OpenNebula is a fully open-source cloud management platform, with excellent performance and scalability to manage tens of thousands of virtual machines, and with the most advanced functionality for building virtualized enterprise data centers and private cloud infrastructures. OpenNebula is the result of many years of research and development in efficient and scalable management of virtual machines on large-scale distributed infrastructures. Its innovative features have been developed to address the requirements of business use cases from leading companies in the context of flagship international projects in cloud computing. OpenNebula is being used by many supercomputing and leading research centers to build HPC and science clouds for hosting virtualized computational environments, such as batch farms and computing clusters, or for providing users with new "HPC as a service" resource provisioning models. The talk describes how to design a cloud architecture with OpenNebula and its innovative features to enable the execution of flexible and elastic cluster and high performance computing services on demand while reducing the associated cost of building the datacenter infrastructure.

1. NASA Ames
NASA Advanced Supercomputing (NAS) Division
California, May 24th, 2012
Cloud Computing Architecture with
OpenNebula
HPC Cloud Use Cases
Ignacio M. Llorente
Project Director
© OpenNebula Project. Creative Commons Attribution-NonCommercial-ShareAlike License

2. Contents
Cloud Computing Architecture with OpenNebula - HPC Cloud Use Cases!
This presentation is about:
• Cloud case studies in HPC
• Private cloud computing using OpenNebula
• The anatomy of a Private Cloud
• OpenNebula innovative features for HPC
Cloud Computing Architecture with OpenNebula - HPC Cloud Use Cases! 2/26

3. What is OpenNebula?
Open Cloud Solution for Building and Managing Virtualized Data Centers!
Public
Private
Hybrid
Cloud Computing Architecture with OpenNebula - HPC Cloud Use Cases! 3/26

4. The HPC Cloud Use Case
The Pre-cloud Era!
Access
Grid Middleware
LRMS (LSF, PBS, SGE…)
Provision
Cloud Computing Architecture with OpenNebula - HPC Cloud Use Cases! 4/26

5. The HPC Cloud Use Case
OpenNebula as an Infrastructure Tool!
Access Service/Provisioning Decoupling!
• Common interfaces
• Grid integration
Grid Middleware
LRMS (LSF, PBS, SGE…)
• Custom environments
Service
• Dynamic elasticity
Virtual Worker Nodes
• Consolidation of WNs
• Simplified management
Provision
• Physical – Virtual WNs
• Dynamic capacity partitioning
• Faster upgrades
Cloud Computing Architecture with OpenNebula - HPC Cloud Use Cases! 5/26

6. The HPC Cloud Use Case
OpenNebula as an Provisioning Tool!
• Simple Provisioning Interface
Access
• Raw/Appliance VMs
IaaS Interface
Pilot Jobs, SSH…
Service
• Dynamic scalable computing
• Custom access to capacity
• Not only batch workloads
• Not only scientific workloads
Provision
• Improve utilization
• Reduced service management
• Cost efficiency
Cloud Computing Architecture with OpenNebula - HPC Cloud Use Cases! 6/26

7. The HPC Cloud Use Case
Examples: CERN’s lxcloud! http://blog.opennebula.org/?p=620
Goal
• Virtualized HTC Batch Nodes: limited life-time, dynamic capacity
• EC2 Query: CernVM
Deployment Notes
• Custom network integration MAC/IP pinning
• Fast image distribution & boot: BitTorrent + LVM snapshots
• OpenNebula tests up to 20,000 VMs (great feedback!)
Cloud Computing Architecture with OpenNebula - HPC Cloud Use Cases! 7/26

8. The HPC Cloud Use Case
Examples: FermiCloud! http://www-fermicloud.fnal.gov/
Goal
• Scientific stakeholders get access to on-demand VMs
• Developers & integrators of new Grid applications
• MPI and legacy applications
Deployment Notes
• VMs access Fermilab Networking and Storage Services
• OpenNebula + X509 support (contributed back!)
• Other areas: HA, Batch queues look-ahead, cluster on-demand…
Cloud Computing Architecture with OpenNebula - HPC Cloud Use Cases! 8/26

9. The HPC Cloud Use Case
Examples: SARA! https://www.cloud.sara.nl/
Goal
• HPC clusters on-demand: Self service, elastic and fully
configurable HPC systems
• Offer production infrastructure for several engineering and
scientific communities: Bioinformatics, ecology…
Deployment Notes
• Low latency network for HPC
• Extensions in accounting, GUI, firewalling… (contributed back!)
Cloud Computing Architecture with OpenNebula - HPC Cloud Use Cases! 9/26

10. The HPC Cloud Use Case
One of Our Main User Communities!
Supercomputing Centers
Research Centers
Distributed Computing Infrastructures
Cloud Computing Architecture with OpenNebula - HPC Cloud Use Cases! 10/26

11. The Anatomy of the Cloud
Different Perspectives of the Cloud!
Cloud User
Cloud Provider
Cloud Integrator
Cloud Computing Architecture with OpenNebula - HPC Cloud Use Cases! 11/26

12. Using the Cloud
How Can I Create and Manage my Virtual Infrastructure?!
… standards (de facto and de jure) Cloud APIs to leverage existing ecosystems
and ensure portability across providers, and self-service portal ….
" Use AWS existing tools
like ElasticFox • Manage images,
" Execute complete networks and
computing cluster instances
" Manage persistent
scientific data
" Catalog with pre-defined
environments
" Sharing between users
or groups
Cloud Computing Architecture with OpenNebula - HPC Cloud Use Cases! 12/26

13. Building the Cloud
What are the Main Components to Build a Cloud Infrastructure?!
Instance Networks
Internet
• Guests
• Public and private networks
" Fair share of resources
" Place VM close to data
Front-end Hosts
• Authentication • Multiple hypervisors
• Authorization • Up to 500 hosts
• ACLs, roles, • Automatic failover and HA
groups… • Automatic resource allocation
• Accounting
• Logging
• Resource quotas
Service Networks Datastores
• Monitoring, control… • VM image storage
• Live migration… • Multiple datastores
• Storage access… • Heterogeneous configurations
• Shared or non-shared
Cloud Computing Architecture with OpenNebula - HPC Cloud Use Cases! 13/26

14. Building the Cloud
Broad Commodity and Enterprise Platform Support!
Instance Network
• VLAN per user (layer2) Internet
• Open vSwitch, 802.1q
• Ebtables
Front-end Hosts
• X509, LDAP, ssh keys • VMware
• ACLs, roles, groups… • Xen
• KVM
Datastores
Service Network • NAS: NFS, Gluster, GlusterFS…
• Ganglia/Nagios • SAN: Fibre Channel, iSCSI, LVM…
• Additional monitor agents • DFS: Bit Torrent, ssh…
Cloud Computing Architecture with OpenNebula - HPC Cloud Use Cases! 14/26

15. Innovative Features
Clustering the Physical Resources!
Clusters
• Pools of hosts that share datastores and networks
• Group different configurations for performance and security
" Different Hw for
different workloads
profiles (HPC vs HTC)
" Isolation and security
levels
Multiple Datastores per Cluster
• Balance I/O operations between storage servers
• Define different SLA policies (e.g. backup) and performance features for
different VM types or users
Cloud Computing Architecture with OpenNebula - HPC Cloud Use Cases! 15/26

16. Innovative Features
Centralized Management of Multiple OpenNebula Instances (Zones)!
" Full Isolation Clouds
Federation of of
performance-sensitive
• Multi-tier architecture
• Scalability
applications
oZones Server • Isolation
" Isolation and security
• Portal • Multiple-site support
levels
Cloud • Cloud API (EC2, OCCI) " Multi-tier
Consumer • Global AuthN " Scalability
" Multiple-site
Cloud Computing Architecture with OpenNebula - HPC Cloud Use Cases! 16/26

17. Innovative Features
On-demand Provision of Virtual Data Centers!
" Virtual Private Cloud Computing with
Provide VOs or Departments
● their own isolated cloud
Typical scenario in large organizations
and cloud providers
compartment
" ● Associated provision of fully-
On-demand to clusters with
specific Hw and isolated VDC with full
configurable and Sw profiles, or
control and capacity to administer its
security levels
users and resources
Cloud Computing Architecture with OpenNebula - HPC Cloud Use Cases! 17/26

18. Innovative Features
Hybrid Cloud Computing!
Hybrid
● Extension of the local private infrastructure with resources from remote clouds
● Cloudbursting to meet peak or fluctuating demands
Cloud User " HTC workloads
" Managed by the admin
Cloud Provider
Cloud Computing Architecture with OpenNebula - HPC Cloud Use Cases! 18/26

19. Operating the Cloud
How Can I Operate my Cloud Infrastructure?!
… programming APIs (create new tools and integrate), web interfaces (simplify
operation), and command lined interface (create scripts)…
" Automatic/Elastic
Management of
Computing Service
CLI
API
● XML-RPC
● OCA (Ruby, Java, Python)
Cloud Computing Architecture with OpenNebula - HPC Cloud Use Cases! 19/26

20. Integrating the Cloud
The Ecosystem!
… truly open (fully open-source, Apache license) and adaptable (modular and
extensible)… because no two data centers are the same
Standards Adapters
CDMI OVF CIMI
Virtualization Drivers Configuration Storage
Cloud Computing Architecture with OpenNebula - HPC Cloud Use Cases! 20/26

21. About the OpenNebula Project
A Project Aimed at Building the Industry Standard Open Cloud Management Tool!
Third party scalability
• Develop & innovate tests: 16,000 VMs
• Support the community
• Collaborate Commercial Support
TP
v1.0 v1.2 v1.4 v2.0 v2.2 V3.0 V3.2 V3.4
2005 2008 2009 2010 2011 2012
dsa group doing
research…
4,000 downloads/
month
European Funding
Cloud Computing Architecture with OpenNebula - HPC Cloud Use Cases! 21/26

22. About the OpenNebula Project
A Quickly Growing Community More than Doubling Each Year!
25,200
Downloads
● 900 downloads in the last week
12,828 ● Linux distro and code repo
4,861
1,865 140% Annual Growth
2008 2009 2010 2011
Site Visits 579,571
● 15,300 visits and 194,000 page
254,186 views in the last week
111,541
35,842 150% Annual Growth
2008 2009 2010 2011
Mails 4,341
● 800 registered users at present
2,293
1,141
227 170% Annual Growth
Date: November 7th, 2011
2008 2009 2010 2011
Cloud Computing Architecture with OpenNebula - HPC Cloud Use Cases! 22/26

23. About the OpenNebula Project
Open Cloud Enabler in the ICT Industry!
Enabling Hosting Companies and Telcos to Offer Cloud Services
Enabling Technology Companies to Offer Cloud Products
Enabling Service Companies to Offer Cloud Consulting and Integration
Cloud Computing Architecture with OpenNebula - HPC Cloud Use Cases! 23/26

24. About the OpenNebula Project
Open Cloud Enabler for Building and Operating Virtualized Data Centers!
Industry
Supercomputing Centers
Research Centers
Cloud Computing Architecture with OpenNebula - HPC Cloud Use Cases! 24/26

25. About the OpenNebula Project
Open Cloud Enabler for Building and Research and Innovation!
Distributed Computing Infrastructures
Research Projects
16,000 VMs!
Cloud Computing Architecture with OpenNebula - HPC Cloud Use Cases! 25/26

26. Questions?
We Will Be Happy to Answer Any Question !
Cloud Computing Architecture with OpenNebula - HPC Cloud Use Cases! 26/26