1. Zoran Pantic & M.Ali Babar
IT University of Copenhagen, Denmark
Nordic Symposium on Cloud Computing &
InternetTechnologies (NordiCloud)
August 21th & 22th,2012,Helsinki,Finland
Building Private Cloud with Open Source
Software for Scientific Environment

2. Zoran Pantic
InfrastructureArchitect & Systems Specialist
Corporate IT @ University of Copenhagen
E-mail: zopa@itu.dk & zoran@pantic.dk
Academic profile: http://itu.academia.edu/ZoranPantic
Blog: http://zoranpantic.wordpress.com
LinkedIn: http://www.linkedin.com/in/zoranpantic

3. M. Ali Babar

4. Agenda
Non-technical part:
Why Private Cloud?
Why OSS?
Technical part:
Reflections on diverse IT-infrastructure aspects
OSS Private Cloud solutions:
UEC/Eucalyptus
OpenNebula
OpenStack
Conclusion
Questions? (also during the session)

5. Tutorial Goals
Understand the role and use of private cloud in specific
environments, e.g., scientific & academic
Gain knowledge of the technologies for setting up a private
cloud with open source software
Learn about the process for designing & implementing a
private cloud solution
Appreciate the socio-technical & technical challenges
involved and some potential strategies

6. Cloud Computing
“Cloud computing is a model for enabling convenient,on-demand network
access to a shared pool of configurable computing resources (e.g.,networks,
servers,storage,applications,and services) that can be rapidly provisioned
and released with minimal management effort or service provider
interaction.”
(A definition by the US National institute of standards and technology - NIST)

7. Main Aspects of a Cloud System
Reproduced from Figure 1 ofThe Future of Cloud Computing: Opportunities for European Cloud Computing beyond 2010.

8. Commercial Efforts & NFRs
FromThe Future of Cloud Computing: Opportunities for European Cloud Computing beyond 2010

9. Service & Deployment Models
Infrastructure as a Service (IaaS)
Amazon EC2 Eucalyptus IBM – Computing
On Demand (DoC)
VMWare
vSphere
Platform as a Service (PaaS)
Google App
Engine
Microsoft
Azure
Force.com Yahoo
Open Strategy
Software as a Service (SaaS)
Google Apps Zoho Salesforce
CRM
Microsoft Cloud
Services
Public
Clouds
Private
Clouds
Community
Clouds
Virtual Private
Clouds
Hybrid Clouds
ServiceModelsDeploymentModels

10. Private Cloud
Private cloud has different meanings to different people
But basically, it’s a cloud infrastructure set up, managed,
and upgraded by an organization or community for their
own use
Commercial vendors are entering in this domain quite
fast and Open Source providers are also there:
Eucalyptus, Open Nebula, OpenStack

11. Steps for Setting Up Private Cloud
Adopt a machine virtualization strategy
Profile application compute,memory, and storage usage
and performance requirements
Design a virtual machine development consultancy
Accounting and recharge policies adapted to self-service
Architect a deployment and deploy a private cloud
Source: Five Steps to Enterprise cloud computing, aWhite paper of Eucalyptus Systems, Inc.

12. Why Private Cloud? 1/2
Usually, the budget is low, and the project should start as soon as
possible
Growing strongly:
The need for processing large data volumes
The need to conserve power by optimizing server utilization
Non-standard highly-adaptable solution needed
Analyzing large amounts of data to get results
Many different research projects in one organization

13. Why Private Cloud? 2/2
Private clouds:
Have higher ROI than traditional infrastructure
Are more customizable
Can quickly respond to changes in demands
Support rapid deployment
Have increased security
Focus on an organization’s core business
Have effort required for running them tending downward

14. Why OSS?
In general:
Lowering the costs (i.e.no licensing headaches!) – the budgets
aren’t growing - but the demands are
Interchangeability & portability (general,avoiding vendor lock-in)
Socio-organizational reasons
Energy efficience
Examples: UEC/Eucalyptus, OpenNebula, OpenStack, Joyent
SmartOS

15. Private Cloud Challenges
Challenges:
Socio-technical
Technical

16. Socio-technical Challenges
Socio-technical challenges: mostly political and economic:
Existing structures oppose implementation of private cloud
Weak transparency of who is in charge of systems and economy,
Researches cannot be market cost-effective,
Administrators de facto in charge - instead of scientific groups
Tendency of implementing things because they are interesting
and “fun”, while maybe there is no need for those systems.

17. Technical Challenges
Private cloud maturity,
Problems porting of programming code,
IT departments should be big enough, with enough
expertise,
OSS: community cannot fix all your problems.

18. Implementing Cloud Solutions
Determine the needs and their nature – extensive interaction
with all the major stakeholders, e.g., project leader
Top-down steering of the process
Design and implement a test case
End users also thoroughly test the solution - free of charge,
Make sure that implementation succeeds first time!
Get a very clear picture of what services are to be offered,
who will use them, what they will use them for, and how!

19. Private Cloud in Scientific Environment
Based on Open Source Software (OSS)
Focus on the logistical and technical challenges, and
strategies of setting up a private cloud for scientific
environment
General scenarios:
Local DIY
OSS Private Cloud
Enterprise Private Cloud (with mgmt solution)
Virtual Private Cloud
... or just going Public Cloud

20. Focus on Scientific Environment
Difference in implementing for “infantry” and “supply troops”
“Infantry” - to support research, scientific computing and
High Performance Computing (HPC)
“Supply” - to support daily operational systems and tasks
i.e. joint administration
Bookkeeping, administration, Communications (telephony, e-
mail, messaging)
“Infantry” – stateless instances vs.“Supply” – stateful
instances

21. Scientific Environment: “Infantry” 1/2
Uses non-standard & advanced research instruments
Applicable in research, scientific computing and HPC, i.e.:
Generally if users needVMs that they administer themselves
(root access) - more appropriate to supply them with machines
from private cloud, then giving access to virtual hosts behind
firewall
Organizations like ITU (Denmark): for numerous different projects
Organizations like DCSC (Denmark): 1/3 of the jobs would be
runnable on private cloud
in HPC: Only in low end, for low memory and low core
number jobs

22. Scientific Environment: “Infantry” 2/2
Summarized suggestions
Have social psychology in mind as important factor
Consult the professor in charge of money for the project
Implement an open source solution – OpenStack, OpenNebula,
UEC based on Eucalyptus, Joyent SmartOS, ...

23. Scientific environment: “Supply”
Needs a stable and supported solution
Summarized suggestions
Have social psychology in mind as important factor
Consult the system owner in charge of money for the project
Implement a proprietary solution from reputable provider
Microsoft Hyper-V,VMwareVirtual Infrastructure, …
Sign a support agreement & negotiate a good SLA

24. CPU and Memory
Processor architecture:
Intel &AMD
Definitely 64-bit – for performance reasons
Multiprocessor, multicore, hyper threading
Virtual Extensions enabled hardware is a must
IntelVT-X orAMD-V virtualization extensions – virtualization
enabled hardware (check by viewing /proc/cpuinfo)
Host’s RAM minimum 4 GB
Enable KSM (Kernel SamePage Merging)

25. Storage Options
Disk intefaces: parallel & serial
Contemporary disk types:
SATA
SCSI
SAS
SSD
Hybrid drives

26. Storage Types 1/3
Local vs Remote storage:
Local storage:
disks in the host itselv
DAS – attached directly to the host
Remote storage:
NAS - File Level Storage (NFS, SMB/CIFS)
Also distributed file systems (see i.e. MooseFS and GlusterFS)
SAN - Block Level Storage (FC/FCoE, iSCSI)
OSS/free SAN/NAS appliance example: Napp-it , based on
ZFS/Nexenta

27. Storage Types 2/3
Storage levels:
Block – bits stored sequentially in a block of fixed size; read & write raw
data blocks; for file systems or DBMSs
File – maintains physical location of files, mapping them to blocks (i.e.
inode number / pointers)
Object – data organized in flexible sized containers, objects, consisting
of data (sequence of bytes) and metadata (extensible attributes
describing the object); for static data; distributed storage spread accross
multiple disk drives and servers; no „central brain“ or „master point“ –
scalable, redundant, durable
Partitioning in Linux using LogicalVolume Manger (LVM)
PhysicalVolume (PV)
LogicalVolume (LV): multiple PVs make one LV
Volume Group (VG): multiple LVs make oneVG

28. Storage Types 3/3
Disk configuration:
Independent disks (JBoD)
RAID:
Multiple drives comprising one logical unit
Can be based on software, hardware or firmware
Some of the RAID levels:
0 – block-level striping without parity or mirroring
1 – mirroring without parity or striping
5 – block-level striping with distributed parity
6 – block-level striping with double distributed parity
01 (0+1) – striped sets in a mirrored set
10 (1+0) – mirrored sets in a striped set

29. Virtualization 1/2
Different types of virtualization:
Hardware
Storage
Network
Memory
Application
Desktop
...

30. Virtualization 2/2
Hardware virtualization:
Full virtualization: guest unmodified, unaware
HW-assisted virtualization: hw architecture supports virtualization
Partial virtualization: partially simulates the physical hardware
of a machine; i.e. each guest has independent address space
Paravirtualization: guest is aware that it’s not „alone“; guest
modification required (drivers)
OS-level virtualization (Container-based virtualization):
physical server virtualized at OS-level, enabling multiple
isolated and secure virtualized servers to run on a single
physical server; guest and host share the same OS

31. Types of Hypervisors
Types of hypervisors:
Native / bare metal – run directly on the host’s hardware
Hosted – run within OS
Major virtualization vendors & technologies used in hypervisor layer:
http://www.cloudcomputeinfo.com/virtualization (source: Paul Morse)
Todays most used hypervisors:
KVM/QEMU
Xen
VirtualBox
VMware
Hyper-V
SmartOS

32. KVM
KVM - „Kernel-basedVirtual Machine“, http://www.linux-kvm.org
Linux kernel module that allows a user space program to utilize the
hardware virtualization features of various processors (Intel and AMD
processors - x86 and x86_64, PPC 440, PPC 970, S/390)
KVM included in kernel » more recent kernel gives updated KVM
features, but is less tested)
virtualization solution that can run multiple virtual machines running
unmodified Linux orWindows guests
Supports .raw, .qcow2 and .vmdk disk image formats
Available as integrated part of every Linux distribution since 2.6.20
Components:
loadable kernel module „kvm.ko“ that provides the core virtualization
processor specific module „kvm-intel.ko“ or „kvm-amd.ko“
KVM is only an intefrace that is called through a special system file,
and requires QEMU to be a full virtualization environment

33. QEMU
QEMU – Quick Emulator - http://wiki.qemu.org
generic open source machine emulator and virtualizer:
Emulator: runs OS’es made for one machine on different machine
Virtualizer:
executes guest code directly on the host CPU
Executed under Xen hypervisor, or using the KVM kernel module

34. Xen
Open Source virtualisation technology - http://www.xen.org
Started as XenoServer project at Cambridge University
Used as standalone hypervisor, or as hypervisor component in
other cloud infrastructure frameworks
Supports .raw and .vmdk disk image formats

35. VirtualBox
OracleVirtualBox - https://www.virtualbox.org
Free software released under GNU GPL
A x86 virtualization platform, created by Innotek, purchased by
Sun, and now owned by Oracle
Installed on a host OS as an application

36. VMware
VMware - http://www.vmware.com
Different hypervisors:
ESX – mainline product; commercial license
ESXi – mainline product, free (not OSS); boot from flash cards supported
Server – free (not OSS), installs on Linux &Windows
Workstation/Player – virtualization on user PC’s
Supports .vmdk disk image format

37. VMware

38. Hyper-V
Microsoft Hyper-V http://www.microsoft.com/en-us/server-
cloud/windows-server/hyper-v.aspx
Released in 2008, new 2012 release expected in November
Virtualization platform that is integral part ofWindows Server
Only for x86-64
Can boot from flash card on servers motherboard
Variants:
Stand-alone product, free, limited to command line interface
As Hyper-V role insideWindows Server
Supports .vhd disk image format

39. SmartOS
Joyent SmartOS - http://smartos.org
Free, gone Open SourceAugust 2011, descent from
OpenSolars - Illumos
Hypervisor powering Joyent’s SmartDataCenter, can run
private, public and hybrid cloud
Enables HW-level and OS-level virtualization in a single OS
Features: KVM, Zones, DTrace, ZFS

40. Networking Services 1/3
Providing basic network services (DNS, GW, NAT, ...) is a
good idea
Physical & virtual networks
Physical network:
Implementing private cloud using 2 or 3 networks:WAN,
Cloud public & Cloud private
Firewall: OSS based pfSense - to make the whole environment
independent of the network infrastructure / environment
where it will be “plugged in”

41. Networking Services 2/3
Virtual networks: (i.e. Nicira, Xsigo)
Independece from network HW
Reproduction of the physical network
Operating model of computing virtualization
Different hypervizor compatibility
Isolation between virtual and physical network, and control
layer
Scalling & performance cloud-like
Programmatic provisioning & control

42. Networking Services 3/3
Network virtualization (example: Nicira)

43. Redundancy
Automatic/manual failover/failback
Clusters: active/active, active/passive (quorum)
Private Cloud
Some HA features, but local to every provider
Work in progress:Corosync + Pacemaker
„Corosync“ – Open Source cluster solution
„Pacemaker“ – Open Source HA cluster resource manager

44. Private Cloud Offerings
List of OSS Private Cloud offerings: (source: Paul Morse)
http://www.cloudcomputeinfo.com/private-clouds
Covered:
Eucalyptus (Ubuntu Enterprise Cloud, UEC)
OpenNebula
OpenStack

45. Eucalyptus
Was bundled with Ubuntu (UEC); now „only“ supported
(Ubuntu is bundling OpenStack from 11.10)
UEC/Eucalyptus is an on-premise private cloud OSS based
platform, sponsored by Eucalyptus Systems
Started as research project in 2007 @ UCSB
Linux based – RHEL, CentOS, Ubuntu
Support forVMware
For scalable private and hybrid clouds
Hybrid clouds achieved byAPI compatibility with Amazon’s
EC2, S3, and IAM services
New feature since UEC: Eucalyptus HA
All figures taken from http://www.eucalyptus.com

46. Requirements
All components must be on physical machines (noVMs!)
Processor Intel orAMD with 2 cores of 2 GHz
Min 4 GB RAM
Storage: min 30 GB for each machine, 100-250 GB and more for
SC & NC recommended
Network: min 1 Gbps NICs, bridges configured on NCs
Linux – if Ubuntu, choose LTS (LongTime Support) version
Hypervisors: (Xen, KVM,VMware)
RHEL & CentOS must have Xen
Ubuntu must have KVM
Vmware
SSH connectivity between machines

47. Components
Designed as a distributed system with a set of 5 (6) elements:
Cloud Controller (CLC)
Walrus Storage Controller (WS3)
Cluster Controller (CC)
Storage Controller (SC)
Node Controller (NC)
VMware Broker (Broker orVB) - optional

48. Architectural Layers
Three levels:
1. Cloud level
Cloud Controller (CLC)
Walrus Storage Controller
(WS3)
2. Cluster level
Cluster Controller (CC)
Storage Controller (SC)
VMware Broker (Broker orVB)
3. Computing level
Node Controller (NC)

49. Cloud Controller (CLC)
Entry point to Eucalyptus cloud
web interfaces for administering the infrastructure
web services interface (EC2/S3 compliant) for end users
/client tools
Frontend for managing the entire UEC infrastructure
Gathers info on usage and availability of the resources in the
cloud
Arbitrates the available resources, dispatching the load to the
clusters

50. Walrus Storage Controller (WS3)
Equivalent to Amazon’s S3
Bucket based storage system with put/get storage model
WS3 is storing the machine images and snapshots
Persistent simple storage service, storing and serving files

51. Cluster Controller (CC)
Entry point to a cluster
Manages NCs and instances running on them
Controls the virtual network available to the instances
Collects information on NCs, reporting it to CLC
One or several per cloud

52. Storage Controller (SC)
Allows creation of block storage similar to Amazon’s Elastic
Block Storage (EBS)
Provides the persistent storage for instances on the cluster
level, in form of block level storage volumes
Supports creation of storage volumes, attaching, detaching
and creation of snapshots
Works with storage volumes that can be attached by aVM or
used as a raw block device (no sharing though)
Works with different storage systems (local, SAN, NAS,
DAS)

53. VMware Broker (Broker or VB)
Optional component for Eucalyptus subscribers
Enables deployingVMs onVMware infrastructure
Responsible for arbitrating interactions between CC and
ESX/ESXi hypervisors
located with CC

54. Node Controller (NC)
Compute node (“work horse”), runs and controls the instances
Supported hypervisors:
KVM (preferred, open source version)
Xen (open source version)
VMware (ESX/ESXi, for subscribers)
Communicating with both OS and the hypervisor running on the
node, and Cluster Controller
Gathers the data about physical resource availability on the node
and their utilization, and data about instances running on that
node, reporting it to CC
One or several per cluster

55. Plan Installation
Integration with LDAP orAD
Support for remote storage (SAN/NAS – check supported devices)
Choosing from “ installing NC on one server and all other on
another“, to „each of components on separate server“
Trade-off between simplicity and performance & HA

56. Installation
Using Ubuntu+Eucalyptus bundled installation (not available
in new versions of Ubuntu, since version 11.10 Ubuntu includes
OpenStack instead)
Manually:
Install OS
Verify network (connectivity, FW,VLAN, DNS...)
Install hypervisor
Configure bridges, NTP and MTA
Install Eucalyptus
Configure Eucalyptus (network modes, hypervisors, runtime
environment)
Eventually configure HA

57. Scale-out Possibilities
2 physical servers
Server 1:
CLC/WS3/CC/SC
Server 2: NC
3 physical servers:
Server 1: CLC/WS3
Server 2: CC/SC
Server 3: NC
4 physical servers
Server 1: CLC
Server 2:WS3
Server 3: CC/SC
Server 4: NC
5 physical servers
Server 1: CLC/WS3
Server 2: CC1/SC1
Server 3: NC1
Server 4: CC2/SC2
Server 5: NC2

58. Scaling Out
NC NC NC NC NC NC
CLUSTER 1 CLUSTER 2
CLUSTER
3
CLOUD

59. Networking
Networking modes
offering different level
of security and
flexibility:
Managed
Managed NoVLAN
System
Static

60. High Availability
Redundancy - Eucalyptus HA:
By configuring HA, primary and secondary cloud and cluster
components are introduced
Hot-swappable components: CLC,Walrus, CC, SC, andVB
Must have 3 NICs if fearing network hardware failure
For HA SCs, supported SANs needed
NCs are not redundant
Externally accessible components (cloud level) must have DNS
Round-Robin support
Arbitrator service uses ICMP messages to test reachability
If all arbitrators fails to reach some component, failover is initiated

61. WebGUI

62. GUI using HybridFox

63. OpenNebula
An Open Source project aiming at implementing the
industry standard for building and managing virtualized data
centres and cloud infrastructure (IaaS)
Sponsors:
EU through various programs (via DSA, RESERVOIR, 4CaaSt,
StratusLab, BonFIRE)
National grants
C12G Labs
Microsoft
All figures taken from http://opennebula.org

64. History

65. Characteristics 1/3
Doesn’t have specific infrastructure requirements, making it easy
to fit in the existing environment
Try it on your laptop! Does not require any special hardware or
software configuration (single server + distro of your choice)
Supports implementations as Private, Hybrid (with both Bursting
and Federation) and Public Cloud
Provides Storage system (storing disk images in datastores; images
can be OS installations, or data blocks),Template Repository
(registeringVM definitions),Virtual Networking & Management
(CLI & Sunstone GUI, features live and cold migration, stop,
resume, cancel)

66. Characteristics 2/3
Has great modularity, which eases the integration with other
solutions
Implemented on a plugin model, making it easy to customize
different aspects (virtualization, storage, authentication &
authorization, ...)
Any action is performed by a bash scirpt
Doesn’t implement a „default“ hypervizor
The core of OpenNebula written in C++, making it robust
and scalable
Monitoring: Configurations ofVM’s and all monitoring
information is stored in a (SQL) database

67. Characteristics 3/3
Uses common open industrial standards – i.e.Amazon EC2
API and Open Cloud Computing Interface (OCCI)
OpenNebula’s native cloud API:
available as Java, Ruby, and XML-RCPAPI
gives access to all the functions
enables integration of own procedures
Security at high level: host communication using SSH (RSA)
and SSL
Quality: relies on Community and own QA
MakingVNC sessions to runningVMs supported

68. Main components

69. Main features (v3.6)
User Security & Multitenancy using Group Management
Virtual Data Centers
Control & Monitoring of Physical &Virtual Infrastructure
Supports multiple hypervisors, data stores, network
integrations, datacenter monitoring (Ganglia)
Distributed Resource Optimization
High Availability
Hybrid Cloud & Bursting
Self-service provisioning portal

70. Internal Architecture 1/4
The three layers of the internal architecture:

71. Internal Architecture 2/4
Drivers communicate directly to the OS
Transfer driver: manage the disk images on the storage
system, that could be NFS or iSCSI, or copying using SSH
Virtual Machine driver: specific to the hypervisor
implemented; manage theVM’s running on the hosts
Information driver: specific to the hypervisor implemented;
showing the current status of hosts andVM’s hosts

72. Internal Architecture 3/4
Set of components to control and monitorVMs,VNs, storage &
hosts:
Request Manager: handles client requests
Virtual Machine Manager: manages & monitorsVMs
Virtual Network Manager: manages virtual networks
Host Manager: manages & monitors physical resources
Database: persistent storage (state)

73. Internal Architecture 4/4
CLI: manual manipulation of the virtual infrastructure
Scheduler: invokes actions onVMs (using XML-RPC interface)
Other: 3rd party tools (using XML-RPC interface or OpenNebula Cloud
API)

74. OpenNebula – hypervisors
Xen
KVM/QEMU
VMware

75. OpenNebula – hardware
Processor requirement: CPU with virtualization support
Memory:
Host: minimum 4 GB
Guest: 256 MB for smallest instance
Storage based on RAID: local disk for PoC, SAN for
production systems
Network: gigabit network card(s), eventually bundling
several cards together (performance & redundance)

76. OpenNebula – system components
Frontend
Hosts
Image Repository
Physical network

77. OpenNebula - networking
Service Network is recommended to be dedicated
network
VM’s network interface is connected to a bridge in
the host (i.e. a host with two NICs, public and private, should
have two bridges)
Create bridges with the same name in all the hosts
Drivers that may be associated with each host:
Dummy
Fw
802.1Q
Ebtables
Ovswitch
VMware
Fw Ovsw 802 ebtbl VMw
KVM Yes Yes Yes Yes No
Xen Yes Yes Yes Yes No
VMware No No No No Yes

78. Installation 1/7
Installation steps:
Planning and preparing the installation
Installing OS
Installing the OpenNebula software
Configuring the OpenNebula components

79. Installation 2/7
Planning & preparing: OpenNebula is a simple setup consisting
of front end(s) and hosts (cluster nodes).
Basic components:
Front end
Host
Datastores
Service Network
VM networks

80. Installation 3/7
Storage types: shared & non-shared
Non-shared storage:
Simple to configure
Initial start of anVM will be slower as image is copied to the host
Shared storage:
Any host has access to the image repository
Any operation on aVM goes quicker because there is direct access
to the images, no copying needed
In smaller environments or PoCs, implemented on front end
In bigger environments, implemented on NAS/SAN

81. Installation 4/7
OS installation:
Choose Linux distribution (i.e. Ubuntu)
Choose installation media: .iso or network
Use default installation steps, except evt. for partitioning
Partitioning:
If HW raid exists, it will appear as single disk; if SW raid should be
configured, can be done after creating partitions
Partitions for system, user and swap files
Default user creation (oneadmin)
The same account and group needed on both Front end & host
All the accounts need the same UID and GID (user & group IDs)

82. Installation 5/7
Front end:
Install OpenNebula software
Requirement:
Needs access to storage (direct or via network)
Needs access to each host
SSH to hosts using SSH keys (without passwords, auto-add to known hosts)
Ruby (≥ v1.8.7)
Hosts:
No OpenNebula software needed
Different hypervisors on different distros inside a cluster possible
Requirements:
Hypervisor
SSH server
Ruby (≥ v1.8.7)
Host should be registered in OpenNebula (onehost)

83. Installation 6/7
Configuring the OpenNebula components:
Hypervisor: KVM by default (and easiest), but other drivers can be
selected / modified
Host monitoring
Storage: shared filesystem used by default, can be changed
Networking
Users & Groups (admins, regular, public & service users; integration
with LDAP infrastructure possible)
Sunstone (Web GUI with same functionality as CLI)
Accounting & Statistics (info on usage, accounting, graphs)
Zones (oZone server, managing Zones andVDCs)
Hybrid clouds (for peak resource usages)
Public clouds (using public interfaces, EC2 query and OCCI)

84. Installation 7/7
Management tasks fter the installation:
Check if deamons are running
Check passwordless inter-host connectivity
Check / enable KSM
Managing hosts:
Registering (adding a host to OpenNebula)
Deleting (deleting a host, i.e. dismissing a host)
Enabling/disabling (no monitoring nor launc of new instances)

85. Hybrid cloud
AWS EC2 or compatibile

86. Public cloud
Giving access to the „outside world“ using:
EC2 Query interface usingAmazon EC2 QueryAPI
Open Cloud Computing Interface (OCCI)
Centralized management using oZone
Zones: several physical hosts with same or different
hypervisors, controlled by one front end
VDCs (Virtual Data Centers): several hosts from the same
zone logically grouped

87. Redundancy
Redundant frontends, but no automatism
Use separate MySQL backend (though oZones currently
suppors only SQLlite)
Sunstone can be deployed on a separate machine (not
necessarily on front end)

88. OpenStack 1/4
IaaS platform for building cloud solutions using any of the
deployment models
Open Source, released underApache license
Co-founded by NASA and Rackspace in 2009 in a joint open
source project, with NASA delivering cloud compute code
(„Nebula“), and Rackspace delivering cloud object storage
(„Cloud Files“)
First release to public in November 2010 – „Austin“
Backed by i.e. HP, Cisco, IBM, RedHat, Dell, Citrix,
Canonical, ...
All figures taken from http://www.openstack.org

89. OpenStack 2/4
Has considerable take-off in use ...
...though NASA reported moving a part of its infrastructure to
Amazon, saving $1 million/yr
(http://www.wired.com/wiredenterprise/2012/06/nasa-web-services-openstack/)
Some contributors left NASA going to the private sector
(Nebula, Piston Cloud Computing, RackSpace, ...)
Active community:
http://forums.openstack.org
http://wiki.openstack.org
http://docs.openstack.org

90. OpenStack 3/4
Supported distros: Ubuntu, Debian, RHEL, CentOS, Fedora,
SUSE, Piston Enterprise OpenStack, SwiftStack,
Cloudscaling & StackOps
Releases:Austin (2010), Bexar, Cactus, Diablo (2011), Essex
(2012, current stable), Folsom (under development)
Hypervisors: KVM, Xen, ESXi
OS-level virtualization also supported, i.e. LXC
Networking modes: Flat (bridging),VLAN (vlan-switch)
Trying it (one or multiple servers):
on free „sandbox“ hosted environment (trystack.org), or
locally using a documented script (devstack.org)

91. OpenStack 4/4
Written in Python
Consists of:Compute, Networking, Storage, Shared Services
Managed through a dashboard
Implements on standard hardware, supported onARM

92. Compute
Provides on-demand computing ressources by provisioningVMs
Access throughAPIs and web GUIs
Scales horizontally (scale-out)
Some features:
Manage CPU, memory, disk, network
Distributed and asynchronous architecture
LiveVM management
Floating IP
Security groups & RBAC (Role Based Access Control)
API with rate limiting and authentication
Resource utilization: allocating, tracking, limiting
VM image management & cashing

93. Storage
Supports both Object Storage and Block Storage:
Object Storage – distributed,API-accessible, scale-out storage used
by applications, for backup, archiving and data retention (static data)
Block Storage - enables block storage to be used byVMs; supports
integration with enterprise storage solutions (i.e. NetApp, Nexenta, ...)
Some features:
Vertical and horizontal scalability
Huge & flat namespace
Built-in replication
RAID not required
Snapshot & Backup API

94. Networking
Managing networks and IP addresses
Pluggable, scalable and API-driven system
Flat networks &VLANs
Static Ips, DHCP & Floating IP

95. Shared services
Dashboard:
GUI for admins and users, brandable
Pluggable / 3rd party: billing, monitoring, additional management
Identity Service:
Central directory of users mapped to services they can access
Queryable list of all of the services deployed
Image Service:
provides discovery, registration and delivery services for disk and
server images
Stores images, snapshots, templates in OpenStack Object Storage
Supports following image formats: raw,AMI,VHD,VDI, qcow2,
VMDK, OVF

96. Service families
Nova - Compute Service
Swift – Object Storage Service
Glance – Image Registry & Delivery Service
Horizon – User Interface Service, „Dashboard“
Keystone – Identity Service
Quantum (in development) –Virtual Network Service

97. Nova
Main part – cloud computing fabric controller
One of 1st projects, descends from NASA’s Nebula
provides API to dynamically request and configureVMs
Two major components: messaging queue (RabbitMQ) and database,
enabling asynchronous orchestration of complex tasks through message
passing and information sharing
Components: Database,Web Dashboard,API,Auth Mgr, ObjectStore,
Scheduler,VolumeWorker, NetworkWorker, ComputeWorker
all of its major components can be run on multiple servers (designed as
distributed application)
Supported virtualization: KVM, Xen, Citrix Xen, ESX/ESXi, Hyper-V,
QEMU, Linux User Mode & Containers
Uses a SQL-based central database (in future, for larger deployments,
aggregated multiple data stores are planned)

98. Swift
Object/blob storage
One of 1st projects, descends from Rackspace’s Cloud Files
Components: Proxy Server, Ring, Object Server, Container
Server,Account Server, Replication, Updaters,Auditors
Can be clustered using Proxy nodes and Storage nodes
Filles cannot be accessed through filesystem, but viaAPI client
Scalability and redundancy: writing multiple copies of each object
to multiple storage servers within separate zones
zone: isolated storage server groups
Isolation levels: different servers, racks, sections of a datacenter,
datacenters
Best practice: write 3 replicas across 5 zones (distributed
writes/reads)

99. Glance
Discovers, registers and retrievesVM images
Uses RESTful API for querying & retrieval
Supports various back-end storage solutions:VM image can
be stored on simple file systems and object storage systems
(Swift)
Components: GlanceAPI server, Registry Server, Store
Adapter
Supported disk formats: raw,VHD,VMDK, qcow2,VDI,
ISO,AMI,ARI,AKI
Supported container formats: OVF,AMI,ARI,AKI

100. Horizon

101. Keystone
Cloud identity service
provides Identity,Token, Catalog and Policy services
implements OpenStack Identity API

102. Quantum
Virtual network service („Networking as a Service“)
Still under development, to be released with the release of
„Folsom“ 27th September 2012
ProvidesAPI to dynamically request and configure virtual
networks
Quantum API supports extensions providing advanced
networking (i.e. Monitoring, QoS,ACLs, ...)
Plugins for Open vSwitch, Cisco, Linux Bridge, Nicira NVP,
Ryu OpenFlow, NEC OpenFlow, MidoNet

103. Advanced setup

104. Redundancy
Comming with „Folsom“
„Corosync“ – open source cluster
Have multiple Swift and Nova servers
Cloud controller – single point of failure (nova-api, nova-
network):
Run multiple instances on multiple hosts (state is saved in DB)
Use (--multi host configuration in Nova)

105. Recommendations
Although still at an early stage, easier to install but still hard
to manage and maintain for a regular admin, and having steep
learning curve (admins & users), implementation is
suggested, at affordable, smaller scale
Implement on a current/modern hardware
Keep the knowledge updated
Keep software platform and hardware updated if possible
Monitor & analyze
costs, available features and complexity, compared to
budget, needs and internal resources available
Asses the implementation possibilities based on the analyses

106. Sources of Further Material 1/5
http://www.openstack.org/
http://opennebula.org/
http://www.ubuntu.com/
http://www.eucalyptus.com
http://www.napp-it.org/index_en.html
http://www.cloudcomputeinfo.com/virtualization
http://www.cloudcomputeinfo.com/private-clouds
http://www.linux-kvm.org
http://wiki.qemu.org
http://www.xen.org
https://www.virtualbox.org
http://www.vmware.com
http://www.microsoft.com/en-us/server-cloud/windows-server/hyper-v.aspx
http://smartos.org

107. Sources of Further Material 2/5
Armbrust, M., et al., 2010,AView of Cloud Computing,ACM,
53(4), pp. 50-58.
Zhang, Q., Cheng, L., Boutaba, R., Cloud Coomputing: state-of-
the-art and research challenges, Journal of Internet Services and
Applications, 2010, 1:7-18.
The Future of Cloud Computing: Opportunities for European
Cloud Computing Beyond 2010.
Chapman et. al. 2010. Software architecture definition for on-
demand cloud provisioning. In Proceedings of the 19th ACM
International Symposium on High Performance Distributed Computing
(HPDC '10), 2010,AView of Cloud Computing,ACM, 53(4), pp.
50-58.
Ali Babar, M.; Chauhan M.A.; ,A tale of migration to cloud
computing for sharing experiences and observations, SECLOUD
'11,ACM.

108. Sources of Further Material 3/5
http://nicira.com/
http://www.xsigo.com/
http://www.reservoir-fp7.eu/
http://www.c12g.com/
http://dsa-research.org/
http://portal.ucm.es/en/web/en-ucm
http://occi-wg.org/
http://opennebula.org/documentation:rel3.6:ganglia
http://www.nasa.gov/
http://www.rackspace.com/
http://www.nebula.com/
http://www.pistoncloud.com/

109. Sources of Further Material 4/5
http://www.pistoncloud.com/openstack-cloud-software
http://swiftstack.com/
http://www.cloudscaling.com/
http://www.stackops.com/
http://www.rabbitmq.com/
http://www.corosync.org/
http://www.clusterlabs.org/
OpenNebula 3 Cloud Computing”, GiovanniToraldo, Packtpub,
May 2012
Eucalyptus Guides, Eucalyptus Systems, Jun 2012
“Deploying OpenStack”, Ken Pepple, Oreilly, July 2011
OpenStack Manuals, docs.openstack.org, May 2012

110. Sources of Further Material 5/5
“Ubuntu Enterprise Cloud Architecture”,TechnicalWhite Paper, Simon
Wardley, Etienne Goyer & Nick Barcet – August 2009
“Building a Private Cloud with Ubuntu Server 10.04 Enterprise Cloud
(Eucalyptus)”, OSCON 2010
“Eucalyptus Beginner's Guide”, UEC Edition, 23 Dec 2010, Johnson D,
KiranMurari, Murthy Raju, Suseendran RB,Yogesh Girikumar
“Dell releases Ubuntu-powered cloud servers”, Joab Jackson, IDG News
Service, NetworkWorld
Interview at Danish Center for Scientific Computing (DCSC), 30th
March 2011
White Paper “Ubuntu - An Introduction to Cloud Computing”
Deployment Guide - Ubuntu Enterprise Cloud on Dell Servers SE
White Paper “Ubuntu Enterprise Cloud Architecture”,Wardley, Goyer,
Barcet,August 2009
“Practical Cloud Evaluation from a Nordic eScience User Perspective”,
Edlund, Koopmans, November 2011.

111. Questions
?

112. Thank you!
Thank you for your attention!
Still having questions?
maba@itu.dk
zopa@itu.dk