Latinoware 2013 - OpenStack RDO - A walkthrough by the Open Source Cloud Comp...
Open stack
1. Openstack => Cloud computing
at your fingertips!
Luan Cestari
February 27 , 2014
1
http://slidesha.re/1gF0PEK
2. Please, let me ask ...
●
●
Is it a hype? What does it means?
●
2
Have you heard about Cloud Computing?
Are you using any cloud service?
http://slidesha.re/1gF0PEK
3. Do you know OpenStack?
●
●
●
3
Is the OpenStack an OS?
Why would I use instead of the machine in my
enterprise?
What is this Open Source thing and what is related to
this talking?
http://slidesha.re/1gF0PEK
4. But what is OpenStack
●
OpenStack is an open source project for
building a private or public
infrastructure-as-a-service (IaaS) cloud
●
OpenStack solution != Virtualization -> let's see why
●
●
4
Cloud!
But that doesn't explain what is the definition of cloud
http://slidesha.re/1gF0PEK
5. The defining the Cloud Computing: It must be ...
●
Scalable
●
Portable
●
On-demand
●
Resource Management
●
Measureable
From: National Institute of Standards and Technology (NIST)
5
http://slidesha.re/1gF0PEK
7. Linux Powers The Cloud
8 out of 10 clouds
are built on Linux1
Amazon EC2
RackSpace
“Linux is twice as
popular as Windows on
Amazon Web Services.”2
“On the RackSpace
cloud, the split is even
starker: 75 %to 25 %,
again in favor of Linux.”2
1 Linux Adoption Trends 2012: A Survey of Enterprise End Users, Linux Foundation, January 18, 2012
2 Windows 8? It Won’t Win Microsoft’s Biggest Battle, Robert Mcmillan, Wired, October 25, 2012
7
http://slidesha.re/1gF0PEK
8. Virtualization to cloud infrastructure continuum
Virtual Infrastructure Management
Private
Distributed
Cloud
Virtualization
Server
Virtualization
Drivers
Consolidation
Reduce Capital Expense
Flexibility & Speed
Reduce Operational Expense
Automation
Less Downtime
Self-Serve Agility
Standardization
IT as a Business
Usage Metering
Hybrid
Cloud
Choice of CAPEX/OPEX model
Increased Flexibility (up and down)
Visibility
Optimization
Agility
Federation
Control
Automation
Self-Service
Brokering
Derived from Gartner Roadmap: From Virtualization to Cloud Computing (reference slide)
8
http://slidesha.re/1gF0PEK
10. Cloud deployment models
Hybrid
Cloud
Private
Cloud
Privately owned
And managed with
Restricted access (but
Could be externally
hosted)
10
Interoperable
combination
of private and
public cloud.
Community
Cloud
http://slidesha.re/1gF0PEK
Public
Clouds
Service Provider
Owned and managed,
Accessed via the web,
Pay for what you use.
11. Types of Cloud
●
Saas (Software as a service)
●
PaaS (Platform as a service)
●
IaaS (Infrastructure as a service)
11
http://slidesha.re/1gF0PEK
12. Cloud Service Models
IaaS
PaaS
SaaS
APPLICATION
APPLICATION PLATFORM
(JBOSS, PHP, RUBY, ETC)
OPERATING SYSTEM
(RHEL)
VIRTUALIZATION
(RHEV)
HARDWARE
(x86)
STORAGE
(RHS)
12
Managed and
Controlled by
Customer (IT, Dev, or
User)
Managed by the Public
or Private Cloud
Offering
Increased Control
Reduced DIY
http://slidesha.re/1gF0PEK
13. A cloud provider view of shared responsibility for
security
Source: Cloud Security Alliance
13
http://slidesha.re/1gF0PEK
13
15. Cloud types & deployment models
Hybrid
Private
Clouds
Public
Clouds
Community
Cloud
SaaS
Google Apps
Salesforce
Many more
PaaS
IaaS
OpenShift
Force.com
Azure
Amazon AWS
RackSpace
OpenStack
vCloud Director
Software-as-a-Service
(SaaS) (hosted apps)
Platform-as-a-Service (PaaS)
(dev platform, apps middleware)
Infrastructure-as-a-Service (IaaS)
(compute, storage, network)
15
http://slidesha.re/1gF0PEK
End-users
Developers
DevOps
IT Admins
16. Streamlining App Dev with PaaS
Physical
How to Build an App:
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
16
Have Idea
Get Budget
Submit hardware acquisition
request
Wait
Get Hardware
Rack and Stack Hardware
Install Operating System
Install Operating System
Patches/Fix-Packs
Create user Accounts
Deploy framework/appserver
Deploy testing tools
Test testing tools
Code
Configure Prod servers (and buy
them if needed)
Push to Prod
Launch
Order more servers to meet
demand
Wait…
Deploy new servers
Etc.
Virtualized
How to Build an App:
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
Have Idea
Get Budget
Submit VM Request request
Wait
Deploy
framework/appserver
Deploy testing tools
Test testing tools
Code
Configure Prod VMs
Push to Prod
Launch
Request More Prod VMs to
meet demand
Wait
Deploy app to new VMs
Etc.
With PaaS
How to Build an App:
1.
2.
3.
4.
5.
6.
Have Idea
Get Budget
Code
Test
Launch
Automatically Scale
“The use of Platform-as-a-Service technologies will
enable IT organizations to become more agile and
more responsive to the business needs.” –Gartner*
More info: www.openshift.com
http://slidesha.re/1gF0PEK
16
17. Don’t take taxonomies too literally
●
IaaS can blend into PaaS
●
●
PaaS can blend into SaaS
●
●
“Value-add” services like DynamoDB, Elastic
MapReduce
PaaS anchored to a SaaS environment
Taxonomy part of broader ecosystem
●
●
APIs/services
●
17
Hybrid cloud IaaS management (CloudForms)
Development tooling
http://slidesha.re/1gF0PEK
18. Back to OpenStack, its history...
●
2010 - Rackspace and Nasa joins forces
●
●
18
Rackspace's Cloud Files platform and NASA's Nebula
OpenStack Object Store (Swift) and OpenStack
Compute Nova
http://slidesha.re/1gF0PEK
28. How does it look like
28
http://slidesha.re/1gF0PEK
29. How does it look like
29
http://slidesha.re/1gF0PEK
30. How does it look like
30
http://slidesha.re/1gF0PEK
31. How to install OpenStack, the easy way
sudo yum install -y http://rdo.fedorapeople.org/rdo-release.rpm
sudo yum install -y openstack-packstack
packstack --allinone
31
http://slidesha.re/1gF0PEK
32. RED HAT LEADS THROUGH OPEN INNOVATION
32
http://slidesha.re/1gF0PEK
33. OpenStack Progression
●
●
●
●
●
●
Open source, communitydeveloped (upstream)
software
Founded by Rackspace
Hosting and NASA
●
●
●
Managed by the OpenStack
Foundation
Vibrant group of developers
collaborating on open source
cloud infrastructure
Software distributed under
the Apache 2.0 license
No certifications, no support
●
●
●
●
Latest OpenStack software,
packaged in a managed
open source community
●
●
Facilitated by Red Hat
Aimed at architects and
developers who want to
create, test, collaborate
●
Freely available, not for sale
●
Six-month release cadence
mirroring community
●
No certification, no support
Installs on Red Hat and
derivatives
●
DOC144908-20130711R4
Enterprise-hardened
OpenStack software
Delivered with an
enterprise life cycle
Six-month release cadence
offset from community
releases to allow testing
Aimed at long-term
production deployments
Certified hardware and
software through the Red
Hat OpenStack Cloud
Infrastructure Partner
Network
Supported by Red Hat
RED HAT ENTERPRISE LINUX – FOUNDATION FOR THE OPEN HYBRID CLOUD
34. PackStack Overview
●
Installer appropriate for smaller scale OpenStack
deployments.
●
Driven by asking questions or an “answer file”
●
Uses SSH and Puppet to set up all nodes
34
http://slidesha.re/1gF0PEK
35. Differences between versions?
Upstream
Midstream
Downstream
Source from GIT
Packages from Brew
Unit Tests
rdopkg
Job Builder/Runner
DevStack
Torpedo / Tempest
Tempest
Tempest
RDO Environments
RHOS Environments
SmokeStack
35
SmokeStack Trunk
RDO Release / Poodle
RHOS Release
http://slidesha.re/1gF0PEK
36. Differences between versions?
●
Upstream:
●
●
Detect rpm based install errors via smokestack
●
●
Contribute tempest tests upstream
Enterprise Linux devstack
Midstream:
●
●
Qualify RDO across supported environments
●
●
Detect packstack, foreman based install errors
Improve the feedback to development
Downstream:
●
●
36
Qualify RHOS across supported environments
Scale and Performance test
http://slidesha.re/1gF0PEK
37. RHEL OPENSTACK PLATFORM VALUE
Why Red Hat vs other community versions?
●
Enterprise-grade OpenStack version with ecosystem, lifecycle, &
support customers expect from Red Hat!
●
●
Based on RHEL and includes required fixes in both OpenStack and
RHEL
●
Enterprise hardened OpenStack code
●
Longer supported lifecycle (starts with 1 year for Folsom/Grizzly)
●
●
●
37
Bug fixes, security errata, selected backports, etc.
Certified ecosystem (Red Hat Certified OpenStack Partner Program and
Red Hat Enterprise Linux ecosystem)
Full support for RHEL and Windows workloads
http://slidesha.re/1gF0PEK
38. Red Hat Enterprise Linux Defines the Open
Hybrid Cloud
OPEN
CLOUD
open innovation,
open standards,
open APIs,
openness vs.
lockin
38
HYBRID
hybrid deployment
models (physical,
virtual, cloud)
hybrid
architectures
public-privatehybrid cloud
scenarios
Scalable
Portable
On-demand
Resource
Management
Measureable
http://slidesha.re/1gF0PEK
55. OpenStack Metering (Ceilometer)
●
The foundation of billing or charge back systems
●
Concepts
●
●
Compute Pollsters
●
Central Pollster
●
Notifications
●
55
Meters
Collectors
http://slidesha.re/1gF0PEK
59. OpenStack Object Storage (Swift)
●
Concepts
●
●
Containers – Organize Your Data
●
Objects – Your Data
●
59
Accounts
Ring – Internal Data Structure
http://slidesha.re/1gF0PEK
60. OpenStack Object Storage (Swift)
API
Storage
Credit: Mark McLoughlin
60
http://slidesha.re/1gF0PEK
64. OpenStack Image Service (Glance)
ReST API
glance-api
ReS
T
Image
Storage
64
glance-registry
DB
http://slidesha.re/1gF0PEK
65. OpenStack Image Service (Glance) Scaling
Load Balancer
glance-api
glance-api
...
glance-api
glance-registry
Image
Storage
65
DB
http://slidesha.re/1gF0PEK
* Scales horizontally the
same way as the API
82. Boot a Server – Step 1
1
Identity
Object
Storage
82
Dashboard
Image
Service
Orchestration
Block
Storage
Metering
Networking
http://slidesha.re/1gF0PEK
Compute
83. Boot a Server – Step 2
1
Identity
Object
Storage
83
2
Dashboard
Image
Service
Orchestration
Block
Storage
Metering
Networking
http://slidesha.re/1gF0PEK
Compute
84. OpenStack Compute (Nova)
ReST API
nova-api
nova-scheduler
AMQP
nova-conductor
nova-compute
Libvirt+KVM
DB
84
http://slidesha.re/1gF0PEK
85. Boot a Server – Step 3
1
Identity
2
Dashboard
Orchestration
Metering
3
Object
Storage
85
Image
Service
Block
Storage
Networking
http://slidesha.re/1gF0PEK
Compute
86. Boot a Server – Step 4
1
Identity
2
Dashboard
Orchestration
Metering
3
Object
Storage
86
Image
Service
Block
Storage
4
Networking
http://slidesha.re/1gF0PEK
Compute
87. Boot a Server – Step 5
1
Identity
2
Dashboard
Orchestration
Metering
3
Object
Storage
87
Image
Service
Block
Storage
4
Networking
http://slidesha.re/1gF0PEK
Compute
5
88. Boot a Server – Step 6
1
Identity
2
Dashboard
Orchestration
Metering
3
Object
Storage
88
Image
Service
Block
Storage
4
6
Networking
http://slidesha.re/1gF0PEK
Compute
5
89. Openstack => Cloud computing
at your fingertips!
Luan Cestari
February 27 , 2014
1
http://slidesha.re/1gF0PEK
90. Please, let me ask ...
●
Have you heard about Cloud Computing?
●
Is it a hype? What does it means?
●
Are you using any cloud service?
2
Scalable
Portable
On-demand
Resource Management
Measureable
http://slidesha.re/1gF0PEK
91. Do you know OpenStack?
●
●
●
3
Is the OpenStack an OS?
Why would I use instead of the machine in my
enterprise?
What is this Open Source thing and what is related to
this talking?
http://slidesha.re/1gF0PEK
3
92. But what is OpenStack
●
OpenStack is an open source project for
building a private or public
infrastructure-as-a-service (IaaS) cloud
●
OpenStack solution != Virtualization -> let's see why
●
●
4
Cloud!
But that doesn't explain what is the definition of cloud
http://slidesha.re/1gF0PEK
The difference in http://www.slideshare.net/CAinc/cloud-expo-session-fromvirtualization-to-cloud-computing-building-an-effective-pragmatic-reliable-cloud
93. The defining the Cloud Computing: It must be ...
●
Scalable
●
Portable
●
On-demand
●
Resource Management
●
Measureable
From: National Institute of Standards and Technology (NIST)
5
http://slidesha.re/1gF0PEK
5
94. 6
http://slidesha.re/1gF0PEK
Open standards: Advanced Research Projects
Agency Network (ARPANET) collaborative
process(in 60s) led to the birth of the Internet (in
1969)
GNU project, 1983, Richard Stallman
The prior slide described the CIO dilemma being
required to do more with less. This will not happen
the old fashion way and a new more disruptive
approach is needed.
1. more use of OSS
2. new service delivery models – cloud
3. new business models – pay as you go and
subscription based
Most clouds are build on OSS and 80% of the
workloads run on Linux.
Open Standards are essential to ensure portability
95. Linux Powers The Cloud
8 out of 10 clouds
are built on Linux1
Amazon EC2
RackSpace
“Linux is twice as
popular as Windows on
Amazon Web Services.”2
“On the RackSpace
cloud, the split is even
starker: 75 %to 25 %,
again in favor of Linux.”2
1 Linux Adoption Trends 2012: A Survey of Enterprise End Users, Linux Foundation, January 18, 2012
2 Windows 8? It Won’t Win Microsoft’s Biggest Battle, Robert Mcmillan, Wired, October 25, 2012
7
http://slidesha.re/1gF0PEK
Today, Linux is the primary platform for a majority
of cloud-based applications. As a case in point, an
October 2012 Wired magazine article cited a
report by Newvem which illustrated Linux's
dominant position, particularly pertaining to
Amazon Web Services:
At Wired’s request, Newvem — a company that
sells management services to Amazon cloud
customers — took at look at about 41,000 cloud
machines run by several hundred customers. Its
conclusion: Linux is twice as popular as Windows
on Amazon Web Services. It was running on 67
percent of machines, compared to Windows’ 33
percent.
7
96. Virtualization to cloud infrastructure continuum
Virtual Infrastructure Management
Private
Cloud
Server
Virtualization
Drivers
Consolidation
Reduce Capital Expense
Hybrid
Cloud
Distributed
Virtualization
Flexibility & Speed
Reduce Operational Expense
Automation
Less Downtime
Self-Serve Agility
Standardization
IT as a Business
Usage Metering
Choice of CAPEX/OPEX model
Increased Flexibility (up and down)
Visibility
Optimization
Agility
Federation
Control
Automation
Self-Service
Brokering
Derived from Gartner Roadmap: From Virtualization to Cloud Computing (reference slide)
8
http://slidesha.re/1gF0PEK
This slide is derived from the Gartner
roadmap “From Virtualization to Cloud
Computing”, and indicates the various
levels of infrastructure enablement that
IT organizations are engaging in today.
Organizations often field more than one
virtual / cloud project may be doing IT
transformation to IaaS while at the same
time running Cloud program initiatives.
Focus has been on deriving economies
and flexibility at one level via server
virtualization and consolidation. Highly
distributed and sprawled virtual
environments that tend to result form
those initiatives require management
optimization and automation to be
efffectivley controlled.
Infrastructure as a Service (IaaS)
capabilities are then engaged to support
agile, self-service access to IT systems
97. 9
http://slidesha.re/1gF0PEK
IT must change and be able to address the questions
of the prior slide. If IT is not able to adapt and change
the business as their customer will go around IT and
buy services directly from public cloud and SaaS
providers which cannot be in the interest of IT and
the corporation overall because shadow IT will
sprawl
IT needs to provide the benefits of a public could
service by eliminating the potential negative sides of
public service like security concerns, governance,
regulatory restrictions, ...
98. Cloud deployment models
Hybrid
Cloud
Private
Cloud
Privately owned
And managed with
Restricted access (but
Could be externally
hosted)
10
Interoperable
combination
of private and
public cloud.
Community
Cloud
http://slidesha.re/1gF0PEK
Public
Clouds
Service Provider
Owned and managed,
Accessed via the web,
Pay for what you use.
99. Types of Cloud
●
Saas (Software as a service)
●
PaaS (Platform as a service)
●
IaaS (Infrastructure as a service)
11
http://slidesha.re/1gF0PEK
100. Cloud Service Models
IaaS
PaaS
SaaS
APPLICATION
APPLICATION PLATFORM
(JBOSS, PHP, RUBY, ETC)
OPERATING SYSTEM
(RHEL)
VIRTUALIZATION
(RHEV)
HARDWARE
(x86)
STORAGE
(RHS)
12
●
Managed and
Controlled by
Customer (IT, Dev, or
User)
Managed by the Public
or Private Cloud
Offering
Increased Control
Reduced DIY
http://slidesha.re/1gF0PEK
To set the proper context for our discussion, let’s take a look at
the three best known cloud service delivery models,
IaaS or Infrastructure as a Service,
PaaS or Platform as a Service,
And SaaS or Software as a Service.
With each of these some portion of the technology stack is
managed and controlled by the Cloud offering (whether public
or private), and the rest is responsibility of the customer of the
Cloud offering. They must manage and maintain the portion
of the stack that is not managed by the Cloud offering. The
benefit is that the customer can also CONTROL the design of
this portion of the stack.
For Infrastructure-as-a-Service, or IaaS, the customer must
manage and control everything from the Operating System up.
They must install the OS, the middleware, and the application
code. This ultimately becomes only a small reduction in effort
compared to bare-metal server management.
101. A cloud provider view of shared responsibility for
security
Source: Cloud Security Alliance
13
http://slidesha.re/1gF0PEK
13
103. Cloud types & deployment models
Hybrid
Private
Clouds
Public
Clouds
Community
Cloud
SaaS
Google Apps
Salesforce
Many more
PaaS
IaaS
OpenShift
Force.com
Azure
Amazon AWS
RackSpace
OpenStack
vCloud Director
Software-as-a-Service
(SaaS) (hosted apps)
Platform-as-a-Service (PaaS)
(dev platform, apps middleware)
Infrastructure-as-a-Service (IaaS)
(compute, storage, network)
15
http://slidesha.re/1gF0PEK
End-users
Developers
DevOps
IT Admins
104. Streamlining App Dev with PaaS
Physical
How to Build an App:
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
Have Idea
Get Budget
Submit hardware acquisition
request
Wait
Get Hardware
Rack and Stack Hardware
Install Operating System
Install Operating System
Patches/Fix-Packs
Create user Accounts
Deploy framework/appserver
Deploy testing tools
Test testing tools
Code
Configure Prod servers (and buy
them if needed)
Push to Prod
Launch
Order more servers to meet
demand
Wait…
Deploy new servers
Etc.
16
Virtualized
How to Build an App:
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
Have Idea
Get Budget
Submit VM Request request
Wait
Deploy
framework/appserver
Deploy testing tools
Test testing tools
Code
Configure Prod VMs
Push to Prod
Launch
Request More Prod VMs to
meet demand
Wait
Deploy app to new VMs
Etc.
With PaaS
How to Build an App:
1.
2.
3.
4.
5.
6.
Have Idea
Get Budget
Code
Test
Launch
Automatically Scale
“The use of Platform-as-a-Service technologies will
enable IT organizations to become more agile and
more responsive to the business needs.” –Gartner*
More info: www.openshift.com
http://slidesha.re/1gF0PEK
16
PaaS allows us to make this leap from a Craftwork methodology
to an Assembly Line methodology. PaaS both streamlines and
standardizes the Application Development process.
Instead of the unique, one-off processes that each application
project used to go through, Utilization of a PaaS allows the
deployment of a standardized methodology with a few distinct
repeatable steps.
Once the idea and budget are secured, then the App Dev team
just simply starts working with the PaaS to code, test, and
launch their application. PaaS provides the platform
environment that allows Developers to start coding immediately.
Server deployment, configuration and administration is all
largely automated. This reduces the burden on Operations and
reduces the delays that impact Development.
And many industry analysts agree that the utilization of PaaS is
going to be the key to allowing IT to be the demands of the
business.
<next slide>
105. Don’t take taxonomies too literally
●
IaaS can blend into PaaS
●
●
PaaS can blend into SaaS
●
●
“Value-add” services like DynamoDB, Elastic
MapReduce
PaaS anchored to a SaaS environment
Taxonomy part of broader ecosystem
●
●
APIs/services
●
17
Hybrid cloud IaaS management (CloudForms)
Development tooling
http://slidesha.re/1gF0PEK
106. Back to OpenStack, its history...
●
2010 - Rackspace and Nasa joins forces
●
●
18
Rackspace's Cloud Files platform and NASA's Nebula
OpenStack Object Store (Swift) and OpenStack
Compute Nova
http://slidesha.re/1gF0PEK
114. Supported Hypervisors
●
KVM
●
LXC (through libvirt)
●
QEMU
●
UML
●
VMWare vSphere
●
Xen
●
Hyper-V
●
Bare Metal
●
Docker
More: https://wiki.openstack.org/wiki/HypervisorSupportMatrix
26
http://slidesha.re/1gF0PEK
KVM - Kernel-based Virtual Machine. The virtual disk
formats that it supports is inherited from QEMU since
it uses a modified QEMU program to launch the
virtual machine. The supported formats include raw
images, the qcow2, and VMware formats.
LXC - Linux Containers (through libvirt), use to run
Linux-based virtual machines.
QEMU - Quick EMUlator, generally only used for
development purposes.
UML - User Mode Linux, generally only used for
development purposes.
VMWare vSphere 4.1 update 1 and newer, runs
VMWare-based Linux and Windows images through
a connection with a vCenter server or directly with an
115. Supported Images
●
●
vhd
●
vmdk
●
iso
●
qcow2
●
vdi
●
aki
●
ari
●
27
raw
ami
http://slidesha.re/1gF0PEK
KVM - Kernel-based Virtual Machine. The virtual disk
formats that it supports is inherited from QEMU since
it uses a modified QEMU program to launch the
virtual machine. The supported formats include raw
images, the qcow2, and VMware formats.
LXC - Linux Containers (through libvirt), use to run
Linux-based virtual machines.
QEMU - Quick EMUlator, generally only used for
development purposes.
UML - User Mode Linux, generally only used for
development purposes.
VMWare vSphere 4.1 update 1 and newer, runs
VMWare-based Linux and Windows images through
a connection with a vCenter server or directly with an
116. How does it look like
28
http://slidesha.re/1gF0PEK
Log in, using your account from keystone
117. How does it look like
29
http://slidesha.re/1gF0PEK
Create an instance
Give it a name
Set parameters (CPUs, RAM, Disk, ...)
Shows you how your request stacks up against your
available quota
118. How does it look like
30
http://slidesha.re/1gF0PEK
Manage existing instances
Networking Parameters
VNC Console
Manage VM lifecycle (such as reboot and Terminate)
119. How to install OpenStack, the easy way
sudo yum install -y http://rdo.fedorapeople.org/rdo-release.rpm
sudo yum install -y openstack-packstack
packstack --allinone
31
http://slidesha.re/1gF0PEK
120. RED HAT LEADS THROUGH OPEN INNOVATION
32
Gerry
http://slidesha.re/1gF0PEK
121. OpenStack Progression
●
●
●
●
●
●
Open source, communitydeveloped (upstream)
software
Founded by Rackspace
Hosting and NASA
●
●
●
Managed by the OpenStack
Foundation
Vibrant group of developers
collaborating on open source
cloud infrastructure
Software distributed under
the Apache 2.0 license
No certifications, no support
●
●
●
●
Latest OpenStack software,
packaged in a managed
open source community
●
●
Facilitated by Red Hat
Aimed at architects and
developers who want to
create, test, collaborate
●
Freely available, not for sale
●
Six-month release cadence
mirroring community
●
No certification, no support
Installs on Red Hat and
derivatives
●
DOC144908-20130711R4
Chuck
Enterprise-hardened
OpenStack software
Delivered with an
enterprise life cycle
Six-month release cadence
offset from community
releases to allow testing
Aimed at long-term
production deployments
Certified hardware and
software through the Red
Hat OpenStack Cloud
Infrastructure Partner
Network
Supported by Red Hat
RED HAT ENTERPRISE LINUX – FOUNDATION FOR THE OPEN HYBRID CLOUD
122. PackStack Overview
●
Installer appropriate for smaller scale OpenStack
deployments.
●
Driven by asking questions or an “answer file”
●
Uses SSH and Puppet to set up all nodes
34
http://slidesha.re/1gF0PEK
123. Differences between versions?
Upstream
Midstream
Downstream
Source from GIT
Packages from Brew
Unit Tests
rdopkg
Job Builder/Runner
DevStack
Torpedo / Tempest
Tempest
Tempest
RDO Environments
RHOS Environments
SmokeStack
35
SmokeStack Trunk
RDO Release / Poodle
RHOS Release
http://slidesha.re/1gF0PEK
35
124. Differences between versions?
●
Upstream:
●
●
Detect rpm based install errors via smokestack
●
●
Contribute tempest tests upstream
Enterprise Linux devstack
Midstream:
●
●
Qualify RDO across supported environments
●
●
Detect packstack, foreman based install errors
Improve the feedback to development
Downstream:
●
●
36
Qualify RHOS across supported environments
Scale and Performance test
http://slidesha.re/1gF0PEK
36
125. ●
●
●
●
●
●
●
●
In many ways, the RHOS and RHEL value propositions are identical with Red Hat
OpenStack (RHOS) being to OpenStack what Red Hat Enterprise Linux (RHEL) is to
Linux.
Red Hat has the proven capability of preparing and delivering an open source
distribution for enterprise use and providing a stable, secure, and common platform
that customers, ISV partners, IHV partners, and service provider partners can use to
test and certify their software and hardware solutions.
Operational competency necessary for maintain a stable platform with backported
bugfixes and features. World-class processes and procedures required for the
rigorous testing and QA necessary for enterprise-class products
Worldwide enterprise class support service with consulting and training services
Certified partner program
Guest certifications – include Microsoft Windows (SVVP)
Hardware certifications – broadest list of certified server, storage and networking
platforms
Leadership and influence of the upstream OpenStack product development process
with the ability and motivation to act as customer advocate with upstream community
126. Red Hat Enterprise Linux Defines the Open
Hybrid Cloud
OPEN
CLOUD
open innovation,
open standards,
open APIs,
openness vs.
lockin
38
HYBRID
hybrid deployment
models (physical,
virtual, cloud)
hybrid
architectures
public-privatehybrid cloud
scenarios
Scalable
Portable
On-demand
Resource
Management
Measureable
http://slidesha.re/1gF0PEK
128. More info in the notes of the slides
40
●
●
●
http://slidesha.re/1gF0PEK
Upstream CI http://www.youtube.com/watch?
v=v0IWQP_m_38
http://ci.openstack.org/devstack-gate.html
http://status.openstack.org/zuul
https://smokestack.openstack.org/
http://stackalytics.com/
Name:
https://wiki.openstack.org/wiki/Release_Naming
Stat:
http://activity.openstack.org/dash/browser/
http://stackalytics.com/
http://www.ohloh.net/p/openstack
https://access.redhat.com/site/documentation/enUS/Red_Hat_OpenStack/2/html/Getting_Started_Gui
de/ch01.html
132. OpenStack Identity (Keystone)
●
Important Concepts
●
Tenants – Groups of Users
●
Users
●
Roles
●
Tokens
●
Services
44
http://slidesha.re/1gF0PEK
Tenants – Groups of users (could be a team, a whole
company, whatever you want), share resources
Users - ... users
Roles - admin or not is all it's used for so far
Tokens – auth to get a token. Token allows access to
all other OpenStack APIs.
133. OpenStack Identity (Keystone)
ReST API
keystone
Token
45
Identity
Services
http://slidesha.re/1gF0PEK
Presents a REST API (most services do)
Separate backends for tokens, identity, and services
Tokens – db, memcache
Identity – db, LDAP
Services – db, flat file backed
134. OpenStack Identity (Keystone) Scaling
Load Balancer
keystone
46
keystone
...
keystone
http://slidesha.re/1gF0PEK
Horizontal Scaling with a load balancer
This pattern will be seen all over OpenStack.
HAProxy is a software load balancer that we support.
All services use the same storage backend (if
MySQL, use strategies to scale MySQL)
137. OpenStack Dashboard (Horizon) Scaling
Load Balancer
httpd
horizon
httpd
horizon
httpd
horizon
OpenStack APIs
Session Storage
49
...
http://slidesha.re/1gF0PEK
Horizontally scale out httpd w/ a load balancer
All horizon instances use the same OpenStack APIs
What's different: Session state shared via Django's
session engine, so whichever backends supported
there
default is local cache
others are memcached, db, db+caching, signed
cookies
139. OpenStack Orchestration (Heat)
●
Consumes all other OpenStack APIs
●
Important concept: template defined stacks
51
http://slidesha.re/1gF0PEK
Why Heat is awesome: Template defines your
application in terms of all of the resources needed
(instances, networks, database, load balancer)
Version your deployment like your software
Repeatable complex deployments that are fully
automated
AWS Cloudformation template compatible, but with
added OpenStack resources
AWS Cloudformation API
Also has an OpenStack API, native template syntax
in the works
Implements HA, auto scaling, was shown in keynote
this AM
140. OpenStack Orchestration (Heat)
ReST API / CFN API
heat-api
AMQP
heat-engine
OpenStack APIs
DB
52
http://slidesha.re/1gF0PEK
Heat-api – serves up the ReST API and/or CFN API
First time AMQP comes up. Explain it. Advanced
Message Queueing Protocol. Scalable messaging
between applications. We use Qpid.
Api talks to engine via AMQP
Engine does the real work of setting up the stack.
Makes many API calls to other OpenStack services
to set up all of the resources defined in the template
More about AMQP
http://docs.openstack.org/developer/nova/devref/rpc.
html
143. OpenStack Metering (Ceilometer)
●
The foundation of billing or charge back systems
●
Concepts
●
●
Compute Pollsters
●
Central Pollster
●
Notifications
●
55
Meters
Collectors
http://slidesha.re/1gF0PEK
Meters – any type of usage data, transformed into
billable items by your own system
How much of a thing, how long, whatever makes
sense (CPU hours, bandwidth)
Notifications – Most projects emit usage notifications
via AMQP
Compute pollsters – poll for other data on compute
nodes
Central pollster – plug point, poll for data from
elsewhere
Collectors – collect meters from all of these places
and store it (in a db, mongodb, mysql/postgres)
145. OpenStack Metering (Ceilometer)
57
http://slidesha.re/1gF0PEK
Another view of the architecture, this one showing
actual services that run, including the API service
A few services funneling meters to the collector
Both collector and API access the backend store
Talk about scaling from this slide:
Horizontal scaling of API, compute pollsters, collector
147. OpenStack Object Storage (Swift)
●
Concepts
●
●
Containers – Organize Your Data
●
Objects – Your Data
●
59
Accounts
Ring – Internal Data Structure
http://slidesha.re/1gF0PEK
Object Storage ... store and retrieve blobs of data
Note that swift manages replicas of data across
multiple storage nodes
Ring internal, basically a distributed hash table, but
exposed to you as an admin. You tell swift info about
your deployment so it can build the ring
148. OpenStack Object Storage (Swift)
API
Storage
Credit: Mark McLoughlin
60
http://slidesha.re/1gF0PEK
Basically two node types, API nodes and storage
nodes
Storage nodes hold a subset of data, one of the
replicas of it
How does scaling work with these 2 node types?
149. OpenStack Object Storage (Swift) Scaling
Load balancer
Proxy
Storage
61
Proxy
Storage
...
Proxy
...
Storage
http://slidesha.re/1gF0PEK
Horizontal scaling like the rest
What's different here is the storage nodes: can run
as many as you need, the ring is generated to
account for added/removed nodes, rebalancing done
as needed
150. OpenStack Image Service (Glance)
Identity
Object
Storage
62
Dashboard
Image
Service
Orchestration
Block
Storage
Metering
Networking
http://slidesha.re/1gF0PEK
Compute
151. OpenStack Image Service (Glance)
●
Concepts
●
●
Metadata
●
63
Images
Storage Backends
http://slidesha.re/1gF0PEK
Registry for Images – disk images for creating VMs
Example metadata:
- name of the image
- size
- access control (public or private?)
- hardware info, hw_vif_driver=e1000 vs virtio
Storage backends
- filesystem backed (local, or glusterfs)
- Swift
152. OpenStack Image Service (Glance)
ReST API
glance-api
ReS
T
Image
Storage
64
glance-registry
DB
http://slidesha.re/1gF0PEK
153. OpenStack Image Service (Glance) Scaling
Load Balancer
glance-api
glance-api
...
glance-api
glance-registry
Image
Storage
65
* Scales horizontally the
same way as the API
DB
http://slidesha.re/1gF0PEK
Horizontal scaling of the api as usual
All accessing the same backend image storage
Glance-registry simplified for the diagram, but can
horizontally scale with a load balancer as well
155. OpenStack Block Storage (Cinder)
●
Concepts
●
●
Snapshots
●
67
Volumes
Storage Backends
http://slidesha.re/1gF0PEK
Volumes can be hotplugged to running Vms, but
that's handled by the compute service
Storage backends
- LVM based storage the default, iSCSI
- Various storage vendors' appliances, iSCSI
- NFS, file backed
- GlusterFS, file backed
156. OpenStack Block Storage (Cinder)
ReST API
cinder-api
cinder-scheduler
AMQP
cinder-volume
DB
68
http://slidesha.re/1gF0PEK
ReST API for external, AMQP internal, as seen
before
Scheduler for placement logic
Volume servers for managing storage
157. OpenStack Block Storage (Cinder) Scaling
Load Balancer
cinder-api
cinder-api
cinder-scheduler
cinder-scheduler
cinder-scheduler
...
cinder-api
AMQP
cinder-volume
cinder-volume
cinder-volume
69
http://slidesha.re/1gF0PEK
Horizontal scaling using patterns seen before
159. OpenStack Networking (...)
●
Concepts
●
●
Routers
●
Subnets
●
Ports
●
71
Networks
Vendor plugins
http://slidesha.re/1gF0PEK
Network. An isolated virtual layer-2 domain. Virtual
networks for your Vms, or provider networks that
map to physical networks in your data center
Routers – Connectivity to external networks, connect
multiple virtual networks together
Subnet. An IP address block. IPs assigned to ports.
Port. A virtual, or logical, switch port on a specified
network. Instances get attached to these.
Like Cinder, Quantum has lots of plugins for various
vendors' networking technologies
The default open source stuff: linux bridges,
Openvswitch
160. OpenStack Networking (...)
72
http://slidesha.re/1gF0PEK
Network. An isolated virtual layer-2 domain. Virtual
networks for your Vms, or provider networks that
map to physical networks in your data center
Routers – Connectivity to external networks, connect
multiple virtual networks together
Subnet. An IP address block. IPs assigned to ports.
Port. A virtual, or logical, switch port on a specified
network. Instances get attached to these.
Like Cinder, Quantum has lots of plugins for various
vendors' networking technologies
The default open source stuff: linux bridges,
Openvswitch
162. OpenStack Compute (Nova)
●
Concepts
●
●
Flavors / Instance Types
●
●
Instances / Servers
Virt drivers
OpenStack API and EC2 API
74
http://slidesha.re/1gF0PEK
Instances – Vms
Flavors – resources (CPUs, RAM, disk)
Virt drivers – choice of hypervisor, KVM most
popular, even supports bare metal provisioning!
Native OpenStack ReST API
EC2 API
163. OpenStack Compute (Nova)
ReST API
nova-api
nova-scheduler
AMQP
nova-conductor
nova-compute
Libvirt+KVM
DB
75
http://slidesha.re/1gF0PEK
Looks very similar to Cinder, (because Cinder came
from Nova)
Api, scheduler, compute for managing hypervisor
nodes
What's different: nova-conductor
For security reasons, want to isolate compute nodes
as much as possible, so no direct db access
164. OpenStack Compute (Nova) Scaling (Step 1)
Load Balancer
nova-api
nova-scheduler
nova-conductor
AMQP
nova-compute
DB
76
Libvirt+KVM
Libvirt+KVM
Libvirt+KVM
http://slidesha.re/1gF0PEK
This works ... to a point (can still support many
thousands of nodes)
DB and message broker become a pain point
There's another level scaling being worked on now:
cells, will show you how cells works, starting with this
picture
165. OpenStack Compute (Nova) Scaling (Step 2)
API Cell
AMQP
Compute Cell
77
Compute Cell
...
Compute Cell
http://slidesha.re/1gF0PEK
Cells, each with their own message broker and
database
Communication over AMQP between the nova-cells
service in each cell
Cells can be local or geographically distributed, all
under a single API endpoint
Next diagrams show what's in a cell
166. OpenStack Compute (Nova) Scaling (Step 2)
Compute Cell
nova-cells
nova-scheduler
AMQP
nova-conductor
nova-compute
DB
78
Libvirt+KVM
Libvirt+KVM
Libvirt+KVM
http://slidesha.re/1gF0PEK
Removed nova-api, added nova-cells service
Its own db and message broker
167. OpenStack Compute (Nova) Scaling (Step 2)
ReST API
API Cell
Load Balancer
nova-api
AMQP
nova-cells
DB
79
http://slidesha.re/1gF0PEK
Removed everything but nova-api, added nova-cells
service
168. OpenStack Compute (Nova) Scaling (Step 2)
API Cell
AMQP
Compute Cell
80
Compute Cell
...
Compute Cell
http://slidesha.re/1gF0PEK
Again, this is what it looks like
Large amounts of nodes grouped into cells,
federated using AMQP
170. Boot a Server – Step 1
1
Identity
Object
Storage
Dashboard
Image
Service
82
Orchestration
Block
Storage
Metering
Networking
Compute
http://slidesha.re/1gF0PEK
Step 1 – get a token
Single keystone service, stores token in its token db
171. Boot a Server – Step 2
1
Identity
Object
Storage
83
2
Dashboard
Image
Service
Orchestration
Block
Storage
Metering
Networking
Compute
http://slidesha.re/1gF0PEK
Step 2 – Request server from compute service
Next slide has nova diagram to talk about how it gets
processed
172. OpenStack Compute (Nova)
ReST API
nova-api
nova-scheduler
AMQP
nova-conductor
nova-compute
Libvirt+KVM
DB
84
http://slidesha.re/1gF0PEK
Boot a server
API -> scheduler -> compute
173. Boot a Server – Step 3
1
Identity
2
Dashboard
Orchestration
Metering
3
Object
Storage
85
Image
Service
Block
Storage
Networking
Compute
http://slidesha.re/1gF0PEK
Step 3 – Download image to compue node if
necessary
Talks to the glance-api service, image data streamed
from storage backend
174. Boot a Server – Step 4
1
Identity
2
Dashboard
Orchestration
Metering
3
Object
Storage
86
Image
Service
Block
Storage
4
Networking
Compute
http://slidesha.re/1gF0PEK
Step 4 –Request block storage volume info so that it
can be attached to an instance
Talks to the cinder-api service
175. Boot a Server – Step 5
1
Identity
2
Dashboard
Orchestration
Metering
3
Object
Storage
87
Image
Service
Block
Storage
4
Networking
Compute
5
http://slidesha.re/1gF0PEK
Step 5 –Request network information
Talks to quantum-api service, tell it the network(s)
that were requested, quantum will allocate port
176. Boot a Server – Step 6
1
Identity
2
Dashboard
Orchestration
Metering
3
Object
Storage
88
Image
Service
Block
Storage
4
6
Networking
Compute
5
http://slidesha.re/1gF0PEK
Step 6 –With all info needed, boot the VM
Back to the nova-compute service, boots the VM by
giving libvirt all necessary info