Understanding OpenStack Deployments - Chris Hoge, OpenStack Foundation

1. Understanding OpenStack
Deployments
Chris Hoge
@hogepodge
!
Interop Engineer
OpenStack Foundation

2. Who is this guy?

3. !
What is OpenStack?

4. • Identity - Keystone
• Network - Neutron
• Compute - Nova
• Block Storage - Cinder
• Image Service - Glance
• Dashboard - Horizon
• Object Storage - Swift
• Telemetry - Ceilometer
• Orchestration - Heat
• Database - Trove
• Map Reduce - Sahara
• More and growing…

5. What does Puppet
have to do with this?

6. A Cartoon View of OpenStack Architecture
Control
Database
Message
Queue
Horizon
Keystone
Neutron API
Nova API
Glance API
Cinder API
Network
Neutron
Agents
Neutron
Agents
Neutron
Agents
Nova
Scheduler
Glance
Registry
Cinder
Scheduler
Compute
Compute
Compute
Compute
Nova
Compute
Nova
Compute
Nova
Compute
Nova
Compute
Network
Agent
Network
Agent
Network
Agent
Network
Agent
Storage
Storage
Storage
Storage
Cinder
Storage
Cinder
Storage
Cinder
Storage
Cinder
Storage

7. Network
Neutron
Agents
Neutron
Agents
Neutron
Agents
Control
Database
Message
Queue
Horizon
Keystone
Neutron
API
Nova API
Glance
API
Cinder
API
Nova
Scheduler
Glance
Registry
Cinder
Scheduler
Compute
Nova
Compute
Nova
Compute
Nova
Compute
Nova
Compute
Network
Network
Network
Network
Agent
Storage
Cinder
Storage
Cinder
Storage
Cinder
Storage
Cinder
Storage
API Network External Network
Data Network
Administrative Network

8. “Le Grand Tour” of the puppet-openstack Modules
• 1:1 correspondence with
OpenStack projects.
• Community developed in
StackForge!
• Builds on Canonical and Red Hat
packaging.
• Tracks the major 6-month
OpenStack releases.
• Modules available on Puppet Forge.

9. OpenStack Composition Modules
• stackforge/packstack
• theforeman/staypuft
• stackforge/fuel-library
• stackforge/puppet-openstack_builder
• enovance/puppet-openstack-cloud
• puppetlabs/puppetlabs-openstack

10. !
class openstack::profile::base {
!
# make sure the parameters are initialized
include ::openstack
!
# everyone also needs to be on the same clock
class { '::ntp': }
!
# all nodes need the OpenStack repository
class { '::openstack::resources::repo': }
!
# database connectors
class { '::openstack::resources::connectors': }
!
$management_network = $::openstack::config::network_management
$management_address = ip_for_network($management_network)
$controller_management_address = $::openstack::config::controller_address_management
$storage_management_address = $::openstack::config::storage_address_management
$management_matches = ($management_address == $controller_management_address)
$storage_management_matches = ($management_address == $storage_management_address)
!
$api_network = $::openstack::config::network_api
$api_address = ip_for_network($api_network)
$controller_api_address = $::openstack::config::controller_address_api
$storage_api_address = $::openstack::config::storage_address_api
!
$api_matches = ($api_address == $controller_api_address)
$storage_api_matches = ($api_address == $storage_api_address)
!
$is_controller = ($management_matches and $api_matches)
$is_storage = ($storage_management_matches and $storage_api_matches)
}

11. Network
Neutron
Agents
Neutron
Agents
Neutron
Agents
Control
Database
Message
Queue
Horizon
Keystone
Neutron
API
Nova API
Glance
API
Cinder
API
Nova
Scheduler
Glance
Registry
Cinder
Scheduler
Compute
Nova
Compute
Nova
Compute
Nova
Compute
Nova
Compute
Network
Network
Network
Network
Agent
Storage
Cinder
Storage
Cinder
Storage
Cinder
Storage
Cinder
Storage
API Network External Network
Data Network
Administrative Network

12. # The profile to install rabbitmq
!
class openstack::profile::rabbitmq {
!
$management_address = $::openstack::config::controller_address_management
!
class { '::nova::rabbitmq':
userid => $::openstack::config::rabbitmq_user,
password => $::openstack::config::rabbitmq_password,
cluster_disk_nodes => [$management_address],
rabbitmq_class => '::rabbitmq',
}
!
if $::osfamily == 'RedHat' {
package { 'erlang':
ensure => installed,
before => Package['rabbitmq-server'],
require => Yumrepo['erlang-solutions'],
}
}
}

13. # The profile to install an OpenStack specific mysql server
!
class openstack::profile::mysql {
!
class { '::mysql::server':
root_password => $::openstack::config::mysql_root_password,
restart => true,
override_options => {
'mysqld' => {
'bind_address' =>
$::openstack::config::controller_address_management,
'default-storage-engine' => 'innodb',
}
}
}
!
!
class { '::mysql::bindings':
python_enable => true,
ruby_enable => true,
}
!
class { 'mysql::server::account_security': }
!
}

14. define openstack::resources::database () {
class { "::${title}::db::mysql":
user => $title,
password => $::openstack::config::mysql_service_password,
dbname => $title,
allowed_hosts => $::openstack::config::mysql_allowed_hosts,
mysql_module => '2.2',
require => Anchor['database-service'],
}
}

15. Network
Neutron
Agents
Neutron
Agents
Neutron
Agents
Control
Database
Message
Queue
Horizon
Keystone
Neutron
API
Nova API
Glance
API
Cinder
API
Nova
Scheduler
Glance
Registry
Cinder
Scheduler
Compute
Nova
Compute
Nova
Compute
Nova
Compute
Nova
Compute
Network
Network
Network
Network
Agent
Storage
Cinder
Storage
Cinder
Storage
Cinder
Storage
Cinder
Storage
API Network External Network
Data Network
Administrative Network

16. # The profile to install the Keystone service
class openstack::profile::keystone {
openstack::resources::controller { 'keystone': }
openstack::resources::database { 'keystone': }
openstack::resources::firewall { 'Keystone API': port => '5000', }
!
include ::openstack::common::keystone
!
class { 'keystone::endpoint':
public_address => $::openstack::config::controller_address_api,
admin_address => $::openstack::config::controller_address_management,
internal_address => $::openstack::config::controller_address_management,
region => $::openstack::config::region,
}
!
$tenants = $::openstack::config::keystone_tenants
$users = $::openstack::config::keystone_users
create_resources('openstack::resources::tenant', $tenants)
create_resources('openstack::resources::user', $users)
}

17. define openstack::resources::user (
$password,
$tenant,
$email,
$admin = false,
$enabled = true,
) {
keystone_user { "$name":
ensure => present,
enabled => $enabled,
password => $password,
tenant => $tenant,
email => $email,
}
!
if $admin == true {
keystone_user_role { "$name@$tenant":
roles => ['_member_', 'admin'],
ensure => present,
}
} else {
keystone_user_role { "$name@$tenant":
roles => ['_member_'],
ensure => present,
}
}
}

18. class openstack::common::keystone {
if $::openstack::profile::base::is_controller {
$admin_bind_host = '0.0.0.0'
} else {
$admin_bind_host = $::openstack::config::controller_address_management
}
!
class { '::keystone':
admin_token => $::openstack::config::keystone_admin_token,
sql_connection => $::openstack::resources::connectors::keystone,
verbose => $::openstack::config::verbose,
debug => $::openstack::config::debug,
enabled => $::openstack::profile::base::is_controller,
admin_bind_host => $admin_bind_host,
mysql_module => '2.2',
}
!
class { '::keystone::roles::admin':
email => $::openstack::config::keystone_admin_email,
password => $::openstack::config::keystone_admin_password,
admin_tenant => 'admin',
}
}

19. # The profile to set up the Nova controller (several services)
class openstack::profile::nova::api {
openstack::resources::controller { 'nova': }
openstack::resources::database { 'nova': }
openstack::resources::firewall { 'Nova API': port => '8774', }
openstack::resources::firewall { 'Nova Metadata': port => '8775', }
openstack::resources::firewall { 'Nova EC2': port => '8773', }
openstack::resources::firewall { 'Nova S3': port => '3333', }
openstack::resources::firewall { 'Nova novnc': port => '6080', }
!
class { '::nova::keystone::auth':
password => $::openstack::config::nova_password,
public_address => $::openstack::config::controller_address_api,
admin_address => $::openstack::config::controller_address_management,
internal_address => $::openstack::config::controller_address_management,
region => $::openstack::config::region,
cinder => true,
}
!
include ::openstack::common::nova
}

20. class openstack::common::nova ($is_compute = false) {
$is_controller = $::openstack::profile::base::is_controller
$management_network = $::openstack::config::network_management
$management_address = ip_for_network($management_network)
$storage_management_address = $::openstack::config::storage_address_management
$controller_management_address =
$::openstack::config::controller_address_management
!
class { '::nova':
sql_connection => $::openstack::resources::connectors::nova,
glance_api_servers => "http://${storage_management_address}:9292",
memcached_servers => ["${controller_management_address}:11211"],
rabbit_hosts => [$controller_management_address],
rabbit_userid => $::openstack::config::rabbitmq_user,
rabbit_password => $::openstack::config::rabbitmq_password,
debug => $::openstack::config::debug,
verbose => $::openstack::config::verbose,
mysql_module => '2.2',
}
!
nova_config { 'DEFAULT/default_floating_pool': value => 'public' }
!
class { '::nova::api':
admin_password => $::openstack::config::nova_password,
auth_host => $controller_management_address,
enabled => $is_controller,
neutron_metadata_proxy_shared_secret =>
$::openstack::config::neutron_shared_secret,
}
!
class { '::nova::vncproxy':
host => $::openstack::config::controller_address_api,
enabled => $is_controller,
}
!
class { [
'nova::scheduler',
'nova::objectstore',
'nova::cert',
'nova::consoleauth',
'nova::conductor'
]:
enabled => $is_controller,
}
!
class { '::nova::compute':
enabled => $is_compute,
vnc_enabled => true,
vncserver_proxyclient_address => $management_address,
vncproxy_host => $::openstack::config::controller_address_api,
}
!
class { '::nova::compute::neutron': }
!
class { '::nova::network::neutron':
neutron_admin_password => $::openstack::config::neutron_password,
neutron_region_name => $::openstack::config::region,
neutron_admin_auth_url => "http://${controller_management_address}:35357/v2.0",
neutron_url => "http://${controller_management_address}:9696",
vif_plugging_is_fatal => false,
vif_plugging_timeout => '0',
}

21. Network
Neutron
Agents
Neutron
Agents
Neutron
Agents
Control
Database
Message
Queue
Horizon
Keystone
Neutron
API
Nova API
Glance
API
Cinder
API
Nova
Scheduler
Glance
Registry
Cinder
Scheduler
Compute
Nova
Compute
Nova
Compute
Nova
Compute
Nova
Compute
Network
Network
Network
Network
Agent
Storage
Cinder
Storage
Cinder
Storage
Cinder
Storage
Cinder
Storage
API Network External Network
Data Network
Administrative Network

22. # The puppet module to set up a Nova Compute node
!
class openstack::profile::nova::compute {
$management_network = $::openstack::config::network_management
$management_address = ip_for_network($management_network)
!
class { 'openstack::common::nova':
is_compute => true,
}
!
class { '::nova::compute::libvirt':
libvirt_type => $::openstack::config::nova_libvirt_type,
vncserver_listen => $management_address,
}
!
file { '/etc/libvirt/qemu.conf':
ensure => present,
source => 'puppet:///modules/openstack/qemu.conf',
mode => '0644',
notify => Service['libvirt'],
}
!
Package['libvirt'] -> File['/etc/libvirt/qemu.conf']
}

23. class openstack::profile::neutron::agent {
include ::openstack::common::neutron
include ::openstack::common::ovs
}
!

24. class openstack::common::neutron {
$controller_management_address = $::openstack::config::controller_address_management
$data_network = $::openstack::config::network_data
$data_address = ip_for_network($data_network)
# neutron auth depends upon a keystone configuration
include ::openstack::common::keystone
!
class { '::neutron':
rabbit_host => $controller_management_address,
core_plugin => 'neutron.plugins.ml2.plugin.Ml2Plugin',
allow_overlapping_ips => true,
rabbit_user => $::openstack::config::rabbitmq_user,
rabbit_password => $::openstack::config::rabbitmq_password,
debug => $::openstack::config::debug,
verbose => $::openstack::config::verbose,
service_plugins => ['neutron.services.l3_router.l3_router_plugin.L3RouterPlugin',
'neutron.services.loadbalancer.plugin.LoadBalancerPlugin',
'neutron.services.vpn.plugin.VPNDriverPlugin',
'neutron.services.firewall.fwaas_plugin.FirewallPlugin',
'neutron.services.metering.metering_plugin.MeteringPlugin'],
}
!
class { '::neutron::keystone::auth':
password => $::openstack::config::neutron_password,
public_address => $::openstack::config::controller_address_api,
admin_address => $::openstack::config::controller_address_management,
internal_address => $::openstack::config::controller_address_management,
region => $::openstack::config::region,
}
!
class { '::neutron::server':
auth_host => $::openstack::config::controller_address_management,
auth_password => $::openstack::config::neutron_password,
database_connection => $::openstack::resources::connectors::neutron,
enabled => $::openstack::profile::base::is_controller,
sync_db => $::openstack::profile::base::is_controller,
mysql_module => '2.2',
}
!
class { '::neutron::server::notifications':
nova_url => "http://${controller_management_address}:8774/v2/",
nova_admin_auth_url => "http://${controller_management_address}:35357/v2.0/",
nova_admin_password => $::openstack::config::nova_password,
nova_region_name => $::openstack::config::region,
}
}

25. class openstack::common::ovs {
$data_network = $::openstack::config::network_data
$data_address = ip_for_network($data_network)
$enable_tunneling = $::openstack::config::neutron_tunneling # true
$tunnel_types = $::openstack::config::neutron_tunnel_types #['gre']
$tenant_network_type = $::openstack::config::neutron_tenant_network_type # ['gre']
$type_drivers = $::openstack::config::neutron_type_drivers # ['gre']
$mechanism_drivers = $::openstack::config::neutron_mechanism_drivers # ['openvswitch']
$tunnel_id_ranges = $::openstack::config::neutron_tunnel_id_ranges # ['1:1000']
!
class { '::neutron::agents::ml2::ovs':
enable_tunneling => $enable_tunneling,
local_ip => $data_address,
enabled => true,
tunnel_types => $tunnel_types,
}
!
class { '::neutron::plugins::ml2':
type_drivers => $type_drivers,
tenant_network_types => $tenant_network_type,
mechanism_drivers => $mechanism_drivers,
tunnel_id_ranges => $tunnel_id_ranges
}
}

26. How You Can Get Involved
• Sign up to be an OpenStack Contributor!
https://wiki.openstack.org/wiki/How_To_Contribute
• Review the Code!
https://review.openstack.org
• Write the Code!
https://wiki.openstack.org/wiki/Gerrit_Workflow
• Hang out with the Amazing Devs!
#puppet-openstack on Freenode
• Share and learn!
puppet-openstack mailing list on Google Groups.

27. QuTehasnkt iyooun! s?