Building complete private cloud architectures that support both traditional application virtualization and emerging distributed cloud systems requires compute and object storage storages. Together Apache CloudStack and Basho's Riak CS provide the infrastructure to support both of workloads. The presentation covers the following topics:
1. Why Private Cloud?
2. Anatomy of a Private Cloud
3. Building a Apache CloudStack Compute Offering
4. Large Object Storage using Riak CS
5. Your Own Private Cloud: The Riak CS Apache CloudStack Integration Roadmap
This presentation was originally delivered to the Silicon Valley Riak group on 27 June 2013.
Human Factors of XR: Using Human Factors to Design XR Systems
Making Cloudy Peanut Butter Cups: Apache CloudStack + Riak CS
1. MAKING CLOUDY PEANUT
BUTTER CUPS: APACHE
CLOUDSTACK + RIAK CS
John Burwell
(jburwell@basho.com | @john_burwell)
Thursday, June 27, 13
2. Who Am I
• Consulting Engineer @ Basho Technologies
• Apache CloudStack PMC Member
• Ran operations and designed automated provisioning for hybrid
analytic/virtualization clouds
• Led architectural design and server-side development of a SaaS
physical security platform
Thursday, June 27, 13
15. Got It ... Why Go Private?
• Leverage existing infrastructure investment
• Customization and Control
• Regulatory/Security Requirements
• Cost
Thursday, June 27, 13
16. Open Source Private Cloud
+
Compute Services Object Storage Services
Thursday, June 27, 13
18. What Is Apache CloudStack?
§Open source software that deploys
and manages large networks of
virtual machines, as a highly
available, highly scalable
Infrastructure as a Service (IaaS)
cloud computing platform.
§CloudStack is a Top-Level Project at
the Apache Software Foundation.
Thursday, June 27, 13
20. Infrastructure Model
• Management Server Farm
• Management and provisioning tasks
• Zone
• Collection of pods, network offerings and secondary storage
• Pod
• Collection of clusters in the same failure boundary
• Cluster
• A grouping of hosts and their associated storage
• Hosts
• Servers onto which services will be provisioned
• Primary Storage
• VM disk storage
• Network
• Logical network associated with service offerings
• Secondary Storage
• Template, snapshot and ISO storage
Zone
CloudStack Pod
Cluster
Host
Host
Network
Primary
Storage
VM
VM
CloudStack Pod
Cluster
Secondary
Storage
Thursday, June 27, 13
21. Management Server Farm
• MS is stateless. MS can be
deployed as physical server
or VM
• Single MS node can manage
up to 10K hosts. Multiple
nodes can be deployed for
scale or redundancy
• RHEL/CentOS (5.4+), Ubuntu
(10.0.4+), Fedora (16+)
User
24. Got Scale?
• Running in production at 30,000 physical hosts, supported by only 4
management server instances
• Even greater scale by using CloudStack in a Regional model
• Remember: The separation of control, management and data planes is
critical for cloud platform scale.
Thursday, June 27, 13
25. Deployment Architecture
• Hypervisor is the basic unit of scale.
• Cluster consists of one ore more hosts of
same hypervisor
• All hosts in cluster have access to shared
(primary) storage
• Pod is one or more clusters, usually with
L2 switches.
• Availability Zone has one or more pods,
has access to secondary storage.
• One or more zones represent a cloud
Pod
1
….
Cluster
N
L2
Host 2
Cluster
1
Host 1
Primary
Storage
Zone
1
….
L3
Secondary
Storage
Pod
N
Management
Server Farm
Internet
Thursday, June 27, 13
28. Networking
• VLAN and Security Group isolation
• Load Balancer Support
• Juniper SRX, F5, Nicira NVP and Open vSwitch integration
Thursday, June 27, 13
29. 4.1: Hot Off The Presses
• AWS style regions
• Event pub-sub framework
(RabbitMQ implementation)
• Advanced search within the UI
• API Server request throttling
• API Discoverer Service
• Self-Service Password Reset
• Users directly changing their
API keys
• EC2 query API
• Cloudmonkey CLI
• Autoscale (F5-specific)
Much, much more ....
Thursday, June 27, 13
30. 4.2: On-Deck
• Midonet and BigSwitch SDN integrations
• Enhanced Baremetal provisioning
• VM I/O Throttling
• Hyper-V 2012 Support
• LXC Support
• Cisco VSG integration
• Cisco ASA 1000V
• VM Affinity Rules
• VM Affinity Rules
• Zone-wide primary storage
• Security group isolation in Advanced Network
zones
• Dedicating resources to domains and accounts
• IP Address reservation w/o a vNIC allocation
• vSphere DVS support
• UI Plugin framework
Thursday, June 27, 13
31. First A Little Riak
• Dynamo-inspired, masterless key-value store
• Apache License
• Always available for writes
• Highly available reads
• Secondary Indexes/Search
• MapReduce
• Operations Friendly
Thursday, June 27, 13
32. ... Then A Little Riak CS
• Built on top Riak
• Apache Licensed
• S3-compatible API
• Large object support and multipart upload
• Multi-tenancy and per-user reporting
• Operations Friendly
Thursday, June 27, 13
53. Apache CloudStack + Riak CS Roadmap
• Now
• S3-backed Secondary Storage
• 4.2
• Partial support for using object stores as secondary storage
• 4.3
• Integrated authentication
• Complete support for using object stores as secondary storage
Thursday, June 27, 13
54. Join Our Community
• Learn more @ http://cloudstack.apache.org
• Grab it @ http://cloudstack.apache.org/downloads.html
• Follow us @cloudstack
• Help from the cloudstack-users@cloudstack.apache.org mailing
list or on Freenode @ #cloudstack
• Contribute on cloudstack-dev@cloudstack.apache.org mailing
list or on Freenode @ #cloudstack-dev
Thursday, June 27, 13
55. Up And Running With Riak CS
• Learn more @ http://basho.com/riak-cloud-storage
• Grab a distribution @ http://docs.basho.com/riakcs/latest/
riakcs-downloads/
• Follow us @basho
• Help from the riak-users@lists.basho.com mailing list or on
FreeNode @ #riak
• Contribute @ http://github.com/basho/riak_cs
Thursday, June 27, 13