Slides from our presentation at the OpenStack Spring Summit hosted by our VP of Product Neil Levine and Dell's Kamesh Pemmaraju.

1. Wicked Easy Ceph Block
Storage & OpenStack
Deployment with Crowbar
Kamesh Pemmaraju, Dell
Neil Levine, Inktank

2. Outline
• Introduction
– Dell OpenStack-Powered Cloud Solution
– Ceph
– Crowbar
• Why Crowbar + Ceph in OpenStack?
– OpenStack Block Storage gap
– Automation, scale
• What we have done to enable things?
• Customer benefits, drivers – an example
• What’s coming next?

3. Clouds Require an Operational Focus
• Clouds demand significant operational
and process controls
SW
OPS • Operational decisions drive hardware and
software decisions
HW
• We are finding ways to productize
operations into best practices

4. Dell OpenStack-Powered Cloud Solution
“Crowbar”
CloudOps
SW
Software
SW
OPS
OPS
Reference
Architecture HW
Services &
Consulting

5. Dell OpenStack-Powered Cloud Solution
HW + SW + Services
• C6220, C6105,R720, R720XD servers
HW reference • Storage and compute
Hardware architecture
• Force 10 S60 and PowerConnect
Configuration • Min 6 nodes. Max 60 nodes
• OpenStack Installer and continuous integration (Crowbar)
Software • OpenStack cloud SW
• Ceph for distributed storage
Software • Ubuntu 12.04 (host)
Operating system • Windows (guests)
• Linux (guests)
Hypervisor • KVM
• Hardware integration – onsite or merge center
• Software installation – onsite
Deployment and
consulting • OpenStack consulting services
Services − Assessment, design and installation
− Delivered via partnerships with Mirantis and Canonical
• HW/Crowbar: Dell ProSupport
Support
• OpenStack support via Canonical (coming soon)

6. Ceph – Unified Storage Software
OBJECTS VIRTUAL DISKS FILES & DIRECTORIES
CEPH CEPH CEPH
OBJECT STORAGE BLOCK STORAGE FILE SYSTEM
A powerful S3- and Swift- A distributed virtual block A distributed, scale-out
compatible gateway that device that delivers high- filesystem with POSIX
brings the power of the performance, cost-effective semantics that provides
Ceph Object Store to storage for virtual machines storage for a legacy and
modern applications and legacy applications modern applications

7. Ceph – Key Differentiators
• Unified storage platform (Object + Block + File)
• Ceph Intelligent Placement (CRUSH)
• Ceph Intelligent Devices (Self-healing, P2P)
• Ceph Intelligent Objects (Embedded Software Classes)
• Ceph Integration (Linux Kernel, OpenStack, Cloudstack)

8. Ceph Architecture
APP APP HOST/VM CLIENT
Ceph Object Ceph Block Ceph File
LIBRADOS
Gateway Device System
A library allowing
apps to directly
(RGW) (RBD) (CephFS)
A bucket-based REST A reliable and fully- A POSIX-compliant
access RADOS,
gateway, compatible distributed block distributed file system,
with support for
C, C++, Java, with S3 and Swift device, with a Linux with a Linux kernel
kernel client and a client and support for
Python, Ruby,
QEMU/KVM driver FUSE
and PHP
Ceph Storage Cluster (RADOS)
A reliable, autonomous, distributed object store comprised of self-healing, self-
managing, intelligent storage nodes

9. Ceph RADOS Components
M
Ceph Monitor Nodes: Ceph OSD Nodes:
– Maintain cluster map • One Ceph OSD daemon per disk
– Provide consensus for distributed (recommended)
decision-making • At least three nodes in a cluster
– Must have an odd number • Serve stored objects to clients
– These do not serve stored objects • Intelligently peer to perform
to clients
replication tasks
• Supports object classes

10. OSD OSD OSD OSD OSD
Ceph OSD btrfs
Node xfs
FS FS FS FS FS ext4
DISK DISK DISK DISK DISK
Ceph
Storage M M M
Cluster
(RADOS)

11. What is Crowbar?
Mission: “A Zero Touch Cloud Installer”
Servers in boxes to full function cloud in under 2 hours
• Fast & Flexible
– Bare metal install including BIOS & RAID config
– Users can choose how their system is configured (“barclamps”)
• DevOps Embracing
– Ongoing Operations Model (DevOps for Clouds)
– Leverages & Wraps Opscode Chef
• Open
– Not specific to OpenStack – Dell using for other Apps
– Not restricted to Dell hardware
– Apache 2 licensed

12. Images vs. Layers: Overview
Images: Single Unit Layers: Stacked Pieces
Configuration Integrations
Configuration
Application Foo
Integrations +
Application Bar
Applications +
Utilities + Operating Utilities
System
Operating System

13. Images vs. Layers: Lifecycle
Images: Replacement Layers: Upgrade
Config Config Config
I I
Foo Foo
Config
Config
I+A+U+O/ I+A+U+O/ I+A+U+O/
Bar v1 Bar v2
S S S
U U
OS OS
Config Bar v2
I+A+U+O/
S

14. Crowbar Uses the Layer Model for Deployment
APIs, User Access, • Includes all the components required to implement an entire
cloud infrastructure including ecosystems partners
& Ecosystem
Partners
Ops Management
Dell “Crowbar”
• Pluggable components deploy cloud infrastructure
Cloud Infrastructure
& Dell IP Extensions • Allow for addition of Dell IP and expansion by the community
services and customers
Core Components & • Can integrate with Dell existing products
Operating Systems • Delivers basic data center services and required cloud
infrastructure.
• Provision bare-metal servers from box to cloud WITHOUT
Physical Resources user intervention (other than racking/cabling and some
minimal configuration questions) NOT EXCLUSIVE TO DELL
HARDWARE
* Crowbar has potential to service other programs beyond OpenStack

15. How Does Crowbar Work?
Status (po
st)
Ops Management
Orchestration
Dell “Crowbar”
State Machine Apps Apps
Crowbar Magic
Run List Network Network
Happens!
Chef-Client Config Config
Chef
Server Chef-C
lient Operating System
Config.
Hardware Config
(uses “Sledgehammer”)
DHCP
Server
t
PXE Boo

16. What is a Barclamp?
Barclamp
Crowbar API & O/S App Components &
Chef Recipes
Partial UI Packages Scripts
Ops Management
Dell “Crowbar”
• Crowbar API & Partial UI
• Adds states and transitions to Orchestration
• Customizes UI specific to barclamp
• Visual extensions to nodes/networks/utils
• Chef Recipes
• DevOps description of application
• Enforced repeatable configuration
• O/A App Packages
• Operating system dependencies (Debs & RPMs)
• Components & Scripts
• Ad hoc configuration not covered by Chef

17. Modular Design: Barclamps The Configuration You
Want to Deploy
Nagios Ganglia Dashboard
APIs, User Access, &
Ecosystem Partners enStratus RightScale Horizon
Ops Management
Dell “Crowbar”
Nova Swift Glance
Cloud Infrastructure
& Dell IP Extensions Ceph Force10 Keystone
Crowbar DNS Logging
Core Components &
Operating Systems Deployer NTP
Provisioner BIOS IPMI
Physical Resources Network RAID

18. The 4 Stages of Cloud Storage Thinking
“We should do “No SPOFs “How do the Web
things the new “App can’t be 2.0 folks do it?”
way!” re-architected!” please!”
Use distributed
Use Object Use local storage! Use this storage on utility
Storage expensive SAN! hardware!

19. Ceph Block Storage for Dell Openstack
1. Foundations: no SPOFs, self-healing, CRUSH
2. Features: Thin-provision, Cloning, Copy-on-Write
3. Openstack: integrated with Cinder, Nova, Glance
4. Deployment: via Crowbar

20. RBD Features

21. RBD Features

22. RBD Features

23. RBD Features

25. Scale-Out Operations With OpenStack
If you can’t automate it, you can’t scale it
I I I I I I I
API & Crowbar Dash Nova Nova Nova Cinder
Config
Config
Config
Config
Config
Config
Config
Nagios
Chef Board Cntrl DB API API
Services U U U U U U U
OS OS OS OS OS OS OS
I I I I I I
Compute Nova Nova Nova Nova Nova Nova
Config
Config
Config
Config
Config
Config
Nodes Node
U
Node
U
Node
U
Node
U
Node
U
Node
U
OS OS OS OS OS OS
I I I I I I
Storage Ceph Mon Ceph Mon Ceph Mon Ceph OSD Ceph OSD Ceph OSD
Config
Config
Config
Config
Config
Config
Nodes Node Node Node Node Node Node
U U U U U U
OS OS OS OS OS OS

26. Ceph Related Barclamps for Crowbar
• https://github.com/ceph/barclamp-ceph
– ceph.com packages into the Crowbar Admin Node
– Ceph OSD roles (ceph-store)
– Ceph Mon roles (ceph-mon)
• https://github.com/crowbar/barclamp-nova
– Hypervisor with RBD (ceph-client)

27. What Have We Done to Enable?
• Dell and Inktank have partnered to bring a complete solution for
OpenStack + Ceph + Automated deployment with Crowbar
• The joint solution provides:
– Crowbar barclamp to
– Deploy Ceph clusters automatically in a very short time
– Connect those clusters to OpenStack
– Professional Services, Support, and Training
› Collaborative Support for Dell hardware customers
– Joint Solution
› Validated against Dell Reference Architectures via the
Technology Partner program

28. Customer:
University to Deploy OpenStack + Ceph Using Crowbar
Situation
• University employs close to 900 researchers and receives • In the end, University
hundreds of million dollars in Grants as a Top 10 Research chose Dell/Inktank to
Institution primarily working on Cancer and Genomic Projects.
The University has a need to provide a centralized data architect a platform
repository for Researchers in order to insure compliances that would be very
concerning retention of data.
• The intent of the Data Repository is to provide 2TB of free
scalable and provide
storage space to each Researcher, with opportunity to lost costs per TB and
purchase from the university more capacity at a very was the best of all
reasonable cost.
Decision Drivers
worlds that provide
• University investigated using an traditional SAN storage solution
compute and storage
which was very expenses on a per TB basis. They also on the same
investigated using public cloud storage options which also hardware.
proved to be too expensive. Finally, they looked at Hadoop for
the project but found that it was not a good fit for this use case.

29. What’s Next?
• May
– Ceph Cuttlefish
• Summer 2013
– Dell Grizzly support
– Ceph Dumpling (August)
• https://github.com/crowbar/crowbar/wiki/Crowbar-2.0
– Crowbar v2.0 work is happening in the open. Check it out and
participate!

30. Don’t Miss the OpenStack Summit Sessions
Havana Sessions:
– Planning the Ceph Roadmap for Openstack
Wednesday April 17, 2013
1:50pm - 2:30pm in Room - B113
– Features for Ceph with Cinder and Beyond
Thursday, April 18, 2013
9:50 – 10:30am in Rooms – C120+121+122

31. Contact Information
Reach Kamesh and Neil for additional information:
Dell.com/OpenStack
Dell.com/Crowbar
Inktank.com/Dell
Kamesh_Pemmaraju@Dell.com
@kpemmaraju
Neil.Levine@Inktank.com
@neilwlevine
Visit the Dell and Inktank booths in the OpenStack Summit Expo Hall

32. Thank you!