Openstack is an open-source cloud computing platform that is widely used. It allows for the provisioning of computing, storage, and networking resources on demand in a manner similar to public cloud services like Amazon Web Services. The presentation discusses Openstack's architecture, current uses, development status, and relationship to high-performance computing. It also covers how Argonne National Labs uses Openstack and potential future directions, like more native support for HPC workloads and integrated application platforms.

1. At the Crossroads of HPC
and
Cloud Computing with Openstack
Presenter: Ryan M. Aydelott of Argonne National Labs
Questions that will be addressed:
• Why is Openstack important?
• What is Openstack currently used for?
• Where is Openstack at in it’s development cycle?
• What does the Openstack architecture/model look like?
• What should you know about Openstack?
• What does Openstack have to do with HPC?
• What will Openstack look like tomorrow?

2. Ryan Aydelott - CELS
Outline
‣ My background
‣ Cloud - what is happening?
‣ The future of programmable stacks
‣ The human factor
‣ Openstack architecture overview
‣ The state of Openstack today
‣ How does Argonne use Openstack?
‣ The future
‣ Discussion/Q&A
2

3. Ryan Aydelott - CELS
My Background
3
‣ Born in the 70’s, took many things apart - put most of them back together.
‣ Started connecting at 300 Baud (3,24 MB/d), Doing Internet things ~1990
‣ First business was shell account/email access to those along the I-88 R&D
Corridor, which blossomed into a full menu ISP.
‣ Left the ISP Business in 1999 and went to work for Lucent, followed by a string of
various employers/startups doing interesting things.
‣ Currently working on various alternative corporate structures, including an
incubator/co-living house on Chicago’s west side.

4. Ryan Aydelott - CELS
4
What is happening?

5. Ryan Aydelott - CELS
The future of programmable stacks
‣ If the infrastructure is programmable, we won’t even consider the *aaS acronym relevant any longer.
‣ Everything looks like code. Infrastructure descriptions can be checked into your source code repo in
tandem with your application. This is great news for developers, interesting news for traditional
sysadmins.
‣ Docker and Vagrant show us how to have textual, executable, repeatable descriptions of platforms with
people providing recipes as part of an ecosystem. Those recipes will come to include clusters of servers
and network topology.
‣ The tools are nascent, (think Mesos/Kubernetes, etc) but these will eventually become the visible face of
what we call “Cloud” today.
5
- http://en.wikipedia.org/wiki/Cloud_computing#Service_models
- https://www.docker.com/
- https://www.vagrantup.com/
- http://kubernetes.io/
- http://mesos.apache.org/

6. Ryan Aydelott - CELS
The Human Factor
6
Sysadmin c.1980-2010
Basic Programming Skills (one off administrator task automation, etc)
Relatively experienced with hardware (RAID, systems architecture, etc)
Networking/Server responsibilities were usually different individuals
Typically responsible for running applications at the server level
Sysadmin c. 2011-
Experienced/Broad Developer (using automation frameworks such as Chef, Saltstack, etc)
Very little hardware expertise (thinks in terms of availability zones)
Deploys networks via SDN frameworks to match the application profile
Responsible for making sure that applications are running well on the platform

7. Ryan Aydelott - CELS
Why is Openstack Important?
‣ Openstack is the single most developed and widely used open source cloud
platform today.
‣ Openstack is a project that has reached the level of adoption that will enable it to
continue to receive considerable development contributions for the foreseeable
near term future.
‣ Openstack’s architecture is modular in nature which allows ease of integration and
customization specific to user environments.
7

8. Ryan Aydelott - CELS
Openstack Architecture (Conceptual)
8

9. Ryan Aydelott - CELS
Openstack Component Description
‣ Dashboard ("Horizon") provides a front end Dashboard to other Openstack services
‣ Compute ("Nova") stores and retrieves virtual disks ("images") and associated metadata in Image
(“Glance”). Images can stored on shared storage such as Gluster, Ceph, NFS, etc. for live migration or
on local disk.
‣ Network ("Neutron") provides virtual networking for Compute. This can either be a single node or in later
versions, multiple-hosts
‣ Block Storage ("Cinder") provides storage volumes for Compute which can be backed by files on ZFS,
LVM, etc. exported most commonly via iSCSI. (Gluster/Ceph supported as well)
‣ Image ("Glance") can store the actual virtual disk files in the Object Store(“Swift”), local disk, Ceph,
Gluster, etc.
‣ All the services authenticate with Identity (“Keystone")
‣ Shared message bus (RabbitMQ, Qpid, or 0MQ) glues it all together.
9

10. Ryan Aydelott - CELS
Openstack Logical Usage
‣ End users can interact through a common web interface (Horizon) or directly to
each service through their API
‣ All services authenticate through a common source (facilitated through keystone)
‣ Individual services interact with each other through their public APIs (except where
privileged administrator commands are necessary)
10

11. Ryan Aydelott - CELS
What is Openstack Used for?
‣ Provide computing services for resale to end users by service operators
‣ Provide internal enterprise computing support as either an alternative or in addition
to existing computing resources
‣ Provide computing resources to research environments across a variety of
disciplines, including research of cloud on cloud computing
‣ Hobbyist/Tinkerers who are interested in learning about/using the software stack
(keep in mind this includes professional hobbyists tasks with exploring Openstack
by their employer)
11

12. Ryan Aydelott - CELS
Where is Openstack at from the standpoint of
maturity?
‣ The largest opinion at the last openstack conference in Paris (November 2014) is
that Openstack is finally ready to enter its maturity phase. This translates to less
features and a focus on robust/stable code.
‣ Currently still on a 6 month major release cycle (moved from a 3 month release
cycle ~2 years ago), there were some murmurs of extending this out even further.
‣ Ready to use/stable for most organizations without a dedicated developer,
provided you are not deploying the stack using non-standard configurations/
hardware.
‣ Many professional service organizations now will run/manage Openstack for you.
(however many typically have their own release)
12

13. Ryan Aydelott - CELS
http://stackalytics.com
13

14. Ryan Aydelott - CELS
How does Argonne Use Openstack?
‣ Currently running some custom software for data analysis as well as standard
stacks (such as Hadoop). Additionally some web facing application stacks are also
running on the system.
‣ At 800 nodes, it was a large system at that time of its deployment (~2010), now
however large systems begin at ~1000 unique hardware entities with many scaling
over 10,000 nodes in service provider environments.
‣ Our system is focusing on vertical performance in specific areas (networking,
memory/compute, storage) rather than overall system size.
‣ A major challenge today is provisioning hardware that allows us the most flexibility
via software. The difficulty lies in the very different nature of partitioning required
for different types of workloads.
14

15. Ryan Aydelott - CELS
HPC and Openstack
‣ Virtual clusters can elastically scale up/down based on demand (Heat Autoscaling)
‣ Not stuck with one distribution - end user can choose
‣ End users can install their own software using the distro package manager
‣ Bring your own runtime in the form of an image (Docker, etc)
‣ Excellent method to combine multiple smaller (1-2 Rack) HPC clusters into a
single managed entity to solve partitioning/underutilization problems
‣ Software that runs most workloads hasn’t fully taken advantage of this new
architecture (yet)
15

16. Ryan Aydelott - CELS
Challenges of HPC and Openstack
Network
‣ Commonly deployed Openstack network is built for features, not performance
‣ MPI performs poorly on common Openstack networks (tunnels, bridges, nat, iptables)
Storage
‣ Presenting the same block device to multiple VMs just becoming available (needed for
distributed filesystems)
‣ Shared filesystem as a service support relatively new feature
Future
‣ SRIOV support, and isolation features for IB are available
‣ SRIOV / pci passthrough is available in recent Openstack releases
‣ IRONIC is an Openstack Project that allows bare metal performance across diverse
systems
16

17. Ryan Aydelott - CELS
Why you should know Openstack
‣ The tools required to work effectively in this industry are changing dramatically to the point
that IT shops are completely retooling their organizational structure/processes.
‣ All future systems will iterate on a version of this architecture. This is as relevant now as
learning about pc’s in the 80’s.
‣ Openstack could be perceived as providing something akin to low-level language for utility
computing.
‣ In research (both corporate and public) tightly coupled compute clusters are falling out of
favor to distributed systems.
‣ Budgets always prefer commodity curve computing at scale.
‣ If you have to customize or are trying to integrate hot-rod hardware, Openstack isn’t a bad
choice.
17

18. Ryan Aydelott - CELS
Openstack has opened up opportunities for
integrators
‣ New applications being deployed can be designed in ways that allow them to fully
utilize the operating environment (I call these architecturally aware applications)
‣ Many cluster management frameworks available today have an interface similar to
the one Openstack and other providers have.
‣ Hardware/Software integrators have a single platform agnostic integration point.
‣ This first group of problem solvers will pave the way for additional management
frameworks to be built on top of Openstack, further abstracting the underlying
infrastructure to the application.
18
- http://www.slideshare.net/hpcloud/openstack-integration-case-study-of-an-application-deployment-on-a-hybrid-cloud
- http://www.slideshare.net/uri1803/open-stack-bigdata

19. Ryan Aydelott - CELS
Building Openstack Aware Applications
‣ HEAT: https://wiki.openstack.org/wiki/Heat
‣ A Heat template describes the infrastructure for a cloud application in a text file that is readable and writable
by humans, and can be checked into version control, diffed, etc.
‣ Infrastructure resources that can be described include: servers, floating ips, volumes, security groups, users,
etc.
‣ Heat also provides an autoscaling service that integrates with Ceilometer, so you can include a scaling group
as a resource in a template.
‣ Templates can also specify the relationships between resources (e.g. this volume is connected to this server).
This enables Heat to call out to the OpenStack APIs to create all of your infrastructure in the correct order to
completely launch your application.
‣ Heat manages the whole lifecycle of the application - when you need to change your infrastructure, simply
modify the template and use it to update your existing stack. Heat knows how to make the necessary
changes. It will delete all of the resources when you are finished with the application, too.
‣ Heat primarily manages infrastructure, but the templates integrate well with software configuration
management tools such as Puppet and Chef. The Heat team is working on providing even better integration
between infrastructure and software.
19
- https://www.openstack.org/summit/openstack-paris-summit-2014/session-videos/presentation/
deploying-and-auto-scaling-applications-on-openstack-with-heat

20. Ryan Aydelott - CELS
Trove - Opensource Database as a Service
‣ https://wiki.openstack.org/wiki/Trove
‣ The goal is to provide a scalable/reliable cloud database as a service for both
relational/non-relational engines
‣ Natively built to run on Openstack
‣ Support a single tenant database within an instance
‣ Still under active development, production ready only with experts.
20
- http://www.slideshare.net/mirantis/trove-d-baa-s-28013400

21. Ryan Aydelott - CELS
Sahara - Hadoop on Openstack
‣ https://wiki.openstack.org/wiki/Sahara
‣ The goal is to provide a scalable/reliable cloud database as a service for both
relational/non-relational engines
‣ API to run analytics jobs
‣ Cluster Provisioning
‣ Still under active development
21
- http://www.slideshare.net/mirantis/savanna-hadoop-on-openstack

22. Ryan Aydelott - CELS
Zaqar - Messaging Service
‣ https://wiki.openstack.org/wiki/Zaqar
‣ Messaging service native to Openstack.
‣ Tenant Queues based on Keystone Project ID’s
‣ HA wHorizontal Scaling
‣ Still under active development
22

23. Ryan Aydelott - CELS
This is just the beginning…
‣ Already a number of these commonly used application stacks are being written
directly against Openstack.
‣ More complicated deployments that consists of groups of applications can be
coordinated with Heat, either exclusively or in tandem with other management
frameworks/systems.
‣ Further development of abstraction/optimization layers will continue, reducing the
human overhead necessary to run complex jobs.
23

24. Ryan Aydelott - CELS
<EOF>
24