Rackspace uses OpenStack to power both its public cloud and many private clouds. Lets take a look at how OpenStack Compute (Nova) works with other OpenStack services to convert a users REST API call into accessible compute resources, be they virtual machines, containers or bare metal. Now you understand how Nova is a highly distributed system, lets have a look at how you can upgrade the control place, spread across thousands of nodes, with minimal downtime.

1. OpenStack Nova Upgrade
John Garbutt, Principal Engineer, Nova PTL
@johnthetubaguy
January 2016 at /dev/winter

2. Image FPO
2
Why is johnthetubaguy talking?
2

3. Image FPO
3
Do you really play the Tuba?
3

4. 4
The Problem

5. 5
to provide massively scalable ,
on demand, self ser vice
access to compute resources
OpenStack Nova’s Mission

6. 6
Old Process
Turn Off
Old
DB
Upgrade
Turn On
New

7. 7
I want
it
now!

8. Image FPO
8
Fix that vulnerability now!

9. 9
The Vision

10. 10
<nothing to say>
Customer Quote

11. 11
Upgrade Needs
• No User Impact
• Scope:
– To the next release
– Continuous Deployment
• Existing Configuration works
• Warn before removing features

12. 12
The Solution

13. 13
Nova Architecture
REST API
Compute Compute Compute Compute Compute
Database
Message
Queue
Conductor(s)
Other Control
Nodes
1
2
4
5
3

14. 14
API Users
The Absent The Active Multi-Cloud Ops & Dev
• Cloud upgrades
• But old script
works
• Uses newest APIs
• Check availability
• Multiple clouds
• Different versions
• Single script
• Who is using
what?
• How to evolve
API?

15. 15
API Evolution
v2.0 v2.1
• First API
• Base + Extensions
• Now Deprecated
• No Extensions
• Evolve using “Micro-versions”
• Better Validation

16. 16
Old Process
Turn Off
Old
DB
Upgrade
Turn On
New

17. 17
Nova Architecture
API Nodes
Behind LB
Compute Compute Compute Compute Compute
Database
Message
Queue
Conductor(s)
Other Control
Nodes
Isolate from DB using
oslo.versionedobjects Versioned RPC Signature
Schema and Data Migrations
Graceful Shutdown
RPC Signature

18. 18
New Upgrade Process
Expand
DB Schema
Restart API
and
Control Plane
Restart
n-compute
SIGHUP
all services

19. 19
New Upgrade Process
API Nodes
Behind LB
Compute Compute Compute Compute Compute
Database
Message
Queue
Conductor(s)
Other Control
Nodes
1
2a
3
4
2b

20. 20
Dev Process

21. 21
The Future of Cloud?

22. 5 M I L L I N G T O N R O A D | H AY E S , U N I T E D K I N G D O M U B 3 4 A Z
U S S A L E S : + 4 4 ( 0 ) 2 0 8 71 2 6 5 07 | U K S U P P O R T: 0 8 0 0 9 8 8 0 3 0 0 | W W W. R AC K S PAC E . C O M
© RACKSPACE LTD. | RACKSPACE® AND FANATICAL SUPPORT® ARE SERVICE MARKS OF RACKSPACE US, INC. REGISTERED IN THE UNITED S TATES AND OTHER COUNTRIES. |
WWW.RACKSPACE.COM
Thank you
@johnthetubaguy