Architecture constraints and benefits for Cloud applications

1. Cloud
Patterns
Nicolas De Loof - cloudbees
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13
1

2. {{
This talk is a mashup
• « PaaS design »
Michael Neale, CloudBees
• « Cloud Best practices »
Eric Bottard, VMWare
• my own …
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13
}}

3. ©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13

4. ©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13

5. ©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13

6. ©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13

7. ©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13

8. <me>
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13

9. {{
}}
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13
5

10. {{
Support engineer
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13
}}

11. {{
}}
Support engineer
suré
cen
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13

12. {{
}}
Support engineer
suré
cen
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13

13. {{
Maven & Jenkins committer
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13
}}

14. {{
}}
JUG Leader
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13

15. </me>
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13

16. {{
End Users
Application
Developers
Infrastructure
Architects
SaaS
PaaS
IaaS
SaaS
PaaS
IaaS
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13
}}
Thousands
Applications
A Dozen
platforms
Few world-scale
providers

17. {{
}}
May be difficult
for legacy apps
Cloud
easy
On premises
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13

18. {{
Cloud
easy
refactor
On premises
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13
}}
May be difficult
for legacy apps

19. {{
Green field applications
• Can select modern solutions
most (not all) frameworks are well designed for Cloud
}}
• Started on cloud, discovers and adapts to constraints
à
devoxx.be 2013
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13

20. {{
« Classic » applications
• Started on premises, single server,
hit cloud constraints
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13
}}
• Not such modern solutions
common frameworks may not fit cloud constraints

21. {{
The Devoxx 2012 apps
• Call for Paper and Registration
• High traffic two months a year
• Self hosted (parleys.com infra)
moved to CloudBees PaaS
• Wicket, Spring, MySQL
No hype, like 99% java applications
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13
}}

22. Scale
Sc
Out
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13

23. {{
Scale up
M1 small
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13
}}

24. {{
Scale up
M1 small
M1 large
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13
}}

25. {{
Scale up
M1 small
M1 large
M3 double extra large
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13
}}

26. {{
Scale up
}}
Then ?
M1 small
M1 large
M3 double extra large
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13

27. {{
Scale up
• Still possible, but will hit a limit
• Not optimal
}}
Then ?
M1 small
M1 large
M3 double extra large
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13

28. {{
M1 extra large
•
•
•
•
64 bits
15 Gb memory
4 Vcore, 2 cpu unit (~2.5Gh)
1.6 Tb HD
aka « my personnal laptop »
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13
}}

29. {{
Scale out
Design for
clustering
Adapt resources to actual load
Pay only for actual use
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13
}}

30. • slice a server into cells
• Multi-tenant app hosting
vitrualisation
{{
What a PaaS actually does …
M1 large
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13
}}

31. • slice a server into cells
• Multi-tenant app hosting
vitrualisation
{{
What a PaaS actually does …
M1 large
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13
}}

32. {{
Multi-tenancy
• Hardware level (IaaS)
• OS level (hypervisor)
• OS virtualization (cgroups, LXC)
• Middleware ?
Java 9 to be multi-tenant ?
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13
}}

33. be
State
Less
… if you can
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13

34. {{
RESTFul, Stateless
• Linear horizontal scaling
But …
• Application - User « conversation » has a state
• Stateless apps mostly use caches
then cache needs to be distributed
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13
}}

35. {{
Stateless, really ?
• Beware your frameworks !
Grails
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13
}}

36. {{
Stateless, really ?
• Beware your frameworks !
Grails
security plugin
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13
}}

37. {{
Stateless, really ?
• Beware your frameworks !
Grails
security plugin
Spring-security
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13
}}

38. {{
Stateless, really ?
• Beware your frameworks !
State
Grails
security plugin
Spring-security
HttpSession
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13
}}

39. {{
Does Stateless really exist ?
• Client side state with browser cookies
à higher network traffic, security
• Use http session (servlet frameworks)
à memcache session replication
à sticky session
• Use a central service (DB)
à SPoF, DB scalability
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13
}}

40. beware
Lock-in
Prefer portable API
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13

41. {{
Standards
• Use standard, portable APIs
(aka Java EE, the good parts)
}}
• Set runtime configuration via
env variables / system properties
• at least use some abstraction to insulate vendorspecific code
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13

42. {{
Some Standards
•
•
•
•
•
•
•
Java EE
Java Servlet
JVM
LAMP
Node.js
RVM
..
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13
}}

43. Beware
File System
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13

44. {{
Cloud uses Schrödinger FS
Looks like it’s alive, but it’s not
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13
}}

45. {{
PaaS != Hosted servers
Your host at this time
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13
}}

46. {{
PaaS != Hosted servers
Your host at this time
Your host after (re)deployment
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13
}}

47. {{
FileSystem is ephemeral
(and not distributed)
File system is not persistent, neither distributed
}}
Use storage engine à la Amazon S3 (or DB BLOB)
... and see local FileSystem as a convenient cache
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13

48. Singletons
… aren’t
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13

49. {{
Sample: Quartz Job Scheduler
Job will be triggered on all nodes !
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13
}}

50. {{
Sample: Quartz Job Scheduler
Job will be triggered on all nodes !
à
Use Persistent (JDBC) Job Store
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13
}}

51. {{
Sample: Quartz Job Scheduler
Job will be triggered on all nodes !
à
Use Persistent (JDBC) Job Store
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13
}}

52. {{
Sample: Quartz Job Scheduler
Job will be triggered on all nodes !
à
Use Persistent (JDBC) Job Store
à
Use dedicated cron service
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13
}}

53. Latency
matters
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13

54. {{
•
app-x
tomcat
}}
Inter Continental
is common
• EU to us-east-1 latency:
120ms
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13

55. {{
Measure
• Chrome DEV Tools
• Google Pagespeed
• YSlow
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13
}}

56. {{
Improve
• Use HTTP cache headers
• Use unique path per deployment
hash, or just ?version=
• Use a CDN
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13
}}

57. Migrate to Cloud
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13

58. Yes!
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13

59. SOA
(the right way)
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13

60. {{
}}
Small is beautifull
• small, specialized, elastic services
• Communicate with
REST on HTTP (sync)
MQ (async)
frontend
Users
Backend
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13
indexer

61. Consume
Services
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13

62. {{
*aaS ecosystem
• *aaS is about service, not software
• Integrate services, don’t try to setup your own
infrastructure
• AWS, the place to be for *aaS
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13
}}

63. {{
*aaS ecosystem
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13
}}
44

64. {{
Private Cloud ?
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13
}}
45

65. {{
Private cloud is non-sense
• Do you produce your own electricity ?
• Security is about humans,
not firewalls
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13
}}

66. Design
for
Failure
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13

67. {{
It
May
Will fail
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13
}}

68. {{
Beware resource
• Don’t hang the app when resource fails
• Be asynchronous and
fault tolerant
• Use Messaging
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13
}}

69. Cloud
Safer
is
anyway
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13

70. {{
Some metrics
• In 2012,
CloudBees suffered 2 major outages
20 then 10 minutes
à 99,99% (What’s your actual availability rate ?)
Cloud outages are
visible
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13
}}

71. {{
Disaster recovery
• All deployed artifact à S3
• DB on EBS, then daily à S3
+ your own backup strategy
http://wiki.cloudbees.com/bin/view/Documentation/BackupPolicies
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13
}}

72. {{
Need more ?
• Multi-zone High-Availability
• Mutli-region redundency
}}
à data sync to handle network latency
à Short TTL DNS
turn key » solution
No «
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13

73. Ops
for
Cloud
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13

74. {{
Infra is managed ... not app
• Need to instrument and monitor
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13
}}

75. {{
Ops for Cloud apps
One team,
One goal,
One platform
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13
}}

76. {{
Cloud is the best place to embrace
DevOps
Traditional
Cloud
Environment
DEV / INT / PROD
identical
Delivery
Mostly manual
full automation
API based
DEV Process
Fire and forget
Continuous delivery
Team
Dev vs Ops vs QA
DevOps
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13
}}

77. Continuous
delivery
Integration
deployment
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13

78. {{
Continuous …
Integration Deployment Delivery
• Git push
• Build
• Test
• Git push
• Build
• Test
• Deploy
• Production
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13
}}
• Git push
• Build
• Test
• Ready for
production
• Production

79. concurrent
deployment
0 downtime
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13

80. {{
0 downtime
http://demo.nicolas.cloudbees.net
rout
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13
}}

81. {{
0 downtime
http://demo.nicolas.cloudbees.net
rout
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13
}}

82. {{
0 downtime
http://demo.nicolas.cloudbees.net
rout
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13
}}

83. {{
0 downtime
http://demo.nicolas.cloudbees.net
rout
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13
}}

84. {{
Resources migration
App Vn running
DB schema Vn
App Vn+1 starting
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13
}}

85. {{
Resources migration
App Vn running
DB schema Vn
App Vn+1 starting
DB schema Vn+1
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13
}}

86. {{
}}
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13

87. {{
• Vn+1 schema to be Vn compatible
• Vn+2 can do some cleanup
i.e. « @deprecated » for DB
Or …
temporary deploy a « maintenance » page
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13
}}

88. Deployment
event
Is not an
anymore
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13

89. {{
Green / Blue
http://martinfowler.com/bliki/BlueGreenDeployment.html
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13
}}

90. {{
A/B testing
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13
}}

91. {{
Canary testing
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13
}}

92. {{
Pretotyping
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13
}}

93. thank
You
©2013 CloudBees, Inc. All Rights Reserved
jeudi 24 octobre 13