Overview of the Cloud stack: infrastructure, platforms, software, development

1. Patrick Chanezon
Director
Enterprise Evangelism
Microsoft
patric@microsoft.com
@chanezon NATO IST Symposium, May 2013
Cloud is such stuff
as dreams are made on
Sunday, May 12, 13

2. French
Polyglot
Server Side
San Francisco
Developer Relations
@chanezon
Sunday, May 12, 13

3. P@ & Military Software
Sunday, May 12, 13

4. Dreams Of my childhood
4
Sunday, May 12, 13

5. Accelerando / Singularity, in a Galaxy far far away
§ Even if we automate ourselves out of a job every 10 years
§ ...I don’t think the singularity is near!
5
Sunday, May 12, 13

6. Moore's Law is for Hardware Only
§ Does not apply to software
§ Productivity gains not keeping up with hardware and bandwidth
§ Writing software is hard, painful, and still very much a craft
6
Sunday, May 12, 13

7. Moore's Law’s free lunch is over
§ Herb Sutter, Welcome to the Jungle
http://herbsutter.com/welcome-to-the-jungle/
7
Sunday, May 12, 13

8. Predictions
“The future is already here
— it's just not very evenly
distributed”
William Gibson
8
Sunday, May 12, 13

9. Architecture Changes: 60’s Mainframe
Sunday, May 12, 13

10. Architecture Changes: 80’s Client-Server
Sunday, May 12, 13

11. Architecture Changes: 90’s Web
Sunday, May 12, 13

12. Architecture Changes: 2010’s Cloud, HTML5, Mobile
Sunday, May 12, 13

13. Back to Client Server: Groovy Baby!
13
Sunday, May 12, 13

14. What is Cloud Computing?
14
Cloud According to my daughter Eliette
Sunday, May 12, 13

15. Cloud Stack - Classic Pyramid
15
Platform As A Service
Infrastructure As A Service
Software
As A Service
Sunday, May 12, 13

16. Cloud Stack - By Number
16
Platform As A Service
Infrastructure
As A Service
Software
As A Service
Sunday, May 12, 13

17. Cloud Stack - By Value
17
Infrastructure
As A Service
Software
As A Service
Platform As A Service
Sunday, May 12, 13

18. Cloud Stack - Neutral
18
Platform As A Service
Infrastructure As A Service
Software
As A Service
Sunday, May 12, 13

19. Crossing the Chasm
§ Build the whole product
§ Cloud getting mainstream: Apple iCloud
§ Opportunities and risks
§ Ecosystems, various platforms
19
Picture from Wikimedia Foundation http://en.wikipedia.org/wiki/File:Technology-Adoption-Lifecycle.png
Sunday, May 12, 13

20. Cloud Stack - History
§ What does cloud mean, 4 main angles
• Software 1994 Netscape
• Infrastructure 2002 Amazon AWS
• Platform 2008 Google
• Development now!
20
§ Industrialization of hardware and
software infrastructure
like electricity beginning of 20th century
§ But software development itself is moving towards craftmanship
Sunday, May 12, 13

21. Cloud started at Consumer websites solving their needs
• Google, Amazon, Yahoo, Facebook, Twitter
• Large Data Sets
• Storage Capacity growing faster than Moore’s Law
• Fast Networks
• Horizontal -> Vertical scalability
• Open Source Software
• Virtualization
• Cloud is a productization of these infrastructures
• Public Clouds Services: Google, Amazon
• Open Source Software: Hadoop, Eucalyptus, Ubuntu, Cloud Foundry
Sunday, May 12, 13

22. Google research papers: good predictor of where BigData is going
• Horizontal scalability 2004: Map/Reduce (Hadoop)
• NoSQL 2006: Bigtable (Mongo, Cassandra, HBase, Riak)
• Real Time analytics 2010: Dremel, BigQuery (Impala)
• Horizontally Scalable SQL 2012: Spanner/ Amazon Redshift
Sunday, May 12, 13

23. 23
Infrastructure
Sunday, May 12, 13

24. IaaS/Virtualization getting mainstream
§ AWS, Azure, Google, Joyent, Rackspace,...
§ Open Source projects: OpenStack,CloudStack, Eucalyptus
§ Automation: Chef, Juju
§ Standardization?
§ Inside the Firewall, Virtualization: VMware, Microsoft HyperV, Xen, KVM
§ 50% of workloads are virtualized
§ Easy to provision, manage instance...BUT
§ Still need to manage backups, software stacks, monitor, upgrades
§ Devops: cultural movement, dev and ops working hand in hand, sharing
tools and processes
24
Sunday, May 12, 13

25. With Infrastructure, you still need to build your own platform
§ Need to build a distributed platform on top of you infrastructure
§ Story of the AWS meltdown from last summer
• http://blog.reddit.com/2011/03/why-reddit-was-down-for-6-of-last-24.html
• http://www.readwriteweb.com/cloud/2010/12/chaos-monkey-how-netflix-
uses.php
• http://news.ycombinator.com/item?id=2477296
• http://stu.mp/2011/04/the-cloud-is-not-a-silver-bullet.html
§ Twilio, Smugmug, SimpleGeo survived it because they built their
own distributed platform on top of IaaS
§ Enterprise customers want to consider Infrastructure like CDNs
• Multi Cloud usage
• Based on Open Source de facto standards, or full standards whenever that
happens
25
Sunday, May 12, 13

26. 26
Platforms
Sunday, May 12, 13

27. Platforms
§ Raise the Unit of currency to be application & services instead of
infrastructure
§ Google App Engine, Microsoft Windows Azure, Cloud Foundry,
SalesForce Heroku, Stax (Cloudbees), Amazon elastic beanstalk,
Red Hat OpenShift, AppFog
§ Single or a few languages, services
§ Start of Multi language Polyglot platforms
§ Enabler for Agile Developers -> Create Business value faster
§ Lack of standards: risk, vendor lock-in
§ Enterprise needs:
•Control, customizability
•Private/Hybrid Cloud: Azure/Windows Server, Cloud Foundry Bosh, Red
Hat Openshift
•Avoid lock-in
27
Sunday, May 12, 13

28. Agile Development Processes
Sunday, May 12, 13

29. Agility as a survival skill
§ Consumer software is becoming like fashion
•Phone apps, social apps, short lifetime, fast lifecycles
•Ab testing
§ Clay shirky situational apps
§ Kent Beck, Usenix 2011 Talk, “Software G-Forces”
change in software process when frequency grows
§ Cloud Platforms enables an Agile culture, driver for innovation
•Scalability is built in the platforms
•Can iterate faster
•Focus on design
§ Cloud Platforms lets developers focus on driving business value
29
Sunday, May 12, 13

30. Cloud Foundry: The Open PaaS
30
CloudProviderInterface
ApplicationService
Private
Clouds
Public
Cloud
Micro
Cloud
Data
Services
Other
Services
Msg Services
vFabric
Postgres
vFabric
RabbitMQTM
• Open Source: Apache 2 Licensed
• multi language/frameworks
• multi services
• multi cloud
Sunday, May 12, 13

31. Windows Azure
flexible open solid
Sunday, May 12, 13

32. Sunday, May 12, 13

33. Global
Footprin
Sunday, May 12, 13

34. Sunday, May 12, 13

35. ß Windows Server and
Linux
Sunday, May 12, 13

36. ß Windows Server and
Linux
ß Flexible Workload
Support
Sunday, May 12, 13

37. Virtual
Machines
ß Windows Server and
Linux
ß Flexible Workload
Support
ß Virtual Private Networking
Sunday, May 12, 13

38. virtual machine
portability
Sunday, May 12, 13

39. Sunday, May 12, 13

40. Windows Azure
Your Data Center
Sunday, May 12, 13

41. Windows Azure
Your Data Center
Sunday, May 12, 13

42. Windows Azure
Your Data Center
Sunday, May 12, 13

43. Your Data Center
Windows Azure
Sunday, May 12, 13

44. Your Data Center
Windows Azure Other Service
Providers
Sunday, May 12, 13

45. Your Data Center
Windows Azure Other Service
Providers
Sunday, May 12, 13

46. Your Data Center
Windows Azure
Sunday, May 12, 13

47. Your Data Center
Windows Azure Other Service
Providers
Sunday, May 12, 13

48. Your Data Center
Windows Azure Other Service
Providers
no lock-in
Sunday, May 12, 13

49. Sunday, May 12, 13

50. ß Build with ASP.NET,
Node.js or PHP
Sunday, May 12, 13

51. ß Build with ASP.NET,
Node.js or PHP
ß Deploy in seconds with
FTP, Git or TFS
Sunday, May 12, 13

52. Web Sites
ß Build with ASP.NET,
Node.js or PHP
ß Deploy in seconds with
FTP, Git or TFS
ß Start for free, scale up as
your traffic grows
Sunday, May 12, 13

53. Sunday, May 12, 13

54. ß Easily build cloud back-ends
Sunday, May 12, 13

55. ß Easily build cloud back-ends
ß Data, identity, push
notifications, background jobs
Sunday, May 12, 13

56. Mobile Services
ß Easily build cloud back-ends
ß Data, identity, push
notifications, background jobs
ß Windows 8, Windows Phone,
iOS, Android, HTML5
Sunday, May 12, 13

57. Sunday, May 12, 13

58. Data ID Services
Sunday, May 12, 13

59. Data ID Services
Sunday, May 12, 13

60. Data ID Services
Custom Script(s)<
//
>
Sunday, May 12, 13

61. Data ID Services
Custom Script(s)<
//
>
Sunday, May 12, 13

62. Sunday, May 12, 13

63. ß Build infinitely scalable
apps and services
Sunday, May 12, 13

64. ß Build infinitely scalable
apps and services
ß Support rich multi-tier
architectures
Sunday, May 12, 13

65. Cloud Services
ß Build infinitely scalable
apps and services
ß Support rich multi-tier
architectures
ß Automated application
management
Sunday, May 12, 13

66. Multiple
Services
storagebig data
caching
CDN
database
identity
media
messagi
ng
networki
ng
traffic
cloud
services
Sunday, May 12, 13

67. Multiple
Services
storagebig data
caching
CDN
database
identity
media
messagi
ng
networki
ng
traffic
cloud
services
Sunday, May 12, 13

68. Multiple
Languages
à http://www.windowsazure.com/en-us/
develop/overview/
Sunday, May 12, 13

69. open
source
à http://github.com/
windowsazure
Sunday, May 12, 13

70. Software: Apps Marketplaces
45
VMware Horizon App Manager
Sunday, May 12, 13

71. BigData Platforms: Hadoop
§ Apache Hadoop, open source version of Google MapReduce, GFS...
§ Cloudera, many others, space heating up
§ EMC, HortonWorks distros
§ Google Bigquery
§ Be your own bitch, today, Cloudera distro
46
Sunday, May 12, 13

72. Services
§ Services
§ Apis, apigee, mashery
§ Telephony, Twilio
§ Geo
§ Social
§ Visualization
47
Sunday, May 12, 13

73. Development
§ Final frontier, happening now
§ Not whole product yet
§ Source: Google code, Github
§ Dev: Cloud9, Exo, Orion
§ Higher level case tools, Wavemaker, Orangescape, Runmyprocess
§ Build: Jenkins, Cloudbees, OpenShift
§ Test: feature of cloud platforms
§ Community Stackoverflow, Github
48
Sunday, May 12, 13

74. 49
Lessons for
Developers
Sunday, May 12, 13

75. Predictions
§ Software is becoming like fashion, design rules
§ Welcome to Babel, use the best tool for the job, embrace multiple
language & heterogeneity
§ Our jobs will change, build yourself out of your current job
§ Sysadmin jobs will morph to hybrid cloud admins
§ Many opportunities open when you embrace change
50
Sunday, May 12, 13

76. Things to Forget
§ First normal form
§ Waterfall model
§ Single server deployment
§ Single language skill
51
Sunday, May 12, 13

77. Things to Learn
• Agile, API Design
• UI Design, Javacript, HTML5, CSS3
• A/B Testing
• Open Source, Open Standards
• Architecture, Distributed Computing (CAP theorem, 8 fallacies)
• Cloud Platforms and APIs
• Multiple types of languages (imperative, object, functional, logic)
• Learn to live in a box (embrace platform limitations) to think outside the box
52
Sunday, May 12, 13

78. What it means for you
§ Build On the shoulders of giants
§ Take risks, to innovate
§ Learn everyday, try different things
§ learn an api or framework / month
§ Learn a language / year
§ Be fast and agile
§ Make money
§ Social and app stores
53
Sunday, May 12, 13

79. Such stuff as dreams are made on
§ Like a kid on a candy store: there's never been a better time to be a
software developer!
§ Welcome to the Cloud: embrace change and reinvent yourselves
§ “The future is already there, not evenly distributed” Gibson
§ We Developers, invent the future today
54
Sunday, May 12, 13

80. Thank You!
@WindowsAzure
@chanezon
Website : www.windowsazure.com/
t
t
Sunday, May 12, 13

81. Books / Articles
§ Nick Carr, The Big Switch
§ Eric Raymond, The Art of Unix Programming
§ Weinberg, Psychology of Computer Programming
§ Wes python book
§ Mark html5 book
§ Kent Beck XP
§ Hunt, Thomas, The Pragmatic Programmer
§ Ade Oshineye, Apprenticeship Patterns
§ Matt Cutt's Ignite Talk IO 2011, Trying different things
§ Josh Bloch talk about api design
§ Larry and Sergey, Anatomy of a Search Engine
§ Rob Pike, The Practice of Programming
56
Sunday, May 12, 13

82. Papers / Talks
§ Simon Wardley, Oscon 09 “Cloud - Why IT Matters”
§ Tim O’Reilly article on internet os
§ Peter Deutsch’s 8 Fallacies of Distributed Computing
§ Brewer’s CAP Theorem
§ Gregor Hohpe’s Starbucks Does Not Use Two-Phase Commit
§ Herb Sutter, Welcome to the Jungle
http://herbsutter.com/welcome-to-the-jungle/
§ Stuff I tag http://www.delicious.com/chanezon/
§ More specifically http://www.delicious.com/chanezon/cloudfoundry
§ My previous Talks http://www.slideshare.net/chanezon
§ My list of favorite books
http://www.chanezon.com/pat/soft_books.html
57
Sunday, May 12, 13

83. Acknowledgement
§ Drawings from my daughters Eliette
58
Sunday, May 12, 13