1. Working Models ...
or how cloud is growing up and thinking about applications
Alex Heneveld / @ahtweetin
Fite Club, 22 May 2013

2. Cloud

3. Cloud
It’s not just about VM’s...

4. It’s about storage, networking, maas, ...
It’s not just about VM’s...
Cloud

5. It’s not just about VM’s...
Ultimately it’s about applications.
Cloud

6. Declarative — AWS CloudFormation and OpenStack Heat
Procedural — Apache Whirr and Cloudsoft Brooklyn
OpenStack Summit — Red Hat, RAX, IBM, OASIS TOSCA and CAMP
DSLs in Progress — OpenStack Heat and CAMP PDP
Ultimately it’s about applications.

7. AWS CloudFormation
{
"AWSTemplateFormatVersion" : "2010-09-09",
"Description" : "WordPress is web software …",
"Parameters" : {
"KeyName" : {
"Description" : "Name of an existing EC2 KeyPair for SSH access",
"Type" : "String" },
... },
"Mappings" : {
"AWSInstanceType2Arch" : {
"m1.small" : { "Arch" : "64" },
"m1.large" : { "Arch" : "64" },
... },
"AWSRegionArch2AMI" : {
"us-east-1" : { "32" : "ami-31814f58", "64" : "ami-1b814f72" }, ... } },
"Resources" : {
"WebServer": {
"Type": "AWS::EC2::Instance",
"Metadata" : {
"AWS::CloudFormation::Init" : {
"config" : {
"packages" : {
"yum" : {
"gcc-c++" : [],
"make" : [],
...

8. {
"AWSTemplateFormatVersion" : "2010-09-09",
"Description" : "WordPress is web software …",
"Parameters" : {
"KeyName" : {
"Description" : "Name of an existing EC2 KeyPair for SSH access",
"Type" : "String" },
... },
"Mappings" : {
"AWSInstanceType2Arch" : {
"m1.small" : { "Arch" : "64" },
"m1.large" : { "Arch" : "64" },
... },
"AWSRegionArch2AMI" : {
"us-east-1" : { "32" : "ami-31814f58", "64" : "ami-1b814f72" }, ... } },
"Resources" : {
"WebServer": {
"Type": "AWS::EC2::Instance",
"Metadata" : {
"AWS::CloudFormation::Init" : {
"config" : {
"packages" : {
"yum" : {
"gcc-c++" : [],
"make" : [],
...
"Resources" : {
"WebServer": {
"Type": "AWS::EC2::Instance"
AWS CloudFormation

9. "Resources" : {
"WebServer": {
"Type": "AWS::EC2::Instance"
Parameters
Mappings
(config metadata)
WaitConditions &
WaitHandles
(orchestration)
Outputs
Functions
AWS CloudFormation

10. "Resources" : {
"WebServer": {
"Type": "AWS::EC2::Instance"
AWS CloudFormation

11. "Resources" : {
"WebServer": {
"Type": "AWS::EC2::Instance"
AWS CloudFormation

12. "Type": "AWS::AutoScaling::AutoScalingGroup"
"Type": "AWS::EC2::EIP"
"Resources" : {
"WebServer": {
"Type": "AWS::EC2::Instance"
"Type": "AWS::SQS::Queue"
"Type": "AWS::CloudFormation::Stack"
"Type": "AWS::S3::Bucket"
AWS CloudFormation

13. "Type": "AWS::AutoScaling::AutoScalingGroup"
"Type": "AWS::EC2::EIP"
"Resources" : {
"WebServer": {
"Type": "AWS::EC2::Instance"
"Type": "AWS::SQS::Queue"
"Type": "AWS::CloudFormation::Stack"
"Type": "AWS::S3::Bucket"
AWS CloudFormation
Composition!

14. — Limited to AWS cloud
— Limited to AWS concepts
— Cumbersome to write
— Limited dependency injection
Nice, but:
AWS CloudFormation

15. — CloudFormation clone
— Targetting OpenStack resources and AWS compatibility
— Adds YAML support (comments and easier)
— Multi-cloud support using Apache Deltacloud
OpenStack Heat

16. (AWS CloudFormation)
{
"AWSTemplateFormatVersion" : "2010-09-09",
"Description" : "WordPress is web software …",
"Parameters" : {
"KeyName" : {
"Description" : "Name of an existing EC2 KeyPair for SSH access",
"Type" : "String" },
... },
"Mappings" : {
"AWSInstanceType2Arch" : {
"m1.small" : { "Arch" : "64" },
"m1.large" : { "Arch" : "64" },
... },
"AWSRegionArch2AMI" : {
"us-east-1" : { "32" : "ami-31814f58", "64" : "ami-1b814f72" }, ... } },
"Resources" : {
"WebServer": {
"Type": "AWS::EC2::Instance",
"Metadata" : {
"AWS::CloudFormation::Init" : {
"config" : {
"packages" : {
"yum" : {
"gcc-c++" : [],
"make" : [],
...
OpenStack Heat

17. OpenStack Heat
HeatTemplateFormatVersion: 2012-12-12
Description: WordPress is web software ...
Parameters:
KeyName:
Description: Name of an existing EC2 KeyPair for SSH access
Type: String
...
Mappings:
AWSInstanceType2Arch:
m1.small: {Arch: '32'}
m1.large: {Arch: '64'}
...
DistroArch2AMI:
F18: {'32': F18-i386-cfntools, '64': F18-x86_64-cfntools}, ...
Resources:
WebServer:
Type: AWS::EC2::Instance
Metadata:
AWS::CloudFormation::Init:
config:
packages:
yum:
gcc-c++: []
make: []
...

18. — CloudFormation clone
— Targetting OpenStack resources and AWS compatibility
— Adds YAML support (comments and easier)
— Multi-cloud support using Apache Deltacloud
OpenStack Heat
Available NOW in Grizzly!

19. OpenStack Heat
— Still limited to selected clouds
— Still limited in concepts
— Only slightly less cumbersome to write
— Very limited dependency injection
But...

20. Procedural Approach
10 INPUT "What app do you like? ", A$
20 INPUT "What cloud do you like? ", C$
30 GOSUB 100
40 END
100 REM deploy C$ to A$
110 RETURN

21. Apache Whirr
provision
install
configure
manage
— Apache top level project
— Deploy multi-node applications
— Hadoop, Hama, Mahout, web,
Cassandra & many more
— Define topologies declaratively
— Add new roles, configuration,
constraints and wiring procedurally

22. $ whirr launch-cluster
--cluster-name=hbase
--instance-templates='1 hbase-master,6 hbase-regionserver'
— Define topologies declaratively
Apache Whirr

23. — Add new roles, configuration,
constraints and wiring procedurally
spec = new ClusterSpec();
spec.setProvider("cloudservers-uk");
spec.setIdentity(apikey);
spec.setCredential(secret);
spec.setClusterName("hbase");
spec.setInstanceTemplates(ImmutableList.of(
new InstanceTemplate(1, "hbase-master"),
new InstanceTemplate(6, "hbase-regionserver")));
cluster = new ClusterController().launchCluster(spec);
Apache Whirr

24. — Limited parameterisation
— Defining new roles requires coding
— Strict phases limit applicability
But...
— Composable to one dimension
— Portable locations support
— Pluggable with Chef, Puppet, and more
Apache Whirr

25. Brooklyn & Cloudsoft AMP
• Brooklyn = Apache-licensed OSS
• AMP = Enterprise + Support
• Deployment and Management
• Hierarchical Autonomic Policies

26. Brooklyn
Contention Management
Custom PaaS
Big Data

27. Region #1
Presentation
Geographic
DNS Load
Balancer
Tomcat
Processing (Monterey)
Data (Gemfire)
Tomcat
Tomcat
Region #2
Presentation
Load
Balancer
Tomcat
Processing (Monterey)
Data (Gemfire)
Tomcat
Tomcat
Provisioning
Monitoring
Management
Brooklyn: Example, Physical Architecture

28. Data (Gemfire)Processing (Monterey)
Region #1
Presentation
Geographic
DNS
Load
Balancer
Tomcat
Processing (Monterey) Data (Gemfire)
Tomcat
Tomcat
Region #2
Presentation
Load
Balancer
Tomcat
Processing (Monterey) Data (Gemfire)
Tomcat
Tomcat
Provisioning Monitoring Management
WebappFabric
TomcatNodes
NginxNode
LoadBalancedWebAppCluster
TomcatNodes
NginxNode
LoadBalancedWebAppCluster
WebappGeoDnsFabric MontereyFabric
MontereyCluster
MontereyNodes
MontereyNodes
MontereySegments
GemfireFabric
MontereyCluster
GemfireCluster
GemfireCluster
Geoscaling
Brooklyn
and friends
GemfireNodes
GemfireNodes
Brooklyn: Example, Logical Architecture

29. Processing (Monterey) Data (Gemfire)
Autonomic Hierarchical Management
Geographic
DNS
Presentation
Load
Balancer
Tomcat Tomcat
WebappFabric
LoadBalancedWebAppCluster
Cluster Cluster Cluster Cluster
MyApplication
Policy
follow-the-
{sun,moon,X}
Policy
resizer
Policy
restart-2x
Policy
Brooklyn
and friends
WebappGeoDnsFabric MontereyFabric GemfireFabric
Brooklyn

30. Processing (Monterey) Data (Gemfire)
Typed Blueprints
replace groups with PaaS tools and
have the same sensors and policies
Geographic
DNS
Presentation
WebappFabric
PaaS
Cluster Cluster Service Cluster
MyApplication
Brooklyn
and friends
WebappGeoDnsFabric MontereyFabric GemfireFabric
Brooklyn
Policy
follow-the-
{sun,moon,X}
Policy
resizer
Policy
restart-2x
Policy

31. Data (Gemfire)
Data (Gemfire)
Processing (Monterey)
Presentation
Geographic
DNS
Load
Balancer
Tomcat
Processing (Monterey)
Tomcat
Tomcat
Presentation
Load
Balancer
Tomcat
Processing (Monterey) Data (Gemfire)
Tomcat
Tomcat
Provisioning Monitoring Management
WebappFabric
TomcatNodes
NginxNode
LoadBalancedWebAppCluster
TomcatNodes
NginxNode
LoadBalancedWebAppCluster
WebappGeoDnsFabric MontereyFabric
MontereyCluster
MontereyNodes
MontereyNodes
MontereySegments
GemfireFabric
MontereyCluster
GemfireCluster
GemfireCluster
Geoscaling
GemfireNodes
GemfireNodes fixed IP
&c
Brooklyn: Run in Many Locations

32. Data (Gemfire)
Data (Gemfire)
Processing (Monterey)
Presentation
Geographic
DNS
Load
Balancer
Tomcat
Processing (Monterey)
Tomcat
Tomcat
Presentation
Load
Balancer
Tomcat
Processing (Monterey) Data (Gemfire)
Tomcat
Tomcat
Provisioning Monitoring Management
WebappFabric
TomcatNodes
NginxNode
LoadBalancedWebAppCluster
TomcatNodes
NginxNode
LoadBalancedWebAppCluster
WebappGeoDnsFabric MontereyFabric
MontereyCluster
MontereyNodes
MontereyNodes
MontereySegments
GemfireFabric
MontereyCluster
GemfireCluster
GemfireCluster
Geoscaling
GemfireNodes
GemfireNodes
Policies
Policies
WAR
file
code schema
credentials
GeoDNS
config actor
descriptor
Regions
Brooklyn: finally { Simplify }

33. public class MyWebCluster extends AbstractApplication
implements MyWebClusterConstants {
public void init() {
MySqlNode mysql = addChild(EntitySpecs.spec(MySqlNode.class)
.configure("creationScriptUrl", DB_SETUP_SQL_URL));
ControlledDynamicWebAppCluster web = addChild(EntitySpecs.spec(
ControlledDynamicWebAppCluster.class)
.configure(WebAppService.HTTP_PORT, PortRanges.fromString("8080+"))
.configure(JavaWebAppService.ROOT_WAR, WAR_PATH)
.configure(javaSysProp("brooklyn.example.db.url"),
formatString("jdbc:%s%s?user=%s&password=%s",
attributeWhenReady(mysql, MySqlNode.MYSQL_URL),
DB_TABLE, DB_USERNAME, DB_PASSWORD)) );
}
}
Brooklyn: Nested Elastic Blueprints

34. public class MyWebCluster extends AbstractApplication
implements MyWebClusterConstants {
public void init() {
MySqlNode mysql = ...;
ControlledDynamicWebAppCluster web = ...;
web.addEnricher(HttpLatencyDetector.builder().url(ROOT_URL).
rollup(10, SECONDS).build());
web.getCluster().addPolicy(AutoScalerPolicy.builder().
metric(REQUESTS_PER_SECOND_IN_WINDOW_PER_NODE).
metricRange(10, 100).sizeRange(2, 5).build());
addEnricher(SensorPropagatingEnricher.newInstanceListeningTo(web,
ROOT_URL, REQUESTS_PER_SECOND_IN_WINDOW, REQUEST_LATENCY_IN_SECONDS_IN_WINDOW));
}
}
Brooklyn: Policies, Enrichers and KPI’s

35. Brooklyn
— Defining blueprints requires coding
But...
— Easily parameterisable blueprints
— Composable and substitutable
— Portable and powerful locations support
— Pluggable with VM images, Heat, Chef, more

36. Applications
— Components (IaaS-resources)
— DeclarativeInfrastructure
— Procedural
— Extensible (new resources)
— Composable (stacks)
— Management
— Portable (hybrid, metal)

37. Heat in Grizzly
Management
Composable
Declarative
Components
Extensible
Portable
thanks largely to Red Hat

38. Portland Design Summit — Heat
Management
Composable
Declarative
Portable
Components
Extensible

39. Portland Design Summit — Heat
Management
Composable
Declarative
Portable
Components
Extensible
lots of new blood (interest and ideas)

40. Portland Design Summit — Heat
Composable
Declarative
Components
Management
Portable
Ease-of-Use
Extensible
ingredients from Brooklyn, Whirr, OoO and others

41. Portland Design Summit — Heat
Composable
Declarative
Components
Management
Portable
Extensible
Auto-wiring
a special ingredient from Rackspace
Ease-of-Use

42. Portland Design Summit — Heat
Composable
Declarative
Components
Management
Extensible
Auto-wiring
Relationships
Ease-of-Use
Portable
a special ingredient from IBM

43. Management
Composable
Declarative
Components
Auto-wiring
Relationships
Extensible
Ease-of-Use
Portable
Portland Design Summit — Heat

44. Heat in Havana
Management
Composable
Declarative
Components
Auto-wiring
Relationships
Extensible
Ease-of-Use
Portable
Concurrent
Distributed
Resilient
Functions / DI

45. Heat in Havana
Management
Composable
Declarative
Components
Auto-wiring
Relationships
Extensible
Ease-of-Use
Portable
Concurrent
Distributed
Resilient
Functions / DI
Standards ?

46. Oasis
TOSCA
http://www.oasis-open.org/committees/tosca/
Topology and Orchestration Specification for Cloud Applications

47. Modeling Topologies with TOSCA
Service Topologies are described using the TOSCA “Meta-model”:
Artifacts
 Describe Installables and Executables required to
instantiate and manage a service. Currently, they
include:
 Implementation Artifacts:
– Executables or Plans that implement a Node’s
or Relationship’s Operations (e.g. a Bash script)
 Deployment Artifacts:
– Installables of the components (e.g. a TAR file)
A service’s Topology Model is included in a TOSCA Service Template which is packaged and shared, along with
all dependent artifacts, as a TOSCA Cloud Service Archive (CSAR)
Service Templates
 Group the nodes and relationships that make up
a service’s topology
– Allowing modeling of sub-topologies
 Service Templates “look like nodes” enabling:
 Composition of applications from one or more
service templates
 Substitution of abstract Node types with
available service templates of the same type
Nodes
 Represent Components of an application or service
and their Properties. Example nodes include:
– Infrastructure: Compute, Network, Storage, etc.
– Platform: OS, VM, DB, Web Server, etc.
– Granular: functional Libraries, Modules, etc.
 Include Operations which are the management
functions for the node
– e.g. deploy(), start(), stop(), connect(), etc.
 Export their dependencies on other nodes
as Requirement and Capabilities
Relationships
 Represent the logical Relationships between
nodes
– e.g. “hostedOn”, “connectsTo”, etc.
 Describes the valid Source and Target nodes they
are designed to couple
– e.g. source “web application” node is designed
to “connectTo” a target “database” node
 Have their own Properties and Constraints
Slide origin: IBM
TOSCA: Modelling Topologies

48. <Definitions id="PayrollDefinitions" targetNamespace="http://www.example.com/ste"
xmlns:pay="http://www.example.com/ste/Types">
<Import namespace="http://www.example.com/ste/Types" location="http://.../PayrollTypes.tosca"
importType=" http://docs.oasis-open.org/tosca/ns/2011/12"/>
<ServiceTemplate id="Payroll" name="Payroll Service Template">
<TopologyTemplate ID="PayrollTemplate">
<NodeTemplate id="Payroll Application" type="pay:ApplicationNodeType">
<DeploymentArtifacts>
<DeploymentArtifact name="PayrollEAR" type="http://..../CSARref">
EARs/Payroll.ear
</DeploymentArtifact>
</DeploymentArtifacts>
</NodeTemplate>
<NodeTemplate id="Application Server" type="pay:ApplicationServerNodeType">
<DeploymentArtifacts>
<DeploymentArtifact name="ApplicationServerImage" type="http://.../AMIref">
ami-edf2cf99
</DeploymentArtifact>
</DeploymentArtifacts>
</NodeTemplate>
<RelationshipTemplate id="deployed_on" type="pay:deployed_on">
<SourceElement ref="Payroll Application"/>
<TargetElement ref="Application Server"/>
</RelationshipTemplate>
</TopologyTemplate>
</ServiceTemplate>
</Definitions>
TOSCA: Sample Definition

49. — Fails the“ease-of-use”test
— Does not standardise an API
But...
— Powerful and broad modelling coverage
— Workflow, locations, substitution
— Relatively mature
TOSCA

50. Oasis
CAMP
www.cloudspecs.org
Cloud Application Management Platform
http://www.oasis-open.org/committees/camp/

51. CAMP: Objectives and Non-Objectives
— Software
— an Orchestration Spec
— a Modelling System
But CAMP is NOT
— a REST API
— for Deployment and Management
— of Applications
It provides a flexible way to use many platforms.
Interoperability.

52. CAMP: Key Concepts
Clients
REST
Platform
Platform
Component
Template
Requirement
Type
Platform
Component
TemplatePlatform
Component
Template
Requirement
Type
App Component
Type
App Component
Type

53. Clients Platform
Platform
Component
Template
Requirement
Type
Platform
Component
TemplatePlatform
Component
Template
Requirement
Type
App Component
Type
App Component
Type
PDP = Platform
Deployment Plan
(blueprint)
— Deployment PlanCAMP: Key Concepts
DSL & REST

54. App Component
Type
PDP = Platform
Deployment Plan
(blueprint)
CAMP: Key Concepts
Clients
REST
Platform
Platform
Component
Template
Requirement
Type
Platform
Component
TemplatePlatform
Component
Template
Requirement
Type
App Component
Type
— Creating Templates
POST

55. App Component
Template
Clients
REST
Platform
Platform
Component
Template
Requirement
Type
Platform
Component
TemplatePlatform
Component
Template
Requirement
Type
App Component
Type
App Component
Type
— ResolutionCAMP: Key Concepts
Assembly
Template
(blueprint)
PDP = Platform
Deployment Plan
(blueprint)POST

56. CAMP: Key Concepts
Clients
REST
Platform
Platform
Component
Template
Requirement
Type
Platform
Component
TemplatePlatform
Component
Template
Requirement
Type
App Component
Type
App Component
Type
App Component
Template
Requirement
Type
Platform
Component
Template
Requirement
Type
App Component
Type
— Resolution
Assembly
Template
(blueprint)
POST

57. Assembly
Template
(blueprint)
CAMP: Key Concepts
Clients
REST
Platform
Platform
Component
Template
Requirement
Type
Platform
Component
TemplatePlatform
Component
Template
Requirement
Type
App Component
Type
App Component
Type
App Component
Template
— Resolution
200
Assembly
Template
POST

58. Assembly
Template
(blueprint)
CAMP: Key Concepts
Clients
REST
Platform
Platform
Component
Template
Requirement
Type
Platform
Component
TemplatePlatform
Component
Template
Requirement
Type
App Component
Type
App Component
Type
App Component
Template
— Deployment

59. Assembly
(blueprint) Platform
Component
Platform
Component
App Component
Assembly
Template
(blueprint)
CAMP: Key Concepts
Clients
REST
Platform
Platform
Component
Template
Requirement
Type
Requirement
Type
App Component
Type
App Component
Type
App Component
Template
— Deployment
POST
Platform
Component
Template
Platform
Component
Template

60. Assembly
(blueprint) Platform
Component
Platform
Component
App Component
CAMP: Key Concepts
Clients
REST
Platform
— Deployment
Assembly
(blueprint) Platform
Component
Platform
Component
App Component
Assembly
Template
(blueprint)
Platform
Component
Template
App Component
Template
Platform
Component
Template
POST
200
Assembly

61. CAMP
— depends on orchestrations supporting this
— wants standardised requirement and component types
— still needs a nice DSL
But...
— all very general
— designed to map on to a wide range of orchestrations
PaaS, TOSCA, Brooklyn, Heat
— OSS code in progress
github.com/brooklyncentral/camp-server

62. CAMP: PDP/Blueprint DSL (work in progress)
Parameters
Components
Requirements
Functions
parameters & sensors
Outputs
*
*

63. CAMP: PDP/Blueprint DSL (work in progress)
Parameters
Functions
parameters & sensors
Outputs
*
*
components:
my_app:
type: WAR
content: { href: hello.war }
requirements:
hosted_on:
javaVersion: [1.7,)

64. Mappings
(config metadata)
WaitConditions &
WaitHandles
(orchestration)
Parameters
components:
my_instance:
type: server
ram: 2048m
requires:
connects_to:
target: my_database
Heat: New HOT DSL (also work in progress)
Functions
Outputs
Heat
Orchestration
Template
Coming in
Havana

65. Wrap-Up
Declarative — AWS CloudFormation and OpenStack Heat
Procedural — Apache Whirr and Cloudsoft Brooklyn
OpenStack Summit — Red Hat, RAX, IBM, OASIS TOSCA and CAMP
DSLs in Progress — OpenStack Heat and CAMP PDP
It’s about the apps.
Come get involved!

66. Thank you.
alex.heneveld@cloudsoftcorp.com
@ahtweetin