Today, there are several trends that are forcing application architectures to evolve. Users expect a rich, interactive and dynamic user experience on a wide variety of clients including mobile devices. Applications must be highly scalable, highly available and run on cloud environments. Organizations often want to frequently roll out updates, even multiple times a day. Consequently, it’s no longer adequate to develop simple, monolithic web applications that serve up HTML to desktop browsers. In this talk we describe the limitations of a monolithic architecture. You will learn how to use the scale cube to decompose your application into a set of narrowly focused, independently deployable back-end services and an HTML 5 client. We will also discuss the role of technologies such as NodeJS and AMQP brokers. You will learn how a modern PaaS such as Cloud Foundry simplifies the development and deployment of this style of application.

1. Decomposing applications for
deployability and scalability
Chris Richardson
Author of POJOs in Action
Founder of the original CloudFoundry.com
@crichardson
chris.richardson@springsource.com
http://plainoldobjects.com
@crichardson

2. Presentation goal
How decomposing applications
improves deployability and
scalability
and
How Cloud Foundry helps
@crichardson

3. About Chris
@crichardson

4. (About Chris)
@crichardson

5. About Chris()
@crichardson

6. About Chris
@crichardson

7. About Chris
http://www.theregister.co.uk/2009/08/19/springsource_cloud_foundry/
@crichardson

8. vmc push About-Chris
Developer Advocate
Signup at http://cloudfoundry.com
@crichardson

9. Agenda
The (sometimes evil) monolith
Decomposing applications into services
How do services communicate?
Presentation layer design
How Cloud Foundry helps
@crichardson

10. Let’s imagine you are building
an e-commerce application
@crichardson

11. Traditional web application
architecture WAR
StoreFrontUI
Accounting
Service MySQL
Browser Apache
Database
Inventory
Service
Shipping
Service
Simple to
develop Tomcat
test
deploy
scale
@crichardson

12. But there are problems with
a monolithic architecture
@crichardson

13. Users expect a rich, dynamic
and interactive experience
h
oug
d en
goo
HTTP Request
is n’t
e
ec tur Java Web
hit
Browser
HTML/Javascript Application
I arc
ty le U
s
Old
Real-time web ≅ NodeJS
@crichardson

14. Intimidates developers
@crichardson

15. Obstacle to frequent
deployments
Need to redeploy everything to change one component
Interrupts long running background (e.g. Quartz) jobs
Increases risk of failure
Fear of change
Updates will happen less often
e.g. Makes A/B testing UI really difficult
@crichardson

16. Overloads your IDE and
container
Slows down development
@crichardson

17. Obstacle to scaling
development
Accounting
E-commerce
Engineering
application
Shipping team
@crichardson

18. Obstacle to scaling
development
WAR
UI team StoreFrontUI
Accounting Accounting
Inventory team InventoryService
Shipping team Shipping
@crichardson

19. Obstacle to scaling
development But
I want the backend is not working
to update the UI yet!
Lots of coordination and
communication required @crichardson

20. Requires long-term commitment
to a technology stack
@crichardson

21. Agenda
The (sometimes evil) monolith
Decomposing applications into services
How do services communicate?
Presentation layer design
How Cloud Foundry helps
@crichardson

22. @crichardson

23. The scale cube
Y axis -
functional
decomposition
Scale by
sim ing
splitting
r
n
ila
litt rtitio
different things
a
ng ing
by ta p
s
da
sp
Sc is -
thi
ax
ale
Z
X axis
- horizontal duplication @crichardson

24. Y-axis scaling - application level
WAR
StoreFrontUI
Accounting
Service
Inventory
Service
Shipping
Service
@crichardson

25. Y-axis scaling - application level
accounting web application
Accounting
Service
Store front web application inventory web application
StoreFrontUI Inventory
Service
shipping web application
Shipping
Service
Apply X axis cloning and/or Z axis partitioning to each service
@crichardson

26. Partitioning strategies...
Partition by verb, e.g. shipping service
Partition by noun, e.g. inventory service
Single Responsibility Principle
Unix utilities - do one focussed thing well
@crichardson

27. ...Partitioning strategies
Too few
Drawbacks of the monolithic architecture
Too many - a.k.a. Nano-service anti-pattern
Runtime overhead
Potential risk of excessive network hops
Potentially difficult to understand system
Something of an art
@crichardson

28. Example micro-service
require 'sinatra'
post '/' do
phone_number = params[:From]
registration_url = "#{ENV['REGISTRATION_URL']}?phoneNumber=#{URI.encode(phone_number, "+")}"
<<-eof
<Response>
<Sms>To complete registration please go to #{registration_url}</Sms>
</Response>
eof
end
For more on micro-services see
@fgeorge52
@crichardson

29. Service deployment options
Isolation, manageability
VM
Linux Container/LXC
JVM
JAR/WAR/OSGI bundle/...
Density/efficiency
@crichardson

30. Real world examples
http://techblog.netflix.com/
http://highscalability.com/amazon-architecture
http://www.addsimplicity.com/downloads/
eBaySDForum2006-11-29.pdf
http://queue.acm.org/detail.cfm?id=1394128
@crichardson

31. There are drawbacks
@crichardson

32. Complexity
See Steve Yegge’s Google Platforms Rant re
Amazon.com
@crichardson

33. Multiple databases
&
Transaction management
@crichardson

34. Implementing features that
span multiple services
@crichardson

35. When to use it?
In the beginning:
•You don’t need it
•It will slow you down
Later on:
•You need it
•Refactoring is painful
@crichardson

36. But there are many benefits
Scales development: develop, deploy and scale each service
independently
Update UI independently
Improves fault isolation
Eliminates long-term commitment to a single technology stack
Modular, polyglot, multi-
framework applications
@crichardson

37. Two levels of architecture
System-level
Services
Inter-service glue: interfaces and communication mechanisms
Slow changing
Service-level
Internal architecture of each service
Each service could use a different technology stack
Pick the best tool for the job
Rapidly evolving
@crichardson

38. If services are small...
Regularly rewrite using a better technology stack
Adapt system to changing requirements and better
technology without a total rewrite
Pick the best developers rather than best <pick a
language> developers polyglot culture
@crichardson

39. The human body as a system
@crichardson

40. 50 to 70 billion of your cells die
each day
@crichardson

41. Yet you (the system) remain you
@crichardson

42. Can we build software systems
with these characteristics?
http://dreamsongs.com/Files/
DesignBeyondHumanAbilitiesSimp.pdf
http://dreamsongs.com/Files/WhitherSoftware.pdf
@crichardson

43. Agenda
The (sometimes evil) monolith
Decomposing applications into services
How do services communicate?
Presentation layer design
How Cloud Foundry helps
@crichardson

44. Inter-service communication
options
Synchronous HTTP asynchronous AMQP
Formats: JSON, XML, Protocol Buffers, Thrift, ...
Even via the database
Asynchronous is preferred
JSON is fashionable but binary format
is more efficient
@crichardson

45. Asynchronous message-based communication
wgrus-billing.war
Accounting
Service
wgrus-store.war wgrus-inventory.war
RabbitMQ
StoreFrontUI (Message InventoryService MySQL
Broker)
wgrus-shipping.war
ShippingService
@crichardson

46. Benefits
Decouples caller from server
Caller unaware of server’s coordinates (URL)
Message broker buffers message when server is down/
slow
Supports a variety of communication patterns, e.g. point-
to-point, pub-sub, ...
@crichardson

47. Drawbacks
Additional complexity of message broker
Request/reply-style communication is more complex
@crichardson

48. Writing code that calls
services
@crichardson

49. The need for parallelism
Service B
b = serviceB()
Call in parallel
c = serviceC()
Service A Service C
d = serviceD(b, c)
Service D
@crichardson

50. Java Futures are a great
concurrency abstraction
http://en.wikipedia.org/wiki/
Futures_and_promises
@crichardson

51. Using Java Futures
public class Client {
private ExecutorService executorService;
private RemoteServiceProxy remoteServiceProxy; Eventually contains result
public void doSomething() throws ... {
Future<Integer> result =
executorService.submit(new Callable<Integer>() {
@Override
public Integer call() throws Exception {
return remoteServiceProxy.invokeRemoteService();
}
});
/// Do other things
When needed wait for result
int r = result.get(500, TimeUnit.MILLISECONDS);
System.out.println(r);
}
} @crichardson

52. Better future implementations
Guava ListenableFutures = better
Scala’s composable Futures = really good
Java 8 CompletableFuture = great
@crichardson

53. Composable Futures
val f1 = Future { ... ; 1 }
val f2 = Future { ... ; 2 }
Transforms Future
val f4 = f2.map(_ * 2)
assertEquals(4, Await.result(f4, 1 second))
Combines two futures
val fzip = f1 zip f2
assertEquals((1, 2), Await.result(fzip, 1 second))
http://doc.akka.io/docs/akka/2.0.1/scala/futures.html
@crichardson

54. Using Scala futures
def callB() : Future[...] = ...
def callC() : Future[...] = ...
def callD() : Future[...] = ... Two calls execute in parallel
val future = for {
(b, c) <- callB() zip callC();
d <- callD(b, c)
} yield d And then invokes D
val result = Await.result(future, 1 second)
http://doc.akka.io/docs/akka/2.0.1/scala/futures.html Get the result of D
@crichardson

55. Spring Integration
Provides the building blocks for a pipes
and filters architecture
Enables development of application
components that are
loosely coupled
insulated from messaging infrastructure
Messaging defined declaratively
@crichardson

56. Handling partial failures
Down?
Slow?
Service A Service B
Down?
Slow?
@crichardson

57. About Netflix
> 1B API calls/day
1 API call average 6 service calls
Fault tolerance is essential
http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html
@crichardson

58. How to run out of threads
HTTP Request X
X
X
Thread 1
Thread 2
Thread 3
X
Thread n
Execute thread
Service A
X
Service B
pool Eventually all threads If service B is
down then thread
Tomcat will be blocked will be blocked
@crichardson

59. Their approach
Network timeouts and retries
Invoke remote services via a bounded thread pool
Use the Circuit Breaker pattern
On failure:
return default/cached data
return error to caller
https://github.com/Netflix/Hystrix
See my talk tomorrow for more details
@crichardson

60. Agenda
The (sometimes evil) monolith
Decomposing applications into services
How do services communicate?
Presentation layer design
How Cloud Foundry helps
@crichardson

61. Presentation layer evolution....
WAR
StoreFrontUI
HTML / HTTP View Controller
Browser
Model
@crichardson

62. ...Presentation layer evolution
Browser Web application
Static
View Controller content
JSON-REST
RESTful
Model Endpoints
HTML 5 - Events Event
JavaScript publisher
No elaborate, server-side web framework
required @crichardson

63. How to publish events to
browser?
@crichardson

64. NodeJS is the fashionable
technology
@crichardson

65. Why NodeJS?
Familiar Javascript
High-performance, scalable event-driven, non-blocking I/O
model
Compact runtime
Over 17,000 modules developed by the community
Simple event publishing using socket.io or SockJS
@crichardson

66. Why not NodeJS?
a.k.a. callback hell
@crichardson

67. A modern web application
Browser
Node JS
RESTful WS Service 1
HTML 5/
Server
Java
application Service 2
Script Events
Socket.io
Socket.io
client
server
Service 3
@crichardson

68. NodeJS - using RESTful WS
and AMQP
REST
Service
REST
Requests
Node JS
Events
socket.io
AMQP AMQP
RabbitMQ Service
@crichardson

69. Socket.io server-side
var express = require('express')
, http = require('http')
, amqp = require(‘amqp’)
....;
Handle
server.listen(8081); socket.io
...
var amqpCon = amqp.createConnection(...);
connection
io.sockets.on('connection', function (socket) {
function amqpMessageHandler(message, headers, deliveryInfo) {
var m = JSON.parse(message.data.toString());
Republish
socket.emit(‘tick’, m); as socket.io
};
amqpCon.queue(“”, {},
event
function(queue) {
queue.bind(“myExchange”, “”);
queue.subscribe(amqpMessageHandler);
}); Subscribe to
}); AMQP queue
https://github.com/cer/nodejs-clock @crichardson

70. Socket.io - client side
<html>
<body>
The event is <span data-bind="text: ticker"></span>
<script src="/socket.io/socket.io.js"></script>
<script src="/knockout-2.0.0.js"></script>
<script src="/clock.js"></script>
Bind to model Connect to
</body>
</html>
socket.io
clock.js
var socket = io.connect(location.hostname);
function ClockModel() {
self.ticker = ko.observable(1); Subscribe
socket.on('tick', function (data) {
self.ticker(data);
to tick event
});
}; Update
ko.applyBindings(new ClockModel()); model
@crichardson

71. Agenda
The (sometimes evil) monolith
Decomposing applications into services
How do services communicate?
Presentation layer design
How Cloud Foundry helps
@crichardson

72. Original architecture
WAR
StoreFrontUI
Accounting
Service
MySQL
Browser Apache
Database
InventoryService
Shipping
Tomcat
@crichardson

73. Modern architecture
p,
lo
Native Mobile HTML5 mobile
Desktop Browser application
application
ve
StoreUI StoreUI StoreUI
st de
NodeJS
?
Asynchronous, Javascript
is
scalable
communication StoreUI
te e
th
w
RabbitMQ
oy
do
Spring/Scala pl Standalone
web application Inventory Shipping
“headless”
ow
Service Service
de
Spring/Java
applications
H
d
MySQL
an
Billing Service Customer Redis Inventory Mongo Order
Database Database Database
@crichardson

74. Traditional tools: monolithic
applications
@crichardson

75. Developing modular apps is
more difficult
Many more moving parts to manage
Platform services: SQL, NoSQL, RabbitMQ
Application services: your code
Who is going to setup the environments:
the developer sandbox?
...
But Cloud Foundry helps...
@crichardson

76. Easy polyglot application deployment and
service provisioning
OSS community
Ap
p lica
'o
vFabric
Postgres Private
n
S
Clouds
Data Services
erv
vFabric Public
i
ce
Msg
RabbitMQTM
Clouds
Services
In
Micro
ter
Other
Services
Clouds
fac
Additional partners
e
services …
@crichardson

77. Creating a platform service
instance
$ vmc create-service mysql mysql1
Creating Service: OK
$ vmc services
......
=========== Provisioned Services ============
+-------------+---------+
| Name | Service |
+-------------+---------+
| mysql1 | mysql |
+-------------+---------+
@crichardson

78. Multi-application manifest -
part 1
---
applications:
inventory/target: Path to application
name: inventory
url: cer-inventory.chrisr.cloudfoundry.me
framework:
name: spring
info:
mem: 512M
description: Java SpringSource Spring Application
exec:
mem: 512M
instances: 1
services: Required platform services
si-rabbit:
type: :rabbitmq
si-mongo:
type: :mongodb
si-redis:
type: :redis
@crichardson

79. Multi-application manifest -
part 2
store/target:
Path to application
name: store
url: cer-store.chrisr.cloudfoundry.me
framework:
name: spring
info:
mem: 512M
description: Java SpringSource Spring Application
exec:
mem: 512M
instances: 1 Required platform services
services:
si-mongo:
type: :mongodb
si-rabbit:
type: :rabbitmq
@crichardson

80. One command to create platform
services and deploy application
$ vmc push
Would you like to deploy from the current directory? [Yn]:
Pushing application 'inventory'...
Creating Application: OK
Creating Service [si-rabbit]: OK
Binding Service [si-rabbit]: OK
Creating Service [si-mongo]: OK vmc push:
Binding Service [si-mongo]: OK
•Reads the manifest file
•Creates the required platform services
Creating Service [si-redis]: OK
Binding Service [si-redis]: OK
Uploading Application:
Checking for available resources: OK
•Deploys all the applications
Processing resources: OK
Packing application: OK
Uploading (12K): OK
Push Status: OK
Staging Application 'inventory': OK
Starting Application 'inventory': OK
Pushing application 'store'...
@crichardson

81. Micro Cloud Foundry: new developer sandbox
App Instances Services
Open source Platform as a Service project
10.04
A PaaS packaged as a VMware Virtual Machine
Use as a developer sandbox
• Use the services from Junit integration tests
• Deploy your application for functional testing
• Remote debugging from STS @crichardson

82. Using Caldecott to tunnel
into your services
@crichardson

83. Caldecott = TCP over HTTP
native
protocol native
HTTP
Service Caldecott Caldecott protocol
Service
client Port gem application
NNN
Your computer Cloud Foundry
@crichardson

84. Using Caldecott…
$ vmc tunnel
1: mysql-135e0
2: mysql1
Which service to tunnel to?: 2
Password: ********
Stopping Application: OK
Redeploying tunnel application 'caldecott'.
Uploading Application:
Checking for available resources: OK
Packing application: OK
Uploading (1K): OK
Push Status: OK
Binding Service [mysql1]: OK
Staging Application: OK
Starting Application: OK
Getting tunnel connection info: OK
Service connection info:
username : uMe6Apgw00AhS
password : pKcD76PcZR7GZ
name : d7cb8afb52f084f3d9bdc269e7d99ab50
Starting tunnel to mysql1 on port 10000.
1: none
2: mysql
Which client would you like to start?: 2
@crichardson

85. …Using Caldecott
Launching 'mysql --protocol=TCP --host=localhost --port=10000 --
user=uMe6Apgw00AhS --password=pKcD76PcZR7GZ
d7cb8afb52f084f3d9bdc269e7d99ab50'
Welcome to the MySQL monitor. Commands end with ; or g.
Your MySQL connection id is 10944342
Server version: 5.1.54-rel12.5 Percona Server with XtraDB (GPL), Release 12.5,
Revision 188
Copyright (c) 2000, 2011, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or 'h' for help. Type 'c' to clear the current input statement.
mysql>
@crichardson

86. Running JUnit test with
Caldecott
Configure your test code to use port + connection info
@crichardson

87. Summary
@crichardson

88. Monolithic applications are
simple to develop and deploy
BUT have significant
drawbacks
@crichardson

89. Apply the scale cube
Modular, polyglot, and
scalable applications
Services developed,
deployed and scaled
independently
@crichardson

90. Cloud Foundry helps
.js
Ap
p
e
lica
Private
fac
Data Services
'o
ter
Clouds
n
S
r
In
erv
ide
Public
i
ov
ce
Msg Services
In
Pr
Clouds
ter
ud
Cl o
fac
Micro
e
Other
Services
Clouds
@crichardson

91. @crichardson chris.richardson@springsource.com
http://plainoldobjects.com
Questions?
www.cloudfoundry.com @crichardson