Decomposing Applications for Scalability and Deployability (April 2012)

Decomposing applications for
deployability and scalability
Chris Richardson

Author of POJOs in Action
Founder of the original CloudFoundry.com

@crichardson
crichardson@vmware.com

1

Presentation goal

Decomposing applications
to improve deployability
and scalability
and
How Cloud Foundry helps
2

About Chris

http://www.theregister.co.uk/2009/08/19/springsource_cloud_foundry/

7

vmc push About-Chris

Developer Advocate for
CloudFoundry.com

Signup at http://cloudfoundry.com

Promo code: CFOpenTour2012
8

Agenda
§ The (sometimes evil) monolith
§ Decomposing applications into services
§ How do services communicate?
§ Presentation layer design
§ Decomposing a monolithic application
§ How Cloud Foundry helps

9

Let’s imagine you are building an e-
commerce application

10

Traditional web application architecture

WAR

StoreFrontUI

Accounting
Service
MySQL
Browser Apache Database
InventoryService

Shipping
Service

Simple to Tomcat
develop
test
deploy
scale
11

But there are problems

12

Users expect a rich, dynamic and
interactive experience on mobile
devices and desktop
h
oug
d en
oo
’t g
HTTP Request
e isn
r
ctu
Java Web
Browser
e
hit
HTML/Javascript Application
c
I ar
ty le U
s
Old

Real-time web ≅ NodeJS
13

One dimensional scalability
§ Scales to handle transaction volumes
§ Higher transaction rate required run more instances
§ Some clouds will do this automatically, others easily
§ No code changes required
BUT
§ Does not scale with increasing data volumes
§ Caching is less effective
§ Requires more memory
§ Requires more I/O

14

Components that don’t cluster
§ Most application components are easily clustered:
• Most services are stateless
• Server-side session state: sticky session routing/state
server

BUT there are exceptions

§ Singletons that maintain global application state:
• e.g. CEP engine analyzing event stream over HTTP
§ Commercial software with a per-server license fee

§ If one application component can’t be clustered you can’t
cluster the application!

15

Obstacle to frequent deployments
§ Need to redeploy everything to change one component
§ Interrupts long running background (e.g. Quartz) jobs
§ Increases risk of failure

Fear of change

§ Updates will happen less often
§ e.g. Makes A/B testing UI really difficult 16

Large application

Slow IDE
Slow start times for web container

17

Scaling development
WAR

StoreFrontUI

Accounting
Scalable
!=
InventoryService development
Shipping

§ Forces teams to synchronize development efforts
§ Teams need to coordinate updates

18

Long-term commitment to a single technology stack
§ Let’s suppose you go with the JVM
• Some polyglot support
• Many (but not all) JVM languages interoperate easily
• e.g. Scala, Groovy modules within a Java application
• But you can’t use non-JVM languages
§ Application depends on a particular:
• Framework - e.g. Spring or Java EE
• Container - e.g. Tomcat
§ Switching technology stack touches entire application
• Painful
• Rarely done

19

Agenda

20

3 dimensions to scaling

Y axis -
functional
decomposition

Scale by

s
ng
splitting

ila g
n

hi
g s oni

rt
different things

iti
rt

im
pa
ta

in
da

ittl
s-

sp
i
ax

by
Z
ale
X axis - horizontal duplication
Sc

Scale by cloning
21

X axis scaling - application level

Load WAR
Balancer

Tomcat

22

X axis scaling - database level
Writes Consistent reads

MySQL
Master
Inconsistent reads

MySQL MySQL MySQL
Slave 1 Slave 2 Slave N

Slaves are clones
23

Benefits and drawbacks of X axis splits
§ Benefits
• Easy and fast to implement
• Handles increasing transaction volumes
• Simple to manage
§ Drawbacks
• Doesn’t scale with increasing data volumes
• Doesn’t address application/development complexity
• Couples entire application to a particular technology stack

24

Z axis scaling - data partitioning
§ Each server runs the identical code base
§ Different servers responsible for different data
§ Route request based on
• Attribute of the request
• Identity/attribute of the requestor/customer
§ For example:
• Sharding - partition data by primary key
• Free customer vs. paying customer

Server 1
Request:
... Some attribute ...

“Router”
Server 2

...
25

Z axis scaling - database sharding
Partition column

RESTAURANT table

ID Name …
1 Ajanta
2 Eggshop
…

X axis scaling too!

Database server 1 Database server 2

ID Name … ID Name …
1 Ajanta 2 Eggshop
… …

26

Z axis scaling - application example
Content Query

Update Router Query Aggregator

Search Service Search Service
Search Service Search Service
Cache Cache

Database server 1 Database server 2

Partition 1 Partition 2 27

Benefits and drawbacks of Z axis splits
§ Benefits
• Each server only deals with a subset of the data
• Improved caching
• Reduced memory usage and reduced I/O
• Improved transaction scalability
• Improved fault isolation - failure only makes part of the data
inaccessible
§ Drawbacks
• Doesn’t scale with increasing development and application
complexity
• Increased management/deployment complexity

28

Y axis scaling - functional partitioning
§ Splits monolithic application into a set of services
§ Each service implements related set of functionality
§ Partitioning schemes:
• Partition functionality by noun or by verb
• Single Responsibility Principle
• e.g. Unix utilities - do one focussed thing well

29

Y axis scaling - application level
billing web application

Accounting
Service

Store front web application inventory web application

Store Front InventoryService

shipping web application

ShippingService

Apply X axis cloning and/or Z axis partitioning to each service
30

Y axis scaling - databases
Application

MySQL MySQL
MySQL MySQL

Order Customer
table table

.... ....

Apply X axis cloning and/or Z axis partitioning to each database
31

Real world examples

http://techblog.netﬂix.com/

Between 100-150 services are accessed to
build a page.
http://highscalability.com/amazon-architecture

http://www.addsimplicity.com/downloads/
eBaySDForum2006-11-29.pdf

http://queue.acm.org/detail.cfm?id=1394128

32

Benefits and drawbacks of Y axis splits
§ Benefits
• Scales development: focussed two pizza devops teams
• Deploy services independently
• Scale services independently
• Improves fault isolation
• Eliminates long-term commitment to a single technology
stack
• Enforces well defined interfaces between components
§ Drawbacks
• Implementation complexity
• Deployment complexity

33

Two levels of architecture
§ System level
• Defines the inter-service glue: interfaces and
communication mechanisms
• Slow changing

Versus

§ Service level
• Defines the internal architecture of each service
• Far fewer constraints on technology
• Each service could use a different technology stack
• Rapid evolving

34

If services are small...
§ Regularly rewrite using a better technology stack
§ Pick the best developers rather than best <pick a
language> developers
§ Adapt system to changing requirements and better
technology without a total rewrite

Fred George
“Developer Anarchy”

35

Moreover: you are not the same you ...
§ Cell lifetimes: Can we build software
• hours - some white blood cells systems with these
characteristics?
• days - stomach lining cells
• years - bone cells Too much technical debt
• lifetime - brain cells component death?
§ 50 to 70 billion of your cells die each day
§ Yet you (the system) remains intact
http://dreamsongs.com/Files/
DesignBeyondHumanAbilitiesSimp.pdf

http://dreamsongs.com/Files/WhitherSoftware.pdf

36

Agenda

37

Inter-service communication options
§ Multiple collaborating services need a communication
mechanism
§ Many choices:
• Synchronous asynchronous
• Transports: HTTP, AMQP, ...
• Formats: JSON, XML, Protocol Buffers, Thrift, ...
• Even via the database
§ Distributed application error handling strategies

38

Synchronous communication
wgrus-billing.war

Accounting
Service

REST
wgrus-store.war
wgrus-shipping.war
SOAP
StoreFrontUI Shipping MySQL
Service
Thrift

... wgrus-inventory.war

InventoryService

Lots of choices
39

Service discovery options
§ Clients need to know the coordinates of their dependencies

§ Option #1: Caller is configured with dependencies’s URL
• e.g. Environment variables, system properties
§ Option #2: Services announce their location
• Relies on ‘broadcast’ protocol or Gossip protocol to
announce location
• e.g. RabbitMQ message broker
§ Option #3: configuration server
• Maintains all the configuration information
• e.g. Zookeeper

40

Benefits and drawbacks
§ Benefits
• Simple to implement and use
§ Drawbacks
• Poor fault isolation: caller is blocked if server is down/slow
• Caller needs to know server’s coordinates (URL, ...)
• Less scalable than asynchronous protocols

41

Asynchronous message-based communication
wgrus-billing.war

Accounting
Service

wgrus-store.war
RabbitMQ wgrus-inventory.war
StoreFrontUI (Message
MySQL
Broker) Widget
InventoryService

wgrus-inventory.war

InventoryService

42

§ Benefits
• Decouples caller from server
• Caller unaware of server’s coordinates (URL)
• Message broker buffers message when server is down/slow
§ Drawbacks
• Additional complexity of message broker
• RPC using messaging is more complex

43

Shared databases?

Order Management Customer Management

Database

Orders
Customers

44

§ Benefits
• Simple
• Single (consistent) source of data
§ Drawbacks
• Defeats the purpose of having multiple services
• Less scalable
• More coupling

45

Better: separate databases

Customer Update Event
Order Management Customer Management

Database
Database
Orders
Customers
Customers (copy)

46

Maintaining consistency without 2PC
begin transaction
update order tables
save intent to update customer tables
commit transaction
for each saved intent
begin transaction
delete intent
queue message to update customer Customer management
commit transaction dequeue message
Order management begin transaction
if (update has not been applied) {
record update as applied
update customer tables
BASE: An Acid Alternative by Dan Pritchett
}
http://queue.acm.org/detail.cfm?id=1394128 commit transaction
acknowledge message 47

Writing code that calls services

48

Composable futures
§ Problem:
• Service A needs to call services B and C and then D
• Makes sense to call B and C parallel
• Yet most concurrency APIs are low-level, error-prone etc
§ Solution:
• Use Akka composable futures = really nice abstraction

val futureB = callB() Two calls execute in parallel
val futureC = callC()

val futureD = for {
b <- futureB.mapTo[SomeType]
c <- futureC.mapTo[SomeType]
d <- callD(b, c) And then invokes D
} yield d

val result = Await.result(futureD, 1 second).
asInstanceOf[SomeOtherType]

Get the result of D
http://doc.akka.io/docs/akka/2.0.1/scala/futures.html
http://en.wikipedia.org/wiki/Futures_and_promises
49

Spring Integration

§ Builds on Spring framework
§ High-level of abstraction for building message
based applications
§ Implements EAI patterns
§ Provides plumbing for exchanging messages
between application components
§ Promotes loosely coupled components
§ Integrates with external messaging infrastructure:
JMS, AMQP, HTTP, Email, File transfer

50

Spring Integration concepts

§ Message channel
• Virtual pipe connecting producer and consumer
§ Message endpoints
• The filter of a pipes-and-filter architecture
• Read from and/or write to channel
§ Endpoint types:
• Transformer
• Filter
• Router
• Splitter
• Aggregator
• ServiceActivator
• Inbound channel adapter - read from external source, writes to channel
• Outbound channel adapter - read from channel write to external destination

51

Spring Integration insulates
components from the underlying
communication mechanism

52

Example of reconfigurability - local
@Service
public class OrderServiceImpl { @Service
public class ShippingServiceImpl {
@Autowired
private ShippingService shippingService; public void shipOrder(String orderId) {
System.out.println("shipped order: " + orderId);
public void placeOrder() { }
String orderId = generateOrderId();
… }
shippingService.shipOrder(orderId);
}

}

Order Shipping
Service service

Messaging
Gateway

Channel Service
Activator
53

Example of reconfigurability - distributed

Code unchanged in new deployment

Order Shipping
Service service

Messaging
Gateway RabbitMQ

Channel AMQP AMQP Channel Service
Activator

54

Handling failure
§ Errors happen (especially in distributed systems)
§ Use timeouts and retries
• Never wait forever
• Some errors are transient so retry
§ Use per-dependency bounded thread pool with bounded queue
• Limits number of outstanding requests
• Fails fast if service is slow or down
§ Use circuit breaker
• High rate of errors stop calling temporarily
• Avoids calling service that has issues
§ On failure
• Returned cached or default data
• Invoke custom error handler

http://techblog.netﬂix.com/2012/02/fault-tolerance-in-high-volume.html
55

Agenda

56

Modular application

Choice of presentation layer technology
+
Redeploy UI frequently/independently

57

NodeJS is the fashionable technology

Many JavaScript client frameworks have a NodeJS counterpart
e.g. socket.io

58

NodeJS isn’t the only game in town

JVM-based http://vertx.io/

59

A modern web application

Service 1
RESTful WS Node JS
Browser

HTML 5 Server Application
Application Service 2
Events
Socket.io Socket.io
client server

...

60

Alternatively: multiple front-end servers

Node JS Service 1

Browser

HTML 5 Node JS Service 2
Application

Node JS ...

Needs single-sign-on
61

NodeJS - using RESTful WS and AMQP

REST
Service

Requests

Node JS

Events
socket.io
AMQP AMQP
RabbitMQ Service

62

Updating the UI is easy
§ Update the UI independently of rest of system
§ Easily run A/B tests
§ Enables fast iteration of the UI

http://theleanstartup.com/principles
63

But coordination with backend changes required
§ Let’s imagine that you are deploying an advanced search
feature:
• Enhancements to search service
• Enhancements to UI
§ Before
• Deploy new war
§ Now:
• Some coordination required
• Deploy updated backend service
• Deploy updated NodeJS and browser code
• Enable feature using feature switch

64

Agenda

65

The challenge
§ In the beginning:
• It would be easy to use Y axis/functional decomposition
• But you don’t need it and it would slow you down
§ Later on:
• You do need it
• But your application logic is tangled and decomposition is
difficult

66

Lower cost way to start with Y axis decomposition
§ Enforce boundaries between services, e.g. DTOs
§ But run in a single JVM with a single database
§ Use Spring Integration
• Course-grained message-based communication
• Via in-memory channels
§ Downside:
• Implementing the glue layers are extra work

67

Special case: Components that don’t cluster

Application

Monitoring Rest of domain classes

A
C

B

D
Esper

Tangled dependencies prevent Y axis scaling

Doesn’t cluster No X axis scaling/cloning 68

Don’t let this happen!

Use well-defined interfaces

Ensure loose coupling

69

Traditional application architecture
Monolithic application

Service A Service B Service C

Shared domain model

A C
Difﬁcult to
untangle

B

X Z

Y

70

Extracting a service = untangling dependencies
Application Service

Service C
Service A Service B

Trouble!
Domain model 1 Domain model 2

X A C

Y B

Z

71

Untangling dependencies
§ Domain object is part of an association class A {
• Read only - replicate private B other;
• Update only - notification interface }

• Update then read - replicate
§ Domain object is a request parameter class A {
• Read-only - replace with DTO void m(B other) {
...
• Update only - notification interface }
• Update then read - tricky }

§ Note - Need acyclic dependencies
• If used object isn’t part of a service = assumption is that it
belongs in calling service

72

Bounded context is a useful idea
§ Different parts of a system different domain models
§ Different services have a different view of some domain
objects, e.g.
• User Management = complex view of user
• Rest of application = PK + ACL + Name

§ Different services can have a different domain model
§ Services exchange messages to synchronize data

73

Untangling dependencies 1


X A C

doSomething(Y)

doSomethingElse()

Y B
getSomething()
Z

74


doSomething(YDTO) C
X A
Facade A

doSomethingElse()
Y B

Facade B
Z

75


C
doSomething(YDTO)
X Facade A A

Message
Broker Listener
Y
Listener B

doSomethingElse()
Z Facade B

76

Agenda

77

Traditional tools: monolithic applications

78

Developing service-oriented apps is harder
§ Many more moving parts to manage
• Infrastructure services: SQL, NoSQL, RabbitMQ
• Application services
§ Who is going to setup the environments:
• the developer sandbox?
• ...
• QA environments?

79

Cloud Foundry makes it easier
§ Easy to deploy applications
§ Easy to provision services
§ Manifest describes the structure of a set of services
§ vmc env-add for binding 'app services' together
§ Micro Cloud Foundry is your personal cloud/sandbox
§ Caldecott exposes services for use by integration tests

80

Services available on Cloud Foundry

81

Multi-application manifest - part 1
---
applications:
inventory/target: Path to application
name: inventory
url: cer-inventory.chrisr.cloudfoundry.me
framework:
name: spring
info:
mem: 512M
description: Java SpringSource Spring Application
exec:
mem: 512M
instances: 1
services:
si-rabbit: Required services
type: :rabbitmq
si-mongo:
type: :mongodb
si-redis:
type: :redis
83

Multi-application manifest - part 2
store/target: Path to application
name: store
url: cer-store.chrisr.cloudfoundry.me
framework:
name: spring
info:
mem: 512M
description: Java SpringSource Spring Application
exec:
mem: 512M
instances: 1
services: Required services
si-mongo:
type: :mongodb
si-rabbit:
type: :rabbitmq
84

One command to create services and deploy application
$ vmc push
Would you like to deploy from the current directory? [Yn]:
Pushing application 'inventory'...
Creating Application: OK
Creating Service [si-rabbit]: OK
Binding Service [si-rabbit]: OK
Creating Service [si-mongo]: OK
Binding Service [si-mongo]: OK
Creating Service [si-redis]: OK
Binding Service [si-redis]: OK
Uploading Application:
Checking for available resources: OK
Processing resources: OK
Packing application: OK
Uploading (12K): OK
vmc push:
Push Status: OK •Reads the manifest ﬁle
•Creates the required services
Staging Application 'inventory': OK
Starting Application 'inventory': OK
Pushing application 'store'...
Creating Application: OK •Deploys all the applications
Binding Service [si-mongo]: OK
Binding Service [si-rabbit]: OK
Processing resources: OK
Uploading (5K): OK
Push Status: OK
Staging Application 'store': OK
85
Starting Application 'store': ...

Application configuration via environment variables

$ vmc env-add cf1
PAYMENT_SVC=http://...
Adding Environment Variable
[PAYMENT_SVC=http://...]: OK
Stopping Application: OK
Staging Application: OK

String value = System.getenv("PAYMENT_SVC")

@Value("#{systemEnvironment['PAYMENT_SVC']}")
private String envVariable;

Micro Cloud Foundry: new developer sandbox

App Instances Services

Open source Platform as a Service project

10.04

A PaaS packaged as a VMware Virtual Machine

Use as a developer sandbox
• Use the services from Junit integration tests
• Deploy your application for functional testing
• Remote debugging from STS
87

Using Caldecott…
$ vmc tunnel
1: mysql-135e0
2: mysql1
Which service to tunnel to?: 2
Password: ********
Stopping Application: OK
Redeploying tunnel application 'caldecott'.
Uploading (1K): OK
Push Status: OK
Binding Service [mysql1]: OK
Staging Application: OK
Starting Application: OK
Getting tunnel connection info: OK

Service connection info:
username : uMe6Apgw00AhS
password : pKcD76PcZR7GZ
name : d7cb8afb52f084f3d9bdc269e7d99ab50

Starting tunnel to mysql1 on port 10000.
1: none
2: mysql
Which client would you like to start?: 2

…Using Caldecott
Launching 'mysql --protocol=TCP --host=localhost --port=10000 --
user=uMe6Apgw00AhS --password=pKcD76PcZR7GZ
d7cb8afb52f084f3d9bdc269e7d99ab50'

Welcome to the MySQL monitor. Commands end with ; or g.
Your MySQL connection id is 10944342
Server version: 5.1.54-rel12.5 Percona Server with XtraDB (GPL),
Release 12.5, Revision 188

Copyright (c) 2000, 2011, Oracle and/or its affiliates. All rights
reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or 'h' for help. Type 'c' to clear the current input
statement.

mysql>

Running JUnit test with Caldecott

Conﬁgure your test code to use port + connection info

90

Summary
§ Monolithic applications are simple to develop and deploy

BUT applying the scale cube

§ Decomposes your application into services
§ Enables scaling for transactions and data volumes
§ Tackles application complexity
§ Enables scaling for development
§ Enables frequent, independent deployments
§ Make it easy to leverage other technologies

AND

§ Cloud Foundry simplifies the development and deployment of
“service-oriented” applications
91

@crichardson crichardson@vmware.com

Questions?
www.cloudfoundry.com promo code CFOpenTour2012

Decomposing Applications for Scalability and Deployability (April 2012)

Recomendados

Recomendados

Más contenido relacionado

La actualidad más candente

La actualidad más candente (20)

Similar a Decomposing Applications for Scalability and Deployability (April 2012)

Similar a Decomposing Applications for Scalability and Deployability (April 2012) (20)

Más de Chris Richardson

Más de Chris Richardson (20)

Último

Último (20)

Decomposing Applications for Scalability and Deployability (April 2012)