Developing applications with a microservice architecture (SVforum, microservices meetup)

1. Microservices: Decomposing @crichardson Applications for Deployability and Scalability Chris Richardson Author of POJOs in Action Founder of the original CloudFoundry.com @crichardson chris@chrisrichardson.net http://plainoldobjects.com

2. Presentation goal How decomposing applications into @crichardson microservices improves deployability and scalability and simplifies the adoption of new technologies

3. @crichardson About Chris

4. @crichardson About Chris Founder of a buzzword compliant (stealthy, social, mobile, big data, machine learning, ...) startup Consultant helping organizations improve how they architect and deploy applications using cloud computing, micro services, polyglot applications, NoSQL, ...

5. @crichardson Agenda The (sometimes evil) monolith Decomposing applications into services Client ⇔ service interaction design Decentralized data management

6. @crichardson Let’s imagine you are building an online store

7. @crichardson Traditional application architecture StoreFrontUI Product Info Service Recommendation Service Tomcat Browser/ Client WAR/EAR MySQL Database Review Service Simple to develop test deploy Load balancer scale Spring MVC Spring Hibernate Order Service HTML REST/JSON

8. But large, complex, monolithic @crichardson applications ⇒ problems

9. @crichardson Intimidates developers

10. @crichardson Obstacle to frequent deployments Need to redeploy everything to change one component Interrupts long running background (e.g. Quartz) jobs Increases risk of failure Fear of change Updates will happen less often - really long QA cycles e.g. Makes A/B testing UI really difficult Eggs in one basket

11. @crichardson Overloads your IDE and container Slows down development

12. @crichardson Obstacle to scaling development I want to update the UI But the backend is not working yet! Lots of coordination and communication required

13. Requires long-term commitment to a technology stack @crichardson

15. @crichardson

16. @crichardson The scale cube X axis - horizontal duplication Z axis - data partitioning Y axis - functional decomposition similar splitting by things Scale Scale by splitting different things

17. @crichardson Y-axis scaling - application level WAR Storefront UI Product Info Service Recommendation Service Review Service Order Service

18. @crichardson Y-axis scaling - application level Storefront UI Product Info Service Recommendation Service Review Service Order Service

19. @crichardson Y-axis scaling - application level Product Info Service Product Info Recommendation Service Review Service Order Service Browse Products UI Checkout UI Order management UI Account management UI Apply X-axis and Z-axis scaling to each service independently

20. Service deployment options @crichardson Isolation, manageability VM or Physical Machine Docker/Linux container JVM JAR/WAR/OSGI bundle/... Density/efficiency

21. @crichardson Partitioning strategies... Partition by noun, e.g. product info service Partition by verb, e.g. Checkout UI Single Responsibility Principle Unix utilities - do one focussed thing well

22. @crichardson Partitioning strategies Too few Drawbacks of the monolithic architecture Too many - a.k.a. Nano-service anti-pattern Runtime overhead Potential risk of excessive network hops Potentially difficult to understand system Something of an art

23. @crichardson Example micro-service

24. @crichardson More service, less micro But more realistically... Focus on building services that make development and deployment easier - not just tiny services

25. @crichardson Real world examples http://techblog.netflix.com/ ~600 services http://highscalability.com/amazon-architecture 100-150 services to build a page http://www.addsimplicity.com/downloads/ eBaySDForum2006-11-29.pdf http://queue.acm.org/detail.cfm?id=1394128

26. @crichardson There are many benefits

27. @crichardson Smaller, simpler apps Easier to understand and develop Less jar/classpath hell - who needs OSGI? Faster to build and deploy Reduced startup time - important for GAE

28. Scales development: develop, deploy and scale each service independently @crichardson

29. @crichardson Improves fault isolation

30. Eliminates long-term commitment to a single technology stack @crichardson Modular, polyglot, multi-framework applications

31. @crichardson Two levels of architecture System-level Services Inter-service glue: interfaces and communication mechanisms Slow changing Service-level Internal architecture of each service Each service could use a different technology stack Pick the best tool for the job Rapidly evolving

32. Easily try other technologies @crichardson ... and fail safely

33. But there are drawbacks @crichardson

34. @crichardson Complexity

35. Complexity of developing a @crichardson distributed system http://highscalability.com/blog/2014/4/8/microservices-not-a-free-lunch.html

36. @crichardson Multiple databases & Transaction management e.g. Fun with eventual consistency

37. @crichardson Complexity of testing a distributed system http://highscalability.com/blog/2014/4/8/microservices-not-a-free-lunch.html

38. Complexity of deploying and operating a distributed @crichardson system http://highscalability.com/blog/2014/4/8/microservices-not-a-free-lunch.html You need a lot of automation

39. @crichardson Developing and deploying features that span multiple services requires careful coordination

40. @crichardson When to use it? In the beginning: •You don’t need it •It will slow you down Later on: •You need it •Refactoring is painful

42. Let’s imagine that you want to display a product’s details... Product Info Reviews Recommendations @crichardson

43. Directly connecting the front-end to the backend @crichardson View Controller Model Product Info service Recommendation Service Review service REST REST AMQP Traditional server-side web application View Controller Model Browser/Native App Chatty API Web unfriendly protocols

44. @crichardson Use an API gateway View Controller Model Product Info service Recommendation Service Review service REST REST AMQP API Gateway View Controller Model Browser/Native App Single entry point Client specific APIs Protocol translation Traditional server-side web application

45. @crichardson Optimized client-specific APIs Web application Mobile App NodeJS API Gateway REST proxy Event publishing Product Info service Recommendation Service Review service REST REST AMQP getProductInfo() getRecomm...() getReviews() getProductDetails()

46. @crichardson Netflix API Gateway http://techblog.netflix.com/2013/01/optimizing-netflix-api.html Device specific end points

47. @crichardson API gateway design challenges Performance and scalability Non-blocking I/O Asynchronous, concurrent code Handling partial failures .... http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html

48. @crichardson Useful frameworks for building an API gateway JVM: Netty, Vertex Netflix Hystrix ... Other: NodeJS

49. How does a browser interact with the partitioned @crichardson web application?

50. @crichardson Partitioned web app ⇒ no longer a single base URL /products Browse Products UI /checkout Checkout UI /orders Order management UI /account Account management UI Browser ?

51. The solution: single entry point that routes based on URL @crichardson /products Browse Products UI /checkout Checkout UI /orders Order management UI /account Account management UI http://acme.com/<service>/... Content Browser Router Hidden from browser Single entry point

52. @crichardson How do the services communicate?

53. Inter-service communication options @crichardson Synchronous HTTP ⇔ asynchronous AMQP Formats: JSON, XML, Protocol Buffers, Thrift, ... Asynchronous is preferred JSON is fashionable but binary format is more efficient

54. Pros and cons of messaging Pros Cons Decouples client from Additional complexity of server message broker Message broker buffers Request/reply-style messages communication is more Supports a variety of complex communication patterns

55. Pros and cons of HTTP Pros Simple and familiar Request/reply is easy Firewall friendly No intermediate broker Cons Only supports request/ reply Server must be available Client needs to discover URL(s) of server(s)

56. @crichardson Discovery option #1: Internal load balancer Load Balancer Product Info Service Product Info Service Product Info Service Product Info Service Client/ API gateway Services register with load balancer Client talks to load balancer Has a well-known location http://docs.aws.amazon.com/ElasticLoadBalancing/latest/DeveloperGuide/USVPC_creating_basic_lb.html

57. @crichardson Discovery option #2: client-side load balancing REST Client Product Info Service Product Info Service Product Info Service Product Info Service Client Service Registry Services register with registry Client polls registry http://techblog.netflix.com/2013/01/announcing-ribbon-tying-netflix-mid.html http://techblog.netflix.com/2012/09/eureka.html

58. @crichardson Lots of moving parts! PProrodducutc tIn Ifnofo Service Recommendation Service Review Service Order Service Browse Products UI Checkout UI Order management UI Account management UI API Gate way Service registry Content Router HTML Browser REST Client Ext. LB Ext. LB

60. @crichardson Decomposed services ⇒ decomposed databases Order management Customer management Order Database Customer Database Separate databases ⇒ less coupling

61. @crichardson Decomposed databases ⇒ polyglot persistence IEEE Software Sept/October 2010 - Debasish Ghosh / Twitter @debasishg

62. Untangling orders and customers Customer management @crichardson Order Service placeOrder() Order management Customer Service availableCredit() updateCustomer() Customer creditLimit ... Order belongs to has orders total Invariant: sum(order.total) <= creditLimit available credit= creditLimit - Trouble! sum(order.total)

63. @crichardson Problems Reads Service A needs to read data owned by service B Updates Transaction must update data owned by multiple services

64. Handling reads: requesting credit limit @crichardson placeOrder() Order management getCreditLimit() Customer management

65. @crichardson Pulling data Benefits Simple to implement Ensures data is fresh Drawbacks Reduces availability Increases response time

66. Customer management @crichardson Handling reads: replicating the credit limit Order Service placeOrder() sum(order.total) <= Order management Customer creditLimit ... Order total Customer’ creditLimit creditLimit changeCreditLimit() Customer Service updateCustomer() Simplified

67. Useful idea: Bounded context @crichardson Different services have a different view of a domain object, e.g. User Management = complex view of user Rest of application: User = PK + ACL + Name Different services can have a different domain model

68. @crichardson Replicating data Benefits Improved availability for reads Improves latency Drawbacks Additional complexity of replication mechanism

69. How to update distributed (and replicated) data? @crichardson

70. Use distributed transactions @crichardson Benefits Simple development Guarantees consistency Drawbacks Operationally complex Reduced availability

71. @crichardson Use event-driven architecture How Services publish events when data changes Subscribing services update their data Benefits: Simpler Better availability Drawbacks: Application has to handle eventually consistent data Application has to handle duplicate events

72. How do services publish @crichardson events?

73. @crichardson To maintain consistency the application must atomically publish an event whenever a domain object changes

74. @crichardson How to generate events? Database triggers Hibernate event listener Ad hoc event publishing code mixed into business logic Domain events - “formal” modeling of events Event Sourcing

75. Atomically publishing events Use distributed transactions to update database and publish to message Database and message broker must support 2PC 2PC is best avoided Use two step eventual consistency mechanism: 1. Update database: new entity state & intent to publish event 2. Publish event & mark event as published • Difficult to implement when using a NoSQL database :-( @crichardson

76. @crichardson Event sourcing An event-centric approach to designing domain models Request becomes a command that is sent to an Aggregate Aggregates handle commands by generating events Apply events to an aggregate to update state Persist events NOT state Replay events to recreate the current state of an aggregate Event Store ≃ database + message broker

77. Request handling in an event-sourced application @crichardson HTTP Handler Event Store pastEvents = findEvents(entityId) Account new() applyEvents(pastEvents) newEvents = processCmd(SomeCmd) saveEvents(newEvents) Microservice A

78. @crichardson Event Store publishes events - consumed by other services Event Store Microservice B Event Subscriber subscribe(EventTypes) publish(event) publish(event) Aggregate NoSQL materialized view update() update()

79. UpdateCreditLimitCommand @crichardson Using event sourcing Customer Service updateCustomer() CustomerCreditLimitUpdatedEvent Event Store Order management Order total Customer’ creditLimit CustomerCreditLimitUpdatedEvent(...) Customer management Customer creditLimit ...

80. @crichardson Customer aggregate case class Customer(customerId: String, creditLimit: BigDecimal) extends ValidatingAggregate[Customer, CustomerCommands.CustomerCommand] { def this() = this(null, null) override def processCommand = { case CreateCustomerCommand(customerId, creditLimit) => Seq(CustomerCreatedEvent(customerId, creditLimit)) case UpdateCreditLimitCommand(newLimit) if newLimit >= 0 => Seq(CustomerCreditLimitUpdatedEvent(newLimit)) } override def applyEvent = { case CustomerCreatedEvent(customerId, creditLimit) => copy(customerId=customerId, creditLimit=creditLimit) case CustomerCreditLimitUpdatedEvent(newLimit) => copy(creditLimit=newLimit) } } Command ⇒ Events Event ⇒ Updated state

81. @crichardson EventStore API trait EventStore { def save[T](entityId: Id, events: Seq[Event]): T def update[T](entityId: Id, version: EntityVersion, events: Seq[Event]): T def load[T](entityType: Class[T], entityId: EntityId): T def subscribe(...) : ... .. }

82. Unfamiliar but it solves many problems @crichardson Eliminates O/R mapping problem Supports both SQL and NoSQL databases Publishes events reliably Reliable eventual consistency framework ...

83. But let’s imagine that you want to display an account and it’s recent transactions... @crichardson

84. Displaying balance + recent transactions We need to do a “join: between the Account and the corresponding TransferTransactions (Assuming Debit/Credit events don’t include other account, ...) @crichardson BUT Event Store = primary key lookup of individual aggregates, ... ⇒ Use Command Query Responsibility Separation Define separate “materialized” query-side views that implement those queries

85. HTTP GET Request View Query Service @crichardson Query-side microservices Updater - microservice View Updater Service Events Event Store Reader - microservice View Store e.g. MongoDB Neo4J CloudSearch update query

86. @crichardson Agenda The (sometimes evil) monolith Decomposing applications into services Client ⇔ service interaction design Decentralized data management Bonus feature: microservices in action

87. API gateway implemented @crichardson using NodeJS

88. Spring Boot provides opinionated Spring application architecture Simplifies applications using Convention over Configuration Packages application as standalone, executable jar Provides production-ready features externalized configuration and health checks @crichardson Microservices built with Spring Boot For more on Spring Boot + Microservices see http://bit.ly/bootmicro1

89. @crichardson Event-driven architecture Business logic implemented using event sourcing-based domain model Views implemented using DynamoDB Cloud Search (text search)

90. @crichardson Jenkins-based deployment pipeline Build & Test micro-service Build & Test Docker image Deploy Docker image to Repository One pipeline per micro-service

91. @crichardson Deployed on Amazon EC2

92. @crichardson Summary

93. Monolithic applications are simple to develop and deploy @crichardson BUT have significant drawbacks

94. @crichardson Apply the scale cube Modular, polyglot, and scalable applications Services developed, deployed and scaled independently

95. Use a modular, polyglot architecture View Controller Product Info @crichardson Model service Recommendation Service Review service REST REST AMQP API Gateway Server-side web application View Controller Model Browser/Native application

96. @crichardson Start refactoring your monolith Monolith Anti-corruption Service layer Glue code Pristine

97. @crichardson chris@chrisrichardson.net @crichardson Questions? http://plainoldobjects.com http://microservices.io