Presented by Jon Eaves (REA), Beth Skurrie (Dius) and Evan Bottcher (ThoughtWorks) at Yow! conference December 2014.

1. From monoliths to microservices

2. @bethesque @evanbottcher @joneaves

3. Why?

4. Why? What?

5. Why? What? How?

6. Residential Commercial Media &
Developer International
Group
Global Architecture
Property
The Money
People & Experiences

7. Why?

8. 1. Long release cycle

9. 1. Long release cycle
2. Coupled releases

10. 1. Long release cycle
2. Coupled releases
3. Long defect fix time

12. Teams at REA are;
self-empowered

13. Teams at REA are;
self-empowered
value independence

14. Teams at REA are;
self-empowered
value independence
trusted

15. Teams at REA are;
self-empowered
value independence
trusted

16. Vision

17. rapid deployment

18. rapid deployment
fault tolerant

19. rapid deployment
fault tolerant
discoverable

20. rapid deployment
fault tolerant
discoverable
independent

21. rapid deployment
fault tolerant
discoverable
independent

22. Microservices
?

23. Microservices
1. Narrow responsibility

24. Microservices
1. Narrow responsibility
2. Small codebase

25. Microservices
1. Narrow responsibility
2. Small codebase
3. There is no #3

26. Starting the journey

27. What?

28. Residential Commercial Media &
Developer International
Group

29. Group

30. (source: flickr)

31. (source: flickr)

32. CRM ERP
Customer
Relationship
Management
Enterprise
Resource
Planning

33. CRM ERP

34. Billing Payment
CRM ERP

35. Billing Payment
CRM ERP

36. Billing
Contract
Sales automation
Product rules
Pricing
Finance

38. Business Systems

39. Line of
Business
Line of
Business
Line of
Business
Customers

40. Line of
Business
Line of
Business
Line of
Business
Customers
Ouch!

42. Billing
Contract
Sales automation
Product rules
Pricing
Finance

43. Billing
Sales automation
Product rules
Pricing
Finance
Contract

44. Billing
Product rules
Contract

45. Billing
Product rules
Contract
New
Apps
New
Apps

46. Billing
Product rules
Contract
New
Apps
New
Apps
New team boundaries?

47. (source: wikimedia)

48. Strategic Goals
Architectural
Principles
Design and Delivery
Practices
Enable scalable
business
More customers/transactions
Self-service for customers
Support entry into
new markets
Flexible operational processes
New products and operational
processes
Support innovation
in existing markets
Flexible operational processes
New products and operational
processes
Reduce inertia
Make choices that favour rapid
feedback and change, with reduced
dependencies across teams.
Eliminate accidental
complexity
Aggressively retire and replace
unnecessarily complex processes,
systems, and integrations so that we
can focus on the essential
complexity.
Consistent interfaces
and data flows
Eliminate duplication of data and
create clear systems of record, with
consistent integration interfaces.
No silver bullets
Off the shelf solutions deliver early
value but create inertia and
complexity.
Standard REST/HTTP
Encapsulate legacy
Eliminate integration
databases
Consolidate and
cleanse data
Published integration
model
Small independent
Services
Continuous
deployment
Minimal customisation
of COTS/SAAS

49. Strategic Goals
Architectural
Principles
Design and Delivery
Practices
Enable scalable
business
More customers/transactions
Self-service for customers
Support entry into
new markets
Flexible operational processes
New products and operational
processes
Support innovation
in existing markets
Flexible operational processes
New products and operational
processes
Reduce inertia
Make choices that favour rapid
feedback and change, with reduced
dependencies across teams.
Eliminate accidental
complexity
Aggressively retire and replace
unnecessarily complex processes,
systems, and integrations so that we
can focus on the essential
complexity.
Consistent interfaces
and data flows
Eliminate duplication of data and
create clear systems of record, with
consistent integration interfaces.
No silver bullets
Off the shelf solutions deliver early
value but create inertia and
complexity.
Standard REST/HTTP
Encapsulate legacy
Eliminate integration
databases
Consolidate and
cleanse data
Published integration
model
Small independent
Services
Continuous
deployment
Minimal customisation
of COTS/SAAS

50. Martin’s first law of distributed objects
Don’t distribute!

51. Single service maintainability
Fit-for-purpose technology choice Integration Complexity
Environment Provisioning
Operational Complexity
Testing Complexity
Microservices Monoliths

52. How?

53. Netscaler configs. Firewall rules. Name all the things consistently.
Configured package repository. Cron jobs. Service healthcheck
Run the specs. Run code hygiene. Keep all the gems up to Configure monitoring. Enable NRPE. Configure apache.
Build an RPM. Configure dependencies. Ruby version. Configure
Passenger configs. Logrotate. Splunk forwarder. Logger
Create users. Configure NewRelic. Load rake tasks. Load
Connect to a database. Make connection pooling work.
Externalise configuration. Run database migrations. Chkconfigs.

54. #noops

55. (source: wikimedia)

57. (source: flickr.com)
#devops

58. If you build it, you support it

59. Integration Complexity
Environment Provisioning
Operational Complexity
Testing Complexity
Single service maintainability
Fit-for-purpose technology choice
Team managed infrastructure
If you build it, you support it
Microservices Monoliths

60. Time
# of Services
3 months

61. Time
# of Services
3 months

62. Time
# of Services
3 months
2 services!

63. Condor
Pricing
Contract
Proposals
Account
Manager
Assignments

64. Condor
Contract
Proposals
Pricing
Account
Manager
Assignments

65. Condor
Customer
Acceptance
Accepted
Contracts
Adapter
Contracts CDM
Contract
Proposals
Pricing
Account
Manager
Assignments

66. Condor
Customer
Acceptance
Accepted
Contracts
Adapter
Contracts CDM
Contract
Proposals
Pricing
Account
Manager
Assignments

67. Condor
Customer
Acceptance
Accepted
Contracts
Adapter
Contracts CDM
Contract
Proposals
Pricing
Account
Manager
Assignments
Anti-corruption

68. (source: flickr.om)

69. ‘Stencil’
(Service Template)

70. Stencil

71. Stencil Service
Clone

72. Service
Service
Stencil Service
Service
Service
Service

73. Service
Service
Stencil Service
Service
Service
Service

74. Service
Service
Stencil Service
Service
Service
Service
Patch

75. Harvest a service template
(but don’t start by writing one)

76. Success brings its
own problems

77. Test Prod

78. Prod
Test
Test
Test

79. Prod
Test
Test
Test

81. Slow Easy to break
Hard to fix Scales BADLY
Integration Tests
Lots of set up False negatives
Lots of infrastructure

82. Why do we do
integration testing?

83. Why do we do
integration testing?

84. Why do we do integration testing?

85. Why do we do Bugs
integration testing?

86. Why do we do Bugs Consequences
integration testing?

87. Bugs Consequences Time Don’t

88. Consequences Time Don’t know

89. Time Don’t know Monitoring

90. Time Don’t know Monitoring

91. Time Don’t know Monitoring
Communication

92. Time Don’t know Monitoring
Communication
Cause

93. Time Don’t know Monitoring
Communication
Cause Logging

94. Time Don’t know Monitoring
Communication
Cause Logging
Production

95. Time Don’t know Monitoring
Communication
Cause Logging
Production Approval process

96. Time Don’t know Monitoring
Communication
Cause Logging
Production Approval process
Integration tests

97. Monitoring
Communication
Logging
Approval process
Integration tests

98. Monitoring
Communication
Logging
Approval process
Integration tests
✓
✓
✓
✓

99. If you build it,
you support it

100. Monitor meaningful
things

101. Just deploy it

102. Throw away
integration tests

103. The problem with mocks
???
CA BP
???
mock

104. Test symmetry
CA BP
mock

105. Test symmetry
CA BP
mock

106. Pact… how it works
C P
mock

107. Pact… how it works
C P
mock

108. Pact… how it works
C P
mock
pact

109. Pact… how it works
C P
mock
pact

110. Pact… how it works
C P
mock
pact

111. Slow Easy to break
Hard to fix Scales BADLY
Integration Tests
Lots of set up False negatives
Lots of infrastructure

112. Fast Stable
Easy to debug Reliable
Pact
Scales linearly Low setup
No extra infrastructure

113. Arrange - set up mock server

114. Arrange - set up mock server

115. Arrange - set up mock server

116. Arrange - set up expectations

117. Arrange - set up expectations

118. Arrange - set up expectations

119. Act and Assert

120. Act and Assert

122. Act and Assert

123. Act and Assert

124. Act and Assert

125. Set up test data

126. Set up test data
Consumer assumed:

127. Set up test data
Consumer assumed:
Provider complies:

128. Set up test data
Consumer assumed:
Provider complies:

129. Verify: Fail

130. Verify: Success!

131. https://github.com/realestate-com-au/pact
@pact_up

132. Fast feedback in isolation
+
Early detection
+
Rapid recovery

133. Single service maintainability
Fit-for-purpose technology choice
Team managed infrastructure
If you build it, you support it
Service Template
Consumer Driven Contracts
Early detection
Rapid Recovery
Integration Complexity
Environment Provisioning
Operational Complexity
Testing Complexity
Microservices Monoliths

134. Time
# of Services
9 months
15 services

135. One
‘Poppenservice’
= Two weeks
(source: flickr.com)

136. To the Cloud!

137. (source: wikimedia)

139. Tribe

140. Tribe
Squad Squad
Delivery
Engineering
Squad

141. “We aim to remove hurdles … and ensure
the whole team has the knowledge to
responsibly operate the system.”
- Hydra’s Vision

142. Single service maintainability
Fit-for-purpose technology choice
Team managed infrastructure++
If you build it, you support it
Service Template
Consumer Driven Contracts
Early detection (MTTD)
Rapid Recovery
Integration Complexity
Environment Provisioning
Operational Complexity
Testing Complexity
Microservices Monoliths

143. One ‘Poppenservice’
= Two hours
(source: flickr.com)

144. Time
# of Services
2 years
60 services

145. (source: flickr.com) Work in Progress

146. Sharing things is hard

147. Service

148. Service Service

149. Service Service Service

150. Service Service Service
Library

151. Service Service Service Service
Service
Library

152. Service Service Service Service Service
Service Service Service Service Service
Service Service Service Service Service
Service Service Service Service
Service
Library

153. Service Service Service Service Service
Service Service Service Service Service
Service Service Service Service Service
Service Service Service Service
Service
Library

154. Sharing == Coupling

157. All of REA
My Tribe
My Squad

158. Be cautious about
shared stuff
(copy/paste may not be wrong)

159. Service
Custodians

160. Establish Service
Custodianship*
(*it’s REALLY hard to make work)

161. Why?

162. Why? What?

163. Why? What? How?

164. Caution.

165. Microservices is a long term
Time
# of Services
strategy

166. If you build it, you support it
Harvest a service template
Early detection of failures
Microservices is a long term
strategy
Fast feedback in isolation
Rapid Recovery
Delivery Engineering
Deliberate sharing
Service Custodianship

167. Thank You!
@bethesque @evanbottcher @joneaves