Presentation: "What has SoundCloud learnt about Microservices?" Track: Monoliths & Microservices / Time: Friday 11:30 - 12:20 / Location: Hall 10 SoundCloud is the largest repository of audio on the web, used by more than 200 million people every month who upload more than 11 hours of audio every minute. Over the past few years we migrated from a typical monolithic architecture to microservices, and during this journey we found out what is real and what is hype about this buzzword. When does "the best tool for the job" break? What is available as open-source software and what will you have to build yourself? How to move from several deploys a day from dozens of deploys of dozens of different apps and keep sanity? How to deal with shared code bases? How does one even test all these moving parts? How big should a microservice be, anyway? In this talk, let's discuss the results of our discovery phase and what technologies, architecture patterns and processes we have adopted as our default toolbox. PHIL CALCADO, SOUNDCLOUD Phil Calcado Biography: Phil Calcado Phil Calçado is a Director of Engineering at SoundCloud, where he writes code and helps teams build a scalable and maintainable service. Before moving to Berlin, he spent four years as a Lead Consultant for ThoughtWorks, where he helped clients build systems in whatever crazy stack they had decided to use. Twitter: @pcalcado Blog: philcalcado.com http://gotocon.com/berlin-2014/presentation/What%20has%20SoundCloud%20learnt%20about%20Microservices?

1. What has SoundCloud
learnt about
microservices?
Phil Calçado
SoundCloud

4. >11 hours of
audio uploaded
every minute

5. ~300
million people
every month

7. What we
2012-13
did in

8. SoundCloud.com

9. Sounds ˝
& Sets
Social
Graph
Activity
Stream
Premium ˝
Features Search
API

10. Sounds ˝
& Sets
Social
Graph
Activity
Stream
API
Premium ˝
Features Search

11. api.soundcloud.com
iOS Android Desktop Widget 3rd Party

12. api-mobile
api web api-partn
iOS Android Desktop Widget
3rd Party

13. More details
http://bit.ly/dealing-with-the-monolith

14. minimise
fixed cost
your
per app
#1

15. every new app answers
What language?
What I/O lib?
What build system?
How do we deploy?
What do we monitor?

16. typical agile project
iteration 0 iteration 1 iteration 2 iteration 3 iteration technical tasks
user stories

17. typical microservices project
iteration 0 iteration 1 iteration 2 iteration 3 iteration

18. You will have lots of
microservices

19. Consider having a
“standard”, well-supported
and
documented stack

20. outsource
as possible
as much
#2

21. micro service x-ray
app
infrastructure

22. Avoid building as much ˝
infrastructure as possible

23. some good options

24. Only infrastructure we
need to build is the glue
between the lib and our
quirks

25. Avoid having quirks

26. acknowledge
complexity
the new
#3

27. user
likes
track

28. typical app
user
likes
track
unit test
single process
run inside an IDE

29. microservice
user
likes
track
integration test
multiple processes
stubs and fakes

30. must be easy to provision
+ +

31. make
visible
everything
#4

33. make interactions visible

34. standardised dashboards

35. wrap your
legacy
with
stranglers
#5

36. api-mobile
api web api-partn
iOS Android Desktop Widget
3rd Party
not exactly true

37. mothership still alive
api.soundcloud.com
api web api-partn
iOS Android Desktop Widget 3rd Party

39. api.soundcloud.com
Client

40. api.soundcloud.com
strangler
Client

41. api.soundcloud.com
strangler
Client

42. api.soundcloud.com
strangler
Client

43. what changed
changed
since
2013?

45. Stop
babysitting
Rails
☑

46. Enable &
empower☑
app devs

47. “Experience
based” ☒
api?

48. What
comes
next?

49. more
devices

50. IDL
based
RPC?

51. ephemeral
test
environments

53. phil calçado
http://philcalcado.com˝
@pcalcado˝
www.soundcloud.com˝