Video and slides synchronized, mp3 and slide download available at URL https://bit.ly/2FRILKh. Aris Koliopoulos talks about how common problems in social media, such as analysing user sessions, counting big numbers and making real-time recommendations can be resolved with a healthy mix of stream processing and functional programming. Filmed at qconlondon.com. Aris Koliopoulos is Backend Engineer at Drivetribe.

1. A SOCIAL NETWORK ON STREAMS
DRIVETRIBE ENGINEERING

2. InfoQ.com: News & Community Site
Watch the video with slide
synchronization on InfoQ.com!
https://www.infoq.com/presentations/
drivetribe
• Over 1,000,000 software developers, architects and CTOs read the site world-
wide every month
• 250,000 senior developers subscribe to our weekly newsletter
• Published in 4 languages (English, Chinese, Japanese and Brazilian
Portuguese)
• Post content from our QCon conferences
• 2 dedicated podcast channels: The InfoQ Podcast, with a focus on
Architecture and The Engineering Culture Podcast, with a focus on building
• 96 deep dives on innovative topics packed as downloadable emags and
minibooks
• Over 40 new content items per week

3. Purpose of QCon
- to empower software development by facilitating the spread of
knowledge and innovation
Strategy
- practitioner-driven conference designed for YOU: influencers of
change and innovation in your teams
- speakers and topics driving the evolution and innovation
- connecting and catalyzing the influencers and innovators
Highlights
- attended by more than 12,000 delegates since 2007
- held in 9 cities worldwide
Presented at QCon London
www.qconlondon.com

4. DRIVETRIBE
DRIVETRIBE
▸ The world biggest motoring
community.
▸ A social platform for petrolheads.
▸ By Clarkson, Hammond and May.
2

5. DRIVETRIBE
DRIVETRIBE
▸ A content destination at the core.
▸ Users consume feeds of content:
images, videos, long-form articles.
▸ Content is organised in
homogenous categories called
“tribes”.
▸ Different users have different
interests and the tribe model allows
to mix and match at will.
3

6. DRIVETRIBE
DRIVETRIBE ARTICLE
▸ Single article by James May.
▸ Contains a plethora of content and
engagement information.
▸ What do we need to compute an
aggregate like this?
4

7. DRIVETRIBE
DRIVETRIBE ARTICLE
▸ getUser(id: Id[User])
5

8. DRIVETRIBE
DRIVETRIBE ARTICLE
▸ getUser(id: Id[User])
▸ getTribe(id: Id[Tribe])
6

9. DRIVETRIBE
DRIVETRIBE ARTICLE
▸ getUser(id: Id[User])
▸ getTribe(id: Id[Tribe])
▸ getArticle(id: Id[Article])
7

10. DRIVETRIBE
DRIVETRIBE ARTICLE
▸ getUser(id: Id[User])
▸ getTribe(id: Id[Tribe])
▸ getArticle(id: Id[Article])
▸ countViews(id: Id[Article])
8

11. DRIVETRIBE
DRIVETRIBE ARTICLE
▸ getUser(id: Id[User])
▸ getTribe(id: Id[Tribe])
▸ getArticle(id: Id[Article])
▸ countViews(id: Id[Article])
▸ countComments(id: Id[Article])
9

12. DRIVETRIBE
DRIVETRIBE ARTICLE
▸ getUser(id: Id[User])
▸ getTribe(id: Id[Tribe])
▸ getArticle(id: Id[Article])
▸ countViews(id: Id[Article])
▸ countComments(id: Id[Article])
▸ countBumps(id: Id[Article])
10

13. DRIVETRIBE
DRIVETRIBE FEED OF ARTICLES
▸ rankArticles(forUserId).flatMap { a
=> … }
▸ getUser(id: Id[User])
▸ getTribe(id: Id[Tribe])
▸ getArticle(id: Id[Article])
▸ countViews(id: Id[Article])
▸ …
11

14. DRIVETRIBE
QUINTESSENTIAL PREREQUISITES
▸ Scalable. Jeremy Clarkson has 7.2M Twitter followers. Cannot really hack
it and worry about it later.
▸ Performant. Low latency is key and mobile networks add quite a bit of it.
▸ Flexible. Almost nobody gets it right the first time around. The ability to
iterate is paramount.
▸ Maintainable. Spaghetti code works like interest on debt.
12

15. DRIVETRIBE
THREE TIER APPROACH
▸ Clients interact with a fleet of stateless servers
(aka “API” servers or “Backend”) via HTTP
(which is stateless).
▸ Global shared mutable state (aka the Database).
▸ Starting simple: Store data in a DB.
▸ Starting simple: Compute the aggregated views
on the fly.
13

16. DRIVETRIBE
DRIVETRIBE ARTICLE
▸ getUser(id: Id[User])
▸ getTribe(id: Id[Tribe])
▸ getArticle(id: Id[Article])
▸ countComments(id: Id[Article])
▸ countBumps(id: Id[Article])
▸ countViews(id: Id[Article])
14

17. DRIVETRIBE
READ TIME AGGREGATION
▸ (6 queries per Item) x (Y items per
page)
▸ Cost of ranking and personalisation.
▸ Quite some work at read time.
▸ Slow. Not really Performant.
15

18. DRIVETRIBE
WRITE TIME AGGREGATION
▸ Compute the aggregation at write
time.
▸ Then a single query can fetch all the
views at once. That scales.
16

19. DRIVETRIBE
WRITE TIME AGGREGATION
▸ Compute the aggregation at write
time.
▸ Then a single query can fetch all the
views at once. That scales.
17

20. DRIVETRIBE
WRITE TIME AGGREGATION
▸ Compute the aggregation at write
time.
▸ Then a single query can fetch all the
views at once. That scales.
18

21. DRIVETRIBE
WRITE TIME AGGREGATION
▸ Compute the aggregation at write
time.
▸ Then a single query can fetch all the
views at once. That scales.
19

22. DRIVETRIBE
WRITE TIME AGGREGATION - EVOLUTION
▸ sendNotification
20

23. DRIVETRIBE
WRITE TIME AGGREGATION - EVOLUTION
▸ sendNotification
▸ updateUserStats
21

24. DRIVETRIBE
WRITE TIME AGGREGATION - EVOLUTION
▸ sendNotification
▸ updateUserStats.
▸ What if we have a cache?
▸ Or a different database for search?
22

25. DRIVETRIBE
WRITE TIME AGGREGATION
▸ A simple user action is triggering a
potentially endless sequence of side
effects.
▸ Most of which need network IO.
▸ Many of which can fail.
23

26. DRIVETRIBE
ATOMICITY
▸ What happens if one of them fails?
What happens if the server fails in
the middle?
▸ We may have transaction support in
the DB, but what about external
systems?
▸ Inconsistent.
24
Atomicity?

27. DRIVETRIBE
CONCURRENCY
▸ Concurrent mutations on a global
shared state entail race conditions.
▸ State mutations are destructive and
can not be (easily) undone.
▸ A bug can corrupt the data
permanently.
25
Concurrency

28. DRIVETRIBE
ITALIAN PASTA
▸ Model evolution becomes difficult.
Reads and writes are tightly
coupled.
▸ Migrations are scary.
▸ This is neither Extensible nor
Maintainable.
26
Extensibility

29. DRIVETRIBE
DIFFERENT APPROACH
▸ Let’s take a step back and try to decouple things.
▸ Clients send events to the API: “John liked Jeremy’s post”, “Maria updated
her profile”
▸ Events are immutable. They capture a user action at some point in time.
▸ Every application state instance can be modelled as a projection of those
events.
27

30. DRIVETRIBE
▸ Persisting those yields an append-
only log of events.
▸ An event reducer can then produce
application state instances.
▸ Even retroactively. The log is
immutable.
▸ This is event sourcing.
28

31. DRIVETRIBE
▸ The write-time model (command
model) and the read time model
(query model) can be separated.
▸ Decoupling the two models opens the
door to more efficient, custom
implementations.
▸ This is known as Command Query
Responsibility Segregation aka
CQRS.
29

32. DRIVETRIBE
EVENT SOURCING APPROACH
30
Like!!

33. DRIVETRIBE
EVENT SOURCING APPROACH
31
Store Like Event
Like!!

34. DRIVETRIBE
EVENT SOURCING APPROACH
32
Store Like Event ArticleStatsReducer
Like!!

35. DRIVETRIBE
EVENT SOURCING APPROACH
33
Store Like Event ArticleStatsReducer
NotificationReducer
Like!!

36. DRIVETRIBE
EVENT SOURCING APPROACH
34
Store Like Event
ArticleStatsReducer
NotificationReducer
UserStatsReducer
Like!!

37. DRIVETRIBE
EVENT SOURCING APPROACH
35
Store Like Event
ArticleStatsReducer
NotificationReducer
UserStatsReducer
And so on..
Like!!

38. DRIVETRIBE
EVENT SOURCING APPROACH
36
Store Like Event
ArticleStatsReducer
NotificationReducer
UserStatsReducer
Sky Is the limit
Get Articles
Like!!

39. DRIVETRIBE
EVENT SOURCING APPROACH
37
Store Like Event
ArticleStatsReducer
NotificationReducer
UserStatsReducer
Sky Is the limit
Get Articles
Like!!
Extensibility?
Maintainability?

40. DRIVETRIBE
EVENT SOURCING APPROACH
38
Store Like Event
ArticleStatsReducer
NotificationReducer
UserStatsReducer
Sky Is the limit
Get Articles
Like!!
Performance?

41. DRIVETRIBE
EVENT SOURCING APPROACH
39
Store Like Event
ArticleStatsReducer
NotificationReducer
UserStatsReducer
Sky Is the limit
Get Articles
Like!!
Atomicity?

42. DRIVETRIBE
EVENT SOURCING APPROACH
40
Store Like Event
ArticleStatsReducer
NotificationReducer
UserStatsReducer
Sky Is the limit
Get Articles
Like!!
Concurrency?

43. IMPLEMENTATION?

44. WE NEED A LOG

45. DRIVETRIBE
APACHE KAFKA
▸ Distributed, fault-tolerant, durable and fast append-only log.
▸ Can scale the thousands of nodes, producers and consumers.
▸ Each business event type can be stored in its own topic.
43

46. WE NEED A STREAM
PROCESSOR

47. DRIVETRIBE
APACHE FLINK
▸ Scalable, performant, mature.
▸ Elegant high level APIs in Scala.
▸ Powerful low level APIs for advanced tuning.
▸ Multiple battle-tested integrations.
▸ Very nice and active community.
45

48. WE NEED DATASTORE

49. DRIVETRIBE
ELASTICSEARCH
▸ Horizontally scalable document store.
▸ Rich and expressive query language.
▸ Dispensable. Can be replaced.
47

50. WE NEED AN API

51. DRIVETRIBE
AKKA HTTP
▸ Asynchronous web application framework.
▸ Written in Scala.
▸ Very expressive routing DSL.
▸ Any modern web application framework would do.
49

52. DRIVETRIBE
EVENT SOURCING IN PRACTICE
50
Store Raw events Consume raw events
Produce aggregated
views
Retrieve aggregated
views

53. DRIVETRIBE
EVENT SOURCING IN PRACTICE
51
Store Raw events Consume raw events
Produce aggregated
views
Retrieve aggregated
views
Stateful

54. DRIVETRIBE
EVENT SOURCING IN PRACTICE
52
Store Raw events Consume raw events
Produce aggregated
views
Retrieve aggregated
views

55. DRIVETRIBE
BLUE/GREEN APPROACH
53
MIRROR

56. A REAL WORLD EXAMPLE

57. DRIVETRIBE
COUNTING BUMPS
▸ Thousands of people like the fact that
Jeremy Clarkson is a really tall guy
▸ Users can “bump” a post if they like it
55

58. DRIVETRIBE
EVENT SOURCING IN PRACTICE
56
Store Raw events Consume raw events
Produce aggregated
views
Retrieve aggregated
views

59. DRIVETRIBE
COUNTING BUMPS
57

60. DRIVETRIBE
COUNTING BUMPS - FIRST ATTEMPT
58

61. DRIVETRIBE
COUNTING BUMPS - FIRST ATTEMPT
59

62. DRIVETRIBE
COUNTING BUMPS - FIRST ATTEMPT
60
▸ Use Flink with at least once
semantics

63. DRIVETRIBE
COUNTING BUMPS - FIRST ATTEMPT
61
▸ Use Flink with at least once
semantics
▸ Our system is eventually consistent

64. DRIVETRIBE
WHAT DO WE KNOW ABOUT OUT COUNTER?
62
▸ we know we’re doing some kind of combine operation over States

65. DRIVETRIBE 63
▸ we know we’re doing some kind of combine operation over States
▸ we want our counter to be idempotent: a |+| a === a
WHAT DO WE KNOW ABOUT OUT COUNTER?

66. DRIVETRIBE 64
▸ we know we’re doing some kind of combine operation over States
▸ we want our counter to be idempotent: a |+| a === a
▸ we also want our counter to be associative:
a |+| (b |+| c) === (a |+| b) |+| c
WHAT DO WE KNOW ABOUT OUT COUNTER?

67. DRIVETRIBE
BAND ALGEBRA
65
▸ Closed
▸ Idempotent
▸ Associative

68. DRIVETRIBE
COUNTING BUMPS - SECOND ATTEMPT
66

69. DRIVETRIBE
COUNTING BUMPS - SECOND ATTEMPT
67
▸ if all components of a case
class have a band then so does
the case class
▸ would normally bring in Set
implementation from a library
▸ normally that library would have
a law testing module

70. DRIVETRIBE
COUNTING BUMPS - PUTTING IT TOGETHER
68

71. DRIVETRIBE
OTHER ALGEBRAS WE USE
69
▸ Adding events: Semigroup/Monoid
▸ Duplicate events: Band
▸ Out of order and duplicate events: Semilattice

72. FIN

73. Watch the video with slide
synchronization on InfoQ.com!
https://www.infoq.com/presentations/
drivetribe