The BBC’s website and apps are used around the world by an audience of millions who read, watch, and interact with a range of content. The BBC handles this scale with an innovative website platform, built on Amazon ElastiCache and Amazon EC2 and based on nanoservices. The BBC has over a thousand nanoservices, powering many of its biggest webpages. Explore its nanoservices platform and use of ElastiCache. Learn how Redis’s ultra-fast queues and pub/sub allow thousands of nanoservices to interact efficiently with low latency. Discover intelligent caching strategies to optimize rendering costs and ensure lightning fast performance. Together, ElastiCache and nanoservices can make real-time systems that can handle thousands of requests per second.

1. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
The Nanoservices Architecture
That Powers BBC Online
M a t t h e w C l a r k , B B C
N o v e m b e r 2 8 , 2 0 1 7
AWS re:INVENT
D A T 3 2 2

4. Background
Pattern
Our implementation
Pros & cons
This is the story of the
nanoservices pattern.
Small units of logic that are
easily understood, deployed,
operated, and shared.
Built on an AWS-hosted
platform that makes extensive
use of Amazon ElastiCache.
4

5. AWS’s hosted service for Redis (and Memcached)
Automates common administrative tasks (e.g., failed nodes,
server optimisation)
Multiple options available (different memory and network
sizes)
Redis is much more than a key-value store: multiple data
types, transactions, Lua scripting, pub/sub, …
Can grow with sharding and/or replication
Amazon ElastiCache
5

7. Make the common things easy
and the special things possible

8. 8
http://2016.chartbeat.com/

9. Multiple teams
Multiple technologies
Smaller and safer deploys
Shareable and flexible
Easily scalable
Pros
9
Microservices Cons
Maintenance overhead
Comms overhead
Multi-version challenges
Harder to refactor
Cost

10. [Microservices] bring an enormous amount of operational complexity to
the table. Suddenly, you need to continuously deploy many different
(possibly containerized) services. New concerns arise: service
discovery, distributed logging, tracing and so on. You are now even more
prone to the fallacies of distributed computing. Versioning of interfaces
and configuration management become a major concern. The list goes
on and on.
–Sander Mak
https://www.oreilly.com/ideas/modules-vs-microservices
10

11. https://en.wikipedia.org/wiki/Microservices

12. Responsible for a single, non-trivial, business-understandable task
Smaller than a microservice, larger than a function
Likely to call other nanoservices, to offer rich functionality
The ideal sized thing to release (or MVT) in a CD environment
Example:
– Get customer order history
– Place order
Nanoservice
12

13. http://www.bbc.co.uk/news

15. Event
page
Top Left Centre Right
Medals
module
Medals
page
Medals
data
Mobile
alert
DB
Multi-
lingual
medals
Internet
connected
TV

18. Architecture
18
Developer
Data (APIs, etc.)
User
Nanoservice
store
Nanoservice
platform
Metadata
store
Content
cache
Queue

19. Every output from every run of
every nanoservice is stored
Cheap, simple, ephemeral
Maximises reuse, speed,
reliability, and scalability
Redis for content
19

20. 4x m4.xlarge (sharded)
Redis: We store 6–8 million items
20

21. Metadata about every request, and its ‘tree’, is
stored in Redis
If the data changes, the metadata informs us what
nanoservices to rerun
The ‘relationship’ is stored both ways:
 ‘A uses B’ and ‘B is used by A’
 Not ‘normal form’, but fast
 Lua can help make atomic
 ‘Hexastore’ approach can do full graphs
Redis for metadata
21
Medals
page
Medals
module
Medals
Data
Mobile
alert
DB

22. Redis, as a queue, is extraordinarily fast (<5 ms end-to-end journey)
It’s the only solution fast enough for this architecture
We comfortably do 10k messages per second (r3.4xlarge)
Redis for queues
22
We use Lua to also write to a set, to then catch de-duplicates
producer
LPUSH queue msg BRPOP queue
consumer

23. Doesn’t really work with replication  Can be a SPOF
Lossy (due to consumer failure), though can be offset with:
 BRPOPLPUSH <queue> <in-progress-queue> <timeout>
 And something to analyse the in-progress-queue
Metrics not so obvious
Harder to scale
No advanced features (dead-letter, prioritization, data pushes)
Redis for queues: Trade-offs
23

24. Cross-AZ Redis calls can be 4x slower* (though still very quick)
Replication helps (but you become eventually consistent)
To minimise latency, consider multiple single-AZ deployments:
An alternative cross-AZ structure
24
* redis-benchmark -c 1 -n 10000 -t GET on m4.large against m4.large ElastiCache, does 7,200/sec same AZ, 1,800/sec different AZ
Availability Zone #1 Availability Zone #2

25. Each nanoservice is developed
& managed independently
Nanoservice
Owner Tests
Connected
nanoservices
Single-click
deploy Analytics UsersStatic files
CodeVersions

26. Demo
26

27. Great ‘Lego bricks’ for sharing
Easy to understand concepts, reduces barrier-to-entry
Should be good value (sharing infrastructure, highly cacheable)
Looks after much of the operational side (scaling, resilience, monitoring,
etc.)
Fast deployment
 Greater development speed
Nanoservices: Advantages
27

28. Single point of failure
Must be conservative, isolate and limit
Can be restrictive
Not very fast
Needs great tooling to manage and understand usage
Challenges
28

29. Multiple teams
Multiple technologies
Smaller and safer deploys
Shareable and flexible
More scalable
Pros
29
Microservices Cons
Maintenance overhead
Comms overhead
Multi-version challenges
Harder to refactor
Cost
✓
✓
✓
✓
?

30. Serverless
AWS Lambda
functions
Azure functions Google Cloud
functions
30
Why should they have all the fun?
The world needs powerful serverless platforms.

31. Is this serverless?
31
We didn’t use Lambda because:
 The need for consistent, <10 ms response
 Instantiation from a Redis queue
 Twiddling thumbs (waiting for dependencies)
We’re considering ‘burst to Lambda’ possibilities
To the developers, this is serverless

32. Serverless will fundamentally change how we build business around
technology and how you code. …
You thought devops was big but it’s chicken feed compared to this…
...expect to see the whole world being overtaken by serverless by 2025
–Simon Wardley, in Why the fuss about serverless?
https://www.linkedin.com/pulse/why-fuss-serverless-simon-wardley
32

33. 33
Is nanoservices a paradigm for
future serverless architecture?

34. https://medium.com/netflix-techblog/developer-experience-lessons-operating-a-serverless-like-platform-at-netflix-
a8bbd5b899a0
It’s not just us
34

35. https://tinyurl.com/nanoservices
But this is (if you want to read more)
35

36. Nanoservices have
brought the BBC:
Sharing
Flexibility
Efficiency
Velocity

37. Summary
Using Redis, you can achieve phenomenal performance, not just in
caching content, but as a control and communication layer.
Which can let you build fast and flexible serverless-like platforms that
allow your organisation to move faster.
The nanoservice pattern is an interesting way to design for serverless,
and can maximise the potential of continuous delivery, reusability, and
flexibility.
37

38. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Thank you!
M a t t h e w C l a r k
m a t t h e w . c l a r k @ b b c . c o . u k
@ m a t t h e w 1 0 0 0