A while ago, I started a project that looked obvious at first. We had to build an IoT system to send messages from devices to the cloud for processing and show everything on a web dashboard. Sounds basic, right? I thought so, too.There was one "little" requirement that took the whole architecture to the ultimate test: It should be able to scale, in a relatively short period of time, to handle bursts of 300K messages/sec. We designed and implemeted the system with the performance requirements in mind, but we had to prove that it actually works.Join me to learn what we learnt from spending 25K$ on testing, to make sure that the designed architecture can withstand the crazy rate of 300K messages per second in a totally serverless system.

1. @AlexPshul
Testing 300K messages/sec Lessons
Alex Pshul
@AlexPshul
alex@pshul.com
https://pshul.com
https://codevalue.net
https://www.meetup.com/Code-Digest/
a serverless IoT system case study

2. @AlexPshul
Kids, I’m going to tell you an incredible story
The story of how we created a beautiful IoT serverless system

3. @AlexPshul
Agenda
▪ Project Requirements
▪ Why Serverless?
▪ Solutions
▪ IoT Hub
▪ IoT Edge
▪ Testing & Conclusions
▪ Would I do that again? (Summary)
3
Agenda

4. @AlexPshul
About Me
4
Alex Pshul
▪ Architect, Consultant, lecturer & tech freak
▪ More than 9 years of hands on experience
▪ Co-organizer of the Code.Digest Meetup
▪ https://www.meetup.com/Code-Digest/
▪ Talk to me about:

5. @AlexPshul
The Project
Requirements
5

6. @AlexPshul
Requirements
6
Icon made by Freepik from www.flaticon.com
IoT DevicesSync DeviceSome Cloud MagicBeautiful Dashboard

7. @AlexPshul
Requirements
7
Icon made by Freepik from www.flaticon.com
Some Cloud MagicBeautiful Dashboard IoT Edge

8. @AlexPshul
Requirements
8
Icon made by Freepik from www.flaticon.com
Some Cloud MagicBeautiful Dashboard IoT Edge

9. @AlexPshul
Requirements
9
Icon made by Freepik from www.flaticon.com
Some Cloud MagicBeautiful Dashboard IoT Edge

10. @AlexPshul
Requirements
10
Icon made by Freepik from www.flaticon.com
Some Cloud MagicBeautiful Dashboard IoT Edge

11. @AlexPshul
Requirements
▪ Support edge to cloud messages
▪ Process the messages
▪ Real-time & Near real-time processing
▪ Minimum 4 messages/sec
▪ Maximum 100K messages/sec
▪ Store data for later use
▪ Cloud to device messages
11

12. @AlexPshul
Serverless & Managed Services!
But why?!
12

13. @AlexPshul
Cost Effective
13

14. @AlexPshul
Scalable
14

15. @AlexPshul
Flexible
▪ Run on cloud
▪ Or in a container
▪ Rich IoT out-of-the-box solutions suite
▪ Easy to start
▪ Easy to deploy
15

16. @AlexPshul
Solutions
Which services and why?
16

17. @AlexPshul
Azure Functions
▪ Easy to start & deploy
▪ Cost efficient
▪ Scalable
▪ Very Flexible
▪ Locally (For debug)
▪ As a Serverless cloud solution
▪ In an AppService/Premium solutions
▪ As a container
▪ Easy microservices implementation
17

18. @AlexPshul
SignalR as a Service
▪ Extreme realtime messaging
▪ Two way communication
▪ Scalable
▪ Free tier for development
▪ Can be run as a library
18

19. @AlexPshul
SignalR as a Service
19

20. @AlexPshul
Azure Storage Tables
▪ Good integration
▪ Scales
▪ Data price = Today + Previous days
▪ Still, Crazy Cheap!
▪ API matches CosmosDB
▪ Write to both – For long and short term storage
20

21. @AlexPshul
Azure Event Hub
▪ Messages between services
▪ High throughput scalable units
▪ Pay-for-use
▪ Easy integration with Azure services
▪ And Apache Kafka
21

22. @AlexPshul
IoT Hub
Edge, Events and everything in between
22

23. @AlexPshul
What is IoT Hub?
23

24. @AlexPshul
IoT Edge
▪ The “edge” of the cloud reach
▪ Runs on the device
▪ Easy OTA services
▪ Version updates
▪ Installations
▪ Provisioning
▪ Runs containers
▪ Cloud Communication
24

25. @AlexPshul
IoT Hub + IoT Edge
▪ Flexible Updates
▪ Locally
▪ Remotely to a large amount of devices
▪ Variety of tiers
▪ By features
▪ By throughput
▪ Units for scaling
▪ Easy two way communication
25

26. @AlexPshul
IoT Hub
2626
Icon made by Freepik from www.flaticon.com
Some Cloud MagicBeautiful Dashboard IoT Edge

27. @AlexPshul
Cloud “Magic”
2727
Icon made by Freepik from www.flaticon.com
Functions
SignalR
Event Hub IoT Hub IoT Edge
Storage

28. @AlexPshul
Cloud “Magic” – Store Data
2828
Icon made by Freepik from www.flaticon.com
IoT Message
Functions
SignalR
Event Hub IoT Hub IoT Edge
Storage

29. @AlexPshul
Cloud “Magic” – Show Data
2929
Icon made by Freepik from www.flaticon.com
Functions
SignalR
Event Hub IoT Hub IoT Edge
Storage

30. @AlexPshul
Cloud “Magic” – Live Data
3030
Icon made by Freepik from www.flaticon.com
Live Data
Functions
SignalR
Event Hub IoT Hub IoT Edge
Storage

31. @AlexPshul
Cloud “Magic” – Update a Device
3131
Icon made by Freepik from www.flaticon.com
Update
Device
Functions
SignalR
Event Hub IoT Hub IoT Edge
Storage

32. @AlexPshul
Testing & Conclusions
Pitfalls and other FAQs
32

33. @AlexPshul
Max Load Test
33
100 messages/second500 messages/second

34. @AlexPshul
Max Load Test
34

35. @AlexPshul
Max Load Test
35
500 messages/second
×200 VMs
100K messages/second
17 × S3 IoT Hub Units
(2 IoT Hubs)
42,500 USD!!!
IoT Hub

36. @AlexPshul
Max Load Test
36

37. @AlexPshul
Optimized Load Test
37

38. @AlexPshul
Optimized Load Test
38
500 messages/second
Pack 20 → 1 messages

39. @AlexPshul
Optimized Load Test
39

40. @AlexPshul
Max Load Test
40

41. @AlexPshul
Max Load Test
41
25 messages/second
×200 VMs
5K messages/second
1 × S3 IoT Hub Units
2,500 USD!!!
IoT Hub

42. @AlexPshul
Max Load Test
42
25 messages/second
×200 VMs
5K messages/second
1 × S3 IoT Hub Units
2,500 USD!!!
IoT Hub

43. @AlexPshul
0 → 100 Instant Scale is Impossible
43
Scale
Scale Scale
Scale
Scale
Scale
Scale
Scale Scale
Scale
Scale Scale
Scale
SCALE ≠ AVAILABILITY

44. @AlexPshul
0 → 100 Instant Scale is Impossible
44
Non HTTP - scales once every 30 seconds
20,000 Entities per second
Partitions # influence the functions scale

45. @AlexPshul
0 → 100 Instant Scale is Impossible
45
Use Premium/AppService plan or convert to containers
Use multiple storage accounts
Trial and error to see optimal configuration

46. @AlexPshul
Summary
Would I go serverless again?
46

47. @AlexPshul
Would I go serverless again?
47

48. @AlexPshul
Would I go serverless again?
▪ Flexible
▪ React to request changes
▪ Different deploy options
▪ Cheap
▪ Neglectable while developing
▪ Cheap in production
▪ If used properly
▪ Easy to deploy
▪ Easy to debug
48

49. Alex Pshul
@AlexPshul
https://www.pshul.com
alexp@codevalue.net
49