Comparing the speed of RPC calls over WebScoket Microservices versus REST based microservices. Using wrk, QBit, and examples in Java we show how much faster WebSocket is for doing RPC service calls.

1. Rick Hightower - Mammatus Technology Inc.
WebSocket
MicroService vs.
REST Microservice
How does the performance of
a REST microservice compare
to a WebSocket
MicorService?

2. Comparison of Speed
❖ WebSocket is faster and uses
less resources for more calls
❖ REST is slower due to each
connection needing to wait for
a response before it can send
another request

3. REST/HTTP vs WebSocket
❖ With REST you have to wait for a reply before you can
send another request (over the same connection)
❖ Unless you use pipelining which seems to work better
in benchmarks than the real world
❖ With WebSocket you can stream requests and stream
responses and then marry them together
❖ Still request/reply but WebSocket can send
responses while it is getting other requests

4. Example uses QBit
❖ QBit supports REST and WebSocket RPC
❖ QBit allows you to stream requests over the wire and
receive responses over the wire. It employs micro-
batching to batch requests and responses into larger
chunks to minimize IO and thread hand-off costs
❖ QBit allows async REST calls and async WebSocket
calls. (As well as internal services based on queues)

5. wrk Testing tool for
REST/HTTP
❖ To perf test REST code we use wrk
❖ wrk load tester capable of generating significant load
when run on a single multi-core CPU
❖ wrk load tester uses multithreaded and event
notification systems
❖ epoll and kqueue
❖ wrk maximizes the number of HTTP requests per
second

6. Trade Service

7. wrk LUA script

8. Caveats About the Test
❖ We can tweak server a bit and reduce the flush rate or reduce the batch size to get higher
throughput with lower connections
❖ We can also tweak the OS so we can have more ephemeral ports available and then use a lot
more connections
❖ experience tells me that we can get close to 90K TPS or so on a MacBook Pro
❖ We could test from two machines
❖ one of those machines being a Linux server with a tweaked stack
❖ This test has disadvantage of all being run on same machine
❖ but same disadvantage that the WebSocket version will have so it is somewhat fair
❖ We could also employ HTTP pipelining to increase the throughput,
❖ great trick for benchmarks rarely works in production environments with real clients
❖ On commodity Linux server, we can get close to 150K TPS to 200K TPS from experience

9. WebSocket Test
❖ This will be a bit harder
❖ No tool like wrk to test WebSocket RPC calls so we will
write our own
❖ RPC calls are QBit centric and rely on JSON plus some
custom ASCII protocol
❖ Some overhead in the protocol because we have to
match responses to clients and requests need unique
identifier (as well as the client needing a unique identifier)

10. To use WebSocket we will need Async interface
to service

11. WebSocket PERF Test code

12. WebSocket Perf code
continued

13. How does the WebSocket
microservice do?
1 Million messages a second.
700K messages for a single WebSocket connection.
7.5x better than REST version.

14. Overhead
❖ Running code through perf showed it was waiting for IO a lot.
Better IO. Better NIC card. More TCP/IP stack tuning. You can
expect better throughput.
❖ QBit has to marry requests with responses/client id combinations.
There is overhead in doing this. QBit also uses JSON/ASCII
protocol which could be optimized to reduce IO (gzip it or use
message pack).
❖ Custom more efficient streaming can be implemented if needed
(from experience, this is another 2x to 4x)
❖ But in general, WebSocket will do better than HTTP

15. More Caveats
❖ When using WebSocket, you also need service
discovery, and you need to implement your own failover
❖ You can’t just throw a bunch of microservices behind
a load balancer
❖ QBit integrates with DNS and consul.io to provide
service discovery (also allows you to use push)

16. Follow up
❖ This slide deck as a blog post:
❖ http://rick-hightower.blogspot.com/2015/12/websocket-microservice-vs-rest.html
❖ The QBit project:
❖ https://github.com/advantageous/qbit
❖ Mammatus Technologies
❖ http://www.mammatustech.com/
❖ Rick Hightower
❖ https://twitter.com/RickHigh
❖ https://www.linkedin.com/in/rickhigh

17. There is a lot more to QBit
than this

18. QBit supports reactive stream over
WebSocket

19. QBit implements Service
Actors

20. Check out QBit
https://github.com/advantageous/qbit