For successful implementation of distributed systems, flexible and scalable messaging layer is one of the most important components. Setting a static messaging infrastructure and provisioning it manually doesn’t fit well in the cloud-centric development model most organisations are adopting lately. The EnMaase project (http://enmasse.io/) provides an open source solution for deploying your own messaging infrastructure in the cloud. It’s based on proven standards and technologies like AMQP and Kubernetes and provides all the features you’d need, from multi-tenancy to simple management and monitoring. This session will cover EnMaase project in details, providing details on the architecture, messaging concepts supported and ways to set and configure it.

1. Messaging for the cloud
Building a scalable messaging service on Kubernetes
and OpenShift
Dejan Bosanac
Senior Software Engineer, Red Hat
@dejanb
1

2. 2
Outline
● What is messaging?
● How to scale messaging in general
● EnMasse messaging cloud platform

3. 3
● Sending messages
○ Internally in distributed systems
○ Externally between systems
● Communication at the application level
● Messages go from sender/producer to receiver/consumer
○ Asynchronously
○ Time decoupling
What is messaging?

4. 4
Basic features
Request-response Publish-subscribe Competing-consumers

5. 5
Basic features
At most once At least once Exactly once
X X

6. 6
Brokered messaging
Store and forward
Producer Broker Consumer
Send message
Accepted
Send message
Accepted

7. 7
Direct Messaging
Router
Producer Router Consumer
Send message
Accepted
Send message
Accepted

8. 8
● Store and Forward
○ Queue
○ Topic
● Direct
○ Anycast
○ Multicast (Broadcast)
Addressing semantics

9. 9
Basic idea
R
R
R
P C
B
B

10. 10
Messaging in the cloud

11. 11
● Freedom of choice
○ On-premise or in the cloud
○ Ability to choose which cloud
○ Open Standards protocols allows users to choose client freely
● Migrating from one to the other can be complex
Cloud provider limitations

12. 12
● Open source cloud messaging running on Kubernetes and OpenShift
● enmasse.io
● github.com/enmasseproject/enmasse
EnMasse
Messaging-as-a-Service

13. 13
● Support for store-and-forward and direct messaging mechanisms
● Multiple communication patterns: request-response, publish-subscribe and competing
consumers
● Scale and elasticity of message brokers
● AMQP 1.0 and MQTT support
● Simple setup, management and monitoring
● Multitenancy: manage multiple independent instances
● Deploy on-premise or in the cloud
EnMasse
Features

14. OpenShift console
14

15. Messaging console
15
●

16. 16
Address
{
"apiVersion": "v3",
"kind": "Address",
"metadata": {
"name": "myqueue"
},
"spec": {
"store_and_forward": true,
"multicast": false,
"flavor": "vanilla-queue"
}
}

17. 17
● Queue
○ store-and-forward = true
○ multicast = false
● Topic
○ store-and-forward = true
○ multicast = true
● Anycast
○ store-and-forward = false
○ multicast = false
● Broadcast
○ store-and-forward = false
○ multicast = true
Address types

18. {
"apiVersion": "v3",
"kind": "Flavor",
"metadata": {
"name": "vanilla-queue"
},
"spec": {
"type": "queue",
"Description": "Simple in-memory queue",
"templateName": "queue-inmemory",
"templateParameters": {
"BROKER_IMAGE": "mybroker:latest",
"STORAGE_CAPACITY": "8Gi"
}
}
}
18
Flavor

19. 19
● Persistence
○ In memory
○ Persisted
● Scaling
○ Single broker
○ Pooled
● HA
Flavor examples

20. 20
Architecture

21. MQTT support
21
● MQTT gateway
○ Handles connections with remote MQTT clients
○ Bridges MQTT - AMQP protocols
● MQTT lwt
○ Provides the “will testament” feature
○ In charge to recover & send the “will” if client dies

22. 22
Scaling (routers and brokers)

23. 23
Scaling routers
R
R
R
R
A

24. 24
Scaling routers (#2)
R
R
R
R
A

25. Scaling brokers
25
a a b b
Router network

26. Adding brokers
26
a a b b
Router network
a

27. Removing brokers
27
a a b b
Router network
a

28. 28
Monitoring

29. 29
● Authentication and authorization
● Service broker API
● HTTP(S)
● Broker address space
○ Message grouping
○ Distributed transactions
○ Message ordering
● Multiple flavors
○ Apache Kafka?
● ...
Future
In progress/TODO

30. Resources
30
● EnMasse - http://enmasse.io
● ActiveMQ Artemis - https://activemq.apache.org/artemis/
● Qpid Dispatch Router - http://qpid.apache.org/components/dispatch-router/

31. Thank you ! Questions ?
@dejanb