Overview of JBoss A-MQ

1. Connecting Applications with
Red Hat JBoss A-MQ
Rob Davies
Technical Director, Fuse
September 2014

2. Rob Davies – Red Hat
• Director, Engineering xPaaS
• Technical Director of Fuse Engineering
• Over 20 years experience of developing
large scale solutions for telcos and finance
• Creator of ActiveMQ and ServiceMix
• Committer on open source projects,
including Apache Camel …
2 RED HAT | Fuse Rocks!
2

3. Why use Message-Oriented Middleware ?
• Robustness to change
• Time Independence
• Location Independence
• Hide Latency
• Scalability
• Event driven
• Simplicity
• Configurable Quality of Service
• Platform and Language Integration
• Fault tolerant
Process
A Queue:Foo
Process
B
Process
3 RED HAT | Fuse Rocks!
3
Broker
Message
in
Message out after enrichment
by Process B
C
Queue:Bar

4. • Top Level Apache Software Foundation Project
• Wildly popular, high performance, reliable message broker
• Connects to nearly everything
•Native Java, C/C++, .Net,
•AMQP 1.0, MQTT 3.1, STOMP (1.0-1.2) and OpenWire
•STOMP enables Ruby, JS, Perl, Python, PHP, ActionScript
…
• Embedded and standalone deployment options
• The key component of JBoss A-MQ!
4 RED HAT | Fuse Rocks!
4
What is Apache ActiveMQ ?

5. Messaging:
The Basics
5 RED HAT | Fuse Rocks!

6. Message Channels and Routing
• Message Channels
Named communication between interested parties
JMS calls them ‘Destinations’
• Can “tune-in” to multiple channels using wildcards
• Can fine-tune message consumption with selectors
• Can route a message based on content
6 RED HAT | Fuse Rocks!
6

7. Message Channels = JMS Destinations
7 RED HAT | Fuse Rocks!
7
Broker Consumer
Consumer
Destination
WIDGET
Destination
ORDER
Producer
Consumer

8. Point-to-Point channel = JMS Queue
8 RED HAT | Fuse Rocks!
8
Consumer
Producer Queue Consumer
Consumer

9. Publish/Subscribe channel = JMS Topic
9 RED HAT | Fuse Rocks!
9
Producer
Consumer
Consumer
Consumer
Topic

10. Message Routing: Selectors
10 RED HAT | Fuse Rocks!
10
Producer Destination
Consume
Colorr =
blue
Consume
Colorr =
red

11. Message Routing: Destination Wildcards
• * matches a subject
• > matches sub-tree
11 RED HAT | Fuse Rocks!
11
Topic:
BAR.BEER.WHITE
Topic:
BAR.WINE.WHITE
Producer
Consumer
Topic:BAR.>
Consumer
Topic:BAR.*.WHITE
Consumer
Topic:BAR.WINE.WHITE
Consumer
Topic:BAR.BEER.WHITE

12. Message Delivery Mode
• Messages can be persisted – allowing for decoupled
applications
• For Queues, any message delivered as PERSISTENT must be
stored on long term storage before being delivered to a
consumer
• For Topics, a message delivered as PERSISTENT will only be
stored if there exists a durable subscriber
• NON-PERSISTENT messages may also be stored on disk if the
memory limits of the broker have been reached
12 RED HAT | Fuse Rocks!
12

13. Messaging:
Features every message broker
should have …
13 RED HAT | Fuse Rocks!

14. Exclusive Consumers
14 RED HAT | Fuse Rocks!
14
Consumer
Consumer
Consumer
Producer Queue

15. Message Groups
• Like Exclusive Consumers – but in parallel
• Guaranteed ordering of related messages across a Queue
• But – load balancing of messages across multiple consumers
• All messages with the same JMSXGroupID go to the same consumer
• How you group messages is down to the application’s producer
• To explicitly close a group, set the JMSXGroupSeq to -1
15 RED HAT | Fuse Rocks!
15

16. Message Groups Code example
16 RED HAT | Fuse Rocks!
16
//Starting a Group …
Message message = session.createTextMessage("<foo>hi
DevNation</foo>");
message.setStringProperty("JMSXGroupID", ”RHT_NYSE");
producer.send(message);
//Close a Group …
Message message = session.createTextMessage("<foo>bye
DevNation</foo>");
message.setStringProperty("JMSXGroupID", ”RHT_NYSE");
message.setStringProperty("JMSXGroupSeq", -1);
producer.send(message);

17. Message Groups
17 RED HAT | Fuse Rocks!
17
Producer
Consumer
Consumer
Consumer
Queue:Prices

18. Deployment
• ActiveMQ Can run standalone or embedded
• As a standalone, or part of a highly available message
broker cluster
• Embedded – easy to use an entire broker in JUnit tests
(no need to Mock)
• In Tomcat, deployed as a war
• In JEE Server – either co-locate – or use client with JCA
ActiveMQ distributions contain a rar for this purpose
18 RED HAT | Fuse Rocks!
18

19. JBoss A-MQ
19 RED HAT | Fuse Rocks!
19

20. Messaging:
Persistence
20 RED HAT | Fuse Rocks!

21. Different Storage Options
• SQL (JDBC)
• KahaDB
• LevelDB
21 RED HAT | Fuse Rocks!
21
Slow but popular
The “current”
default
The “best yet”

22. ActiveMQ Message Storage
22 22 RED HAT | Fuse Rocks!

23. LevelDB vs KahaDB
• Fewer index entries per message than KahaDB
• Faster recovery when a broker restarts
• LevelDB index out-perform Btree index at sequential
access
• LevelDB indexes support concurrent read access
• Pause-less data log file garbage collection cycles
• Fewer IOPS to load stored messages
• It exposes it's status via JMX for monitoring
23 RED HAT | Fuse Rocks!
23

24. LevelDB vs KahaDB (cont.)
24 RED HAT | Fuse Rocks!
24

25. Messaging:
Protocols
25 RED HAT | Fuse Rocks!

26. OpenWire
Advantages: Disadvantages:
26 RED HAT | Fuse Rocks!
26
• Fast – optimized for
ActiveMQ
• client failover
• automatic reconnect
• Client load balancing
• Flow control
• Many advanced features
• Not a recognized
standard
• Only Java, C/C++/.Net

27. MQTT
Advantages: Disadvantages:
27 RED HAT | Fuse Rocks!
27
• “light weight” telemetry
transport
• Standard out of OASIS
• Adoption by a large cross-section,
from Facebook
(Messenger) to
embedded devices
• Version 3.x only
supports
publish/subscribe
• No advance features,
such as flow control

28. AMQP
Advantages: Disadvantages:
28 RED HAT | Fuse Rocks!
28
• AMQP 1.0 OSASIS
standard
• Feature rich messaging
protocol
• C/C++/Java clients and
more in development
• Currently, only client to
broker supported in
ActiveMQ

29. STOMP
Advantages: Disadvantages:
29 RED HAT | Fuse Rocks!
29
• Easy to implement
• Defacto standard:
• ActiveMQ + Apollo
• HornetQ
• RabbitMQ
• PocoMQ
• StompServer
• OpenMQ
• Many more …
• Not as fast a binary
formats

30. Websockets
• STOMP is a natural wire protocol for WebSockets
• And so is MQTT!
HTTP(S)
Request/Response
30 RED HAT | Fuse Rocks!
30
ActiveMQ
Client
(Browser)
WebSocket
Connector
Stomp
Converter
ActiveMQ
(Backend)
TCP
(WebSocket)

31. Configuring protocols
31 RED HAT | Fuse Rocks!
31
<transportConnector uri = “tcp://0.0.0.0:61616”/>
<transportConnector uri = “nio://0.0.0.0:61615”/>
<transportConnector uri = “stomp+ssl://0.0.0.0:61617”/>
<transportConnector uri = “mqtt+nio+ssl://0.0.0.0:1883”/>
<transportConnector uri = “amqp+nio+ssl://0.0.0.0:5672”/>
<transportConnector uri = “wss://0.0.0.0:61614”/>

32. High Availability
32 RED HAT | Fuse Rocks!

33. Master/Slave
• Shared Message Store
• Replicated LevelDB Master/Slave
33 RED HAT | Fuse Rocks!
33

34. Messaging:
High Availability
34 RED HAT | Fuse Rocks!

35. Failover
Supported by both Java and C++ OpenWire
clients
ActiveMQ
Cluster
ActiveMQ
Client
35 RED HAT | Fuse Rocks!

36. High Availability with shared storage
ActiveMQ
Cluster
ActiveMQ
Client
36 RED HAT | Fuse Rocks!

37. Replicated LevelDB
37 RED HAT | Fuse Rocks!
Registry
(ZooKeeper)
ActiveMQ
Client

38. Messaging:
Store and Forward (Networks)
38 RED HAT | Fuse Rocks!

39. Networks of Brokers
Link Brokers together
Use Store and Forward
Are uni-directional by default
All Destinations are global
39 RED HAT | Fuse Rocks!
39

40. Networks of Brokers
40 RED HAT | Fuse Rocks!
40

41. Store and Forward
41 RED HAT | Fuse Rocks!
41

42. Bi-directional network
42 RED HAT | Fuse Rocks!
42

43. Bi-directional Configuration
<networkConnectors>
<networkConnector name="backoffice"
uri="static://(tcp://backoffice:61617)”
duplex="true">
</networkConnector>
</networkConnectors>
43 RED HAT | Fuse Rocks!
43
<networkConnectors>
<networkConnector name="backoffice"
uri="static://(tcp://backoffice:61617)”
duplex="true">
</networkConnector>
</networkConnectors>

44. Reducing cross-talk
<networkConnectors>
<networkConnector uri="static:
(tcp://remote:61617) "/>
<dynamicallyIncludedDestinations>
<queue physicalName=”free.food.>"/>
<queue physicalName=”free.beer.>"/>
<topic physicalName=”cricket.scores.>"/>
</dynamicallyIncludedDestinations>
</networkConnectors>
44 RED HAT | Fuse Rocks!
44
<networkConnectors>
<networkConnector uri="static:
(tcp://remote:61617) "/>
<dynamicallyIncludedDestinations>
<queue physicalName=”free.food.>"/>
<queue physicalName=”free.beer.>"/>
<topic physicalName=”cricket.scores.>"/>
</dynamicallyIncludedDestinations>
</networkConnectors>

45. Scaling Networks
45 RED HAT | Fuse Rocks!
45
• Brokers share routing information across networks
• All destinations are considered Global – easy to use
• Though it starts to get problematic with 1000’s of
brokers
• However – we can do filtering to shape the traffic
across networks
• But this involves a lot of manual configuration
Time to talk about Apache Camel …

46. ActiveMQ and Apache Camel
46 RED HAT | Fuse Rocks!
46
S e rvice
C o nsum e r
C o nsum e r
A ctiveM Q B ro ke r

47. ActiveMQ Broker
47 RED HAT | Fuse Rocks!
47
Topics
Queues
Message Store
JMX JaaS Advisor
y
schedul
er

48. ActiveMQ Broker
48 RED HAT | Fuse Rocks!
48
Topics
Queues
Message Store
JMX JaaS Advisor
y
schedul
er
Broker

49. ActiveMQ with embedded Camel
<beans>
<broker brokerName="testBroker"
xmlns="http://activemq.apache.org/schema/core">
<transportConnectors>
<transportConnector uri="tcp://localhost:61616"/>
</transportConnectors>
</broker>
<import resource="camel.xml"/>
</beans>
49 49 RED HAT | Fuse Rocks!

50. Setup Camel Context in the usual way
<camelContext errorHandlerRef="errorHandler" xmlns=http://camel.apache.org/schema/spring>
<route>
<from uri="activemq:queue:test.queue"/>
<choice>
<when>
<xpath>$foo = 'bar'</xpath>
<to uri="activemq:topic:topic.bar"/>
</when>
<when>
<xpath>$foo = 'cheese'</xpath>
<to uri="activemq:topic:topic.cheese"/>
</when>
<otherwise>
<to uri="activemq:topic:topic.all"/>
</otherwise>
</choice>
</route>
</camelContext>
50 50 RED HAT | Fuse Rocks!

51. Scaling Networks with Camel
51 RED HAT | Fuse Rocks!
51
• Allows for non-chatty networks to be established
• Routing information can be externalized to the broker
• Successfully used in production for 1000’s of brokers
today

52. Messaging:
More Problems with large
deployments
52 RED HAT | Fuse Rocks!

53. Deployment and Maintenance
53 RED HAT | Fuse Rocks!
53
Main problems
Installing brokers on multiple hosts
ssh, untar, set directories and environment
Setting configuration manually for every broker
copying xml config, tweaking, testing
Updating configuration across cluster
Upgrading brokers

54. Best practices for ActiveMQ only
54 RED HAT | Fuse Rocks!
54
• Keep XML as a template and configure node-specific
details through properties
• Keep configuration in SVC system (git, svn, …)
• Keep configuration separate from installation for
easier upgrades

55. Clients
55 RED HAT | Fuse Rocks!
55
• Topology is very “static”
• Clients need to be aware of topology
• Clients need to know broker locations
• Changes are not easy as clients need to be updated
• Adding new resources (brokers) requires client updates
• Not suitable for “cloud” deployments

56. Fabric8 to the rescue!
56 RED HAT | Fuse Rocks!

57. Messaging:
Use cases
57 RED HAT | Fuse Rocks!

58. Not space - but close
58 RED HAT | Fuse Rocks!
58

59. Running the biggest machine on the planet
59 RED HAT | Fuse Rocks!
59

60. Messaging:
The Internet of Things
60 RED HAT | Fuse Rocks!

61. The Internet of Things (IoT)
61 RED HAT | Fuse Rocks!
61
• Uniquely identifiable objects and their virtual
representations in an internet-like structure
• Originally defined by Kevin Ashton, co-founder of Auto-ID
center at MIT
• Today its meaning has evolved to encompass a world
where physical objects are integrated into the information
network, and where physical objects participate in business
processes.

62. Telemetry
62 RED HAT | Fuse Rocks!
62
• M2M involves collection and transmission of event level
data (used to be called telemetry) from intelligent devices
• Machines can interact over the net – and/or attached
networks – by wired and wireless networks
• Payloads are typically minimal (temperature, pressure,
location, metering etc.)

63. IoT predicted Growth
50 Billion
10 Billion
RFID tags for
Inventory control
1 Billion
Ubiquitous
positioning –
locating people and
everyday objects
Geo-located devices
Microcontrollers
internet enabled
63 RED HAT | Fuse Rocks!
63
Vending machines, lone
workers, POS, vehicles,
planes, tolls, power
systems, transport,
telemedicine, home
appliances, power
systems, etc.
Time
Connected Devices on the
internet
Advanced sensors,
Enterprises fully
connected: integration
of everything
2000 2010 2020

64. Collect IOT information
Messaging Platform
(Multi-protocol,
Multi-platform)
JBoss A-MQ
Mobile
wireless
Medical Devices
Industrial
Machines
Distribution/Deli
very Vehicles
Energy Monitoring
Equipment
Sensors
MQTT
AMQP
64 RED HAT | Fuse Rocks!
Aggregate IOT data
reliably
Real-time notification
Lightweight, embeddable
messaging

65. Integrate IOT information
Messaging Platform
(Multi-protocol,
Multi-platform)
JBoss A-MQ
IOT data
Enterprise Service
Bus
(Mediate,
Transform, Route,
Connect)
JBoss Fuse
65 RED HAT | Fuse Rocks!
SaaS/Cloud
application
s
Enterprise
application
s
CRM
application
Unlock
information,
services,
processes
Lightweight, embeddable,
enterprise integration
pattern based platform

66. IOT – Red Hat JBoss solution
66 RED HAT | Fuse Rocks!

67. Messaging:
IoT Demo
67 RED HAT | Fuse Rocks!

68. What is Arduino ?
68 RED HAT | Fuse Rocks!
68
• Open Source Hardware microcontroller
• Cheap and easily available.
• Open Source Software.
• Widespread: many projects.
• Extra HW (shields) available

69. USB
7-12 v
3 v GRD5 v
GRD
69 69 RED HAT | Fuse Rocks!
Transmitter/Receiver
Serial Connection
Microcontroller
ATmega328
Operating Voltage 5V
Input Voltage
(recommended)7-12V
Input Voltage (limits)6-
20V
Digital I/O Pins 14
(of which 6 provide PWM
output)
Analog Input Pins 6
DC Current per I/O Pin
40 mA
DC Current for 3.3V Pin
50 mA
Digital Input/Output Pins
Pins with ~ are PWM
[Analog Output]
Analog Input Pins

70. Arduino Programming
70 RED HAT | Fuse Rocks!
70
• 3 types of memory on Arduino
Flash memory (program space) – 32 Kb
SRAM – used by sketches – 2 Kb
EPROM – long term memory – 1 Kb
• Programming language based on Wiring: C/C++ library
designed to make input/output easier. Programs are called
sketches.
• Arduino has a simple IDE, available on Windows, Linux and
Mac

71. Arduino MQTT to WebSockets
71 RED HAT | Fuse Rocks!
71
Arduino
AMQP
MQTT
ActiveMQ
MQTT/AMQP
Gateway
ActiveMQ
AMQP/WebSockets
Broker HTML 5/WebSockets

72. QQuueessttiioonnss
aanndd
DDiissccuussssiioonn
72 RED HAT | Fuse Rocks!