Embedded Java and MQTT

1
© 2013 IBM Corporation
MQTT and Java
Peter Niblett
IBM Senior Technical Staff Member
OASIS MQTT Technical Committee
Eclipse M2M Working Group

InfoQ.com: News & Community Site
• 750,000 unique visitors/month
• Published in 4 languages (English, Chinese, Japanese and Brazilian
Portuguese)
• Post content from our QCon conferences
• News 15-20 / week
• Articles 3-4 / week
• Presentations (videos) 12-15 / week
• Interviews 2-3 / week
• Books 1 / month
Watch the video with slide
synchronization on InfoQ.com!
http://www.infoq.com/presentations
/embedded-java-mqtt

Presented at QCon New York
www.qconnewyork.com
Purpose of QCon
- to empower software development by facilitating the spread of
knowledge and innovation
Strategy
- practitioner-driven conference designed for YOU: influencers of
change and innovation in your teams
- speakers and topics driving the evolution and innovation
- connecting and catalyzing the influencers and innovators
Highlights
- attended by more than 12,000 delegates since 2007
- held in 9 cities worldwide

22 © 2013 IBM Corporation
Mobile is a Prelude to the “Internet of Things”

3
Agenda
What is MQTT?
– What is it for?
– How does it compare with HTTP?
– What's its relationship with JMS and WebSocket?
Details of MQTT – the protocol
– Protocol features
– Example data flows
Developing with MQTT
– What you need
– Java API walkthrough

An Open Approach to Connectivity for Mobile, M2M and IoT
MQTT is a lightweight publish/subscribe protocol with reliable bi-directional
message delivery
Lossy or
Constrained
Network
Lossy or
Constrained
Network
Monitoring &
Analytics
Server
Real-World Aware Business Processing
High volumes of data/events
IT Systems
In this arena, open source and standards are
essential
1999 Invented by Dr. Andy Stanford-Clark (IBM),
Arlen Nipper (now Cirrus Link Solutions)
1999 Invented by Dr. Andy Stanford-Clark (IBM),
Arlen Nipper (now Cirrus Link Solutions)
2011 - Eclipse PAHO MQTT
open source project
2011 - Eclipse PAHO MQTT
open source project
2004 MQTT.org open community2004 MQTT.org open community
2013 – MQTT Technical
Committee formed
2013 – MQTT Technical
Committee formed
Cimetrics, Cisco, Eclipse,
dc-Square, Eurotech, IBM,
INETCO Landis & Gyr, LSI,
Kaazing, M2Mi, Red Hat,
Solace, Telit Comms,
Software AG, TIBCO,
WSO2
Evolution of an open technology

Sensors ActuatorsEmbedded
Controllers
Sense and ControlVisualise and Respond
Intelligence
and Analytics
Messaging server
Edge
Gateways
Mobile Web
Systems of
Record
BigData
Sense
Data/Alert
Respond
ControlM2M
What’s MQTT for?
MQTT

6
Volume (cost) of data being transmitted
(especially in M2M with limited data plans)
Power consumption (battery powered
devices)
Responsiveness (near-real time delivery of
information)
Reliable delivery over fragile connections
Security and privacy
Scalability
Some challenges facing Mobile and M2M developers

7
Original MQTT design principles
■ Expect and cater for frequent network
disruption – built for low bandwidth, high
latency, unreliable, high cost networks
■ Expect that client applications may have
very limited resources available.
■ Minimal the on-the-wire footprint.
■ Publish/subscribe messaging paradigm as
required by the majority of SCADA and
sensor applications.
■ Provide traditional messaging qualities of
service where the environment allows.
■ Publish and standardize the protocol for
ease of adoption by device vendors and
third-party client software.

8
Central
Systems
●
Monitoring
- temp, pressure...
●
Control
- valves…
4000 devices integrated, need to add 8000 more BUT:
• Satellite network saturated due to polling of device
• VALMET system CPU at 100%
• Other applications needed access to data ("SCADA prison")
Proprietary polling protocol
Billing
Maintenance
SCADA
low-bandwidth,
expensive comms
Real Life - Pipeline integration
challenges

9
Central
Systems
Billing
Maintenance
SCADA
low-bandwidth,
expensive comms
Scalability for whole pipeline!
Network traffic much lower - events pushed to/from devices and report by exception
Network cost reduced
Lower CPU utilization
Broken out of the SCADA prison – data accessible to other applications
Message Broker
pub sub
transformation
Enterprise MessagingMQTT
20 Field
Devices to 1
Concentrator
Enterprise to physical world solution
with MQTT

Industry Examples of use
Automotive Connected Cars
Healthcare Medical Devices
Home monitoring
Transport & Supply Chain Asset tracking
Ticketing
Oil and Gas Pipeline management
Energy & Utilities Smart metering and demand
management
Manufacturing Process Control
Finance Customer alerting
Telecomms Generic push notification
M2M services
Mobile apps Facebook Messenger
Some areas where MQTT is being used

11
And Home Hackers as well

MQTT serverMQTT serverMQTT serverMQTT server
Reliable
asynchronous
transactions
User submits a transaction. One
or more responses may come
back over time.
MQTT provides reliability and store/forward of
requests and responses if needed – reducing the
amount of application code
Continuous
update of
real-time
information
Server-side data is “streamed” to
the device and used to update
the UI. In most cases this is only
required when the app is in the
foreground
Small MQTT header size reduces battery
consumption and network traffic. One->many
publish/subscribe reduces load on application
Push
Notification
Sending alert or other
informational message to the
device. The app may or may not
be running at the time.
Avoidance of polling reduces battery
consumption and network traffic.
Store/forward of important notifications if
app/device is not contactable
Collection of
data from
device
Data sent to the server coming
either from User Interface, of
from onboard sensors or from
devices attached to the phone
Small MQTT header size reduces battery
consumption and network traffic.
Store/forward of messages.
One->many publish/subscribe
Mobile Usage patterns
MQTT

13
• The HTTP standard revolutionized how we consume
data
‒ A single simple model: Send a request, read the
response
‒ Available via any tablet, laptop, phone, PC etc.
‒ Good for requesting data from a known source
• However, HTTP wasn’t designed for mobile use
• MQTT provides support for transactions and an
event-oriented paradigm, engineered around wireless:
Reliably and securely completing mobile business
transactions over unreliable networks
Pushing information over unreliable networks
Transmitting information one to many
Listening for events whenever they happen
Distributing minimal packets of data in huge
volumes
Why isn't HTTP enough?

14
MQTT compared with...
Java Message Service (JMS)
– JMS is a fully-featured messaging API designed for Enterprise apps. It
contains additional features that aren't included in MQTT
• Rich set of message headers
• Explicit Map and Stream message body types
• Rollback transactional capabilities
• Explicit point-point domain
– JMS and MQTT can inter-operate via a shared topic space (IBM
implementations do this).
 WebSocket
– Provides basic (socket like) bidirectional comms to web apps.
– MQTT can run over a WebSocket to provide reliable delivery and
publish/subscribe to HTML5 apps in a manner that
• Reduces the coding required by the application developer
• Is inter-operable with other channels and applications

A producer publishes a message (publication) on a topic (subject)
A consumer subscribes (makes a subscription) for messages on a topic (subject)
A message server matches publications to subscriptions
If none of them match the message is discarded
If one or more matches the message is delivered to each matching consumer
Publish / Subscribe has three important characteristics:
1. It decouples message senders and receivers, allowing for more flexible applications
2. It can take a single message and distribute it to many consumers
3. This collection of consumers can change over time, and vary based on the nature
of the message.
Publish / Subscribe Messaging (One to Many)

16
MQTT Topics
All subscriptions are to a topic space
All messages are published to an individual topic
Topic names are hierarchical
– Levels separated by “/”
– Single-level wildcards “+” can appear anywhere in
the topic string
– Multi-level wildcards “#” must appear at the end of
the string
• Wildcards must be next to a separator
• Can't use wildcards when publishing
MQTT topic names can be 64KB long
Fruit
Grape
Red White
Fruit/#
Fruit/Grape/+
Fruit/+/Red

17
Agenda
What is MQTT?
– What is it for?
– What you need

18
MQTT Protocol Deep Dive - Headers
MQTT protocol flows:
– Fixed header (2 bytes)
– Variable header (optional, length varies)
– Message payload (optional, length encoded, up to 256MB)
Fixed header indicates the Message Packet type, the length of
the payload and Quality of Service
Variable header contents depends on the Message Packet type
– Message ID, Topic name and so on.
Fixed Variable Payload

19
A 2 byte flow? What can we do with that?!
Each bit in each byte is important!
Disconnect and pings only
need the fixed header
Remaining length allows for a
256MB payload
– Use 1 byte for up to 127 bytes,
– 2 bytes for up to 16383 bytes
– Max. 4 bytes = 256MB

20
MQTT API Flows
Most flows have a corresponding
acknowledgment
Connect
– Can restart a previous session
– Can specify a “Last Will and Testament”
message and topic
Subscribes can specify multiple topics
Publish flows
– Flow depends on QoS level
– Sent from client → server to publish a
message, or
– Server → client to send messages
Connection
Management
CONNECT
CONNACK
DISCONNECT
PINGREQ
PINGRESP
Subscription
Management
SUBSCRIBE
SUBACK
UNSUBSCRIBE
UNSUBACK
Message
Delivery
PUBLISH
PUBACK
PUBREC
PUBREL
PUBCOMP

21
Qualities of Service
QoS 0: At most once delivery (non-
persistent)
– No retry semantics are defined in the protocol.
– The message arrives either once or not at all.
QoS 1: At least once delivery (persistent,
dups possible)
– Client sends message with Message ID in the message header
– Server acknowledges with a PUBACK control message
– Message resent with a DUP bit set If the PUBACK message is not
seen
QoS 2: Exactly once delivery (persistent)
– Uses additional flows to ensure that message is not duplicated
– Server acknowledges with a PUBREC control message
– Client releases message with a PUBREL control message
– Server acknowledges completion with a PUBCOMP control message

22
MQTT Keep Alive
Protocol includes support for client and server to detect failed
connections
– At connection time, a keep alive can be specified
If client does not send a PINGREQ request to the server, the
server assumes the connection has failed.
If client does not receive a PINGRESP after the keep alive time
has passed after sending the request, the connection is failed.
Maximum keep alive interval of 18 hours.
– Can specify a value of 0 to disable keep alive
??

23
MQTT Last Will & Testament
During connection, a Will message and topic can be
specified
– Abnormal disconnections will cause WMQ to publish the
message
– Clean disconnects will not cause the message to publish
Can set the message as retained
– Message is published to a subscriber when registering
Useful to report the connection status of the client
– Will message is a retained “down”
– Upon connecting, client publishes a retained “up” message.

24
The life of a MQTT client
MQTT serverCONNECT
SUBACK
SUBSCRIBE
PINGREQ
CONNACK
PINGRESP
Has a subscriber
connected on a topic
Is connected, and
is awaiting
messages
Is the connection
still active? Yes!

25
The life of a MQTT client (2)
MQTT Server
PUBLISH
PUBLISH
PUBREC
Send some important
messages
Send some low
importance messages
PUBREL
PUBCOMP
DISCONNECTI'm done!

26
Agenda
What is MQTT?
– What is it for?
– What you need

27
MQTT server options
 Commercial
‒ IBM WebSphere MQ
‒ IBM IBM MessageSight (Appliance)
‒ Software AG my-Channels Nirvana
‒ dcSquare HiveMQ
 Open Source or Free download
‒ Mosquitto (mosquitto.org)
‒ RSMB (IBM developerWorks)
‒ ActiveMQ (Apache)
‒ RabbitMQ (vmWare)
‒ mqtt.js , eMQTT (GitHub)
 Web-hosted
‒ Eclipse Paho (implemented using Mosquitto)
‒ Everywhere Device Cloud
‒ m2m.io

28
MQTT Clients and APIs
The published wire format is useful when implementing very
lightweight sensors
However, a prebuilt client library simplifies MQTT programming
– No need to know the bits and bytes flowing over the wire
– No need to code thread management for background processing
– Makes code more maintainable
Open Source clients available in Eclipse Paho project
– Java, C and JavaScript
Clients for other languages are available, see mqtt.org/software
– C++, Delphi, Erlang, Lua, .Net, Objective-C, PERL, PHP, Python,
Ruby

29
Eclipse Paho Java APIs for MQTT
MqttClient - “Synchronous” or “blocking” API
– This was the original API
– Some processing is done on background threads, but most calls
block until their processing is complete
– Slightly simpler to program to than the Asynch API
MqttAsyncClient – New “Asynchronous” API
– Better fit to asynchronous environments, e.g. Android
– All significant processing is done on background threads
The synchronous client is actually implemented as a thin layer
on top of the Asynchronous one
Both interfaces are included in the same Jar file
– org.eclipse.paho.client.mqttv3.jar

30
The synchronous Java API
 MqttClient object is the starting point
 Incoming messages are received via callbacks
 Messages can be published synchronously or asynchronously
 Local persistence stores used to for QoS 1 and 2
– Persistence store implementation can be replaced
MqttClient
subscribe
MqttConnectOptions
MqttCallback
MqttClientPersistence
MqttTopic
setCallBack
MqttMessage
connect
getTopic
publish
publish

31
Synchronous API : Connecting
 The first step is to create an MqttClient object. This represents the
connection of an application to the MQTT server
– Specify details of the server (in a URI format), a ClientID and
optionally a persistence implementation
– The ClientID must be unique for the server that it is connecting to
 Then specify connection options, and connect. In this example:
– Keep alive of 480 seconds
– A retained publication Will message of QoS 1

32
Synchronous API : Sending a Message
 The Message object contains a payload and a few message properties, such as
the QoS level.
– The payload is always a byte array, though this might be an encoded string
 Messages can be sent synchronously, using the MqttClient object
– This call does not complete until all acknowledgements (if any) have been
received:
 Messages can be sent asynchronously, using an MqttTopic object:
Both forms are overloaded so you don't have to create an MqttMessage object first

33
Synchronous API : Subscriptions
After connecting, subscribe by providing the topic string:
You can subscribe to multiple topics at the same time, and
provide QoS levels:
Messages will then be delivered to the messageArrived callback
Call unsubscribe to stop getting messages. Again, you can
optionally provide a list of containing multiple topics:

34
Asynchronous MQTT API
To use the new non-blocking API, you use an MqttAsyncClient
object instead of using MqttClient
– In this class the methods for connect, disconnect, publish,
subscribe and unsubscribe all operate asynchronously
– As you will see, it's quite easy to achieve the effect of a blocking
call using the new MqttAsyncClient. We therefore recommend that
you use MqttAsyncClient in place of MqttClient
You construct the MqttAsyncClient and set callbacks just like
you do for the MqttClient:

35
Asynchronous MQTT API
 MqttAsyncClient provides two asynchronous invocation styles
 In the first style, each call looks like its MqttClient counterpart except that it
returns a token, which you can then use to track completion of the operation:
– You can register a listener callback against the token which will get called when the
operation completes:
– You can block on the token waiting for its completion, which in effect makes the call
into a blocking call:
 In the second style you supply the listener callback as a parameter on the
method invocation itself.

36
Asynchronous API: Listener callbacks
 The ActionListener contains two callback methods, one for Success and one
for Failure:
 Applications may pass application-specific context in the iMqttToken which can
then be used by the callback method implementations

37
Receiving messages – both APIs
Messages are delivered to the app via a callback mechanism
– Callbacks are also used to indicate when the connection has
broken, and when an asynchronous publish has has completed
Note that the messageArrived callback is scoped to the
MqttClient or MqttAsyncClient object and is driven whenever a
message arrives on any of its subscriptions.

38
Further notes
To resume a previous session, set the clean session option to
false when connecting
– You must also use the same client identifier that was used last time
If you set cleanSession to true, you still need to provide a clientID,
there's a convenience method to generate a random clientID for
this case
Client-side state is persisted using the initial URI and clientID as its
key.
– The latest version of the MQTT client has the capability to connect to
a server which has a different URI, using the same client state. This
is for HA configurations where you want to failover to a server which
has a different IP address

39
MQTT
Messaging optimized for mobile, smart sensors and telemetry devices
Enables intelligent decision-making based on remote real-world events
Management of static or moving assets, people, locations
Simple APIs for Java, JavaScript and other languages reduce the
burden for application developers
An open protocol with Industry leadership & mindshare
• MQTT Protocol and client code contributed to open source
• see MQTT.org and Eclipse Paho
• Open licence allows development communities to provide further
client code & device support
• 16+ MQTT servers and 40+ MQTT clients
• Standardisation process is under way at OASIS
Summary

40
Useful Links
MQTT information
– http://mqtt.org
MQTT 3.1 Specification
– http://www.ibm.com/developerworks/webservices/library/ws-mqtt/index.html
RSMB (server implementation)
– http://www.alphaworks.ibm.com/tech/rsmb/
Eclipse Paho project
– http://www.eclipse.org/paho
Eclipse M2M Industry Working Group
– http://wiki.eclipse.org/Machine-to-Machine
OASIS MQTT Technical Committee
– https://www.oasis-open.org/committees/tc_home.php?wg_abbrev=mqtt
MQTT: the Smarter Planet Protocol
– http://andypiper.co.uk/2010/08/05/mqtt-the-smarter-planet-protocol

Watch the video with slide synchronization on
InfoQ.com!
http://www.infoq.com/presentations/embedded
-java-mqtt

Embedded Java and MQTT

Recomendados

Recomendados

Más contenido relacionado

Destacado

Destacado (10)

Más de C4Media

Más de C4Media (20)

Último

Último (20)

Embedded Java and MQTT