Is your project dragging you down? Are you stuck with the same old technologies? Are you bored with coding? If you answer “yes” to any of these questions, you may have lost your coding mojo—just like this session’s speaker did a few years back. Come hear how he learned new technologies and rediscovered his coding mojo by building an IoT/robotics prototype: a voice-controlled robot. Along the way, you’ll hear about HTML5 speech recognition, controlling hardware with Node.js and Johnny-Five, using WebSocket and MQTT for communication between components, and finally how you can easily combine the Raspberry Pi and Arduino platforms to gain ultimate power over your own projects.

1. How
I
Got
Back
my
Coding
Mojo!
Mark
West

2. Safe
Harbour

3. What
is
Mojo?

7.
“Any
applica+on
that
can
be
wri3en
in
JavaScript,
will
eventually
be
wri3en
in
JavaScript”
James
Atwood
(founder,
stackoverflow.com)

10. NodeBot
Rover

11. NodeBot
Rover
Demo

12. NodeBot
Rover
Component
Overview
Hardware
Computer
Micro
Controller
Camera
Chassis
Pan
&
Tilt
SoAware
Speech
Robot
Control
Web
Server
IntegraFon
Other

13. The
Arduino
Ecosystem
• MicroController
plaLorm.
• Many
different
Arduino
models.
• Open
Source
design.
• MicroControllers
extensible
via
“Shields”.

14. PuSng
the
Hardware
Together

18. Two
Servos
for
Camera
Pan
&
Tilt
Servos
wired
to
Arduino
Chassis
Raspberry
PI
In
PlasFc
Case
WIFI
Dongle

19. Raspberry
PI
Camera
Wired
directly
to
the
Raspberry
PI

20.
Actuators
Sensors
Nervous
System
Raspberry
PI
2
Raspberry
PI
Cam
Arduberry
Microcontroller
Arduino
Chassis
Servo
Servo
Brain
NodeBot
Rover
Hardware

21. Combining
the
Raspberry
PI
&
Arduino
Raspberry
PI
• Linux
PC.
• Supports
USB
peripherals.
• Programming.
Arduino
PlaNorm
• MicroController.
• Robust.
• Flexibility
(input/output).
The
whole
is
greater
than
the
sum
of
parts!

22.
Actuators
Sensors
Nervous
System
Raspberry
PI
2
Raspberry
PI
Cam
Arduberry
Microcontroller
Arduino
Chassis
Servo
Servo
Brain
NodeBot
Rover
Hardware

23. NodeBot
Rover
Component
Overview
Hardware
Raspberry
PI
Arduberry
(Arduino)
Pi
Cam
Zumo
Chassis
Servos
SoAware
Speech
Robot
Control
Web
Server
IntegraFon
Other

24.
Raspberry
PI
2
Arduberry
Servo
Soware
Communica]on
across
Hardware
Layers
Servo
Raspberry
PI
Chassis
JavaScriptPI
Binary
(Compiled
C
/
C++)

25. The
NodeBots
Movement
Source
:
nodebots.io

27. 1. Maturity
2. Community
3. DSL
4. Portability
5. Open
Source
6. Node.js
ecosystem
7. REPL
Benefits
of
using
Johnny-­‐Five

28. var five = require("johnny-five");
var myBoard = new five.Board();
myBoard.on("ready", function() {
var myLed = new five.Led(13);
myLed.blink(500);
this.repl.inject({
replLed: myLed
});
});
Imports
J5
Dependancy
Adds
LED
instance
to
REPL
Declares
LED
as
connected
to
UNO
Pin
13
Blinks
LED
every
500
milliseconds
Code
block
triggered
by
UNO
”Ready”
Event
Ini]alises
UNO
Johnny-­‐Five
Code
Example

29. hhps://vimeo.com/134953425

30.
Arduberry
Servo
Bridging
the
gap
with
Firmata
Servo
Raspberry
PI
Chassis
Johnny-­‐Five
(Firmata
Client)
Std.
Firmata
(Firmata
Server)
JavaScriptPI
Binary
(Compiled
C
/
C++)

32. GeSng
Started
with
Johnny-­‐Five
1. Buy
an
Arduino
Experimenters
Kit.
2. Follow
the
tutorials
at
hSp://node-­‐ardx.org.
3. Visit
hSp://johnny-­‐five.io
for
more
informa]on
and
inspira]on.

33. NodeBot
Rover
Component
Overview
Hardware
Raspberry
PI
Arduberry
(Arduino)
Pi
Cam
Zumo
Chassis
Servos
SoAware
Speech
Johnny-­‐
Five
Web
Server
IntegraFon
Other

34. Speech
Recogni]on
Requirements
Need
to
have
• Quality
and
speed
of
speech
recogni]on.
• Free,
no
restric]ons.
• Soware
based.
• JavaScript.
Nice
to
have
• Speech
to
text.
• One
stop
service.
• Bahle
tested.

37. Web
Speech
API
(limited
to
Google
Chrome)

38. Web
Speech
API
:
Configura]on

39. Web
Speech
API
:
Events

40. Web
Speech
API
:
Control

41. Speech
Demo

42. Buhon
press
and
release
connected
to
Web
Speech
API
start
and
stop
Control
methods.
Con+nuous
dicta+on
switched
on,
to
avoid
cuSng
commands
short.
Interim
results
switched
on
–
shown
in
green
text
(final
results
in
white).
Each
set
of
results
checked
for
uniqueness
to
avoid
sending
duplicate
commands
to
the
Robot.
Web
Speech
API
:
Robot

43. NodeBot
Rover
Component
Overview
Hardware
Raspberry
PI
Arduberry
(Arduino)
Pi
Cam
Zumo
Chassis
Servos
SoAware
HTML5
Speech
Johnny-­‐
Five
Web
Server
IntegraFon
Other

45. Node.js
Express
Web
Server
(17
lines
of
code)

46. NodeBot
Rover
Component
Overview
Hardware
Raspberry
PI
Arduberry
(Arduino)
Pi
Cam
Zumo
Chassis
Servos
SoAware
Speech
Johnny-­‐
Five
Express
IntegraFon
Other

47. Johnny-­‐Five
Process
Express
Process
Raspberry
PI
2
WE
Browser
HTTPS
NodeBot
Rover
Hardware
Firmata

48. Johnny-­‐Five
Process
Express
Process
Raspberry
PI
2
WE
Browser
HTTPS
NodeBot
Rover
Hardware
Firmata
MQTT
Message
Broker
MQTT
MQTT
over
WebSockets
Commands
Acknowledgements
MQTT.js
Client
MQTT.js
Client

49. Why
Add
a
Message
Broker?
• Sepera]on
of
concerns.
• Isolate
main
components
for
easier
tes]ng.

50. MQTT
–
MQ
Telemetry
Transport
• Internet
of
Things
connec]vity
protocol.
• Designed
to
be
lightweight
with
a
small
footprint
and
lihle
overhead.
• Is
a
protocol
and
a
Pub-­‐Sub
Message
Broker.
• Used
by
Facebook
for
pushing
updates
to
mobile
clients.

51. Adding
MQTT
to
the
NodeBot
Rover
Broker
• Public
MQTT
Broker
– Many
Public
Brokers
exist.
– One
less
process
to
run
on
Raspberry
PI.
Client
• MQTT.js
– Provides
an
MQTT
client
library
for
Node.js.
– Extremely
simple
to
use.
– Integrates
seamlessly
with
Johnny-­‐Five.

52. MQTT.js
(Node.js)
Client
Example

53. MQTT
Over
WebSockets
with
MQTT.js
• MQTT.js
is
a
Node.js
package
and
is
meant
for
use
on
the
Server
Side.
• To
allow
MQTT.js
to
run
from
the
Browser,
we
first
needed
to
”Browserify”
the
MQTT
library.
• We
could
then
access
the
”Browserified”
MQTT
client
library
from
the
Browser.
• This
solu]on
requires
that
your
MQTT
Broker
has
a
WebSocket
endpoint.

54. MQTT
Over
WebSockets
Demo

55. NodeBot
Rover
Component
Overview
Hardware
Raspberry
PI
Arduberry
(Arduino)
Pi
Cam
Zumo
Chassis
Servos
SoAware
HTML5
Speech
Johnny-­‐
Five
Express
MQTT
&
WebSocket
Other

56. Node.js
Twiher
Client
• Fully
fledged
Twiher
Client:
– Asynchronous.
– Supports
REST
API
(write
and
write)
– Supports
Streaming
API
(events
and
tweets).
• Requires
developer
creden]als
from
Twiher:
– Trivial
to
get
hold
of.

57. Node.js
Twiher
Client
REST
Example

58. Video
Streaming
via
Mo]on
• MoFon:
Soware
Mo]on
Detector.
• Provides
streaming
video
with
possibility
to
create
snapshots.
• Good
performance
on
the
Raspberry
PI.
• PotenFal
side
project:
get
Robot
to
follow
moving
objects?

59. Other
things
I
picked
up
• Git
/
GitHub
• HTML5
• CSS
• Linux

61. NodeBot
Rover
Component
Overview
Hardware
Raspberry
PI
Arduberry
(Arduino)
Pi
Cam
Zumo
Chassis
Servos
SoAware
HTML5
Speech
Johnny-­‐
Five
Express
MQTT
&
WebSocket
TwiSer/
MoFon/
Git/GitHub/
CSS/Linux

62. So
Did
I
Get
My
Coding
Mojo
Back?

64. Land
Turn
Take-­‐
off

67. @markawest