#AI + #ML + #Robotics combination is a game-changer, so #ServerlessTO members were lucky to have Alex Barbosa Coqueiro - Public Sector Solutions Architect Manager at AWS Canada, introduce us to AWS Robomaker & AWS DeepRacer! Alex also talked about managed #ReinforcementLearning (RL) with Amazon SageMaker, and compared DeepRacer to Donkey Car – open source project for small-scale self-driving cars. Video at: https://youtu.be/t8bo9gOveoo

1. Wednesday, July 31, 2019
1. Intro & Activity Update
2. Community Open Mic
3. Alex Barbosa Coqueiro - Public
Sector SA Manager @ AWS
Canada: “Racing with AI”
4. Networking
1
ServerlessToronto.org Meetup Agenda

2. Why we do what we do?
2
Serverless is New Agile
Serverless Dev (gluing
other people’s APIs
and managed services)
We're obsessed
helping Startups (and
creating meaningful
MVPs & products)
We build bridges
between Serverless
Community (“Dev leg”),
and Front-end & Voice-
First folks (“UX leg”),
and empower UX
developers
Achieve agility NOT by
“sprinting” faster (like in
Scrum), but by working
smarter (by using
bigger building blocks
and less Ops)

3. Why AI/ML topics at Serverless meetup?
3
AWS Machine Learning Stack

4. #ServerlessTO
Sponsors
4

5. Manning Publications 2019 giveaways:
1. www.manning.com/books/serverless-applications-with-nodejs
2. www.manning.com/livevideo/production-ready-serverless
3. www.manning.com/livevideo/production-ready-serverless
4. www.manning.com/livevideo/serverless-applications-with-AWS
5. www.manning.com/livevideo/serverless-applications-with-AWS
6. www.manning.com/books/serverless-architectures-on-aws
7. www.manning.com/books/http2-in-action
8. www.manning.com/books/event-streams-in-action
9. www.manning.com/books/the-design-of-everyday-apis
10. www.manning.com/livevideo/graphql-in-motion
11. www.manning.com/books/voice-applications-for-alexa-and-google-assistant
12. www.manning.com/livevideo/machine-learning-for-mere-mortals
13. www.manning.com/books/classic-computer-science-problems-in-python
14. www.manning.com/books/getting-mean-with-mongo-express-angular-and-node
5Check out MEAP program from our Learning Sponsor!

6. Venue Sponsor
6
As Certified B Corporation, Myplanet is purpose-driven and
creates benefit for all stakeholders, not just shareholders!

7. Catering Sponsor
7
Also an active Serverless Community member – check out
their “Data Pipelines using Serverless Architectures” talk!

8. Devinity
8

9. Devinity… continued
9

10. Community Open Mic
10
10 seconds of freedom
to pitch yourself, or
your company

11. Feature Talk
Racing with AI
11

12. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Racing with Artificial Intelligence
Alex Coqueiro
Head of Public Sector Solutions Architecture for Canada, Latin America and Caribbean
AWS
@alexbcbr

13. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Rubik’s cube challenge

14. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
43,252,003,274,489,856,000
43 QUINTILLION
UNIQUE COMBINATIONS

15. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Don’t code the patterns, let the
system learn through data

16. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
F2 U' R' L F2 R L' U'
ModelData

17. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
F2 U' R' L F2 R L' U'
Confidence
1%
accuracy
R U r U R U2 r U2%
accuracy
Training Models
Model

18. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Confidence
20%
accuracy
40%
accuracy
60%
accuracy
80%
accuracy
95%
accuracy
2%
accuracy
Training Models
Model

19. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Confidence
95%
accuracy
?
F2 R F R′ B′ D F D′ B D F
Inference
Model

20. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

21. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
SOLVED IN 0.9 SECONDS

22. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Let’s apply it into the business

23. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
https://youtu.be/xC-tikvEvzo

24. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
10,000+ customers | 2x customer references | 85% of TensorFlow projects in the cloud happen on
AWS

25. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
LEA (Robot Care Systems)
https://youtu.be/r-US8rs8EY0

26. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
tuSimple (Autonomous Vehicle)

27. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Show me how to do it

28. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.Use Case – Autonomous Driving

29. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Our problem re-formulation

30. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Different problems require different learning strategies
labeled training data
Complexityofdecisions
Supervised learning
Non-labeled training data

31. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Autonomous Driving Development

32. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Robocar (Donkey Car Project)
Donkey Car Project
https://github.com/sunilmallya/donkey/tree/master/sagetrain
http://awsrobocar.s3-website-us-east-1.amazonaws.com/
https://github.com/tescal2/donkey

33. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
SSD MultiBox — Real-Time Object Detection +
Behavioral Cloning

34. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon SageMaker:
Build, Train, and Deploy ML Models at Scale
1
2
3

35. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Different problems require different learning strategies
labeled training data
Complexityofdecisions
Supervised learning
Unsupervised
learning
Reinforcement
Learning
Non-labeled training data

36. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Reinforcement learning in the real world
Reward positive
behavior
Don’t reward
negative
behavior
The result!

37. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
• Build machine learning models in Amazon
SageMaker
• Train, test, and iterate on the track using the AWS
DeepRacer 3D racing simulator
• Compete in the world’s first global autonomous
racing league, to race for prizes and a chance to
advance to win the coveted AWS DeepRacer Cup
AWS DeepRacer
A fully autonomous 1/18th-scale race car designed to help you learn about
reinforcement learning through autonomous driving

38. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Robotic autonomous
race car
DeepRacer: An exciting way for developers to get hands-on experience with
Reinforcement Learning
Racing LeagueVirtual simulator, to
train and experiment

39. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Track components
TRACK CENTER
TRACK WALL
TRACK SURFACE aka ON-TRACK
FIELD aka OFF-TRACK
TRACK BOUNDARIES

40. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Action space

41. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
The reward function in a race grid
S G = 2
GOALAGENT

42. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Incentivizing centerline behavior
0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1
S 2 2 2 2 2 2 G = 2
0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1
REWARD FUNCTION
8.6 9.5 8.5 7.5 6.3 5.0 3.5 1.9
S 10.4 9.4 8.2 6.9 5.4 3.8 G = 2
8.6 9.5 8.5 7.5 6.3 5.0 3.5 1.9
MAX VALUE OF EACH STATE
AFTER LOTS OF EXPLORING
Discount per step
0.9

43. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Programming your own reward function

44. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Let’s go deeper
Let’s go deeper…

45. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS DeepRacer Neural Network Architecture
An overview of the network architecture that AWS DeepRacer uses:
Output

46. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon
Sagemaker RL

47. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Reinforcement Learning Algorithms Compared
Value Approximation Policy Approximation
Advantages
More stable performance when it works, and tends to
converge on global optimum
Effective in continuous action spaces, can learn stochastic policies,
and faster convergence
Disadvantages
Difficult to converge if too many (state, action)
combinations, slower convergence in general, and can’t
learn stochastic properties
Typically converges to a local rather than global optimum, high
variance in estimating the gradient adversely affects stability, and
evaluating a policy is generally inefficient
Examples Q-Learning, Deep Q Network, Deep Double Q Network Policy Gradient, Proximal Policy Optimization (PPO)

48. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Hyper parameters control the training algorithm

49. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Coordinate system and track waypoints
OUTER BOUNDARY WAYPOINTS
TRACK CENTER WAYPOINTS
INNER BOUNDARY WAYPOINTS
X
Y
TRACK WIDTH
CAR DIRECTION

50. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS DeepRacer Car Specifications
CAR 18th scale 4WD with monster truck chassis
CPU Intel Atom™ Processor
MEMORY 4GB RAM
STORAGE 32GB (expandable)
WI-FI 802.11ac
CAMERA 4 MP camera with MJPEG
DRIVE BATTERY 7.4V/1100mAh lithium polymer
COMPUTE BATTERY 13600mAh USB-C PD
SENSORS Integrated accelerometer and gyroscope
PORTS 4x USB-A, 1x USB-C, 1x Micro-USB, 1x HDMI
SOFTWARE Ubuntu OS 16.04.3 LTS, Intel® OpenVINO™
toolkit, ROS Kinetic

51. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Robotic Arms
International Space Station
Drones
Education
Water
Home
Self-Driving Vehicles
Autonomous Walker
Rover
Robot landscape

52. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Robotics trends
in 2018
Robotics is undergoing fundamental
change in collaboration, autonomous
mobility, and increasing intelligence
Source: IDTechEx
• Logistics
• Construction
• Retail
• Hospitality
• Healthcare
Robots are being put to work every
day across many industries
• Agriculture
• Energy Management
• Oil and Gas
• Facilities Management
• Household chores
By 2023, it’s estimated that mobile autonomous robots will
emerge as the standard for logistic and fulfillment processes
By 2030, 70% of all mobile material
handling equipment will be autonomous

53. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Robotic development cycle
2) Develop
robotics
application
1) Select robotics
software
framework
1) Deploy and
manage
application
3) Test and
simulate
application
New application release and update

54. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Most widely used software framework for teaching and learning about robotics – over 16 million .deb (Linux Debian)
packages downloaded in 2018, a 400% increase since 2014
Founded in Stanford labs over 10 year ago, now managed by the Open Source Robotics Foundation (OSRF)
Global open-source community supports two products—Robot Operating System (ROS) and Gazebo
ROS
A set of software libraries and tools, from drivers to algorithms,
that help developers build robot applications
Gazebo
Robust physics engine, high-quality graphics, and programmatic
and graphical interfaces to help developers simulate robots
Robot Operating System (ROS) primer

55. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Introducing AWS RoboMaker
A service that makes it easy for
developers to develop, test, and
deploy robotics applications, as
well as build intelligent robotics
functions using cloud services

56. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS RoboMaker service suite
Development
Environment
SimulationCloud Extensions for
ROS
Fleet
Management

57. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS RoboMaker
Sample Robot Applications
Hello
World
Navigation
and Person
Recognition
Voice
Commands
Robot
Monitoring
Object-
following using
RL
Self-
driving
using RL

58. AWS Cloud
AWS
DeepRacer
NAT gateway
VPC
AWS DeepRacer
Models
Simulation
video
Metrics
AWS DeepRacer Simulation Architecture

59. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
ROS Msg Node
Stored File
ROS Nodes
Model
Optimizer
Video
M-JPEG
Web Server
Video
Inference
Results
Web
Server
Publisher
Autonomous
Drive
Control
Node
Optimized
Model
Media engine
Camera
Model
Inference
engine
Manual
Drive
Navigation
Node
Servo & Motor
AWS DeepRacer Software Architecture

60. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
DATA

61. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
We are not spectators,
but actors of the future
Herb Simon,
2000

62. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
ml.aws
@alexbcbr

63. Thank you!

64. Future Talks
2019
1

65. Upcoming Serverless Toronto Meetups
2
1. August 12, 2019: 1. A look at Google Cloud Functions
– Matt Welke // 2. Firebase Crash Course – Kudz Murefu
2. September 12, 2019: Serverless Design Patterns and
Best Practices – Mike Apted, Startup SA at AWS
3. October, 2019: Serverless CI/CD pipelines with AWS
CodePipeline and CodeBuild, vs CircleCI, vs Travis, vs
Seed – Frank Wang & Jay V
4. November or December: Serverless Heroes (authors
of “Serverless Applications with Node.js”) Slobodan
Stojanović & Aleksandar Simović will be here!
5. December or January, 2020: re:Invent recap –
Jonathan Dion, Senior Technical Evangelist at AWS