by Dario Rivera, Solutions Architect, AWS Artificial Intelligence (AI) services on the AWS cloud bring deep learning (DL) technologies like natural language understanding (NLU), automatic speech recognition (ASR), image recognition and computer vision (CV), text-to-speech (TTS), and machine learning (ML) within reach of every developer. In this session, you will be introduced to several new AI services: Amazon Lex, to build sophisticated text and voice chatbots; Amazon Rekognition, for deep learning-based image recognition; and Amazon Polly, for turning text into lifelike speech. The opportunities to apply one or more of these DL services are nearly boundless and this session will provide a number of examples and use cases to help you get started.

1. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Dario Rivera
AWS Solutions Architect
Sept 2017
AI and Deep Learning
at Amazon

2. A Flywheel For Data
Machine Learning
Deep Learning
AI
More Users Better Products
More Data Better Analytics
Object Storage
Databases
Data warehouse
Streaming analytics
BI
Hadoop
Spark/Presto
Elasticsearch
Click stream
User activity
Generated content
Purchases
Clicks
Likes
Sensor data

3. Machine Learning &
Artificial Intelligence
Big Data
More Users Better Products
More Data Better Analytics
A Flywheel For Data

4. Algorithms
Data
Programming
Models
GPUs &
Acceleration
The Advent of Deep Learning
image understanding
natural language
processing
speech recognition
autonomy

5. AI Applications on AWS
Zillow
• Zestimate (using Apache Spark)
Howard Hughes Corp
• Lead scoring for luxury real estate
purchase predictions
FINRA
• Anomaly detection, sequence matching,
regression analysis, network/tribe analysis
Netflix
• Recommendation engine
Pinterest
• Image recognition search
Fraud.net
• Detect online payment fraud
DataXu
• Leverage automated & unattended ML at
large scale (Amazon EMR + Spark)
Mapillary
• Computer vision for crowd sourced maps
Hudl
• Predictive analytics on sports plays
Upserve
• Restaurant table mgmt & POS for
forecasting customer traffic
TuSimple
• Computer Vision for Autonomous Driving
Clarifai
• Computer Vision APIs

6. AI Applications on AWS
Pinterest Lens
Netflix Recommendation Engine

7. Thousands Of Employees Across The Company Focused on AI
Discovery &
Search
Fulfilment &
Logistics
Enhance
Existing Products
Define New
Product
Categories
Bring Machine
Learning To All
Artificial Intelligence At Amazon

8. Artificial Intelligence
At Amazon (1995)

10. Can We Help Customers
Put Intelligence At The Heart Of
Every Application & Business?

11. Amazon AI
Intelligent Services Powered By Deep Learning

12. Amazon AI: New Deep Learning Services
Life-like Speech
Polly Lex
Conversational
Engine
Rekognition
Image Analysis
Deep Learning
Frameworks
MXNet, TensorFlow,
Theano, Caffe, Torch

13. DIY Deep Learning
for Custom Models
AI Enabled
Managed API
Services
Amazon AI: New Deep Learning Services
Polly LexRekognition
Deep Learning
Frameworks
MXNet, TensorFlow, Theano, Caffe, Torch
CONTROL
USABILITY&
SIMPLICITY

14. One-Click GPU
Deep Learning
AWS Deep Learning AMI
Up to~40k CUDA cores
MXNet
TensorFlow
Theano
Caffe
Torch
Pre-configured CUDA drivers
Anaconda, Python3
+ CloudFormation template
+ Container Image

15. MXNet: Scalable Deep Learning Framework

16. Converts text
to life-like speech
47 voices 24 languages Low latency,
real time
Fully managed
Polly: Life-like Speech Service
Voice Quality & Pronunciation
1. Automatic, Accurate Text Processing
2. Intelligible and Easy to Understand
3. Add Semantic Meaning to Text
4. Customized Pronunciation
Articles and Blogs
Training Material
Chatbots (Lex)
Public Announcements

17. 1st Gen: Machine-oriented
interactions
2nd Gen: Control-oriented
& translated
3rd Gen:
Intent-oriented
The Advent Of Conversational Interactions

18. Voice & Text
“Chatbots”
Powers
Alexa
Voice interactions
on mobile, web
& devices
Text interaction
with Slack & Messenger
Enterprise
Connectors
(with more coming) Salesforce
Microsoft Dynamics
Marketo
Zendesk
Quickbooks
Hubspot
Lex: Build Natural, Conversational
Interactions In Voice & Text
Improving human interactions…
• Contact, service, and support center interfaces (text + voice)
• Employee productivity and collaboration (minutes into seconds)

19. Origin
Destination
Departure Date
Flight Booking
“Book a flight
to London”
Automatic
Speech Recognition
Natural Language
Understanding
Book Flight
London
Utterances
Flight booking
London Heathrow
Intent /
Slot model
London Heathrow

20. Origin
Destination
Departure Date
Flight Booking
“Book a flight
to London”
Automatic
Speech Recognition
Natural Language
Understanding
Book Flight
London
Utterances
Flight booking
London Heathrow
Intent /
Slot model
London Heathrow
LocationLocation
Seattle

21. Origin
Destination
Departure Date
Flight Booking
“Book a flight
to London”
Automatic
Speech Recognition
Natural Language
Understanding
Book Flight
London
Utterances
Flight booking
London Heathrow
Intent /
Slot model
London Heathrow
LocationLocation
Seattle
Prompt
“When would you like to fly?”
“When would you
like to fly?”
Polly

22. Origin
Destination
Departure Date
Flight Booking
London Heathrow
Seattle
Prompt
“When would you like to fly?”
“When would you
like to fly?”
Polly
“Next Friday”

23. Origin
Destination
Departure Date
Flight Booking
“Next Friday”
Automatic
Speech Recognition
Next Friday
Utterances
Natural Language
Understanding
Flight booking
02 / 24 / 2017
Intent /
Slot model
London Heathrow
Seattle
02/24/2017

24. Origin
Destination
Departure Date
Flight Booking
“Next Friday”
Automatic
Speech Recognition
Next Friday
Utterances
Natural Language
Understanding
Flight booking
02 / 24 / 2017
Intent /
Slot model
London Heathrow
Seattle
02/24/2017
Confirmation
“Your flight is booked for next Friday”
“Your flight is booked
for next Friday”
Polly

25. Origin
Destination
Departure Date
Flight Booking
“Next Friday”
Automatic
Speech Recognition
Next Friday
Utterances
Natural Language
Understanding
Flight booking
02 / 24 / 2017
Intent /
Slot model
London Heathrow
Seattle
02/24/2017
Hotel Booking

26. Amazon Rekognition
Deep learning-based image recognition service
Search, verify, and organize millions of images
Object and Scene
Detection
Facial
Analysis
Face
Comparison
Facial
Recognition
Integrated with S3, Lambda, Polly, Lex

27. Object and Scene Detection
Generate labels for thousands of objects, scenes, and
concepts, each with a confidence score
• Search, filter, and
curate image
libraries
• Smart searches for
user generated
content
• Photo, travel, real
estate, vacation
rental applications
Maple
Plant
Villa
Garden
Water
Swimming Pool
Tree
Potted Plant
Backyard

28. Facial Analysis
Locate faces within images and analyze face attributes to
detect emotion, pose, facial landmarks, and features
• Avoid faces when cropping
images and overlaying ads
• Capture user demographics
and sentiment
• Recommend the best photos
• Improve online dating match
recommendations
• Dynamic, personalized ads

29. Face Comparison
Measure the likelihood that faces in two images are of the
same person
• Add face verification to
applications and devices
• Extend physical security
controls
• Provide guest access to
VIP-only facilities
• Verify users for online
exams and polls

30. Facial Recognition
Identify people in images by finding the closest match for an
input face image against a collection of stored face vectors
• Add friend tagging to
social and messaging apps
• Assist public safety officers
find missing persons
• Identify employees as they
access sensitive locations
• Identify celebrities in
historical media archives

31. Media Case Study
Identify who is on camera at what time for each of 8 networks
so that recorded video streams can be indexed and searched
Video frame-sampling facial recognition solution using
Amazon Rekognition:
• Indexed 97,000 people into a face collection in 1 day
• Sample frames every 6 secs and test for image variance
• Upload images to S3 and call Rekognition to find best facial match
• Store time stamp and faceID metadata

32. Influencer Marketing Case Study
Associate influencers with objects and scenes in social media
images in order to create high impact campaigns for clients
Using Rekognition for metadata extraction:
• Create rich media indexes of images from social media feeds, which
the application associates with influencers
• Enable analytics to profile environments where influence is strongest
• Connect client brands with the influencers most likely to have impact

33. Rekognition Customers
Media and Entertainment
Public Safety
Law Enforcement
Digital Asset Management
Influencer Marketing
Digital Advertising
Education
Consumer Storage

34. Amazon AI Services
• Leveraging Amazon internal experiences with AI / ML
• Managed API services with embedded AI for maximum
accessibility and simplicity
• Full stack of platforms and engines for specialized deep
learning applications

35. Thank you!
pearsond@amazon.com