Motion detection triggers have reduced the amount of video recorded by modern devices. But maybe you want to reduce that further—maybe you only care if a car or a person is on-camera before recording or sending a notification. Security cameras and smart doorbells can use Amazon Rekognition to reduce the number of false alarms. Learn how device makers and home enthusiasts are building their own smart layers of person and car detection to reduce false alarms and limit video volume. Learn too how you can use face detection and recognition to notify you when a friend has arrived.

1. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS re:Invent
Mak ing IoT Smar ter w ith AW S R ek ognition
N i c k R o b i n s o n , S o f t w a r e D e v e l o p e r
M C L 3 0 6

2. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
What to Expect from the Session
• How to leverage AWS to add intelligence to IoT
devices
• Smart Door Cam
• Smart Object Detector
• A reference architecture for performing real-time
recognition across a number of categories

3. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Problem

4. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Reference Architecture
AWSAWS
AWS

5. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Detection Pipeline
AWSAWS
AWS

6. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS IoT
AWSAWS
AWS

7. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
State of IoT
• Connected Device Growth
• 2016 - 6.38 Billion
• 2017 – 8.38 Billion
• 2020 Projections – 20.42
Billion
• Rapid Growth of Sector
• Growth in Consumer Spending
• 2016 - $532B
• 2017- $725B
• 2020 Projections - $1.5T

8. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Common Problems in IoT
• Security
• Limited Compute Power
• Limited Storage

9. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS IoT Overview

10. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS IoT Rules Engine

11. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS IoT Rules Engine
• Rule syntax
• Example Rule Definition
• SELECT resource_id as id, cycle
as c_number FROM ‘pm/topic’ WHERE
failure = 1

12. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS IoT Setup
• Provision new device through the console
• Download connection kit
• Example Pub/Sub Application

13. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

14. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon Rekogntion
AWSAWS
AWS

15. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon Rekognition Overview
Object and Scene
Detection
Facial
Analysis
Face
Comparison
Facial
Recognition
Celebrity
Recognition
Image Moderation Text Recognition

16. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Facial Analysis
Image Quality
Facial Landmarks
Demographic Data Emotion Expressed
General Attributes
Facial Pose
Brightness 23.6%
Sharpness 99.9%
EyeLeft,EyeRight,Nose
RightPupil,LeftPupil
MouthRight,LeftEyeBrowUp
Bounding Box...
Age Range 29-45
Gender:Male 96.5%
Happy 83.8%
Surprised 0.65%
Smile:True 23.6%
EyesOpen:True 99.8%
Beard:True 99.5%
Mustache:True 99.9%...
Pitch 1.446
Roll 5.725
Yaw 4.383
DetectFaces

17. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Facial Recognition
Search
Index
Collection
SearchFacesByImage

18. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Object And Scene Detection
Flower
Arrangement
Chair
Coffee Table
Living Room Indoors
Furniture
Cushion
Vase
Maple
Villa
Plant
Garden
Water
Swimming Pool
Tree
Potted Plant
Backyard
Patio
DetectLabels

19. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Interface with Amazon Rekognition
• S3 input for API calls - max image size of 15MB
• 5MB limit for non-S3 (Base64 encoded) API calls
• Minimum image resolution (x or y) of 80 pixels
• Image data supported in PNG or JPG format
• Face collections support tens of millions of faces
• The max matching faces the search API returns is 4096
• Size of face should occupy 5%+ of image for detection
• Collections are for faces!

20. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Potential Use Cases
• Smart Body Cameras with alerts for persons of interest
• Dog bowl with embedded camera that sends you updates during the day
when your pet is spotted
• Adaptive digital advertising, real-time content decisions based on crowd
demographics
• Vision systems with configurable match criteria

21. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS Lambda
AWSAWS
AWS

22. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
IoT Message Processing
• AWS Lambda invoked from IoT action based on topic
• Receives base64 encoded jpeg image in JSON message
• Sends image to Amazon Rekognition for label detection
• Filters out accepted labels
• Communicates back to IoT device

23. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Setting Up Our Lambda Function
• Creating Lambda Functions
• Connect to AWS IoT
• Creating topics
• Configuring Actions

24. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
object_list = ['Person', 'People', 'Human']
def lambda_handler(event, context):
client = boto3.client('rekognition')
response = client.detect_labels(
Image={
'Bytes': base64.b64decode(event['image']),
},
MaxLabels=5 )
client = boto3.client('iot-data', region_name='us-east-1')
for item in response['Labels']:
if item['Name'] in object_list:
client.publish(
topic='sdk/objectDetected',
qos=1,
payload=json.dumps({"object":item['Name']})
)
Lambda Handler

25. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

26. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

27. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Delivery Pipeline
AWSAWS
AWS

28. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Customer Notifications – AWS SNS
AWSAWS
AWS

29. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
• Deliver notifications to your customers in a scalable way
• Setup a topic
• Subscribe your users
• Publish notifications and have them multiplexed to subscribed
users
Amazon SNS

30. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Deliver Video
AWSAWS
AWS

31. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
• Amazon S3 Events provides powerful tool for event generation
• Deliver video content to bucket
• Can trigger variety of events across a number of AWS services
including AWS Lambda
Amazon S3 Events

32. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
• Setup steps
• Generating Signed URLs
• Sending notifications to our users
Setting up our Notification Lambda

33. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
• Modify our Lambda function to add Face Detection and metadata
collection
• Perform facial recognition against a face collection for more
detailed notifications
• Use for real-time content moderation
Extending the Pipeline

34. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

35. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
• Come out to the skills session!
• SKL69
Learn More

36. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Questions?
Making IoT Smarter with Amazon Rekognition

37. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
THANK YOU!