This document summarizes a presentation about using IoT and AI services from AWS in healthcare. It discusses how IoT can collect data from devices and sensors, examples of how Philips Healthcare and Emory University are using AWS IoT, and how Alexa and Amazon Lex can be used to build voice applications. It also provides a case study of how Ohio Health is using Alexa to provide health information to patients. Finally, it discusses potential ways the American Heart Association could use Amazon Lex to provide cardiovascular health information through conversational interfaces.

1. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
IoT and AI Services in Healthcare
Anthony Nguyen
Senior Big Data Consultant
June 13, 2017

2. Agenda
• What is IoT?
• Customer examples of IoT on AWS in healthcare
• How Alexa and AWS can help customers think big
• Pollexy
• American Heart Association’s story
• Q+A

3. What exactly is IoT anyway?
“The Internet of Things is the
network of physical objects
that contain embedded
technology to communicate
and sense or interact with
their internal states or the
external environment.”

4. Internet of Things (IoT)
26 billion connected units will be installed by 2020
Key factors include:
 Lower price point of sensors, processors, and networking
 Increased connectivity
 Reducing cost of data storage and compute
Sources:
https://www.gartner.com/doc/3086918/market-guide-iot-platforms
http://www.mckinsey.com/insights/business_technology/the_internet_of_things_the_value_of_digitizing_the_physical_world
Economic value impact is estimated to be up to $11T by 2025
 Large quantities of data to be analyzed and processed
 Every industry vertical and domain expected to be impacted

5. The promise of the Internet of
Things
- Smarter products that get better with time
- Processes that are more efficient and predictable
- Closer relationship with your customers
- Offerings that were not possible before

6. However…
- Devices are hard to connect, manage
- Things do not interoperate out of the box
- Low signal-to-noise ratio in collected data
- Applications and things do not always match

7. AWS IoT
“Securely connect billions of
devices to AWS,
so they can interact with
applications and other
devices”

8. Heavy lifting with IoT
Devices Network Security Data Collection Smarts
1

9. AWS IoT
Fully managed cloud platform that lets connected devices easily and securely
interact with cloud applications and other devices
Devices Network Security Data Collection Smarts
1
Connect billions
of devices
Lightweight
communication protocol
X509 certificates DynamoDB,
Amazon Kinesis,
and S3
Trigger Lambda
functions

10. AWS customers are connecting physical things to the
cloud in every industry imaginable
Healthcare and Life
Sciences
Municipal Infrastructure Smart Home Retail
Manufacturing, Logistics, &
Supply Chain
Agriculture Education Automotive

11. AWS customers are connecting physical things to the
cloud in every industry imaginable
Healthcare and Life Sciences

12. Philips Healthcare
Uses AWS IoT to collect and act on critical data across different devices
Philips is a leading health-tech company,
working to create a new era of connected and
personalized digital health and care.
With the addition of AWS IoT, we will
greatly accelerate the pursuit of our
vision by making it easy to acquire,
process, and act on data from
heterogeneous devices in real time.
 The Philips HealthSuite digital
platform analyzes and stores 15 PB
of patient data gathered from 390
million imaging studies, medical
records, and patient inputs.
 Running on AWS provides the
reliability, performance, and
scalability that Philips needs to help
protect patient data as its global
digital platform grows at the rate of
one petabyte per month.
Jeroen Tas
CEO Healthcare Informatics Solutions and Services, Philips
”
“

13. Emory: The Healthy Aging Study
The challenge: identify predictive AD biomarker during middle age (who, when)
The opportunity: 100,000 patients as partners → the Emory Healthy Aging Study
Extensive phenotyping of health — from MRI to mHealth
Physiology patches, actigraphy meters, behavioral data scraping, off-body
sensors…

17. How can Alexa help
in healthcare?

18. High utility Low utility
Doing
Performs a task
“Alexa, ask Scout to arm
away mode.”
“Away mode armed. You
have 45 seconds to leave
the house.”
Searching
Identifies specific info
“Alexa, ask Vendor if there
are Pearl Jam tickets
available for this
weekend.”
“There are a limited number
of tickets, ranging from $49
to $279.”
Telling
Provides a quick
reference point
“Alexa, tell me a cat fact.”
“It is well known that dogs
are superior to cats.”
Browsing
Gives info on a broad
subject
“Alexa, ask Amazon
what’s on sale.”
“The following items are on
sale right now...”

19. CASE STUDY: OHIO
HEALTH
Case study: Ohio Health

21. How about Alexa without
an Echo?

22. How about Alexa without
an Echo?

23. Think of the possibilities!

24. Think of the possibilities!

25. Think of the possibilities!

26. Think of the possibilities!

27. Alexa + AWS
Alexa Voice
Services
Amazon
Echo
Alexa
Skills KitIoT Apps, Back
End System, Big
Data, Machine
Learning, Storage
Companion
Apps
Lambda
AWS IoT
Voice Control
• Alexa - With the Alexa Skills Kit,
you can easily build and add your
own skills to Alexa.
• AWS IoT manages the connection
devices and integration with ASK,
as well as other backend systems
and companion apps.
• Build skills for Alexa using AWS
Lambda. Simply write the code and
upload it as a Lambda function.
• Alexa Voice Service brings voice-
powered experiences to any
connected devices.
DeviceConnection

28. Introducing Amazon AI
Bringing powerful artificial intelligence to all developers

29. Amazon Rekognition
Image Recognition and Analysis
Powered by Deep Learning
1

30. Amazon Rekognition: Images in,
categories and facial analysis out
Amazon
Rekognition
Car
Outside
Daytime
Driving
Objects
& Scenes
Female
Smiling
Sunglasses
Faces

31. Amazon Rekognition: Recognize,
search, & understand images
Batch
analysis
Real-time
analysis
Low costContinually
improving
Easy to use

32. Amazon Polly
Text to Speech Powered by Deep Learning

33. Amazon Polly: Text in, lifelike speech out
Amazon Polly
“The temperature
in WA is 75°F”
“The temperature
in Washington is 75 degrees
Fahrenheit”

34. Amazon Polly: Text in, lifelike speech out
Returns an MP3
audio stream
Unlimited
replay
Fully managedLightning fast
responses

35. Amazon ALEXA
(It’s what’s inside Alexa)
Natural Language Understanding (NLU) &
Automatic Speech Recognition (ASR) Powered by Deep Learning

36. Amazon Lex: Speech
Recognition & Natural
Language Understanding
Amazon Lex
Automatic Speech Recognition
Natural Language Understanding
“What’s the weather
forecast?”
Weather
Forecast

37. Amazon Lex: Speech
Recognition & Natural
Language Understanding
Amazon Lex
Automatic Speech Recognition
Natural Language Understanding
“What’s the weather
forecast?”
“It will be sunny
and 75°F”
Weather
Forecast

38. Amazon Lex: Build natural, conversational
interactions in voice & text
Integrated
development in the
AWS console
Fully
managed
Trigger
Lambda
functions
Continually improving
ASR & NLU models
Enterprise
connectors
Multi-step
conversations

40. CASE STUDY: OHIO
HEALTH
Case study: Ohio Health

41. High utility Low utility
Doing
Performs a task
“Alexa, ask Scout to arm
away mode.”
“Away mode armed. You
have 45 seconds to leave
the house.”
Searching
Identifies specific info
“Alexa, ask Vendor if there
are Pearl Jam tickets
available for this
weekend.”
“There are a limited number
of tickets, ranging from $49
to $279.”
Telling
Provides a quick
reference point
“Alexa, tell me a cat fact.”
“It is well known that dogs
are superior to cats.”
Browsing
Gives info on a broad
subject
“Alexa, ask Amazon
what’s on sale.”
“The following items are on
sale right now...”

42. Think Big!
Amazon Polly + Amazon Lex
+ Alexa = Pollexy

43. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
David Woody
June 13, 2017
Advancing Our Mission on the Alexa
and Amazon Lex Platforms
The American Heart Association

44. David Woody, Director of Innovative Solutions
Business Technology
Advancing Our Mission on Alexa and Amazon Lex
The American Heart Association

45. Our Mission
To build healthier lives, free of cardiovascular diseases and stroke.

46. Who we are and what we do…
The American Heart Association is the nation’s oldest and largest voluntary
organization dedicated to fighting heart disease and stroke. Founded by six
cardiologists in 1924, our organization now includes more than 22.5 million
volunteers and supporters.
We fund innovative research, fight for stronger public health policies, and
provide critical tools and information to save and improve lives.
Since 1949, the AHA has
invested more than $4 billion in
research to enhance our
knowledge of cardiovascular
diseases and stroke.
Thirteen scientists funded by the
American Heart Association have
won Nobel Prizes, including nine for
research we wholly or partially
supported.
The AHA’s ECC programs train
more than 18 million people every
year by educating healthcare
providers, caregivers, and the
general public.

47. Advancing our mission on Alexa and Amazon Lex
The world is rapidly changing and we are constantly rethinking traditional
approaches to reach people at the pace that they live their everyday lives. We
are committed to the role that technology plays in enabling consumers,
patients, and physicians to achieve better health outcomes and to further the
mission of the AHA.
• Ease of access to relevant AHA information
• Frictionless engagement with the organization

48. The AHA on Alexa

49. The AHA on Alexa – CPR instructions
Why this matters ...
 Through this skill, we are able to reach people where they are — at home.
 More than 350,000 out-of-hospital cardiac arrests happen annually in the
United States, with a survival rate of about 11 percent.
 Nearly 70 percent of these occur in the home.

50. The AHA on Amazon Lex
We need to provide access to the vast scientific research of the American Heart Association via a
personable, reliable, always-available solution. Our stakeholders are always looking for information
via questions on health, nutrition, and events, that needs to be engaging and convenient by being
highly accessible, even for accepting donations.
Science
Access to AHA research
data requires a go-to
resource for those seeking
reliable heart- and stroke-
related information.
Engagement
The tone and personality of
the American Heart
Association brand are
brought to life via natural
language conversations.
Community
AHA seeks to become an embedded member of the AHA community.

51. Community
Fundraising
Healthy
Living
Science
Mission
Our Vision 360º access to AHA information and services
TRANSFORMATION

52. NEC
(Service
Center)
Heart.org
Content
Fundraising
DonateNow
HW
YM
Social Events
3 Proofs of Concept
Content areas
• 7 major content areas
• 8 additional top-level areas
• Multiple sub-areas
• Public & professional sites
• 5 languages supported
• ASA site
• 5 major content
areas
• 250,000 calls annually
(about 20,000 per month)
• Additional volume of calls
and emails going to ECC
program
• Primary donations site
• 3 major fundraising events
• Heart Walks
• Youth Market
events
• Social events
• Multiple other events

53. POC 1 : Heart Walk Events
• The AHA engages nearly 1 million participants nationwide through our premier Heart
Walk events, to further our mission of saving lives.
• The Heart Walk is the AHA's premiere event designed to bring together communities,
promote physical activity, and raise funds in the fight against this country's no. 1 and no. 5
killers, heart disease and stroke.
• AHA sponsors nearly 300 Heart Walk events annually in the U.S., totaling nearly 1 million
participants.
• The Heart Walk is the largest fundraiser for the American Heart Association with 300
events nationwide raising $130M ($1.6B since it’s inception).

54. Our Approach
The current process of registration…
• Decidedly not frictionless
• Manual completion of a standard web form
• Multiple pages with multiple questions to comprehend and then answer per page
Traditional optimizations efforts do not completely address the core issue…
• The repeated need to read questions, comprehend what was asked, select an
answer, and then type that answer creates a frictional event.
• The introduction of a natural language engagement eliminates the need to read
text, accelerates comprehension, and replaces manual input with speech.
However, we did not want
• A device-specific solution
• A solution requiring an app
Enter….Voice-activated Bot technology
POC 1

55. AHA Heart Walk bot
• Browser-based solution to register participants using text or speech
• Participants provide details such as email address to look up and
register with new teams
• Decision trees are employed for different workflows: returning
participant, starting new team, joining as an individual
• Integrated with Teamraiser utilizing Luminate APIs via Lamda functions
• Participant is presented with T&C to opt-in and a summary of input
• Participants can confirm their information or request to edit certain input
• Email notification is sent upon completion
Key Features
POC 1

56. Architecture
Teamraiser
1. Heart Walk participants
initiate registration on their
phone or laptop
2. Amazon Lex
recognizes that a
registration request
has been initiated
3. Amazon Lex asks the
registrant a series of questions
and collects the responses
4. Registration
information is passed
to Team Raiser via
Luminate APIs
5. Registrant receives email
confirmation and link to their
unique participant center
POC 1

57. Heart Walk Registration botPOC 1

58. HW botPOC 1

59. POC 2 : Healthy for Good Tracking

60. HFG tracking bot Demo

61. POC 3 : FAQ bot
NEC
(Service Center)
Heart.org
• 7 major content areas
• 8 additional top-level areas
• Multiple sub-areas
• Public & professional sites
• 5 languages supported
• ASA site
• 5 major content areas
• 250,000 calls annually
(about 20,000 per month)
• Additional volume of calls and
emails going to ECC program
Content Areas
• Fast, relevant answers
• Natural Language search
• Use of response cards to
return more than text
• Images
• Links
• Graphical elements;
charts, graphs,
infographics, etc.
• Input and selection
via buttons
Potential
• Geo-location elements
• Sitecore profiles

62. FAQ bot on heart.org

64. FAQ bot on heart.org

65. FAQ bot on heart.org

66. Q+A
Thank you!
aanwin@amazon.com
david.woody@heart.org