AWS Summit Milan 2018 Sviluppa, addestra e distribuisci modelli di Machine learning su qualsiasi scala con Amazon SageMaker

1. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Giuseppe Angelo Porcelli
Solutions Architect, Amazon Web Services
Paolo Genta
Senior Software Engineer, Edizioni Condè Nast Italia S.p.A.
Sviluppa, addestra e distribuisci
modelli di Machine Learning
su qualsiasi scala

2. LONG HISTORY OF ML AT AMAZON
THOUSANDS OF ENGINEERS ACROSS THE COMPANY FOCUSED ON AI
Personalized
recommendations
Inventing
entirely new
customer
experiences
Fulfillment
automation and
inventory
management
Drones Voice-driven
interactions

3. OUR MISSION AT AWS IS TO PUT
MACHINE LEARNING IN THE HANDS OF
EVERY DEVELOPER AND DATA
SCIENTIST

4. Amazon Polly
Amazon
Transcribe
Amazon Rekognition
Amazon Rekognition
Video
Amazon Translate
Amazon
Comprehend
Amazon Lex
VISION SPEECH LANGUAGE CHATBOT
SERVICES
Amazon SageMakerPLATFORMS Amazon ML Spark & EMR
Amazon
Mechanical Turk
MXNET
FRAMEWORKS
TensorFlow
Caffe2
& Caffe
Gluon KerasCNTKPyTorch
GPUINFRASTRUCTURE CPU
IoT
(Greengrass)
Mobile FPGAServerless
DEEP LEARNING AMI
AWS DeepLens
AMAZON AI
DEMOCRATIZED ARTIFICIAL INTELLIGENCE

5. Running AI in Production on AWS Today

6. THE MACHINE LEARNING PROCESS
Data Visualization &
Analysis
Business Problem –
ML problem framing Data Collection
Data Integration
Data Preparation &
Cleaning
Feature Engineering
Model Training &
Parameter Tuning
Model Evaluation
Are Business
Goals met?
Model Deployment
Monitoring &
Debugging
– Predictions
YesNo
DataAugmentation
Feature
Augmentation
Re-training
Help formulate the right
questions
• Domain Knowledge

7. Data Visualization &
Analysis
Business Problem –
ML problem framing Data Collection
Data Integration
Data Preparation &
Cleaning
Feature Engineering
Model Training &
Parameter Tuning
Model Evaluation
Are Business
Goals met?
Model Deployment
Monitoring &
Debugging
– Predictions
YesNo
DataAugmentation
Feature
Augmentation
Re-training
Help formulate the right
questions
• Domain Knowledge
DISCOVERY

8. Data Visualization &
Analysis
Business Problem –
ML problem framing Data Collection
Data Integration
Data Preparation &
Cleaning
Feature Engineering
Model Training &
Parameter Tuning
Model Evaluation
Are Business
Goals met?
Model Deployment
Monitoring &
Debugging
– Predictions
YesNo
DataAugmentation
Feature
Augmentation
Re-training
Build the data platform
• Amazon S3
• AWS Glue
• Amazon Athena
• Amazon EMR
• Amazon Redshift Spectrum
INTEGRATION

9. Data Visualization &
Analysis
Business Problem –
ML problem framing Data Collection
Data Integration
Data Preparation &
Cleaning
Feature Engineering
Model Training &
Parameter Tuning
Model Evaluation
Are Business
Goals met?
Model Deployment
Monitoring &
Debugging
– Predictions
YesNo
DataAugmentation
Feature
Augmentation
Re-training
• Setup and manage Notebook
Environments
• Setup and manage Training
Clusters
• Write Data Connectors
• Scale ML algorithms to large
datasets
• Distribute ML training
algorithm to multiple machines
• Secure Model artifacts
TRAINING

10. Data Visualization &
Analysis
Business Problem –
ML problem framing Data Collection
Data Integration
Data Preparation &
Cleaning
Feature Engineering
Model Training &
Parameter Tuning
Model Evaluation
Are Business
Goals met?
Model Deployment
Monitoring &
Debugging
– Predictions
YesNo
DataAugmentation
Feature
Augmentation
Re-training
• Setup and manage Model
Inference Clusters
• Manage and Auto-Scale Model
Inference APIs
• Monitor and Debug Model
Predictions
• Models versioning and
performance tracking
• Automate New Model version
promotion to production (A/B
testing)
DEPLOYMENT

11. A managed service
that provides the quickest and easiest way for
data scientists and developers to get
ML models from idea to production
Amazon SageMaker

12. AMAZON SAGEMAKER
Highly-optimized
machine learning
algorithms
BuildPre-built notebook
instances

13. AMAZON SAGEMAKER
One-click training
for ML, DL, and
custom algorithms
Easier training with
hyperparameter
optimization
Highly-optimized
machine learning
algorithms
BuildPre-built notebook
instances
Train

14. AMAZON SAGEMAKER
One-click training
for ML, DL, and
custom algorithms
Easier training with
hyperparameter
optimization
Highly-optimized
machine learning
algorithms
Deployment
without
engineering effort
Fully-managed
hosting at scale
BuildPre-built notebook
instances
Deploy
Train

15. AMAZON SAGEMAKER LAUNCH CUSTOMERS
“
- Ashok Srivastava, Chief Data Officer, Intuit
With Amazon SageMaker, we can accelerate our Artificial
Intelligence initiatives at scale by building and deploying
our algorithms on the platform. We will create novel large-
scale machine learning and AI algorithms and deploy them
on this platform to solve complex problems that can power
prosperity for our customers.
"

16. AMAZON SAGEMAKER KEY BENEFITS @
INTUIT
Ad-hoc setup and management
of notebook environments
Limited choices for model
deployment
Competing for compute
resources across teams
Easy data exploration
in SageMaker notebooks
Building around virtualization
for flexibility
Auto-scalable model hosting
environment
From To

17. AMAZON SAGEMAKER LAUNCH CUSTOMERS
“
- Dr. Walter Scott, CTO of Maxar Technologies
and founder of DigitalGlobe
"
As the world’s leading provider of high-resolution Earth imagery, data and
analysis, DigitalGlobe works with enormous amounts of data every day.
DigitalGlobe is making it easier for people to find, access, and run compute
against our entire 100PB image library, which is stored in AWS’s cloud, to
apply deep learning to satellite imagery. We plan to use Amazon
SageMaker to train models against petabytes of Earth observation imagery
datasets using hosted Jupyter notebooks, so DigitalGlobe's Geospatial Big
Data Platform (GBDX) users can just push a button, create a model, and
deploy it all within one scalable distributed environment at scale.

18. Summit Milan - 27 March 2018
Paolo Genta
Senior Software Engineer
@gentax

19. AWS Summit Milan - March 2018
30M Unique Visitors 250M Page Views
46% SEO 29% Social
CN.numbers // by month
20% Desktop80% Mobile

20. AWS Summit Milan - March 2018
CN.challange().engageUsers()
Behaviors analysis on authenticated
& not authenticated users
Users clustering and profiling
Better advertiser performance:
targeting ads, newsletter and e-commerce
Prediction: success of a content based on user
navigation and social network reaction

21. AWS Summit Milan - March 2018
CN.genius()
CN.genius().improve()

22. AWS Summit Milan - March 2018
CN.genius().showMobile()
Tailored made
related articles
Engage users
with ad hoc overlay
Show content
based on your
history

23. AWS Summit Milan - March 2018
CN.analyze().overview()
text
text
text
text
text
I’m a user

24. AWS Summit Milan - March 2018
CN.analyze().overview().addAI()
I’m a
HAPPY
user!!!

25. AWS Summit Milan - March 2018
CN.genius().user()CN.genius().user().idendity()CN.genius().user().clickstream()CN.genius().suggestedArticles()

26. AWS Summit Milan - March 2018
CN.genius().images()
more than
10M images
Automatic celeb
gallery
Gallery tag driven
Tags manager

27. AWS Summit Milan - March 2018
Collaborative filtering with
Amazon Neptune graph
DB
Data from videos:
Amazon Rekogniton Video
class nextstep extends CN.genius{…}
Use Amazon Comprehend
to analyze text with NLP
Stylist detection
{
“dress”: “Giorgio Armani”,
“bag”: “Gucci”
}
E-commerce:
click to shop

28. AWS Summit Milan - March 2018
class nextstep extends CN.genius.Sagemaker()
• We are not MXNet / Tensorflow experts: we need to
speed up
• Use a fully-managed machine learning compute instance
running a Jupyter Notebook and leverage on Amazon
SageMaker built-in algorithms
• Build -> Train -> Deploy all in one service: we can
decouple explorative analysis from training
• Ability to leverage on auto-scaling inference, with the
possibility to use your own algorithm packaged in a
Docker container
• DeepAR algorithm for forecasting scalar time series to
let ADV/Editors choose best moments to publish a
content

29. Thank you
Paolo Genta
Senior Software Engineer
@gentax

30. Amazon SageMaker Deep Dive

31. AMAZON SAGEMAKER COMPONENTS
Amazon’s fast, scalable algorithms
Idiomatic, distributed TensorFlow & MXNet
Bring your own algorithm
Hyperparameter optimization
UX HostingTraining

32. AMAZON SAGEMAKER COMPONENTS
Amazon’s fast, scalable algorithms
Idiomatic, distributed TensorFlow & MXNet
Bring your own algorithm
Hyperparameter optimization
UX HostingTraining

33. UX
… or Apache Spark
through EMR and
the SageMaker
Spark SDK...
Use SageMaker‘s
hosted Notebook
Instances...
... or SageMaker‘s
Console for a point
and click
experience...
... or your own
device (EC2,
laptop, etc.)

34. AMAZON SAGEMAKER COMPONENTS
Amazon’s fast, scalable algorithms
Idiomatic, distributed TensorFlow & MXNet
Bring your own algorithm
Hyperparameter optimization
UX HostingTraining

35. TRAINING
Zero setup Streaming datasets
+ distributed
compute
Docker / ECS Deploy trained models
locally or to
SageMaker,
Greengrass,
DeepLens

36. AMAZON SAGEMAKER COMPONENTS
Amazon’s fast, scalable algorithms
Idiomatic, distributed TensorFlow & MXNet
Bring your own algorithm
Hyperparameter optimization
UX HostingTraining

37. HOSTING
One step
deployment
Low latency, high
throughput, and
high reliability
A/B testing Use your own
model

38. AMAZON SAGEMAKER COMPONENTS
Amazon’s fast, scalable algorithms
Idiomatic, distributed TensorFlow & MXNet
Bring your own algorithm
Hyperparameter optimization
UX HostingTraining

39. BUILT-IN ALGORITHMS
XGBoost, FM,
Linear, and
Forecasting for
supervised learning
Kmeans, PCA, and
Word2Vec for
clustering and pre-
processing
Image
classification with
convolutional
neural networks
LDA and NTM for
topic modeling,
seq2seq for
translation

40. AMAZON SAGEMAKER COMPONENTS
Amazon’s fast, scalable algorithms
Idiomatic, distributed TensorFlow & MXNet
Bring your own algorithm
Hyperparameter optimization
UX HostingTraining

41. TensorFlow AND MXNet CONTAINERS
… explore and
refine models in a
single Notebook
Instance
… deploy to
production
Sample your
data…
Use the same code
to train on the full
dataset in a cluster
of GPU
instances…

42. AMAZON SAGEMAKER COMPONENTS
Amazon’s fast, scalable algorithms
Idiomatic, distributed TensorFlow & MXNet
Bring your own algorithm
Hyperparameter optimization
UX HostingTraining

43. BRING YOUR OWN ALGORITHM
... add algorithm
code to a Docker
container...
Pick your preferred
framework...
... publish to ECS
Amazon
ECS

44. AMAZON SAGEMAKER COMPONENTS
Amazon’s fast, scalable algorithms
Idiomatic, distributed TensorFlow & MXNet
Bring your own algorithm
Hyperparameter optimization
UX HostingTraining

45. HYPERPARAMETER OPTIMIZATION
Run a large set of training
jobs with varying
hyperparameters...
... and search the
hyperparameter space for
improved accuracy.

46. CALL TO ACTION
• Getting started with Amazon SageMaker: https://aws.amazon.com/sagemaker/
• Use the Amazon SageMaker high-level SDK:
• For Python: https://github.com/aws/sagemaker-python-sdk
• For Spark: https://github.com/aws/sagemaker-spark
• SageMaker Examples: https://github.com/awslabs/amazon-sagemaker-examples
• Let us know what you build!

47. Enabling Machine Learning and
learning from Amazon’s experience

48. AMAZON ML SOLUTIONS LAB
Lots of companies
doing Machine
Learning
Unable to unlock
business potential
Brainstorming Modeling Teaching
Lack ML
expertise
Leverage Amazon experts with decades of ML
experience with technologies like Amazon Echo,
Amazon Alexa, Prime Air and Amazon Go
Amazon ML Lab
provides the missing
ML expertise

49. AMAZON ML TECHNOLOGY PARTNERS
Annotation, Wrangling,
Procurement
Media NLP Optimization Other SaaS/API
Data
Annotations Generation ‘Wrangling’
Data management
Predictive model training
Model evaluation
Model deployment
Model management
Offline and online prediction
Computer Vision
Natural Language Processing
Recommendation Engines
Conversational Interfaces
Event Prediction
Anomaly Detection
Data Science &
Machine Learning
Platforms

50. D a t a S o l u t i o n s M L & D a t a S c i e n c e I n t e l l i g e n t S o l u t i o n s
aws.amazon.com/mp/ai
AWS MARKETPLACE

51. Thank You!
https://aws.amazon.com/machine-learning/