Amazon SageMaker è un servizio gestito per sviluppatori e data scientist che consente di progettare, addestrare e distribuire modelli di Machine Learning su larga scala. In questo webinar esploreremo le funzionalità di questo servizio, dalle istanze notebook Jupyter ai servizi di training e hosting, per poi discutere di aspetti come il labeling di dataset e l’ottimizzazione dei modelli. Successivamente, vedremo in modo pratico come utilizzare il servizio per implementare, addestrare e distribuire un modello di esempio.

1. AWS Online Tech Talks
La serie di webinar AWS in Italiano
Amazon SageMaker in action: modelli di Machine Learning da idee a produzione
Speaker
Giuseppe Porcelli
Solutions Architect, AWS
Obiettivi
 Fornire una panoramica del servizio Amazon SageMaker
 Fornire linee guida su sviluppo, training e distribuzione di modelli di
Machine Learning
 Mostrare in modo pratico come iniziare ad utilizzare Amazon SageMaker

2. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon SageMaker in Action
Modelli di Machine Learning da idee a produzione
Giuseppe A. Porcelli
Sr. Solutions Architect
Amazon Web Services EMEA

3. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Machine Learning at Amazon
Voice driven
interactions
Fulfillment automation
& inventory management
Personalized
recommendations
Drones
Inventing entirely new
customer experiences

4. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS MISSION
Put Machine Learning in the
hands of every developer and
data scientist
A long history of successfully
running large-scale
Machine Learning
DEMOCRATIZING ARTIFICIAL INTELLIGENCE
AMAZON’S EXPERIENCE

5. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
THE AWS MACHINE LEARNING STACK
M L F R A M E W O R K S &
I N F R A S T R U C T U R E
A I S E R V I C E S
M L P L A T F O R M S
R E K O G N I T I O N
I M A G E
P O L L Y T R A N S C R I B E T R A N S L A T E C O M P R E H E N D L E XR E K O G N I T I O N
V I D E O
Vision Speech Language Chatbots
F O R E C A S T
Forecasting
T E X T R A C T P E R S O N A L I Z E
Recommendations

6. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
THE AWS MACHINE LEARNING STACK
M L F R A M E W O R K S &
I N F R A S T R U C T U R E
A I S E R V I C E S
R E K O G N I T I O N
I M A G E
P O L L Y T R A N S C R I B E T R A N S L A T E C O M P R E H E N D L E XR E K O G N I T I O N
V I D E O
Vision Speech Language Chatbots
F O R E C A S T
Forecasting
T E X T R A C T P E R S O N A L I Z E
Recommendations
M L P L A T F O R M S
A M A Z O N
S A G E M A K E R
B U I L D T R A I N D E P L O Y
Pre-built algorithms & notebooks
Data labeling (G R O U N D T R U T H )
One-click model training & tuning
Optimization (N E O )
One-click deployment & hosting
Reinforcement learningAlgorithms & models ( A W S M A R K E T P L A C E
F O R M A C H I N E L E A R N I N G )

7. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
THE AWS MACHINE LEARNING STACK
M L F R A M E W O R K S &
I N F R A S T R U C T U R E
A I S E R V I C E S
R E K O G N I T I O N
I M A G E
P O L L Y T R A N S C R I B E T R A N S L A T E C O M P R E H E N D L E XR E K O G N I T I O N
V I D E O
Vision Speech Language Chatbots
F O R E C A S T
Forecasting
T E X T R A C T P E R S O N A L I Z E
Recommendations
M L P L A T F O R M S
F r a m e w o r k s I n t e r f a c e s
I n f r a s t r u c t u r e
E C 2 P 3
& P 3 D N
E C 2 C 5 F P G A s G R E E N G R A S S E L A S T I C
I N F E R E N C E
A M A Z O N
S A G E M A K E R
B U I L D T R A I N D E P L O Y
Pre-built algorithms & notebooks
Data labeling (G R O U N D T R U T H )
One-click model training & tuning
Optimization (N E O )
One-click deployment & hosting
Reinforcement learningAlgorithms & models ( A W S M A R K E T P L A C E
F O R M A C H I N E L E A R N I N G )
A W S D E E P L E A R N I N G A M I

8. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Scaling machine learning

9. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Machine Learning Process is Hard, and Time Consuming
Experimentation
• Setup and manage
Notebook environments
• Get data to notebooks securely
Training
• Setup and manage clusters
• Scale/distribute ML algorithms
Deployment
• Setup and manage
inference clusters
• Manage and auto scale
inference APIs
• Testing, versioning,
and monitoring
Fetch data
Clean & format
data
Prepare &
transform data
Train modelEvaluate model
Integrate with
prod
Monitor/
debug/refresh
6–18
months
Data Wrangling
• Manage data ingestion
• Execute ETL

10. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon SageMaker
A managed service that provides the quickest and
easiest way for data scientists and developers to get
ML models from idea to production

11. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AMAZON SAGEMAKER
Pre-built
notebooks for
common problems
Built-in, high
performance
algorithms
BUI L D
T R AI N & T UNE D E P L O Y
BUILD, TRAIN, TUNE AND HOST YOUR OWN MODELS

12. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AMAZON SAGEMAKER
Pre-built
notebooks for
common problems
Built-in, high
performance
algorithms
One-click
training
BUI L D T R AI N & T UNE
D E P L O Y
Hyperparameter
optimization
BUILD, TRAIN, TUNE AND HOST YOUR OWN MODELS

13. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AMAZON SAGEMAKER
Fully managed
hosting with auto-
scaling
One-click
deployment
Pre-built
notebooks for
common problems
Built-in, high
performance
algorithms
One-click
training
BUI L D T R AI N & T UNE D E P L O Y
Hyperparameter
optimization
BUILD, TRAIN, TUNE AND HOST YOUR OWN MODELS

14. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AMAZON SAGEMAKER
Fully managed
hosting with auto-
scaling
One-click
deployment
Pre-built
notebooks for
common problems
Built-in, high
performance
algorithms
One-click
training
BUI L D T R AI N & T UNE D E P L O Y
End-to-end encryption with KMS
End-to-end VPC support
Compliance and audit capabilities Pay as you go
Hyperparameter
optimization
BUILD, TRAIN, TUNE AND HOST YOUR OWN MODELS

15. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

16. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Three areas we are improving for ML developers
CostData
Cost
We’re improving both training and
inference speed & cost
Data
Preparing data for ML is major
expensive, complex, and time consuming
Ease of use
We continue to want to reduce the
barrier of entry to ML for all developers

17. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Improving Training & Inference Cost

18. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
The largest P3 instance,
optimized for distributed training

19. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon EC2 P3dn instance
The largest P3 instance, optimized for distributed training
Reduce machine
learning training time
Better GPU
utilization
Support larger, more
complex models
K E Y F E A T U R E S
100Gbps of networking
bandwidth
8 NVIDIA Tesla
V100 GPUs
32GB of
memory per GPU
(2x more P3)
96 Intel
Skylake vCPUs
(50% more than P3)
with AVX-512

20. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Training gets a lot of attention,
but what about inference?

21. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Prediction Training
Inference
(Prediction)
90%
Training
10%
Predictions drive
complexity and
cost in production

22. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
The challenges of prediction in production
One size does not fit all Elasticity is important

23. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Add GPU acceleration to any Amazon EC2 instance or Amazon SageMaker
for faster inference at much lower cost (up to 75% savings)

24. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Elastic Inference
Reduce deep learning inference costs up to 75%
Match capacity
to demand
Available between 1 to 32
TFLOPS
K E Y F E A T U R E S
Integrated with
Amazon EC2,
Amazon SageMaker, and
Amazon DL AMIs
Support for TensorFlow,
Apache MXNet, and ONNX
with PyTorch coming soon
Single and
mixed-precision
operations
Lower inference costs

25. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Making it easier to obtain high
quality labeled data

26. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Build highly accurate training datasets and reduce
data labeling costs by up to 70% using machine
learning

27. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon SageMaker ground truth
Label machine learning training data easily and accurately

28. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Driving Ease of Use

29. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS Marketplace for machine learning
ML algorithms and models available instantly

30. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Optimization is extremely complex

31. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
A deep learning model compiler that lets customers train models once,
and run them anywhere with up to 2X improvement in performance

32. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon SageMaker Neo
Train once, run anywhere with 2x the performance
K E Y F E A T U R E S
Compiler & run-time are open source 1/10th the size of original models

33. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
What’s next for
machine learning?

34. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Supervised learning
Unsupervised learning
Types of Machine LearningSOPHISTICATIONOFMLMODELS
AMOUNT OF TRAINING DATA REQUIRED

35. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Types of Machine Learning
AMOUNT OF TRAINING DATA REQUIRED
Supervised learning
Unsupervised learning
SOPHISTICATIONOFMLMODELS

36. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Types of Machine Learning
Reinforcement learning
(RL)
Supervised learning
Unsupervised learning
AMOUNT OF TRAINING DATA REQUIRED
SOPHISTICATIONOFMLMODELS

37. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
How does RL work?
USE CASES

38. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
New machine learning capabilities in Amazon SageMaker to
build, train and deploy with reinforcement learning

39. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon SageMaker RL
Reinforcement learning for every developer and data scientist
Broad support
for frameworks
Broad support for simulation
environments including
SimuLink and MatLab
K E Y F E A T U R E S
TensorFlow, Apache
MXNet, Intel Coach,
and Ray RL support
2D & 3D physics
environments and
OpenAI Gym support
Supports Amazon Sumerian and
Amazon RoboMaker
Fully
managed
Example notebooks
and tutorials

40. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Other ways to
getting started…

41. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Machine Learning University
Uses the same
materials used to train
Amazon developers
Foundational
knowledge with
real-world application
Structured
courses and
specialist certification

42. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon ML Solutions Lab
Brainstorming Modeling Teaching
Leverage Amazon experts with decades of ML
experience with technologies like Amazon Echo,
Amazon Alexa, Prime Air and Amazon Go
Amazon ML Solutions
Lab provides ML
expertise
Quick turnaround
Demonstration of
functionality
Static data
Limited live data-sets
Core functionality
Limited integration with
production systems
Initial Architecture
Full live data-sets
Enhanced functionality
Link to production systems
Scaled Architecture
Iteration on MVP2
Fully operationally
Fully supported
Live security-approved
account

43. Thank you!
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Giuseppe A. Porcelli
Sr. Solutions Architect
Amazon Web Services EMEA