2° episodio - Speaker: Giuseppe Porcelli, Solutions Architect AWS

1. © 2019, Amazon Web Services, Inc. or its Affiliates.
Giuseppe Angelo Porcelli
Principal Solutions Architect
Amazon Web Services EMEA
Sviluppa, addestra e distribuisci
modelli di Machine Learning
Serie webinar in Italiano

2. © 2019, Amazon Web Services, Inc. or its Affiliates.
Artificial Intelligence. Get Started.
Serie webinar in Italiano
Today
27 Giugno25 Giugno
11 Giugno

3. © 2019, Amazon Web Services, Inc. or its Affiliates.
THE AWS MACHINE LEARNING STACK
M L F R A M E W O R K S &
I N F R A S T R U C T U R E
A I S E R V I C E S
M L P L A T F O R M S
R E K O G N I T I O N
I M A G E
P O L L Y T R A N S C R I B E T R A N S L A T E C O M P R E H E N D L E XR E K O G N I T I O N
V I D E O
Vision Speech Language Chatbots
F O R E C A S T
Forecasting
T E X T R A C T P E R S O N A L I Z E
Recommendations

4. © 2019, Amazon Web Services, Inc. or its Affiliates.
THE AWS MACHINE LEARNING STACK
M L F R A M E W O R K S &
I N F R A S T R U C T U R E
A I S E R V I C E S
R E K O G N I T I O N
I M A G E
P O L L Y T R A N S C R I B E T R A N S L A T E C O M P R E H E N D L E XR E K O G N I T I O N
V I D E O
Vision Speech Language Chatbots
F O R E C A S T
Forecasting
T E X T R A C T P E R S O N A L I Z E
Recommendations
M L P L A T F O R M S
A M A Z O N
S A G E M A K E R
B U I L D T R A I N D E P L O Y
Pre-built algorithms & notebooks
Data labeling (G R O U N D T R U T H )
One-click model training & tuning
Optimization (N E O )
One-click deployment & hosting
Reinforcement learningAlgorithms & models ( A W S M A R K E T P L A C E
F O R M A C H I N E L E A R N I N G )

5. © 2019, Amazon Web Services, Inc. or its Affiliates.
THE AWS MACHINE LEARNING STACK
M L F R A M E W O R K S &
I N F R A S T R U C T U R E
A I S E R V I C E S
R E K O G N I T I O N
I M A G E
P O L L Y T R A N S C R I B E T R A N S L A T E C O M P R E H E N D L E XR E K O G N I T I O N
V I D E O
Vision Speech Language Chatbots
F O R E C A S T
Forecasting
T E X T R A C T P E R S O N A L I Z E
Recommendations
M L P L A T F O R M S
F r a m e w o r k s I n t e r f a c e s
I n f r a s t r u c t u r e
E C 2 P 3
& P 3 D N
E C 2 C 5 F P G A s G R E E N G R A S S E L A S T I C
I N F E R E N C E
A M A Z O N
S A G E M A K E R
B U I L D T R A I N D E P L O Y
Pre-built algorithms & notebooks
Data labeling (G R O U N D T R U T H )
One-click model training & tuning
Optimization (N E O )
One-click deployment & hosting
Reinforcement learningAlgorithms & models ( A W S M A R K E T P L A C E
F O R M A C H I N E L E A R N I N G )
A W S D E E P L E A R N I N G A M I

6. © 2019, Amazon Web Services, Inc. or its Affiliates.
MORE MACHINE LEARNING HAPPENS ON AWS THAN ANYWHERE ELSE
10,000+ customers | 2x the customer references | 85% of TensorFlow projects in the cloud happen on AWS

7. © 2019, Amazon Web Services, Inc. or its Affiliates.
THE MACHINE LEARNING PROCESS
1
2
3
1
2
3

8. © 2019, Amazon Web Services, Inc. or its Affiliates.
Amazon SageMaker
A managed service that provides the quickest and
easiest way for data scientists and developers to get
ML models from idea to production

9. © 2019, Amazon Web Services, Inc. or its Affiliates.
AMAZON SAGEMAKER: BUILD, TRAIN AND DEPLOY ML MODELS
1
2
3

10. © 2019, Amazon Web Services, Inc. or its Affiliates.
AMAZON SAGEMAKER GROUND TRUTH
Easily integrate
human labelers
Get accurate
results
K E Y F E A T U R E S
Automatic labeling via
machine learning
Ready-made and
custom workflows for
image bounding box,
segmentation, and text
Label
management
Quickly label
training data
Private and public
human workforce
Label machine learning training data easily and accurately

11. © 2019, Amazon Web Services, Inc. or its Affiliates.
Amazon SageMaker
AMAZON SAGEMAKER NOTEBOOK INSTANCES
• Fully-managed Jupyter Notebook with
flexible choice of ML compute instances
• Anaconda packages and libraries for
common Deep Learning platforms
• Jupyter/JupyterLab interfaces
• Over 200 example notebooks
• Git integration
• Lifecycle Configurations
• Access to Terminal through Jupyter
• Root/non-root access
• VPC & encryption
VPC
ML Compute
Instance
CPU
GPU
ML Storage
CreateNotebookInstance()
HTTPS Endpoint
Client Browser

12. © 2019, Amazon Web Services, Inc. or its Affiliates.
AMAZON SAGEMAKER: BUILD, TRAIN AND DEPLOY ML MODELS

13. © 2019, Amazon Web Services, Inc. or its Affiliates.
AMAZON SAGEMAKER BUILT-IN ALGORITHMS
Algorithms for
“infinite scale”
Distributed by
default
Train on a
data stream
Checkpoint
for re-training
Single pass
training
Not memory
bound
K-Means
K-Nearest Neighbors (k-NN)
PCA
Latent Dirichlec Allocation (LDA)
Factorization Machines
Linear Learner
Neural Topic Model (NTM)
Random Cut Forest (RCF)
Sequence to Sequence (seq2seq)
XGBoost
Image Classification
Object Detection
DeepAR Forecasting
BlazingText
IP Insights
Semantic Segmentation
Object2Vec
UnsupervisedSupervised

14. © 2019, Amazon Web Services, Inc. or its Affiliates.
AMAZON SAGEMAKER: BUILD, TRAIN AND DEPLOY ML MODELS

15. © 2019, Amazon Web Services, Inc. or its Affiliates.
AWS IS FRAMEWORK AGNOSTIC
Choose from popular frameworks
Run them fully managed Or run them yourself

16. © 2019, Amazon Web Services, Inc. or its Affiliates.
APACHE MXNET ON AWS
Near-linear scalability up to 256 GPUs
Dynamic training
MXNet Model Server

17. © 2019, Amazon Web Services, Inc. or its Affiliates.
AMAZON SAGEMAKER TRAINING
Amazon Simple Storage
Service (S3) Bucket
Training Data
Amazon Simple Storage
Service (S3) Bucket
Model Artifacts
BUILT-IN ALGORITHMS
BRING YOUR OWN SCRIPT
BRING YOUR OWN CONTAINER
AWS MARKETPLACE
Elastic Container Registry
Training Config
Instance Type
Intance Count
Container URI
Hyperparameters
etc.
CreateTrainingJob()
Amazon SageMaker Training
ML Compute
Instance
Training Code
ML Compute
Instance
Training Code
ML Compute
Instance
Training Code
…
VPC
ML Storage ML Storage ML Storage

18. © 2019, Amazon Web Services, Inc. or its Affiliates.
AWS MARKETPLACE FOR MACHINE LEARNING
Subscribe in a
single click
Available in
Amazon SageMaker
K E Y F E A T U R E S
Automatic labeling via machine learning
IP protection
Automated billing and metering
Browse or search
AWS Marketplace
S E L L E R S
Broad selection of paid, free, and
open-source algorithms and models
Data protection
B U Y E R S
ML algorithms and models available instantly

19. © 2019, Amazon Web Services, Inc. or its Affiliates.
AMAZON SAGEMAKER REINFORCEMENT LEARNING
Broad support
for frameworks
Broad support for simulation
environments including
SimuLink and MatLab
K E Y F E A T U R E S
TensorFlow, Apache
MXNet, Intel Coach,
and Ray RL support
2D & 3D physics
environments and
OpenAI Gym support
Supports Amazon Sumerian and
Amazon RoboMaker
Fully
managed
Example notebooks
and tutorials
Reinforcement learning for every developer and data scientist

20. © 2019, Amazon Web Services, Inc. or its Affiliates.
AMAZON SAGEMAKER: BUILD, TRAIN AND DEPLOY ML MODELS

21. © 2019, Amazon Web Services, Inc. or its Affiliates.
Amazon SageMaker Hyperparameter Tuning
AMAZON SAGEMAKER HYPERPARAMETER TUNING
Amazon Simple Storage
Service (S3) Bucket
Training Data
Amazon Simple Storage
Service (S3) Bucket
Model Artifacts
BUILT-IN ALGORITHMS
BRING YOUR OWN SCRIPT
BRING YOUR OWN CONTAINER
AWS MARKETPLACE
Elastic Container Registry
Training Config
Instance Type
Intance Count
Container URI
Hyperparameters
etc.
CreateHyperParameterTuningJob()
Tuning Config
Objective Metric
Parameter Ranges
Resource Limits
Strategy
Bayesian Optimization
Random Search
Amazon SageMaker Training
ML Compute
Instance
Training Code
ML Compute
Instance
Training Code
ML Compute
Instance
Training Code
…
VPC
ML Storage ML Storage ML Storage

22. © 2019, Amazon Web Services, Inc. or its Affiliates.
AMAZON SAGEMAKER NEO
K E Y F E A T U R E S
Compiler & run-time are open source 1/10th the size of original models
Train once, run anywhere with 2x the performance

23. © 2019, Amazon Web Services, Inc. or its Affiliates.
AMAZON SAGEMAKER: BUILD, TRAIN AND DEPLOY ML MODELS

24. © 2019, Amazon Web Services, Inc. or its Affiliates.
AMAZON SAGEMAKER HOSTING
CreateModel() > CreateEndpointConfig() > CreateEndpoint()
Amazon Simple Storage
Service (S3) Bucket
Model Artifacts
Amazon SageMaker Hosting
ML Compute
Instance
Model
ML Compute
Instance
Model
ML Compute
Instance
Model
…
BUILT-IN ALGORITHMS
BRING YOUR OWN SCRIPT
BRING YOUR OWN CONTAINER
AWS MARKETPLACE
Elastic Container Registry
Hosting Config
Instance Type
Intance Count
Container URI
etc.
HTTPS Endpoint
Client Application
Input Data (request)
Inference (response)
Endpoint Configuration
VPC

25. © 2019, Amazon Web Services, Inc. or its Affiliates.
AMAZON SAGEMAKER BATCH TRANSFORM
Amazon Simple Storage
Service (S3) Bucket
Input Data
Amazon Simple Storage
Service (S3) Bucket
Results
Amazon SageMaker Batch Transform
…
BUILT-IN ALGORITHMS
BRING YOUR OWN SCRIPT
BRING YOUR OWN CONTAINER
AWS MARKETPLACE
Elastic Container Registry
CreateTransformJob()
Transform Config
Model Name
Transform Resources
Transform Output
Batch Strategy
etc.
ML Compute Instance
Model
Agent
Request
Data
Transformed
Data
ML Compute Instance
Model
Agent
Request
Data
Transformed
Data
VPC

26. © 2019, Amazon Web Services, Inc. or its Affiliates.
AMAZON SAGEMAKER: BUILD, TRAIN AND DEPLOY ML MODELS

27. © 2019, Amazon Web Services, Inc. or its Affiliates.
AUTOSCALING, A/B TESTING AND INFERENCE PIPELINES
Amazon SageMaker Hosting
HTTPS Endpoint
Client Application
Input Data (request)
Inference (response)
Endpoint Configuration
ML Compute Instance
Model 1 Model n
…
Pipeline Model
Production Variant
ML Compute Instance
Model 1 Model n
…
Pipeline Model
ML Compute Instance
Model 1 V2 Model n V2
…
Pipeline Model
Production Variant
VPC
AWS Auto Scaling

28. © 2019, Amazon Web Services, Inc. or its Affiliates.
AMAZON ELASTIC INFERENCE
Match capacity
to demand
Available between 1 to 32
TFLOPS
K E Y F E A T U R E S
Integrated with
Amazon EC2,
Amazon SageMaker, and
Amazon DL AMIs
Support for TensorFlow,
Apache MXNet, and ONNX
with PyTorch coming soon
Single and
mixed-precision
operations
Lower inference costs
Reduce deep learning inference costs up to 75%

29. © 2019, Amazon Web Services, Inc. or its Affiliates.
END-TO-END ML WORKFLOWS AND AUTOMATION
Use end-to-end or pick what you need
Automate with AWS Step Functions, Apache AirFlow or your own orchestrator

30. © 2019, Amazon Web Services, Inc. or its Affiliates.
HOW TO USE AMAZON SAGEMAKER
AWS CLI
AWS Console
AWS SDKs
SageMaker Python SDK

31. © 2019, Amazon Web Services, Inc. or its Affiliates.© 2019, Amazon Web Services, Inc. or its Affiliates.
DEMO

32. © 2019, Amazon Web Services, Inc. or its Affiliates.© 2019, Amazon Web Services, Inc. or its Affiliates.
WAYS TO GET STARTED

33. © 2019, Amazon Web Services, Inc. or its Affiliates.
ONLINE RESOURCES
Amazon SageMaker landing page https://aws.amazon.com/sagemaker/
Amazon SageMaker documentation https://docs.aws.amazon.com/sagemaker/index.html
SageMaker Python SDK https://github.com/aws/sagemaker-python-sdk
SageMaker MXNet Container https://github.com/aws/sagemaker-mxnet-container
SageMaker Tensorflow Container https://github.com/aws/sagemaker-tensorflow-container
SageMaker PyTorch Container https://github.com/aws/sagemaker-pytorch-container
SageMaker Chainer Container https://github.com/aws/sagemaker-chainer-container
SageMaker Scikit-learn Container https://github.com/aws/sagemaker-scikit-learn-container
SageMaker Containers https://github.com/aws/sagemaker-containers
Amazon SageMaker Examples https://github.com/awslabs/amazon-sagemaker-examples
Amazon SageMaker Workshop https://github.com/awslabs/amazon-sagemaker-workshop

34. © 2019, Amazon Web Services, Inc. or its Affiliates.
MACHINE LEARNING UNIVERSITY
Uses the same
materials used to train
Amazon developers
Foundational
knowledge with
real-world application
Structured
courses and
specialist certification
https://aws.amazon.com/training/learning-paths/machine-learning/

35. © 2019, Amazon Web Services, Inc. or its Affiliates.
AMAZON ML SOLUTIONS LAB
Brainstorming Modeling Teaching
Leverage Amazon experts with decades of ML
experience with technologies like Amazon Echo,
Amazon Alexa, Prime Air and Amazon Go
Amazon ML Solutions
Lab provides ML
expertise
Quick turnaround
Demonstration of
functionality
Static data
Limited live data-sets
Core functionality
Limited integration with
production systems
Initial Architecture
Full live data-sets
Enhanced functionality
Link to production systems
Scaled Architecture
Iteration on MVP2
Fully operationally
Fully supported
Live security-approved
account
https://aws.amazon.com/ml-solutions-lab/

36. © 2019, Amazon Web Services, Inc. or its Affiliates.
Artificial Intelligence. Get Started.
Serie webinar in Italiano
Today
27 Giugno25 Giugno
11 Giugno

37. © 2019, Amazon Web Services, Inc. or its Affiliates.
Thank You
Giuseppe Angelo Porcelli
Principal Solutions Architect
Amazon Web Services EMEA