SlideShare una empresa de Scribd logo

Machine Learning Models with Apache MXNet and AWS Fargate

•
•
Amazon Web Services
Amazon Web Services

by Ahmad Khan, Sr. Solutions Architect, AWS Deep Learning has been delivering state of the art results across a growing number of domains and use cases. Correspondingly, Deep Learning models are being deployed across a growing number of applications across segments. In this session, we will dive deep into serving machine learning models in production, and demonstrate how to efficiently deploy and serve models over serverless infrastructure using the open source project Model Server for Apache MXNet, Containers and AWS Fargate.

1 de 32
©2017, AmazonWebServices, Inc. or its Affiliates. All rights reserved. Serverless Model Serving with MXNet and Fargate Hagay Lupesko, Amazon AI ©2018 Amazon Web Services, Inc. or its affiliates, All rights reserved
©2017, AmazonWebServices, Inc. or its Affiliates. All rights reserved. The Amazon Machine Learning Stack FRAMEWORKS & INTERFACES Caffe2 CNTK Apache MXNet PyTorch TensorFlow Chainer Keras Gluon AWSDeepLearningAMIs AmazonSageMaker Rekognition Transcribe Translate Polly Comprehend Lex AWS DeepLens EDUCATION PLATFORM SERVICES APPLICATION SERVICES AmazonMechanical Turk
©2017, AmazonWebServices, Inc. or its Affiliates. All rights reserved. Brief Intro to Deep Learning AI Machine Learning Deep Learning Can machines think? Can machines do what we can? (Turing, 1950) Machine Learning Data Answers Rules Traditional Programming Data Rules Answers
©2017, AmazonWebServices, Inc. or its Affiliates. All rights reserved. Inspired by the brain’s Neurons We have ~100B of them, and ~1Q Synapses w1 w2 wn x1 x2 xn Σ φ Inputs Weights Non-Linear Activation ! … ! = #(% &'( ) *+,+) Brief Intro to Deep Learning – Artificial Neurons
©2017, AmazonWebServices, Inc. or its Affiliates. All rights reserved. Brief Intro to Deep Learning – Neural Networks Output Layer Input Layer Hidden Layers Many More… • Non linear • Hierarchical feature learning • Scalable architecture • Computationally intensive
©2017, AmazonWebServices, Inc. or its Affiliates. All rights reserved. Deep Learning is a Big Deal It has a growing impact on our lives Personalization Logistics Voice Autonomous Vehicles
©2017, AmazonWebServices, Inc. or its Affiliates. All rights reserved. Deep Learning is a Big Deal It’s able to do better than humans (and ML)
©2017, AmazonWebServices, Inc. or its Affiliates. All rights reserved. Model Model Server Mobile Desktop IoT Internet So what does a deployed model looks like?
©2017, AmazonWebServices, Inc. or its Affiliates. All rights reserved. Performance Availability Networking Monitoring Model Decoupling Cross Framework Cross Platform The Undifferentiated Heavy Lifting of Model Serving Model Server for MXNet
©2017, AmazonWebServices, Inc. or its Affiliates. All rights reserved. Apache MXNet - Background • Apache (incubating) open source project • Framework for building and training DNNs • Created by academia (CMU and UW) • Adopted by AWS as DNN framework of choice, Nov 2016 http://mxnet.io
©2017, AmazonWebServices, Inc. or its Affiliates. All rights reserved. Apache MXNet - Highlights • Imperative, Symbolic and Dynamic APIs • APIs in Python, Scala, C, C++, R (and more) • Examples and tutorials Ease of Use • Optimized for CPU, GPU, ARM (and more) • Highly scalable distributed training • Quantization, Sparse, NCCL, and more… Performance • Train on cloud, predict on edge • Model serving framework • ONNX support Portability
©2017, AmazonWebServices, Inc. or its Affiliates. All rights reserved. MXNet Model Server • Machine learning model server • Serves MXNet and ONNX models • Automated HTTP endpoints setup • Auto-scales to all available CPUs and GPUs • Pre-built and configured containers • CLI to package model artifacts for serving • Open source project under AWS Labs http://modelserver.io
©2017, AmazonWebServices, Inc. or its Affiliates. All rights reserved. It’s Demo Time!
©2017, AmazonWebServices, Inc. or its Affiliates. All rights reserved. Model Archive REST and OpenAPI Containerization ONNX Support Operational Metrics
©2017, AmazonWebServices, Inc. or its Affiliates. All rights reserved. Trained Network Model Signature Custom Code Auxiliary Assets Model Archive Model Export CLI Model Archive Back
©2017, AmazonWebServices, Inc. or its Affiliates. All rights reserved. Demo: Model Archive
©2017, AmazonWebServices, Inc. or its Affiliates. All rights reserved. Model Archive REST and OpenAPI Containerization ONNX Support Operational Metrics
©2017, AmazonWebServices, Inc. or its Affiliates. All rights reserved. REST and OpenAPI REST-like endpoint: <model-name>/predict Endpoint auto-generated from the model’s signature.json JSON encoding by default Binary input via request payload OpenAPI support – client code-gen and tooling Back
©2017, AmazonWebServices, Inc. or its Affiliates. All rights reserved. Demo: Endpoints
©2017, AmazonWebServices, Inc. or its Affiliates. All rights reserved. Model Archive REST and OpenAPI Containerization ONNX Support Operational Metrics
©2017, AmazonWebServices, Inc. or its Affiliates. All rights reserved. Containerization • Production-ready orchestration tools: ECS, Docker, Kubernetes • Easy to scale out • Robust and scalable images • Automatically leverages all GPUs and CPUs on host • Pre-configured images on DockerHub: awsdeeplearningteam/mms_cpu Back
©2017, AmazonWebServices, Inc. or its Affiliates. All rights reserved. MMS Docker Image Pull or Build Run Containerization Container Cluster MMS Container MMS ContainerMMS Container MXNet NGINX MXNet Model Server Load Balancer
©2017, AmazonWebServices, Inc. or its Affiliates. All rights reserved. Demo: Containers
©2017, AmazonWebServices, Inc. or its Affiliates. All rights reserved. Model Archive REST and OpenAPI Containerization ONNX Support Operational Metrics
©2017, AmazonWebServices, Inc. or its Affiliates. All rights reserved. • Requests • Latencies • Resources Metrics • Model Name • Host NameDimensions • Log / CSV • AWS CloudWatchTarget Operational Metrics Back
©2017, AmazonWebServices, Inc. or its Affiliates. All rights reserved.
©2017, AmazonWebServices, Inc. or its Affiliates. All rights reserved. Model Archive REST and OpenAPI Containerization ONNX Support Operational Metrics
©2017, AmazonWebServices, Inc. or its Affiliates. All rights reserved. O(n2) Pairs M XNet Caffe2 PyTorch TF CNTKCoreM L TensorRT NGraph SNPEMany Frameworks ONNX Support Many Platforms ONNX: Common IR Supported in MMS
©2017, AmazonWebServices, Inc. or its Affiliates. All rights reserved. + MXNet Model Server AWS Fargate Serverless Model Serving
©2017, AmazonWebServices, Inc. or its Affiliates. All rights reserved. AWS Fargate • Deploy containers without managing hosts or clusters • Seamless scaling • Pay only for CPU and Memory your container consumed Back
©2017, AmazonWebServices, Inc. or its Affiliates. All rights reserved. Demo: MMS on Fargate
©2017, AmazonWebServices, Inc. or its Affiliates. All rights reserved. Try it out, Star and Contribute! http://mxnet.io http://modelserver.io

Recomendados

DataPalooza - A Music Festival themed ML + IoT Workshop
DataPalooza - A Music Festival themed ML + IoT WorkshopDataPalooza - A Music Festival themed ML + IoT Workshop
DataPalooza - A Music Festival themed ML + IoT WorkshopAmazon Web Services
 
Using Amazon SageMaker to build, train, and deploy your ML Models
Using Amazon SageMaker to build, train, and deploy your ML ModelsUsing Amazon SageMaker to build, train, and deploy your ML Models
Using Amazon SageMaker to build, train, and deploy your ML ModelsAmazon Web Services
 
Workshop: Build Deep Learning Applications with TensorFlow and SageMaker
Workshop: Build Deep Learning Applications with TensorFlow and SageMakerWorkshop: Build Deep Learning Applications with TensorFlow and SageMaker
Workshop: Build Deep Learning Applications with TensorFlow and SageMakerAmazon Web Services
 
Build Deep Learning Applications with TensorFlow & SageMaker
Build Deep Learning Applications with TensorFlow & SageMakerBuild Deep Learning Applications with TensorFlow & SageMaker
Build Deep Learning Applications with TensorFlow & SageMakerAmazon Web Services
 
Using Amazon SageMaker to build, train, & deploy your ML Models
Using Amazon SageMaker to build, train, & deploy your ML ModelsUsing Amazon SageMaker to build, train, & deploy your ML Models
Using Amazon SageMaker to build, train, & deploy your ML ModelsAmazon Web Services
 
Kate Werling - Using Amazon SageMaker to build, train, and deploy your ML Mod...
Kate Werling - Using Amazon SageMaker to build, train, and deploy your ML Mod...Kate Werling - Using Amazon SageMaker to build, train, and deploy your ML Mod...
Kate Werling - Using Amazon SageMaker to build, train, and deploy your ML Mod...Amazon Web Services
 
AWS Machine Learning Week SF: Build, Train & Deploy ML Models Using SageMaker
AWS Machine Learning Week SF: Build, Train & Deploy ML Models Using SageMakerAWS Machine Learning Week SF: Build, Train & Deploy ML Models Using SageMaker
AWS Machine Learning Week SF: Build, Train & Deploy ML Models Using SageMakerAmazon Web Services
 
AWS Machine Learning Week SF: Amazon SageMaker & TensorFlow
AWS Machine Learning Week SF: Amazon SageMaker & TensorFlowAWS Machine Learning Week SF: Amazon SageMaker & TensorFlow
AWS Machine Learning Week SF: Amazon SageMaker & TensorFlowAmazon Web Services
 

Recomendados

DataPalooza - A Music Festival themed ML + IoT Workshop
DataPalooza - A Music Festival themed ML + IoT WorkshopDataPalooza - A Music Festival themed ML + IoT Workshop
DataPalooza - A Music Festival themed ML + IoT WorkshopAmazon Web Services
 
Using Amazon SageMaker to build, train, and deploy your ML Models
Using Amazon SageMaker to build, train, and deploy your ML ModelsUsing Amazon SageMaker to build, train, and deploy your ML Models
Using Amazon SageMaker to build, train, and deploy your ML ModelsAmazon Web Services
 
Workshop: Build Deep Learning Applications with TensorFlow and SageMaker
Workshop: Build Deep Learning Applications with TensorFlow and SageMakerWorkshop: Build Deep Learning Applications with TensorFlow and SageMaker
Workshop: Build Deep Learning Applications with TensorFlow and SageMakerAmazon Web Services
 
Build Deep Learning Applications with TensorFlow & SageMaker
Build Deep Learning Applications with TensorFlow & SageMakerBuild Deep Learning Applications with TensorFlow & SageMaker
Build Deep Learning Applications with TensorFlow & SageMakerAmazon Web Services
 
Using Amazon SageMaker to build, train, & deploy your ML Models
Using Amazon SageMaker to build, train, & deploy your ML ModelsUsing Amazon SageMaker to build, train, & deploy your ML Models
Using Amazon SageMaker to build, train, & deploy your ML ModelsAmazon Web Services
 
Kate Werling - Using Amazon SageMaker to build, train, and deploy your ML Mod...
Kate Werling - Using Amazon SageMaker to build, train, and deploy your ML Mod...Kate Werling - Using Amazon SageMaker to build, train, and deploy your ML Mod...
Kate Werling - Using Amazon SageMaker to build, train, and deploy your ML Mod...Amazon Web Services
 
AWS Machine Learning Week SF: Build, Train & Deploy ML Models Using SageMaker
AWS Machine Learning Week SF: Build, Train & Deploy ML Models Using SageMakerAWS Machine Learning Week SF: Build, Train & Deploy ML Models Using SageMaker
AWS Machine Learning Week SF: Build, Train & Deploy ML Models Using SageMakerAmazon Web Services
 
AWS Machine Learning Week SF: Amazon SageMaker & TensorFlow
AWS Machine Learning Week SF: Amazon SageMaker & TensorFlowAWS Machine Learning Week SF: Amazon SageMaker & TensorFlow
AWS Machine Learning Week SF: Amazon SageMaker & TensorFlowAmazon Web Services
 
Building Deep Learning Applications with TensorFlow and Amazon SageMaker
Building Deep Learning Applications with TensorFlow and Amazon SageMakerBuilding Deep Learning Applications with TensorFlow and Amazon SageMaker
Building Deep Learning Applications with TensorFlow and Amazon SageMakerAmazon Web Services
 
A New Way to Learn Machine Learning
A New Way to Learn Machine LearningA New Way to Learn Machine Learning
A New Way to Learn Machine LearningAmazon Web Services
 
Keith Steward - SageMaker Algorithms Infinitely Scalable Machine Learning_VK.pdf
Keith Steward - SageMaker Algorithms Infinitely Scalable Machine Learning_VK.pdfKeith Steward - SageMaker Algorithms Infinitely Scalable Machine Learning_VK.pdf
Keith Steward - SageMaker Algorithms Infinitely Scalable Machine Learning_VK.pdfAmazon Web Services
 
SageMaker Algorithms Infinitely Scalable Machine Learning
SageMaker Algorithms Infinitely Scalable Machine LearningSageMaker Algorithms Infinitely Scalable Machine Learning
SageMaker Algorithms Infinitely Scalable Machine LearningAmazon Web Services
 
Speed up your Machine Learning workflows with build-in algorithms
Speed up your Machine Learning workflows with build-in algorithmsSpeed up your Machine Learning workflows with build-in algorithms
Speed up your Machine Learning workflows with build-in algorithmsJulien SIMON
 
Amazon SageMaker workshop
Amazon SageMaker workshopAmazon SageMaker workshop
Amazon SageMaker workshopJulien SIMON
 
Using Amazon SageMaker to build, train, and deploy your ML Models
Using Amazon SageMaker to build, train, and deploy your ML ModelsUsing Amazon SageMaker to build, train, and deploy your ML Models
Using Amazon SageMaker to build, train, and deploy your ML ModelsAmazon Web Services
 
Integrating Deep Learning into your Enterprise
Integrating Deep Learning into your EnterpriseIntegrating Deep Learning into your Enterprise
Integrating Deep Learning into your EnterpriseAmazon Web Services
 
Using Amazon SageMaker to build, train, and deploy your ML Models
Using Amazon SageMaker to build, train, and deploy your ML ModelsUsing Amazon SageMaker to build, train, and deploy your ML Models
Using Amazon SageMaker to build, train, and deploy your ML ModelsAmazon Web Services
 
Build, Train, & Deploy ML Models Using SageMaker
Build, Train, & Deploy ML Models Using SageMakerBuild, Train, & Deploy ML Models Using SageMaker
Build, Train, & Deploy ML Models Using SageMakerAmazon Web Services
 
Demystifying Machine Learning on AWS
Demystifying Machine Learning on AWSDemystifying Machine Learning on AWS
Demystifying Machine Learning on AWSAmazon Web Services
 
Build text analytics solutions with AWS ML Services
Build text analytics solutions with AWS ML ServicesBuild text analytics solutions with AWS ML Services
Build text analytics solutions with AWS ML ServicesAmazon Web Services
 
Automate for Efficiency with Amazon Transcribe and Amazon Translate
Automate for Efficiency with Amazon Transcribe and Amazon TranslateAutomate for Efficiency with Amazon Transcribe and Amazon Translate
Automate for Efficiency with Amazon Transcribe and Amazon TranslateAmazon Web Services
 
Time series modeling workd AMLD 2018 Lausanne
Time series modeling workd AMLD 2018 LausanneTime series modeling workd AMLD 2018 Lausanne
Time series modeling workd AMLD 2018 LausanneSunil Mallya
 
Accelerate your Machine Learning workflows with Amazon SageMaker
Accelerate your Machine Learning workflows with Amazon SageMakerAccelerate your Machine Learning workflows with Amazon SageMaker
Accelerate your Machine Learning workflows with Amazon SageMakerJulien SIMON
 
Supercharge your Machine Learning Solutions with Amazon SageMaker
Supercharge your Machine Learning Solutions with Amazon SageMakerSupercharge your Machine Learning Solutions with Amazon SageMaker
Supercharge your Machine Learning Solutions with Amazon SageMakerAmazon Web Services
 
Introducing aws deep lens
Introducing aws deep lensIntroducing aws deep lens
Introducing aws deep lensRichard Harvey
 
Optimize your Machine Learning Workloads on AWS (July 2019)
Optimize your Machine Learning Workloads on AWS (July 2019)Optimize your Machine Learning Workloads on AWS (July 2019)
Optimize your Machine Learning Workloads on AWS (July 2019)Julien SIMON
 
Optimize your ML workloads_converted.pdf
Optimize your ML workloads_converted.pdfOptimize your ML workloads_converted.pdf
Optimize your ML workloads_converted.pdfAmazon Web Services
 
Accelerating Apache MXNet Models on Apple Platforms Using Core ML - MCL311 - ...
Accelerating Apache MXNet Models on Apple Platforms Using Core ML - MCL311 - ...Accelerating Apache MXNet Models on Apple Platforms Using Core ML - MCL311 - ...
Accelerating Apache MXNet Models on Apple Platforms Using Core ML - MCL311 - ...Amazon Web Services
 
Emotion recognition in images: from idea to a model in production - Nordic DS...
Emotion recognition in images: from idea to a model in production - Nordic DS...Emotion recognition in images: from idea to a model in production - Nordic DS...
Emotion recognition in images: from idea to a model in production - Nordic DS...Hagay Lupesko
 
Model Serving for Deep Learning with MXNet Model Server
Model Serving for Deep Learning with MXNet Model ServerModel Serving for Deep Learning with MXNet Model Server
Model Serving for Deep Learning with MXNet Model ServerAmazon Web Services
 

Más contenido relacionado

La actualidad más candente

Building Deep Learning Applications with TensorFlow and Amazon SageMaker
Building Deep Learning Applications with TensorFlow and Amazon SageMakerBuilding Deep Learning Applications with TensorFlow and Amazon SageMaker
Building Deep Learning Applications with TensorFlow and Amazon SageMakerAmazon Web Services
 
A New Way to Learn Machine Learning
A New Way to Learn Machine LearningA New Way to Learn Machine Learning
A New Way to Learn Machine LearningAmazon Web Services
 
Keith Steward - SageMaker Algorithms Infinitely Scalable Machine Learning_VK.pdf
Keith Steward - SageMaker Algorithms Infinitely Scalable Machine Learning_VK.pdfKeith Steward - SageMaker Algorithms Infinitely Scalable Machine Learning_VK.pdf
Keith Steward - SageMaker Algorithms Infinitely Scalable Machine Learning_VK.pdfAmazon Web Services
 
SageMaker Algorithms Infinitely Scalable Machine Learning
SageMaker Algorithms Infinitely Scalable Machine LearningSageMaker Algorithms Infinitely Scalable Machine Learning
SageMaker Algorithms Infinitely Scalable Machine LearningAmazon Web Services
 
Speed up your Machine Learning workflows with build-in algorithms
Speed up your Machine Learning workflows with build-in algorithmsSpeed up your Machine Learning workflows with build-in algorithms
Speed up your Machine Learning workflows with build-in algorithmsJulien SIMON
 
Amazon SageMaker workshop
Amazon SageMaker workshopAmazon SageMaker workshop
Amazon SageMaker workshopJulien SIMON
 
Using Amazon SageMaker to build, train, and deploy your ML Models
Using Amazon SageMaker to build, train, and deploy your ML ModelsUsing Amazon SageMaker to build, train, and deploy your ML Models
Using Amazon SageMaker to build, train, and deploy your ML ModelsAmazon Web Services
 
Integrating Deep Learning into your Enterprise
Integrating Deep Learning into your EnterpriseIntegrating Deep Learning into your Enterprise
Integrating Deep Learning into your EnterpriseAmazon Web Services
 
Using Amazon SageMaker to build, train, and deploy your ML Models
Using Amazon SageMaker to build, train, and deploy your ML ModelsUsing Amazon SageMaker to build, train, and deploy your ML Models
Using Amazon SageMaker to build, train, and deploy your ML ModelsAmazon Web Services
 
Build, Train, & Deploy ML Models Using SageMaker
Build, Train, & Deploy ML Models Using SageMakerBuild, Train, & Deploy ML Models Using SageMaker
Build, Train, & Deploy ML Models Using SageMakerAmazon Web Services
 
Demystifying Machine Learning on AWS
Demystifying Machine Learning on AWSDemystifying Machine Learning on AWS
Demystifying Machine Learning on AWSAmazon Web Services
 
Build text analytics solutions with AWS ML Services
Build text analytics solutions with AWS ML ServicesBuild text analytics solutions with AWS ML Services
Build text analytics solutions with AWS ML ServicesAmazon Web Services
 
Automate for Efficiency with Amazon Transcribe and Amazon Translate
Automate for Efficiency with Amazon Transcribe and Amazon TranslateAutomate for Efficiency with Amazon Transcribe and Amazon Translate
Automate for Efficiency with Amazon Transcribe and Amazon TranslateAmazon Web Services
 
Time series modeling workd AMLD 2018 Lausanne
Time series modeling workd AMLD 2018 LausanneTime series modeling workd AMLD 2018 Lausanne
Time series modeling workd AMLD 2018 LausanneSunil Mallya
 
Accelerate your Machine Learning workflows with Amazon SageMaker
Accelerate your Machine Learning workflows with Amazon SageMakerAccelerate your Machine Learning workflows with Amazon SageMaker
Accelerate your Machine Learning workflows with Amazon SageMakerJulien SIMON
 
Supercharge your Machine Learning Solutions with Amazon SageMaker
Supercharge your Machine Learning Solutions with Amazon SageMakerSupercharge your Machine Learning Solutions with Amazon SageMaker
Supercharge your Machine Learning Solutions with Amazon SageMakerAmazon Web Services
 
Introducing aws deep lens
Introducing aws deep lensIntroducing aws deep lens
Introducing aws deep lensRichard Harvey
 
Optimize your Machine Learning Workloads on AWS (July 2019)
Optimize your Machine Learning Workloads on AWS (July 2019)Optimize your Machine Learning Workloads on AWS (July 2019)
Optimize your Machine Learning Workloads on AWS (July 2019)Julien SIMON
 
Optimize your ML workloads_converted.pdf
Optimize your ML workloads_converted.pdfOptimize your ML workloads_converted.pdf
Optimize your ML workloads_converted.pdfAmazon Web Services
 
Accelerating Apache MXNet Models on Apple Platforms Using Core ML - MCL311 - ...
Accelerating Apache MXNet Models on Apple Platforms Using Core ML - MCL311 - ...Accelerating Apache MXNet Models on Apple Platforms Using Core ML - MCL311 - ...
Accelerating Apache MXNet Models on Apple Platforms Using Core ML - MCL311 - ...Amazon Web Services
 

La actualidad más candente (20)

Building Deep Learning Applications with TensorFlow and Amazon SageMaker
Building Deep Learning Applications with TensorFlow and Amazon SageMakerBuilding Deep Learning Applications with TensorFlow and Amazon SageMaker
Building Deep Learning Applications with TensorFlow and Amazon SageMaker
 
A New Way to Learn Machine Learning
A New Way to Learn Machine LearningA New Way to Learn Machine Learning
A New Way to Learn Machine Learning
 
Keith Steward - SageMaker Algorithms Infinitely Scalable Machine Learning_VK.pdf
Keith Steward - SageMaker Algorithms Infinitely Scalable Machine Learning_VK.pdfKeith Steward - SageMaker Algorithms Infinitely Scalable Machine Learning_VK.pdf
Keith Steward - SageMaker Algorithms Infinitely Scalable Machine Learning_VK.pdf
 
SageMaker Algorithms Infinitely Scalable Machine Learning
SageMaker Algorithms Infinitely Scalable Machine LearningSageMaker Algorithms Infinitely Scalable Machine Learning
SageMaker Algorithms Infinitely Scalable Machine Learning
 
Speed up your Machine Learning workflows with build-in algorithms
Speed up your Machine Learning workflows with build-in algorithmsSpeed up your Machine Learning workflows with build-in algorithms
Speed up your Machine Learning workflows with build-in algorithms
 
Amazon SageMaker workshop
Amazon SageMaker workshopAmazon SageMaker workshop
Amazon SageMaker workshop
 
Using Amazon SageMaker to build, train, and deploy your ML Models
Using Amazon SageMaker to build, train, and deploy your ML ModelsUsing Amazon SageMaker to build, train, and deploy your ML Models
Using Amazon SageMaker to build, train, and deploy your ML Models
 
Integrating Deep Learning into your Enterprise
Integrating Deep Learning into your EnterpriseIntegrating Deep Learning into your Enterprise
Integrating Deep Learning into your Enterprise
 
Using Amazon SageMaker to build, train, and deploy your ML Models
Using Amazon SageMaker to build, train, and deploy your ML ModelsUsing Amazon SageMaker to build, train, and deploy your ML Models
Using Amazon SageMaker to build, train, and deploy your ML Models
 
Build, Train, & Deploy ML Models Using SageMaker
Build, Train, & Deploy ML Models Using SageMakerBuild, Train, & Deploy ML Models Using SageMaker
Build, Train, & Deploy ML Models Using SageMaker
 
Demystifying Machine Learning on AWS
Demystifying Machine Learning on AWSDemystifying Machine Learning on AWS
Demystifying Machine Learning on AWS
 
Build text analytics solutions with AWS ML Services
Build text analytics solutions with AWS ML ServicesBuild text analytics solutions with AWS ML Services
Build text analytics solutions with AWS ML Services
 
Automate for Efficiency with Amazon Transcribe and Amazon Translate
Automate for Efficiency with Amazon Transcribe and Amazon TranslateAutomate for Efficiency with Amazon Transcribe and Amazon Translate
Automate for Efficiency with Amazon Transcribe and Amazon Translate
 
Time series modeling workd AMLD 2018 Lausanne
Time series modeling workd AMLD 2018 LausanneTime series modeling workd AMLD 2018 Lausanne
Time series modeling workd AMLD 2018 Lausanne
 
Accelerate your Machine Learning workflows with Amazon SageMaker
Accelerate your Machine Learning workflows with Amazon SageMakerAccelerate your Machine Learning workflows with Amazon SageMaker
Accelerate your Machine Learning workflows with Amazon SageMaker
 
Supercharge your Machine Learning Solutions with Amazon SageMaker
Supercharge your Machine Learning Solutions with Amazon SageMakerSupercharge your Machine Learning Solutions with Amazon SageMaker
Supercharge your Machine Learning Solutions with Amazon SageMaker
 
Introducing aws deep lens
Introducing aws deep lensIntroducing aws deep lens
Introducing aws deep lens
 
Optimize your Machine Learning Workloads on AWS (July 2019)
Optimize your Machine Learning Workloads on AWS (July 2019)Optimize your Machine Learning Workloads on AWS (July 2019)
Optimize your Machine Learning Workloads on AWS (July 2019)
 
Optimize your ML workloads_converted.pdf
Optimize your ML workloads_converted.pdfOptimize your ML workloads_converted.pdf
Optimize your ML workloads_converted.pdf
 
Accelerating Apache MXNet Models on Apple Platforms Using Core ML - MCL311 - ...
Accelerating Apache MXNet Models on Apple Platforms Using Core ML - MCL311 - ...Accelerating Apache MXNet Models on Apple Platforms Using Core ML - MCL311 - ...
Accelerating Apache MXNet Models on Apple Platforms Using Core ML - MCL311 - ...
 

Similar a Machine Learning Models with Apache MXNet and AWS Fargate

Emotion recognition in images: from idea to a model in production - Nordic DS...
Emotion recognition in images: from idea to a model in production - Nordic DS...Emotion recognition in images: from idea to a model in production - Nordic DS...
Emotion recognition in images: from idea to a model in production - Nordic DS...Hagay Lupesko
 
Model Serving for Deep Learning with MXNet Model Server
Model Serving for Deep Learning with MXNet Model ServerModel Serving for Deep Learning with MXNet Model Server
Model Serving for Deep Learning with MXNet Model ServerAmazon Web Services
 
Machine Learning State of the Union - MCL210 - re:Invent 2017
Machine Learning State of the Union - MCL210 - re:Invent 2017Machine Learning State of the Union - MCL210 - re:Invent 2017
Machine Learning State of the Union - MCL210 - re:Invent 2017Amazon Web Services
 
Artificial Intelligence (Machine Learning) on AWS: How to Start
Artificial Intelligence (Machine Learning) on AWS: How to StartArtificial Intelligence (Machine Learning) on AWS: How to Start
Artificial Intelligence (Machine Learning) on AWS: How to StartVladimir Simek
 
Deep Learning Using Caffe2 on AWS - MCL313 - re:Invent 2017
Deep Learning Using Caffe2 on AWS - MCL313 - re:Invent 2017Deep Learning Using Caffe2 on AWS - MCL313 - re:Invent 2017
Deep Learning Using Caffe2 on AWS - MCL313 - re:Invent 2017Amazon Web Services
 
ONNX - The Lingua Franca of Deep Learning
ONNX - The Lingua Franca of Deep LearningONNX - The Lingua Franca of Deep Learning
ONNX - The Lingua Franca of Deep LearningHagay Lupesko
 
Maschinelles Lernen auf AWS für Entwickler, Data Scientists und Experten
Maschinelles Lernen auf AWS für Entwickler, Data Scientists und ExpertenMaschinelles Lernen auf AWS für Entwickler, Data Scientists und Experten
Maschinelles Lernen auf AWS für Entwickler, Data Scientists und ExpertenAWS Germany
 
New AI/ML services at AWS re:Invent 2017
New AI/ML services at AWS re:Invent 2017New AI/ML services at AWS re:Invent 2017
New AI/ML services at AWS re:Invent 2017Julien SIMON
 
Emotion Recognition in Images
Emotion Recognition in ImagesEmotion Recognition in Images
Emotion Recognition in ImagesApache MXNet
 
MCL303-Deep Learning with Apache MXNet and Gluon
MCL303-Deep Learning with Apache MXNet and GluonMCL303-Deep Learning with Apache MXNet and Gluon
MCL303-Deep Learning with Apache MXNet and GluonAmazon Web Services
 
Artificial Intelligence (Machine Learning) on AWS: How to Start
Artificial Intelligence (Machine Learning) on AWS: How to StartArtificial Intelligence (Machine Learning) on AWS: How to Start
Artificial Intelligence (Machine Learning) on AWS: How to StartVladimir Simek
 
Build a Java Spring Application on Amazon ECS - CON332 - re:Invent 2017
Build a Java Spring Application on Amazon ECS - CON332 - re:Invent 2017Build a Java Spring Application on Amazon ECS - CON332 - re:Invent 2017
Build a Java Spring Application on Amazon ECS - CON332 - re:Invent 2017Amazon Web Services
 
Stack Mastery: Create and Optimize Advanced AWS CloudFormation Templates - DE...
Stack Mastery: Create and Optimize Advanced AWS CloudFormation Templates - DE...Stack Mastery: Create and Optimize Advanced AWS CloudFormation Templates - DE...
Stack Mastery: Create and Optimize Advanced AWS CloudFormation Templates - DE...Amazon Web Services
 
AWS User Group Wellington - re:Invent 2017 Recap
AWS User Group Wellington - re:Invent 2017 RecapAWS User Group Wellington - re:Invent 2017 Recap
AWS User Group Wellington - re:Invent 2017 RecapAPI Talent
 
Integrating Deep Learning into your Enterprise
Integrating Deep Learning into your EnterpriseIntegrating Deep Learning into your Enterprise
Integrating Deep Learning into your EnterpriseAmazon Web Services
 
AWS Machine Learning Week SF: Integrating Deep Learning into Your Enterprise
AWS Machine Learning Week SF: Integrating Deep Learning into Your EnterpriseAWS Machine Learning Week SF: Integrating Deep Learning into Your Enterprise
AWS Machine Learning Week SF: Integrating Deep Learning into Your EnterpriseAmazon Web Services
 
Moving to Amazon ECS – the Not-So-Obvious Benefits - CON356 - re:Invent 2017
Moving to Amazon ECS – the Not-So-Obvious Benefits - CON356 - re:Invent 2017Moving to Amazon ECS – the Not-So-Obvious Benefits - CON356 - re:Invent 2017
Moving to Amazon ECS – the Not-So-Obvious Benefits - CON356 - re:Invent 2017Amazon Web Services
 
Integrating Deep Learning In the Enterprise
Integrating Deep Learning In the EnterpriseIntegrating Deep Learning In the Enterprise
Integrating Deep Learning In the EnterpriseAmazon Web Services
 
Semplificare la gestione dei container con i servizi AWS
Semplificare la gestione dei container con i servizi AWSSemplificare la gestione dei container con i servizi AWS
Semplificare la gestione dei container con i servizi AWSAmazon Web Services
 
High Performance Computing on AWS
High Performance Computing on AWSHigh Performance Computing on AWS
High Performance Computing on AWSAmazon Web Services
 

Similar a Machine Learning Models with Apache MXNet and AWS Fargate (20)

Emotion recognition in images: from idea to a model in production - Nordic DS...
Emotion recognition in images: from idea to a model in production - Nordic DS...Emotion recognition in images: from idea to a model in production - Nordic DS...
Emotion recognition in images: from idea to a model in production - Nordic DS...
 
Model Serving for Deep Learning with MXNet Model Server
Model Serving for Deep Learning with MXNet Model ServerModel Serving for Deep Learning with MXNet Model Server
Model Serving for Deep Learning with MXNet Model Server
 
Machine Learning State of the Union - MCL210 - re:Invent 2017
Machine Learning State of the Union - MCL210 - re:Invent 2017Machine Learning State of the Union - MCL210 - re:Invent 2017
Machine Learning State of the Union - MCL210 - re:Invent 2017
 
Artificial Intelligence (Machine Learning) on AWS: How to Start
Artificial Intelligence (Machine Learning) on AWS: How to StartArtificial Intelligence (Machine Learning) on AWS: How to Start
Artificial Intelligence (Machine Learning) on AWS: How to Start
 
Deep Learning Using Caffe2 on AWS - MCL313 - re:Invent 2017
Deep Learning Using Caffe2 on AWS - MCL313 - re:Invent 2017Deep Learning Using Caffe2 on AWS - MCL313 - re:Invent 2017
Deep Learning Using Caffe2 on AWS - MCL313 - re:Invent 2017
 
ONNX - The Lingua Franca of Deep Learning
ONNX - The Lingua Franca of Deep LearningONNX - The Lingua Franca of Deep Learning
ONNX - The Lingua Franca of Deep Learning
 
Maschinelles Lernen auf AWS für Entwickler, Data Scientists und Experten
Maschinelles Lernen auf AWS für Entwickler, Data Scientists und ExpertenMaschinelles Lernen auf AWS für Entwickler, Data Scientists und Experten
Maschinelles Lernen auf AWS für Entwickler, Data Scientists und Experten
 
New AI/ML services at AWS re:Invent 2017
New AI/ML services at AWS re:Invent 2017New AI/ML services at AWS re:Invent 2017
New AI/ML services at AWS re:Invent 2017
 
Emotion Recognition in Images
Emotion Recognition in ImagesEmotion Recognition in Images
Emotion Recognition in Images
 
MCL303-Deep Learning with Apache MXNet and Gluon
MCL303-Deep Learning with Apache MXNet and GluonMCL303-Deep Learning with Apache MXNet and Gluon
MCL303-Deep Learning with Apache MXNet and Gluon
 
Artificial Intelligence (Machine Learning) on AWS: How to Start
Artificial Intelligence (Machine Learning) on AWS: How to StartArtificial Intelligence (Machine Learning) on AWS: How to Start
Artificial Intelligence (Machine Learning) on AWS: How to Start
 
Build a Java Spring Application on Amazon ECS - CON332 - re:Invent 2017
Build a Java Spring Application on Amazon ECS - CON332 - re:Invent 2017Build a Java Spring Application on Amazon ECS - CON332 - re:Invent 2017
Build a Java Spring Application on Amazon ECS - CON332 - re:Invent 2017
 
Stack Mastery: Create and Optimize Advanced AWS CloudFormation Templates - DE...
Stack Mastery: Create and Optimize Advanced AWS CloudFormation Templates - DE...Stack Mastery: Create and Optimize Advanced AWS CloudFormation Templates - DE...
Stack Mastery: Create and Optimize Advanced AWS CloudFormation Templates - DE...
 
AWS User Group Wellington - re:Invent 2017 Recap
AWS User Group Wellington - re:Invent 2017 RecapAWS User Group Wellington - re:Invent 2017 Recap
AWS User Group Wellington - re:Invent 2017 Recap
 
Integrating Deep Learning into your Enterprise
Integrating Deep Learning into your EnterpriseIntegrating Deep Learning into your Enterprise
Integrating Deep Learning into your Enterprise
 
AWS Machine Learning Week SF: Integrating Deep Learning into Your Enterprise
AWS Machine Learning Week SF: Integrating Deep Learning into Your EnterpriseAWS Machine Learning Week SF: Integrating Deep Learning into Your Enterprise
AWS Machine Learning Week SF: Integrating Deep Learning into Your Enterprise
 
Moving to Amazon ECS – the Not-So-Obvious Benefits - CON356 - re:Invent 2017
Moving to Amazon ECS – the Not-So-Obvious Benefits - CON356 - re:Invent 2017Moving to Amazon ECS – the Not-So-Obvious Benefits - CON356 - re:Invent 2017
Moving to Amazon ECS – the Not-So-Obvious Benefits - CON356 - re:Invent 2017
 
Integrating Deep Learning In the Enterprise
Integrating Deep Learning In the EnterpriseIntegrating Deep Learning In the Enterprise
Integrating Deep Learning In the Enterprise
 
Semplificare la gestione dei container con i servizi AWS
Semplificare la gestione dei container con i servizi AWSSemplificare la gestione dei container con i servizi AWS
Semplificare la gestione dei container con i servizi AWS
 
High Performance Computing on AWS
High Performance Computing on AWSHigh Performance Computing on AWS
High Performance Computing on AWS
 

Más de Amazon Web Services

Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...Amazon Web Services
 
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...Amazon Web Services
 
Esegui pod serverless con Amazon EKS e AWS Fargate
Esegui pod serverless con Amazon EKS e AWS FargateEsegui pod serverless con Amazon EKS e AWS Fargate
Esegui pod serverless con Amazon EKS e AWS FargateAmazon Web Services
 
Costruire Applicazioni Moderne con AWS
Costruire Applicazioni Moderne con AWSCostruire Applicazioni Moderne con AWS
Costruire Applicazioni Moderne con AWSAmazon Web Services
 
Come spendere fino al 90% in meno con i container e le istanze spot
Come spendere fino al 90% in meno con i container e le istanze spot Come spendere fino al 90% in meno con i container e le istanze spot
Come spendere fino al 90% in meno con i container e le istanze spot Amazon Web Services
 
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...Amazon Web Services
 
OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...Amazon Web Services
 
Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads
Microsoft Active Directory su AWS per supportare i tuoi Windows WorkloadsMicrosoft Active Directory su AWS per supportare i tuoi Windows Workloads
Microsoft Active Directory su AWS per supportare i tuoi Windows WorkloadsAmazon Web Services
 
Database Oracle e VMware Cloud on AWS i miti da sfatare
Database Oracle e VMware Cloud on AWS i miti da sfatareDatabase Oracle e VMware Cloud on AWS i miti da sfatare
Database Oracle e VMware Cloud on AWS i miti da sfatareAmazon Web Services
 
Crea la tua prima serverless ledger-based app con QLDB e NodeJS
Crea la tua prima serverless ledger-based app con QLDB e NodeJSCrea la tua prima serverless ledger-based app con QLDB e NodeJS
Crea la tua prima serverless ledger-based app con QLDB e NodeJSAmazon Web Services
 
API moderne real-time per applicazioni mobili e web
API moderne real-time per applicazioni mobili e webAPI moderne real-time per applicazioni mobili e web
API moderne real-time per applicazioni mobili e webAmazon Web Services
 
Database Oracle e VMware Cloud™ on AWS: i miti da sfatare
Database Oracle e VMware Cloud™ on AWS: i miti da sfatareDatabase Oracle e VMware Cloud™ on AWS: i miti da sfatare
Database Oracle e VMware Cloud™ on AWS: i miti da sfatareAmazon Web Services
 
Tools for building your MVP on AWS
Tools for building your MVP on AWSTools for building your MVP on AWS
Tools for building your MVP on AWSAmazon Web Services
 
How to Build a Winning Pitch Deck
How to Build a Winning Pitch DeckHow to Build a Winning Pitch Deck
How to Build a Winning Pitch DeckAmazon Web Services
 
Building a web application without servers
Building a web application without serversBuilding a web application without servers
Building a web application without serversAmazon Web Services
 
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...Amazon Web Services
 
Introduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container ServiceIntroduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container ServiceAmazon Web Services
 

Más de Amazon Web Services (20)

Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
 
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
 
Esegui pod serverless con Amazon EKS e AWS Fargate
Esegui pod serverless con Amazon EKS e AWS FargateEsegui pod serverless con Amazon EKS e AWS Fargate
Esegui pod serverless con Amazon EKS e AWS Fargate
 
Costruire Applicazioni Moderne con AWS
Costruire Applicazioni Moderne con AWSCostruire Applicazioni Moderne con AWS
Costruire Applicazioni Moderne con AWS
 
Come spendere fino al 90% in meno con i container e le istanze spot
Come spendere fino al 90% in meno con i container e le istanze spot Come spendere fino al 90% in meno con i container e le istanze spot
Come spendere fino al 90% in meno con i container e le istanze spot
 
Open banking as a service
Open banking as a serviceOpen banking as a service
Open banking as a service
 
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
 
OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...
 
Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads
Microsoft Active Directory su AWS per supportare i tuoi Windows WorkloadsMicrosoft Active Directory su AWS per supportare i tuoi Windows Workloads
Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads
 
Computer Vision con AWS
Computer Vision con AWSComputer Vision con AWS
Computer Vision con AWS
 
Database Oracle e VMware Cloud on AWS i miti da sfatare
Database Oracle e VMware Cloud on AWS i miti da sfatareDatabase Oracle e VMware Cloud on AWS i miti da sfatare
Database Oracle e VMware Cloud on AWS i miti da sfatare
 
Crea la tua prima serverless ledger-based app con QLDB e NodeJS
Crea la tua prima serverless ledger-based app con QLDB e NodeJSCrea la tua prima serverless ledger-based app con QLDB e NodeJS
Crea la tua prima serverless ledger-based app con QLDB e NodeJS
 
API moderne real-time per applicazioni mobili e web
API moderne real-time per applicazioni mobili e webAPI moderne real-time per applicazioni mobili e web
API moderne real-time per applicazioni mobili e web
 
Database Oracle e VMware Cloud™ on AWS: i miti da sfatare
Database Oracle e VMware Cloud™ on AWS: i miti da sfatareDatabase Oracle e VMware Cloud™ on AWS: i miti da sfatare
Database Oracle e VMware Cloud™ on AWS: i miti da sfatare
 
Tools for building your MVP on AWS
Tools for building your MVP on AWSTools for building your MVP on AWS
Tools for building your MVP on AWS
 
How to Build a Winning Pitch Deck
How to Build a Winning Pitch DeckHow to Build a Winning Pitch Deck
How to Build a Winning Pitch Deck
 
Building a web application without servers
Building a web application without serversBuilding a web application without servers
Building a web application without servers
 
Fundraising Essentials
Fundraising EssentialsFundraising Essentials
Fundraising Essentials
 
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
 
Introduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container ServiceIntroduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container Service
 

Machine Learning Models with Apache MXNet and AWS Fargate

  • 1. ©2017, AmazonWebServices, Inc. or its Affiliates. All rights reserved. Serverless Model Serving with MXNet and Fargate Hagay Lupesko, Amazon AI ©2018 Amazon Web Services, Inc. or its affiliates, All rights reserved
  • 2. ©2017, AmazonWebServices, Inc. or its Affiliates. All rights reserved. The Amazon Machine Learning Stack FRAMEWORKS & INTERFACES Caffe2 CNTK Apache MXNet PyTorch TensorFlow Chainer Keras Gluon AWSDeepLearningAMIs AmazonSageMaker Rekognition Transcribe Translate Polly Comprehend Lex AWS DeepLens EDUCATION PLATFORM SERVICES APPLICATION SERVICES AmazonMechanical Turk
  • 3. ©2017, AmazonWebServices, Inc. or its Affiliates. All rights reserved. Brief Intro to Deep Learning AI Machine Learning Deep Learning Can machines think? Can machines do what we can? (Turing, 1950) Machine Learning Data Answers Rules Traditional Programming Data Rules Answers
  • 4. ©2017, AmazonWebServices, Inc. or its Affiliates. All rights reserved. Inspired by the brain’s Neurons We have ~100B of them, and ~1Q Synapses w1 w2 wn x1 x2 xn Σ φ Inputs Weights Non-Linear Activation ! … ! = #(% &'( ) *+,+) Brief Intro to Deep Learning – Artificial Neurons
  • 5. ©2017, AmazonWebServices, Inc. or its Affiliates. All rights reserved. Brief Intro to Deep Learning – Neural Networks Output Layer Input Layer Hidden Layers Many More… • Non linear • Hierarchical feature learning • Scalable architecture • Computationally intensive
  • 6. ©2017, AmazonWebServices, Inc. or its Affiliates. All rights reserved. Deep Learning is a Big Deal It has a growing impact on our lives Personalization Logistics Voice Autonomous Vehicles
  • 7. ©2017, AmazonWebServices, Inc. or its Affiliates. All rights reserved. Deep Learning is a Big Deal It’s able to do better than humans (and ML)
  • 8. ©2017, AmazonWebServices, Inc. or its Affiliates. All rights reserved. Model Model Server Mobile Desktop IoT Internet So what does a deployed model looks like?
  • 9. ©2017, AmazonWebServices, Inc. or its Affiliates. All rights reserved. Performance Availability Networking Monitoring Model Decoupling Cross Framework Cross Platform The Undifferentiated Heavy Lifting of Model Serving Model Server for MXNet
  • 10. ©2017, AmazonWebServices, Inc. or its Affiliates. All rights reserved. Apache MXNet - Background • Apache (incubating) open source project • Framework for building and training DNNs • Created by academia (CMU and UW) • Adopted by AWS as DNN framework of choice, Nov 2016 http://mxnet.io
  • 11. ©2017, AmazonWebServices, Inc. or its Affiliates. All rights reserved. Apache MXNet - Highlights • Imperative, Symbolic and Dynamic APIs • APIs in Python, Scala, C, C++, R (and more) • Examples and tutorials Ease of Use • Optimized for CPU, GPU, ARM (and more) • Highly scalable distributed training • Quantization, Sparse, NCCL, and more… Performance • Train on cloud, predict on edge • Model serving framework • ONNX support Portability
  • 12. ©2017, AmazonWebServices, Inc. or its Affiliates. All rights reserved. MXNet Model Server • Machine learning model server • Serves MXNet and ONNX models • Automated HTTP endpoints setup • Auto-scales to all available CPUs and GPUs • Pre-built and configured containers • CLI to package model artifacts for serving • Open source project under AWS Labs http://modelserver.io
  • 13. ©2017, AmazonWebServices, Inc. or its Affiliates. All rights reserved. It’s Demo Time!
  • 14. ©2017, AmazonWebServices, Inc. or its Affiliates. All rights reserved. Model Archive REST and OpenAPI Containerization ONNX Support Operational Metrics
  • 15. ©2017, AmazonWebServices, Inc. or its Affiliates. All rights reserved. Trained Network Model Signature Custom Code Auxiliary Assets Model Archive Model Export CLI Model Archive Back
  • 16. ©2017, AmazonWebServices, Inc. or its Affiliates. All rights reserved. Demo: Model Archive
  • 17. ©2017, AmazonWebServices, Inc. or its Affiliates. All rights reserved. Model Archive REST and OpenAPI Containerization ONNX Support Operational Metrics
  • 18. ©2017, AmazonWebServices, Inc. or its Affiliates. All rights reserved. REST and OpenAPI REST-like endpoint: <model-name>/predict Endpoint auto-generated from the model’s signature.json JSON encoding by default Binary input via request payload OpenAPI support – client code-gen and tooling Back
  • 19. ©2017, AmazonWebServices, Inc. or its Affiliates. All rights reserved. Demo: Endpoints
  • 20. ©2017, AmazonWebServices, Inc. or its Affiliates. All rights reserved. Model Archive REST and OpenAPI Containerization ONNX Support Operational Metrics
  • 21. ©2017, AmazonWebServices, Inc. or its Affiliates. All rights reserved. Containerization • Production-ready orchestration tools: ECS, Docker, Kubernetes • Easy to scale out • Robust and scalable images • Automatically leverages all GPUs and CPUs on host • Pre-configured images on DockerHub: awsdeeplearningteam/mms_cpu Back
  • 22. ©2017, AmazonWebServices, Inc. or its Affiliates. All rights reserved. MMS Docker Image Pull or Build Run Containerization Container Cluster MMS Container MMS ContainerMMS Container MXNet NGINX MXNet Model Server Load Balancer
  • 23. ©2017, AmazonWebServices, Inc. or its Affiliates. All rights reserved. Demo: Containers
  • 24. ©2017, AmazonWebServices, Inc. or its Affiliates. All rights reserved. Model Archive REST and OpenAPI Containerization ONNX Support Operational Metrics
  • 25. ©2017, AmazonWebServices, Inc. or its Affiliates. All rights reserved. • Requests • Latencies • Resources Metrics • Model Name • Host NameDimensions • Log / CSV • AWS CloudWatchTarget Operational Metrics Back
  • 26. ©2017, AmazonWebServices, Inc. or its Affiliates. All rights reserved.
  • 27. ©2017, AmazonWebServices, Inc. or its Affiliates. All rights reserved. Model Archive REST and OpenAPI Containerization ONNX Support Operational Metrics
  • 28. ©2017, AmazonWebServices, Inc. or its Affiliates. All rights reserved. O(n2) Pairs M XNet Caffe2 PyTorch TF CNTKCoreM L TensorRT NGraph SNPEMany Frameworks ONNX Support Many Platforms ONNX: Common IR Supported in MMS
  • 29. ©2017, AmazonWebServices, Inc. or its Affiliates. All rights reserved. + MXNet Model Server AWS Fargate Serverless Model Serving
  • 30. ©2017, AmazonWebServices, Inc. or its Affiliates. All rights reserved. AWS Fargate • Deploy containers without managing hosts or clusters • Seamless scaling • Pay only for CPU and Memory your container consumed Back
  • 31. ©2017, AmazonWebServices, Inc. or its Affiliates. All rights reserved. Demo: MMS on Fargate
  • 32. ©2017, AmazonWebServices, Inc. or its Affiliates. All rights reserved. Try it out, Star and Contribute! http://mxnet.io http://modelserver.io