From data ingestion, processing, model deployment to prediction - machine learning is hard! Join me to learn how serverless can make it all easier so you can stop worrying about the underlying infrastructure layer, and focus on getting the most value out of your data and development time.
2. About Me
Product Manager on Azure Functions. Pythonista
working on Serverless for over a year.
Passionate about developer experiences, data
science and machine learning.
Asavari Tayal
@tayalasavari
3. Plan
for
today
1
Machine Learning
Introduction, process of ML and roles in ML.
Serverless and FaaS
How can they help with ML?
Use Cases
Real world examples of using Serverless ML.
Learnings and
ChallengesBenefits of Serverless ML, shortcomings and
open questions.
2
3
4
4. What is Machine Learning?
Practice of examining large amounts of data to find
patterns and then generating code to find those
same patterns in brand new data.
ML can help you build smarter
applications.
5. Process of Machine Learning
Model Training
1
Candidate
Model
Chosen Model
6. Roles in Machine Learning
Model Training
Data Scientist
App
Developer
Model
Deployment
? How big is my training dataset?
? Which algorithm should I use?
? What’s the confusion matrix?
? How often should I retrain the model?
? How many servers do I need?
? How can I scale my application?
? Who monitors my app?
? How should I patch my servers?
? What packages should be on my
server?
7. Roles in Machine Learning
Model Training
Data Scientist
App
Developer
Model
Deployment
? How big is my training dataset?
? Which algorithm should I use?
? What’s the confusion matrix?
? How often should I retrain the model?
Full abstraction of servers
Developers can just focus on their code—there are
no distractions around server management, capacity
planning, or availability.
Simplified starting experience
Easier experimentation
Scale at your pace—don’t bet the farm on Day 1
Flexibility
10. Solution I - PaaS
Python Runtime
WSGI HTTP Server
Dependencies
Docker
Image
Scoring script
Application
Gateway
App Service
Web App
Input data
HttpTrigger
11. Solution II - Serverless
Scoring script
Azure
Function
Input Data
Configuration
(trigger,
dependencies)
HttpTrigger
12. Learnings
1 Python is the preferred language for
inferencing using Functions.
2 FaaS provides a simple and flexible
approach to model deployment,
empowering Data Scientists to create
their own web service.
3 Serverless provides a low-cost
solution for event-driven
inferencing pipelines.
15. Learnings
Training
Serverless is not preferable for
compute-intensive training jobs.
Short-lived, low-compute
executions are still feasible.
Integration
Triggers and bindings
simplify integrating with
data sources. Data
Scientists can focus on the
training algorithm and
scoring script, not the SDKs.
Performance
Downloading model from
storage increases startup
time, adding latency to the
inferencing step.
16. Data processing - OCR
Vehicle Number
Plates
Blob Trigger
Result
Raw image
Re-shaped
image
Image
Vehicle
Number
HTTP Webhook
17. Learnings
Beyond FaaS
Serverless beyond
Functions is relevant to
Machine Learning
workloads:
• Eventing infrastructure
• Workflow orchestration
• Pay-per-use Cognitive
APIs
Role of
Functions
Functions serve as the glue
between different services
for data processing.
19. Challenges
Limits
Serverless Functions
have strict limits on
execution time and
memory used.
Dedicated hosting
Durable Functions
Hardware
Some inferencing
scenarios can require
GPU or FPGA enabled
hardware for
execution.
Functions
container on GPU
enabled K8s
cluster
Versioning
Data Science teams
prefer to version
models independent of
their applications. This
can increase startup
time.
Custom container
on Azure
Functions on
Linux
Boundaries
Applications dealing
with sensitive data
require more advanced
security and
networking options.
App Service
Environment
Premium Plan
20. Takeaways
Serverless is indeed a good fit for Machine Learning.
FaaS programming model and event-driven pipelines
can empower Data Scientists to quickly build smart
applications.
Let’s continue to bring the goodness of Serverless to
more scenarios and developer communities.
