Modern MLOps toolchain
2023

About Me
Dmytro Spodarets
● DevOps Architect at Grid Dynamics
● Founder and chief editor of Data Phoenix
AWS | Infrastructure | DevOps/MLOps

Agenda
● What is MLOps?
● DevOps vs MLOps
● MLOps Stack
● Use cases:
○ From research to production
○ Versioning & retraining
○ IaaC & K8s
○ Using GitOps for Machine Learning

CRISP-DM

Hidden Technical Debt in Machine Learning Systems
https://papers.nips.cc/paper/2015/file/86df7dcfd896fcaf2674f757a2463eba-Paper.pdf

The goal of MLOps is to reduce technical friction to get the model from an idea
into production in the shortest possible time with as little risk as possible.

DevOps vs MLOps

MLOps is not a single tool or platform

MLOps is about agreeing to do ML the right way and then supporting it.

A few shared principles will take you a long way…
ML should be collaborative ML should be reproducible ML should be continuous ML should be tested &
monitored

A few shared principles will take you a long way…
ML should be collaborative ML should be reproducible ML should be continuous ML should be tested &
monitored
Shared Infrastructure
Versioning for Code,
Data and Metadata
Machine Learning Pipelines Model Deployment
and Monitoring

Continuous X
MLOps is an ML engineering culture that includes the following practices:
● Continuous Integration (CI) extends the testing and validating code and
components by adding testing and validating data and models.
● Continuous Delivery (CD) concerns with delivery of an ML training pipeline
that automatically deploys another the ML model prediction service.
● Continuous Training (CT) is unique to ML systems property, which
automatically retrains ML models for re-deployment.
● Continuous Monitoring (CM) concerns with monitoring production data and
model performance metrics, which are bound to business metrics.

MLOps levels
● Level 0: No MLOps
● Level 1: DevOps no MLOps
● Level 2: Automated Training
● Level 3: Automated Model
Deployment
● Level 4: Full MLOps Automated
Retraining
https://learn.microsoft.com/en-us/azure/architecture/example-scenario/mlops/mlops-maturity-model

MLOps Stack

AWS ML Stack

Use cases
○ From research to production
○ Versioning & retraining
○ IaaC & K8s
○ Using GitOps for Machine Learning

From research to production

From research to production

Data versioning & retraining

Data versioning & retraining

IaaC & K8s

Using GitOps for Machine Learning
https://docs.dstack.ai/

Large language models / Generative models

3D-parallelism
https://aws.amazon.com/blogs/machine-learning/training-large-language-models-on-amazon-sagemaker-best-practices/

Scalable HPC

Questions?
Dmytro Spodarets
d.spodarets@dataphoenix.info
https://dataphoenix.info

https://www.eventbrite.com/o/data-phoenix-events-23453295848
https://dataphoenix.info/subscribe/