MLOps Using MLflow
•
3 recomendaciones•2,496 vistas
MLflow is an MLOps tool that enables data scientist to quickly productionize their Machine Learning projects. To achieve this, MLFlow has four major components which are Tracking, Projects, Models, and Registry. MLflow lets you train, reuse, and deploy models with any library and package them into reproducible steps. MLflow is designed to work with any machine learning library and require minimal changes to integrate into an existing codebase. In this session, we will cover the common pain points of machine learning developers such as tracking experiments, reproducibility, deployment tool and model versioning. Ready to get your hands dirty by doing quick ML project using mlflow and release to production to understand the ML-Ops lifecycle.
Recomendados
Más contenido relacionado
La actualidad más candente
La actualidad más candente (20)
Similar a MLOps Using MLflow
Similar a MLOps Using MLflow (20)
Más de Databricks
Más de Databricks (20)
Último
Último (20)
MLOps Using MLflow
- 1. MLOPs Using MLflow Nagaraj Sengodan Senior Technical Manager, HCL Technologies Nitin Raj Soundararajan Technical Consultant, Cognizant Worldwide Limited
- 2. Presenter Nagaraj Sengodan is a Senior Technical Manager in Data and Analytics Practice at HCL Technologies. Nitin Raj Soundararajan is a technical consultant focusing on advanced data analytics, data engineering, cloud scale analytics and data science.
- 3. Agenda MLOps MLOps Process and stages. Challenges MLFlow Components Mapping with MLOps stages Demo Real-world example for mlflow Q & A
- 4. MLOps
- 5. Why
- 6. Challenges Reproducibility in ML models ML Operations Model Management Tools and Technologies
- 7. MLOps Software took months to release; now it releases daily. How we deploy software From curve fitting to neural networks… How we use data + maths to train models Evolution of Software Engineering from punch card to Distributed version control… Github! How we write software Write Software Engineering Deploy DevOps Train ML Engineer MLOps
- 8. Collaboration in Nature Data Source PREPARE DATA Data Engineer Data Repository TRAIN MODEL Modelling Pipeline FEATURE TRAIN EVALUATE RELEASE MODEL Release Pipeline DEPLOY APPROVE PROFILE Data Scientist Model Registry ML Engineer Data Pipeline VALIDATE PACKAGE Collect
- 9. MLOps – Life-Cycle ML Orchestration ML Health Business Impact Model Governance CI / CD Machine Learning Models Business Value Collaboration Collaboration MLOps Productionizing Machine Learning Models
- 10. Life Cycle Coding Unit Test Cases Peer –Review Approval Commit Test Release Prod Release Software Writing Code for specific requirement / functionality Writing Unit test cases for each functionality and block to cover all boundary condition Peer review the code and make sure code meets the standard and best practices Code is approved to check-in to repository / version control Approved code is merged with main / feature branch for release rollover. Release the feature / main branch for Testing. Move the Tested version to production.
- 11. Life Cycle Coding Unit Test Cases Peer –Review Approval Commit Test Release Prod Release Analyse Data Data Preparation Building Model Evaluate the Model Model Optimization Deploy Model Monitor and Re-Train Software ML
- 12. Life Cycle Coding Unit Test Cases Peer –Review Approval Commit Test Release Prod Release Analyse Data Data Preparation Building Model Evaluate the Model Model Optimization Deploy Model Monitor and Re-Train Software ML Cover Functional Requirements Optimize Metrics GOAL
- 13. Life Cycle Coding Unit Test Cases Peer –Review Approval Commit Test Release Prod Release Analyse Data Data Preparation Building Model Evaluate the Model Model Optimization Deploy Model Monitor and Re-Train Software ML Cover Functional Requirements Optimize Metrics Depends on the Code Depends on Data, Choice of algorithm, params, etc… GOAL QUALITY
- 14. Life Cycle Analyse Data Data Preparation Building Model Evaluate the Model Model Optimization Deploy Model Monitor and Re-Train Software ML Cover Functional Requirements Optimize Metrics Depends on the Code Depends on Data, Choice of algorithm, params, etc… GOAL QUALITY Mostly one tech stack Combinations of many libraries and tools TECHNOLOGY Coding Unit Test Cases Peer –Review Approval Commit Test Release Prod Release
- 15. Life Cycle Coding Unit Test Cases Peer –Review Approval Commit Test Release Prod Release Analyse Data Data Preparation Building Model Evaluate the Model Model Optimization Deploy Model Monitor and Re-Train Software ML Cover Functional Requirements Optimize Metrics Depends on the Code Depends on Data, Choice of algorithm, params, etc… GOAL QUALITY Mostly one tech stack Combinations of many libraries and tools TECHNOLOGY Works deterministically Changing based on data… OUTCOME
- 16. A platform for the Complete Machine Learning Lifecycle
- 17. MLFlow Components Machine Learning lifecycle MLflow Tracking - Record and query experiments: code, data, config, and results MLflow Projects - Package data science code in a format to reproduce runs on any platform MLflow Models - Deploy machine learning models in diverse serving environments MLflow Registry - Store, annotate, discover, and manage models in a central repository
- 18. Mlflow Workflow Models Tracking Model Registry V0 V1 V1 V2 V3
- 19. Mlflow Tracking Tracking Server Notebooks Local Apps Cloud Jobs UI API Spark Data Source Parameters Metrics Artifacts Metadata Models
- 20. Mlflow Projects Project Spec Local Execution Remote Execution Code Config Metadata
- 21. Mlflow Models
- 22. Mlflow Registry
- 23. Demo # Enable MLflow Autologging mlflow.keras.autolog() X, y = get_training_data()opt = keras.optimizers.Adam(lr=params["learning_rate"], beta_1=params["beta_1"], beta_2=params["beta_2"], epsilon=params["epsilon"] model = Sequential()model.add(Dense(int(params["units"]), ...) model.add(Dense(1)) model.compile(loss="mse", optimizer=opt) rest = model.fit(X, y, epochs=50, batch_size=64, validation_split=.2)
- 24. What Next? To get started with mlflow, just pip install mlflow Docs and Tutorials mlflow.org Session materials and demo https://github.com/krsnagaraj/dataaisummit-mlflow Connect with us ▪ https://uk.linkedin.com/in/nagarajsengodan ▪ https://www.linkedin.com/in/nitinrajs
- 25. Thank You
- 26. Feedback Your feedback is important to us. Don’t forget to rate and review the sessions.