Enviar búsqueda
Cargar
ODSC data science to DataOps
•
0 recomendaciones
•
238 vistas
Christopher Bergh
Seguir
Applications of dataops to data science and AI
Leer menos
Leer más
Tecnología
Denunciar
Compartir
Denunciar
Compartir
1 de 37
Descargar ahora
Descargar para leer sin conexión
Recomendados
DataOps - Big Data and AI World London - March 2020 - Harvinder Atwal
DataOps - Big Data and AI World London - March 2020 - Harvinder Atwal
Harvinder Atwal
seven steps to dataops @ dataops.rocks conference Oct 2019
seven steps to dataops @ dataops.rocks conference Oct 2019
DataKitchen
How to add security in dataops and devops
How to add security in dataops and devops
Ulf Mattsson
Overcoming DataOps hurdles for ML in Production
Overcoming DataOps hurdles for ML in Production
Sandeep Uttamchandani
ODSC May 2019 - The DataOps Manifesto
ODSC May 2019 - The DataOps Manifesto
DataKitchen
The Importance of DataOps in a Multi-Cloud World
The Importance of DataOps in a Multi-Cloud World
DATAVERSITY
Webinar: Attaining Excellence in Big Data Integration
Webinar: Attaining Excellence in Big Data Integration
SnapLogic
Starting Your Modern DataOps Journey
Starting Your Modern DataOps Journey
CloverDX
Recomendados
DataOps - Big Data and AI World London - March 2020 - Harvinder Atwal
DataOps - Big Data and AI World London - March 2020 - Harvinder Atwal
Harvinder Atwal
seven steps to dataops @ dataops.rocks conference Oct 2019
seven steps to dataops @ dataops.rocks conference Oct 2019
DataKitchen
How to add security in dataops and devops
How to add security in dataops and devops
Ulf Mattsson
Overcoming DataOps hurdles for ML in Production
Overcoming DataOps hurdles for ML in Production
Sandeep Uttamchandani
ODSC May 2019 - The DataOps Manifesto
ODSC May 2019 - The DataOps Manifesto
DataKitchen
The Importance of DataOps in a Multi-Cloud World
The Importance of DataOps in a Multi-Cloud World
DATAVERSITY
Webinar: Attaining Excellence in Big Data Integration
Webinar: Attaining Excellence in Big Data Integration
SnapLogic
Starting Your Modern DataOps Journey
Starting Your Modern DataOps Journey
CloverDX
Journey to Cloud Analytics
Journey to Cloud Analytics
Datavail
Agile Leadership: Guiding DataOps Teams Through Rapid Change and Uncertainty
Agile Leadership: Guiding DataOps Teams Through Rapid Change and Uncertainty
TamrMarketing
Webinar: The Death of Traditional Data Integration
Webinar: The Death of Traditional Data Integration
SnapLogic
DATAOPS: THE NEXT BIG WAVE ON YOUR DATA JOURNEY - Big Data Expo
DATAOPS: THE NEXT BIG WAVE ON YOUR DATA JOURNEY - Big Data Expo
webwinkelvakdag
How to Streamline DataOps on AWS
How to Streamline DataOps on AWS
Enterprise Management Associates
DataOps: Nine steps to transform your data science impact Strata London May 18
DataOps: Nine steps to transform your data science impact Strata London May 18
Harvinder Atwal
Accidental DataOps
Accidental DataOps
Steve Ross
Moving to the Cloud: Modernizing Data Architecture in Healthcare
Moving to the Cloud: Modernizing Data Architecture in Healthcare
Perficient, Inc.
Operational Analytics
Operational Analytics
Eckerson Group
TDWI Boston Keynote - The New BI/Analytics Synergy - 7 30-2015 - tdwi keynote
TDWI Boston Keynote - The New BI/Analytics Synergy - 7 30-2015 - tdwi keynote
Eckerson Group
Customer Spotlight: How WellCare Accelerated Big Data Delivery to Improve Ana...
Customer Spotlight: How WellCare Accelerated Big Data Delivery to Improve Ana...
VMware Tanzu
Predictive and Prescriptive Analytics Expert Session Webinar
Predictive and Prescriptive Analytics Expert Session Webinar
ibi
Strata+hadoop data kitchen-seven-steps-to-high-velocity-data-analytics-with d...
Strata+hadoop data kitchen-seven-steps-to-high-velocity-data-analytics-with d...
DataKitchen
Integrating the CDO Role Into Your Organization; Managing the Disruption (MIT...
Integrating the CDO Role Into Your Organization; Managing the Disruption (MIT...
Caserta
Accelerating Fast Data Strategy with Data Virtualization
Accelerating Fast Data Strategy with Data Virtualization
Denodo
Scaling Your Data: Data Democratisation and DataOps
Scaling Your Data: Data Democratisation and DataOps
Juan Sebastián Urrego Escobar
Traditional BI vs. Business Data Lake – A Comparison
Traditional BI vs. Business Data Lake – A Comparison
Capgemini
Architecting Data For The Modern Enterprise - Data Summit 2017, Closing Keynote
Architecting Data For The Modern Enterprise - Data Summit 2017, Closing Keynote
Caserta
A modern, flexible approach to Hadoop implementation incorporating innovation...
A modern, flexible approach to Hadoop implementation incorporating innovation...
DataWorks Summit
The Future of Data Warehousing and Data Integration
The Future of Data Warehousing and Data Integration
Eric Kavanagh
Washington DC DataOps Meetup -- Nov 2019
Washington DC DataOps Meetup -- Nov 2019
DataKitchen
Your Data Nerd Friends Need You!
Your Data Nerd Friends Need You!
DataKitchen
Más contenido relacionado
La actualidad más candente
Journey to Cloud Analytics
Journey to Cloud Analytics
Datavail
Agile Leadership: Guiding DataOps Teams Through Rapid Change and Uncertainty
Agile Leadership: Guiding DataOps Teams Through Rapid Change and Uncertainty
TamrMarketing
Webinar: The Death of Traditional Data Integration
Webinar: The Death of Traditional Data Integration
SnapLogic
DATAOPS: THE NEXT BIG WAVE ON YOUR DATA JOURNEY - Big Data Expo
DATAOPS: THE NEXT BIG WAVE ON YOUR DATA JOURNEY - Big Data Expo
webwinkelvakdag
How to Streamline DataOps on AWS
How to Streamline DataOps on AWS
Enterprise Management Associates
DataOps: Nine steps to transform your data science impact Strata London May 18
DataOps: Nine steps to transform your data science impact Strata London May 18
Harvinder Atwal
Accidental DataOps
Accidental DataOps
Steve Ross
Moving to the Cloud: Modernizing Data Architecture in Healthcare
Moving to the Cloud: Modernizing Data Architecture in Healthcare
Perficient, Inc.
Operational Analytics
Operational Analytics
Eckerson Group
TDWI Boston Keynote - The New BI/Analytics Synergy - 7 30-2015 - tdwi keynote
TDWI Boston Keynote - The New BI/Analytics Synergy - 7 30-2015 - tdwi keynote
Eckerson Group
Customer Spotlight: How WellCare Accelerated Big Data Delivery to Improve Ana...
Customer Spotlight: How WellCare Accelerated Big Data Delivery to Improve Ana...
VMware Tanzu
Predictive and Prescriptive Analytics Expert Session Webinar
Predictive and Prescriptive Analytics Expert Session Webinar
ibi
Strata+hadoop data kitchen-seven-steps-to-high-velocity-data-analytics-with d...
Strata+hadoop data kitchen-seven-steps-to-high-velocity-data-analytics-with d...
DataKitchen
Integrating the CDO Role Into Your Organization; Managing the Disruption (MIT...
Integrating the CDO Role Into Your Organization; Managing the Disruption (MIT...
Caserta
Accelerating Fast Data Strategy with Data Virtualization
Accelerating Fast Data Strategy with Data Virtualization
Denodo
Scaling Your Data: Data Democratisation and DataOps
Scaling Your Data: Data Democratisation and DataOps
Juan Sebastián Urrego Escobar
Traditional BI vs. Business Data Lake – A Comparison
Traditional BI vs. Business Data Lake – A Comparison
Capgemini
Architecting Data For The Modern Enterprise - Data Summit 2017, Closing Keynote
Architecting Data For The Modern Enterprise - Data Summit 2017, Closing Keynote
Caserta
A modern, flexible approach to Hadoop implementation incorporating innovation...
A modern, flexible approach to Hadoop implementation incorporating innovation...
DataWorks Summit
The Future of Data Warehousing and Data Integration
The Future of Data Warehousing and Data Integration
Eric Kavanagh
La actualidad más candente
(20)
Journey to Cloud Analytics
Journey to Cloud Analytics
Agile Leadership: Guiding DataOps Teams Through Rapid Change and Uncertainty
Agile Leadership: Guiding DataOps Teams Through Rapid Change and Uncertainty
Webinar: The Death of Traditional Data Integration
Webinar: The Death of Traditional Data Integration
DATAOPS: THE NEXT BIG WAVE ON YOUR DATA JOURNEY - Big Data Expo
DATAOPS: THE NEXT BIG WAVE ON YOUR DATA JOURNEY - Big Data Expo
How to Streamline DataOps on AWS
How to Streamline DataOps on AWS
DataOps: Nine steps to transform your data science impact Strata London May 18
DataOps: Nine steps to transform your data science impact Strata London May 18
Accidental DataOps
Accidental DataOps
Moving to the Cloud: Modernizing Data Architecture in Healthcare
Moving to the Cloud: Modernizing Data Architecture in Healthcare
Operational Analytics
Operational Analytics
TDWI Boston Keynote - The New BI/Analytics Synergy - 7 30-2015 - tdwi keynote
TDWI Boston Keynote - The New BI/Analytics Synergy - 7 30-2015 - tdwi keynote
Customer Spotlight: How WellCare Accelerated Big Data Delivery to Improve Ana...
Customer Spotlight: How WellCare Accelerated Big Data Delivery to Improve Ana...
Predictive and Prescriptive Analytics Expert Session Webinar
Predictive and Prescriptive Analytics Expert Session Webinar
Strata+hadoop data kitchen-seven-steps-to-high-velocity-data-analytics-with d...
Strata+hadoop data kitchen-seven-steps-to-high-velocity-data-analytics-with d...
Integrating the CDO Role Into Your Organization; Managing the Disruption (MIT...
Integrating the CDO Role Into Your Organization; Managing the Disruption (MIT...
Accelerating Fast Data Strategy with Data Virtualization
Accelerating Fast Data Strategy with Data Virtualization
Scaling Your Data: Data Democratisation and DataOps
Scaling Your Data: Data Democratisation and DataOps
Traditional BI vs. Business Data Lake – A Comparison
Traditional BI vs. Business Data Lake – A Comparison
Architecting Data For The Modern Enterprise - Data Summit 2017, Closing Keynote
Architecting Data For The Modern Enterprise - Data Summit 2017, Closing Keynote
A modern, flexible approach to Hadoop implementation incorporating innovation...
A modern, flexible approach to Hadoop implementation incorporating innovation...
The Future of Data Warehousing and Data Integration
The Future of Data Warehousing and Data Integration
Similar a ODSC data science to DataOps
Washington DC DataOps Meetup -- Nov 2019
Washington DC DataOps Meetup -- Nov 2019
DataKitchen
Your Data Nerd Friends Need You!
Your Data Nerd Friends Need You!
DataKitchen
Enabling Agility Through DevOps
Enabling Agility Through DevOps
Leland Newsom CSP-SM, SPC5, SDP
Bdf16 big-data-warehouse-case-study-data kitchen
Bdf16 big-data-warehouse-case-study-data kitchen
Christopher Bergh
From zero to one - How we evolved our test automation processes and mindset i...
From zero to one - How we evolved our test automation processes and mindset i...
Jen-Chieh Ko
Comcast Labs Connect - PHLAI Conference Philadelphia 2018
Comcast Labs Connect - PHLAI Conference Philadelphia 2018
Open Data Group
Data science tools of the trade
Data science tools of the trade
Fangda Wang
ENT206 Product Development in the Cloud
ENT206 Product Development in the Cloud
Amazon Web Services
Agile-plus-DevOps Testing for Packaged Applications
Agile-plus-DevOps Testing for Packaged Applications
Worksoft
Intelligent data summit: Self-Service Big Data and AI/ML: Reality or Myth?
Intelligent data summit: Self-Service Big Data and AI/ML: Reality or Myth?
SnapLogic
DataOps - The Foundation for Your Agile Data Architecture
DataOps - The Foundation for Your Agile Data Architecture
DATAVERSITY
Building a Marketing Data Warehouse from Scratch - SMX Advanced 202
Building a Marketing Data Warehouse from Scratch - SMX Advanced 202
Christopher Gutknecht
Product Development in the Cloud
Product Development in the Cloud
Amazon Web Services
Cloud-native Enterprise Data Science Teams
Cloud-native Enterprise Data Science Teams
Boston Consulting Group
Transforming Product Development in the Cloud (ENT306) - AWS re:Invent 2018
Transforming Product Development in the Cloud (ENT306) - AWS re:Invent 2018
Amazon Web Services
Transforming Product Development - Transformation Day Montreal 2018
Transforming Product Development - Transformation Day Montreal 2018
Amazon Web Services
Product Development in the Cloud - ENT206 - Chicago AWS Summit
Product Development in the Cloud - ENT206 - Chicago AWS Summit
Amazon Web Services
12Nov13 Webinar: Big Data Analysis with Teradata and Revolution Analytics
12Nov13 Webinar: Big Data Analysis with Teradata and Revolution Analytics
Revolution Analytics
Business Process and Technology Evolution - Product Creation
Business Process and Technology Evolution - Product Creation
Vikram Singla FCILT
PayPal Notebooks at Jupytercon 2018
PayPal Notebooks at Jupytercon 2018
Romit Mehta
Similar a ODSC data science to DataOps
(20)
Washington DC DataOps Meetup -- Nov 2019
Washington DC DataOps Meetup -- Nov 2019
Your Data Nerd Friends Need You!
Your Data Nerd Friends Need You!
Enabling Agility Through DevOps
Enabling Agility Through DevOps
Bdf16 big-data-warehouse-case-study-data kitchen
Bdf16 big-data-warehouse-case-study-data kitchen
From zero to one - How we evolved our test automation processes and mindset i...
From zero to one - How we evolved our test automation processes and mindset i...
Comcast Labs Connect - PHLAI Conference Philadelphia 2018
Comcast Labs Connect - PHLAI Conference Philadelphia 2018
Data science tools of the trade
Data science tools of the trade
ENT206 Product Development in the Cloud
ENT206 Product Development in the Cloud
Agile-plus-DevOps Testing for Packaged Applications
Agile-plus-DevOps Testing for Packaged Applications
Intelligent data summit: Self-Service Big Data and AI/ML: Reality or Myth?
Intelligent data summit: Self-Service Big Data and AI/ML: Reality or Myth?
DataOps - The Foundation for Your Agile Data Architecture
DataOps - The Foundation for Your Agile Data Architecture
Building a Marketing Data Warehouse from Scratch - SMX Advanced 202
Building a Marketing Data Warehouse from Scratch - SMX Advanced 202
Product Development in the Cloud
Product Development in the Cloud
Cloud-native Enterprise Data Science Teams
Cloud-native Enterprise Data Science Teams
Transforming Product Development in the Cloud (ENT306) - AWS re:Invent 2018
Transforming Product Development in the Cloud (ENT306) - AWS re:Invent 2018
Transforming Product Development - Transformation Day Montreal 2018
Transforming Product Development - Transformation Day Montreal 2018
Product Development in the Cloud - ENT206 - Chicago AWS Summit
Product Development in the Cloud - ENT206 - Chicago AWS Summit
12Nov13 Webinar: Big Data Analysis with Teradata and Revolution Analytics
12Nov13 Webinar: Big Data Analysis with Teradata and Revolution Analytics
Business Process and Technology Evolution - Product Creation
Business Process and Technology Evolution - Product Creation
PayPal Notebooks at Jupytercon 2018
PayPal Notebooks at Jupytercon 2018
Último
Vector Search -An Introduction in Oracle Database 23ai.pptx
Vector Search -An Introduction in Oracle Database 23ai.pptx
Remote DBA Services
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
apidays
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Victor Rentea
Corporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptx
Rustici Software
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Orbitshub
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Edi Saputra
Exploring Multimodal Embeddings with Milvus
Exploring Multimodal Embeddings with Milvus
Zilliz
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Juan lago vázquez
Mcleodganj Call Girls 🥰 8617370543 Service Offer VIP Hot Model
Mcleodganj Call Girls 🥰 8617370543 Service Offer VIP Hot Model
Deepika Singh
WSO2's API Vision: Unifying Control, Empowering Developers
WSO2's API Vision: Unifying Control, Empowering Developers
WSO2
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Safe Software
Elevate Developer Efficiency & build GenAI Application with Amazon Q
Elevate Developer Efficiency & build GenAI Application with Amazon Q
Bhuvaneswari Subramani
Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf
Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf
Orbitshub
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
Jeffrey Haguewood
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
ThousandEyes
MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024
MIND CTI
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
MadyBayot
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Victor Rentea
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
sammart93
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Safe Software
Último
(20)
Vector Search -An Introduction in Oracle Database 23ai.pptx
Vector Search -An Introduction in Oracle Database 23ai.pptx
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Corporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptx
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Exploring Multimodal Embeddings with Milvus
Exploring Multimodal Embeddings with Milvus
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Mcleodganj Call Girls 🥰 8617370543 Service Offer VIP Hot Model
Mcleodganj Call Girls 🥰 8617370543 Service Offer VIP Hot Model
WSO2's API Vision: Unifying Control, Empowering Developers
WSO2's API Vision: Unifying Control, Empowering Developers
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Elevate Developer Efficiency & build GenAI Application with Amazon Q
Elevate Developer Efficiency & build GenAI Application with Amazon Q
Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf
Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
ODSC data science to DataOps
1.
Copyright © 2017
by DataKitchen, Inc. All Rights Reserved.
2.
Copyright © 2018
by DataKitchen, Inc. All Rights Reserved. Agenda How to go from Data Science to Data Operations (#DataOps) Introductions Data Science Challenges What is DataOps? Seven Shocking Steps to DataOps Pulling it together
3.
Copyright © 2018
by DataKitchen, Inc. All Rights Reserved. Keep this question in mind What can I take from this session and use on Monday?
4.
Copyright © 2018
by DataKitchen, Inc. All Rights Reserved. For slides contact gil@DataKitchen.io
5.
Copyright © 2018
by DataKitchen, Inc. All Rights Reserved. Speaker – co-Founder of DataKitchen Gil Benghiat, Founder, VP of Products gil@datakitchen.io A series of data centric software projects 🎓 Applied Math / Biology @ Brown 🎓 Computer Science @ Stanford 🏢 Bell Labs, Sybase, PhaseForward, LeapFrogRx
6.
Copyright © 2018
by DataKitchen, Inc. All Rights Reserved. DataKitchen DataOps Software Platform Main Features 1. Orchestrate complex data pipelines 2. Deploy new ideas to production 3. Automate tests and monitor quality Enables 1. Fast delivery of analytics 2. High data quality 3. Using your favorite tools and data stores
7.
Copyright © 2018
by DataKitchen, Inc. All Rights Reserved. Agenda Introductions • Data Science Challenges What is DataOps? Seven Shocking Steps to DataOps Pulling it together
8.
Copyright © 2018
by DataKitchen, Inc. All Rights Reserved. Figure 1: Only a small fraction of real-world ML systems is composed of the ML code, as shown by the small black box in the middle. The required surrounding infrastructure is vast and complex. Google Advances in Neural Information Processing Systems 28 (NIPS 2015)
9.
Copyright © 2018
by DataKitchen, Inc. All Rights Reserved. Business Need Prep Data Feature Extraction Build Model Evaluate Model Deploy Model Monitor Model Iterate, Test and Improve Model building
10.
Copyright © 2018
by DataKitchen, Inc. All Rights Reserved. Agenda Introductions Data Science Challenges • What is DataOps? Seven Shocking Steps to DataOps Pulling it together
11.
Copyright © 2018
by DataKitchen, Inc. All Rights Reserved. Genesis of DataOps People, Process, Organization Technical Environment = 7 steps
12.
Copyright © 2018
by DataKitchen, Inc. All Rights Reserved. Data Engineer Data Scientist Data Analyst Agile Development is a mindset: 1. Collaborate with your customers 2. Respond to change 3. Measure progress by working analytics 4. Release frequently (most important first) 5. Get feedback on your releases 6. Adjust your behavior to become more effective 4 Values 12 Principles Be Pragmatic Not Dogmatic DataOps: It Began With Agile Business Partner
13.
Copyright © 2018
by DataKitchen, Inc. All Rights Reserved. Focus on Value
14.
Copyright © 2018
by DataKitchen, Inc. All Rights Reserved. Agenda Introductions Data Science Challenges What is DataOps? • Seven Shocking Steps to DataOps Pulling it together
15.
Copyright © 2018
by DataKitchen, Inc. All Rights Reserved. Seven Steps to DataOps 1. Orchestrate Two Journeys 2. Add Tests 3. Use a Version Control System 4. Branch and Merge 5. Use Multiple Environments 6. Reuse & Containerize 7. Parameterize Your Processing
16.
Copyright © 2018
by DataKitchen, Inc. All Rights Reserved. Journey 1: Orchestrate data to customer value Analytic process are like manufacturing: materials (data) and production outputs (refined data, charts, graphs, models) Access: Python Code Transform: SQL Code, ETL Model: R Code Visualize: Tableau Workbook Report: Tableau Online ❶
17.
Copyright © 2018
by DataKitchen, Inc. All Rights Reserved. Journey 2: Speed ideas to production Analytic processes are like software development: deliverables continually move from development to production ❶ Data Engineers Data Scientists Data Analysts Diverse Team Diverse Tools Diverse Customers Business Customer Products & Systems
18.
Copyright © 2018
by DataKitchen, Inc. All Rights Reserved. Innovation and Value Pipeline Together Focus on both orchestration and deployment while automating & monitoring quality Don’t want break production when I deploy my changes Don’t want to learn about data quality issues from my customers ❶
19.
Copyright © 2018
by DataKitchen, Inc. All Rights Reserved. Add Tests Monitor quality Data Quality Monitoring: To ensure that during in the Value Pipeline, the data quality remains high. Code Quality Monitoring: Before promoting work, running new and old tests gives high confidence that the change did not break anything in the Innovation Pipeline ❷
20.
Copyright © 2018
by DataKitchen, Inc. All Rights Reserved. Automate Monitoring & Tests In Production Test Every Step And Every Tool in Your Value Pipeline Are your outputs consistent? And Save Test Results! Are data inputs free from issues? Is your business logic still correct? Access: Python Code Transform: SQL Code, ETL Model: R Code Visualize: Tableau Workbook Report: Tableau Online ❷
21.
Copyright © 2018
by DataKitchen, Inc. All Rights Reserved. Support Multiple Types Of Tests Testing Data Is Not Just Pass/Fail in Your Value Pipeline Support Test Types • Error – stop the line • Warning – investigate later • Info – list of changes Keep Test History • Statistical Process Control ❷
22.
Copyright © 2018
by DataKitchen, Inc. All Rights Reserved. Types of Tests ❷
23.
Copyright © 2018
by DataKitchen, Inc. All Rights Reserved. Example Tests Simple ❷
24.
Copyright © 2018
by DataKitchen, Inc. All Rights Reserved. For the Innovation Pipeline Tests Are For Also Code: Keep Data Fixed Deploy Feature Run all tests here before promoting ❷
25.
Copyright © 2018
by DataKitchen, Inc. All Rights Reserved. Use a Version Control System At The End Of The Day, Analytic Work Is All Just Code Access: Python Code Transform: SQL Code, ETL Code Model: R Code Visualize: Tableau Workbook XML Report: Tableau Online Source Code Control ❸
26.
Copyright © 2018
by DataKitchen, Inc. All Rights Reserved. Branch & Merge Source Code Control Branching & Merging enables people to safely work on their own tasks ❹
27.
Copyright © 2018
by DataKitchen, Inc. All Rights Reserved. ❹ Example branch and merge pattern Sprint 1 Sprint 2 f1 f2 f3 main / master / trunk f5
28.
Copyright © 2018
by DataKitchen, Inc. All Rights Reserved. Access: Python Code Transform: SQL Code, ETL Code Model: R Code Visualize: Tableau Workbook XML Report: Tableau Online Use Multiple Environments Analytic Environment Your Analytic Work Requires Coordinating Tools And Hardware ❺
29.
Copyright © 2018
by DataKitchen, Inc. All Rights Reserved. Use Multiple Environments Provide an Analytic Environment for each branch • Analysts and Data Scientists need a controlled environment for their experiments • Engineers need a place to develop outside of production • Update Production only after all tests are run! ❺
30.
Copyright © 2018
by DataKitchen, Inc. All Rights Reserved. Use Multiple Environments ❺ Provide an Analytic Environment for each branch • Analysts and Data Scientists need a controlled environment for their experiments • Engineers need a place to develop outside of production • Update Production only after all tests are run!
31.
Copyright © 2018
by DataKitchen, Inc. All Rights Reserved. Reuse & Containerize Containerize 1. Manage the environment for each model (e.g. Docker, VM, AMI) 2. Practice Environment Version Control make production and development areas identical Reuse 1. The code 2. Data ❻
32.
Copyright © 2018
by DataKitchen, Inc. All Rights Reserved. Parameterize Your Processing Think Of Your Pipeline Like A Big Function • Named sets of parameters will increase your velocity • With parameters, you can vary • Inputs • Outputs • Steps in the workflow • You can make a time machine ❼
33.
Copyright © 2018
by DataKitchen, Inc. All Rights Reserved. Agenda Introductions Data Science Challenges What is DataOps? Seven Shocking Steps to DataOps • Pulling it together
34.
Copyright © 2018
by DataKitchen, Inc. All Rights Reserved. Business Need Prep Data Feature Extraction Build Model Evaluate Model Deploy Model Monitor Model Iterate, Test and Improve Model building
35.
Copyright © 2018
by DataKitchen, Inc. All Rights Reserved. The 7 Steps and Data Science Journeys Tests Version Control Branch and Merge Environments Reuse / Containerize Parameterize Business Need Agile Prep Data x x x x x x x Feature Extraction x x x x x x x Build Model x x x x x x x Evaluate Model x Deploy Model x x x x x x x Monitor Model x
36.
Copyright © 2018
by DataKitchen, Inc. All Rights Reserved. Make a note to yourself What can I take from this session and use on Monday?
37.
Copyright © 2018
by DataKitchen, Inc. All Rights Reserved. For slides contact gil@DataKitchen.io Thank you for attending
Descargar ahora