SlideShare una empresa de Scribd logo
1 de 10
Descargar para leer sin conexión
USING DOCKER FOR DATA 
SCIENCE
WHY DOCKER 
Portable environment 
Isolated between projects 
Stateless 
Fast local file access 
Hetrogenous
GET DOCKER 
https://docs.docker.com/installation/ 
boot2docker .dmg or .exe 
apt-get install docker.io ...
RUN SCIPYSERVER 
$ docker run -d -e "PASSWORD=YourPassword?" ipython/scipyserver 
$ docker run  
-d  
-e "PASSWORD=YourPassword?"  
--name dev_notebook  
-p 443:8888  
ipython/scipyserver 
https://localhost:443 
https://{boot2docker ip}:443
CREATE DATA-ONLY CONTAINERS 
$ docker run  
-d  
-v ~/notebooks:/notebooks  
--name notebooks_container  
ubuntu 
echo notebooks 
$ docker run -d -v ~/data:/data --name data_container ubuntu echo data
MOUNT DATA-ONLY CONTAINERS 
$ docker stop dev_notebook 
$ docker rm dev_notebook 
$ docker run  
-d  
-e "PASSWORD=YourPassword?"  
--name dev_notebook  
-p 443:8888  
--volumes-from data_container  
--volumes-from notebooks_container  
ipython/scipyserver
CREATE A DOCKERFILE 
FROM ipython/scipyserver 
MAINTAINER Calvin Giles <calvin.giles@gmail.com> 
COPY requirements.txt /requirements.txt 
RUN pip2 install -r /requirements.txt 
RUN pip3 install -r /requirements.txt 
$ docker build  
-t calvingiles/ds-notebook  
. 
$ docker run  
-d  
-e "PASSWORD=YourPassword?"  
--name dev_notebook  
-p 443:8888  
--volumes-from data_container  
--volumes-from notebooks_container  
calvingiles/ds-notebook
NEXT TIME 
Connecting to local database containers 
Tweaking the boot2docker vm memory from 2GB to 8 (or 
more...) 
Linking to private git repositories 
Automated builds with github linking
MORE? 
Docker User Guide: 
http://docs.docker.com/userguide/ 
Docker Reference: 
http://docs.docker.com/reference/commandline/cli/ 
ipython docker images: 
https://registry.hub.docker.com/repos/ipython/ 
my docker image: 
https://github.com/calvingiles/ds-notebook 
https://registry.hub.docker.com/u/calvingiles/ds-notebook/
ABOUT ME 
Calvin Giles 
Data Scientist at Adthena 
PyData Meetup Organiser 
untangleconsulting.io 
calvin.giles@gmail.com 
@calvingiles on twitter, github, docker hub (and many more)

Más contenido relacionado

La actualidad más candente

Drupal Camp Brighton 2015: Ansible Drupal Medicine show
Drupal Camp Brighton 2015: Ansible Drupal Medicine showDrupal Camp Brighton 2015: Ansible Drupal Medicine show
Drupal Camp Brighton 2015: Ansible Drupal Medicine show
George Boobyer
 
Ops for everyone - John Britton
Ops for everyone - John BrittonOps for everyone - John Britton
Ops for everyone - John Britton
Devopsdays
 
Docker & FieldAware
Docker & FieldAwareDocker & FieldAware
Docker & FieldAware
Jakub Jarosz
 

La actualidad más candente (18)

Shared Object images in Docker: What you need is what you want.
Shared Object images in Docker: What you need is what you want.Shared Object images in Docker: What you need is what you want.
Shared Object images in Docker: What you need is what you want.
 
2009 cluster user training
2009 cluster user training2009 cluster user training
2009 cluster user training
 
Hadoop installation
Hadoop installationHadoop installation
Hadoop installation
 
Drupal Camp Brighton 2015: Ansible Drupal Medicine show
Drupal Camp Brighton 2015: Ansible Drupal Medicine showDrupal Camp Brighton 2015: Ansible Drupal Medicine show
Drupal Camp Brighton 2015: Ansible Drupal Medicine show
 
Ops for everyone - John Britton
Ops for everyone - John BrittonOps for everyone - John Britton
Ops for everyone - John Britton
 
rake puppetexpert:create - Puppet Camp Silicon Valley 2014
rake puppetexpert:create - Puppet Camp Silicon Valley 2014rake puppetexpert:create - Puppet Camp Silicon Valley 2014
rake puppetexpert:create - Puppet Camp Silicon Valley 2014
 
Drupal from scratch
Drupal from scratchDrupal from scratch
Drupal from scratch
 
Medicine show2 Drupal Bristol Camp 2015
Medicine show2 Drupal Bristol Camp 2015Medicine show2 Drupal Bristol Camp 2015
Medicine show2 Drupal Bristol Camp 2015
 
JDD 2017: Nginx + Lua = OpenResty (Marcin Stożek)
JDD 2017: Nginx + Lua = OpenResty (Marcin Stożek)JDD 2017: Nginx + Lua = OpenResty (Marcin Stożek)
JDD 2017: Nginx + Lua = OpenResty (Marcin Stożek)
 
Ansible - Swiss Army Knife Orchestration
Ansible - Swiss Army Knife OrchestrationAnsible - Swiss Army Knife Orchestration
Ansible - Swiss Army Knife Orchestration
 
Ansible, Simplicity, and the Zen of Python
Ansible, Simplicity, and the Zen of PythonAnsible, Simplicity, and the Zen of Python
Ansible, Simplicity, and the Zen of Python
 
Top Node.js Metrics to Watch
Top Node.js Metrics to WatchTop Node.js Metrics to Watch
Top Node.js Metrics to Watch
 
PuppetCamp SEA 1 - Use of Puppet
PuppetCamp SEA 1 - Use of PuppetPuppetCamp SEA 1 - Use of Puppet
PuppetCamp SEA 1 - Use of Puppet
 
The Puppet Debugging Kit: Building Blocks for Exploration and Problem Solving...
The Puppet Debugging Kit: Building Blocks for Exploration and Problem Solving...The Puppet Debugging Kit: Building Blocks for Exploration and Problem Solving...
The Puppet Debugging Kit: Building Blocks for Exploration and Problem Solving...
 
Puppet Camp Phoenix 2015: Managing Files via Puppet: Let Me Count The Ways (B...
Puppet Camp Phoenix 2015: Managing Files via Puppet: Let Me Count The Ways (B...Puppet Camp Phoenix 2015: Managing Files via Puppet: Let Me Count The Ways (B...
Puppet Camp Phoenix 2015: Managing Files via Puppet: Let Me Count The Ways (B...
 
Docker & FieldAware
Docker & FieldAwareDocker & FieldAware
Docker & FieldAware
 
Configuration surgery with Augeas (OggCamp 12)
Configuration surgery with Augeas (OggCamp 12)Configuration surgery with Augeas (OggCamp 12)
Configuration surgery with Augeas (OggCamp 12)
 
AnsibleFest 2014 - Role Tips and Tricks
AnsibleFest 2014 - Role Tips and TricksAnsibleFest 2014 - Role Tips and Tricks
AnsibleFest 2014 - Role Tips and Tricks
 

Destacado

Agile deployment predictive analytics on hadoop
Agile deployment predictive analytics on hadoopAgile deployment predictive analytics on hadoop
Agile deployment predictive analytics on hadoop
DataWorks Summit
 

Destacado (8)

Using docker for data science - part 2
Using docker for data science - part 2Using docker for data science - part 2
Using docker for data science - part 2
 
Using Docker for GPU Accelerated Applications
Using Docker for GPU Accelerated ApplicationsUsing Docker for GPU Accelerated Applications
Using Docker for GPU Accelerated Applications
 
Using Docker Containers to Improve Reproducibility in Software and Web Engine...
Using Docker Containers to Improve Reproducibility in Software and Web Engine...Using Docker Containers to Improve Reproducibility in Software and Web Engine...
Using Docker Containers to Improve Reproducibility in Software and Web Engine...
 
Deploying Data Science with Docker and AWS
Deploying Data Science with Docker and AWSDeploying Data Science with Docker and AWS
Deploying Data Science with Docker and AWS
 
Running Docker clusters on AWS (November 2016)
Running Docker clusters on AWS (November 2016)Running Docker clusters on AWS (November 2016)
Running Docker clusters on AWS (November 2016)
 
Deep Learning with AWS (November 2016)
Deep Learning with AWS (November 2016)Deep Learning with AWS (November 2016)
Deep Learning with AWS (November 2016)
 
Reproducible bioinformatics pipelines with Docker and Anduril
Reproducible bioinformatics pipelines with Docker and AndurilReproducible bioinformatics pipelines with Docker and Anduril
Reproducible bioinformatics pipelines with Docker and Anduril
 
Agile deployment predictive analytics on hadoop
Agile deployment predictive analytics on hadoopAgile deployment predictive analytics on hadoop
Agile deployment predictive analytics on hadoop
 

Similar a Using python and docker for data science

Going live with BommandBox and docker Into The Box 2018
Going live with BommandBox and docker Into The Box 2018Going live with BommandBox and docker Into The Box 2018
Going live with BommandBox and docker Into The Box 2018
Ortus Solutions, Corp
 

Similar a Using python and docker for data science (20)

Docker workshop DevOpsDays Amsterdam 2014
Docker workshop DevOpsDays Amsterdam 2014Docker workshop DevOpsDays Amsterdam 2014
Docker workshop DevOpsDays Amsterdam 2014
 
Docker - from development to production (PHPNW 2017-09-05)
Docker - from development to production (PHPNW 2017-09-05)Docker - from development to production (PHPNW 2017-09-05)
Docker - from development to production (PHPNW 2017-09-05)
 
Docker for Web Developers: A Sneak Peek
Docker for Web Developers: A Sneak PeekDocker for Web Developers: A Sneak Peek
Docker for Web Developers: A Sneak Peek
 
Into The Box 2018 Going live with commandbox and docker
Into The Box 2018 Going live with commandbox and dockerInto The Box 2018 Going live with commandbox and docker
Into The Box 2018 Going live with commandbox and docker
 
Going live with BommandBox and docker Into The Box 2018
Going live with BommandBox and docker Into The Box 2018Going live with BommandBox and docker Into The Box 2018
Going live with BommandBox and docker Into The Box 2018
 
Play With Docker
Play With DockerPlay With Docker
Play With Docker
 
Docker workshop
Docker workshopDocker workshop
Docker workshop
 
Running Docker in Development & Production (DevSum 2015)
Running Docker in Development & Production (DevSum 2015)Running Docker in Development & Production (DevSum 2015)
Running Docker in Development & Production (DevSum 2015)
 
From Docker Run To Docker Compose
From Docker Run To Docker ComposeFrom Docker Run To Docker Compose
From Docker Run To Docker Compose
 
Docker tips & tricks
Docker  tips & tricksDocker  tips & tricks
Docker tips & tricks
 
Docker for dev
Docker for devDocker for dev
Docker for dev
 
파이썬 개발환경 구성하기의 끝판왕 - Docker Compose
파이썬 개발환경 구성하기의 끝판왕 - Docker Compose파이썬 개발환경 구성하기의 끝판왕 - Docker Compose
파이썬 개발환경 구성하기의 끝판왕 - Docker Compose
 
Docker Introductory workshop
Docker Introductory workshopDocker Introductory workshop
Docker Introductory workshop
 
Hands-On Session Docker
Hands-On Session DockerHands-On Session Docker
Hands-On Session Docker
 
Create a Database Application Development Environment with Docker
Create a Database Application Development Environment with DockerCreate a Database Application Development Environment with Docker
Create a Database Application Development Environment with Docker
 
Docker by Example - Basics
Docker by Example - Basics Docker by Example - Basics
Docker by Example - Basics
 
2017-03-11 02 Денис Нелюбин. Docker & Ansible - лучшие друзья DevOps
2017-03-11 02 Денис Нелюбин. Docker & Ansible - лучшие друзья DevOps2017-03-11 02 Денис Нелюбин. Docker & Ansible - лучшие друзья DevOps
2017-03-11 02 Денис Нелюбин. Docker & Ansible - лучшие друзья DevOps
 
Docker security
Docker securityDocker security
Docker security
 
Architecting .NET Applications for Docker and Container Based Deployments
Architecting .NET Applications for Docker and Container Based DeploymentsArchitecting .NET Applications for Docker and Container Based Deployments
Architecting .NET Applications for Docker and Container Based Deployments
 
Lessons from running potentially malicious code inside Docker containers
Lessons from running potentially malicious code inside Docker containersLessons from running potentially malicious code inside Docker containers
Lessons from running potentially malicious code inside Docker containers
 

Último

%+27788225528 love spells in new york Psychic Readings, Attraction spells,Bri...
%+27788225528 love spells in new york Psychic Readings, Attraction spells,Bri...%+27788225528 love spells in new york Psychic Readings, Attraction spells,Bri...
%+27788225528 love spells in new york Psychic Readings, Attraction spells,Bri...
masabamasaba
 
+971565801893>>SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHAB...
+971565801893>>SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHAB...+971565801893>>SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHAB...
+971565801893>>SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHAB...
Health
 
introduction-to-automotive Andoid os-csimmonds-ndctechtown-2021.pdf
introduction-to-automotive Andoid os-csimmonds-ndctechtown-2021.pdfintroduction-to-automotive Andoid os-csimmonds-ndctechtown-2021.pdf
introduction-to-automotive Andoid os-csimmonds-ndctechtown-2021.pdf
VishalKumarJha10
 

Último (20)

call girls in Vaishali (Ghaziabad) 🔝 >༒8448380779 🔝 genuine Escort Service 🔝✔️✔️
call girls in Vaishali (Ghaziabad) 🔝 >༒8448380779 🔝 genuine Escort Service 🔝✔️✔️call girls in Vaishali (Ghaziabad) 🔝 >༒8448380779 🔝 genuine Escort Service 🔝✔️✔️
call girls in Vaishali (Ghaziabad) 🔝 >༒8448380779 🔝 genuine Escort Service 🔝✔️✔️
 
Announcing Codolex 2.0 from GDK Software
Announcing Codolex 2.0 from GDK SoftwareAnnouncing Codolex 2.0 from GDK Software
Announcing Codolex 2.0 from GDK Software
 
%+27788225528 love spells in new york Psychic Readings, Attraction spells,Bri...
%+27788225528 love spells in new york Psychic Readings, Attraction spells,Bri...%+27788225528 love spells in new york Psychic Readings, Attraction spells,Bri...
%+27788225528 love spells in new york Psychic Readings, Attraction spells,Bri...
 
Payment Gateway Testing Simplified_ A Step-by-Step Guide for Beginners.pdf
Payment Gateway Testing Simplified_ A Step-by-Step Guide for Beginners.pdfPayment Gateway Testing Simplified_ A Step-by-Step Guide for Beginners.pdf
Payment Gateway Testing Simplified_ A Step-by-Step Guide for Beginners.pdf
 
W01_panagenda_Navigating-the-Future-with-The-Hitchhikers-Guide-to-Notes-and-D...
W01_panagenda_Navigating-the-Future-with-The-Hitchhikers-Guide-to-Notes-and-D...W01_panagenda_Navigating-the-Future-with-The-Hitchhikers-Guide-to-Notes-and-D...
W01_panagenda_Navigating-the-Future-with-The-Hitchhikers-Guide-to-Notes-and-D...
 
%in kempton park+277-882-255-28 abortion pills for sale in kempton park
%in kempton park+277-882-255-28 abortion pills for sale in kempton park %in kempton park+277-882-255-28 abortion pills for sale in kempton park
%in kempton park+277-882-255-28 abortion pills for sale in kempton park
 
+971565801893>>SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHAB...
+971565801893>>SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHAB...+971565801893>>SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHAB...
+971565801893>>SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHAB...
 
Generic or specific? Making sensible software design decisions
Generic or specific? Making sensible software design decisionsGeneric or specific? Making sensible software design decisions
Generic or specific? Making sensible software design decisions
 
%in Hazyview+277-882-255-28 abortion pills for sale in Hazyview
%in Hazyview+277-882-255-28 abortion pills for sale in Hazyview%in Hazyview+277-882-255-28 abortion pills for sale in Hazyview
%in Hazyview+277-882-255-28 abortion pills for sale in Hazyview
 
Unlocking the Future of AI Agents with Large Language Models
Unlocking the Future of AI Agents with Large Language ModelsUnlocking the Future of AI Agents with Large Language Models
Unlocking the Future of AI Agents with Large Language Models
 
%in Durban+277-882-255-28 abortion pills for sale in Durban
%in Durban+277-882-255-28 abortion pills for sale in Durban%in Durban+277-882-255-28 abortion pills for sale in Durban
%in Durban+277-882-255-28 abortion pills for sale in Durban
 
Microsoft AI Transformation Partner Playbook.pdf
Microsoft AI Transformation Partner Playbook.pdfMicrosoft AI Transformation Partner Playbook.pdf
Microsoft AI Transformation Partner Playbook.pdf
 
Exploring the Best Video Editing App.pdf
Exploring the Best Video Editing App.pdfExploring the Best Video Editing App.pdf
Exploring the Best Video Editing App.pdf
 
%in kaalfontein+277-882-255-28 abortion pills for sale in kaalfontein
%in kaalfontein+277-882-255-28 abortion pills for sale in kaalfontein%in kaalfontein+277-882-255-28 abortion pills for sale in kaalfontein
%in kaalfontein+277-882-255-28 abortion pills for sale in kaalfontein
 
%in Stilfontein+277-882-255-28 abortion pills for sale in Stilfontein
%in Stilfontein+277-882-255-28 abortion pills for sale in Stilfontein%in Stilfontein+277-882-255-28 abortion pills for sale in Stilfontein
%in Stilfontein+277-882-255-28 abortion pills for sale in Stilfontein
 
introduction-to-automotive Andoid os-csimmonds-ndctechtown-2021.pdf
introduction-to-automotive Andoid os-csimmonds-ndctechtown-2021.pdfintroduction-to-automotive Andoid os-csimmonds-ndctechtown-2021.pdf
introduction-to-automotive Andoid os-csimmonds-ndctechtown-2021.pdf
 
%in Bahrain+277-882-255-28 abortion pills for sale in Bahrain
%in Bahrain+277-882-255-28 abortion pills for sale in Bahrain%in Bahrain+277-882-255-28 abortion pills for sale in Bahrain
%in Bahrain+277-882-255-28 abortion pills for sale in Bahrain
 
Direct Style Effect Systems - The Print[A] Example - A Comprehension Aid
Direct Style Effect Systems -The Print[A] Example- A Comprehension AidDirect Style Effect Systems -The Print[A] Example- A Comprehension Aid
Direct Style Effect Systems - The Print[A] Example - A Comprehension Aid
 
%in Harare+277-882-255-28 abortion pills for sale in Harare
%in Harare+277-882-255-28 abortion pills for sale in Harare%in Harare+277-882-255-28 abortion pills for sale in Harare
%in Harare+277-882-255-28 abortion pills for sale in Harare
 
The Top App Development Trends Shaping the Industry in 2024-25 .pdf
The Top App Development Trends Shaping the Industry in 2024-25 .pdfThe Top App Development Trends Shaping the Industry in 2024-25 .pdf
The Top App Development Trends Shaping the Industry in 2024-25 .pdf
 

Using python and docker for data science

  • 1. USING DOCKER FOR DATA SCIENCE
  • 2. WHY DOCKER Portable environment Isolated between projects Stateless Fast local file access Hetrogenous
  • 3. GET DOCKER https://docs.docker.com/installation/ boot2docker .dmg or .exe apt-get install docker.io ...
  • 4. RUN SCIPYSERVER $ docker run -d -e "PASSWORD=YourPassword?" ipython/scipyserver $ docker run -d -e "PASSWORD=YourPassword?" --name dev_notebook -p 443:8888 ipython/scipyserver https://localhost:443 https://{boot2docker ip}:443
  • 5. CREATE DATA-ONLY CONTAINERS $ docker run -d -v ~/notebooks:/notebooks --name notebooks_container ubuntu echo notebooks $ docker run -d -v ~/data:/data --name data_container ubuntu echo data
  • 6. MOUNT DATA-ONLY CONTAINERS $ docker stop dev_notebook $ docker rm dev_notebook $ docker run -d -e "PASSWORD=YourPassword?" --name dev_notebook -p 443:8888 --volumes-from data_container --volumes-from notebooks_container ipython/scipyserver
  • 7. CREATE A DOCKERFILE FROM ipython/scipyserver MAINTAINER Calvin Giles <calvin.giles@gmail.com> COPY requirements.txt /requirements.txt RUN pip2 install -r /requirements.txt RUN pip3 install -r /requirements.txt $ docker build -t calvingiles/ds-notebook . $ docker run -d -e "PASSWORD=YourPassword?" --name dev_notebook -p 443:8888 --volumes-from data_container --volumes-from notebooks_container calvingiles/ds-notebook
  • 8. NEXT TIME Connecting to local database containers Tweaking the boot2docker vm memory from 2GB to 8 (or more...) Linking to private git repositories Automated builds with github linking
  • 9. MORE? Docker User Guide: http://docs.docker.com/userguide/ Docker Reference: http://docs.docker.com/reference/commandline/cli/ ipython docker images: https://registry.hub.docker.com/repos/ipython/ my docker image: https://github.com/calvingiles/ds-notebook https://registry.hub.docker.com/u/calvingiles/ds-notebook/
  • 10. ABOUT ME Calvin Giles Data Scientist at Adthena PyData Meetup Organiser untangleconsulting.io calvin.giles@gmail.com @calvingiles on twitter, github, docker hub (and many more)