Data science team from roles and methodologies to data Integration pipeline

1. Data Scientist Team
A practice to tackle real world problems
Omid Mogharian
V0.4.3 - 07.06.2018

2. … it’s a mistake to treat data science teams like any old product
group ... To build teams that create great data products, you have to
find people with the skills and the curiosity to ask the big questions.
You have build cross-disciplinary groups with people who are
comfortable creating together…
DJ Patil, U.S. Chief Data Scientist at White House
Office of Science and Technology Policy

3. Roles
AI/Machine
Learning Engineer
Data
Engineer/Architect
Data Analyst
Math &
Statistics
Data
Analysis/
Visualisation
AI/ML
Math &
Statistics
Artificial
Intelligence/
Machine
Learning
Developing
Developing
Infrastructure
Design
AI/ML
Operation
Programming
Data Scientist
Core Team Skills

4. Relations
Environment
Data Science
Team
Business
Departments
BI
Operations/
Infrastructure
Leadership
(PM/PMO)
Development

5. Process Method
CRISP-DM
TheDataScienceProcess Communications with Customer/business side

6. Data Scientist vs. Developer
The Battle
Efficient Algorithms
+Communication, Improvement
Process Oriented
Lorem ipsum dolor sit amet at
nec at adipiscing
Project/Product Oriented
Working Software
+Monitoring, Maintaining
Turning Data to knowledge
Historical data & Business
Requirements
Data, Machine Learning,
Analytics and Algorithms
Building High quality Software
Product
Platform, Programming,
Frameworks, API, Web/App,
Deployment
Tools
End-users Requirements
GOAL
Focus
Approach
Result

7. Two Env. Solution
AnalyseEnv.
DeliveryEnv.
Storage Layer

8. Big data & Fast data
Why data Eng./Architect

9. Data Integration Pipeline
Streaming Engine
Continuous Analytics
Serving layer
Batch Engine
Analysis Env.
Storage layer
Data Streams
Services/
Web apps
Data Processing
Monitoring/ Performance Analytics
Data Ingest
Publish Streaming /
Batch tasks
Sent feedback
Storingrawdata
Insert Offline Models
Update Models
Send Logs & KPI

10. Data Quality
To bring accuracy

11. References
● https://dzone.com/articles/lambda-architecture-with-apache-spark
● https://en.wikipedia.org/wiki/Lambda_architecture
● https://www.mapr.com/developercentral/lambda-architecture
● http://www.kdnuggets.com/2015/11/different-data-science-roles-industry.html
● http://www.kdnuggets.com/2016/03/data-science-process-rediscovered.html
● http://www.datascienceassn.org/sites/default/files/Building%20Data%20Science%20T
eams.pdf