1.
Hadoop &
Germany &
2016
uweseiler

2.
/whoami
&
/disclaimer
Hadoop & Germany & 2016

3.
We finally stopped talking
infrastructure!
Hadoop & Germany & 2016

4.
We now talk
architectures and use cases!
Hadoop & Germany & 2016

5.
#1
The Big Data Lake is an illusion!
Hadoop & Germany & 2016

6.
DataSourcesDataSystemsApplications
Traditional Sources
RDBMS OLTP OLAP …
Traditional Systems
RDBMS EDW MPP …
Business
Intelligence
Business
Applications
Custom
Applications
Operation
Manage
&
Monitor
Dev Tools
Build
&
Test
New Sources
Logs Mails Sensor …Social
Media
Enterprise
Hadoop
Plattform
#1 The Vision of the Big Data Lake

7.
Hadoop is not the one tool
to rule them all
#1 Vision & Reality

8.
Embrace heterogeneity!
(and learn to deal with the complexity)
#1 After the reality shock…

9.
#1 Real world architecture - Insurance
DataSourcesDataSystemsApplications
Traditional Sources
RDBMS OLTP OLAP …
Traditional Systems
DWH
Business
Intelligence
New Sources
Logs Sensor …Social
Media
Enterprise Hadoop Plattform
SAS LASR Server
Apache Zeppelin

10.
#2
Speed is the new king!
Hadoop & Germany & 2016

11.
#2 The “classic“ Lambda Architecture

12.
Batch Layer
Speed Layer
Data Ingestion
Data Processing
Data Storage
Data Storage Data Analysis
Visualization
Visualization
…
Data
Channels
ms - s
min - h
#2 Lambda in Action - (e)Commerce

13.
SMACK
Spark
Mesos
Akka
Cassandra
Kafka
#2 The lust for speed

14.
Data Ingestion
Data Processing
Raw Data
#2 Cassandra & Hadoop - AdServing
Data Processing
User Journey
Aggregated Data
Web Frontend
Aggregated Data
< 120 days
Data Science

15.
#3
Data Science to the help!
Hadoop & Germany & 2016

16.
Hadoop is about to become
commodity
#3 Let’s face it..

17.
Algorithms will be the new
differentiator
#3 We need new challenges…

18.
Batch Layer
Speed Layer
Data Ingestion
Stream Processing
ms - s
min - h
#3 Fraud detection - Financial services
Data
Import
Data
Preparation
Model
Generation
Model
Validation
Feature &
Parameter
Selection
Manual or automatic
Iterations to tune
parameters
Use
Model
Refresh Model from
latest input data

19.
Every major company is
building teams of unicorns
#3 The solution?

20.
#4
Hadoop for good!
Hadoop & Germany & 2016

21.
Hadoop User Group Rhein-Main
http://www.meetup.com/de-DE/HUG-Rhein-Main/
Next Meetup: 23.06.2016, Talks welcome