CS-Op Analytics

1. 1
©
Cloudera,
Inc.
All
rights
reserved.
Smarter
Decisions
in
Less
Time
Opera?onal
Analy?cs
with
Cloudera

2. 2
©
Cloudera,
Inc.
All
rights
reserved.
Opera?onalizing
Reports,
Models,
or
Rules
Recommenda)on
Engine
Event
Detec)on
Model
Scoring
Point
Solu)ons
Custom
Development
3rd
Party
Data
Discovery
&
Analy8cs

3. 3
©
Cloudera,
Inc.
All
rights
reserved.
Custom
Development
Use
Cases
Recommenda)on
Engine
Event
Detec)on
Model
Scoring
Fraud
Detec?on
Spam
Filter
Marke?ng
Alerts
Embedded
Analy?cs
Analy?c
Aggregates
Reports
Next
Best
Offer
Content
Rec
Services
Rec

4. 4
©
Cloudera,
Inc.
All
rights
reserved.
The
Process
of
Opera?onal
Analy?cs
Data
Discovery
Advanced
Analy8cs
Data
Volumes
Stream
&
Batch
Processing
Data
Genera?on
Opera8onal
Analy8cs
Flow
Op?mize
Analy?c
Func?on
Processing
Respond
to
Data
Feed
Data
Applica?on
Act
and
Measure
Model
Flexibility
Scalability
Embedded
Analy8cs
Reports

5. 5
©
Cloudera,
Inc.
All
rights
reserved.
Opera?onal
Analy?c
Needs
Scale
Embed
Analy8cs
Enterprise
Data
Warehouse
Data
Data
Sources
ETL
Structured
Unstructured
Database
ELT
Store
&
Process
Tradi8onal
Architecture
Archive
Serve
Ac?on
Model
Process
f
(D1,
DN)
Structured
Unstructured
Machine
Drill
Down
Human
API
Ingest
LiHle
Latency

6. 6
©
Cloudera,
Inc.
All
rights
reserved.
Challenges
with
Tradi?onal
Opera?onal
Analy?c
1)
Limited
Data
3)
Analy8c
Latency
2)
Drill
Down
Performance
Enterprise
Data
Warehouse
Data
Data
Sources
ETL
Structured
Unstructured
Database
ELT
Store
&
Process
Tradi8onal
Architecture
Archive
Serve
Ac?on
Model
Process
f
(D1,
DN)
Structured
Unstructured
Machine
Drill
Down
Human
API
Ingest
1
2
1
3

7. 7
©
Cloudera,
Inc.
All
rights
reserved.
A
New
Way
Forward
1)
Data
Scale
3)
LiHle
Latency
2)
Drill
Down
Speed
Enterprise
Data
Warehouse
Data
Data
Sources
ETL
Structured
Unstructured
Enterprise
Data
Hub
ELT
Store
&
Process
Modern
Architecture
Serve
Ac?on
Process
f
(D1,
DN)
Structured
Unstructured
Machine
Drill
Down
Human
API
Ingest
1
1
2
3

8. 8
©
Cloudera,
Inc.
All
rights
reserved.
Opower
Customer
Story

9. 9
©
Cloudera,
Inc.
All
rights
reserved.
Opower
Overview
The
Company
• Serving
95+
u?li?es
in
9
countries
• Over
5TWh
saved
to
date
• 40%
of
US
household
data
under
management
totaling
300
billion
reads
Our
DNA
• Behavioral
science
so^ware
• Data
analy?cs
• Consumer
marke?ng
• User-­‐centric
design
A
So^ware
as
a
Service
Customer
Engagement
Pla`orm

10. 10
©
Cloudera,
Inc.
All
rights
reserved.
Opower’s
Personalized
Insights
Neighbor
comparisons
Usage
trend
analysis

11. 11
©
Cloudera,
Inc.
All
rights
reserved.
Ini?al
Hadoop
Architecture
1
2
3
Ingest
performance
Complex
query
paths
1
3
2
Challenges
Mul?ple
workloads

12. 12
©
Cloudera,
Inc.
All
rights
reserved.
Modern
Hadoop
Architecture
Offline
Analysis
and
Experimenta?on
Product
Analy?cs
Ingest
Performance
Workload
separa?on
3
1
2
Improvements
En?ty-­‐centric
HBase
schema
2
1
3

13. 13
©
Cloudera,
Inc.
All
rights
reserved.
Insight
Crea?on
Environments
Insight
Delivery
Insight
Calcula?on
Product
Calcula8on
and
Delivery
Offline
Analysis
and
Experimenta8on
Meter reads
(gas)
Meter reads
(electric)
Bill forecast
insight
MapReduce
HBase Site Row
Insight Service
Application
Bulkload
ETL
Hive BI
Raw
MR
Batch Tools
HDFS
Reporting
External
Feeds
HBase Export
Non-product
Insights

14. 14
©
Cloudera,
Inc.
All
rights
reserved.
What
does
this
mean
to
end
users?
Batch
Analy8c
Calcula8ons
Individual
Insight
Query
Latency
Pre-­‐Hadoop
Modern
Hadoop
Hours
12
24
48
Hours
Days
Pre-­‐Hadoop
Seconds
1
2
3
~10ms
3
secs
Analy8c
Development
Time
Pre-­‐Hadoop
Months
1
3
5
Weeks
Months
Modern
Hadoop
Modern
Hadoop

15. Thank
you.