More Related Content Similar to Cloud expo june 2013: Building a Real Time Analytics Platform on Big Data in the Cloud (20) Cloud expo june 2013: Building a Real Time Analytics Platform on Big Data in the Cloud1. 1
Building a Real Time Analytics Platform on
Big Data in the Cloud
Sanjay Sharma
Principal Architect
June 12 2013
2. Impetus
©
2013
Impetus
Technologies
• Software Solutions and Services Company
• Leader in Innovation led Technology services
• 17 years of customer success, 1500 people
• Big Data, Enterprise Mobility,Test and
Performance Engineering, Carrier Grade Large
Systems
3. ©
2013
Impetus
Technologies
?
Why
s9ll
Batch?
$$$
Credits
*
hAp://thenextweb.com/insider/2013/02/07/mailboxs-‐user-‐access-‐queue-‐is-‐a-‐work-‐of-‐frustra9ng-‐genius/
**
hAp://en.wikipedia.org/wiki/Moore's_law
Batch
to
Real
Time
Now
is
the
Time
4. ©
2013
Impetus
Technologies
OLTP
Online
Transac'ons
Processing
NOW
OLAP/
DW
Online
Analy'cal
Processing/DW
ZZZZZZZZZ…
Transac9ons
&
Analy9cal
Processing
Now
is
the
Time
5. ©
2013
Impetus
Technologies
Age
of
Horizontal
Scalability
-‐Divide
and
Conquer
Online
Transac'ons
Processing
Online
Analy'cal
Processing/
DW
OLTP
OLAP
6. ©
2013
Impetus
Technologies
Online
Transac9ons
Processing
Online
Analy9cal
Processing/
DW
On
Demand
Scalability
-‐CLOUD
OLTP
OLAP
7. ©
2013
Impetus
Technologies
State
of
Play:
Analy9cs/
BI
Database
RESULTS
Time
Time
Ingest/ETL
SQL
Machine
Data
8. ©
2013
Impetus
Technologies
Faster
Analy9cs:
Streaming
Analy9cs/
Complex
Event
Processing
Database
RESULTS
Time
Time
RESULTS
Ingest/ETL
SQL
Approach
1
9. ©
2013
Impetus
Technologies
Faster
Analy9cs:
In-‐memory
Analy9cs
Database
RESULTS
Time
Time
RESULTS
Ingest/ETL
SQL
Approach
2
10. • Streaming/CEP Analytics
– Storm by Twitter
– Spark Streaming by Berkley Amplabs
– IBM Streams
– SQLstream
– MS StreamInsight
– StreamBase
– Oracle CEP/Streams
– Esper
©
2013
Impetus
Technologies
• In-Memory Analytics
– SAP Hana
– Oracle Exalytics
– SAS in-memory analytics
– MS Powerpivot
Some
Real
Time
So_ware
11. ©
2013
Impetus
Technologies
• Others
– NoSQL databases
• Cassandra, MongoDB, HBase
– Grid solutions
– MPP and NewSQL
– Druid, Kognitio
– Solr/Elastic Search/Marklogic
– Splunk
– BI products with OLAP
• Hadoop based
– Summarization/ cube
preparation engine
– Impala/Stinger/Hawq/
Drill
– Hstreaming
Some
Real
Time
So_ware
• Specialized
– Hardware accelerators / GPU
cards
• Sisense
12. ©
2013
Impetus
Technologies
Real
Time
Analy9cs
Strategy
-‐Key
Components
RESULT
S
Ingest
SQL
Machine
Data
OLTP
OLAP/
DW
ETL
Discovery
& Search
Deeper
Analytics
Streaming
Analytics
Real time
Analytics
13. ©
2013
Impetus
Technologies
NoSQL
+
Search
Machine
Data
Real
Time
Analy9cs
Strategy
-‐Big
Data
Reference
Architecture
Example
14. ©
2013
Impetus
Technologies
Real
Time
Analy9cs
Strategy
-‐Data
Inges9on/
Streaming
–
Customer
Example
Machine
Data
Inges9on
Engine
(Real
9me
+
Batch
components)
Real
Time
Processing
Engine
(CEP/Analy9cs/
Rule
Engine)
Real
Time
Data
Storage
Engine
(Store
+
Indexing/
Search)
Business
Process
Engine
(Business
Process/
Rule
management)
Kafa
Storm
+
Esper
HBase+
Solr
JBoss
Drools/jBPM
15. ©
2013
Impetus
Technologies
NoSQL
+
Search
Machine
Data
Real
Time
Analy9cs
Strategy
-‐Big
Data
Reference
Architecture
Example
16. Impetus provides vendor-neutral Consulting &
Services for Big Data Analytics
We take enterprises through the journey of
successful Big Data adoption
We deliver: Low-risk acceleration to
Business value at moderate cost
©
2013
Impetus
Technologies
Impetus