Introction to Big Data/ Hadoop - Big Data Overview

1. Trendwise Analytics
Introduction to
Big Data
S.Mohan Kumar
Copyright Trendwise Analytics

2. Trendwise Analytics
Big Data – Hype or Reality?
Gartner's Hype Cycles for Emerging Technologies, 2011-2013
Copyright Trendwise Analytics

3. Trendwise Analytics
Market opportunity
Source: wikibon 2014

4. Trendwise Analytics
Market opportunity
Source: wikibon 2014

5. Trendwise Analytics
InformationWeek 2013 Analytic
Copyright Trendwise Analytics
InformationWeek 2013 Analytic

6. Trendwise Analytics
Data Growth?
Source: The Digital Universe in 2020: Big Data, Bigger Digital Shadows, and Biggest Growth in the Far East (IDC & EMC, December 2012)
Copyright Trendwise Analytics

7. Trendwise Analytics
What is Big Data?

8. Trendwise Analytics
Internet Minute ....
Copyright Trendwise Analytics

9. Trendwise Analytics
Other sources
Copyright Trendwise Analytics
· A commercial aircraft
generates 3GB of
flight sensor data in 1
hour
An ERP system for an mid
size company grows by
1-2TB annually
A Video Suveillance
Camera generates 1-
3TB data in 3 months
Airtel or Vodafone
generates 3TB of
Call Details
Records (CDR)
every day
Every day 2.5 quintillion
(2.5×10^18) bytes of data
is created
i.e., 2,500,000TB

10. Trendwise Analytics
How are Organizations
using Big Data
Technology?

11. Trendwise Analytics
Watson wins Jeopardy!
Feb 14th 2011 – Watson wins Jeopardy!
beating its human opponents.
Watson is IBM’s super computer built
using Big Data Technology.
Copyright Trendwise Analytics 9 Dec 2013 11

12. Trendwise Analytics
Across All Industries
Web app
optimization
Smart meter
Equipment monitoring
monitoring
Advertising
analysis
Life sciences
research
Fraud
detection
Healthcare
outcomes
Weather
forecasting
Natural
resource
exploration
Social
network
analysis
Churn
analysis
Traffic flow
optimization
IT
infrastructure
optimization
Legal
discovery

13. Trendwise Analytics
Real Life Cases
Other Examples:
i. UPS implemented Big Data to track data on 16.3 million
packages per day for 8.8 million customers, with an
average of 39.5 million tracking requests from customers
per day. The company stores over 16 petabytes of data.
ii. Caesars Entertainment uses Big Data to analyze customer
data from its Total Rewards loyalty program,web clickstreams,
and from real-time play in slot machines for its Marketing
and Service. They plan to implement video analytics to increase
customer satisfaction.
Source: http://www.csc.com/big_data/success_stories
Copyright Trendwise Analytics

14. Trendwise Analytics
Some more...
Copyright Trendwise Analytics

15. Trendwise Analytics
TECHNOLOGY
Copyright Trendwise Analytics

16. Trendwise Analytics
Hadoop
NoSQL
SAP Hana
Copyright Trendwise Analytics

17. Trendwise Analytics
What is Hadoop?
Open source project started by Doug Cutting
A platform to manage Big Data
Helps in Distributed computing
Runs on Commodity Hardware

18. Trendwise Analytics
Core components of Hadoop
 Data storage (HDFS)
Runs on commodity hardware
Horizontally scalable
 Processing (MapReduce)
Parallelized (scalable) processing
Fault Tolerant
 Other Tools / Frameworks
HBase, Hive, Pig, Mahout

19. Trendwise Analytics
Hadoop Components
Copyright Trendwise Analytics

20. Trendwise Analytics
Commercial Hadoop Distributions
• Cloudera
• Hortonworks
• Greenplum, A Division of EMC
• IBM InfoSphere BigInsights
Copyright Trendwise Analytics

21. Trendwise Analytics
Where to start?
1. http://hadoop.apache.org/
2. Pre-requisite:
- Linux ( Preferred OS)
- Java JDK
3. Install and run a single node cluster – pseudo mode

22. Trendwise Analytics
How it looks?

23. Trendwise Analytics
Web Interface – NameNode Tracker

24. Trendwise Analytics
Hadoop
NoSQL
SAP Hana
Copyright Trendwise Analytics

25. Trendwise Analytics
What is NoSQL
It does not mean No SQL...
But it means Not only SQL
- It is a schema less database
- Or NonRelational Database
Copyright Trendwise Analytics

26. Trendwise Analytics
Why NoSQL?
Source: www.Benstopford.com
Copyright Trendwise Analytics

27. Trendwise Analytics
Advantages of NoSQL
• Flexible (schema-less)
• Highly scalable
– Horizontal scalability
Copyright Trendwise Analytics
• Cheap
– Most of them are open source

28. Trendwise Analytics
Types of NoSQL
• Key-Value Store (persistent & Volatile)
• Column store
• Document database
• Graph database
Copyright Trendwise Analytics

29. Trendwise Analytics
Examples
Copyright Trendwise Analytics
Key Value
Store
Column
Family
Document Graph
Riak Cassandra MongoDB Neo4j
Redis HBase CouchDB

30. Trendwise Analytics
Contact us
Website: www.TrendwiseAnalytics.com
Email: info@TrendwiseAnalytics.com
US Tollfree: +1 877 268 2872
India Number: +91 80 4094 9600
Copyright Trendwise Analytics