Big data originated from the need to index and store massive amounts of web data in a cost-effective way, leading Google to pioneer technologies like MapReduce and BigTable. Hadoop was later developed at Yahoo as an open-source software framework for reliable, distributed storage and processing of large datasets across commodity hardware. It has since been adopted by many companies as a primary platform for big data applications and is now managed by the Apache Software Foundation.

1. BIG DATA AND HADOOP
Big Data History,Hadoop, and Industry Trends
E S T H E R K U N D I N
B L O O M B E R G L P

2. BIG DATA – WHAT IS IT?

3. WHAT IS BIG DATA?

4. BIG DATA ORIGINS
Indexing the web requires lots of storage - Petabytes of data!
Economic problem – reliable servers expensive!
Solution:
 Cram in as many cheap machines as possible
 Replace them when they fail
 Solve reliability via software!
Google publishes papers about:
 GFS (2000)
 MapReduce (2004)
 BigTable (2006)

5. HADOOP – OPEN SOURCE BIG DATA SOLUTION
Hadoop, originally developed at
Yahoo, accepted as Apache top-level
project in 2008

6. CURRENT INDUSTRY TRENDS

7. QUESTIONS?
Thank you!