Separating Hadoop Myths from Reality by ROB ANDERSON at Big Data Spain 2013

•

0 recomendaciones•474 vistas

According to Gartner, Hadoop is near the top of the Hype Cycle. While some customers have questions about the enterprise capabilities of Hadoop, the answers are clear as production deployments continue to expand. This session will use successful customer experiences to highlight the power of Hadoop and separate the myths from reality.

Tecnología Empresariales

Separating Hadoop Myths from Reality

Rob Anderson

The
Myths
&
Reali.es

Surrounding
Hadoop

Rob
Anderson

VP
Systems
Engineering

1

Hadoop
Changes
Analy.cs

“Simple
algorithms
and
lots
of
data
trump

complex
models
”

Halevy,

Norvig,
and

Pereira,
Google

IEEE
Intelligent
Systems

Loca7on

Click

Streams

Sensor

Data

Enterprise

Data
Hub

Produc7on

Data

Web
Logs

Public

Social

Media

Sales

SCM

CRM

2

Billing

Variety

Volume

Velocity

Data

Warehouse

5

Big Data is hard to move…because it’s

7

BIG

What
was
the
genius
of
Hadoop?

Fueling
an
industry
revolu7on

by
providing
inﬁnite
capability

to
store
and
process
big
data

§  Expanding
analy7cs
across
data

types

§  Compelling
economics

§ 

– 
20
to
100X
more
cost
eﬀec7ve

than
alterna7ves

8

Random
Wri.ng
in
MapR

Client

wri.ng

data

Write

next
chunk

to
S2

S1

Ask
for

64M
block

Create
cont.

aZach

CLDB

S1, S2, S4
S1, S3
S1, S4, S5

S2
Picks
master

and
2
replica
slaves

S2, S4, S5
S3

S2, S3, S5

S4

S3

10

S5

TwiZer

TwiZer

API

MapR

Spout

DFS

TwiZerLogger

MapR

Op7onal

MapReduce

Storm

12

hZp://www.ﬂickr.com/photos/onemoreshotrog/8085462024/

13

Hadoop:
The
Disrup.ve
Technology

at
the
Core
of
Big
Data

Architecture
Comparison

HBase

JVM

HDFS

JVM

ext3/ext4

MapR
Data
System

Disks

Disks

Other
Distribu7ons

MapR
M7

Architecture
Results

Results
with
other

distribu.ons

Results
with

MapR
M7

1000+
Nodes

Fortune
100
Financial
Services
Company

23

Produc7on
Hadoop
in

Waste
Management

25

Separating Hadoop Myths from Reality by ROB ANDERSON at Big Data Spain 2013

Más contenido relacionado

La actualidad más candente

Jeff Fletcher - Building a Hadoop based infrastructure as a service product a...Saratoga

BigData Hadoop Kumari Surabhi

Big data use casesRussell Lankenau

IBM Big Data in the CloudRob Thomas

Big Data LDN 2017: The Rise Of The Streaming PlatformMatt Stubbs

Big data hadoopAgnieszka Zdebiak

Meetup at AI NextCon 2019: In-Stream data process, Data Orchestration & MoreAlluxio, Inc.

Big Data Landscape 2016Josef Adersberger

Big Data Sameer Sawhney

Data Orchestration for AI, Big Data, and CloudAlluxio, Inc.

Introduction of Big data and Hadoop Arohi Khandelwal

IBM Big Data Platform, 2012Rob Thomas

Introducing the Big Data Ecosystem with Caserta Concepts & TalendCaserta

Cloudera 助力台灣大數據產業的發展Etu Solution

SQLSat 245 - Por Onde Começar no BigDataDiego Nogare

OpenDataSoft platform designed for big data issuesOpenDataSoft

Présentation on radoop siliconsudipt

Gail Zhou on "Big Data Technology, Strategy, and Applications"Gail Zhou, MBA, PhD

Introducing the hadoop ecosystemGeert Van Landeghem

How is smart data cooked?Ontotext

La actualidad más candente (20)

Jeff Fletcher - Building a Hadoop based infrastructure as a service product a...

BigData Hadoop

Big data use cases

IBM Big Data in the Cloud

Big Data LDN 2017: The Rise Of The Streaming Platform

Big data hadoop

Meetup at AI NextCon 2019: In-Stream data process, Data Orchestration & More

Big Data Landscape 2016

Big Data

Data Orchestration for AI, Big Data, and Cloud

Introduction of Big data and Hadoop

IBM Big Data Platform, 2012

Introducing the Big Data Ecosystem with Caserta Concepts & Talend

Cloudera 助力台灣大數據產業的發展

SQLSat 245 - Por Onde Começar no BigData

OpenDataSoft platform designed for big data issues

Présentation on radoop

Gail Zhou on "Big Data Technology, Strategy, and Applications"

Introducing the hadoop ecosystem

How is smart data cooked?

Destacado

Capturing Value from Big Financial Data by MARCELO SORIA-RODRÍGUEZ at Big Dat...Big Data Spain

The top five questions to ask about NoSQL. JONATHAN ELLIS at Big Data Spain 2012Big Data Spain

From Hadoop to the Future - Opening note by OSCAR MENDEZ at Big Data Spain 2012Big Data Spain

Intro to the Big Data Spain 2014 conferenceBig Data Spain

Workshop - Hadoop + R by CARLOS GIL BELLOSTA at Big Data Spain 2013Big Data Spain

Big data meets scalable visualizations by JAVIER DE LA TORRE at Big Data Spai...Big Data Spain

The Big Business of Big Data. JON BRUNER at Big Data Spain 2012Big Data Spain

Putting Hadoop on any Cloud. NATI SHALOM at Big Data Spain 2012Big Data Spain

Health Insurance Predictive Analysis with Hadoop and Machine Learning. JULIEN...Big Data Spain

Crunching Data with Google BigQuery. JORDAN TIGANI at Big Data Spain 2012Big Data Spain

Building a heterogeneous Hadoop Olap system with Microsoft BI stack. PABLO DO...Big Data Spain

Destacado (11)

Capturing Value from Big Financial Data by MARCELO SORIA-RODRÍGUEZ at Big Dat...

The top five questions to ask about NoSQL. JONATHAN ELLIS at Big Data Spain 2012

From Hadoop to the Future - Opening note by OSCAR MENDEZ at Big Data Spain 2012

Intro to the Big Data Spain 2014 conference

Workshop - Hadoop + R by CARLOS GIL BELLOSTA at Big Data Spain 2013

Big data meets scalable visualizations by JAVIER DE LA TORRE at Big Data Spai...

The Big Business of Big Data. JON BRUNER at Big Data Spain 2012

Putting Hadoop on any Cloud. NATI SHALOM at Big Data Spain 2012

Health Insurance Predictive Analysis with Hadoop and Machine Learning. JULIEN...

Crunching Data with Google BigQuery. JORDAN TIGANI at Big Data Spain 2012

Building a heterogeneous Hadoop Olap system with Microsoft BI stack. PABLO DO...

Similar a Separating Hadoop Myths from Reality by ROB ANDERSON at Big Data Spain 2013

Data Warehouse Evolution RoadshowMapR Technologies

Forecast of Big Data TrendsIMC Institute

What is hadoopAsis Mohanty

EMC Isilon Database Converged deckKeithETD_CTO

Cascading User Group MeetVinoth Kannan

Data Lakes on Public Cloud: Breaking Data Management MonolithsItai Yaffe

AWS Partner Webcast - Hadoop in the Cloud: Unlocking the Potential of Big Dat...Amazon Web Services

Big Data on Public CloudIMC Institute

Is your cloud ready for Big Data? Strata NY 2013Richard McDougall

Big Data in Action – Real-World Solution ShowcaseInside Analysis

Red Hat - Presentation at Hortonworks Booth - Strata 2014Hortonworks

Introduction to Microsoft Azure HD Insight by Dattatrey Sindhol HARMAN Services

My Other Computer is a Data Center (2010 v21)Robert Grossman

Million Monkeys User GroupJesse Anderson

Big Data Everywhere Chicago: Leading a Healthcare Company to the Big Data Pro...BigDataEverywhere

Sharing Experiences in Cloud Adoption: Burlington, MANuoDB

Leonard Austin (Ravelin) - DevOps in a Machine Learning WorldOutlyer

Vmware Serengeti - Based on Infochimps IronfanJim Kaskade

Refactoring your EDW with Mobile Analytics ProductsLuke Han

GPU 101: The Beast In Data CentersRommel Garcia

Similar a Separating Hadoop Myths from Reality by ROB ANDERSON at Big Data Spain 2013 (20)

Data Warehouse Evolution Roadshow

Forecast of Big Data Trends

What is hadoop

EMC Isilon Database Converged deck

Cascading User Group Meet

Data Lakes on Public Cloud: Breaking Data Management Monoliths

AWS Partner Webcast - Hadoop in the Cloud: Unlocking the Potential of Big Dat...

Big Data on Public Cloud

Is your cloud ready for Big Data? Strata NY 2013

Big Data in Action – Real-World Solution Showcase

Red Hat - Presentation at Hortonworks Booth - Strata 2014

Introduction to Microsoft Azure HD Insight by Dattatrey Sindhol

My Other Computer is a Data Center (2010 v21)

Million Monkeys User Group

Big Data Everywhere Chicago: Leading a Healthcare Company to the Big Data Pro...

Sharing Experiences in Cloud Adoption: Burlington, MA

Leonard Austin (Ravelin) - DevOps in a Machine Learning World

Vmware Serengeti - Based on Infochimps Ironfan

Refactoring your EDW with Mobile Analytics Products

GPU 101: The Beast In Data Centers

Más de Big Data Spain

Big Data, Big Quality? by Irene Gonzálvez at Big Data Spain 2017Big Data Spain

Scaling a backend for a big data and blockchain environment by Rafael Ríos at...Big Data Spain

AI: The next frontier by Amparo Alonso at Big Data Spain 2017Big Data Spain

Disaster Recovery for Big Data by Carlos Izquierdo at Big Data Spain 2017Big Data Spain

Presentation: Boost Hadoop and Spark with in-memory technologies by Akmal Cha...Big Data Spain

Data science for lazy people, Automated Machine Learning by Diego Hueltes at ...Big Data Spain

Training Deep Learning Models on Multiple GPUs in the Cloud by Enrique Otero ...Big Data Spain

Unbalanced data: Same algorithms different techniques by Eric Martín at Big D...Big Data Spain

State of the art time-series analysis with deep learning by Javier Ordóñez at...Big Data Spain

Trading at market speed with the latest Kafka features by Iñigo González at B...Big Data Spain

Unified Stream Processing at Scale with Apache Samza by Jake Maes at Big Data...Big Data Spain

The Analytic Platform behind IBM’s Watson Data Platform by Luciano Resende a...Big Data Spain

Artificial Intelligence and Data-centric businesses by Óscar Méndez at Big Da...Big Data Spain

Why big data didn’t end causal inference by Totte Harinen at Big Data Spain 2017Big Data Spain

Meme Index. Analyzing fads and sensations on the Internet by Miguel Romero at...Big Data Spain

Vehicle Big Data that Drives Smart City Advancement by Mike Branch at Big Dat...Big Data Spain

End of the Myth: Ultra-Scalable Transactional Management by Ricardo Jiménez-P...Big Data Spain

Attacking Machine Learning used in AntiVirus with Reinforcement by Rubén Mart...Big Data Spain

More people, less banking: Blockchain by Salvador Casquero at Big Data Spain ...Big Data Spain

Make the elephant fly, once again by Sourygna Luangsay at Big Data Spain 2017Big Data Spain

Más de Big Data Spain (20)

Big Data, Big Quality? by Irene Gonzálvez at Big Data Spain 2017

Scaling a backend for a big data and blockchain environment by Rafael Ríos at...

AI: The next frontier by Amparo Alonso at Big Data Spain 2017

Disaster Recovery for Big Data by Carlos Izquierdo at Big Data Spain 2017

Presentation: Boost Hadoop and Spark with in-memory technologies by Akmal Cha...

Data science for lazy people, Automated Machine Learning by Diego Hueltes at ...

Training Deep Learning Models on Multiple GPUs in the Cloud by Enrique Otero ...

Unbalanced data: Same algorithms different techniques by Eric Martín at Big D...

State of the art time-series analysis with deep learning by Javier Ordóñez at...

Trading at market speed with the latest Kafka features by Iñigo González at B...

Unified Stream Processing at Scale with Apache Samza by Jake Maes at Big Data...

The Analytic Platform behind IBM’s Watson Data Platform by Luciano Resende a...

Artificial Intelligence and Data-centric businesses by Óscar Méndez at Big Da...

Why big data didn’t end causal inference by Totte Harinen at Big Data Spain 2017

Meme Index. Analyzing fads and sensations on the Internet by Miguel Romero at...

Vehicle Big Data that Drives Smart City Advancement by Mike Branch at Big Dat...

End of the Myth: Ultra-Scalable Transactional Management by Ricardo Jiménez-P...

Attacking Machine Learning used in AntiVirus with Reinforcement by Rubén Mart...

More people, less banking: Blockchain by Salvador Casquero at Big Data Spain ...

Make the elephant fly, once again by Sourygna Luangsay at Big Data Spain 2017

Último

Vertex AI Gemini Prompt Engineering TipsMiki Katsuragi

E-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptxnull - The Open Security Community

Anypoint Exchange: It’s Not Just a Repo!Manik S Magar

Ensuring Technical Readiness For Copilot in Microsoft 3652toLead Limited

Nell’iperspazio con Rocket: il Framework Web di Rust!Commit University

Advanced Test Driven-Development @ php[tek] 2024Scott Keck-Warren

Install Stable Diffusion in windows machinePadma Pradeep

Unleash Your Potential - Namagunga Girls Coding ClubKalema Edgar

"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...Fwdays

Unraveling Multimodality with Large Language Models.pdfAlex Barbosa Coqueiro

SIP trunking in Janus @ Kamailio World 2024Lorenzo Miniero

Training state-of-the-art general text embeddingZilliz

What's New in Teams Calling, Meetings and Devices March 2024Stephanie Beckett

DMCC Future of Trade Web3 - Special EditionDubai Multi Commodity Centre

Artificial intelligence in cctv survelliance.pptxhariprasad279825

My Hashitalk Indonesia April 2024 PresentationRidwan Fadjar

CloudStudio User manual (basic edition):comworks

"ML in Production",Oleksandr BaganFwdays

Story boards and shot lists for my a level piececharlottematthew16

Integration and Automation in Practice: CI/CD in Mule Integration and Automat...Patryk Bandurski

Separating Hadoop Myths from Reality by ROB ANDERSON at Big Data Spain 2013

1. Separating Hadoop Myths from Reality Rob Anderson

2. The Myths & Reali.es Surrounding Hadoop Rob Anderson VP Systems Engineering 1

3. Hadoop Changes Analy.cs “Simple algorithms and lots of data trump complex models ” Halevy, Norvig, and Pereira, Google IEEE Intelligent Systems Loca7on Click Streams Sensor Data Enterprise Data Hub Produc7on Data Web Logs Public Social Media Sales SCM CRM 2 Billing

4. 3

5. 4

6. Variety Volume Velocity Data Warehouse 5

7. 6

8. Big Data is hard to move…because it’s 7 BIG

9. What was the genius of Hadoop? Fueling an industry revolu7on by providing inﬁnite capability to store and process big data §  Expanding analy7cs across data types §  Compelling economics §  –  20 to 100X more cost eﬀec7ve than alterna7ves 8

10. 9

11. Random Wri.ng in MapR Client wri.ng data Write next chunk to S2 S1 Ask for 64M block Create cont. aZach CLDB S1, S2, S4 S1, S3 S1, S4, S5 S2 Picks master and 2 replica slaves S2, S4, S5 S3 S2, S3, S5 S4 S3 10 S5

12. 11

13. TwiZer TwiZer API MapR Spout DFS TwiZerLogger MapR Op7onal MapReduce Storm 12

14. hZp://www.ﬂickr.com/photos/onemoreshotrog/8085462024/ 13

15. Hadoop Distribu.ons 14

16. Hadoop: The Disrup.ve Technology at the Core of Big Data

17. 16

18. The Reality is Architecture MaHers 17

19. Architecture Comparison HBase JVM HDFS JVM ext3/ext4 MapR Data System Disks Disks Other Distribu7ons MapR M7

20. Architecture Results Results with other distribu.ons Results with MapR M7

21. 20

22. Produc.on Success with Hadoop

23. 2000+ Nodes Fortune 100 Retailer 22