SlideShare a Scribd company logo
1 of 23
Trends in Big Data
Selvaraaju Murugesan
https://www.linkedin.com/in/selvaraaju/
Overview
• Big data evolution and role of data
• Big data ecosystem and tools
• Demo of MapR cluster
• Trends in next 5 years
• My personal recommendations
• Conclusion
When big data was small ?
• Most of the transactional data was/(is) stored in
databases
• Social media was getting some traction
• Mobile phone penetration was very low
• Managers made decision on reports that is based on
static data (there is no live feed of data that influences
their decision at right time)
Big data landscape has changed !
Old Vs New Paradigm
Distributed Storage
Big data ≠ Hadoop
Main players
MapR Converged Data Platform
David vs Goliath
https://www.dezyre.com/article/cloudera-vs-hortonworks-vs-mapr-hadoop-distribution-comparison-/190
Ecosystem tools
Big data process
Ingestion Storage Analysis Presentation
Data Ingestion : Flume / Streamsets / Impala
Hive / Hue
Self-Service Data Exploration
Data Agility with Less IT Required
Single SQL Interface for Structured and
Semi-Structured Data
Data Exploration
Data Analytics – R / Spark R
Operationalise – Spark
Trends in next 5 years
• Every home will not have super computers but powerful nodes that
can do distributed computing and storage
• Analysing data and decision making will be performed by 5 year old
using standard AI libraries and cheap hardware
• Big data will empower deep learning
• Bots will try to mimic human services
What is after big data ?
Recommendation
• Big data platform can be implemented in many ways ; Hadoop is not
the only option !
• Analyse important data bytes that is relevant to make business
decisions
• Beware of cloud providers and their traps
• Data driven decision making but hunch is very important
Creativity > big data
Trends in big data

More Related Content

What's hot

What's hot (16)

Use case and integration of ClickHouse with Apache Superset & Dremio
Use case and integration of ClickHouse with Apache Superset & DremioUse case and integration of ClickHouse with Apache Superset & Dremio
Use case and integration of ClickHouse with Apache Superset & Dremio
 
Top 6 Information Management and Data podcasts
Top 6 Information Management and Data podcastsTop 6 Information Management and Data podcasts
Top 6 Information Management and Data podcasts
 
Big data analytics presented at meetup big data for decision makers
Big data analytics presented at meetup big data for decision makersBig data analytics presented at meetup big data for decision makers
Big data analytics presented at meetup big data for decision makers
 
IBM and Apache Spark
IBM and Apache SparkIBM and Apache Spark
IBM and Apache Spark
 
BIg Data Trends in 2016
BIg Data Trends in 2016BIg Data Trends in 2016
BIg Data Trends in 2016
 
Realtime interactive dashboard 2015
Realtime interactive dashboard 2015Realtime interactive dashboard 2015
Realtime interactive dashboard 2015
 
Big Data and the pursuit of African "indigenuity"
Big Data and the pursuit of African "indigenuity"Big Data and the pursuit of African "indigenuity"
Big Data and the pursuit of African "indigenuity"
 
MongoDB IoT City Tour LONDON: Hadoop and the future of data management. By, M...
MongoDB IoT City Tour LONDON: Hadoop and the future of data management. By, M...MongoDB IoT City Tour LONDON: Hadoop and the future of data management. By, M...
MongoDB IoT City Tour LONDON: Hadoop and the future of data management. By, M...
 
Big Data, Big Deal? (A Big Data 101 presentation)
Big Data, Big Deal? (A Big Data 101 presentation)Big Data, Big Deal? (A Big Data 101 presentation)
Big Data, Big Deal? (A Big Data 101 presentation)
 
Big data
Big dataBig data
Big data
 
Big Data Landscape 2016
Big Data Landscape 2016 Big Data Landscape 2016
Big Data Landscape 2016
 
Lessons from Digital Natives: How Retailers Power their Businesses with DataOps
Lessons from Digital Natives: How Retailers Power their Businesses with DataOpsLessons from Digital Natives: How Retailers Power their Businesses with DataOps
Lessons from Digital Natives: How Retailers Power their Businesses with DataOps
 
Thilga
ThilgaThilga
Thilga
 
Big Data Landscape 2018
Big Data Landscape 2018Big Data Landscape 2018
Big Data Landscape 2018
 
Big data
Big dataBig data
Big data
 
David Ottewell - data journalism
David Ottewell - data journalism David Ottewell - data journalism
David Ottewell - data journalism
 

Viewers also liked

"Своя игра"(игра по теме "Давление")
"Своя игра"(игра по теме "Давление")"Своя игра"(игра по теме "Давление")
"Своя игра"(игра по теме "Давление")
sveta7940
 
Sports and Big data
Sports and Big dataSports and Big data
Sports and Big data
DeZyre
 

Viewers also liked (20)

Presentation romuald cetkovic
Presentation romuald cetkovicPresentation romuald cetkovic
Presentation romuald cetkovic
 
Gibi acessibilidade
Gibi acessibilidadeGibi acessibilidade
Gibi acessibilidade
 
Circuito electrico taller 11 2
Circuito electrico taller 11 2Circuito electrico taller 11 2
Circuito electrico taller 11 2
 
Taller 2
Taller 2Taller 2
Taller 2
 
Dev Wednesday - Swiss Transport in Real Time: Tribulations in the Big Data Stack
Dev Wednesday - Swiss Transport in Real Time: Tribulations in the Big Data StackDev Wednesday - Swiss Transport in Real Time: Tribulations in the Big Data Stack
Dev Wednesday - Swiss Transport in Real Time: Tribulations in the Big Data Stack
 
Trại hè tiếng Anh trẻ em tại trường Anh ngữ CPILS
Trại hè tiếng Anh trẻ em tại trường Anh ngữ CPILSTrại hè tiếng Anh trẻ em tại trường Anh ngữ CPILS
Trại hè tiếng Anh trẻ em tại trường Anh ngữ CPILS
 
Ira
IraIra
Ira
 
Tom a3
Tom a3Tom a3
Tom a3
 
A 63 copy
A 63 copyA 63 copy
A 63 copy
 
CATALOGO BONG BLACK LEAF 2017 >> By PuntoG
CATALOGO BONG BLACK LEAF  2017 >> By PuntoGCATALOGO BONG BLACK LEAF  2017 >> By PuntoG
CATALOGO BONG BLACK LEAF 2017 >> By PuntoG
 
Sam a3
Sam a3Sam a3
Sam a3
 
"Своя игра"(игра по теме "Давление")
"Своя игра"(игра по теме "Давление")"Своя игра"(игра по теме "Давление")
"Своя игра"(игра по теме "Давление")
 
Secuencia de clase 9 copia
Secuencia de clase  9   copiaSecuencia de clase  9   copia
Secuencia de clase 9 copia
 
Iot intro
Iot introIot intro
Iot intro
 
Max a6
Max a6Max a6
Max a6
 
Badanie i implementacja aspektu QoE (ang. Quality of Experience) w aplikacjac...
Badanie i implementacja aspektu QoE (ang. Quality of Experience) w aplikacjac...Badanie i implementacja aspektu QoE (ang. Quality of Experience) w aplikacjac...
Badanie i implementacja aspektu QoE (ang. Quality of Experience) w aplikacjac...
 
Big Data Trends
Big Data TrendsBig Data Trends
Big Data Trends
 
What is Big Data?
What is Big Data?What is Big Data?
What is Big Data?
 
Customer success management(csm)
Customer success management(csm)Customer success management(csm)
Customer success management(csm)
 
Sports and Big data
Sports and Big dataSports and Big data
Sports and Big data
 

Similar to Trends in big data

02 a holistic approach to big data
02 a holistic approach to big data02 a holistic approach to big data
02 a holistic approach to big data
Raul Chong
 
Big-Data-Seminar-6-Aug-2014-Koenig
Big-Data-Seminar-6-Aug-2014-KoenigBig-Data-Seminar-6-Aug-2014-Koenig
Big-Data-Seminar-6-Aug-2014-Koenig
Manish Chopra
 
Capturing big value in big data
Capturing big value in big data Capturing big value in big data
Capturing big value in big data
BSP Media Group
 

Similar to Trends in big data (20)

02 a holistic approach to big data
02 a holistic approach to big data02 a holistic approach to big data
02 a holistic approach to big data
 
INTRODUCTION TO BIG DATA AND HADOOP
INTRODUCTION TO BIG DATA AND HADOOPINTRODUCTION TO BIG DATA AND HADOOP
INTRODUCTION TO BIG DATA AND HADOOP
 
Hadoop Perspectives for 2017
Hadoop Perspectives for 2017Hadoop Perspectives for 2017
Hadoop Perspectives for 2017
 
Oh! Session on Introduction to BIG Data
Oh! Session on Introduction to BIG DataOh! Session on Introduction to BIG Data
Oh! Session on Introduction to BIG Data
 
Hadoop and the Data Warehouse: When to Use Which
Hadoop and the Data Warehouse: When to Use Which Hadoop and the Data Warehouse: When to Use Which
Hadoop and the Data Warehouse: When to Use Which
 
Recipes for Unlocking Value from Big Data
Recipes for Unlocking Value from Big DataRecipes for Unlocking Value from Big Data
Recipes for Unlocking Value from Big Data
 
Big Data and BI Tools - BI Reporting for Bay Area Startups User Group
Big Data and BI Tools - BI Reporting for Bay Area Startups User GroupBig Data and BI Tools - BI Reporting for Bay Area Startups User Group
Big Data and BI Tools - BI Reporting for Bay Area Startups User Group
 
Are You Prepared For The Future Of Data Technologies?
Are You Prepared For The Future Of Data Technologies?Are You Prepared For The Future Of Data Technologies?
Are You Prepared For The Future Of Data Technologies?
 
Big data management
Big data managementBig data management
Big data management
 
Big-Data-Seminar-6-Aug-2014-Koenig
Big-Data-Seminar-6-Aug-2014-KoenigBig-Data-Seminar-6-Aug-2014-Koenig
Big-Data-Seminar-6-Aug-2014-Koenig
 
Total Data Industry Report
Total Data Industry ReportTotal Data Industry Report
Total Data Industry Report
 
Introduction to Big Data and Hadoop
Introduction to Big Data and HadoopIntroduction to Big Data and Hadoop
Introduction to Big Data and Hadoop
 
Big Data - Part II
Big Data - Part IIBig Data - Part II
Big Data - Part II
 
Fast and Furious: From POC to an Enterprise Big Data Stack in 2014
Fast and Furious: From POC to an Enterprise Big Data Stack in 2014Fast and Furious: From POC to an Enterprise Big Data Stack in 2014
Fast and Furious: From POC to an Enterprise Big Data Stack in 2014
 
IRJET- Survey of Big Data with Hadoop
IRJET-  	  Survey of Big Data with HadoopIRJET-  	  Survey of Big Data with Hadoop
IRJET- Survey of Big Data with Hadoop
 
Big and fast data strategy 2017 jr
Big and fast data strategy 2017 jrBig and fast data strategy 2017 jr
Big and fast data strategy 2017 jr
 
Big Data Hadoop
Big Data HadoopBig Data Hadoop
Big Data Hadoop
 
Introduction to Big Data and Hadoop
Introduction to Big Data and HadoopIntroduction to Big Data and Hadoop
Introduction to Big Data and Hadoop
 
Capturing big value in big data
Capturing big value in big data Capturing big value in big data
Capturing big value in big data
 
Big data
Big dataBig data
Big data
 

Recently uploaded

+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
?#DUbAI#??##{{(☎️+971_581248768%)**%*]'#abortion pills for sale in dubai@
 
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Victor Rentea
 
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Victor Rentea
 

Recently uploaded (20)

Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...
Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...
Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...
 
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost SavingRepurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
 
CNIC Information System with Pakdata Cf In Pakistan
CNIC Information System with Pakdata Cf In PakistanCNIC Information System with Pakdata Cf In Pakistan
CNIC Information System with Pakdata Cf In Pakistan
 
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
 
MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024
 
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
 
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 
Exploring Multimodal Embeddings with Milvus
Exploring Multimodal Embeddings with MilvusExploring Multimodal Embeddings with Milvus
Exploring Multimodal Embeddings with Milvus
 
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
 
MS Copilot expands with MS Graph connectors
MS Copilot expands with MS Graph connectorsMS Copilot expands with MS Graph connectors
MS Copilot expands with MS Graph connectors
 
Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...
Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...
Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...
 
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
 
Platformless Horizons for Digital Adaptability
Platformless Horizons for Digital AdaptabilityPlatformless Horizons for Digital Adaptability
Platformless Horizons for Digital Adaptability
 
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
 
Introduction to Multilingual Retrieval Augmented Generation (RAG)
Introduction to Multilingual Retrieval Augmented Generation (RAG)Introduction to Multilingual Retrieval Augmented Generation (RAG)
Introduction to Multilingual Retrieval Augmented Generation (RAG)
 
Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf
Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdfRising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf
Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf
 
AWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of TerraformAWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of Terraform
 
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 AmsterdamDEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
 

Trends in big data

Editor's Notes

  1. With the volume and velocity of IoT data an important advantage is the ability to explore the data directly, without requiring IT to set up the data This is where Apache Drill comes in. This is a SQL query engine that supports self-service data exploration without the need to predefine a schema. Drill is ANSI SQL compliant and plugs right into all of those BI tools you are accustomed to using. With Drill you simply query your data in place; there is no need to perform ETL or to move your data. After all, if you currently use business intelligence tools, you should be enabled to still use them. Data exploration…before and after…. No IT step required…
  2. With the volume and velocity of IoT data an important advantage is the ability to explore the data directly, without requiring IT to set up the data This is where Apache Drill comes in. This is a SQL query engine that supports self-service data exploration without the need to predefine a schema. Drill is ANSI SQL compliant and plugs right into all of those BI tools you are accustomed to using. With Drill you simply query your data in place; there is no need to perform ETL or to move your data. After all, if you currently use business intelligence tools, you should be enabled to still use them. Data exploration…before and after…. No IT step required…
  3. With the volume and velocity of IoT data an important advantage is the ability to explore the data directly, without requiring IT to set up the data This is where Apache Drill comes in. This is a SQL query engine that supports self-service data exploration without the need to predefine a schema. Drill is ANSI SQL compliant and plugs right into all of those BI tools you are accustomed to using. With Drill you simply query your data in place; there is no need to perform ETL or to move your data. After all, if you currently use business intelligence tools, you should be enabled to still use them. Data exploration…before and after…. No IT step required…