SlideShare una empresa de Scribd logo
1 de 32
OLLIE PARSLEY

@OLLIEPARSLEY
REALTIME TWITTER DATA MINING


                               @DATASIFT
MINING THE TWEET




170 MILLION (daily)
MINING THE TWEET




170 MILLION (daily)
MINING THE TWEET




                                         REAL-TIME OUTPUT


170 MILLION (daily)
DATA FILTERING
CSDL - Curation Stream Definition Language
AUGMENTATIONS



KLOUT                PEERINDEX            QWERLY




        LEXALYTICS               TRENDS
NEW FEATURES


BIGSTORE

STREAMTAGS
BIGSTORE
BIGSTORE



                                 REAL-TIME OUTPUT




170 MILLION (daily)
BIGSTORE



                                 REAL-TIME OUTPUT




170 MILLION (daily)                  BIGSTORE
BIGSTORE




HBASE
BIGSTORE




HBASE   POST-PROCESSING
BIGSTORE




HBASE   POST-PROCESSING   MAP-REDUCE
BIGSTORE




HBASE   POST-PROCESSING   MAP-REDUCE   EXPORTING
STREAMTAGS
STREAMTAGS
STREAMTAGS
STREAMTAGS
WHAT CAN YOU BUILD?
@STOCKSIFT
@STOCKSIFT
@STOCKSIFT
@CLUSTERAPP
@CLUSTERAPP
@CLUSTERAPP
USE CASES / VERTICALS
• Below	
  are	
  some	
  of	
  the	
  use	
  cases	
  from	
  the	
  Alpha	
  programme.	
  
  This	
  is	
  a	
  con7nuously	
  evolving	
  set	
  of	
  applica7ons,	
  based	
  upon	
  market	
  feedback.         Mining
                                                                                                                          Oil & Gas
                                                                                                                          Exploration
                                                                                                     Sports               Utilities

                                                                                                     Energy               Automobiles
                                                                    Crisis Management
                                                                                                     Industrials          Aerospace / Defence
                                                                                                                          Basic Resources
                                          Events / Trigger          Reputation Management            Banking
                                                                                                                          Chemicals
                                                                                                     Financial Services   Construction
                       Monitoring                                   Risk Analysis
                                         Data Analysis                                               Insurance            Industrial Goods
                                                                     Research
                                                                                                     Property
                                          Brand Marketing             Viral Campaigns                                     Airlines
                     Influencing
                                                                                                     Media                Shipping
                                                          Qualitative Feedback                       Transportation       Rail
                                          CRM
                                                                                                     Health               Pharmaceuticals
                                                         Augmented Research
                                                                                                                          Healthcare
                                                                                                     Technology

                                                                                                      Government          Telecommunications

                                                                                                     Retail / Consumer    F&B
                                                                                                                          Personal Goods
                                                                                                                          Travel & Leisure
WHERE'S MY INVITE??
WHERE'S MY INVITE??


1) Request an invite from datasift.net

2) Email ollie@mediasift.com

3) Enjoy!
WE'RE HIRING!
WE'RE HIRING!


- Various posts: Developers/Operations/Sales

- Perks: Free copy of StarCraft II and brand new office

- Interested? Come talk to me!
ANY QUESTIONS?

Más contenido relacionado

Destacado

Big Data at Twitter, Chirp 2010
Big Data at Twitter, Chirp 2010Big Data at Twitter, Chirp 2010
Big Data at Twitter, Chirp 2010Kevin Weil
 
Twitter as a data mining source
Twitter  as  a data mining sourceTwitter  as  a data mining source
Twitter as a data mining sourceAtaxo Group
 
Social media data for Social science research
Social media data for Social science researchSocial media data for Social science research
Social media data for Social science researchDavide Bennato
 
Data Mining on Twitter
Data Mining on TwitterData Mining on Twitter
Data Mining on TwitterPulkit Goyal
 
Apache Flume and its use case in Manufacturing
Apache Flume and its use case in ManufacturingApache Flume and its use case in Manufacturing
Apache Flume and its use case in ManufacturingRapheephan Thongkham-Uan
 
Analyse Tweets using Flume, Hadoop and Hive
Analyse Tweets using Flume, Hadoop and HiveAnalyse Tweets using Flume, Hadoop and Hive
Analyse Tweets using Flume, Hadoop and HiveIMC Institute
 
Twitter Text Mining with Web scraping, R, Shiny and Hadoop - Richard Sheng
Twitter Text Mining with Web scraping, R, Shiny and Hadoop - Richard Sheng Twitter Text Mining with Web scraping, R, Shiny and Hadoop - Richard Sheng
Twitter Text Mining with Web scraping, R, Shiny and Hadoop - Richard Sheng Richard Sheng
 
Challenges of social media analysis in the real world
Challenges of social media analysis in the real worldChallenges of social media analysis in the real world
Challenges of social media analysis in the real worldDiana Maynard
 
Social media mining PPT
Social media mining PPTSocial media mining PPT
Social media mining PPTChhavi Mathur
 
Hadoop, Pig, and Twitter (NoSQL East 2009)
Hadoop, Pig, and Twitter (NoSQL East 2009)Hadoop, Pig, and Twitter (NoSQL East 2009)
Hadoop, Pig, and Twitter (NoSQL East 2009)Kevin Weil
 
Sentiment Analysis of Twitter Data
Sentiment Analysis of Twitter DataSentiment Analysis of Twitter Data
Sentiment Analysis of Twitter DataSumit Raj
 

Destacado (13)

Big Data at Twitter, Chirp 2010
Big Data at Twitter, Chirp 2010Big Data at Twitter, Chirp 2010
Big Data at Twitter, Chirp 2010
 
Twitter as a data mining source
Twitter  as  a data mining sourceTwitter  as  a data mining source
Twitter as a data mining source
 
Social media data for Social science research
Social media data for Social science researchSocial media data for Social science research
Social media data for Social science research
 
PPT FOR BIG
PPT FOR BIGPPT FOR BIG
PPT FOR BIG
 
Data Mining on Twitter
Data Mining on TwitterData Mining on Twitter
Data Mining on Twitter
 
Apache Flume and its use case in Manufacturing
Apache Flume and its use case in ManufacturingApache Flume and its use case in Manufacturing
Apache Flume and its use case in Manufacturing
 
Analyse Tweets using Flume, Hadoop and Hive
Analyse Tweets using Flume, Hadoop and HiveAnalyse Tweets using Flume, Hadoop and Hive
Analyse Tweets using Flume, Hadoop and Hive
 
Twitter Text Mining with Web scraping, R, Shiny and Hadoop - Richard Sheng
Twitter Text Mining with Web scraping, R, Shiny and Hadoop - Richard Sheng Twitter Text Mining with Web scraping, R, Shiny and Hadoop - Richard Sheng
Twitter Text Mining with Web scraping, R, Shiny and Hadoop - Richard Sheng
 
Challenges of social media analysis in the real world
Challenges of social media analysis in the real worldChallenges of social media analysis in the real world
Challenges of social media analysis in the real world
 
A data analyst view of Bigdata
A data analyst view of Bigdata A data analyst view of Bigdata
A data analyst view of Bigdata
 
Social media mining PPT
Social media mining PPTSocial media mining PPT
Social media mining PPT
 
Hadoop, Pig, and Twitter (NoSQL East 2009)
Hadoop, Pig, and Twitter (NoSQL East 2009)Hadoop, Pig, and Twitter (NoSQL East 2009)
Hadoop, Pig, and Twitter (NoSQL East 2009)
 
Sentiment Analysis of Twitter Data
Sentiment Analysis of Twitter DataSentiment Analysis of Twitter Data
Sentiment Analysis of Twitter Data
 

Similar a DataSift Update - May 3rd 2011 - Devnest

2012 09 Oracle bigdata_architecture
2012 09 Oracle bigdata_architecture2012 09 Oracle bigdata_architecture
2012 09 Oracle bigdata_architecturejdijcks
 
BI Forum 2012 - Integrované systémy
BI Forum 2012 - Integrované systémyBI Forum 2012 - Integrované systémy
BI Forum 2012 - Integrované systémyOKsystem
 
Business: Security & Privacy
Business: Security & PrivacyBusiness: Security & Privacy
Business: Security & PrivacyJeremy Hilton
 
MphasiS - insurance and technology
MphasiS -  insurance and technologyMphasiS -  insurance and technology
MphasiS - insurance and technologyMphasis
 
Cup Moscow >> Eng >> 1st Place >> Sorrento (1st Round Case)
Cup Moscow >> Eng >> 1st Place >> Sorrento (1st Round Case)Cup Moscow >> Eng >> 1st Place >> Sorrento (1st Round Case)
Cup Moscow >> Eng >> 1st Place >> Sorrento (1st Round Case)Changellenge >> Capital
 
Global Pharmaceuticals & Biotechnology Industry
Global Pharmaceuticals & Biotechnology IndustryGlobal Pharmaceuticals & Biotechnology Industry
Global Pharmaceuticals & Biotechnology IndustryGavin Pathross
 
M Life 2009 Modern Government And Mobility Eduardo Fernandez Rojo
M Life 2009 Modern Government And Mobility   Eduardo Fernandez RojoM Life 2009 Modern Government And Mobility   Eduardo Fernandez Rojo
M Life 2009 Modern Government And Mobility Eduardo Fernandez RojoEd Fernandez
 
SOA and Cloud in Life Sciences
SOA and Cloud in Life SciencesSOA and Cloud in Life Sciences
SOA and Cloud in Life SciencesSandeep Bhat
 
Global Services Jan 2012
Global Services Jan 2012Global Services Jan 2012
Global Services Jan 2012ajfitzer
 
Kantar health china credential deck 20110621
Kantar health china credential deck 20110621Kantar health china credential deck 20110621
Kantar health china credential deck 20110621simonlxg
 
George W. Buckley Chairman, President and Chief Executive Officer
George W. Buckley Chairman, President and Chief Executive OfficerGeorge W. Buckley Chairman, President and Chief Executive Officer
George W. Buckley Chairman, President and Chief Executive Officerfinance10
 
Admonsters OPS Mobile Keynote Presentation
Admonsters OPS Mobile Keynote PresentationAdmonsters OPS Mobile Keynote Presentation
Admonsters OPS Mobile Keynote PresentationPaul Gelb
 
BPC Overview
BPC OverviewBPC Overview
BPC Overviewevolfi
 
ADMA Macquarie Campaign Measurement
ADMA Macquarie Campaign MeasurementADMA Macquarie Campaign Measurement
ADMA Macquarie Campaign MeasurementDatalicious
 

Similar a DataSift Update - May 3rd 2011 - Devnest (20)

2012 09 Oracle bigdata_architecture
2012 09 Oracle bigdata_architecture2012 09 Oracle bigdata_architecture
2012 09 Oracle bigdata_architecture
 
Ict’s in value chains v1
Ict’s in value chains v1Ict’s in value chains v1
Ict’s in value chains v1
 
BI Forum 2012 - Integrované systémy
BI Forum 2012 - Integrované systémyBI Forum 2012 - Integrované systémy
BI Forum 2012 - Integrované systémy
 
Business: Security & Privacy
Business: Security & PrivacyBusiness: Security & Privacy
Business: Security & Privacy
 
MphasiS - insurance and technology
MphasiS -  insurance and technologyMphasiS -  insurance and technology
MphasiS - insurance and technology
 
Cup Moscow >> Eng >> 1st Place >> Sorrento (1st Round Case)
Cup Moscow >> Eng >> 1st Place >> Sorrento (1st Round Case)Cup Moscow >> Eng >> 1st Place >> Sorrento (1st Round Case)
Cup Moscow >> Eng >> 1st Place >> Sorrento (1st Round Case)
 
Greenplum hadoop
Greenplum hadoopGreenplum hadoop
Greenplum hadoop
 
Greenplum hadoop
Greenplum hadoopGreenplum hadoop
Greenplum hadoop
 
Global Pharmaceuticals & Biotechnology Industry
Global Pharmaceuticals & Biotechnology IndustryGlobal Pharmaceuticals & Biotechnology Industry
Global Pharmaceuticals & Biotechnology Industry
 
M Life 2009 Modern Government And Mobility Eduardo Fernandez Rojo
M Life 2009 Modern Government And Mobility   Eduardo Fernandez RojoM Life 2009 Modern Government And Mobility   Eduardo Fernandez Rojo
M Life 2009 Modern Government And Mobility Eduardo Fernandez Rojo
 
Hacking Trust
Hacking TrustHacking Trust
Hacking Trust
 
SOA and Cloud in Life Sciences
SOA and Cloud in Life SciencesSOA and Cloud in Life Sciences
SOA and Cloud in Life Sciences
 
M3 En 03 2009 Founders Profile
M3 En 03 2009 Founders ProfileM3 En 03 2009 Founders Profile
M3 En 03 2009 Founders Profile
 
Global Services Jan 2012
Global Services Jan 2012Global Services Jan 2012
Global Services Jan 2012
 
Kantar health china credential deck 20110621
Kantar health china credential deck 20110621Kantar health china credential deck 20110621
Kantar health china credential deck 20110621
 
George W. Buckley Chairman, President and Chief Executive Officer
George W. Buckley Chairman, President and Chief Executive OfficerGeorge W. Buckley Chairman, President and Chief Executive Officer
George W. Buckley Chairman, President and Chief Executive Officer
 
Noronesc
NoronescNoronesc
Noronesc
 
Admonsters OPS Mobile Keynote Presentation
Admonsters OPS Mobile Keynote PresentationAdmonsters OPS Mobile Keynote Presentation
Admonsters OPS Mobile Keynote Presentation
 
BPC Overview
BPC OverviewBPC Overview
BPC Overview
 
ADMA Macquarie Campaign Measurement
ADMA Macquarie Campaign MeasurementADMA Macquarie Campaign Measurement
ADMA Macquarie Campaign Measurement
 

Último

08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking MenDelhi Call girls
 
GenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationGenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationMichael W. Hawkins
 
Artificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsArtificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsJoaquim Jorge
 
Presentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreterPresentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreternaman860154
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)Gabriella Davis
 
What Are The Drone Anti-jamming Systems Technology?
What Are The Drone Anti-jamming Systems Technology?What Are The Drone Anti-jamming Systems Technology?
What Are The Drone Anti-jamming Systems Technology?Antenna Manufacturer Coco
 
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Igalia
 
The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024Rafal Los
 
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationRadu Cotescu
 
[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdfhans926745
 
Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slidevu2urc
 
08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking MenDelhi Call girls
 
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdfThe Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdfEnterprise Knowledge
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonAnna Loughnan Colquhoun
 
The Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxThe Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxMalak Abu Hammad
 
Breaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountBreaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountPuma Security, LLC
 
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptxEIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptxEarley Information Science
 
Boost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivityBoost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivityPrincipled Technologies
 
Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)wesley chun
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc
 

Último (20)

08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
 
GenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationGenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day Presentation
 
Artificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsArtificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and Myths
 
Presentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreterPresentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreter
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)
 
What Are The Drone Anti-jamming Systems Technology?
What Are The Drone Anti-jamming Systems Technology?What Are The Drone Anti-jamming Systems Technology?
What Are The Drone Anti-jamming Systems Technology?
 
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
 
The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024
 
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organization
 
[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf
 
Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slide
 
08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men
 
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdfThe Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt Robison
 
The Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxThe Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptx
 
Breaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountBreaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path Mount
 
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptxEIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
 
Boost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivityBoost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivity
 
Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
 

DataSift Update - May 3rd 2011 - Devnest