SlideShare una empresa de Scribd logo
1 de 32
OLLIE PARSLEY

@OLLIEPARSLEY
REALTIME TWITTER DATA MINING


                               @DATASIFT
MINING THE TWEET




170 MILLION (daily)
MINING THE TWEET




170 MILLION (daily)
MINING THE TWEET




                                         REAL-TIME OUTPUT


170 MILLION (daily)
DATA FILTERING
CSDL - Curation Stream Definition Language
AUGMENTATIONS



KLOUT                PEERINDEX            QWERLY




        LEXALYTICS               TRENDS
NEW FEATURES


BIGSTORE

STREAMTAGS
BIGSTORE
BIGSTORE



                                 REAL-TIME OUTPUT




170 MILLION (daily)
BIGSTORE



                                 REAL-TIME OUTPUT




170 MILLION (daily)                  BIGSTORE
BIGSTORE




HBASE
BIGSTORE




HBASE   POST-PROCESSING
BIGSTORE




HBASE   POST-PROCESSING   MAP-REDUCE
BIGSTORE




HBASE   POST-PROCESSING   MAP-REDUCE   EXPORTING
STREAMTAGS
STREAMTAGS
STREAMTAGS
STREAMTAGS
WHAT CAN YOU BUILD?
@STOCKSIFT
@STOCKSIFT
@STOCKSIFT
@CLUSTERAPP
@CLUSTERAPP
@CLUSTERAPP
USE CASES / VERTICALS
• Below	
  are	
  some	
  of	
  the	
  use	
  cases	
  from	
  the	
  Alpha	
  programme.	
  
  This	
  is	
  a	
  con7nuously	
  evolving	
  set	
  of	
  applica7ons,	
  based	
  upon	
  market	
  feedback.         Mining
                                                                                                                          Oil & Gas
                                                                                                                          Exploration
                                                                                                     Sports               Utilities

                                                                                                     Energy               Automobiles
                                                                    Crisis Management
                                                                                                     Industrials          Aerospace / Defence
                                                                                                                          Basic Resources
                                          Events / Trigger          Reputation Management            Banking
                                                                                                                          Chemicals
                                                                                                     Financial Services   Construction
                       Monitoring                                   Risk Analysis
                                         Data Analysis                                               Insurance            Industrial Goods
                                                                     Research
                                                                                                     Property
                                          Brand Marketing             Viral Campaigns                                     Airlines
                     Influencing
                                                                                                     Media                Shipping
                                                          Qualitative Feedback                       Transportation       Rail
                                          CRM
                                                                                                     Health               Pharmaceuticals
                                                         Augmented Research
                                                                                                                          Healthcare
                                                                                                     Technology

                                                                                                      Government          Telecommunications

                                                                                                     Retail / Consumer    F&B
                                                                                                                          Personal Goods
                                                                                                                          Travel & Leisure
WHERE'S MY INVITE??
WHERE'S MY INVITE??


1) Request an invite from datasift.net

2) Email ollie@mediasift.com

3) Enjoy!
WE'RE HIRING!
WE'RE HIRING!


- Various posts: Developers/Operations/Sales

- Perks: Free copy of StarCraft II and brand new office

- Interested? Come talk to me!
ANY QUESTIONS?

Más contenido relacionado

Destacado

Big Data at Twitter, Chirp 2010
Big Data at Twitter, Chirp 2010Big Data at Twitter, Chirp 2010
Big Data at Twitter, Chirp 2010Kevin Weil
 
Twitter as a data mining source
Twitter  as  a data mining sourceTwitter  as  a data mining source
Twitter as a data mining sourceAtaxo Group
 
Social media data for Social science research
Social media data for Social science researchSocial media data for Social science research
Social media data for Social science researchDavide Bennato
 
Data Mining on Twitter
Data Mining on TwitterData Mining on Twitter
Data Mining on TwitterPulkit Goyal
 
Apache Flume and its use case in Manufacturing
Apache Flume and its use case in ManufacturingApache Flume and its use case in Manufacturing
Apache Flume and its use case in ManufacturingRapheephan Thongkham-Uan
 
Analyse Tweets using Flume, Hadoop and Hive
Analyse Tweets using Flume, Hadoop and HiveAnalyse Tweets using Flume, Hadoop and Hive
Analyse Tweets using Flume, Hadoop and HiveIMC Institute
 
Twitter Text Mining with Web scraping, R, Shiny and Hadoop - Richard Sheng
Twitter Text Mining with Web scraping, R, Shiny and Hadoop - Richard Sheng Twitter Text Mining with Web scraping, R, Shiny and Hadoop - Richard Sheng
Twitter Text Mining with Web scraping, R, Shiny and Hadoop - Richard Sheng Richard Sheng
 
Challenges of social media analysis in the real world
Challenges of social media analysis in the real worldChallenges of social media analysis in the real world
Challenges of social media analysis in the real worldDiana Maynard
 
Social media mining PPT
Social media mining PPTSocial media mining PPT
Social media mining PPTChhavi Mathur
 
Hadoop, Pig, and Twitter (NoSQL East 2009)
Hadoop, Pig, and Twitter (NoSQL East 2009)Hadoop, Pig, and Twitter (NoSQL East 2009)
Hadoop, Pig, and Twitter (NoSQL East 2009)Kevin Weil
 
Sentiment Analysis of Twitter Data
Sentiment Analysis of Twitter DataSentiment Analysis of Twitter Data
Sentiment Analysis of Twitter DataSumit Raj
 

Destacado (13)

Big Data at Twitter, Chirp 2010
Big Data at Twitter, Chirp 2010Big Data at Twitter, Chirp 2010
Big Data at Twitter, Chirp 2010
 
Twitter as a data mining source
Twitter  as  a data mining sourceTwitter  as  a data mining source
Twitter as a data mining source
 
Social media data for Social science research
Social media data for Social science researchSocial media data for Social science research
Social media data for Social science research
 
PPT FOR BIG
PPT FOR BIGPPT FOR BIG
PPT FOR BIG
 
Data Mining on Twitter
Data Mining on TwitterData Mining on Twitter
Data Mining on Twitter
 
Apache Flume and its use case in Manufacturing
Apache Flume and its use case in ManufacturingApache Flume and its use case in Manufacturing
Apache Flume and its use case in Manufacturing
 
Analyse Tweets using Flume, Hadoop and Hive
Analyse Tweets using Flume, Hadoop and HiveAnalyse Tweets using Flume, Hadoop and Hive
Analyse Tweets using Flume, Hadoop and Hive
 
Twitter Text Mining with Web scraping, R, Shiny and Hadoop - Richard Sheng
Twitter Text Mining with Web scraping, R, Shiny and Hadoop - Richard Sheng Twitter Text Mining with Web scraping, R, Shiny and Hadoop - Richard Sheng
Twitter Text Mining with Web scraping, R, Shiny and Hadoop - Richard Sheng
 
Challenges of social media analysis in the real world
Challenges of social media analysis in the real worldChallenges of social media analysis in the real world
Challenges of social media analysis in the real world
 
A data analyst view of Bigdata
A data analyst view of Bigdata A data analyst view of Bigdata
A data analyst view of Bigdata
 
Social media mining PPT
Social media mining PPTSocial media mining PPT
Social media mining PPT
 
Hadoop, Pig, and Twitter (NoSQL East 2009)
Hadoop, Pig, and Twitter (NoSQL East 2009)Hadoop, Pig, and Twitter (NoSQL East 2009)
Hadoop, Pig, and Twitter (NoSQL East 2009)
 
Sentiment Analysis of Twitter Data
Sentiment Analysis of Twitter DataSentiment Analysis of Twitter Data
Sentiment Analysis of Twitter Data
 

Similar a DataSift Update - May 3rd 2011 - Devnest

2012 09 Oracle bigdata_architecture
2012 09 Oracle bigdata_architecture2012 09 Oracle bigdata_architecture
2012 09 Oracle bigdata_architecturejdijcks
 
BI Forum 2012 - Integrované systémy
BI Forum 2012 - Integrované systémyBI Forum 2012 - Integrované systémy
BI Forum 2012 - Integrované systémyOKsystem
 
Business: Security & Privacy
Business: Security & PrivacyBusiness: Security & Privacy
Business: Security & PrivacyJeremy Hilton
 
MphasiS - insurance and technology
MphasiS -  insurance and technologyMphasiS -  insurance and technology
MphasiS - insurance and technologyMphasis
 
Cup Moscow >> Eng >> 1st Place >> Sorrento (1st Round Case)
Cup Moscow >> Eng >> 1st Place >> Sorrento (1st Round Case)Cup Moscow >> Eng >> 1st Place >> Sorrento (1st Round Case)
Cup Moscow >> Eng >> 1st Place >> Sorrento (1st Round Case)Changellenge >> Capital
 
Global Pharmaceuticals & Biotechnology Industry
Global Pharmaceuticals & Biotechnology IndustryGlobal Pharmaceuticals & Biotechnology Industry
Global Pharmaceuticals & Biotechnology IndustryGavin Pathross
 
M Life 2009 Modern Government And Mobility Eduardo Fernandez Rojo
M Life 2009 Modern Government And Mobility   Eduardo Fernandez RojoM Life 2009 Modern Government And Mobility   Eduardo Fernandez Rojo
M Life 2009 Modern Government And Mobility Eduardo Fernandez RojoEd Fernandez
 
SOA and Cloud in Life Sciences
SOA and Cloud in Life SciencesSOA and Cloud in Life Sciences
SOA and Cloud in Life SciencesSandeep Bhat
 
Global Services Jan 2012
Global Services Jan 2012Global Services Jan 2012
Global Services Jan 2012ajfitzer
 
Kantar health china credential deck 20110621
Kantar health china credential deck 20110621Kantar health china credential deck 20110621
Kantar health china credential deck 20110621simonlxg
 
George W. Buckley Chairman, President and Chief Executive Officer
George W. Buckley Chairman, President and Chief Executive OfficerGeorge W. Buckley Chairman, President and Chief Executive Officer
George W. Buckley Chairman, President and Chief Executive Officerfinance10
 
Admonsters OPS Mobile Keynote Presentation
Admonsters OPS Mobile Keynote PresentationAdmonsters OPS Mobile Keynote Presentation
Admonsters OPS Mobile Keynote PresentationPaul Gelb
 
BPC Overview
BPC OverviewBPC Overview
BPC Overviewevolfi
 
ADMA Macquarie Campaign Measurement
ADMA Macquarie Campaign MeasurementADMA Macquarie Campaign Measurement
ADMA Macquarie Campaign MeasurementDatalicious
 

Similar a DataSift Update - May 3rd 2011 - Devnest (20)

2012 09 Oracle bigdata_architecture
2012 09 Oracle bigdata_architecture2012 09 Oracle bigdata_architecture
2012 09 Oracle bigdata_architecture
 
Ict’s in value chains v1
Ict’s in value chains v1Ict’s in value chains v1
Ict’s in value chains v1
 
BI Forum 2012 - Integrované systémy
BI Forum 2012 - Integrované systémyBI Forum 2012 - Integrované systémy
BI Forum 2012 - Integrované systémy
 
Business: Security & Privacy
Business: Security & PrivacyBusiness: Security & Privacy
Business: Security & Privacy
 
MphasiS - insurance and technology
MphasiS -  insurance and technologyMphasiS -  insurance and technology
MphasiS - insurance and technology
 
Cup Moscow >> Eng >> 1st Place >> Sorrento (1st Round Case)
Cup Moscow >> Eng >> 1st Place >> Sorrento (1st Round Case)Cup Moscow >> Eng >> 1st Place >> Sorrento (1st Round Case)
Cup Moscow >> Eng >> 1st Place >> Sorrento (1st Round Case)
 
Greenplum hadoop
Greenplum hadoopGreenplum hadoop
Greenplum hadoop
 
Greenplum hadoop
Greenplum hadoopGreenplum hadoop
Greenplum hadoop
 
Global Pharmaceuticals & Biotechnology Industry
Global Pharmaceuticals & Biotechnology IndustryGlobal Pharmaceuticals & Biotechnology Industry
Global Pharmaceuticals & Biotechnology Industry
 
M Life 2009 Modern Government And Mobility Eduardo Fernandez Rojo
M Life 2009 Modern Government And Mobility   Eduardo Fernandez RojoM Life 2009 Modern Government And Mobility   Eduardo Fernandez Rojo
M Life 2009 Modern Government And Mobility Eduardo Fernandez Rojo
 
Hacking Trust
Hacking TrustHacking Trust
Hacking Trust
 
SOA and Cloud in Life Sciences
SOA and Cloud in Life SciencesSOA and Cloud in Life Sciences
SOA and Cloud in Life Sciences
 
M3 En 03 2009 Founders Profile
M3 En 03 2009 Founders ProfileM3 En 03 2009 Founders Profile
M3 En 03 2009 Founders Profile
 
Global Services Jan 2012
Global Services Jan 2012Global Services Jan 2012
Global Services Jan 2012
 
Kantar health china credential deck 20110621
Kantar health china credential deck 20110621Kantar health china credential deck 20110621
Kantar health china credential deck 20110621
 
George W. Buckley Chairman, President and Chief Executive Officer
George W. Buckley Chairman, President and Chief Executive OfficerGeorge W. Buckley Chairman, President and Chief Executive Officer
George W. Buckley Chairman, President and Chief Executive Officer
 
Noronesc
NoronescNoronesc
Noronesc
 
Admonsters OPS Mobile Keynote Presentation
Admonsters OPS Mobile Keynote PresentationAdmonsters OPS Mobile Keynote Presentation
Admonsters OPS Mobile Keynote Presentation
 
BPC Overview
BPC OverviewBPC Overview
BPC Overview
 
ADMA Macquarie Campaign Measurement
ADMA Macquarie Campaign MeasurementADMA Macquarie Campaign Measurement
ADMA Macquarie Campaign Measurement
 

Último

Story boards and shot lists for my a level piece
Story boards and shot lists for my a level pieceStory boards and shot lists for my a level piece
Story boards and shot lists for my a level piececharlottematthew16
 
"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr BaganFwdays
 
"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii SoldatenkoFwdays
 
What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024Stephanie Beckett
 
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Mark Simos
 
SAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptxSAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptxNavinnSomaal
 
Install Stable Diffusion in windows machine
Install Stable Diffusion in windows machineInstall Stable Diffusion in windows machine
Install Stable Diffusion in windows machinePadma Pradeep
 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada
 
SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024Lorenzo Miniero
 
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmaticsKotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmaticscarlostorres15106
 
The Future of Software Development - Devin AI Innovative Approach.pdf
The Future of Software Development - Devin AI Innovative Approach.pdfThe Future of Software Development - Devin AI Innovative Approach.pdf
The Future of Software Development - Devin AI Innovative Approach.pdfSeasiaInfotech2
 
Vector Databases 101 - An introduction to the world of Vector Databases
Vector Databases 101 - An introduction to the world of Vector DatabasesVector Databases 101 - An introduction to the world of Vector Databases
Vector Databases 101 - An introduction to the world of Vector DatabasesZilliz
 
Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!Manik S Magar
 
Human Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsHuman Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsMark Billinghurst
 
Unraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfUnraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfAlex Barbosa Coqueiro
 
Search Engine Optimization SEO PDF for 2024.pdf
Search Engine Optimization SEO PDF for 2024.pdfSearch Engine Optimization SEO PDF for 2024.pdf
Search Engine Optimization SEO PDF for 2024.pdfRankYa
 
Powerpoint exploring the locations used in television show Time Clash
Powerpoint exploring the locations used in television show Time ClashPowerpoint exploring the locations used in television show Time Clash
Powerpoint exploring the locations used in television show Time Clashcharlottematthew16
 
"Federated learning: out of reach no matter how close",Oleksandr Lapshyn
"Federated learning: out of reach no matter how close",Oleksandr Lapshyn"Federated learning: out of reach no matter how close",Oleksandr Lapshyn
"Federated learning: out of reach no matter how close",Oleksandr LapshynFwdays
 
Vertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering TipsVertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering TipsMiki Katsuragi
 

Último (20)

Story boards and shot lists for my a level piece
Story boards and shot lists for my a level pieceStory boards and shot lists for my a level piece
Story boards and shot lists for my a level piece
 
"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan
 
"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko
 
What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024
 
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
 
SAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptxSAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptx
 
Install Stable Diffusion in windows machine
Install Stable Diffusion in windows machineInstall Stable Diffusion in windows machine
Install Stable Diffusion in windows machine
 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
 
E-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptx
E-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptxE-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptx
E-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptx
 
SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024
 
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmaticsKotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
 
The Future of Software Development - Devin AI Innovative Approach.pdf
The Future of Software Development - Devin AI Innovative Approach.pdfThe Future of Software Development - Devin AI Innovative Approach.pdf
The Future of Software Development - Devin AI Innovative Approach.pdf
 
Vector Databases 101 - An introduction to the world of Vector Databases
Vector Databases 101 - An introduction to the world of Vector DatabasesVector Databases 101 - An introduction to the world of Vector Databases
Vector Databases 101 - An introduction to the world of Vector Databases
 
Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!
 
Human Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsHuman Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR Systems
 
Unraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfUnraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdf
 
Search Engine Optimization SEO PDF for 2024.pdf
Search Engine Optimization SEO PDF for 2024.pdfSearch Engine Optimization SEO PDF for 2024.pdf
Search Engine Optimization SEO PDF for 2024.pdf
 
Powerpoint exploring the locations used in television show Time Clash
Powerpoint exploring the locations used in television show Time ClashPowerpoint exploring the locations used in television show Time Clash
Powerpoint exploring the locations used in television show Time Clash
 
"Federated learning: out of reach no matter how close",Oleksandr Lapshyn
"Federated learning: out of reach no matter how close",Oleksandr Lapshyn"Federated learning: out of reach no matter how close",Oleksandr Lapshyn
"Federated learning: out of reach no matter how close",Oleksandr Lapshyn
 
Vertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering TipsVertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering Tips
 

DataSift Update - May 3rd 2011 - Devnest