Se ha denunciado esta presentación.
Utilizamos tu perfil de LinkedIn y tus datos de actividad para personalizar los anuncios y mostrarte publicidad más relevante. Puedes cambiar tus preferencias de publicidad en cualquier momento.

Alphago vs Lee Se-Dol : Tweeter Analysis using Hadoop and Spark

334 visualizaciones

Publicado el

Tweeter data analysis in Alphago using Big Data Hadoop Spark and DashDB MPP warehouse.

Publicado en: Ingeniería
  • Inicia sesión para ver los comentarios

  • Sé el primero en recomendar esto

Alphago vs Lee Se-Dol : Tweeter Analysis using Hadoop and Spark

  1. 1. Alphago vs Lee Se-Dol Tweeter Analysis using Hadoop and Spark March 18 2016 Jongwook Woo, PhD
  2. 2. Content Hadoop and Spark IBM DashDB Conclusion
  3. 3. Hadoop and Spark Environment  Systems Azure HDInsights Spark 8 Nodes  40 cores: 2.4GHz Intel Xeon  Memory - Each Node: 28 GB  Data Source Keyword ‘alphago’ from Tweeter via Apache NiFi  Data Size  63,193 tweets  Real Time Data Collection period 03/12 – 03/17/2016  No data collected on 03/13
  4. 4. Top 10 Countries that Tweets “Alphago” Positive Negative
  5. 5. Top 10 Countries  # of Tweets per Country USA: > 11,000 Japan: > 9,000 Korea: > 1,900 Russia, UK: > 1,600 Thai Land, France : > 1,000  Netherland, Spain, Ukraine: > 600
  6. 6. Top 10 Countries Sentiment Positive Negative
  7. 7. Top 10 Countries Most Tweeted Countries  All countries show more positive tweets  Korea, Japan, USA Country Positive Negative USA 5070 3567 Japan 8118 217 … Korea 1053 407 …
  8. 8. Daily Tweets in 03/12 – 03/17/2016 0 5000 10000 15000 20000 25000 30000 35000 40000 45000 50000 3/12/2016 3/13/2016 3/14/2016 3/15/2016 3/16/2016 3/17/2016 Alphago vs Lee Sedol Game 4: Mar 13 Lee Se-Dol win Game 5: Mar 15 Game 3: Mar 12
  9. 9. Ngram words  3 word in row right after Go-Champion “sedol” and “se- dol” sedol  se-dol 3-grams Frequency Again-to-win 1,187 Is-something-I’ll 369 Is-something-i 199 In-go-tournament 168
  10. 10. Sentiment Map of Alphago Positive Negative
  11. 11. Sentiment Map of Lee Se-Dol vs Alphago YouTube video: the sentiment of the World https://youtu.be/vAzdnj4fkOg?list=PLaEg1tCLuW0BYLqVS5RTb ToiB8wQ2w14a
  12. 12. Tweeter Analysis using IBM DashDB  Environment: DashDB and Tweets Services of IBM Bluemix  Load existing data  Period: by March 16 2016  Authors and Followers of the Tweets
  13. 13. Top 10 Tweet Countries  With Hashtag “#Alphago” United States: >10,000 Japan: >8,000 Korea: >1,800
  14. 14. Hashtags Frequency
  15. 15. Sentiment at #Alphago
  16. 16. Gender Counts Who Tweets female male unknown Unknown
  17. 17. Tweets counts per months 0 2000 4000 6000 8000 10000 12000 Aug-2014 Feb-2015 Feb-2016 Jan-2015 Jan-2016 Mar-2016 Tweets counts per months
  18. 18. Daily Tweets During Games 0 500 1000 1500 2000 2500 3000 3500 3/9/2016 3/10/2016 3/11/2016 3/12/2016 3/13/2016 3/14/2016 3/15/2016 3/16/2016 Daily Tweets during Games Game 4: Mar 13 Lee Se-Dol win Game 5: Mar 15 Game 3: Mar 12 Game 1: Mar 9 Game 2: Mar 10
  19. 19. Conclustion  Analyze Tweeters with “Alphago”  USA and Japan dominates the tweets More than Korea European countries as well  More Positive tweets Alphago and Lee Sedol both become popular

×