Using Topic Models for Twitter hashtag recommendation

•Download as PPTX, PDF•

4 likes•2,730 views

fgodin

Presentation given at the Making Sense of Micropost Worksop at the World Wide Web conference of 2013

Technology

ELIS – Multimedia Lab
Fréderic Godin, Viktor Slavkovikj, Wesley De
Neve, Benjamin Schrauwen and Rik Van de Walle
Using Topic Models for
Twitter Hashtag Recommendation
Multimedia Lab, Ghent University – iMinds, Belgium
Reservoir Lab, Ghent University, Belgium
Image and Video Systems Lab, KAIST, South Korea

2
ELIS – Multimedia Lab
Using Topic Models for Twitter Hashtag Recommendation
Fréderic Godin, Viktor Slavkovikj, Wesley De Neve, Benjamin Schrauwen and Rik Van de Walle
Making Sense of Microposts Workshop @ World Wide Web Conference 2013
Introduction (1)
Indexing
Search
Linking
General Topic
Memes Grouping
Information retrieval

3
ELIS – Multimedia Lab
Using Topic Models for Twitter Hashtag Recommendation
Fréderic Godin, Viktor Slavkovikj, Wesley De Neve, Benjamin Schrauwen and Rik Van de Walle
Making Sense of Microposts Workshop @ World Wide Web Conference 2013
Introduction (2)
±10% of tweets contain a hashtag
3% of the hashtags are used more than 5 times
Indexing
Search
Linking
General Topic
Memes
Grouping

4
ELIS – Multimedia Lab
Using Topic Models for Twitter Hashtag Recommendation
Fréderic Godin, Viktor Slavkovikj, Wesley De Neve, Benjamin Schrauwen and Rik Van de Walle
Making Sense of Microposts Workshop @ World Wide Web Conference 2013
Goal
Suggest keywords that resemble the general topic of a tweet
and that could be used as a hashtag
Promote hashtags for effective indexing
Allow for effective search of tweets through hashtags
Reduce the use of sparse hashtags

5
ELIS – Multimedia Lab
Using Topic Models for Twitter Hashtag Recommendation
Fréderic Godin, Viktor Slavkovikj, Wesley De Neve, Benjamin Schrauwen and Rik Van de Walle
Making Sense of Microposts Workshop @ World Wide Web Conference 2013
Architectural overview
Basic filterTweet
Language
identification
Topic
distribution
Hashtag
suggestion
Hashtagged
tweet

6
ELIS – Multimedia Lab
Using Topic Models for Twitter Hashtag Recommendation
Fréderic Godin, Viktor Slavkovikj, Wesley De Neve, Benjamin Schrauwen and Rik Van de Walle
Making Sense of Microposts Workshop @ World Wide Web Conference 2013
Basic filter
Clean up the tweet: URLs, special HTML entities, digits,
punctuations, the hash character, …
During training:
Remove tweets with just one word
Remove retweets

7
ELIS – Multimedia Lab
Using Topic Models for Twitter Hashtag Recommendation
Fréderic Godin, Viktor Slavkovikj, Wesley De Neve, Benjamin Schrauwen and Rik Van de Walle
Making Sense of Microposts Workshop @ World Wide Web Conference 2013
Language identification
Why We need to build a language-dependent topic model.
Goal Build unsupervised classifier that discriminates between
English and non-English tweets.
How Using Naive Bayes and the Expectation-Maximization
algorithm + character n-gram features
Result Evaluation on a test set of 1000 randomly selected tweets
Lui & Baldwin (LangID.py) Our algorithm
Precision 97.9% 97.0%
Recall 91.8% 97.8%
F1 94.8% 97.4%

8
ELIS – Multimedia Lab
Using Topic Models for Twitter Hashtag Recommendation
Fréderic Godin, Viktor Slavkovikj, Wesley De Neve, Benjamin Schrauwen and Rik Van de Walle
Making Sense of Microposts Workshop @ World Wide Web Conference 2013
Calculating the topic distribution
Idea Find the general topic(s) of a tweet
How Using Latent Dirichlet Allocation to find
the topic distribution in an unsupervised manner
Training 1.8 million tweets pre-filtered on 4000 keywords
200 topics, α=0.1, β=0.1
Example “Please RT!! sign Bernie Sanders petition for the
fiscal cliff! http://..”
0 1 2 3 57 199
[0.1; 0.0 ; 0.0 ; 0.0 ; … ; 0.8 ; … ; 0.05]
Topic 57:
1. Fiscal
2. Political
3. President
…

9
ELIS – Multimedia Lab
Using Topic Models for Twitter Hashtag Recommendation
Fréderic Godin, Viktor Slavkovikj, Wesley De Neve, Benjamin Schrauwen and Rik Van de Walle
Making Sense of Microposts Workshop @ World Wide Web Conference 2013
Hashtag suggestion (1)
Idea Suggest a number of hashtags based on
the topic distribution of the tweet
How Sample the topic distribution and suggest
the top ranked keywords
Yay, we got sixth period today school business light time period
Please RT!! Sign Bernie Sanders
petition for the fiscall! Http://..
fiscal political traffic president policy
comfort, elegance, prettiness little good love relationship god
Example

10
ELIS – Multimedia Lab
Using Topic Models for Twitter Hashtag Recommendation
Fréderic Godin, Viktor Slavkovikj, Wesley De Neve, Benjamin Schrauwen and Rik Van de Walle
Making Sense of Microposts Workshop @ World Wide Web Conference 2013
Hashtag suggestion (2)
0
5
10
15
20
25
30
35
0 1 2 3 4 5 6 7 8 9 10
Percentageoftweets(%)
Number of correctly suggested hashtags
5 hashtags
10 hashtags
Evaluation of 100 tweets

11
ELIS – Multimedia Lab
Using Topic Models for Twitter Hashtag Recommendation
Fréderic Godin, Viktor Slavkovikj, Wesley De Neve, Benjamin Schrauwen and Rik Van de Walle
Making Sense of Microposts Workshop @ World Wide Web Conference 2013
Conclusions and Future Work
We built a hashtag recommendation system:
Suggests general keywords
Unsupervised
In the future:
Use more context information: semantic web,
social graph,…
Adopt a hybrid approach between general and specific
hashtags

12
ELIS – Multimedia Lab
Using Topic Models for Twitter Hashtag Recommendation
Fréderic Godin, Viktor Slavkovikj, Wesley De Neve, Benjamin Schrauwen and Rik Van de Walle
Making Sense of Microposts Workshop @ World Wide Web Conference 2013
#Questions @frederic_godin

The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf

Enterprise Knowledge

Building Digital Trust in a Digital Economy Veronica Tan, Director - Cyber Security Agency of Singapore Apidays Singapore 2024: Connecting Customers, Business and Technology (April 17 & 18, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...

apidays

CNv6 Instructor Chapter 6 Quality of Service

giselly40

Advantages of Hiring UIUX Design Service Providers for Your Business

Pixlogix Infotech

The 7 Things I Know About Cyber Security After 25 Years | April 2024

Rafal Los

Handwritten Text Recognition for manuscripts and early printed texts

Maria Levchenko

Slack Application Development 101 Slides

praypatel2

Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024

The Digital Insurer

Sara Mae O’Brien Scott and Tatiana Baquero Cakici, Senior Consultants at Enterprise Knowledge (EK), presented “AI Fast Track to Search-Focused AI Solutions” at the Information Architecture Conference (IAC24) that took place on April 11, 2024 in Seattle, WA. In their presentation, O’Brien-Scott and Cakici focused on what Enterprise AI is, why it is important, and what it takes to empower organizations to get started on a search-based AI journey and stay on track. The presentation explored the complexities of enterprise search challenges and how IA principles can be leveraged to provide AI solutions through the use of a semantic layer. O’Brien-Scott and Cakici showcased a case study where a taxonomy, an ontology, and a knowledge graph were used to structure content at a healthcare workforce solutions organization, providing personalized content recommendations and increasing content findability. In this session, participants gained insights about the following: Most common types of AI categories and use cases; Recommended steps to design and implement taxonomies and ontologies, ensuring they evolve effectively and support the organization’s search objectives; Taxonomy and ontology design considerations and best practices; Real-world AI applications that illustrated the value of taxonomies, ontologies, and knowledge graphs; and Tools, roles, and skills to design and implement AI-powered search solutions.

IAC 2024 - IA Fast Track to Search Focused AI Solutions

Enterprise Knowledge

Histor y of HAM Radio presentation slide

vu2urc

How to convert PDF to text with Nanonets

naman860154

Discord is a free app offering voice, video, and text chat functionalities, primarily catering to the gaming community. It serves as a hub for users to create and join servers tailored to their interests. Discord’s ecosystem comprises servers, each functioning as a distinct online community with its own channels dedicated to specific topics or activities. Users can engage in text-based discussions, voice calls, or video chats within these channels. Understanding Discord Servers Discord servers are virtual spaces where users congregate to interact, share content, and build communities. Servers may revolve around gaming, hobbies, interests, or fandoms, providing a platform for like-minded individuals to connect. Communication Features Discord offers a range of communication tools, including text channels for messaging, voice channels for real-time audio conversations, and video channels for face-to-face interactions. These features facilitate seamless communication and collaboration. What Does NSFW Mean? The acronym NSFW stands for “Not Safe For Work,” indicating content that may be inappropriate for professional or public settings. NSFW Content NSFW content encompasses material that is sexually explicit, violent, or otherwise graphic in nature. It often includes nudity, profanity, or depictions of sensitive topics.

Understanding Discord NSFW Servers A Guide for Responsible Users.pdf

UK Journal

Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...

Neo4j

🐬 The future of MySQL is Postgres 🐘

RTylerCroy

How to Troubleshoot Apps for the Modern Connected Worker

ThousandEyes

BooK Now Call us at +918448380779 to hire a gorgeous and seductive call girl for sex. Take a Delhi Escort Service. The help of our escort agency is mostly meant for men who want sexual Indian Escorts In Delhi NCR. It should be noted that any impersonator will get 100 attention from our Young Girls Escorts in Delhi. They will assume the position of reliable allies. VIP Call Girl With Original Photos Book Tonight +918448380779 Our Cheap Price 1 Hour not available 2 Hours 5000 Full Night 8000 TAG: Call Girls in Delhi, Noida, Gurgaon, Ghaziabad, Connaught Place, Greater Kailash Delhi, Lajpat Nagar Delhi, Mayur Vihar Delhi, Chanakyapuri Delhi, New Friends Colony Delhi, Majnu Ka Tilla, Karol Bagh, Malviya Nagar, Saket, Khan Market, Noida Sector 18, Noida Sector 76, Noida Sector 51, Gurgaon Mg Road, Iffco Chowk Gurgaon, Rajiv Chowk Gurgaon All Delhi Ncr Free Home Deliver

08448380779 Call Girls In Friends Colony Women Seeking Men

Delhi Call girls

The presentation explores the development and application of artificial intelligence (AI) from its inception to its current status in the modern world. The term "artificial intelligence" was first coined by John McCarthy in 1956 to describe efforts to develop computer programs capable of performing tasks that typically require human intelligence. This concept was first introduced at a conference held at Dartmouth College, where programs demonstrated capabilities such as playing chess, proving theorems, and interpreting texts. In the early stages, Alan Turing contributed to the field by defining intelligence as the ability of a being to respond to certain questions intelligently, proposing what is now known as the Turing Test to evaluate the presence of intelligent behavior in machines. As the decades progressed, AI evolved significantly. The 1980s focused on machine learning, teaching computers to learn from data, leading to the development of models that could improve their performance based on their experiences. The 1990s and 2000s saw further advances in algorithms and computational power, which allowed for more sophisticated data analysis techniques, including data mining. By the 2010s, the proliferation of big data and the refinement of deep learning techniques enabled AI to become mainstream. Notable milestones included the success of Google's AlphaGo and advancements in autonomous vehicles by companies like Tesla and Waymo. A major theme of the presentation is the application of generative AI, which has been used for tasks such as natural language text generation, translation, and question answering. Generative AI uses large datasets to train models that can then produce new, coherent pieces of text or other media. The presentation also discusses the ethical implications and the need for regulation in AI, highlighting issues such as privacy, bias, and the potential for misuse. These concerns have prompted calls for comprehensive regulations to ensure the safe and equitable use of AI technologies. Artificial intelligence has also played a significant role in healthcare, particularly highlighted during the COVID-19 pandemic, where it was used in drug discovery, vaccine development, and analyzing the spread of the virus. The capabilities of AI in healthcare are vast, ranging from medical diagnostics to personalized medicine, demonstrating the technology's potential to revolutionize fields beyond just technical or consumer applications. In conclusion, AI continues to be a rapidly evolving field with significant implications for various aspects of society. The development from theoretical concepts to real-world applications illustrates both the potential benefits and the challenges that come with integrating advanced technologies into everyday life. The ongoing discussion about AI ethics and regulation underscores the importance of managing these technologies responsibly to maximize their their benefits while minimizing potential harms.

Artificial Intelligence: Facts and Myths

Joaquim Jorge

A Call to Action for Generative AI in 2024

Results

Created by Mozilla Research in 2012 and now part of Linux Foundation Europe, the Servo project is an experimental rendering engine written in Rust. It combines memory safety and concurrency to create an independent, modular, and embeddable rendering engine that adheres to web standards. Stewardship of Servo moved from Mozilla Research to the Linux Foundation in 2020, where its mission remains unchanged. After some slow years, in 2023 there has been renewed activity on the project, with a roadmap now focused on improving the engine’s CSS 2 conformance, exploring Android support, and making Servo a practical embeddable rendering engine. In this presentation, Rakhi Sharma reviews the status of the project, our recent developments in 2023, our collaboration with Tauri to make Servo an easy-to-use embeddable rendering engine, and our plans for the future to make Servo an alternative web rendering engine for the embedded devices industry. (c) Embedded Open Source Summit 2024 April 16-18, 2024 Seattle, Washington (US) https://events.linuxfoundation.org/embedded-open-source-summit/ https://ossna2024.sched.com/event/1aBNF/a-year-of-servo-reboot-where-are-we-now-rakhi-sharma-igalia

A Year of the Servo Reboot: Where Are We Now?

Igalia

Recently uploaded (20)

Factors to Consider When Choosing Accounts Payable Services Providers.pptx

The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf

Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...

CNv6 Instructor Chapter 6 Quality of Service

Advantages of Hiring UIUX Design Service Providers for Your Business

The 7 Things I Know About Cyber Security After 25 Years | April 2024

Handwritten Text Recognition for manuscripts and early printed texts

Slack Application Development 101 Slides

Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024

IAC 2024 - IA Fast Track to Search Focused AI Solutions

Histor y of HAM Radio presentation slide

How to convert PDF to text with Nanonets

Understanding Discord NSFW Servers A Guide for Responsible Users.pdf

Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...

🐬 The future of MySQL is Postgres 🐘

How to Troubleshoot Apps for the Modern Connected Worker

08448380779 Call Girls In Friends Colony Women Seeking Men

Artificial Intelligence: Facts and Myths

A Call to Action for Generative AI in 2024

A Year of the Servo Reboot: Where Are We Now?

Using Topic Models for Twitter hashtag recommendation

1. ELIS – Multimedia Lab Fréderic Godin, Viktor Slavkovikj, Wesley De Neve, Benjamin Schrauwen and Rik Van de Walle Using Topic Models for Twitter Hashtag Recommendation Multimedia Lab, Ghent University – iMinds, Belgium Reservoir Lab, Ghent University, Belgium Image and Video Systems Lab, KAIST, South Korea

2. 2 ELIS – Multimedia Lab Using Topic Models for Twitter Hashtag Recommendation Fréderic Godin, Viktor Slavkovikj, Wesley De Neve, Benjamin Schrauwen and Rik Van de Walle Making Sense of Microposts Workshop @ World Wide Web Conference 2013 Introduction (1) Indexing Search Linking General Topic Memes Grouping Information retrieval

3. 3 ELIS – Multimedia Lab Using Topic Models for Twitter Hashtag Recommendation Fréderic Godin, Viktor Slavkovikj, Wesley De Neve, Benjamin Schrauwen and Rik Van de Walle Making Sense of Microposts Workshop @ World Wide Web Conference 2013 Introduction (2) ±10% of tweets contain a hashtag 3% of the hashtags are used more than 5 times Indexing Search Linking General Topic Memes Grouping

4. 4 ELIS – Multimedia Lab Using Topic Models for Twitter Hashtag Recommendation Fréderic Godin, Viktor Slavkovikj, Wesley De Neve, Benjamin Schrauwen and Rik Van de Walle Making Sense of Microposts Workshop @ World Wide Web Conference 2013 Goal Suggest keywords that resemble the general topic of a tweet and that could be used as a hashtag Promote hashtags for effective indexing Allow for effective search of tweets through hashtags Reduce the use of sparse hashtags

5. 5 ELIS – Multimedia Lab Using Topic Models for Twitter Hashtag Recommendation Fréderic Godin, Viktor Slavkovikj, Wesley De Neve, Benjamin Schrauwen and Rik Van de Walle Making Sense of Microposts Workshop @ World Wide Web Conference 2013 Architectural overview Basic filterTweet Language identification Topic distribution Hashtag suggestion Hashtagged tweet

6. 6 ELIS – Multimedia Lab Using Topic Models for Twitter Hashtag Recommendation Fréderic Godin, Viktor Slavkovikj, Wesley De Neve, Benjamin Schrauwen and Rik Van de Walle Making Sense of Microposts Workshop @ World Wide Web Conference 2013 Basic filter Clean up the tweet: URLs, special HTML entities, digits, punctuations, the hash character, … During training: Remove tweets with just one word Remove retweets

7. 7 ELIS – Multimedia Lab Using Topic Models for Twitter Hashtag Recommendation Fréderic Godin, Viktor Slavkovikj, Wesley De Neve, Benjamin Schrauwen and Rik Van de Walle Making Sense of Microposts Workshop @ World Wide Web Conference 2013 Language identification Why We need to build a language-dependent topic model. Goal Build unsupervised classifier that discriminates between English and non-English tweets. How Using Naive Bayes and the Expectation-Maximization algorithm + character n-gram features Result Evaluation on a test set of 1000 randomly selected tweets Lui & Baldwin (LangID.py) Our algorithm Precision 97.9% 97.0% Recall 91.8% 97.8% F1 94.8% 97.4%

8. 8 ELIS – Multimedia Lab Using Topic Models for Twitter Hashtag Recommendation Fréderic Godin, Viktor Slavkovikj, Wesley De Neve, Benjamin Schrauwen and Rik Van de Walle Making Sense of Microposts Workshop @ World Wide Web Conference 2013 Calculating the topic distribution Idea Find the general topic(s) of a tweet How Using Latent Dirichlet Allocation to find the topic distribution in an unsupervised manner Training 1.8 million tweets pre-filtered on 4000 keywords 200 topics, α=0.1, β=0.1 Example “Please RT!! sign Bernie Sanders petition for the fiscal cliff! http://..” 0 1 2 3 57 199 [0.1; 0.0 ; 0.0 ; 0.0 ; … ; 0.8 ; … ; 0.05] Topic 57: 1. Fiscal 2. Political 3. President …

9. 9 ELIS – Multimedia Lab Using Topic Models for Twitter Hashtag Recommendation Fréderic Godin, Viktor Slavkovikj, Wesley De Neve, Benjamin Schrauwen and Rik Van de Walle Making Sense of Microposts Workshop @ World Wide Web Conference 2013 Hashtag suggestion (1) Idea Suggest a number of hashtags based on the topic distribution of the tweet How Sample the topic distribution and suggest the top ranked keywords Yay, we got sixth period today school business light time period Please RT!! Sign Bernie Sanders petition for the fiscall! Http://.. fiscal political traffic president policy comfort, elegance, prettiness little good love relationship god Example

10. 10 ELIS – Multimedia Lab Using Topic Models for Twitter Hashtag Recommendation Fréderic Godin, Viktor Slavkovikj, Wesley De Neve, Benjamin Schrauwen and Rik Van de Walle Making Sense of Microposts Workshop @ World Wide Web Conference 2013 Hashtag suggestion (2) 0 5 10 15 20 25 30 35 0 1 2 3 4 5 6 7 8 9 10 Percentageoftweets(%) Number of correctly suggested hashtags 5 hashtags 10 hashtags Evaluation of 100 tweets

11. 11 ELIS – Multimedia Lab Using Topic Models for Twitter Hashtag Recommendation Fréderic Godin, Viktor Slavkovikj, Wesley De Neve, Benjamin Schrauwen and Rik Van de Walle Making Sense of Microposts Workshop @ World Wide Web Conference 2013 Conclusions and Future Work We built a hashtag recommendation system: Suggests general keywords Unsupervised In the future: Use more context information: semantic web, social graph,… Adopt a hybrid approach between general and specific hashtags

12. 12 ELIS – Multimedia Lab Using Topic Models for Twitter Hashtag Recommendation Fréderic Godin, Viktor Slavkovikj, Wesley De Neve, Benjamin Schrauwen and Rik Van de Walle Making Sense of Microposts Workshop @ World Wide Web Conference 2013 #Questions @frederic_godin

Editor's Notes

Footer: Micropost -> Microposts
… and that could be used …Allow for effective search of tweets (through hashtags)
Remove the full stopsLanguage dependent -> Language-dependentWhy? -> Why (for reasons of consistency)
Those 4000 keywords are used to getsomemeaningfultweets. Otherwise the set was to big for training the algorithm. Ifyoutake a smaller sample than 4 days, thenagainyou of to few coherent tweets to train the model. Thosekeywordsdon’tbecome the most important keywordwithin a topic. Ex. Keyword president. The topic was fiscalcliff and politicalproblems.
Misschienverduidelijken hoe je de verdeling van de topics bemonsterd?Op de vorige slide misschienookverduidelijken hoe je de topics hebtgeselecteerd?
an hashtag -> a hashtagsocial graph -> social graph, …To suggest general keywords -> Suggests general keywordsFuture work: anderetechniekenom topics tebepalen? Bayesian inference, deep learning, … ;-)?

Using Topic Models for Twitter hashtag recommendation

Recommended

Recommended

More Related Content

Similar to Using Topic Models for Twitter hashtag recommendation

Similar to Using Topic Models for Twitter hashtag recommendation (20)

More from fgodin

More from fgodin (7)

Recently uploaded

Recently uploaded (20)

Using Topic Models for Twitter hashtag recommendation

Editor's Notes