Se ha denunciado esta presentación.
Utilizamos tu perfil de LinkedIn y tus datos de actividad para personalizar los anuncios y mostrarte publicidad más relevante. Puedes cambiar tus preferencias de publicidad en cualquier momento.

Justifying Recommendations through Aspect-based Sentiment Analysis of Users Reviews

425 visualizaciones

Publicado el

UMAP 2019 Presentation - Larnaca Cyprus - June 2019 - User Modelind Adaptation and Personalization Conference

Publicado en: Tecnología
  • Can you earn $7000 a month from home? Are you feeling trapped by your life? Stuck in a dead-end job you hate, but too scared to call it quits, because after all, the rent's due on the first of the month, right? Are you ready to change your life for the better? ★★★ http://scamcb.com/ezpayjobs/pdf
       Responder 
    ¿Estás seguro?    No
    Tu mensaje aparecerá aquí
  • MADE $30 ON MY FIRST DAY! Being a fresh graduate and having lots of free time, I stumbled upon your site when I was searching for work at home opportunities, good thing I did! Just on my first day of joining I already made $30! Now I'm averaging close to $80 a day just for filling out surveys! ■■■ http://t.cn/AieXAuZz
       Responder 
    ¿Estás seguro?    No
    Tu mensaje aparecerá aquí
  • accessibility Books Library allowing access to top content, including thousands of title from favorite author, plus the ability to read or download a huge selection of books for your pc or smartphone within minutes DOWNLOAD THIS BOOKS INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... Download Full PDF EBOOK here { https://urlzs.com/UABbn } ......................................................................................................................... Download Full EPUB Ebook here { https://urlzs.com/UABbn } ......................................................................................................................... ...................................ALL FOR EBOOKS................................................. Cookbooks, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy,
       Responder 
    ¿Estás seguro?    No
    Tu mensaje aparecerá aquí
  • How can I improve my memory before an exam? How can I improve my study skills? learn more... ●●● https://bit.ly/2GEWG9T
       Responder 
    ¿Estás seguro?    No
    Tu mensaje aparecerá aquí

Justifying Recommendations through Aspect-based Sentiment Analysis of Users Reviews

  1. 1. @cataldomusto cataldo.musto@uniba.it Justifying Recommendations through Aspect-based Sentiment Analysis of Users’ Reviews CATALDO MUSTO, PASQUALE LOPS, MARCO DE GEMMIS, GIOVANNI SEMERARO UNIVERSITÀ DEGLI STUDI DI BARI ‘ALDO MORO’ - ITALY UMAP 2019 – 27th International Conference on User Modeling, Adaptation and Personalization Larnaca (Cyprus) – June 10, 2019
  2. 2. Recommender Systems Cataldo Musto, Pasquale Lops, Marco de Gemmis, Giovanni Semeraro. Justifying Recommendations through Aspect-based Sentiment Analysis of Users’ Reviews. UMAP 2019 – Larnaca (Cyprus) – June 10, 2019
  3. 3. Motivations Recommendation Cataldo Musto, Pasquale Lops, Marco de Gemmis, Giovanni Semeraro. Justifying Recommendations through Aspect-based Sentiment Analysis of Users’ Reviews. UMAP 2019 – Larnaca (Cyprus) – June 10, 2019 I suggest you
  4. 4. Motivations Cataldo Musto, Pasquale Lops, Marco de Gemmis, Giovanni Semeraro. Justifying Recommendations through Aspect-based Sentiment Analysis of Users’ Reviews. UMAP 2019 – Larnaca (Cyprus) – June 10, 2019 ? Recommendation
  5. 5. Intuition Cataldo Musto, Pasquale Lops, Marco de Gemmis, Giovanni Semeraro. Justifying Recommendations through Aspect-based Sentiment Analysis of Users’ Reviews. UMAP 2019 – Larnaca (Cyprus) – June 10, 2019 To identify relevant and distinguishing characteristics of the recommended item by mining users’ reviews, and to use these characteristics to build a natural language justification
  6. 6. Intuition Cataldo Musto, Pasquale Lops, Marco de Gemmis, Giovanni Semeraro. Justifying Recommendations through Aspect-based Sentiment Analysis of Users’ Reviews. UMAP 2019 – Larnaca (Cyprus) – June 10, 2019 Breath-taking images Excellent actors Excellent sound design Wonderful storytelling To identify relevant and distinguishing characteristics of the recommended item by mining users’ reviews, and to use these characteristics to build a natural language justification
  7. 7. Intuition Cataldo Musto, Pasquale Lops, Marco de Gemmis, Giovanni Semeraro. Justifying Recommendations through Aspect-based Sentiment Analysis of Users’ Reviews. UMAP 2019 – Larnaca (Cyprus) – June 10, 2019 "I recommend you 300 because people who liked the movie think that the war scenes are really well done. Moreover, people liked 300 since it has a wonderful storytelling and the actors are excellent" A potential review-based justification
  8. 8. Solution Cataldo Musto, Pasquale Lops, Marco de Gemmis, Giovanni Semeraro. Justifying Recommendations through Aspect-based Sentiment Analysis of Users’ Reviews. UMAP 2019 – Larnaca (Cyprus) – June 10, 2019
  9. 9. Aspect Extraction Cataldo Musto, Pasquale Lops, Marco de Gemmis, Giovanni Semeraro. Justifying Recommendations through Aspect-based Sentiment Analysis of Users’ Reviews. UMAP 2019 – Larnaca (Cyprus) – June 10, 2019
  10. 10. Aspect Extraction Cataldo Musto, Pasquale Lops, Marco de Gemmis, Giovanni Semeraro. Justifying Recommendations through Aspect-based Sentiment Analysis of Users’ Reviews. UMAP 2019 – Larnaca (Cyprus) – June 10, 2019 Goal: to identify the aspects that are discussed when people talk about the item Strategy: to use natural language processing techniques (specifically, part-of-speech tagging) to extract the names mentioned in users’ reviews
  11. 11. Aspect Extraction Cataldo Musto, Pasquale Lops, Marco de Gemmis, Giovanni Semeraro. Justifying Recommendations through Aspect-based Sentiment Analysis of Users’ Reviews. UMAP 2019 – Larnaca (Cyprus) – June 10, 2019 Goal: to identify the aspects that are discussed when people talk about the item Strategy: to use natural language processing techniques (specifically, part-of-speech tagging) to extract the names mentioned in users’ reviews
  12. 12. Aspect Extraction Cataldo Musto, Pasquale Lops, Marco de Gemmis, Giovanni Semeraro. Justifying Recommendations through Aspect-based Sentiment Analysis of Users’ Reviews. UMAP 2019 – Larnaca (Cyprus) – June 10, 2019 reviews aspects Input: reviews of the item i R = {ri1, ri2 … rin} Output: aspects A = {ai1, ai2 … aik}
  13. 13. Aspect Extraction Cataldo Musto, Pasquale Lops, Marco de Gemmis, Giovanni Semeraro. Justifying Recommendations through Aspect-based Sentiment Analysis of Users’ Reviews. UMAP 2019 – Larnaca (Cyprus) – June 10, 2019 reviews aspects Input: reviews of the item i R = {ri1, ri2 … rin} Output: aspects A = {ai1, ai2 … aik} Why only names? Findings from previous work in the area Why no bigrams? No significant improvement emerged
  14. 14. Aspect Ranking Cataldo Musto, Pasquale Lops, Marco de Gemmis, Giovanni Semeraro. Justifying Recommendations through Aspect-based Sentiment Analysis of Users’ Reviews. UMAP 2019 – Larnaca (Cyprus) – June 10, 2019
  15. 15. Aspect Ranking Cataldo Musto, Pasquale Lops, Marco de Gemmis, Giovanni Semeraro. Justifying Recommendations through Aspect-based Sentiment Analysis of Users’ Reviews. UMAP 2019 – Larnaca (Cyprus) – June 10, 2019 Aspect Extraction Goal: to identify the aspects that are discussed when people talk about the item Aspect Ranking Goal: to identify distinguishing aspects that are discussed with a positive sentiment when people talk about the item
  16. 16. Aspect Ranking Cataldo Musto, Pasquale Lops, Marco de Gemmis, Giovanni Semeraro. Justifying Recommendations through Aspect-based Sentiment Analysis of Users’ Reviews. UMAP 2019 – Larnaca (Cyprus) – June 10, 2019 Aspect Extraction Goal: to identify the aspects that are discussed when people talk about the item Aspect Ranking Goal: to identify distinguishing aspects that are discussed with a positive sentiment when people talk about the item Strategy: to calculate a relevance score for each aspect previously detected
  17. 17. Aspect Ranking Cataldo Musto, Pasquale Lops, Marco de Gemmis, Giovanni Semeraro. Justifying Recommendations through Aspect-based Sentiment Analysis of Users’ Reviews. UMAP 2019 – Larnaca (Cyprus) – June 10, 2019 Goal: to identify distinguishing aspects that are discussed with a positive sentiment when people talk about the item
  18. 18. Aspect Ranking Cataldo Musto, Pasquale Lops, Marco de Gemmis, Giovanni Semeraro. Justifying Recommendations through Aspect-based Sentiment Analysis of Users’ Reviews. UMAP 2019 – Larnaca (Cyprus) – June 10, 2019 Goal: to identify distinguishing aspects that are discussed with a positive sentiment when people talk about the item How many times aspect ‘a’ appears in the reviews of item ‘i’ (frequency of the aspect)
  19. 19. Aspect Ranking Cataldo Musto, Pasquale Lops, Marco de Gemmis, Giovanni Semeraro. Justifying Recommendations through Aspect-based Sentiment Analysis of Users’ Reviews. UMAP 2019 – Larnaca (Cyprus) – June 10, 2019 Goal: to identify distinguishing aspects that are discussed with a positive sentiment when people talk about the item How many times aspect ‘a’ appears in the reviews of item ‘i’ (frequency of the aspect) How positive is the opinion of the users when they talk about aspect ‘a’ (opinion towards the aspect)
  20. 20. Aspect Ranking Cataldo Musto, Pasquale Lops, Marco de Gemmis, Giovanni Semeraro. Justifying Recommendations through Aspect-based Sentiment Analysis of Users’ Reviews. UMAP 2019 – Larnaca (Cyprus) – June 10, 2019 Goal: to identify distinguishing aspects that are discussed with a positive sentiment when people talk about the item How many times aspect ‘a’ appears in the reviews of item ‘i’ (frequency of the aspect) How positive is the opinion of the users when they talk about aspect ‘a’ (opinion towards the aspect) How distinguishing is the aspect ‘a’ (inverse popularity)
  21. 21. Aspect Ranking Cataldo Musto, Pasquale Lops, Marco de Gemmis, Giovanni Semeraro. Justifying Recommendations through Aspect-based Sentiment Analysis of Users’ Reviews. UMAP 2019 – Larnaca (Cyprus) – June 10, 2019 Goal: to identify distinguishing aspects that are discussed with a positive sentiment when people talk about the item Intuition: our formula gives an higher score to the aspects that are frequently mentioned in the reviews with a positive sentiment. Moreover, it also rewards less popular aspects (higher IAF).
  22. 22. Aspect Ranking Cataldo Musto, Pasquale Lops, Marco de Gemmis, Giovanni Semeraro. Justifying Recommendations through Aspect-based Sentiment Analysis of Users’ Reviews. UMAP 2019 – Larnaca (Cyprus) – June 10, 2019 aspects top-k aspects Input: aspects A = {ai1, ai2 … aim} Output: top-k aspects A = {ai1, ai2 … aik}
  23. 23. Generation Cataldo Musto, Pasquale Lops, Marco de Gemmis, Giovanni Semeraro. Justifying Recommendations through Aspect-based Sentiment Analysis of Users’ Reviews. UMAP 2019 – Larnaca (Cyprus) – June 10, 2019
  24. 24. Generation Cataldo Musto, Pasquale Lops, Marco de Gemmis, Giovanni Semeraro. Justifying Recommendations through Aspect-based Sentiment Analysis of Users’ Reviews. UMAP 2019 – Larnaca (Cyprus) – June 10, 2019 Goal: to generate a natural language justification that relies on the most relevant aspects identified by the ASPECT RANKING module. For each aspect ’a’ returned by the ASPECT RANKING module Browse the available reviews Look for a compliant excerpt containing ‘a’ If the sentence has a positive sentiment Add the sentence to the justification Strategy
  25. 25. Generation Cataldo Musto, Pasquale Lops, Marco de Gemmis, Giovanni Semeraro. Justifying Recommendations through Aspect-based Sentiment Analysis of Users’ Reviews. UMAP 2019 – Larnaca (Cyprus) – June 10, 2019 Question: when does an excerpt is a compliant sentence? Answer: an excerpt is compliant if it follows one of the 18 justification patterns we defined Example: the excerpt must have a third personal singular verb
  26. 26. Generation Cataldo Musto, Pasquale Lops, Marco de Gemmis, Giovanni Semeraro. Justifying Recommendations through Aspect-based Sentiment Analysis of Users’ Reviews. UMAP 2019 – Larnaca (Cyprus) – June 10, 2019 Question: when does an excerpt is a compliant sentence? Answer: an excerpt is compliant if it follows one of the 18 justification patterns we defined “I really liked the cast” Not compliant “The cast was great” Compliant Example: the excerpt must have a third personal singular verb
  27. 27. Generation – Final Output Cataldo Musto, Pasquale Lops, Marco de Gemmis, Giovanni Semeraro. Justifying Recommendations through Aspect-based Sentiment Analysis of Users’ Reviews. UMAP 2019 – Larnaca (Cyprus) – June 10, 2019 "I recommend you 300 because people who liked the movie think that the war scenes are really well done. Moreover, people liked 300 since it has a wonderful storytelling and the actors are excellent" Legenda Blue: randomized sentences Green: recommendation Red: aspects (k=3) Black: compliant excerpts
  28. 28. Experimental Evaluation Cataldo Musto, Pasquale Lops, Marco de Gemmis, Giovanni Semeraro. Justifying Recommendations through Aspect-based Sentiment Analysis of Users’ Reviews. UMAP 2019 – Larnaca (Cyprus) – June 10, 2019 Research Question 1 How effective are the justifications generated through the pipeline, on varying of different combinations of the parameters? Research Question 2 How does our justifications perform with respect to a classic feature-based explanation [*] ? (e.g.) “I suggest you 300 since you like other movies by Zack Snyder, as Watchmen” Experimental Design User Study with a Web Application 286 subjects Movie and Books Domain Metrics: Transparency, Engagement, Persuasion, Trust, Effectiveness [^] Between-subjects for Research Question 1, Within-subjects for Research Question 2 [*] Musto, C., Narducci, F., Lops, P., De Gemmis, M., Semeraro G. ExpLOD: A Framework for Explaining Recommendations based on the Linked Open Data Cloud. In Proceedings of the 10th ACM Conference on Recommender Systems. pp. 151-154. 2016 [^] Tintarev, N., & Masthoff, J. Designing and evaluating explanations for recommender systems. In Recommender systems handbook. pp. 479-510. Springer, Boston, MA. 2011
  29. 29. Experimental Evaluation Cataldo Musto, Pasquale Lops, Marco de Gemmis, Giovanni Semeraro. Justifying Recommendations through Aspect-based Sentiment Analysis of Users’ Reviews. UMAP 2019 – Larnaca (Cyprus) – June 10, 2019 Parameters of the system - Length of the justifications (short vs. long justifications) short → top-2 aspects long -> top-4 aspects - Vocabulary of aspects (statics vs. complete) static → bounded to a fixed and pre-defined set of relevant aspects. No aspect extraction, just aspect ranking complete → not bounded. All the aspects are discovered by the aspect extractor - Four different configurations Implementation Details Recommendations generated through Personalized PageRank, aspect extraction through CoreNLP POS-tagger and sentiment analysis through Stanford algorithm
  30. 30. Experimental Evaluation Cataldo Musto, Pasquale Lops, Marco de Gemmis, Giovanni Semeraro. Justifying Recommendations through Aspect-based Sentiment Analysis of Users’ Reviews. UMAP 2019 – Larnaca (Cyprus) – June 10, 2019 Statistics of the datasets Movies Books #items 307 333 #reviews 153,566 52,560 avg. reviews/item 500.21 157.80 avg. words/reviews 138.38 126.65 Dataset are obtained by mapping items from MovieLens and BookCrossing datasets with reviews obtained from Amazon data (dataset available)
  31. 31. Web Application Cataldo Musto, Pasquale Lops, Marco de Gemmis, Giovanni Semeraro. Justifying Recommendations through Aspect-based Sentiment Analysis of Users’ Reviews. UMAP 2019 – Larnaca (Cyprus) – June 10, 2019 Recommendation
  32. 32. Web Application (Research Question 1) Cataldo Musto, Pasquale Lops, Marco de Gemmis, Giovanni Semeraro. Justifying Recommendations through Aspect-based Sentiment Analysis of Users’ Reviews. UMAP 2019 – Larnaca (Cyprus) – June 10, 2019 Recommendation Justification Questionnaire
  33. 33. Web Application (Research Question 2) Cataldo Musto, Pasquale Lops, Marco de Gemmis, Giovanni Semeraro. Justifying Recommendations through Aspect-based Sentiment Analysis of Users’ Reviews. UMAP 2019 – Larnaca (Cyprus) – June 10, 2019 I propose you “Aliens” because you sometimes like movies edited by Canadian film editors, American fiction films and 1980s films, as The Terminator. I recommend you “Aliens” because people who liked this movie think that the Alien series is one of the best sci-fi movies and that the ending is awesome with some fantastic action scenes. Review-based Explanation Feature-based Explanation
  34. 34. Results (Research Question 1) Cataldo Musto, Pasquale Lops, Marco de Gemmis, Giovanni Semeraro. Justifying Recommendations through Aspect-based Sentiment Analysis of Users’ Reviews. UMAP 2019 – Larnaca (Cyprus) – June 10, 2019 MOVIES Transparency Persuasion Engagement Trust Effectiveness Static Short 3.40 3.13 3.09 3.23 0.64 Static Long 3.77 3.68 3.55 3.73 0.55 Complete Short 3.91 3.60 3.25 3.70 0.53 Complete Long 3.74 3.48 3.35 3.46 0.59
  35. 35. Results (Research Question 1) Cataldo Musto, Pasquale Lops, Marco de Gemmis, Giovanni Semeraro. Justifying Recommendations through Aspect-based Sentiment Analysis of Users’ Reviews. UMAP 2019 – Larnaca (Cyprus) – June 10, 2019 Finding 1 With the ‘complete’ set of aspects, shorter justifications have the best results MOVIES Transparency Persuasion Engagement Trust Effectiveness Static Short 3.40 3.13 3.09 3.23 0.64 Static Long 3.77 3.68 3.55 3.73 0.55 Complete Short 3.91 3.60 3.25 3.70 0.53 Complete Long 3.74 3.48 3.35 3.46 0.59
  36. 36. Results (Research Question 1) Cataldo Musto, Pasquale Lops, Marco de Gemmis, Giovanni Semeraro. Justifying Recommendations through Aspect-based Sentiment Analysis of Users’ Reviews. UMAP 2019 – Larnaca (Cyprus) – June 10, 2019 Finding 1 With the ‘complete’ set of aspects, shorter justifications have the best results Finding 2 With the ‘static’ set of aspects, longer justifications have the best results MOVIES Transparency Persuasion Engagement Trust Effectiveness Static Short 3.40 3.13 3.09 3.23 0.64 Static Long 3.77 3.68 3.55 3.73 0.55 Complete Short 3.91 3.60 3.25 3.70 0.53 Complete Long 3.74 3.48 3.35 3.46 0.59
  37. 37. Results (Research Question 1) Cataldo Musto, Pasquale Lops, Marco de Gemmis, Giovanni Semeraro. Justifying Recommendations through Aspect-based Sentiment Analysis of Users’ Reviews. UMAP 2019 – Larnaca (Cyprus) – June 10, 2019 Finding 1 With the ‘complete’ set of aspects, shorter justifications have the best results Finding 2 With the ‘static’ set of aspects, longer justifications have the best results Overall Long justifications based on static aspects have the best results in the Movie Domain MOVIES Transparency Persuasion Engagement Trust Effectiveness Static Short 3.40 3.13 3.09 3.23 0.64 Static Long 3.77 3.68 3.55 3.73 0.55 Complete Short 3.91 3.60 3.25 3.70 0.53 Complete Long 3.74 3.48 3.35 3.46 0.59
  38. 38. Results (Research Question 1) Cataldo Musto, Pasquale Lops, Marco de Gemmis, Giovanni Semeraro. Justifying Recommendations through Aspect-based Sentiment Analysis of Users’ Reviews. UMAP 2019 – Larnaca (Cyprus) – June 10, 2019 BOOKS Transparency Persuasion Engagement Trust Effectiveness Static Short 3.57 3.40 3.07 3.31 0.74 Static Long 3.64 3.82 3.45 3.71 0.45 Complete Short 3.82 3.59 3.37 3.51 0.47 Complete Long 3.61 3.41 3.31 3.30 0.57 Findings confirmed for the Books domain. Static list of aspects + Long justifications = Best Performance
  39. 39. Results (Research Question 2) Cataldo Musto, Pasquale Lops, Marco de Gemmis, Giovanni Semeraro. Justifying Recommendations through Aspect-based Sentiment Analysis of Users’ Reviews. UMAP 2019 – Larnaca (Cyprus) – June 10, 2019 MOVIES Review -based Feature- based Indifferent Transparency 47.4% 38.6% 14.0% Persuasion 51.7% 43.3% 5.0% Engagement 66.7% 25.0% 8.3% Trust 53.3% 35.5% 11.7% Effectiveness 57.9% 35.0% 7.1% Outcome: Users preferred Review-based Justifications Confirmed for all the metrics and both the domains BOOKS Review- based Feature- based Indifferent Transparency 58.1% 36.0% 5.9% Persuasion 61.8% 29.0% 9.2% Engagement 54.6% 27.3% 18.1% Trust 58.2% 27.2% 14.6% Effectiveness 59.9% 31.1% 10.0%
  40. 40. Recap Exploitation of Users’ Review to Generate Natural Language Justification Natural Language Processing and Sentiment Analysis pipeline ◦ Aspect Extraction ◦ Aspect Ranking ◦ Generation of the Justification Outcomes of the Experiments ◦ If we bound the generation to a static list of aspects, we can generate longer justifications ◦ Otherwise shorter justifications are preferred (due to some noise?) ◦ In general, review-based justifications are preferred to classic feature-based explanations Cataldo Musto, Pasquale Lops, Marco de Gemmis, Giovanni Semeraro. Justifying Recommendations through Aspect-based Sentiment Analysis of Users’ Reviews. UMAP 2019 – Larnaca (Cyprus) – June 10, 2019
  41. 41. Future Work To evaluate new and different domains To improve the implementation of the single modules ◦ Aspect Extraction (try to extract bigrams or entities) ◦ Aspect Ranking (more sophisticated techniques can be exploited) ◦ Generation of the Justification (make the natural language generation more dynamic) To introduce methods to generate hybrid explanations ◦ Combinations of review data with descriptive features Cataldo Musto, Pasquale Lops, Marco de Gemmis, Giovanni Semeraro. Justifying Recommendations through Aspect-based Sentiment Analysis of Users’ Reviews. UMAP 2019 – Larnaca (Cyprus) – June 10, 2019
  42. 42. Thank you! cataldo.musto@uniba.it @cataldomusto Contacts Cataldo Musto, Pasquale Lops, Marco de Gemmis, Giovanni Semeraro. Justifying Recommendations through Aspect-based Sentiment Analysis of Users’ Reviews. UMAP 2019 – Larnaca (Cyprus) – June 10, 2019

×