SlideShare una empresa de Scribd logo
1 de 40
User Engagement –  A Scientific Challenge Mounia Lalmas Yahoo! Research Barcelona [email_address] Ioannis Arapakis Ricardo Baeza-Yates Georges Dupret Janette Lehmann Lori McCay-Peet Vidhya Navalpakkam Elad Yom-Tov Collaborators
Outline ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Motivation ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Would a user engage with this web site? http://www.nhm.ac.uk/
Would a user engage with this web site? http://www.amazingthings.org/  (art event calendar)
Would a user engage with this web site? http://www.lowpriceskates.com/   (e-commerce – skating)
Would a user engage with this web site? http://chiptune.com/  (music repository)
Would a user engage with this web site? http://www.theosbrinkagency.com/  (photographer)
What is user engagement? ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Characteristics of user engagement (I)
Characteristics of user engagement (II)
Forrester Research – The four I’s Measuring Engagement, Forrester Research, June 2008
Peterson etal  Engagement measure – 8 indices ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Peterson etal. Measuring the immeasurable: visitor engagement, WebAnalyticsDemystified, September 2008
Subjective measures Record a user’s perception (generally self-reported) of the technology Two-part processes: Increased use of crowd-sourcing based studies
Objective measures ,[object Object],[object Object]
Objective measures –  Online activities   Proxy of user engagement ,[object Object]
Characteristics and measures S. Attfield, G. Kazai, M. Lalmas and B. Piwowarski. Towards a science of user engagement (Position Paper),  WSDM Workshop on User Modelling for Web Applications, 2011.
Diagnostic and what we can do ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Page stylistics + layout  +links + saliency Measurements and methodologies + online analytics metrics (dwell time, …) + questionnaires + crowd-sourcing - biometrics (eye tracking, mouse tracking, …)  Goals + Models of user engagement - Metrics of user engagement Engagement types + attention + emotion + activity + popularity + loyalty - functionality +/- intent & interest user engagement  within and across site
Focus – Two “opposite” studies models USER ENGAGEMENT 1. Models of user engagement 2. On saliency, attention and  positive effect  activity metrics loyalty metrics popularity metrics hobbies navigation social media e-commerce magazine sport news search weather mail weekly news … focused attention affect (emotion) saliency
1. Models of user engagement ,[object Object],[object Object],[object Object],[object Object],[object Object]
Data and Metrics ,[object Object],Popularity #Users Number of distinct users #Visits Number of visits #Clicks Number of clicks Activity ClickDepth Average number of page views per visit. DwellTimeA Average time per visit Loyalty ActiveDays Number of days a user visited the site ReturnRate Number of times a user visited the site DwellTimeL Average time a user spend on the site.
Diversity in user engagement ,[object Object],[object Object],[object Object],[object Object],[object Object],Engagement of a site depends on users and time mail, social media shopping, entertainment media (special events) daily activity, navigation media, entertainment
Methodology General models User-based models Time-based models Dimensions 8 metrics 5 user groups 8 metrics per user group weekdays, weekend 8 metrics per time span #Dimensions 8 40 16 Kernel k-means with  Kendall tau rank correlation kernel Nb of clusters based on eigenvalue distribution of kernel matrix Significant metric values with Kruskal-Wallis/Bonferonni #Clusters (Models) 6 7 5 Analysing cluster centroids = models
Models of user engagement ,[object Object],[object Object],[object Object],[object Object],[object Object],Models based on engagement metrics interest-specific e-commerce, configuration periodic media
Models of user engagement ,[object Object],[object Object],[object Object],[object Object],Models based on engagement metrics, user and time navigation game, sport hobbies, interest-specific daily news ,[object Object],[object Object]
Relationships between models ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Variance of Information  [0,5.61] General User Time General 0.00 3.50 4.23 User 3.50 0.00 4.25 Time 4.23 4.25 0.00
Models of user engagement –  Recap & Next ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Focus – Two “opposite” studies models USER ENGAGEMENT 1. Models of user engagement 2. On saliency, attention and  positive effect  activity metrics loyalty metrics popularity metrics hobbies navigation social media e-commerce magazine sport news search weather mail weekly news … focused attention affect (emotion) saliency
2.  Saliency, attention and positive affect ,[object Object],[object Object],[object Object],[object Object],[object Object]
Manipulating saliency salient condition non-salient condition Web page screenshot Saliency maps
Study design ,[object Object],[object Object],[object Object],[object Object],[object Object]
PANAS  (10 positive items and 10 negative items) ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],D. Watson, L.A. Clark & A. Tellegen. Development and validation of brief measures of positive and negative affect:  The PANAS Scales.  Journal of Personality and Social Psychology, 47 (1988).
7-item focused attention subscale  (part of the 31-item user engagement scale)   5-point scale (strong disagree to strong agree) ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],H.L. O'Brien.  Defining and Measuring Engagement in User Experiences with Technology.  PhD Thesis, 2008.
Study protocol ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Saliency and positive affect ,[object Object],[object Object],[object Object],[object Object],[object Object]
Saliency  and  focused attention ,[object Object],[object Object],[object Object]
Saliency and user engagement –  Recap & Next ,[object Object],[object Object],[object Object],[object Object],[object Object],L. McCay-Peet, M. Lalmas, V. Navalpakkam. On saliency, affecr and focused attention,  CHI 2012
The big picture ……….… my vision Page stylistics + layout  +links + saliency Measurements and methodologies + online analytics metrics (dwell time, …) + questionnaires + crowd-sourcing - biometrics (eye tracking, mouse tracking, …)  Goals + Models of user engagement - Metrics of user engagement Engagement types + attention + emotion + activity + popularity + loyalty - functionality +/- intent & interest user engagement  within and across site
Gracias ,[object Object],[object Object]

Más contenido relacionado

La actualidad más candente

User Engagement: from Sites to a Network of Sites or The Network Effect Matters!
User Engagement: from Sites to a Network of Sites or The Network Effect Matters!User Engagement: from Sites to a Network of Sites or The Network Effect Matters!
User Engagement: from Sites to a Network of Sites or The Network Effect Matters!Mounia Lalmas-Roelleke
 
From “Selena Gomez” to “Marlon Brando”: Understanding Explorative Entity Search
From “Selena Gomez” to “Marlon Brando”: Understanding Explorative Entity SearchFrom “Selena Gomez” to “Marlon Brando”: Understanding Explorative Entity Search
From “Selena Gomez” to “Marlon Brando”: Understanding Explorative Entity SearchMounia Lalmas-Roelleke
 
A Journey into Evaluation: from Retrieval Effectiveness to User Engagement
A Journey into Evaluation: from Retrieval Effectiveness to User EngagementA Journey into Evaluation: from Retrieval Effectiveness to User Engagement
A Journey into Evaluation: from Retrieval Effectiveness to User EngagementMounia Lalmas-Roelleke
 
Tutorial on Online User Engagement: Metrics and Optimization
Tutorial on Online User Engagement: Metrics and OptimizationTutorial on Online User Engagement: Metrics and Optimization
Tutorial on Online User Engagement: Metrics and OptimizationMounia Lalmas-Roelleke
 
User Engagement: A Cross-Disciplinary Framework (Nim Dvir)
User Engagement: A Cross-Disciplinary Framework (Nim Dvir)User Engagement: A Cross-Disciplinary Framework (Nim Dvir)
User Engagement: A Cross-Disciplinary Framework (Nim Dvir)Nim Dvir
 
Metrics, Engagement & Personalization
Metrics, Engagement & Personalization Metrics, Engagement & Personalization
Metrics, Engagement & Personalization Mounia Lalmas-Roelleke
 
User Research & Usability Testing - NavigationArts
User Research & Usability Testing - NavigationArtsUser Research & Usability Testing - NavigationArts
User Research & Usability Testing - NavigationArtsKirsten Miller
 
Engagement, metrics and "recommenders"
Engagement, metrics and "recommenders"Engagement, metrics and "recommenders"
Engagement, metrics and "recommenders"Mounia Lalmas-Roelleke
 
A Rubric for Assessing the UX of Online Museum Collections: Preliminary Findi...
A Rubric for Assessing the UX of Online Museum Collections: Preliminary Findi...A Rubric for Assessing the UX of Online Museum Collections: Preliminary Findi...
A Rubric for Assessing the UX of Online Museum Collections: Preliminary Findi...craigmmacdonald
 
An Exploratory Study of the Motivations and Satisfactions on Mobile Web Browsing
An Exploratory Study of the Motivations and Satisfactions on Mobile Web BrowsingAn Exploratory Study of the Motivations and Satisfactions on Mobile Web Browsing
An Exploratory Study of the Motivations and Satisfactions on Mobile Web BrowsingRuby Kuo
 
Design Patterns on Social Navigation
Design Patterns on Social NavigationDesign Patterns on Social Navigation
Design Patterns on Social NavigationRuby Kuo
 
Digital Communication Power Tools: Speakers Notes version
Digital Communication Power Tools: Speakers Notes versionDigital Communication Power Tools: Speakers Notes version
Digital Communication Power Tools: Speakers Notes versionMarilyn Herie
 
UX Librarians: User Advocates, User Researchers, Usability Evaluators, or All...
UX Librarians: User Advocates, User Researchers, Usability Evaluators, or All...UX Librarians: User Advocates, User Researchers, Usability Evaluators, or All...
UX Librarians: User Advocates, User Researchers, Usability Evaluators, or All...craigmmacdonald
 
Toward a More Robust Usability concept with Perceived Enjoyment in the contex...
Toward a More Robust Usability concept with Perceived Enjoyment in the contex...Toward a More Robust Usability concept with Perceived Enjoyment in the contex...
Toward a More Robust Usability concept with Perceived Enjoyment in the contex...Waqas Tariq
 
Assessing the User Experience (UX) of Online Museum Collections: Perspectives...
Assessing the User Experience (UX) of Online Museum Collections: Perspectives...Assessing the User Experience (UX) of Online Museum Collections: Perspectives...
Assessing the User Experience (UX) of Online Museum Collections: Perspectives...craigmmacdonald
 
Lecture 1: Human-Computer Interaction Introduction (2014)
Lecture 1: Human-Computer Interaction Introduction (2014)Lecture 1: Human-Computer Interaction Introduction (2014)
Lecture 1: Human-Computer Interaction Introduction (2014)Lora Aroyo
 
What Does it Mean for a System to be Useful? An Exploratory Study of Usefulness
What Does it Mean for a System to be Useful? An Exploratory Study of UsefulnessWhat Does it Mean for a System to be Useful? An Exploratory Study of Usefulness
What Does it Mean for a System to be Useful? An Exploratory Study of Usefulnesscraigmmacdonald
 

La actualidad más candente (20)

User Engagement: from Sites to a Network of Sites or The Network Effect Matters!
User Engagement: from Sites to a Network of Sites or The Network Effect Matters!User Engagement: from Sites to a Network of Sites or The Network Effect Matters!
User Engagement: from Sites to a Network of Sites or The Network Effect Matters!
 
An engaging click
An engaging clickAn engaging click
An engaging click
 
From “Selena Gomez” to “Marlon Brando”: Understanding Explorative Entity Search
From “Selena Gomez” to “Marlon Brando”: Understanding Explorative Entity SearchFrom “Selena Gomez” to “Marlon Brando”: Understanding Explorative Entity Search
From “Selena Gomez” to “Marlon Brando”: Understanding Explorative Entity Search
 
A Journey into Evaluation: from Retrieval Effectiveness to User Engagement
A Journey into Evaluation: from Retrieval Effectiveness to User EngagementA Journey into Evaluation: from Retrieval Effectiveness to User Engagement
A Journey into Evaluation: from Retrieval Effectiveness to User Engagement
 
Tutorial on Online User Engagement: Metrics and Optimization
Tutorial on Online User Engagement: Metrics and OptimizationTutorial on Online User Engagement: Metrics and Optimization
Tutorial on Online User Engagement: Metrics and Optimization
 
User Engagement: A Cross-Disciplinary Framework (Nim Dvir)
User Engagement: A Cross-Disciplinary Framework (Nim Dvir)User Engagement: A Cross-Disciplinary Framework (Nim Dvir)
User Engagement: A Cross-Disciplinary Framework (Nim Dvir)
 
Metrics, Engagement & Personalization
Metrics, Engagement & Personalization Metrics, Engagement & Personalization
Metrics, Engagement & Personalization
 
User Research & Usability Testing - NavigationArts
User Research & Usability Testing - NavigationArtsUser Research & Usability Testing - NavigationArts
User Research & Usability Testing - NavigationArts
 
Engagement, metrics and "recommenders"
Engagement, metrics and "recommenders"Engagement, metrics and "recommenders"
Engagement, metrics and "recommenders"
 
A Rubric for Assessing the UX of Online Museum Collections: Preliminary Findi...
A Rubric for Assessing the UX of Online Museum Collections: Preliminary Findi...A Rubric for Assessing the UX of Online Museum Collections: Preliminary Findi...
A Rubric for Assessing the UX of Online Museum Collections: Preliminary Findi...
 
An Exploratory Study of the Motivations and Satisfactions on Mobile Web Browsing
An Exploratory Study of the Motivations and Satisfactions on Mobile Web BrowsingAn Exploratory Study of the Motivations and Satisfactions on Mobile Web Browsing
An Exploratory Study of the Motivations and Satisfactions on Mobile Web Browsing
 
Design Patterns on Social Navigation
Design Patterns on Social NavigationDesign Patterns on Social Navigation
Design Patterns on Social Navigation
 
Digital Communication Power Tools: Speakers Notes version
Digital Communication Power Tools: Speakers Notes versionDigital Communication Power Tools: Speakers Notes version
Digital Communication Power Tools: Speakers Notes version
 
Social Media
Social MediaSocial Media
Social Media
 
SMAG Evaluation
SMAG EvaluationSMAG Evaluation
SMAG Evaluation
 
UX Librarians: User Advocates, User Researchers, Usability Evaluators, or All...
UX Librarians: User Advocates, User Researchers, Usability Evaluators, or All...UX Librarians: User Advocates, User Researchers, Usability Evaluators, or All...
UX Librarians: User Advocates, User Researchers, Usability Evaluators, or All...
 
Toward a More Robust Usability concept with Perceived Enjoyment in the contex...
Toward a More Robust Usability concept with Perceived Enjoyment in the contex...Toward a More Robust Usability concept with Perceived Enjoyment in the contex...
Toward a More Robust Usability concept with Perceived Enjoyment in the contex...
 
Assessing the User Experience (UX) of Online Museum Collections: Perspectives...
Assessing the User Experience (UX) of Online Museum Collections: Perspectives...Assessing the User Experience (UX) of Online Museum Collections: Perspectives...
Assessing the User Experience (UX) of Online Museum Collections: Perspectives...
 
Lecture 1: Human-Computer Interaction Introduction (2014)
Lecture 1: Human-Computer Interaction Introduction (2014)Lecture 1: Human-Computer Interaction Introduction (2014)
Lecture 1: Human-Computer Interaction Introduction (2014)
 
What Does it Mean for a System to be Useful? An Exploratory Study of Usefulness
What Does it Mean for a System to be Useful? An Exploratory Study of UsefulnessWhat Does it Mean for a System to be Useful? An Exploratory Study of Usefulness
What Does it Mean for a System to be Useful? An Exploratory Study of Usefulness
 

Similar a User Engagement - A Scientific Challenge

Tutorial on metrics of user engagement -- Applications to Search & E- commerce
Tutorial on metrics of user engagement -- Applications to Search & E- commerceTutorial on metrics of user engagement -- Applications to Search & E- commerce
Tutorial on metrics of user engagement -- Applications to Search & E- commerceMounia Lalmas-Roelleke
 
The Verification Of Virtual Community Member’s Socio-Demographic Profile
The Verification Of Virtual Community Member’s Socio-Demographic ProfileThe Verification Of Virtual Community Member’s Socio-Demographic Profile
The Verification Of Virtual Community Member’s Socio-Demographic Profileacijjournal
 
THE VERIFICATION OF VIRTUAL COMMUNITY MEMBER’S SOCIO-DEMOGRAPHIC PROFILE
THE VERIFICATION OF VIRTUAL COMMUNITY MEMBER’S SOCIO-DEMOGRAPHIC PROFILE THE VERIFICATION OF VIRTUAL COMMUNITY MEMBER’S SOCIO-DEMOGRAPHIC PROFILE
THE VERIFICATION OF VIRTUAL COMMUNITY MEMBER’S SOCIO-DEMOGRAPHIC PROFILE acijjournal
 
Determining Strategic Value of Online Social Engagements
Determining Strategic Value of Online Social EngagementsDetermining Strategic Value of Online Social Engagements
Determining Strategic Value of Online Social Engagementsinventionjournals
 
A Study On Social Media Habits of Internet Users
A Study On Social Media Habits of Internet Users A Study On Social Media Habits of Internet Users
A Study On Social Media Habits of Internet Users Huseyin Kiran
 
Chapter 6 presentation
Chapter 6 presentationChapter 6 presentation
Chapter 6 presentationMiles223
 
Team Lecture on Blog
Team Lecture on BlogTeam Lecture on Blog
Team Lecture on Blogmcleanq
 
Chapter 6 presentation
Chapter 6 presentationChapter 6 presentation
Chapter 6 presentationsabucher
 
customer behavior analysis for social media
customer behavior analysis for social mediacustomer behavior analysis for social media
customer behavior analysis for social mediaINFOGAIN PUBLICATION
 
Running Research Communities in Asian markets
Running Research Communities in Asian marketsRunning Research Communities in Asian markets
Running Research Communities in Asian marketsInSites Consulting
 
Social Sentiment Analysis and Its Use in Communication Campaigns
Social Sentiment Analysis and Its Use in Communication CampaignsSocial Sentiment Analysis and Its Use in Communication Campaigns
Social Sentiment Analysis and Its Use in Communication Campaignssvladovic
 
The Effects of Social Media Marketing In the Hotel Industry: Conceptual Model...
The Effects of Social Media Marketing In the Hotel Industry: Conceptual Model...The Effects of Social Media Marketing In the Hotel Industry: Conceptual Model...
The Effects of Social Media Marketing In the Hotel Industry: Conceptual Model...inventionjournals
 
The Effects of Social Media Marketing In the Hotel Industry: Conceptual Model...
The Effects of Social Media Marketing In the Hotel Industry: Conceptual Model...The Effects of Social Media Marketing In the Hotel Industry: Conceptual Model...
The Effects of Social Media Marketing In the Hotel Industry: Conceptual Model...Samaan Al-Msallam
 
Schivinski, Christodoulides, Dabrowski 2016 - measuring consumers ’ engagem...
Schivinski, Christodoulides, Dabrowski   2016 - measuring consumers ’ engagem...Schivinski, Christodoulides, Dabrowski   2016 - measuring consumers ’ engagem...
Schivinski, Christodoulides, Dabrowski 2016 - measuring consumers ’ engagem...Roger Smith
 
Consumer activity in social media managerial approaches to consumers' social...
Consumer activity in social media  managerial approaches to consumers' social...Consumer activity in social media  managerial approaches to consumers' social...
Consumer activity in social media managerial approaches to consumers' social...Anupam Lav
 
interacting with social media content about events
interacting with social media content about eventsinteracting with social media content about events
interacting with social media content about eventsmor
 

Similar a User Engagement - A Scientific Challenge (20)

Tutorial on metrics of user engagement -- Applications to Search & E- commerce
Tutorial on metrics of user engagement -- Applications to Search & E- commerceTutorial on metrics of user engagement -- Applications to Search & E- commerce
Tutorial on metrics of user engagement -- Applications to Search & E- commerce
 
Session, focus and engagement
Session, focus and engagementSession, focus and engagement
Session, focus and engagement
 
The Verification Of Virtual Community Member’s Socio-Demographic Profile
The Verification Of Virtual Community Member’s Socio-Demographic ProfileThe Verification Of Virtual Community Member’s Socio-Demographic Profile
The Verification Of Virtual Community Member’s Socio-Demographic Profile
 
THE VERIFICATION OF VIRTUAL COMMUNITY MEMBER’S SOCIO-DEMOGRAPHIC PROFILE
THE VERIFICATION OF VIRTUAL COMMUNITY MEMBER’S SOCIO-DEMOGRAPHIC PROFILE THE VERIFICATION OF VIRTUAL COMMUNITY MEMBER’S SOCIO-DEMOGRAPHIC PROFILE
THE VERIFICATION OF VIRTUAL COMMUNITY MEMBER’S SOCIO-DEMOGRAPHIC PROFILE
 
Determining Strategic Value of Online Social Engagements
Determining Strategic Value of Online Social EngagementsDetermining Strategic Value of Online Social Engagements
Determining Strategic Value of Online Social Engagements
 
A Study On Social Media Habits of Internet Users
A Study On Social Media Habits of Internet Users A Study On Social Media Habits of Internet Users
A Study On Social Media Habits of Internet Users
 
Chapter 6 presentation
Chapter 6 presentationChapter 6 presentation
Chapter 6 presentation
 
Team Lecture on Blog
Team Lecture on BlogTeam Lecture on Blog
Team Lecture on Blog
 
Chapter 6 presentation
Chapter 6 presentationChapter 6 presentation
Chapter 6 presentation
 
H018144450
H018144450H018144450
H018144450
 
customer behavior analysis for social media
customer behavior analysis for social mediacustomer behavior analysis for social media
customer behavior analysis for social media
 
Running Research Communities in Asian markets
Running Research Communities in Asian marketsRunning Research Communities in Asian markets
Running Research Communities in Asian markets
 
Expertise Social Media Research - eng- out 2013
Expertise   Social Media Research - eng- out 2013Expertise   Social Media Research - eng- out 2013
Expertise Social Media Research - eng- out 2013
 
Social Sentiment Analysis and Its Use in Communication Campaigns
Social Sentiment Analysis and Its Use in Communication CampaignsSocial Sentiment Analysis and Its Use in Communication Campaigns
Social Sentiment Analysis and Its Use in Communication Campaigns
 
The Effects of Social Media Marketing In the Hotel Industry: Conceptual Model...
The Effects of Social Media Marketing In the Hotel Industry: Conceptual Model...The Effects of Social Media Marketing In the Hotel Industry: Conceptual Model...
The Effects of Social Media Marketing In the Hotel Industry: Conceptual Model...
 
The Effects of Social Media Marketing In the Hotel Industry: Conceptual Model...
The Effects of Social Media Marketing In the Hotel Industry: Conceptual Model...The Effects of Social Media Marketing In the Hotel Industry: Conceptual Model...
The Effects of Social Media Marketing In the Hotel Industry: Conceptual Model...
 
Schivinski, Christodoulides, Dabrowski 2016 - measuring consumers ’ engagem...
Schivinski, Christodoulides, Dabrowski   2016 - measuring consumers ’ engagem...Schivinski, Christodoulides, Dabrowski   2016 - measuring consumers ’ engagem...
Schivinski, Christodoulides, Dabrowski 2016 - measuring consumers ’ engagem...
 
Consumer activity in social media managerial approaches to consumers' social...
Consumer activity in social media  managerial approaches to consumers' social...Consumer activity in social media  managerial approaches to consumers' social...
Consumer activity in social media managerial approaches to consumers' social...
 
interacting with social media content about events
interacting with social media content about eventsinteracting with social media content about events
interacting with social media content about events
 
Dissertation
DissertationDissertation
Dissertation
 

Más de Mounia Lalmas-Roelleke

Engagement, Metrics & Personalisation at Scale
Engagement, Metrics &  Personalisation at ScaleEngagement, Metrics &  Personalisation at Scale
Engagement, Metrics & Personalisation at ScaleMounia Lalmas-Roelleke
 
Personalizing the listening experience
Personalizing the listening experiencePersonalizing the listening experience
Personalizing the listening experienceMounia Lalmas-Roelleke
 
Recommending and Searching (Research @ Spotify)
Recommending and Searching (Research @ Spotify)Recommending and Searching (Research @ Spotify)
Recommending and Searching (Research @ Spotify)Mounia Lalmas-Roelleke
 
An introduction to system-oriented evaluation in Information Retrieval
An introduction to system-oriented evaluation in Information RetrievalAn introduction to system-oriented evaluation in Information Retrieval
An introduction to system-oriented evaluation in Information RetrievalMounia Lalmas-Roelleke
 
Friendly, Appealing or Both? Characterising User Experience in Sponsored Sear...
Friendly, Appealing or Both? Characterising User Experience in Sponsored Sear...Friendly, Appealing or Both? Characterising User Experience in Sponsored Sear...
Friendly, Appealing or Both? Characterising User Experience in Sponsored Sear...Mounia Lalmas-Roelleke
 
Social Media and AI: Don’t forget the users
Social Media and AI: Don’t forget the usersSocial Media and AI: Don’t forget the users
Social Media and AI: Don’t forget the usersMounia Lalmas-Roelleke
 
Describing Patterns and Disruptions in Large Scale Mobile App Usage Data
Describing Patterns and Disruptions in Large Scale Mobile App Usage DataDescribing Patterns and Disruptions in Large Scale Mobile App Usage Data
Describing Patterns and Disruptions in Large Scale Mobile App Usage DataMounia Lalmas-Roelleke
 
Story-focused Reading in Online News and its Potential for User Engagement
Story-focused Reading in Online News and its Potential for User EngagementStory-focused Reading in Online News and its Potential for User Engagement
Story-focused Reading in Online News and its Potential for User EngagementMounia Lalmas-Roelleke
 
Mobile advertising: The preclick experience
Mobile advertising: The preclick experienceMobile advertising: The preclick experience
Mobile advertising: The preclick experienceMounia Lalmas-Roelleke
 
Predicting Pre-click Quality for Native Advertisements
Predicting Pre-click Quality for Native AdvertisementsPredicting Pre-click Quality for Native Advertisements
Predicting Pre-click Quality for Native AdvertisementsMounia Lalmas-Roelleke
 
Improving Post-Click User Engagement on Native Ads via Survival Analysis
Improving Post-Click User Engagement on Native Ads via Survival AnalysisImproving Post-Click User Engagement on Native Ads via Survival Analysis
Improving Post-Click User Engagement on Native Ads via Survival AnalysisMounia Lalmas-Roelleke
 
Promoting Positive Post-click Experience for In-Stream Yahoo Gemini Users
Promoting Positive Post-click Experience for In-Stream Yahoo Gemini UsersPromoting Positive Post-click Experience for In-Stream Yahoo Gemini Users
Promoting Positive Post-click Experience for In-Stream Yahoo Gemini UsersMounia Lalmas-Roelleke
 
Social Media News Communities: Gatekeeping, Coverage, and Statement Bias
 Social Media News Communities: Gatekeeping, Coverage, and Statement Bias Social Media News Communities: Gatekeeping, Coverage, and Statement Bias
Social Media News Communities: Gatekeeping, Coverage, and Statement BiasMounia Lalmas-Roelleke
 
On the Reliability and Intuitiveness of Aggregated Search Metrics
On the Reliability and Intuitiveness of Aggregated Search MetricsOn the Reliability and Intuitiveness of Aggregated Search Metrics
On the Reliability and Intuitiveness of Aggregated Search MetricsMounia Lalmas-Roelleke
 
Penguins in Sweaters, or Serendipitous Entity Search on User-generated Content
 Penguins in Sweaters, or Serendipitous Entity Search on User-generated Content Penguins in Sweaters, or Serendipitous Entity Search on User-generated Content
Penguins in Sweaters, or Serendipitous Entity Search on User-generated ContentMounia Lalmas-Roelleke
 
Evaluating Heterogeneous Information Access (Position Paper)
Evaluating Heterogeneous Information Access (Position Paper)Evaluating Heterogeneous Information Access (Position Paper)
Evaluating Heterogeneous Information Access (Position Paper)Mounia Lalmas-Roelleke
 

Más de Mounia Lalmas-Roelleke (19)

Engagement, Metrics & Personalisation at Scale
Engagement, Metrics &  Personalisation at ScaleEngagement, Metrics &  Personalisation at Scale
Engagement, Metrics & Personalisation at Scale
 
Recommending and searching @ Spotify
Recommending and searching @ SpotifyRecommending and searching @ Spotify
Recommending and searching @ Spotify
 
Personalizing the listening experience
Personalizing the listening experiencePersonalizing the listening experience
Personalizing the listening experience
 
Recommending and Searching (Research @ Spotify)
Recommending and Searching (Research @ Spotify)Recommending and Searching (Research @ Spotify)
Recommending and Searching (Research @ Spotify)
 
Search @ Spotify
Search @ Spotify Search @ Spotify
Search @ Spotify
 
An introduction to system-oriented evaluation in Information Retrieval
An introduction to system-oriented evaluation in Information RetrievalAn introduction to system-oriented evaluation in Information Retrieval
An introduction to system-oriented evaluation in Information Retrieval
 
Friendly, Appealing or Both? Characterising User Experience in Sponsored Sear...
Friendly, Appealing or Both? Characterising User Experience in Sponsored Sear...Friendly, Appealing or Both? Characterising User Experience in Sponsored Sear...
Friendly, Appealing or Both? Characterising User Experience in Sponsored Sear...
 
Social Media and AI: Don’t forget the users
Social Media and AI: Don’t forget the usersSocial Media and AI: Don’t forget the users
Social Media and AI: Don’t forget the users
 
Advertising Quality Science
Advertising Quality ScienceAdvertising Quality Science
Advertising Quality Science
 
Describing Patterns and Disruptions in Large Scale Mobile App Usage Data
Describing Patterns and Disruptions in Large Scale Mobile App Usage DataDescribing Patterns and Disruptions in Large Scale Mobile App Usage Data
Describing Patterns and Disruptions in Large Scale Mobile App Usage Data
 
Story-focused Reading in Online News and its Potential for User Engagement
Story-focused Reading in Online News and its Potential for User EngagementStory-focused Reading in Online News and its Potential for User Engagement
Story-focused Reading in Online News and its Potential for User Engagement
 
Mobile advertising: The preclick experience
Mobile advertising: The preclick experienceMobile advertising: The preclick experience
Mobile advertising: The preclick experience
 
Predicting Pre-click Quality for Native Advertisements
Predicting Pre-click Quality for Native AdvertisementsPredicting Pre-click Quality for Native Advertisements
Predicting Pre-click Quality for Native Advertisements
 
Improving Post-Click User Engagement on Native Ads via Survival Analysis
Improving Post-Click User Engagement on Native Ads via Survival AnalysisImproving Post-Click User Engagement on Native Ads via Survival Analysis
Improving Post-Click User Engagement on Native Ads via Survival Analysis
 
Promoting Positive Post-click Experience for In-Stream Yahoo Gemini Users
Promoting Positive Post-click Experience for In-Stream Yahoo Gemini UsersPromoting Positive Post-click Experience for In-Stream Yahoo Gemini Users
Promoting Positive Post-click Experience for In-Stream Yahoo Gemini Users
 
Social Media News Communities: Gatekeeping, Coverage, and Statement Bias
 Social Media News Communities: Gatekeeping, Coverage, and Statement Bias Social Media News Communities: Gatekeeping, Coverage, and Statement Bias
Social Media News Communities: Gatekeeping, Coverage, and Statement Bias
 
On the Reliability and Intuitiveness of Aggregated Search Metrics
On the Reliability and Intuitiveness of Aggregated Search MetricsOn the Reliability and Intuitiveness of Aggregated Search Metrics
On the Reliability and Intuitiveness of Aggregated Search Metrics
 
Penguins in Sweaters, or Serendipitous Entity Search on User-generated Content
 Penguins in Sweaters, or Serendipitous Entity Search on User-generated Content Penguins in Sweaters, or Serendipitous Entity Search on User-generated Content
Penguins in Sweaters, or Serendipitous Entity Search on User-generated Content
 
Evaluating Heterogeneous Information Access (Position Paper)
Evaluating Heterogeneous Information Access (Position Paper)Evaluating Heterogeneous Information Access (Position Paper)
Evaluating Heterogeneous Information Access (Position Paper)
 

Último

Boost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivityBoost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivityPrincipled Technologies
 
Artificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsArtificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsJoaquim Jorge
 
🐬 The future of MySQL is Postgres 🐘
🐬  The future of MySQL is Postgres   🐘🐬  The future of MySQL is Postgres   🐘
🐬 The future of MySQL is Postgres 🐘RTylerCroy
 
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...DianaGray10
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfsudhanshuwaghmare1
 
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodPolkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodJuan lago vázquez
 
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps ScriptAutomating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Scriptwesley chun
 
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...Martijn de Jong
 
Manulife - Insurer Innovation Award 2024
Manulife - Insurer Innovation Award 2024Manulife - Insurer Innovation Award 2024
Manulife - Insurer Innovation Award 2024The Digital Insurer
 
Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...apidays
 
Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)wesley chun
 
HTML Injection Attacks: Impact and Mitigation Strategies
HTML Injection Attacks: Impact and Mitigation StrategiesHTML Injection Attacks: Impact and Mitigation Strategies
HTML Injection Attacks: Impact and Mitigation StrategiesBoston Institute of Analytics
 
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Drew Madelung
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationSafe Software
 
Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherStrategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherRemote DBA Services
 
MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024MIND CTI
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonAnna Loughnan Colquhoun
 
presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century educationjfdjdjcjdnsjd
 
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...apidays
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc
 

Último (20)

Boost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivityBoost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivity
 
Artificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsArtificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and Myths
 
🐬 The future of MySQL is Postgres 🐘
🐬  The future of MySQL is Postgres   🐘🐬  The future of MySQL is Postgres   🐘
🐬 The future of MySQL is Postgres 🐘
 
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
 
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodPolkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
 
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps ScriptAutomating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Script
 
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...
 
Manulife - Insurer Innovation Award 2024
Manulife - Insurer Innovation Award 2024Manulife - Insurer Innovation Award 2024
Manulife - Insurer Innovation Award 2024
 
Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...
 
Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)
 
HTML Injection Attacks: Impact and Mitigation Strategies
HTML Injection Attacks: Impact and Mitigation StrategiesHTML Injection Attacks: Impact and Mitigation Strategies
HTML Injection Attacks: Impact and Mitigation Strategies
 
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
 
Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherStrategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a Fresher
 
MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt Robison
 
presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century education
 
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
 

User Engagement - A Scientific Challenge

  • 1. User Engagement – A Scientific Challenge Mounia Lalmas Yahoo! Research Barcelona [email_address] Ioannis Arapakis Ricardo Baeza-Yates Georges Dupret Janette Lehmann Lori McCay-Peet Vidhya Navalpakkam Elad Yom-Tov Collaborators
  • 2.
  • 3.
  • 4. Would a user engage with this web site? http://www.nhm.ac.uk/
  • 5. Would a user engage with this web site? http://www.amazingthings.org/ (art event calendar)
  • 6. Would a user engage with this web site? http://www.lowpriceskates.com/ (e-commerce – skating)
  • 7. Would a user engage with this web site? http://chiptune.com/ (music repository)
  • 8. Would a user engage with this web site? http://www.theosbrinkagency.com/ (photographer)
  • 9.
  • 10. Characteristics of user engagement (I)
  • 11. Characteristics of user engagement (II)
  • 12. Forrester Research – The four I’s Measuring Engagement, Forrester Research, June 2008
  • 13.
  • 14. Subjective measures Record a user’s perception (generally self-reported) of the technology Two-part processes: Increased use of crowd-sourcing based studies
  • 15.
  • 16.
  • 17. Characteristics and measures S. Attfield, G. Kazai, M. Lalmas and B. Piwowarski. Towards a science of user engagement (Position Paper), WSDM Workshop on User Modelling for Web Applications, 2011.
  • 18.
  • 19. Page stylistics + layout +links + saliency Measurements and methodologies + online analytics metrics (dwell time, …) + questionnaires + crowd-sourcing - biometrics (eye tracking, mouse tracking, …) Goals + Models of user engagement - Metrics of user engagement Engagement types + attention + emotion + activity + popularity + loyalty - functionality +/- intent & interest user engagement within and across site
  • 20. Focus – Two “opposite” studies models USER ENGAGEMENT 1. Models of user engagement 2. On saliency, attention and positive effect activity metrics loyalty metrics popularity metrics hobbies navigation social media e-commerce magazine sport news search weather mail weekly news … focused attention affect (emotion) saliency
  • 21.
  • 22.
  • 23.
  • 24. Methodology General models User-based models Time-based models Dimensions 8 metrics 5 user groups 8 metrics per user group weekdays, weekend 8 metrics per time span #Dimensions 8 40 16 Kernel k-means with Kendall tau rank correlation kernel Nb of clusters based on eigenvalue distribution of kernel matrix Significant metric values with Kruskal-Wallis/Bonferonni #Clusters (Models) 6 7 5 Analysing cluster centroids = models
  • 25.
  • 26.
  • 27.
  • 28.
  • 29. Focus – Two “opposite” studies models USER ENGAGEMENT 1. Models of user engagement 2. On saliency, attention and positive effect activity metrics loyalty metrics popularity metrics hobbies navigation social media e-commerce magazine sport news search weather mail weekly news … focused attention affect (emotion) saliency
  • 30.
  • 31. Manipulating saliency salient condition non-salient condition Web page screenshot Saliency maps
  • 32.
  • 33.
  • 34.
  • 35.
  • 36.
  • 37.
  • 38.
  • 39. The big picture ……….… my vision Page stylistics + layout +links + saliency Measurements and methodologies + online analytics metrics (dwell time, …) + questionnaires + crowd-sourcing - biometrics (eye tracking, mouse tracking, …) Goals + Models of user engagement - Metrics of user engagement Engagement types + attention + emotion + activity + popularity + loyalty - functionality +/- intent & interest user engagement within and across site
  • 40.

Notas del editor

  1. As said here it is important to differentiate between user experience and user engagement, the latter is an aspect of the former. User engagement related to the positive aspect of the interaction including that of being captivated. Providing technologies with which user engaged is very important, and this is true of any web service. And those are two nice quotes about the importance of engagement.
  2. Characteristics elaborate the notion of engagement over 3 broad dimensions: emotional, cognitive and behavioural. We identified 8 of them based on previous works / suggested by us. Attention -> exclusivity Affect -> lack of fun can lead as a barrier to shopping (O’Brian) Aesthetics -> promotes attention, stimulate curiosity Endurability -> does the experience meet the expectation?
  3. Novelty -> freshness of content / has to be carefully balanced Richness = complexity of thoughts and actions, increased complexity/ Affect = relates to the emotion experienced during the interaction (fun and enjoyment)
  4. The way it often work  A two-part process. One is very exploratory, it is important to “probe” carefully, so that indentify all dimension, usually conducted with a small number of users. Highly qualitative, and really the aim is to identify the important characteristics (work of Tom et al did this in 4 areas, online shopping, web searching, educational web casting, and video games). People in HCI knows how to do this well. Then from there, construct more focused studies, but larger scale, and actually measure them in a particular interactive experience, and again going back to the Dalhousie work, they did so for the online shopping scenario. Constructed an online questionnaire, and got 440 responses. Cleaning of data, and factor analysis to reduce the set of characteristics (100) to few key ones (down to 6). Again what I want to make it clear is the methodology, and here the fact we have qualitative data. Would be important to see how the questionnaire developed by them can be used in other area, other domain (IM different to news portal). Health  trust Film  aesthetics
  5. In the paper we propose a mapping between objective measures and engagement characteristics, e.g. focused attention can be measured by distorted perception of time, follow-on task performance, and eye tracking Online behaviour = can be large scale IR = special status (metrics) -> Simulated search, user models, linking to user satisfaction