SlideShare una empresa de Scribd logo
1 de 24
Descargar para leer sin conexión
RecommenderSystems
and the Human Factor
Mark Graus
Netherlands Machine Learning Meetup
2016/03/16
I’m
50% Machine
Learner
50%
Psychologist
I’ve been
working with
‘recommender
systems’ since
2009
 Movie Recommender Systems
 Website
 Personalization
 App
 Personalization
200920112016
Content
 WhatAre Recommender Systems
 Why Machine Learning is not Enough
What are
Recommender
Systems?
The Machine
Learning
Behind
Recommender
Systems
 We use historical item-user data to predict
unobserved item-user data
 Typically big datasets
 i.e. billions of observations
 millions of users
 tons of items
 Numerous Specifically Designed Algorithms
How I see
Recommender
Algorithms
Implicit Feedback Explicit Feedback
Collaborative
Content-Based
Distinction 1:
Implicit versus
Explicit
Feedback
Implicit
My actual behavior
 watching
 skipping/stopping
Explicit
The feedback I give
 star rating
Distinction 1:
Considerations
“Oh no! MyTiVo thinks I’m gay”
Jeffrey Zaslow,TheWall Street Journal, December 2002
What I Like versus What I Say I Like
Solution: Use a bit of both implicit and explicit
Distinction 2:
Content-Based
versus
Collaborative
Filtering
 Supervised learning
 Features are extracted from ‘metadata’
 Target variable is rating (explicit) or whether the movie will be
watched (implicit)
Genre Director Main
Actor
Year Rating
The Usual
Suspects
Crime Bryan
Singer
Kevin
Spacey
1995
Titanic Drama James
Cameron
Leonardo
DiCaprio
1997
Die Hard Action John
McTiernan
Bruce
Willis
1988
?
Distinction 2:
Content-Based
versus
Collaborative
Filtering
KNN,SlopeOne
?
?
Matrix
Factorization
butalso FunkSVD,
SVD+
UsualSuspects
Titanic
DieHard
TheGodfather
Jack
Dylan
Olivia
Mark
?
?
?
? ? ?
?
 Dimensionality Reduction
Matrix
Factorization
butalso FunkSVD,
SVD+
Jack
Mark
Olivia Dylan
Content-Based
versus
Collaborative
Considerations
 Metadata availability
 Need for explaining
MyApproach
 Start with Open Source Software
 Lenskit (Java)
 MyMediaLite (C#)
 Mahout (Python)
 Learn about Recommender Systems and User Base
 Scale Up
 Cassandra
 Akka
State-of-the-
Art
 We can do predictions really well
 Challenges
 Cold Start Problem
 Context-Aware Recommendations
 Social Recommendations
 “Merged accounts”
Why Machine Learning
is Not Enough
Recommender
System Data is
Observable
Behavior
Recommendations
Behavior
Recommender
System
User
Experience
Examples of
Things Data
CannotTellUs
 Do I feel my privacy invaded?
 Am I happy to have American Pie 2 recommended?
 Why do people react to recommendations the way they do?
 Presentation?
 Bad Recommendations?
 Choice Overload?
We need to do
A/B testing
andUX
measurement
System A System B
What did we
learn from
surveys?
 Satisfaction =
Recommendation Set
Attractiveness - Choice
Difficulty
 More views != Satisfaction
 Diversity influences
Satisfaction
 Long Lists = Difficult to
Choose
 Short Lists = Easier to
Choose, but not enough
choice
 Right Balance = Short
Lists of Diverse Items
Take Home
Message
 The Machine Learning is just the beginning of Recommender
Systems
Thank you for
listening!
Some Pointers
 Recommender Algorithms
 Yehuda Koren, Google
 Introduction to Recommender Systems,Coursera/GroupLens
 Infrastructure
 NetflixTech Blog
 A/BTesting
 Ron Kohavi, Microsoft Research
 User Experience Evaluation in Recommender Systems
 Bart Knijnenburg, Clemson University

Más contenido relacionado

La actualidad más candente

9B_1_Trust in web gis a preliminary investigation of the environment agencys ...
9B_1_Trust in web gis a preliminary investigation of the environment agencys ...9B_1_Trust in web gis a preliminary investigation of the environment agencys ...
9B_1_Trust in web gis a preliminary investigation of the environment agencys ...
GISRUK conference
 
How NOT to Aggregrate Polling Data
How NOT to Aggregrate Polling DataHow NOT to Aggregrate Polling Data
How NOT to Aggregrate Polling Data
DataCards
 

La actualidad más candente (20)

Thesis Presentation
Thesis PresentationThesis Presentation
Thesis Presentation
 
Interaction Design Patterns in Recommender Systems
Interaction Design Patterns in Recommender SystemsInteraction Design Patterns in Recommender Systems
Interaction Design Patterns in Recommender Systems
 
Press One for Yes: Using Clickers for Library Instruction and Information Li...
Press One for Yes: Using Clickers for Library Instruction and Information Li...Press One for Yes: Using Clickers for Library Instruction and Information Li...
Press One for Yes: Using Clickers for Library Instruction and Information Li...
 
Get to know your users
Get to know your users Get to know your users
Get to know your users
 
Social tags and controlled vocabularies as filters for searching and browsing...
Social tags and controlled vocabularies as filters for searching and browsing...Social tags and controlled vocabularies as filters for searching and browsing...
Social tags and controlled vocabularies as filters for searching and browsing...
 
Movie recommendation system using collaborative filtering system
Movie recommendation system using collaborative filtering system Movie recommendation system using collaborative filtering system
Movie recommendation system using collaborative filtering system
 
Amazon seniment
Amazon senimentAmazon seniment
Amazon seniment
 
[WI 2017] Context Suggestion: Empirical Evaluations vs User Studies
[WI 2017] Context Suggestion: Empirical Evaluations vs User Studies[WI 2017] Context Suggestion: Empirical Evaluations vs User Studies
[WI 2017] Context Suggestion: Empirical Evaluations vs User Studies
 
Process of instrument develoment
Process of instrument develomentProcess of instrument develoment
Process of instrument develoment
 
VA Mobile Applications for Mental Health: Designing a Patient-Centered User E...
VA Mobile Applications for Mental Health: Designing a Patient-Centered User E...VA Mobile Applications for Mental Health: Designing a Patient-Centered User E...
VA Mobile Applications for Mental Health: Designing a Patient-Centered User E...
 
Movies recommendation system in R Studio, Machine learning
Movies recommendation system in  R Studio, Machine learning Movies recommendation system in  R Studio, Machine learning
Movies recommendation system in R Studio, Machine learning
 
Mobile User Experience & Solve the Outbreak
Mobile User Experience & Solve the OutbreakMobile User Experience & Solve the Outbreak
Mobile User Experience & Solve the Outbreak
 
Ml part2
Ml part2Ml part2
Ml part2
 
9B_1_Trust in web gis a preliminary investigation of the environment agencys ...
9B_1_Trust in web gis a preliminary investigation of the environment agencys ...9B_1_Trust in web gis a preliminary investigation of the environment agencys ...
9B_1_Trust in web gis a preliminary investigation of the environment agencys ...
 
How NOT to Aggregrate Polling Data
How NOT to Aggregrate Polling DataHow NOT to Aggregrate Polling Data
How NOT to Aggregrate Polling Data
 
Web application of Antonucci's hierarchical approach for measuring social net...
Web application of Antonucci's hierarchical approach for measuring social net...Web application of Antonucci's hierarchical approach for measuring social net...
Web application of Antonucci's hierarchical approach for measuring social net...
 
Recommender Systems
Recommender SystemsRecommender Systems
Recommender Systems
 
215 Towards a Framework to Support Large Scale Sampling in Software Engineeri...
215 Towards a Framework to Support Large Scale Sampling in Software Engineeri...215 Towards a Framework to Support Large Scale Sampling in Software Engineeri...
215 Towards a Framework to Support Large Scale Sampling in Software Engineeri...
 
Social Media Sentiments Analysis
Social Media Sentiments AnalysisSocial Media Sentiments Analysis
Social Media Sentiments Analysis
 
Molly russell
Molly russellMolly russell
Molly russell
 

Destacado

Factorization Meets the Item Embedding: Regularizing Matrix Factorization wit...
Factorization Meets the Item Embedding: Regularizing Matrix Factorization wit...Factorization Meets the Item Embedding: Regularizing Matrix Factorization wit...
Factorization Meets the Item Embedding: Regularizing Matrix Factorization wit...
Dawen Liang
 

Destacado (8)

BYU Social Media Committee Meeting - Finding Followers
 BYU Social Media Committee Meeting - Finding Followers BYU Social Media Committee Meeting - Finding Followers
BYU Social Media Committee Meeting - Finding Followers
 
Major
MajorMajor
Major
 
What is Socialism
What is SocialismWhat is Socialism
What is Socialism
 
Kdd 2014 Tutorial - the recommender problem revisited
Kdd 2014 Tutorial -  the recommender problem revisitedKdd 2014 Tutorial -  the recommender problem revisited
Kdd 2014 Tutorial - the recommender problem revisited
 
Past, Present & Future of Recommender Systems: An Industry Perspective
Past, Present & Future of Recommender Systems: An Industry PerspectivePast, Present & Future of Recommender Systems: An Industry Perspective
Past, Present & Future of Recommender Systems: An Industry Perspective
 
Factorization Meets the Item Embedding: Regularizing Matrix Factorization wit...
Factorization Meets the Item Embedding: Regularizing Matrix Factorization wit...Factorization Meets the Item Embedding: Regularizing Matrix Factorization wit...
Factorization Meets the Item Embedding: Regularizing Matrix Factorization wit...
 
Recsys 2016 tutorial: Lessons learned from building real-life recommender sys...
Recsys 2016 tutorial: Lessons learned from building real-life recommender sys...Recsys 2016 tutorial: Lessons learned from building real-life recommender sys...
Recsys 2016 tutorial: Lessons learned from building real-life recommender sys...
 
Balancing Discovery and Continuation in Recommendations
Balancing Discovery and Continuation in RecommendationsBalancing Discovery and Continuation in Recommendations
Balancing Discovery and Continuation in Recommendations
 

Similar a Recommender Systems and the Human Factor

How to Get Survey Answers That Actually Help Your Business
How to Get Survey Answers That Actually Help Your BusinessHow to Get Survey Answers That Actually Help Your Business
How to Get Survey Answers That Actually Help Your Business
Kissmetrics on SlideShare
 

Similar a Recommender Systems and the Human Factor (20)

The subtle art of recommendation
The subtle art of recommendationThe subtle art of recommendation
The subtle art of recommendation
 
How AI is transforming learning
How AI is transforming learningHow AI is transforming learning
How AI is transforming learning
 
Ai demystified for HR and TA leaders
Ai demystified for HR and TA leadersAi demystified for HR and TA leaders
Ai demystified for HR and TA leaders
 
Reflection In Web2.0 Style
Reflection In Web2.0 StyleReflection In Web2.0 Style
Reflection In Web2.0 Style
 
Recommender Systems in TEL
Recommender Systems in TELRecommender Systems in TEL
Recommender Systems in TEL
 
Tech for Tracking & Assessment
Tech for Tracking & AssessmentTech for Tracking & Assessment
Tech for Tracking & Assessment
 
The Hive Think Tank: Machine Learning at Pinterest by Jure Leskovec
The Hive Think Tank: Machine Learning at Pinterest by Jure LeskovecThe Hive Think Tank: Machine Learning at Pinterest by Jure Leskovec
The Hive Think Tank: Machine Learning at Pinterest by Jure Leskovec
 
Recommender systems to help people move forward
Recommender systems to help people move forwardRecommender systems to help people move forward
Recommender systems to help people move forward
 
Sweeny group think-ias2015
Sweeny group think-ias2015Sweeny group think-ias2015
Sweeny group think-ias2015
 
Top 10 Tips for Smart Software Selection Success Before the Search
Top 10 Tips for Smart Software Selection Success Before the SearchTop 10 Tips for Smart Software Selection Success Before the Search
Top 10 Tips for Smart Software Selection Success Before the Search
 
Learning Analytics Primer: Getting Started with Learning and Performance Anal...
Learning Analytics Primer: Getting Started with Learning and Performance Anal...Learning Analytics Primer: Getting Started with Learning and Performance Anal...
Learning Analytics Primer: Getting Started with Learning and Performance Anal...
 
What We Learned from Three Years of Sciencing the Crap Out of DevOps
What We Learned from Three Years of Sciencing the Crap Out of DevOpsWhat We Learned from Three Years of Sciencing the Crap Out of DevOps
What We Learned from Three Years of Sciencing the Crap Out of DevOps
 
What we learned from three years sciencing the crap out of devops
What we learned from three years sciencing the crap out of devopsWhat we learned from three years sciencing the crap out of devops
What we learned from three years sciencing the crap out of devops
 
Algorithmic Impact Assessment: Fairness, Robustness and Explainability in Aut...
Algorithmic Impact Assessment: Fairness, Robustness and Explainability in Aut...Algorithmic Impact Assessment: Fairness, Robustness and Explainability in Aut...
Algorithmic Impact Assessment: Fairness, Robustness and Explainability in Aut...
 
How to Get Survey Answers That Actually Help Your Business
How to Get Survey Answers That Actually Help Your BusinessHow to Get Survey Answers That Actually Help Your Business
How to Get Survey Answers That Actually Help Your Business
 
Everything You Always Wanted to Know About Bad UX Research But Were Afraid to...
Everything You Always Wanted to Know About Bad UX Research But Were Afraid to...Everything You Always Wanted to Know About Bad UX Research But Were Afraid to...
Everything You Always Wanted to Know About Bad UX Research But Were Afraid to...
 
Impersonal Recommendation system on top of Hadoop
Impersonal Recommendation system on top of HadoopImpersonal Recommendation system on top of Hadoop
Impersonal Recommendation system on top of Hadoop
 
DIY: Research on a shoestring budget
DIY: Research on a shoestring budgetDIY: Research on a shoestring budget
DIY: Research on a shoestring budget
 
Fuzzy Logic Based Recommender System
Fuzzy Logic Based Recommender SystemFuzzy Logic Based Recommender System
Fuzzy Logic Based Recommender System
 
Planning your analytics journey - webinar slides
Planning your analytics journey  - webinar slidesPlanning your analytics journey  - webinar slides
Planning your analytics journey - webinar slides
 

Último

Jual Obat Aborsi Surabaya ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...
Jual Obat Aborsi Surabaya ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...Jual Obat Aborsi Surabaya ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...
Jual Obat Aborsi Surabaya ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...
ZurliaSoop
 
Reconciling Conflicting Data Curation Actions: Transparency Through Argument...
Reconciling Conflicting Data Curation Actions:  Transparency Through Argument...Reconciling Conflicting Data Curation Actions:  Transparency Through Argument...
Reconciling Conflicting Data Curation Actions: Transparency Through Argument...
Bertram Ludäscher
 
Top profile Call Girls In bhavnagar [ 7014168258 ] Call Me For Genuine Models...
Top profile Call Girls In bhavnagar [ 7014168258 ] Call Me For Genuine Models...Top profile Call Girls In bhavnagar [ 7014168258 ] Call Me For Genuine Models...
Top profile Call Girls In bhavnagar [ 7014168258 ] Call Me For Genuine Models...
gajnagarg
 
Sealdah % High Class Call Girls Kolkata - 450+ Call Girl Cash Payment 8005736...
Sealdah % High Class Call Girls Kolkata - 450+ Call Girl Cash Payment 8005736...Sealdah % High Class Call Girls Kolkata - 450+ Call Girl Cash Payment 8005736...
Sealdah % High Class Call Girls Kolkata - 450+ Call Girl Cash Payment 8005736...
HyderabadDolls
 
Computer science Sql cheat sheet.pdf.pdf
Computer science Sql cheat sheet.pdf.pdfComputer science Sql cheat sheet.pdf.pdf
Computer science Sql cheat sheet.pdf.pdf
SayantanBiswas37
 
Top profile Call Girls In Indore [ 7014168258 ] Call Me For Genuine Models We...
Top profile Call Girls In Indore [ 7014168258 ] Call Me For Genuine Models We...Top profile Call Girls In Indore [ 7014168258 ] Call Me For Genuine Models We...
Top profile Call Girls In Indore [ 7014168258 ] Call Me For Genuine Models We...
gajnagarg
 
+97470301568>>weed for sale in qatar ,weed for sale in dubai,weed for sale in...
+97470301568>>weed for sale in qatar ,weed for sale in dubai,weed for sale in...+97470301568>>weed for sale in qatar ,weed for sale in dubai,weed for sale in...
+97470301568>>weed for sale in qatar ,weed for sale in dubai,weed for sale in...
Health
 

Último (20)

Jual Obat Aborsi Surabaya ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...
Jual Obat Aborsi Surabaya ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...Jual Obat Aborsi Surabaya ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...
Jual Obat Aborsi Surabaya ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...
 
Reconciling Conflicting Data Curation Actions: Transparency Through Argument...
Reconciling Conflicting Data Curation Actions:  Transparency Through Argument...Reconciling Conflicting Data Curation Actions:  Transparency Through Argument...
Reconciling Conflicting Data Curation Actions: Transparency Through Argument...
 
20240412-SmartCityIndex-2024-Full-Report.pdf
20240412-SmartCityIndex-2024-Full-Report.pdf20240412-SmartCityIndex-2024-Full-Report.pdf
20240412-SmartCityIndex-2024-Full-Report.pdf
 
Digital Transformation Playbook by Graham Ware
Digital Transformation Playbook by Graham WareDigital Transformation Playbook by Graham Ware
Digital Transformation Playbook by Graham Ware
 
Top profile Call Girls In bhavnagar [ 7014168258 ] Call Me For Genuine Models...
Top profile Call Girls In bhavnagar [ 7014168258 ] Call Me For Genuine Models...Top profile Call Girls In bhavnagar [ 7014168258 ] Call Me For Genuine Models...
Top profile Call Girls In bhavnagar [ 7014168258 ] Call Me For Genuine Models...
 
Gulbai Tekra * Cheap Call Girls In Ahmedabad Phone No 8005736733 Elite Escort...
Gulbai Tekra * Cheap Call Girls In Ahmedabad Phone No 8005736733 Elite Escort...Gulbai Tekra * Cheap Call Girls In Ahmedabad Phone No 8005736733 Elite Escort...
Gulbai Tekra * Cheap Call Girls In Ahmedabad Phone No 8005736733 Elite Escort...
 
Digital Advertising Lecture for Advanced Digital & Social Media Strategy at U...
Digital Advertising Lecture for Advanced Digital & Social Media Strategy at U...Digital Advertising Lecture for Advanced Digital & Social Media Strategy at U...
Digital Advertising Lecture for Advanced Digital & Social Media Strategy at U...
 
Aspirational Block Program Block Syaldey District - Almora
Aspirational Block Program Block Syaldey District - AlmoraAspirational Block Program Block Syaldey District - Almora
Aspirational Block Program Block Syaldey District - Almora
 
7. Epi of Chronic respiratory diseases.ppt
7. Epi of Chronic respiratory diseases.ppt7. Epi of Chronic respiratory diseases.ppt
7. Epi of Chronic respiratory diseases.ppt
 
5CL-ADBA,5cladba, Chinese supplier, safety is guaranteed
5CL-ADBA,5cladba, Chinese supplier, safety is guaranteed5CL-ADBA,5cladba, Chinese supplier, safety is guaranteed
5CL-ADBA,5cladba, Chinese supplier, safety is guaranteed
 
Sealdah % High Class Call Girls Kolkata - 450+ Call Girl Cash Payment 8005736...
Sealdah % High Class Call Girls Kolkata - 450+ Call Girl Cash Payment 8005736...Sealdah % High Class Call Girls Kolkata - 450+ Call Girl Cash Payment 8005736...
Sealdah % High Class Call Girls Kolkata - 450+ Call Girl Cash Payment 8005736...
 
Computer science Sql cheat sheet.pdf.pdf
Computer science Sql cheat sheet.pdf.pdfComputer science Sql cheat sheet.pdf.pdf
Computer science Sql cheat sheet.pdf.pdf
 
DATA SUMMIT 24 Building Real-Time Pipelines With FLaNK
DATA SUMMIT 24  Building Real-Time Pipelines With FLaNKDATA SUMMIT 24  Building Real-Time Pipelines With FLaNK
DATA SUMMIT 24 Building Real-Time Pipelines With FLaNK
 
Vadodara 💋 Call Girl 7737669865 Call Girls in Vadodara Escort service book now
Vadodara 💋 Call Girl 7737669865 Call Girls in Vadodara Escort service book nowVadodara 💋 Call Girl 7737669865 Call Girls in Vadodara Escort service book now
Vadodara 💋 Call Girl 7737669865 Call Girls in Vadodara Escort service book now
 
Fun all Day Call Girls in Jaipur 9332606886 High Profile Call Girls You Ca...
Fun all Day Call Girls in Jaipur   9332606886  High Profile Call Girls You Ca...Fun all Day Call Girls in Jaipur   9332606886  High Profile Call Girls You Ca...
Fun all Day Call Girls in Jaipur 9332606886 High Profile Call Girls You Ca...
 
Predicting HDB Resale Prices - Conducting Linear Regression Analysis With Orange
Predicting HDB Resale Prices - Conducting Linear Regression Analysis With OrangePredicting HDB Resale Prices - Conducting Linear Regression Analysis With Orange
Predicting HDB Resale Prices - Conducting Linear Regression Analysis With Orange
 
Top profile Call Girls In Indore [ 7014168258 ] Call Me For Genuine Models We...
Top profile Call Girls In Indore [ 7014168258 ] Call Me For Genuine Models We...Top profile Call Girls In Indore [ 7014168258 ] Call Me For Genuine Models We...
Top profile Call Girls In Indore [ 7014168258 ] Call Me For Genuine Models We...
 
+97470301568>>weed for sale in qatar ,weed for sale in dubai,weed for sale in...
+97470301568>>weed for sale in qatar ,weed for sale in dubai,weed for sale in...+97470301568>>weed for sale in qatar ,weed for sale in dubai,weed for sale in...
+97470301568>>weed for sale in qatar ,weed for sale in dubai,weed for sale in...
 
Dubai Call Girls Peeing O525547819 Call Girls Dubai
Dubai Call Girls Peeing O525547819 Call Girls DubaiDubai Call Girls Peeing O525547819 Call Girls Dubai
Dubai Call Girls Peeing O525547819 Call Girls Dubai
 
SAC 25 Final National, Regional & Local Angel Group Investing Insights 2024 0...
SAC 25 Final National, Regional & Local Angel Group Investing Insights 2024 0...SAC 25 Final National, Regional & Local Angel Group Investing Insights 2024 0...
SAC 25 Final National, Regional & Local Angel Group Investing Insights 2024 0...
 

Recommender Systems and the Human Factor