An introduction to recommendation systems: types of recommenders, types of data (input), user-user collaborative filtering, item-item collaborative filtering, running Mahout on a single node and on multiple nodes (on Hadoop)
2. Agenda
❖
What is a recommender?!
❖
Types of recommenders, input data!
❖
Recommendations with Mahout: single node!
❖
Recommendations with Mahout: multiple nodes
5/11/2013 Warszawa-JUG
@adamwarski
3. Who Am I?
❖
Day: coding @ SoftwareMill!
❖
Afternoon: playgrounds, Duplos, etc.!
❖
Evenings: blogging, open-source!
❖
Original author of Hibernate Envers!
❖
ElasticMQ, Veripacks, MacWire!
❖
http://www.warski.org
5/11/2013 Warszawa-JUG
@adamwarski
4. Me + recommenders
❖
Yap.TV!
❖
Recommends shows to users
basing on their favorites!
❖
❖
❖
from Yap!
from Facebook!
Some business rules
5/11/2013 Warszawa-JUG
@adamwarski
17. User neighbourhood
❖
Threshold on similarity!
❖
Top-N neighbours!
❖
❖
25-100: good starting point!
Variations:!
❖
trust networks, e.g. friends in a social network
5/11/2013 Warszawa-JUG
@adamwarski
19. Item-Item CF
❖
Similar to UU-CF, only with items!
❖
Item similarity!
❖
Pearson correlation on item ratings!
❖
co-occurences
5/11/2013 Warszawa-JUG
@adamwarski
20. Evaluating recommenders
❖
How to tell if a recommender is good?!
❖
❖
❖
compare implementations!
are the recommendations ok?!
Business metrics!
❖
what leads to increased sales
5/11/2013 Warszawa-JUG
@adamwarski
21. Leave one out
❖
Remove one preference!
❖
Rebuild the model!
❖
See if it is recommended!
❖
Or, see what is the predicted rating
5/11/2013 Warszawa-JUG
@adamwarski
23. Precision/recall
❖
Precision: % of recommended items that are relevant!
❖
Recall: % of relevant items that are recommended!
❖
Precision@N, recall@N
Recall
Ideal recs
Precision
Recs
5/11/2013 Warszawa-JUG
All items
@adamwarski
24. Problems
❖
Novelty: hard to measure!
❖
❖
leave one out: punishes recommenders which
recommend new items!
Test on humans!
❖
A/B testing
5/11/2013 Warszawa-JUG
@adamwarski
26. Netflix challenge
❖
$1m for the best recommender!
❖
Used mean error!
❖
Winning algorithm won on predicting low ratings
5/11/2013 Warszawa-JUG
@adamwarski