Se ha denunciado esta presentación.
Utilizamos tu perfil de LinkedIn y tus datos de actividad para personalizar los anuncios y mostrarte publicidad más relevante. Puedes cambiar tus preferencias de publicidad en cualquier momento.
Online
Learning to Rank
for Recommending
Daan Odijk

Lead data scientist @ Blendle
@dodijk
Mission
Help you discover and support 

the world’s best journalism

International
May 2014: The Netherlands
Sept ...
@dodijk
In Blendle you can
browse through all
quality
newspapers and
magazines
@dodijk
You only pay for
what you read,
with a single
click
Scale at Blendle
@dodijk
Articles
> 6M in total
> 7K new every day
> 30% is read
Users
> 1M users
~ 1 in 5 converts to a
p...
@dodijk
Our editors select the best
articles for our email
newsletter every day
Our personalisation
algorithms create a
pe...
@anneschuthBlendle
@dodijk
@dodijk
Why we personalize
@dodijk
Why we personalize
@dodijk
@dodijk
Random Forest classifier trained
on a year of editorial picks
Clustered based on Cosine
similarity with TF.IDF vect...
Sales→
Today
1w
|
2w
|
3w
|
Short shelve life
Daily cold start
• >7K new articles every night
• Our newsletter is an
important traffic driver
• No usage info to rank the...
Article enrichment pipeline
Author
extraction
Semantic
linking
Sentiment
analysis
Stylometry
Named Entity
Recognition
PoS-...
User Profiling
Enrich
Aggregate Profile
• Reads
• Views
• Negative feedback
Learning to rank: preference learning
Model
Enrich EnrichProfile Profile
Extract ML Features
Learning to predict
Learning to rank: preference learning
Model
Enrich EnrichProfile Profile
Extract ML Features
Learning to predict
Enrich Profi...
Online Learning to Rank
• Learning with a user in the loop
• Daily updates to our model
query
[Yue et al, 2009; Hofmann et al., 2011]
Dueling Bandit Gradient Descent
wAuthor
wTopic
wAuthor
wTopic
Explorative Ra...
Interleaved Ranking Explorative RankingExploitative Ranking
A
B
C
D
E
F
C
G
D
A
B
E
query
TeamDraft Interleave
Radlinski, ...
Interleaved Ranking Explorative RankingExploitative Ranking
A
B
D
E
F
C
G
D
A
B
E
query
TeamDraft Interleave
Radlinski, F....
Interleaved Ranking Explorative RankingExploitative Ranking
AB
D
E
F
C
G
D
B
E
query
TeamDraft Interleave
Radlinski, F., K...
Interleaved Ranking
A
B
E
C
G
D
query
TeamDraft Interleave
Radlinski, F., Kurup, M., & Joachims, T. (2008). How does click...
Interleaved Ranking
A
B
E
C
G
D
query
TeamDraft Interleave
Radlinski, F., Kurup, M., & Joachims, T. (2008). How does click...
query
wAuthor
wTopic
Explorative RankerExploitative Ranker
wAuthor
wTopic
G
Dueling Bandit Gradient Descent
[Yue et al, 20...
query
wAuthor
wTopic
Explorative RankerExploitative Ranker
wAuthor
wTopic
G
Dueling Bandit Gradient Descent
[Yue et al, 20...
query
Exploitative Ranker
wAuthor
wTopic
G
Dueling Bandit Gradient Descent
[Yue et al, 2009; Hofmann et al., 2011]
Online Evaluation
@dodijk
-2%
0%
2%
4%
6%
Online Learning A/B Test
Lift in
articles
read
Days →
User Cold Start
@dodijk
User Cold Start
@dodijk
+ Newspapers
RankerWeight
inScore
0%
50%
100%
Articles Read
0 5
Onboarding Reading
5% lift in
arti...
@dodijkBlendle
Relevance
Foryou
✓
Diversification
@dodijkBlendle
Relevance
Foryou
✓
Similarity
Diversification
@dodijkBlendle
Relevance
Foryou
✓
Similarity
✓
Diversification
@dodijkBlendle
Relevance
Foryou
✓
Similarity
✓
Diversification
@dodijkBlendle
Relevance
Foryou
✓
Similarity
✓ ✓
Diversification
@dodijkBlendle
Relevance
Foryou
✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓
Diversification
@dodijkBlendle
Breaking Bubbles
Editorial selection
Must reads
Diversification
Little effect of popularity
Filter Bubbles
F...
Timing problem
• Our editors wake up at 5am
and are done reading at
7am
• Which is also when we want
to send our newslette...
@dodijk
article
published
articles
enrich
article
enrich
article
enrich
article
articles updates
persist
update
articles
p...
personalize
bundle
@dodijk
scores
editor sends
newsletter
newsletters
users
select users
user newsletters personalized new...
Cold start problem
• So we enrich our
content
Timing problem
• So we precompute
as much as
possible
Explanations: #FATREC
Why this article?
Because you seem to like a long
read article every now and then.
@dodijk
@dodijk research.blendle.io
Próxima SlideShare
Cargando en…5
×

Blendle @ RecSys'17: Online Learning to Rank for Recommender Systems

3.976 visualizaciones

Publicado el

Online Learning to Rank for Recommender Systems
by Daan Odijk (Blendle)
https://recsys.acm.org/recsys17/industry-session-3/

Every morning, at Blendle, we have a huge cold-start problem when over 8.000 new articles from the latest editions of newspapers arrive in our system. At that moment, these articles are read by virtually no-one and we are tasked with sending out personalised newsletters to over 1 million users. We can thus not rely on collaborative filtering type of recommendations, nor can we use the popularity of the articles as clues for what our user might want to read. We overcome our cold-start problem by a mix of curation by our editorial team and an automated analysis of the content of these articles. We extract named entities, semantic links, authors, the language and plenty of stylometrics. For each of our users, we build a very fine grained profile based on the attributes of the articles that they read. The combination of enriched articles and user profiles is fed into our machine learning pipeline. We are currently experimenting with an online learning to rank setup, where each of our users is exposed to a slightly perturbed version of our ranking model. We observe the interactions of our users to infer in which direction we should be updating the model.
Our editorial team gets up at around 5am every morning to read what was published over night. They are done reading and recommending their selection of articles around 8am, which is also the time we would ideally send out the newsletter so that our users, on their commute to work, can read our newsletter. These timing restrictions pose yet another challenge: our content analysis and machine learning pipeline needs to be really fast. We solve this by using a streaming infrastructure build on Kafka. In this infrastructure, an article is analysed and scored for relevance towards each of our users as soons as it arrives. This has the advantage that at 8am, when our editorial team is done reading, personalisation is much more lightweight. We use the precomputed relevance scores and balance them with diversity to arrive at a unique ranking for each of our users. In this talk, I will detail how we enrich articles in a streaming fashion and how we use online learning methods to learn a ranking model. I will also talk about how we deal with the time constraints of the problem we are trying to solve.

ABOUT THE SPEAKER
Daan Odijk is lead data scientist at Blendle, a New York Times backed startup that builds a platform where users can explore and support the world’s best journalism and only pay for what they read. Daan heads a team of eight data scientists and engineers who work on personalised recommendations. Daan has a PhD in information retrieval and has worked on leveraging context when searching for news.

Publicado en: Ciencias
  • Like to know how to take easy surveys and get huge checks - then you need to visit us now! Having so many paid surveys available to you all the time let you live the kind of life you want. learn more...➤➤ https://tinyurl.com/make2793amonth
       Responder 
    ¿Estás seguro?    No
    Tu mensaje aparecerá aquí
  • Hello! Get Your Professional Job-Winning Resume Here - Check our website! https://vk.cc/818RFv
       Responder 
    ¿Estás seguro?    No
    Tu mensaje aparecerá aquí
  • great talk on online learning for publishing for the day for temporally valuable content
       Responder 
    ¿Estás seguro?    No
    Tu mensaje aparecerá aquí

Blendle @ RecSys'17: Online Learning to Rank for Recommender Systems

  1. 1. Online Learning to Rank for Recommending Daan Odijk
 Lead data scientist @ Blendle
  2. 2. @dodijk Mission Help you discover and support 
 the world’s best journalism
 International May 2014: The Netherlands Sept 2015: Germany March 2016: United States Publisher-backed a.o. NY Times, Nikkei, Axel Springer 70 employees 10 journalists & 50 developers Blendle
  3. 3. @dodijk In Blendle you can browse through all quality newspapers and magazines
  4. 4. @dodijk You only pay for what you read, with a single click
  5. 5. Scale at Blendle @dodijk Articles > 6M in total > 7K new every day > 30% is read Users > 1M users ~ 1 in 5 converts to a paying user Events ~ 2B in total > 2M new every day ! "
  6. 6. @dodijk Our editors select the best articles for our email newsletter every day Our personalisation algorithms create a personal bundle from this
  7. 7. @anneschuthBlendle
  8. 8. @dodijk
  9. 9. @dodijk
  10. 10. Why we personalize @dodijk
  11. 11. Why we personalize @dodijk
  12. 12. @dodijk Random Forest classifier trained on a year of editorial picks Clustered based on Cosine similarity with TF.IDF vectors Prioritised Selection
  13. 13. Sales→ Today 1w | 2w | 3w | Short shelve life
  14. 14. Daily cold start • >7K new articles every night • Our newsletter is an important traffic driver • No usage info to rank the newsletter before we send the newsletter
  15. 15. Article enrichment pipeline Author extraction Semantic linking Sentiment analysis Stylometry Named Entity Recognition PoS-tagging Length, word variation, vocabulary richness, … Polarity scores (negative, positive) Locations, persons, organizations Wikipedia concepts @dodijk Language detection Topic modeling Tokenization Spark LDA EM
  16. 16. User Profiling Enrich Aggregate Profile • Reads • Views • Negative feedback
  17. 17. Learning to rank: preference learning Model Enrich EnrichProfile Profile Extract ML Features Learning to predict
  18. 18. Learning to rank: preference learning Model Enrich EnrichProfile Profile Extract ML Features Learning to predict Enrich Profile Extract ML Features Rank Ranking
  19. 19. Online Learning to Rank • Learning with a user in the loop • Daily updates to our model
  20. 20. query [Yue et al, 2009; Hofmann et al., 2011] Dueling Bandit Gradient Descent wAuthor wTopic wAuthor wTopic Explorative RankerExploitative Ranker For Blendle the useris the query
  21. 21. Interleaved Ranking Explorative RankingExploitative Ranking A B C D E F C G D A B E query TeamDraft Interleave Radlinski, F., Kurup, M., & Joachims, T. (2008). How does clickthrough data reflect retrieval quality? In CIKM ’08.
  22. 22. Interleaved Ranking Explorative RankingExploitative Ranking A B D E F C G D A B E query TeamDraft Interleave Radlinski, F., Kurup, M., & Joachims, T. (2008). How does clickthrough data reflect retrieval quality? In CIKM ’08.
  23. 23. Interleaved Ranking Explorative RankingExploitative Ranking AB D E F C G D B E query TeamDraft Interleave Radlinski, F., Kurup, M., & Joachims, T. (2008). How does clickthrough data reflect retrieval quality? In CIKM ’08.
  24. 24. Interleaved Ranking A B E C G D query TeamDraft Interleave Radlinski, F., Kurup, M., & Joachims, T. (2008). How does clickthrough data reflect retrieval quality? In CIKM ’08.
  25. 25. Interleaved Ranking A B E C G D query TeamDraft Interleave Radlinski, F., Kurup, M., & Joachims, T. (2008). How does clickthrough data reflect retrieval quality? In CIKM ’08. note: the interleaving method is NOT part of DBGD, it just provides feedback
  26. 26. query wAuthor wTopic Explorative RankerExploitative Ranker wAuthor wTopic G Dueling Bandit Gradient Descent [Yue et al, 2009; Hofmann et al., 2011]
  27. 27. query wAuthor wTopic Explorative RankerExploitative Ranker wAuthor wTopic G Dueling Bandit Gradient Descent [Yue et al, 2009; Hofmann et al., 2011]
  28. 28. query Exploitative Ranker wAuthor wTopic G Dueling Bandit Gradient Descent [Yue et al, 2009; Hofmann et al., 2011]
  29. 29. Online Evaluation @dodijk -2% 0% 2% 4% 6% Online Learning A/B Test Lift in articles read Days →
  30. 30. User Cold Start @dodijk
  31. 31. User Cold Start @dodijk + Newspapers RankerWeight inScore 0% 50% 100% Articles Read 0 5 Onboarding Reading 5% lift in articles read
  32. 32. @dodijkBlendle Relevance Foryou ✓ Diversification
  33. 33. @dodijkBlendle Relevance Foryou ✓ Similarity Diversification
  34. 34. @dodijkBlendle Relevance Foryou ✓ Similarity ✓ Diversification
  35. 35. @dodijkBlendle Relevance Foryou ✓ Similarity ✓ Diversification
  36. 36. @dodijkBlendle Relevance Foryou ✓ Similarity ✓ ✓ Diversification
  37. 37. @dodijkBlendle Relevance Foryou ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ Diversification
  38. 38. @dodijkBlendle Breaking Bubbles Editorial selection Must reads Diversification Little effect of popularity Filter Bubbles Foryou MustReads Making Bubbles Onboarding Reading history Explicit feedback
  39. 39. Timing problem • Our editors wake up at 5am and are done reading at 7am • Which is also when we want to send our newsletter • We simply can’t wait for a batch process
  40. 40. @dodijk article published articles enrich article enrich article enrich article articles updates persist update articles picks editor picks article create pick update picked articles article features article features users user article features user article features ranker A ranker B ranker Z user article scores persist scorescores filled nightly witha batch process scores arriveafter seconds #users x #updates = 200M #users x #updates x #rankers = 600M #updates = 200
  41. 41. personalize bundle @dodijk scores editor sends newsletter newsletters users select users user newsletters personalized newsletter send newsletter computed realtime filled with abatch process #newsletters = 1 #newsletters x #users = 1M #newsletters x #users = 1M ~7am ~7.15am personalizepersonalize bundle experimentation
  42. 42. Cold start problem • So we enrich our content
  43. 43. Timing problem • So we precompute as much as possible
  44. 44. Explanations: #FATREC Why this article? Because you seem to like a long read article every now and then. @dodijk
  45. 45. @dodijk research.blendle.io

×