This document provides an overview of using deep learning techniques for recommender systems. It begins with establishing the need for recommender systems due to increasing information overload. It then gives a basic introduction and agenda for the talk, covering motivation, basics, deep learning for vehicle recommendations, and scalability/production. The talk discusses using deep learning approaches like wide and deep learning as well as sequential models to improve recommendation relevance for applications like vehicle recommendations. It provides details on preprocessing, training a classifier, candidate generation and ranking for recommendations. The document concludes with discussing deploying such a system at scale and current trends in recommender system research.
3. 3
1. Motivation
2. Basics and Overview
3. Deep Learning for Vehicle Recommendations
4. Scalability and Production
Agenda
4. 4
Annual Data Sphere increases exponentially
International Data Corporation: Data Age 2025 study, April 2017
Information Load
à Humans
Human Processing
Capacity
5. 5
Information and Choice Overload
https://www.linkedin.com/pulse/its-information-overload-filter-failure-productivity-industry-zayats/
https://en.wikipedia.org/wiki/Clay_Shirky
“It‘s not information overload. It‘s filter failure." - Clay Shirky
6. 6
- Covington et al.
2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019
Google Trends for
“Deep Learning“
“Deep Learning becomes a
general-purpose solution for
nearly all learning problems."
9. 9 Gomez-Uribe, Carlos A. and Hunt, Neil: The Netflix Recommender System: Algorithms, Business Value, and Innovation (2015)
„Our recommender system […]
in total influences choice for about
80% of hours streamed at Netflix.
The remaining 20% comes from search
[...]“
Suche
Empfehlungen
Recommendations
Search
10. 10 Gomez-Uribe, Carlos A. and Hunt, Neil: The Netflix Recommender System: Algorithms, Business Value, and Innovation (2015)
„Reduction of monthly churn both increases the lifetime value of an existing
subscriber, and reduces the number of new subscribers we need to acquire to
replace cancelled members.
We think the combined effect of
personalization and recommendations
save us more than $1B per year.“
Suche
Empfehlungen
11. 11
1. Motivation
2. Basics and Overview
3. Deep Learning for Vehicle Recommendations
4. Scalability and Production
Agenda
20. see Slide on References, Details: https://bit.ly/2WuS4Zq
Domains and Types for DLRS
20
DNNs
CNNs
RNNs
AEs
Other
Other
2017
2018
2009
2015 2015
2017
2016
2015
2018
2016
2013
2018
2018
2017
2018
2018
2018
2018
2018
2017
https://bit.ly/2WuS4Zq
21. Cheng, Heng-Tze et al.: Wide and Deep Learning for Recommender Systems (2016)
Wide and Deep Learning for App-Recos
Combine Memorization and Generalization
21
22. Cheng, Heng-Tze et al.: Wide and Deep Learning for Recommender Systems (2016)
Wide and Deep Learning for App-Recos
Combine Memorization and Generalization
22
Deep
Component
Embeddings
Wide
Component
23. Session-based Recommendations
Leverage Sequential Information to Improve Relevance
www.netflix.com23
t
DESIGNATED
SURVIVOR
DARK
DESIGNATED
SURVIVOR
DARK
› HOUSE OF CARDS
› STRANGER THINGS
› HOUSE OF CARDS
› STRANGER THINGS
STRANGER
THINGS
HOUSE OF
CARDS
24. Session-based Recommendations
Leverage Sequential Information to Improve Relevance
Quadrana et al.: Personalizing Session-based Recommendations with Hierarchical Recurrent Neural Networks (2017)24
25. 25
1. Motivation
2. Basics and Overview
3. Deep Learning for Vehicle Recommendations
4. Scalability and Production
Agenda
33. 33
categorical features
many-hot-encoding one-hot-encoding
feature values
ucat icat
eclimatisation
icont
embeddinguser
consumption first_reg price...
embeddingi, cont
ucont
embeddingu,cont
...
outlier removal
z-normalisation
ELU (256)
ELU (128)
ELU (64)
embeddingitem
...
...
climatisation color
ecolor etransmission
transmission
Probability that user u
likes vehicle i
meanconsumption meanprice
stddevconsumption stddevprice
...
concatenateconcatenate
PreprocessingEmbeddingDeepComponent
outlier removal
z-normalisation
34. 34
categorical features
many-hot-encoding one-hot-encoding
feature values
ucat icat
eclimatisation
icont
embeddinguser
consumption first_reg price...
embeddingi, cont
ucont
embeddingu,cont
...
outlier removal
z-normalisation
ELU (256)
ELU (128)
ELU (64)
embeddingitem
...
...
climatisation color
ecolor etransmission
transmission
Probability that user u
likes vehicle i
meanconsumption meanprice
stddevconsumption stddevprice
...
concatenateconcatenate
PreprocessingEmbeddingDeepComponent
outlier removal
z-normalisation
UserNet ItemNet
RankNet
35. minimize
minimize
Adam Optimizer: Stochastic Gradient Descent with adaptive learning rate and adaptive momentum
Approach: Classifier Training
35 35
RankNet
eu
u
UserNet
ei
i
ItemNet
p ( i | u )
class_loss
sim_loss
Adam
Optimizer
Adam
Optimizer
43. Results: DLRS Recommendation Relevance
43 MAP: mean average precision, comparative results after optimization of hyperparameters
0,20%
0,30%
0,40%
0,50%
0,60%
0,70%
0,80%
0,90%
1,00%
1,10%
k = 1 k = 5 k = 10 k = 30 k = 100
MAP@k
Deep Learning
Hybrid CF-CBF (d=700)
CF (d=100)
1.10%
1.00%
0.90%
0.80%
0.70%
0.60%
0.50%
0.40%
0.30%
0.20%
"
+73%
+143%
44. 44
1. Motivation
2. Basics and Overview
3. Deep Learning for Vehicle Recommendations
4. Scalability and Production
Agenda
45. Deploying Vehicle Recommendations at Scale
45
item
storage
embeddings
RankNet
UserNet
ItemNet
ANNOY
ANN index
Candidate ServiceRanking Service
Webservice
User Profile API
Recommendation Service
k recommendations
rank candidates
{ei} for eu
get u
get eu
get T
candidates
{ei}
get i
get ei
index
all ei
ANN
search
46. 46
Deep Learning Solved – What’s next?
http://dlrs-workshop.org/wp-content/uploads/2018/10/dlrs2018_welcome.pdf
48. 48
"We can only see a short distance ahead,
but we can see plenty there that needs to
be done."
- Alan Turing
49. Thank You
Marcel Kurovski
Data Scientist
inovex GmbH
Kupferhütte 1.13,
Schanzenstr. 6-20
51063 Cologne
marcel.kurovski@inovex.de
+49 173 3181 088
Dr. Florian Wilhelm
Principal Data Scientist
Julian Hatzky
Data Science Working Student
50. References
50
[1] Quadrana, Massimo, Karatzoglou, Alexandros, Hidasi, Balázs, Cremonesi, Paolo. “Personalizing Session-based Recommendations with Hierarchical Recurrent Neural
Networks“ Proceedings of the 11th ACM Conference on Recommender Systems. 2017
[2] Cheng, Heng-Tze, et al. "Wide & deep learning for recommender systems." Proceedings of the 1st Workshop on Deep Learning for Recommender Systems. ACM, 2016.
[3] Covington, Paul, Jay Adams, and Emre Sargin. "Deep neural networks for youtube recommendations." Proceedings of the 10th ACM Conference on Recommender
Systems. ACM, 2016.
[4] Goodfellow, Ian, Yoshua Bengio, and Aaron Courville. Deep learning. MIT Press, 2016.
[5] Heaton, Jeff. Artificial Intelligence for Humans: Deep Learning and Neural Networks. 2015.
[6] Ricci, Francesco and Rokach, Lior and Shapira, Bracha. Recommender Systems Handbook. Springer-Verlag. 2015
[7] Reinsel, David, Gantz, John, Rydning, John. “Data Age 2025: The Evolution of Data to Life-Critical Don't Focus on Big Data; Focus on the Data That's Big“ International
Data Corporation (IDC). 2017
[8] Gomez-Uribe, Carlos A. and Hunt, Neil: The Netflix Recommender System: Algorithms, Business Value, and Innovation. 2015
[9] JP Mangalindan: Amazon's recommendation secret. 2012
[10] Christ Johnson: Algorithmis Music Discovery at Spotify. 2014
[11] Maya Hristakeva: Overview of Recommender Algorithms - Part 2. 2015
[12] Alex Gude: The Nine Must-Have Datasets for Investigating Recommender Systems. 2016
[13] Erik Bernhardsson: Approximate nearest news. 2016
[14] Balász Hidasi. 3rd Workshop on Deep Learning for Recommender Systems. 2018
[15] CartStack LLC: Comparison could be killing your online business. 2017
[16] Marina Zayats: “It‘s not information overload; it‘s filter failure.“ Productivity in the Industry 4.0. 2016