Martijn Willemsen, Bart Knijnenburg, Mark Graus, Linda Velter-Bremmers, Kai Fu

1. Using latent features diversification
to reduce choice difficulty in
recommendation lists
Martijn Willemsen
Together with:
Mark Graus, Bart Knijnenburg
Linda Velter-Bremmers and Kai Fu

2. Remember this?

3. Choice Overload in Recommenders
Recommenders reduce information overload…
But large personalized sets cause choice overload!
Less attractive
30% sales More attractive
Higher purchase
satisfaction 3% sales
From Iyengar and Lepper (2000)
Satisfaction decreases with larger sets as increased
attractiveness is counteracted by choice difficulty

4. Satisfaction and item set length
More options provide more benefits in terms of
finding the right option
But result in higher opportunity costs
More comparisons required
Increased potential regret
Larger expectations for larger
sets
Paradox of choice
(Barry Schwartz)

5. Choice Overload in Recommenders
(Bollen, Knijnenburg, Willemsen & Graus, RecSys 2010)
Top-20 Lin-20
vs Top-5 recommendations vs Top-5 recommendations
.401 (.189) -.540 (.196) -.633 (.177) .938 (.249)
p < .05 p < .01 p < .001 p < .001
+ + - +
perceived recommendation + perceived recommendation + choice
variety .449 (.072) quality .445 (.102) difficulty .496 (.152)
p < .001 p < .001 p < .005
+ +
.170 (.069) .172 (.068) .346 (.125) -.217 (.070)
p < .05 p < .05 p < .01 p < .005
Choice satisfaction + -
0.5
0.4 movie choice
Objective System Aspects (OSA)
0.3 expertise satisfaction Subjective System Aspects (SSA)
0.2 Experience (EXP)
Personal Characteristics (PC)
0.1
Interaction (INT)
0
-0.1
Top-5 Top-20 Lin-20

6. Research on Choice overload
Choice overload is not omnipresent
Meta-analysis (Scheibehenne et al., JCR 2010)
suggests an overall effect size of zero
Choice overload stronger when:
No strong prior preferences
Little difference in attractiveness items
In consumer literature, most item
sets are not personalized…
Prior studies did not control for
the diversity of the item set

7. Choice Difficulty and Diversity
Larger sets are often more difficult because of the
increased uniformity of these sets (Fasolo et al., 2009;
Reutskaja et al., 2009)
High Density
Larger item sets have many small tradeoffs
similar options 200
# text messages
small inter-product distances 150
and small tradeoffs 100
High density! 50
Choice Difficulty related to 0
50 150 250 350
lack of justification minutes

8. Choice difficulty and trade-offs
As item sets become more diverse (less dense)
tradeoff size increases
High Density Low Density,
small tradeoffs large tradeoffs
200 200
# text messages
# text messages
150 150
100 100
50 50
0 0
50 150 250 350 50 150 250 350
minutes minutes
Tradeoffs are effortful…
give up one aspect for another
But can be justified very easily!

9. Double Mediation Model for difficulty
(Scholten and Sherman, JEP:G 2006)
U-shaped relation between diversity and difficulty:
Choosing from uniform set is
hard to justify but has no uniform diverse
Difficulty
difficult tradeoffs
Choosing from a diverse set
encompasses difficult tradeoffs
but is easy to justify
Diversity / tradeoff size
Does this also apply to personalized item sets?
Can we use diversity to reduce choice overload?
And to what level of diversity?

10. User Study
Manipulate Diversity in a personalized item set
while keeping attractiveness constant!
Recommenders provide the perfect tools for this…
Latent Features as means of diversification
Factorization algorithms describe movies and users as
vectors on a set of latent features
Preference dimensions related to real word concepts
(e.g. Escapist/serious) (Koren, Bell and Volinsky,2009)
Parallel to how choice sets are described in
MAUT(multi-attribute utility theory)

11. Matrix Factorization
Usual Suspects
The Godfather
Die Hard
Jack ? Titanic ?
Dylan ? ?
Olivia
Mark ? ? ?

12. Diversifying attractive items
Olivia Dylan
Jack
Mark

13. Diversity manipulation in detail
10-dimensional MF model
Personalized top 200
(close in attractiveness)
Low: closest to centroid
Greedy algorithm
Select N movies with
highest inter-item distance
Medium:
100 items closest to centroid
High: from top 200

14. System characteristics
MF recommender based on MyMedia project
10M MovieLens dataset: movies from 1994
5.6M ratings for 70k users and 5.4k movies
RMSE of 0.854, MAE of 0.656
Movies shown with title and predicted rating:
hovering the mouse over the title reveals additional
information: short synopsis, cast, director and image

15. Experimental design/procedure
Pre-questionnaire
Personal characteristics
Rating task to train the system (10 ratings)
Assess three lists of recommendations
Within subjects: low / mid / high diversification
Between subjects: number of items (5,10,15,20,25)
After each list we measured:
Perceived Diversity & Attractiveness
Expected Trade-Off Difficulty & Choice Difficulty

16. Pre-questionnaire
Strength of preference
3 items, e.g. “I know what kind of movies I like”
Movie expertise
4 items, e.g. “Compared to others I watch a lot of
movies”
Maximizing tendency
2 items, e.g. “I try to reach the highest standards in
everything I do”

17. After each list
Perceived recommendation diversity
5 items, e.g. “The list of movies was varied”
Perceived recommendation attractiveness
5 items, e.g. “The list of recommendations was
attractive”
Choice difficulty (indicator)
“I would find it difficult to choose a movie from this list”
Tradeoff difficulty (indicator)
“I had to put a lot of effort into comparing the different
aspects of the movies”

18. Participants and Manipulation checks
97 Participants from an online database
Paid for participation
Mean age: 29 years, 52 females and 45 males
Low, medium and high diversification
differed in the feature score range
Average predicted ratings of the sets were not different!
Average feature
score range Predicted SD predicted
(AFSR) rating rating
diversity mean (SE) mean (SE) mean (SE)
Low 0.959 (0.015) 4.486 (0.042) 0.163 (0.010)
Medium 1.273 (0.016) 4.486 (0.041) 0.184 (0.011)
High 1.744 (0.024) 4.527 (0.039) 0.206 (0.013)

19. Results Structural Equation Model
attractiveness diversity
standardized score
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
low mid high
diversification
choice diff. tradeoff diff.
0.2
scale difference
0
-0.2
-0.4
-0.6
-0.8
-1
low mid high
diversification

20. Conclusions & Discussion
choice diff. tradeoff diff.
Diversifying on latent features 0.2
scale difference
0
-0.2
Increases attractiveness/diversity -0.4
-0.6
Reduces trade-off difficulty (high) -0.8
-1
low mid high
Reduces choice difficulty (linearly) diversification
No evidence for U-shaped difficulty model
High diversity does not result in trade-off conflicts
(perhaps due to the nature of the domain/MF?)
No effect of number on items
Small sets benefit as much from diversification
Diversification on MF features seems
promising to increase attractiveness!

21. Future Work
In this study, no actual choice was made
Explains limited effect of number of items
We could not measure choice satisfaction or
justification-based processes
We will use diversification with list length as two
factors in a new choice overload experiment
Item size: 5, 10 and 20
Low and high diversity
We expect choice overload to be more prominent
for low diversity sets