1.
Bringing Science to the Art of Retail SM
Modeling Choice with
Limited Data
Vivek Farias, CTO @ Celect
Robert N. Noyce Professor @ MIT

2.
An Important Question
©2015 Celect, Inc. All Rights Reserved.2

3.
An Important Question
Comparisons are eas(ier) …
… but also, comparisons are everywhere.
©2015 Celect, Inc. All Rights Reserved.3

4.
Comparisons Are Everywhere
©2015 Celect, Inc. All Rights Reserved.4
Traditional Reviews: Peter prefers Tamarind to Cafe Boulud
Peter G.
Peter G.’s Profile
About Peter G.
Peter G.

5.
Comparisons Are Everywhere
5
Click Stream
Data©2015 Celect, Inc. All Rights Reserved.

6.
Comparisons Are Everywhere
6
Click Stream Data: Vinay is more in the mood for The
Expendables 3 vs. Cosmos or The Bank Job or ... ?
©2015 Celect, Inc. All Rights Reserved.

7.
Comparisons Are Everywhere
©2015 Celect, Inc. All Rights Reserved.7
(Brick and Mortar) Retail Data

8.
Comparisons Are Everywhere
8
(Brick and Mortar) Retail Data: I prefer Neutrogena to Dove,
and Aveeno, and …
©2015 Celect, Inc. All Rights Reserved.

9.
Two Applications
• Helping a department store build “hyper-local” experiences
‒ Fully data driven approach to modeling choice
• Helping a top 10 US Retail website personalize to intent
‒ A new view of what personalization online means
9
“Harvesting and Learning From
Comparisons”
©2015 Celect, Inc. All Rights Reserved.

10.
A Mental Picture Of A Choice Model
10
Representative Customer
©2015 Celect, Inc. All Rights Reserved.

11.
A Generic Choice Model
11
30% 30% 7% 7% 20% 6%
©2015 Celect, Inc. All Rights Reserved.

12.
A Generic Choice Model
12
= 30+30 = 60%
30% 30% 7% 7% 20% 6%
©2015 Celect, Inc. All Rights Reserved.

13.
A Generic Choice Model
13
30% 30% 7% 7% 20% 6%
©2015 Celect, Inc. All Rights Reserved.

14.
A Generic Choice Model
14
= 30+30+20 = 80%
30% 30% 7% 7% 20% 6%
©2015 Celect, Inc. All Rights Reserved.

15.
A Generic Choice Model
15
$200
$300
$100
30% 30% 7% 7% 20% 6%
©2015 Celect, Inc. All Rights Reserved.

16.
A Generic Choice Model
16
$100 $100$200 $200 $300 $300
30% 30% 7% 7% 20% 6%
©2015 Celect, Inc. All Rights Reserved.

17.
A Generic Choice Model
17
30% 30% 7% 7% 20% 6%
$100 $100$200 $200 $300 $300
©2015 Celect, Inc. All Rights Reserved.
$200 $300

18.
A Generic Choice Model
18
$200 $300$200 $200 $300 $300
30% 30% 7% 7% 20% 6%
©2015 Celect, Inc. All Rights Reserved.

19.
Revenue Estimation
19
?
Challenge: How to deal with sparse data & a high dim model?
Traditional Approach: Assume comes from a parametric family
©2015 Celect, Inc. All Rights Reserved.
Generic Model Limited Data
Predicted Rev
Rate

20.
Revenue Estimation
20
Challenge: How to deal with sparse data & a high dim model?
Our Approach: Assume that comes from a sparse model
©2015 Celect, Inc. All Rights Reserved.
Generic Model Limited Data
?Predicted Rev
Rate

21.
Our Approach
21
“Simple” “Conservative”
©2015 Celect, Inc. All Rights Reserved.
What do we know?

22.
Case Study: Choice Modeling + Assortments
• Mid-Size Retailer (~$4B in revenue. Close to 300 stores)
• Task
‒ Optimize assortment using our approach to learn choice patterns at
individual stores
• Data
‒ In-store inventory + transactions over time
‒ Online browse/ transactions
• Controlled experiment at 10 representative stores (Q3 14)
22 ©2015 Celect, Inc. All Rights Reserved.

23.
Case Study: Choice Modeling + Assortments
©2015 Celect, Inc. All Rights Reserved.23
Test Stores Control Stores
08/03 - 09/27
06/15 - 07/26
Test Period

24.
Case Study: Choice Modeling + Assortments
©2015 Celect, Inc. All Rights Reserved.24
Test Stores Control Stores
08/03 - 09/27
06/15 - 07/26
Trailing 6
Weeks

25.
Case Study: Choice Modeling + Assortments
©2015 Celect, Inc. All Rights Reserved.25
5.71% -2.91%
-.54% -2.51%
Test Stores Control Stores
08/03 - 09/27
06/15 - 07/26
Test Stores Net Relative
5.71 - -0.54 = 6.25%

26.
Case Study: Choice Modeling + Assortments
©2015 Celect, Inc. All Rights Reserved.26
5.71% -2.91%
-.54% -2.51%
Test Stores Control Stores
08/03 - 09/27
06/15 - 07/26
Control Stores Net Relative
-2.91 - -2.51 = -0.40%

27.
Case Study: Choice Modeling + Assortments
27
5.71% -2.91%
-.54% -2.51%
Test Stores Control Stores
08/03 - 09/27
06/15 - 07/26
Adjusted Net Relative
6.25 - -0.40 = +6.65%
©2015 Celect, Inc. All Rights Reserved.

28.
Case Study Summary
• Celect modifiers were applied to a limited portion of each test
store
• Adjusting against control stores and for trailing 6-wk trends:
‒ Celect modified departments yielded an incremental 6.65% increase in
revenue
‒ We predicted 6.43%, so our performance is predictable
• Test stores in total yielded an incremental 2.98% increase in
revenue potentially due to positive spillover effects of a better
assorted store
©2015 Celect, Inc. All Rights Reserved.28

29.
Personalization
29 ©2015 Celect, Inc. All Rights Reserved.
“If you liked this then you will also like these other things”

30.
Personalization
30
“If you liked this then you will also like these other things”
• What does it mean to personalize online?
• Explosion of ad-hoc heuristics -- tailor made to application
• All have a ‘nearest neighbors’ flavor
• How does choice play a role?
©2015 Celect, Inc. All Rights Reserved.

31.
Personalization
31
“If you liked this then you will also like these other things”
• What does it mean to personalize online?
• Explosion of ad-hoc heuristics -- tailor made to application
• All have a ‘nearest neighbors’ flavor
• Our Approach: Build a choice model, and optimize assuming
choice
©2015 Celect, Inc. All Rights Reserved.

32.
Implementation at a Gigantic Retailer
• Retailer (~$36B in revenue. ~ 10 Monthly Uniques)
• Task
‒ Personalized offerings for search and checkout
‒ < 5 ms RTT tolerance
• Data
‒ Online browse/ transactions
‒ CRM
• Live on traffic
©2015 Celect, Inc. All Rights Reserved.32

33.
Implementation at a Gigantic Retailer
©2015 Celect, Inc. All Rights Reserved.33
Category Incumbents Celect Increase
Appliances $4.38 $5.57 27.17%
Clothing $0.64 $0.77 20.31%
Lawn & Garden $3.26 $4.50 38.04%
Tools $1.36 $1.56 14.71%
Toys $0.41 $0.67 63.41%
Outdoor Living $4.49 $5.03 12.03%
Total $2.86 $3.43 19.93%
20% increase in
monetizing site traffic.
Distinct incumbents in
each category

34.
In Summary
• Understanding choice is valuable
• We can model choice by learning from atomic comparisons
‒ Comparisons are everywhere
• Data-Driven Choice modeling can re-shape offline retail functions
‒ Hyper-Localized Buying, Planning and Allocation
• Data-Driven Choice modeling can re-shape personalization
‒ Understanding choice beyond nearest-neighbor algorithms
34 ©2015 Celect, Inc. All Rights Reserved.

35.
A Top Strategic Initiative
"Finally, our localization initiative is important to our growth plan as we look to offer
increasingly local-centric assortments. We are testing new software called Celect, a
predictive modeling tool that allows us to leverage customers' omnichannel
purchase and browse behavior to identify merchandise localization opportunities.
We believe that localization can allow us to further increase market share in our
low-volume doors.”
Kathy Bufano, CEO, Bon-Ton Stores
BONT Earnings Call, May 21, 2015
©2015 Celect, Inc. All Rights Reserved.35

36.
Thank you!
©2015 Celect, Inc. All Rights Reserved.36
Live Webcast – October 15th at 1pm ET
www.celect.com/screaming-data-webcast
Visit blog.celect.com
Follow us on Twitter! @celect