Users are constantly searching for new content and to stay competitive organizations must act immediately based on up-to-date data. Outdated recommendations decrease the likelihood of presenting the right offer and make it harder to maintain customer loyalty. In order to provide the most relevant recommendations and increase engagement, organizations must track customer interactions and re-score recommendations on the fly.
Data sources have expanded dramatically to include a wealth of historical data and a constant influx of behavior data. The key to moving from predictive models, applied in batch, to models that provide responses in real time, is to focus on the efficiency of model application. The speed that recommendations can be served is influenced by:
Architecture of the recommendation serving platform
Choice of recommendation algorithm
Datastore access patterns
In this presentation, we’ll discuss how developers can use open source components like HBase and Kiji to develop low-latency recommendation models that can be easily deployed by e-commerce companies. We will give practical advice on how to choose models and design data stores that make use of the architecture and quickly serve new recommendations.
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Real-time Recommendations for Retail: Architecture, Algorithms, and Design
1. Juliet Hougland and Jonathan Natkins
REAL-TIME RECOMMENDATIONS FOR RETAIL:
ARCHITECTURE, ALGORITHMS, AND DESIGN
2. Who Are We?
Jonathan Natkins
Field Engineer at
WibiData
Before that, Cloudera
Software Engineer
Before that, Vertica
Software/Field Engineer
Juliet Hougland
Data Scientist, previously
at WibiData
MS in Applied Math
BA in Math-Physics
6. Recommender Contexts
Taste History
Based on everything you know about a user
Interests over months/years
Current Taste
Based on a user’s immediate history
Interests over minutes/hours
Ephemeral
Extreme version of current taste
For example, location
Demographic*
Similar to taste history, but less subjective
Geographic region, age bracket, etc.
9. Requirements for a Real-Time System
General System Requirements
Handle millions of customers/users
Support collection and storage of complex data
Static and event-series
Real-Time System Requirements
Quickly retrieve subsets of data for a single user
Aggregate/derive new, first-class data per user
13. How Can We Make Real-Time Models?
Population interests
change slowly
Individual interests
change quickly
14. How Can We Make Real-Time Models?
Population interests
change slowly
Models don’t need
to be retrained
frequently
Individual interests
change quickly
15. How Can We Make Real-Time Models?
Population interests
change slowly
Models don’t need
to be retrained
frequently
Application of a model
should be fast
Individual interests
change quickly
16. A Common Workflow
Train a model over
the entire dataset
Save fitted model
parameters to a file or
another table
Access the model
parameters when
generating new
recommendations
based on new data
This is
EXPENSIVE
17. Developing Models
KijiExpress
Scala interface for interacting with Kiji data
Uses Scalding for designing complex dataflows
Model Lifecycle
Allows analysts and data scientists to break apart
a model into phases
19. Scoring Models in Real-Time
Batch isn’t real-time
Number of
Users
Number of Interactions
20. Scoring Models in Real-Time
Batch isn’t real-time
Number of
Users
A few users with
many interactions
Number of Interactions
21. Scoring Models in Real-Time
Batch isn’t real-time
A lot of users with
few interactions
Number of
Users
A few users with
many interactions
Number of Interactions
26. Fresheners Compute Lazily
Read a column
Get from HBase
Client
Freshness
Policy
Scorer
Yes, return to client
Write back for next time
KijiScoring Server
HBase
29. Kiji Model Repository
Link between application and models
Stores Freshener metadata
FreshnessPolicy, Scorer, attached column
Location of trained model
Stores Scorer code
Code repository makes model scoring code available
to the application from a central location
New models can be deployed to the Model
Repository and made immediately available to
the application
33. Content-Based Recommenders
Build models around entities using
features that we think reflect
inherent characteristics
Orange-Nosed
Lab Assistant
Meeps a lot
40. Similar Entities
What do we mean by similar?
Jaccard Index: a measure of set similarity
Cosine Similarity: the angle between two vectors
Pearson Correlation: statistical measure, similar to cosine
Naively, we could compare every entity to each other
…But that would not scale
will with increasing
numbers of entities
42. Collaborative Filtering: Is This Useful?
Problem: Too much data!
Tracking user preferences and all their events generates huge
amounts of data
Problem: Too little data!
Dimensions of user-space and item-space are usually very large
More variables makes it more difficult to generate user
preferences
Problem: Cold start
If you don’t know anything about a user, what should you
recommend?
Problem: More ratings means slower computations
Identifying neighborhoods of entities is expensive
43. Collaborative Filtering: Why Is It Useful?
Because it works
Content-agnostic
All that matters is co-occurrence of events
44. Amazon: Item-Item Collaborative Filtering
>
Used for personalized recommendations
Fill screen real estate with related items
Produces specific, but non-creepy
recommendations
Linden, G.; Smith, B.; York, J., "Amazon.com recommendations: item-to-item collaborative filtering," Internet Computing, IEEE , vol.7,
no.1, pp.76,80, Jan/Feb 2003
45. Item-Item Collaborative Filtering
Beaker buys a banana slicer
Then:
Generate list of candidate items to predict ratings for
Predict ratings for candidate items
Select Top-N items
46. Accessing External Data
KeyValueStore API enables external data access
when applying a model
External data might be…
Trained model parameters
Hierarchical/Taxonomic data
Geo-lookup
Store external data flexibly
Text files, sequence files, Kiji tables, etc.
Data access is decoupled from use during execution
If the data doesn’t fit in memory, put it in a table
47. How Much Less Work Can We Do?
We can choose a
predictor that allows
us to truncate a sum
There are two ways
terms in the sum of
our predictor can be
small
No rating
Small similarity
48. How Much Less Work Can We Do?
We can choose a
predictor that allows
us to truncate a sum
There are two ways
terms in the sum of
our predictor can be
small
No rating
Small similarity
49. How Much Less Work Can We Do?
We can choose a
predictor that allows
us to truncate a sum
Ignore unrated items
There are two ways
terms in the sum of
our predictor can be
small
No rating
Small similarity
50. How Much Less Work Can We Do?
We can choose a
predictor that allows
us to truncate a sum
Ignore dissimilar items
There are two ways
terms in the sum of
our predictor can be
small
No rating
Small similarity
51. How Much Less Work Can We Do?
If we only present a few recommendations,
we don’t need to predict ratings for all items
Choose your candidate set to estimate ratings
wisely or infer from nearest neighbors
54. Want to Know More?
The Kiji Project
kiji.org
github.com/kijiproject
Questions about this presentation?
Twitter: @JulietHougland or @nattyice
Email: natty@wibidata.com
Notas del editor
Natty, thanks for that great description of the infrastructure that Kiji provides. Now we want to take the next step- go from infrastructure to actually showing our customers items they may be interested in.1. Recommending a group of items. If you are taking the time to make predictions you may as well present people with many options. This means recommendations arent about finding the best item to recommend. It is about predicting to best group of items. This gives us some leeway in terms of exact value.2. Users often will not be logged in at first. If we want to present good recs we need to be able to do it based on their current sessions browsing history. You can’t have personalized recs for non logged in users without real-time recs.3. Online retails want to present vast catalogs to their large user base. For most online retail sites the point of recommendations is organizing information in a way that is relevant and useful to their customer.
I want to give us a broad overview of the types of approaches to recommendations available. We have got a finite amount of time here, so I will focus a lot of what I say on a simple implementation of collaborative filtering. I don’t want to give the impression that it is the only, or absolute best solution to the recommendations problem. Like with any prediction problem, there are many ways to tackle retail recommendation.There are two main types of recommendation algorithms. In a realistic system, they willalways be used together
Use item descriptions, user generated tags, expert generated tags (Pandora) in order to build representations.A major pro of content based models is that they better handle unrated items. Good way to get around the cold start problem for a rec systems. Good way to bootstrap your way to getting user ratings or augment other methods of recommendation.The down side is that processingand building models around textual information can be very challenging.
Just look at it.In a content based system, the hope is that the content you are basing your recs on is a good indicator of other items that are related in a relevant way.So, if we had a good content based recommender, after observing an interest in a banana slicer, it would recommend that you trying using a butter knife quickly.
Pandora is the first recommendation system I remember consciously interacting with.Pandora: Expert TaggingThey have a team of musicians (domain expertise in invaluable) to listen to music and apply tags to songs.From a seeded station they begin to present to you variations on the original attributes of the song you started with. Your likes and dislikes as expressed to their systems helps it learn what attributes you like and dislike.Expert tagging is expensive and is a bottleneck in introducing new items.
In collaborative filtering recommendation algorithms, we base our prediction based purely on expressed preferences.we think of storing user-item ratings as a matrix where the rows correspond to users and the columns correspond to items. We collectexplicit ratings and record them. Unfortunately, people don’t provide many ratings. Also, people lie.
Gather feedback as explicit ratings or implicitly through user behavior (page views, put in shopping cart, starred/saved for later, bought)Lots of work in rec systems has been done around explicitly rated items.People lying about their preferences. They are aspirational.I put ken burns in my queue, but I watch a lot of the deadliest catch.New data can be added incrementally to the model.
We can rely on implicit affinities for items instead of explicit ratings. We can track viewed or bought items and use a unary representation.Meaning, the happen, or… nothing, null, the void.
Users that have expressed similar taste in the past should express similar taste in the future. (We represent users as a vector of ratings for items.)Items that have had similar profiles of user interest should continue to appeal to a similar collection of users. (we represent items as a vector of ratings by user.)For a target entity, we predict the unknown rating using information from other similar entities. A simple and common approach is to take the weighted average of ratings, where weights are some function of the similarity between entities. r_{i} = sum_{j}w_{ij}r_{j}
identify items to select recommendations fromgenerate predicted scores, often through weighted averages in neighborhoods.return a list of top rated items
- we had the cold start problem => content based recs- where do we keep our data? What do we need access to when we generate recs?design tablestrain model
Too much data!Too little data! Data sparsity. Large number of users and items makes it hard to get a good sample of the potential “taste space.” Did you see gravity? Did the emptiness fo space strike you? This is like that, but much much emptier.Cold start: Troublesome because it requires ratings. If your system has no recorded interactions or explicit ratings, you can’t do this. Usually systems will be bootstrapped from recommenders that don’t require having user interactions recorded.Content based recommenders. Use item descriptions or tags to infer similarity between items, or generate profiles for users.Use data volunteered by users during registration or pulled in from facebook profiles to begin recommending items.The more ratings data you have, the slower you computation goes.
Useful because it is content agnostic. Can be used for any variety of content, you just need items, users, and ratings. Can be used across languages. (If you are google, this is very important.)It performs well. It used used in many succesful commerical applications. Amazon, Netflix, Google. It just works well.
Conceptual reasons amazon users item-item CFAmazon has more users than items. Computationally cheaper to focus model building around item relationships since their are less items.The relationships between items is also often simplerCan be used for personalized recs. It is especially useful when the only information you have about your user is a few items they have viewed in their current session.Item based CF is specific in the types of items is recommends, user based CF is more serendipitous in the types of items it recommends.Less creepy to get recommended very similar items to the one you are currently looking at than to have an accurate prediction when people don’t think Can fill screen real estate with similar items easily. “Customer who bought X also bought..” You are already doing the needed computation during the model training phase.
Use banana slicer + banana slicer pile pic and eq hereSteps in generating rec:1. We don’t need to estimate rating for every item in the catalog if we will only present a few recs.Choose your candidate set to estimate ratings wisely or infer from nearest neighbors.2. Precompute item-item similarities- (N^2)M operation. In practice NM since ratings are sparse. At scoring time, aggregate ratings and similarities to predict ratings for unrated items.How can you organize the model data in such a way that what you need to generate predictions is accessible at scoring time?
We need to be able to quickly access item-item similarities on a per item basis. How can we do this, quickly?Kiji provides the ability to access outside data source while freshening, running a MapReduce job, or testing through the KeyValue Stores. The KeyValueStore interface has many existing useful implementations you may use, or you may define your own custom one. Depending on you access need and total size of the data you need access to you may use a file backed KeyValue Store, or another KijiTable itself.Since item-item CF as we have stated it requires that we are able to access all item-item similarity pairs, our best choice is to use another Kiji Table to store this information. Accesing this information quickly then becomes an issue of table layout design.
no rating => we should be able to query item-item similarities on a per items basis and
Organization of data in your tables depends on your prediction function. We can see that in standard neighborhood based interpolation in CF that we need to be able to access all of a users ratings.Two tablesUsers table contains user info, product ratings, views, purchases, etc.Products table contains product info, and will be augmented with related/similar products
Organization of data in your tables depends on your prediction function. We can see that in standard neighborhood based interpolation in CF that we need to be able to access all of a users ratings.