Más contenido relacionado Dynamic Pricing in Mobile Games1. © 2015. Company Confidential and Not for Redistribution. info@scientificrevenue.com 07/22/15 1
Dynamic Pricing In Mobile Games
July 22, 2015
2. © 2015. Company Confidential and Not for Redistribution. info@scientificrevenue.com 07/22/15 2
About Me
• CEO and Founder,
Scientific Revenue
• Previously CTO / SVP
Product for Live Gamer
(Payments Aggregator
focused on F2P Gaming )
• Long history in data and
artificial intelligence
3. © 2015. Company Confidential and Not for Redistribution. info@scientificrevenue.com 07/22/15 3
The Last AI Conference I Spoke At ….
4. © 2015. Company Confidential and Not for Redistribution. info@scientificrevenue.com 07/22/15 4
About Scientific Revenue
Scientific Revenue provides a dynamic price management solution for mobile
games that boosts in-app purchase revenue. We match the right prices with
the right players at the right times, to keep players engaged, increase
conversion, and grow profits for game publishers.
5. © 2015. Company Confidential and Not for Redistribution. info@scientificrevenue.com 07/22/15 5
What Heather Said
6. © 2015. Company Confidential and Not for Redistribution. info@scientificrevenue.com 07/22/15 6
Earlier Today ….
• The heavy lifting going on around knowing what
players are doing has to do with prediction and
classification
• Classic territory
• Systems today are more strongly on detection and
diagnosis, not action side
• We’re starting to get solid predictive analytics
• Scientific Revenue is about a control framework
• Well, that and some pretty nice machine learning.
• This talk is mainly about control frameworks.
7. © 2015. Company Confidential and Not for Redistribution. info@scientificrevenue.com 07/22/15 7
What is a Price
8. © 2015. Company Confidential and Not for Redistribution. info@scientificrevenue.com 07/22/15 8
An Offer To Sell a Good or Service for “Real Money”
9. © 2015. Company Confidential and Not for Redistribution. info@scientificrevenue.com 07/22/15 9
An Offer To Sell a Good or Service for “Real Money”
10. © 2015. Company Confidential and Not for Redistribution. info@scientificrevenue.com 07/22/15 10
Lots of Decisions
Different:
• Prices
• Coin amounts
• (volume discounts)
• Framing text and cues
• Default selection
• Different bonus types
Same:
• Call to action
11. © 2015. Company Confidential and Not for Redistribution. info@scientificrevenue.com 07/22/15 11
These are Also “Pricing Decisions”
12. © 2015. Company Confidential and Not for Redistribution. info@scientificrevenue.com 07/22/15 12
Not Just “How Much for How Much”
• Pricing decisions are also
• What additional inducements do you offer?
• When do you make the offer?
• What channels you make the offer in?
• What messages accompany the offer?
• How long the user has to act on the offer?
• ….
13. © 2015. Company Confidential and Not for Redistribution. info@scientificrevenue.com 07/22/15 13
The Problem
14. © 2015. Company Confidential and Not for Redistribution. info@scientificrevenue.com 07/22/15 14
Increase LTR (“R” = “Revenue”)
• Pricing optimization is a tool to revenue maximization
• Without causing adverse reactions
• NOT Looking at things at the “individual transaction
level”
• For games with very high churn and very short
retention times, these approaches overlap
• But if you’re keeping your users around and hoping for
more revenue later (second purchases, advertising, …)
then there are other aspects to consider
15. © 2015. Company Confidential and Not for Redistribution. info@scientificrevenue.com 07/22/15 15
LTR
• 20% of all purchases occurred on day 1
• All spending was done by day 40
• 27% of all first purchases occur on day 1
• 80% of all first purchases occur in week 1
• 49% of purchasers bought a second time
• Half of all purchases occurred in week 1
• Second purchases were the same size as first
purchases
16. © 2015. Company Confidential and Not for Redistribution. info@scientificrevenue.com 07/22/15 16
The Settled Science isn’t Very Useful
• Classical Economics involves pricing to the demand
curve (and, maybe, estimating demand curves using
multi-armed bandits)
17. © 2015. Company Confidential and Not for Redistribution. info@scientificrevenue.com 07/22/15 17
Reality is Actually …. Kind of Unsettled
• Training effects?
• Framing Effects?
• Volume discounts?
• Churn Impacts?
• Community Impact?
• Moral Hazard?
18. © 2015. Company Confidential and Not for Redistribution. info@scientificrevenue.com 07/22/15 18
Moral Hazard
This is absolutely disgusting. I'll be sure to tell everyone
about this creepy, exploitive tracking of players.
19. © 2015. Company Confidential and Not for Redistribution. info@scientificrevenue.com 07/22/15 19
Our Predecessors Made Many Simplifying Assumptions ….
• Non-negotiated pricing
• Flexible return policy
• Segmentable market demand
• Highly competitive markets / little or no vendor
loyalty
• Publically available ratecards
• Pre-existing anchoring on pricing and rates
• Infrequent, large-dollar amount purchases
• Customers return months or years later
• Low variable costs
• Fixed capacity
• Inventory can be changed from one product to
another
• Perishable inventory
20. © 2015. Company Confidential and Not for Redistribution. info@scientificrevenue.com 07/22/15 20
The Architecture
21. © 2015. Company Confidential and Not for Redistribution. info@scientificrevenue.com 07/22/15 21
In Block Form
22. © 2015. Company Confidential and Not for Redistribution. info@scientificrevenue.com 07/22/15 22
Three Distinct Requirements
• Data Collection
• This isn’t an algorithm problem. It’s a modeling and feature problem
and it requires a well designed data set informed by the machine
learning goals
• Control Framework
• The point is to change prices. By itself, that’s actually pretty hard to
do (in the “lot of code” sense)
• Asynchronous Evaluation Framework
• Most of our model building and training is done asynchronously
23. © 2015. Company Confidential and Not for Redistribution. info@scientificrevenue.com 07/22/15 23
Evaluation Framework: Global Cycle
Calibrate
Measure
Diagnose
Propose
Promote Test
24. © 2015. Company Confidential and Not for Redistribution. info@scientificrevenue.com 07/22/15 24
Evaluation Framework: Calibration
• Solving “Cold Start”
• Have a canned set of 70+ segments (that are “known”
to exhibit pricing and behavioral differences).
• Have a predefined set of 250+ additional features
• Have a diagnostic framework that can exhaustively
measure a large number of metrics and which can
evaluate features for predictive power
25. © 2015. Company Confidential and Not for Redistribution. info@scientificrevenue.com 07/22/15 25
Example: Part of Day / Day of Week
• Left: Number of new users by date and hour, lighter = more
• Right: Number of purchasers by date and hour, lighter = more
• People who join at noon are 4 times more likely to spend than people who join at 5 AM (in this
game)
26. © 2015. Company Confidential and Not for Redistribution. info@scientificrevenue.com 07/22/15 26
Evaluation Framework: Calibration
• Run for three weeks to train the models against the
initial segments and get baseline performance data
• Compare the initial segments to each other to get an
idea of variation and benchmark good performance
• Spot “underperformers” and “overperformers”
• Automated diagnostics to explore reasons why
27. © 2015. Company Confidential and Not for Redistribution. info@scientificrevenue.com 07/22/15 27
Directional Metrics
• Traditional KPIs can indicate issues,
but don’t help much with action
• ARPU dropping? What is the
automated, or partially automated,
outcome?
• Part of the power of our approach
comes from putting features against
finer-grained behavioral metrics
• And then automating the sifting
• Example:
• Purchase Index
• Default Acceptance Ratio
• Upsell %
• Downsell %
28. © 2015. Company Confidential and Not for Redistribution. info@scientificrevenue.com 07/22/15 28
Evaluation Framework: Segmentation
• Predictive Analytics
• Churn Prediction, Likely To Purchase, Potential Whale, …
• Custom Models
• Disposable Income, Affluence, Gamerness, Mobile Native Ness, …
• “Possibly Important” Features
• Lots of these
• Propose Segments and Pricing Policies
• Based on important features, create segments and compare
29. © 2015. Company Confidential and Not for Redistribution. info@scientificrevenue.com 07/22/15 29
Evaluation Framework: User Lifecycle
(start) Join
Initial
Profile
Baseline
Modeling
Baseline
Prediction
Observe Adjust
Initial
Pricing Set
Reprice
30. © 2015. Company Confidential and Not for Redistribution. info@scientificrevenue.com 07/22/15 30
Dealing With Intuitive Ideas
31. © 2015. Company Confidential and Not for Redistribution. info@scientificrevenue.com 07/22/15 31
Intuitive Physics
32. © 2015. Company Confidential and Not for Redistribution. info@scientificrevenue.com 07/22/15 32
Intuitive Economics
33. © 2015. Company Confidential and Not for Redistribution. info@scientificrevenue.com 07/22/15 33
What is Stickershock
A feeling of surprise and disappointment caused by
learning that something you want to buy is very
expensive.
Astonishment and dismay experienced on being
informed of a product’s unexpectedly high price.
34. © 2015. Company Confidential and Not for Redistribution. info@scientificrevenue.com 07/22/15 34
Formalizing Stickershock
Before:
• D0 to D3 timeframe
• A user visits a payment wall (or purchase opportunity) early in their lifecycle (and unusually
early)
• We have other signals that they are likely to buy (usually behavior-oriented)
During:
• They don’t buy
• They abandon relatively quickly
After:
• They don’t come back to the payment wall
• They either grind or leave the game entirely
Supporting Evidence:
• Low affluence signals
35. © 2015. Company Confidential and Not for Redistribution. info@scientificrevenue.com 07/22/15 35
Options for Dealing with Stickershock
• Give out more currency early
• Initial Framing Offer
• Targeted Intervention
• Reorder baseline prices and reset default
• Different set of Baseline Prices
• Different set of Baseline Prices with windowing