Date: 15th November 2017 Location: AI Lab Theatre Time: 11:10 - 11:40 Speaker: Alexandre Hubert Organisation: Dataiku

1. Marketing attribution
Big Data London 2017

2. Marketing departments are more important than ever to serve the ambitions of
organisations.
NG LEADERS MUST DEMONSTRATE THE RETURNS THEY

3. After 3 years of consecutive growth,
marketing budget
growth
has stalled
67%
of CMOs plan to increase
their spending in Digital
advertising
Numbers and Insights - Taking action

4. Marketing executives are required to track their spending.
Transforming the business
from this … to that

5. Marketing attribution: transforming a business question into a data science
problem
• How can I quantify the
influence an advertisement
has on a customer decision to
make a purchase?
• How can we measure the
effectiveness of each channel?

6. Marketing attribution: which channel should be rewarded ?

7. Marketing attribution: heuristic modelling
• Arbitrarily decide to reward on
segment of the journey
Last click ?
Major problem: will introduce a bias
towards retargeting. What if my customers
are not that sensitive to retargeting but
more to a good old Google search?
First click ?
Major problem: will introduce a bias
towards Google search. What if my
customers are sensitive to social media ?

8. Marketing attribution: algorithmic modelling
• All rule-based models
introduce some human bias
whatever the « complexity » .
• How can we use data
science to solve that
problem?

9. Introducing deep recurrent artificially
convoluted networks for distributed machine
learning models in marketing attribution
• How does that sound? Cool? Smart? Trendy?
• Data science solves well-framed problems in a well-framed manner.
• Weird…? Too complicated? Indeed. And it should not.

10. Attribution Modelling = Track your channels + Record the conversions
1. Markov Chain Modeling
3. Game theory and Shapley value
2. How to validate the best approach ?
Define and validate a model

11. A good data science solution starts … with good data
Prerequisites
• Track all your user
actions on each
targeted channel
• Leverage all the
progress IT has done
recently.
• And keep in mind that
there will certainly be
some limitations and
missing data

12. Marketing attribution: Markov Chain modelling
The methodology: from raw log files to a Markov Model
• From user actions database, compute each
customer journey
• From each customer journey, compute the
Markov transition matrix of a typical
customer
• From the Markov transition matrix, compute
the standard likelihood to convert
• From the standard likelihood to convert,
compute the removal effect of each
channel.
• Really hard to do A/B
testing in practice
• Too expensive and risky
to stop a channel
• Sessionalization in database (30
minutes windows)
• Distributed aggregation with
Spark

13. Marketing attribution: Markov Chain modelling
C1
(start)
C2 C3
(Null)
(Conv)
Computing the standard likelihood to convert
66.7%
33.3%
50%
100%
50%
50%
Let’s compute P(conv) which is the likelihood of any given customer to
convert.
50%

14. Marketing attribution: Monte Carlo Markov Chain
C1
(start)
C2 C3
(Null)
(Conv)
Computing the removal effect of a given channel
66.7%
33.3%
100%
50%
50%
Let’s compute how the likelihood of conversion will be affected when we
remove a channel P(conv_without_C1)
Removal effect = P(conv_standard) - P(conv_without_C1)/P(conv_standard)

15. Marketing attribution: Monte Carlo Markov Chain

16. Marketing attribution: Markov Chain modelling
Each channel has a removal effect; we can
now
• Compare them over time
• Spend the budget based on the actual
effect of each one
Easy, robust and fast technique
• can easily be deployed in production
• lead to quick win in the organisation
Summary
Currently some live implementations and
on our own data and some of our clients.
Promising results :)
By the way, here is a non-exhaustive list of
our happy customers (where we might have tested this
approach)
Consumer Goods E-Retail
Travel & HospitalityServices

17. Game theory and Shapley Value

18. How much value does a
channel bring to an already
existing group of channels ?
Marketing attribution: Game theory and Shapley Value
Facebook + Twitter = 2% of
conversions.
What if we add SnapChat
down the line ?
2 %
3 %
Increased likelihood
of purchase by 50%
Likelihood
of purchase
Customer journey
leading to conversion

19. Marketing attribution: Game theory and Shapley Value
Methodology and mathematical reasoning
• From user actions database, compute
each customer journey and the
number of conversions.
• Compare the added value of a
channel on the number of conversions
to every coalitions channels.
• Normalize data: study % of
conversions and not number
conversions
• Scientific approach: keep in
mind some basic reasoning
like symmetry
Work in database.
Use SparkSql and
PySpark

20. Marketing attribution: Game theory and Shapley Value
{Facebook} = 5 conversions
{Twitter} = 2 conversions
{SnapChat} = 3 conversions
{Facebook , Twitter} = 8 conversions
{Facebook , SnapChat} = 13
conversions
{Twitter , SnapChat} = 7 conversions
{Facebook , Twitter , SnapChat} = 16
conversions
Facebook contribution
(5 + 6 + 10 + 9)/ 4 = 7.5
Twitter contribution
(2 + 3 + 4 + 5) /4 = 3.5
SnapChat contribution
(3 + 8 + 5 + 8 )/4 = 6
A practical example from the different customer journey leading to conversion

21. How to validate and choose a
model?

22. Marketing attribution: model validation strategy
Practical difficulties of the real world
How do you observe the real removal
impact? Quantify the real contribution of a channel?
Hard to verify the grand truth of what you
can compute fro the database.
A/B testing
• Potential disastrous effect for the business to stop
one channel even for a day.
• Client acquisition is more challenging than ever.
Why should you do that ?
• To choose the best approach
• To monitor the potential drift in real time

23. Marketing attribution: model validation strategy
Methodology for validation strategy
Random simulation
of customers journey
Apply attribution models
Define conversions
given an hypothesis H
How did models
capture H?
Deploy the best model
to allocate
marketing budget
Increase your
conversion rate

24. Marketing attribution: model validation strategy
Results on Dataiku logs
Hypothesis: noisy data
Shapley can capture that
much better than any other
(No money should be
spent there)
Caution!!
Be careful that there is
a natural bias in the data
Logs in database are
probably … the results of
actual marketing campaign

25. A DSS workflow eases the process of
iterative data science
• Handle the mandatory data
preparation
steps
• Handle some complex coding
transformation in SQL/Python/R/Scala
• Leverage the underlying infrastructure
Dataiku DSS the most complete Data Science Platform
A DSS workflow offers some features to
easily deploy in production
• Scheduling and automation of jobs
• Advance usage of the internal API
• Deployment without recoding anything

26. Thank you for your attention !