Automating sample is an important step toward delivering on industry demands for speed, cost savings and quality. Where is the sample industry when it comes to automating processes? Where should we be? This session provides a vision for automation that goes beyond greater efficiency and lower costs, ultimately enabling suppliers to deliver better data and agility for clients. From respondent satisfaction all the way to effective fraud mitigation, using this technology to its full potential can help to build true efficiencies that result in better quality from start to finish.

1. 10 Ways Sample Companies Should Be Using Automation AI and Machine Learning
JD Deitch, P2 Sample
The Future of AI
& Automation

2. 10 Ways Sample Companies Should Be Using Automation AI and Machine Learning
JD Deitch, P2 Sample
The Future of AI
& Automation
● Social scientist & econometrician
● 20+ years in market research
(client & supplier)
● Senior roles at NPD Group, Ipsos
● Written and spoken extensively on
evolution of market research
About Me

3. 10 Ways Sample Companies Should Be Using Automation AI and Machine Learning
JD Deitch, P2 Sample
The Future of AI
& Automation
AUTOMATION

4. 10 Ways Sample Companies Should Be Using Automation AI and Machine Learning
JD Deitch, P2 Sample
The Future of AI
& Automation
The Process PANEL RECRUITMENT AND PROJECT EXECUTION
What’s Already Being Done SPEED AND COST REDUCTION
The Coming Wave SOLVING THE BIG PROBLEMS
Today’s discussion

5. 10 Ways Sample Companies Should Be Using Automation AI and Machine Learning
JD Deitch, P2 Sample
The Future of AI
& Automation
The Process

6. 10 Ways Sample Companies Should Be Using Automation AI and Machine Learning
JD Deitch, P2 Sample
The Future of AI
& Automation
RECRUITMENT PROFILING SELECTION REWARD
panel

7. 10 Ways Sample Companies Should Be Using Automation AI and Machine Learning
JD Deitch, P2 Sample
The Future of AI
& Automation
project
PLANNING COMMISSIONING FIELDING ANALYSIS

8. 10 Ways Sample Companies Should Be Using Automation AI and Machine Learning
JD Deitch, P2 Sample
The Future of AI
& Automation
panel
project
RECRUITMENT PROFILING SELECTION REWARD
PLANNING COMMISSIONING FIELDING ANALYSIS

9. 10 Ways Sample Companies Should Be Using Automation AI and Machine Learning
JD Deitch, P2 Sample
The Future of AI
& Automation
FEASIBILITY
BUY/SELL
panel
project
RECRUITMENT PROFILING SELECTION REWARD
PLANNING COMMISSIONING FIELDING ANALYSIS

10. 10 Ways Sample Companies Should Be Using Automation AI and Machine Learning
JD Deitch, P2 Sample
The Future of AI
& Automation
FEASIBILITY
BUY/SELL
QUOTA MGMT
FIELD MONITORING
panel
project
L
A
U
N
C
H
RECRUITMENT PROFILING SELECTION REWARD
PLANNING COMMISSIONING FIELDING ANALYSIS

11. 10 Ways Sample Companies Should Be Using Automation AI and Machine Learning
JD Deitch, P2 Sample
The Future of AI
& Automation
FEASIBILITY
BUY/SELL
QUOTA MGMT
FIELD MONITORING
INCENTIVE
DATA DELIVERY
panel
project
L
A
U
N
C
H
C
L
O
S
E
RECRUITMENT PROFILING SELECTION REWARD
PLANNING COMMISSIONING FIELDING ANALYSIS

12. 10 Ways Sample Companies Should Be Using Automation AI and Machine Learning
JD Deitch, P2 Sample
The Future of AI
& Automation
What’s Happening Now

13. 10 Ways Sample Companies Should Be Using Automation AI and Machine Learning
JD Deitch, P2 Sample
The Future of AI
& Automation
FEASIBILITY
BUY/SELL
R QUALITY SCORE
QUOTA MGMT
FIELD MONITORING
INCENTIVE
DATA DELIVERY
RECRUITMENT PROFILING SELECTION
REWARD
PLANNING COMMISSIONING FIELDING
ANALYSIS
panel
project
1 1. RECRUITMENT
Onboarding new panelists is already quite automated.
Suppliers typically have standing orders with companies that
source participants and direct them into a “new member”
workflow.

14. 10 Ways Sample Companies Should Be Using Automation AI and Machine Learning
JD Deitch, P2 Sample
The Future of AI
& Automation
R QUALITY SCORE
QUOTA MGMT
FIELD MONITORING
INCENTIVE
DATA DELIVERY
PLANNING COMMISSIONING FIELDING
ANALYSIS
panel
project
2
RECRUITMENT PROFILING SELECTION REWARD
2. PROFILING
The collection of demographic profile data
for targeting is quite automated.
● Detailed demographic data is
common.
● The collection of extensive behavioral
profiling varies considerably.
(And, disappointingly, respondents are still
too frequently asked to provide this data in
field despite it already being known!)

15. 10 Ways Sample Companies Should Be Using Automation AI and Machine Learning
JD Deitch, P2 Sample
The Future of AI
& Automation
INCENTIVE
DATA DELIVERY
PLANNING COMMISSIONING FIELDING
ANALYSIS
panel
project
3
3. INCENTIVE FULFILLMENT
This has been automated for a while. It
became essential as participation rates
started to decline many years ago.
RECRUITMENT PROFILING SELECTION REWARD

16. 10 Ways Sample Companies Should Be Using Automation AI and Machine Learning
JD Deitch, P2 Sample
The Future of AI
& Automation
PLANNING COMMISSIONING FIELDING ANALYSIS
FEASIBILITY
BUY/SELL
R QUALITY SCORE
QUOTA MGMT
FIELD MONITORING
INCENTIVE
DATA DELIVERY
panel
project
RECRUITMENT PROFILING SELECTION REWARD
4. BUYING & SELLING
This is what people are referring to when they talk
about programmatic sampling.
That said, there are still plenty of companies
(including some large agencies) that remain
highly manual!4

17. 10 Ways Sample Companies Should Be Using Automation AI and Machine Learning
JD Deitch, P2 Sample
The Future of AI
& Automation
FEASIBILITY
BUY/SELL
QUOTA MGMT
FIELD MONITORING
INCENTIVE
DATA DELIVERY
panel
project
L
A
U
N
C
H
C
L
O
S
E
1 2
4
Though big strides have been made, automation remains largely piecemeal,
and little has been done to improve conditions for respondents.
3
RECRUITMENT PROFILING SELECTION REWARD
PLANNING COMMISSIONING FIELDING ANALYSIS

18. 10 Ways Sample Companies Should Be Using Automation AI and Machine Learning
JD Deitch, P2 Sample
The Future of AI
& Automation
The Coming Wave

19. 10 Ways Sample Companies Should Be Using Automation AI and Machine Learning
JD Deitch, P2 Sample
The Future of AI
& Automation
PLANNING COMMISSIONING FIELDING ANALYSIS
RECRUITMENT PROFILING SELECTION REWARD
FEASIBILITY
BUY/SELL
R QUALITY SCORE
QUOTA MGMT
FIELD MONITORING
INCENTIVE
DATA DELIVERY
panel
project
L
A
U
N
C
H
C
L
O
S
E
5
5. FEASIBILITY
Are we really still doing this in spreadsheets?
We have loads of respondent data and client
data beyond LOI and Incidence, and the
processing power to crunch the data.
Automation creates far more accurate
estimates that take into account the numerous
conditions that impact feasibility.

20. 10 Ways Sample Companies Should Be Using Automation AI and Machine Learning
JD Deitch, P2 Sample
The Future of AI
& Automation
PLANNING COMMISSIONING FIELDING ANALYSIS
RECRUITMENT PROFILING SELECTION REWARD
FEASIBILITY
BUY/SELL
R QUALITY SCORE
QUOTA MGMT
FIELD MONITORING
INCENTIVE
DATA DELIVERY
panel
project
L
A
U
N
C
H
C
L
O
S
E
6
6. LAUNCH
We should be able to go from bid to
launch (soft or full) in a single click.
It sounds simple, but it requires
automation of the quotation process and
a tight link to the operational platform.
But when you do it, you can literally
launch in seconds.

21. 10 Ways Sample Companies Should Be Using Automation AI and Machine Learning
JD Deitch, P2 Sample
The Future of AI
& Automation
RECRUITMENT PROFILING SELECTION REWARD
FEASIBILITY
BUY/SELL
QUOTA MGMT
FIELD MONITORING
INCENTIVE
DATA DELIVERY
panel
project
L
A
U
N
C
H
C
L
O
S
E
PLANNING COMMISSIONING FIELDING
ANALYSIS
7
7. QUOTA MGMT
We should know when
they are full.
We should therefore
know to stop sending
sample, yet overquotas
still plague respondents.
(This is an upstream problem,
isn’t it!)
8
8. FIELD MONITORING
Why so manual, still?!
We have field data. We
have computers. We
should be able to judge
if we’re on track. If we’re
not, the system should
send immediate alerts
so we can fix the
problem in time!

22. 10 Ways Sample Companies Should Be Using Automation AI and Machine Learning
JD Deitch, P2 Sample
The Future of AI
& Automation
PLANNING COMMISSIONING FIELDING ANALYSIS
RECRUITMENT PROFILING SELECTION REWARD
FEASIBILITY
BUY/SELL
R QUALITY SCORE
QUOTA MGMT
FIELD MONITORING
S QUALITY SCORE
INCENTIVE
DATA DELIVERY
panel
project
L
A
U
N
C
H
C
L
O
S
E
9. RESPONDENT
QUALITY SCORE
Consistent, detailed and
thoughtful responses
are good. Speeding and
straightlining are bad.
Automation allows us to
assign scores in real-
time and eliminate and
replace bad
respondents in field.
10. SURVEY
QUALITY SCORE
Dropout, overquota and
term rates tell us a lot
about survey quality, as
do respondent ratings.
We have the data. Good
surveys should be
promoted. Bad surveys
should be quarantined.
9
10

23. 10 Ways Sample Companies Should Be Using Automation AI and Machine Learning
JD Deitch, P2 Sample
The Future of AI
& Automation
FEASIBILITY
BUY/SELL
R QUALITY SCORE
QUOTA MGMT
FIELD MONITORING
S QUALITY SCORE
INCENTIVE
DATA DELIVERY
panel
project
L
A
U
N
C
H
C
L
O
S
E
APIapplication programming interfaces
We are only scratching the surface
on what can be communicated
between systems.
11
RECRUITMENT PROFILING SELECTION REWARD
PLANNING COMMISSIONING FIELDING ANALYSIS

24. 10 Ways Sample Companies Should Be Using Automation AI and Machine Learning
JD Deitch, P2 Sample
The Future of AI
& Automation
FEASIBILITY
BUY/SELL
INCENTIVE
DATA DELIVERY
R QUALITY SCORE
QUOTA MGMT
FIELD MONITORING
S QUALITY SCORE
panel
project
L
A
U
N
C
H
C
L
O
S
E
1 2
The magic happens when you put all of these things together.
3
6
7
8
9
10
11 API
4
5
RECRUITMENT PROFILING SELECTION REWARD
PLANNING COMMISSIONING FIELDING ANALYSIS

25. 10 Ways Sample Companies Should Be Using Automation AI and Machine Learning
JD Deitch, P2 Sample
The Future of AI
& Automation
FEASIBILITY
BUY/SELL
INCENTIVE
DATA DELIVERY
R QUALITY SCORE
QUOTA MGMT
FIELD MONITORING
S QUALITY SCORE
L
A
U
N
C
H
panel
project
C
L
O
S
E
1 2
3
6
7
8
9
10
11 API
4
5
Fully automated operations massively improves SPEED, EFFICIENCY and
DEPENDABILITY.
RECRUITMENT PROFILING SELECTION REWARD
PLANNING COMMISSIONING FIELDING ANALYSIS

26. 10 Ways Sample Companies Should Be Using Automation AI and Machine Learning
JD Deitch, P2 Sample
The Future of AI
& Automation
L
A
U
N
C
H
FEASIBILITY
BUY/SELL
INCENTIVE
DATA DELIVERY
R QUALITY SCORE
QUOTA MGMT
FIELD MONITORING
S QUALITY SCORE
panel
project
C
L
O
S
E
1 2
3
6
7
8
9
10
11 API
4
5
DATA & RESPONSE QUALITY improve dramatically.
RECRUITMENT PROFILING SELECTION REWARD
PLANNING COMMISSIONING FIELDING ANALYSIS

27. 10 Ways Sample Companies Should Be Using Automation AI and Machine Learning
JD Deitch, P2 Sample
The Future of AI
& Automation
● Significantly reduces demoralizing overquotas, soul-crushing routing,
unnecessary terms and panel-killing dropouts.
This arises only when we combine profiling data and algorithms and fully map and communicate data in APIs, and then
use engagement metrics as KPIs.
● Delivers better data, and we have proof of this.
NB: Our industry has done tons of research proving bad experiences = bad data. We have third party data demonstrating
that the practice of promoting good experiences and quarantining bad ones delivers more engaged respondents. Plus,
stopping a study that’s crashing in field and alerting clients before it’s too late makes absolute sense!
● Used up the supply chain, improves conversion rates for sourcing
partners and leads to long-term quality & economic benefits.
Doing all these things creates a virtuous circle, whereas current practices create a vicious one.
Automation can massively improve
the respondent experience.

28. 10 Ways Sample Companies Should Be Using Automation AI and Machine Learning
JD Deitch, P2 Sample
The Future of AI
& Automation
R QUALITY SCORE
QUOTA MGMT
FIELD MONITORING
S QUALITY SCORE
9
L
A
U
N
C
H
FEASIBILITY
BUY/SELL
INCENTIVE
DATA DELIVERY
R QUALITY SCORE
QUOTA MGMT
FIELD MONITORING
S QUALITY SCORE
panel
project
C
L
O
S
E
1 2
3
6
7
8
9
10
11 API
4
5
Breakthrough improvements in RESPONDENT VALIDATION & FRAUD DETECTION.
RECRUITMENT PROFILING SELECTION REWARD
PLANNING COMMISSIONING FIELDING ANALYSIS

29. 10 Ways Sample Companies Should Be Using Automation AI and Machine Learning
JD Deitch, P2 Sample
The Future of AI
& Automation
250,000,000 data points per year
● demographics
● survey responses
● behaviours
● device information
● field statistics
Data and processing power
enable significant improvement in
respondent validation and deduplication.

30. 10 Ways Sample Companies Should Be Using Automation AI and Machine Learning
JD Deitch, P2 Sample
The Future of AI
& Automation
5-20%+
are fraudulent✻ meta-estimate from various sources: Robbins and Kuriakose (from Science Mag, with Pew rebuttal), TrueSample, P2Sample, Lucid and
discussions with informed colleagues

31. 10 Ways Sample Companies Should Be Using Automation AI and Machine Learning
JD Deitch, P2 Sample
The Future of AI
& Automation

32. 10 Ways Sample Companies Should Be Using Automation AI and Machine Learning
JD Deitch, P2 Sample
The Future of AI
& Automation
32
75%mixed human & machine
source: ana/whiteops

33. 10 Ways Sample Companies Should Be Using Automation AI and Machine Learning
JD Deitch, P2 Sample
The Future of AI
& Automation
Email validation
Address validation
Digital fingerprinting
Trap questions
Honeypots
Captcha
Expert rules
Open end analysis

34. 10 Ways Sample Companies Should Be Using Automation AI and Machine Learning
JD Deitch, P2 Sample
The Future of AI
& Automation
34
Artificial Intelligence
artificial intelligence

35. 10 Ways Sample Companies Should Be Using Automation AI and Machine Learning
JD Deitch, P2 Sample
The Future of AI
& Automation
1. Domain experience
2. Data (a LOT)
3. Algorithm(s)
Artificial Intelligence requires three things:

37. 10 Ways Sample Companies Should Be Using Automation AI and Machine Learning
JD Deitch, P2 Sample
The Future of AI
& Automation
66%REDUCTION IN FRAUD

38. 10 Ways Sample Companies Should Be Using Automation AI and Machine Learning
JD Deitch, P2 Sample
The Future of AI
& Automation
MIGHT BE FRAUDULENT,
THEY ARE DISENGAGED THE NEXT TIME!
~1.5%

39. 10 Ways Sample Companies Should Be Using Automation AI and Machine Learning
JD Deitch, P2 Sample
The Future of AI
& Automation
1. Addressing the industry’s
toughest problems.
2. Meaningfully differentiating
sample providers now.
3. Automate or go home.
The Second Wave of Automation

40. 10 Ways Sample Companies Should Be Using Automation AI and Machine Learning
JD Deitch, P2 Sample
The Future of AI
& Automation
More on automation and objective
factors to evaluate sample suppliers,
with commentary & forewords from Ray,
Lenny Murphy (GreenBook/IIeX), Kristin
Luck, and Patrick Comer (Lucid).
We will send it to you (free).

41. 10 Ways Sample Companies Should Be Using Automation AI and Machine Learning
JD Deitch, P2 Sample
The Future of AI
& Automation
Jonathan D. Deitch, Ph. D.
P2Sample
+33 7 60 62 68 94
+1 703 493 0422
jd@p2sample.com
or find me in Berlin at ESOMAR!
Questions?
p2sample.com/automation

42. 10 Ways Sample Companies Should Be Using Automation AI and Machine Learning
JD Deitch, P2 Sample
The Future of AI
& Automation
Q & A
JD Deitch
P2Sample
Ray Poynter
NewMR

43. 10 Ways Sample Companies Should Be Using Automation AI and Machine Learning
JD Deitch, P2 Sample
The Future of AI
& Automation
NewMR 2018 Sponsors
Communica:on
Gold
Silver