Alternative to ANA's end to end supply chain transparency study v final

June 2021 / Page 0
marketing.science
consulting group, inc.
linkedin.com/in/augustinefou
End-to-End Supply Chain Study
Result Scenarios / Recommendations
June 2021
Augustine Fou, PhD.
acfou [at] mktsci.com

June 2021 / Page 1
marketing.science
"The industry must now move to rectify the
issues and restore the productivity of billions
of dollars of marketing investments,“ said
Bob Liodice, CEO of the ANA

June 2021 / Page 2
marketing.science
What the ANA RFP asked for …
The new study will add detail to prior studies, and separate “cost efficiency” from “ad effectiveness”
“Cost Efficiency” “Ad Effectiveness”
https://www.ana.net/content/show/id/pr-2021-programmatic-rfp

June 2021 / Page 3
marketing.science
But …
• The numerous forms of hidden arbitrage, waste, and fraud cannot be depicted in a
simple waterfall chart (“if the definition of transparency is how much of my budget
makes it into the publishers’ hands, that's a very flawed baseline.”)
• Attempts to depict the losses and waste assumes you can see all of those problems
and requires you to make assumptions about what average is “most representative”
when in most cases “the extent is not known”
• The reality of “productive portion of an ad dollar” is too scary for a trade association
to publish truthfully; it is likely they will do what they have done before: position the
result with PR, which masks the truth, is misleading, and therefore is useless
• The recommendation “buy less programmatic” will not be tenable

June 2021 / Page 4
marketing.science
Executive Summary
OBJECTIVE OF THIS STUDY
• Educate marketers on the inefficiencies, costs, hidden arbitrage, and risks associated
with buying through programmatic channels and how these outweigh the alleged
benefits of reach, targeting, and performance
• Enable marketers to take immediate action on these new insights, without re-
tooling, to optimize their digital ad spending, reduce waste and risk, and improve
actual business outcomes
All of the data compiled in this study come from 1) first-hand campaigns
where all variables are known, 2) documented cases published by others
publicly, and 3) data crowdsourced from practitioners anonymously

June 2021 / Page 5
marketing.science
Historical Context

June 2021 / Page 6
marketing.science
Publishers’ Studies of Supply Chain
Wide variation in outcomes: 30c on the dollar went to pub; < 1% went to publisher buying own inventory
Ad Budget Publisher
30%
100%
80%
60%
40%
20% https://mediatel.co.uk/newsline/2016/10/04/where-did-the-
money-go-guardian-buys-its-own-ad-inventory
2016
The Guardian
“for every pound an advertiser
spends programmatically on the
Guardian only 30 pence actually
goes to the publisher.”
Ad Budget Publisher
100%
80%
60%
40%
20%
2017
BusinessInsider
“$40,000 worth of ad inventory
through the open exchanges, the
publication only saw $97.”
http://adage.com/article/digital/business-insider-york-times-
shed-details-ad-industry-s-biggest-problem/311081/
< 1%

June 2021 / Page 7
marketing.science
Prior Transparency Studies
Supply chain studies since 2014 corroborate that only 40 – 61% of dollar makes it to pubs for showing ads
ISBA 2020: 51%
“publishers received half of advertiser spend.
15% of advertiser spend – the unknown
delta, representing around 1/3 of supply
chain costs – could not be attributed.”
Source: ISBA, May 2020
“went missing”
Source: WFA, Sep 2014 Source: ANA, May 2017
WFA 2014: 40%
“We have little or no clear understanding of what
percentage (of digital spend) is being delivered to
the media owner and what is being taken in fees
from either the agency or middlemen. ”
ANA 2017: 60%
“Programmatic remains complex and often
non-transparent. Our study revealed that
this lack of transparency makes it difficult for
advertisers to manage, measure and audit
programmatic media investments.”

June 2021 / Page 8
marketing.science
WARC/MAGNA 2018 Study
40% of the dollar reaches publishers; before any ad fraud is accounted for

June 2021 / Page 9
marketing.science
Estimates of productivity over years
13% productive (2014), 10% productive (2017), 1% productive (2019)
https://www.slideshare.net/augustinefou/display-ad-
productivity-by-augustine-fou
13% (2014) 10% (2017)
https://www.slideshare.net/augustinefou/effectiveness-of-
digital-ads-q2-2017-update-by-augustine-fou/3
1% (2019)
https://www.slideshare.net/augustinefou/ad-
fraud-is-more-than-just-bots-jan-2019

June 2021 / Page 10
marketing.science
Illustrative Scenarios

June 2021 / Page 11
marketing.science
Advertisers thought they bought…
Many loopholes and risks are not accounted for by simple waterfall chart; scenarios are needed
Ad revenue
diverted by
domain spoofing
Pay on bids won
vs ads served
Unspent media
dollars go missing
Random or
replayed deviceID
Faked valid traffic,
viewable ads
Analytics and
measurement
manipulation
Attribution
scamming
Residential proxies
to disguise traffic
Media
dollars to
agency
Set up
campaign
in DSP
Buy
white list
domains
Buy
valid,
viewable
Buy
real
devices
Buy
“fraud
free”
Buy
“verified”
Did ad
render?
#marketers who think that having bot detection in place is enough
clearly don’t know about these other “measurable issues”
Catching the 1%
IVT, missing rest
Advertisers
Publishers

June 2021 / Page 12
marketing.science
56%
“Best Case” Scenario
Shortest possible supply path, no agency, no targeting/verification, direct publisher
Ad Budget Agency
Fees
DSP Ad Auction
Funds
SSP Publisher Ad Quality
“discounts”
Audience
100%
12%
10%
79%
23%
70%
average
viewability
Tech, Data,
Verification
Fees
Bid: $1.00
30%
non-viewable
Auction Funds: $0.88
Source: 10 first-hand campaigns, Jan – June 2021
100%
80%
60%
40%
20%

June 2021 / Page 13
marketing.science
35%
“Ideal Conditions” Scenario
No “unknown delta” (no arbitrage), minimized DSP, SSP fees, 10% data fee
Ad Budget Agency
Fees
DSP Ad Auction
Funds
SSP
Exchange
Publisher
Ad Quality
“discounts”
Audience
7%
7%
60%
85%
Source: $TTD 10-Qs
Tech, Data,
Verification
Fees
100%
80%
60%
40%
20%
15%
10%
20%
10%
55%
65%
Source: ISBA
8%
10%
Source: ISBA
Advertiser to Publisher
“Cost Efficiency”
40 cents on the dollar go to ‘non-working’ costs
75%
65%
69.8%
viewability
1.5%
ad
fraud
5.8%
brand
risk
Source: IAS H2 2020 MQR
viewable
no
G-IVT
brand
safe
“Ad Effectiveness”
Publisher to Audience
25 more cents eaten up by ad quality issues
“Using the most conservative
estimates of ad quality issues”
– IAS H2 2020 MQR

June 2021 / Page 14
marketing.science
Ranges and Averages must be used
Grouping together known costs, separating from “measurable issues,” ranges and averages cited
100%
80%
60%
40%
20%
IVT / NHT
58%
42%
Domain
Spoofing
45%
55%
Not
Viewable
20%
80%
Ad
Blocking
85%
15%
Off Target
34%
66%
Ads Not
Displayed
75%
25%
Not Bot
Ads.txt
Matched Viewable
Not Ad
Blocked On Target
Ads
Displayed
Agency
90%
10%
Not
Agency
Exchange
DSP / SSP
80%
20%
Not
Trading
Data /
Verification
80%
20%
Not
Fees
Known Costs Measurable Issues

June 2021 / Page 15
marketing.science
“Ignore-All-Issues” Scenario
“Measurable issues” eat away at “working media” (productive = 99% · 99% · 99% · 99% ·62% ·50%)
100%
80%
60%
40%
20%
IVT / NHT
Not Bot
Domain
Spoofing
99%
1%
Ads.txt
Matched
Not
Viewable
62%
38%
Viewable
Ad
Blocking
Not Ad
Blocked
Off Target
50%
50%
On Target
Ads Not
Displayed
Ads
Displayed
Agency
90%
10%
Not
Agency
Exchange
DSP / SSP
80%
20%
Not
Trading
Data /
Verification
80%
20%
Not
Fees
10%
20%
20%
50%
“working
media”
35%
15%
“likely
productive
ads”
99%
1% 99%
1% 99%
1%
assume
1%
domain
spoofing
assume
1%
ad
blocking
assume
1%
IVT/NHT
assume
1%
ad
serving
problems
Source:
IAS
Q1
MQR
Source:
Nielsen

June 2021 / Page 16
marketing.science
“More Realistic” Scenario
“Measurable issues” eat away at “working media” (productive = 85% ·75% ·58% ·45% ·34% ·20%)
100%
80%
60%
40%
20%
IVT / NHT
58%
42%
Not Bot
Domain
Spoofing
45%
55%
Ads.txt
Matched
Not
Viewable
20%
80%
Viewable
Ad
Blocking
85%
15%
Not Ad
Blocked
Off Target
34%
66%
On Target
Ads Not
Displayed
75%
25%
Ads
Displayed
Agency
90%
10%
Not
Agency
Exchange
DSP / SSP
80%
20%
Not
Trading
Data /
Verification
80%
20%
Not
Fees
10%
20%
20%
50%
“working
media”
49%
1%
“likely
productive
ads”
Slide 18 Slide 19 Slide 20 Slide 21 Slide 22 Slide 23

June 2021 / Page 17
marketing.science
Supporting Data

June 2021 / Page 18
marketing.science
Domain Spoofing – ads.txt matching
Match ads.txt sellerIDs to IDs in sellers.json
Match type “direct” vs “reseller” correctly
1/3 matched
1/2 matched
45% matched
https://deepsee.io/blog/evaluating-the-ecosystem

June 2021 / Page 19
marketing.science
Directly Measured Ad Blocking
Direct measurement is necessary, measurement done on-site
• Ad blocking must be measured on-site. In-ad measurements are invalid
• Desktop and mobile must be separated because ad blocking in mobile is
very low (no plugins for mobile browsers, and most consumers don’t
regularly use ad blocking browsers, they use built-in browsers)
• Bots and “not measurable” must be excluded because bots don’t block
ads (their job is to cause them to load)
Business sites desktop mobile
ad block rate 8 - 17% 0.6 – 0.8%
Consumer sites desktop mobile
ad block rate 11 - 12% 0.9 - 1%

June 2021 / Page 20
marketing.science
IVT/NHT – Display Ads Only
Not Measurable 0%
No Client-Side Data 8%
DESKTOP GIVT/SIVT Humans Other
Disguised
Traffic
Fake
Device
App
Fraud
67% 37% 2% 61%
0% 6% 3%
MOBILE GIVT/SIVT Humans Other
33% 7% 5% 88%
DEFINITIONS
Not measurable – no tags sent (this should be zero, ads are called by JS)
No Client-Side Data – no data sent back, ad blocker or browser block (e.g. Brave)
Other – not enough blue or red labels to confirm
Disguised Traffic – fake traffic, bounced through residential proxies
Fake Device – multiple factors indicating fake device (software in datacenter)
App Fraud – apps loading webpages and other non-bot fraud
IAS: 1.4% (US)Source: IAS Media Quality Report H1 2020
IAS: 0.8% (US)Source: IAS Media Quality Report H1 2020
mobile app
non-app

June 2021 / Page 21
marketing.science
Bids won vs impressions served
In most ideal conditions, 25% of ads purchased were never rendered
“The more fraudulent the site, the larger the
discrepancy, up to 100% (no ads served).
1. DSP: bids won by domain
2. Ad Server: ads served by domain
1 2 1 2
Low CPM
campaign

June 2021 / Page 22
marketing.science
Viewability – Display, Video
DEFINITIONS
measured viewable = intersection >50% + visibilityState=visible
viewable-MRC = measured viewable + 1 second
Display
desktop
only
mobile web
only app only
measured viewable 25% 58% 18%
viewable-MRC 20% 7% 9%
Video
desktop
only
mobile web
only app only
measured viewable 9% 7% 2%
viewable-MRC 7% 4% 2%
IAS: 71% (US)
Source: IAS Media Quality Report H1 2020
IAS: 68% (US)
IAS: 68% (US)
Source: IAS Media Quality Report H1 2020
IAS: 66% (US) IAS: 79% (US)

June 2021 / Page 23
marketing.science
Bad targeting, worse than “random”
Ad tech targeting is based on segments inferred from website visitation patterns
1 parameter: gender 42% accuracy
2 params: gender+age 24% accuracy

June 2021 / Page 24
marketing.science
Other Issues Not Depicted in
Waterfall Chart Scenarios

June 2021 / Page 25
marketing.science
All the fraud missed by IVT detection
Other forms of fraud are severely under-reported by standard IVT detection tech vendors
1.3%
fake traffic
(bots)
severely
under-reported
overall fraud
bot detection
sees this
bot detection
misses all these
58%
57%
+ =
other fraud
(sites/apps)

June 2021 / Page 26
marketing.science
Incorrect measurements by IVT vendor
Accredited IVT vendors are measuring incorrectly, no way to troubleshoot because “black box”
false positives
false negatives
discrepancies
Domain (spoofed) % SIVT
esquire.com 77%
travelchannel.com 76%
foodnetwork.com 76%
popularmechanics.com 74%
latimes.com 72%
reuters.com 71%
Good domains are
incorrectly marked

June 2021 / Page 27
marketing.science
Vast botnets - primary use: ad fraud
The most lucrative use for vast botnets is ad fraud
2019 2020
2017 2018
2015 2016
2013 2014
2011 2012
2010
IceBucket
CTV fraud, 2B
imps /day
Coreflood
DDoS, web
scraping,
credential
stealing
Grum
40B spam
emails /day
ZeroAccess
Click fraud
Bitcoin mining
488 TB traffic
/day
Avalanche
Ransomeware,
RATs, banking
trojans
Mirai
IoT malware, 665
Gbps DDoS
Methbot
Video ad fraud,
300M /day Hyphbot
Ad fraud, 34k
websites
3ve
Ad fraud 1.7M
devices, 12B ad
req /day
Conficker
DDoS,10M
computers
Judy
Ad fraud, 41
apps, 37M
devices
Fireball
Ad fraud browser
hijack, 250M
devices
SndApps
IMEI/personal
info stealing,
adware
Chameleon
Click fraud, fake
mouse moves,
ad fraud
Rustock
Email spam,
2,000 /second
RottenSys
Android adware,
malware
WannaCry
Kelihos
Necurs
Zeus
Dridex
Emotet
Spam, banking
trojans
Ransomware
WePurchaseApps
127 apps,
millions of
downloads
Cheetah
600 apps, 4+
billion downloads
DO Global
6 apps, 50M
downloads, ad
fraud
VidMate
500M installs, ad
fraud
iHandy
46 apps, 100M
installs, ad fraud
DiCaprio
CTV fraud Grindr
faking Roku
streams
CooTek
60 apps, 50M
downloads, ad
fraud
ClearSkins
Fake Sites, 80M
video views /day
Adap.tv
Video ad fraud
(unnamed)
5,000 mobile
apps, display ad
fraud
Boris
Network for
sites for ad fraud
FreeStreams
Network of sites,
audience
overlap ad fraud
DDoS
Spam, banking, ransomware
Ad fraud
Google Digital Attack Map

June 2021 / Page 28
marketing.science
Traffic available to purchase by credit card
Thousands of sources to buy “traffic” widely and easily available
Google
“buy real human traffic”
Select vendor and
“traffic quality level”
Host your own bots
(cost $3.99/mo)

June 2021 / Page 29
marketing.science
“Dark pooling” - ads.txt abuse
Fraudsters spoof top domains with 40 – 42X more “unpermissioned” bid requests
https://www.linkedin.com/pulse/unique-cookie-viewability-analysis-most-spoofed-domains-augustine-fou/

June 2021 / Page 30
marketing.science
Oxylabs[.]io Smartproxy[.]com
Method for disguising fake traffic
Fraudsters disguise data center bots to appear to come from residential IP addresses, avoid detection
32 million
IP addresses
40 million
IP addresses
100 million
IP addresses
72 million
IP addresses

June 2021 / Page 31
marketing.science
Missing Media Dollars
Hidden arbitrage, principal trading, unspent media dollars
Display Campaign 1
$3,300,000 (+10%)
(agency invoice to advertiser)
$3,000,000
DSP invoice to agency (paid by
advertiser)
$1,800,000 (-40%)
DSP log total (bids won)
On average, 24% of advertisers’ media spend was wasted/missing
Mobile Campaign 2
$950,000
(campaign spend)
$330,000 (35%)
(invalid devices/fraud)
Display Campaign 3
$7,900,000
(campaign spend)
$1,500,000 (19%)
Display Campaign 3
$3,900,000
(campaign spend)
$550,000 (14%)
Display Campaign 4
$7,000,000
(campaign spend)
$1,700,000 (24%)
Source: Method Media Intelligence
DSP logs show only $1.8 million of bids
won, compared to the $3.0 million paid;
$1.2 million went “missing” (not
returned/credited to advertiser)

June 2021 / Page 32
marketing.science
Malicious code to alter measurements
Common practice to manipulate/alter measurements to trick fraud detection
“the [malicious] code used by NMG is
designed to interfere with the ability of
third-party measurement systems to
determine how much of a digital ad
was viewable during a browsing
session.
This code manipulated data to
ensure that otherwise
unviewable ads showed up in
measurement systems as valid
impressions, which resulted in
payment being made for the
ad.”
Buzzfeed, March 2018

June 2021 / Page 33
marketing.science
tOLKnMEzcnARvLTvChnt
Random vs Replayed Device IDs for fraud
Collecting device IDs and replaying them to defeat fraud detection and frequency caps
RANDOM deviceIDs
lXvBEeRXPURtcKILYFYE
IdUkQeWgqshMmfMdzlAx
INIjBzHJHywhgRsMdQPe
tiAnxwuKBNCjoMetZaPN
UjtRbuUTvYUwmABhmPGH
MDSUUgkENQkQDztavzfl
iljoJEXUcLCEFwSdrwZn
APbLSRUvlrIoofIchhLg
NZXVVKCbymRYBSStNRYz
UiSBmuDpYLkNvsHBKcri
LZyhgblHtMIMaAliHWYB
vKFknsnhGouIucYgxmdu
• If fraud detection hasn’t seen a device
before, the default action is to let the ad
serve
• Frequency caps based on deviceIDs are
defeated, each device appears as new
• Valid deviceIDs are harvested from real
devices and sent to fake devices or apps to
replay
• Replayed deviceIDs are used by fraudulent
apps to defeat fraud detection
REPLAYED deviceIDs
validated deviceID harvested

June 2021 / Page 34
marketing.science
Fraud hidden in averages
Waterfall charts require assumptions and average numbers to be plotted, hides fraud
… lots of
clicks but not
from humans
visible in line-item details
9.4% avg click rate

June 2021 / Page 35
marketing.science
Suboptimal campaigns remain hidden
Ad impressions used up by 4a, very low reach – not visible in totals/averages
hourly details
domain-level reach analysis

June 2021 / Page 36
marketing.science
False data inserted into Google Analytics
Faking mousemoves, clicks, etc; inserting false data into Google Analytics
demo of faked mouse movement demo of faked Google Analytics traffic
https://www.forbes.com/sites/augustinefou/2021/04/22/google-analytics-vulnerabilities-put-marketers-at-risk/

June 2021 / Page 37
marketing.science
Supply path leakage, spoofing, arb
Using more than 1 exchange to buy the same domain results in leakage/inefficiencies
1 domain, all exchanges 1 domain, one exchange
“Even with an allow-list of 1 domain, 24%
of impressions ‘went elsewhere’ if multiple
exchanges were used for buying; find the
primary/preferred exchange of the
publisher and buy only through that one.”
24%

June 2021 / Page 38
marketing.science
Duplicate bid requests for same ad slot
Simultaneous bid requests increases risk of duplicate bid requests, buyers bidding against themselves
https://deepsee.io/blog/header-bidding-activity-benchmarks-for-2021-so-far

June 2021 / Page 39
marketing.science
Ad Dollars Funding Fake News/Disinformation
Ad dollars flow away from real sites to fake news, hate speech, and disinformation sites
Discovery of Fake Sites fake local news sites ads blocked on real sites

June 2021 / Page 40
marketing.science
Brand safety issues continue
Ad dollars flow to disinfo sites via programmatic; monitor exactly where your ads go
Global Disinformation Index
https://disinformationindex.org/wp-
content/uploads/2021/06/GDI_AdD
eck_ClimateChange_2June2021.pdf
TAG (Trustworthy Accountability Group)
BSI (Brand Safety Institute)
giving framed “certified” plaques and scarves
to members who paid the fee
(doesn’t reduce brand safety issues)

June 2021 / Page 41
marketing.science
Useless certifications, accreditations
“content-level brand safety” for YouTube but porn,child abuse videos remain rampant
https://www.adweek.com/programmatic/youtube-receives-
content-level-brand-safety-accreditation-from-media-rating-
council/
MRC
(Media Ratings Council)
“they measure what they
said they would measure”
TAG
(Trustworthy Accountability Group)
“self-attested, pay-to-play
certification”

June 2021 / Page 42
marketing.science
Fake audience segments by bots
Bots can emulate any desirable audience segment to earn higher prices from marketers
Journal of Clinical Oncology
Bots pretend to be oncologists
by collecting/replaying cookie
bots pretend to be
desirable audiences
Adweek, Feb 2018
https://www.forbes.com/sites/augustinefou/2021/01/08/bots-emulate-any-audience-you-want-to-target-heres-how/

June 2021 / Page 43
marketing.science
So What?

June 2021 / Page 44
marketing.science
Industry Associations’ View
Trade associations look at invalid traffic (IVT) by fraud detection; severely underestimates
“The 11 percent decline in
two years is particularly
noteworthy” – ANA (2019)
“IVT is between
1 – 3%”
– GroupM (2019)
“IVT is 1.05%”
– TAG (2020)
https://www.ana.net/miccontent/s
how/id/rr-2019-bot-baseline
https://www.tagtoday.net/pressrele
ases/2020-us-fraud-study-release
https://www.nexttv.com/news/groupm-
puts-risk-of-ad-fraud-at-22-4b-globally

June 2021 / Page 45
marketing.science
Site
Analytics
Ad
Analytics
“see fou yourself”
• alternative to Google Analytics
• secure, hardened against attack
• shows all details, no black box
• refined/updated w/ practitioners
• verify your own media/ads
• secure, hardened against attack
• shows details for decisioning
• recommended optimizations
for #publishers for #marketers

June 2021 / Page 46
marketing.science
FouAnalytics to continuously monitor
Measuring for bots/fraud is not enough; measure for humans, remove bad domains/apps
programmatic campaign large branding campaign
good publisher 1 good publisher 2
good publisher 3

June 2021 / Page 47
marketing.science
Turn off bad domains and apps
All clicks came between 12a – noon; many suspicious mobile apps causing large numbers of clicks
Clicks came from
highly suspicious
casino, gambling,
crime, lotto, scratch
off mobile apps
All impressions used up before noon

June 2021 / Page 48
marketing.science
Measure for Humans, not just for fraud
Optimizing for low CPMs may increase risk of ad fraud; optimize for humans instead
Lower quality paid
sources mean higher
cost per human – like
11X higher cost.
Sources of different
quality send widely
different amounts of
humans to landing pages.
“showing ads to humans is
the single most important
factor for better digital.”

June 2021 / Page 49
marketing.science
Improve outcomes by shifting spend
Shifting dollars from programmatic channels [B] to good publishers [A], direct buys help
Measure
Ads
Measure
Arrivals
Measure
Conversions
clean, good media
low-cost media, ad
exchanges
346
1743
5
156
30X better
outcomes
• More arrivals
• Better quality
A
B

June 2021 / Page 50
marketing.science
Recommendations
• Previous supply chain studies showed that 50% of a marketer’s dollar goes to supply
chain costs, leaving 50 cents for “working media” in ideal conditions
• This study provides additional examples of “measurable issues” that further eat
away at the “working media” leaving far less money for showing ads.
• Many other issues are not depicted in the waterfall chart; but they remain sources
of cost and risk for marketers buying ads through programmatic channels
• Marketers should run their own “turn-off” experiments; also experiment with using
fewer targeting parameters; business outcomes and ROI should go up
• Marketers should shift dollars to buying direct from good publishers, to reduce and
eliminate most of the supply chain costs and associated risks and waste

Alternative to ANA's end to end supply chain transparency study v final

Recomendados

Recomendados

Más contenido relacionado

La actualidad más candente

La actualidad más candente (20)

Similar a Alternative to ANA's end to end supply chain transparency study v final

Similar a Alternative to ANA's end to end supply chain transparency study v final (20)

Más de Dr. Augustine Fou - Independent Ad Fraud Researcher

Más de Dr. Augustine Fou - Independent Ad Fraud Researcher (20)

Último

Último (20)

Alternative to ANA's end to end supply chain transparency study v final