FouAnalytics - site analytics and media analytics for practitioners to detect fraud and take action themselves - on-site tags and in-ad tags measure sites and ad impressions, respectively
Section 3 - Technical Sales Foundations for IBM QRadar for Cloud (QRoC)V1 P10...
FouAnalytics DIY site media analytics fraud detection baked in
1. June 2020 / Page 0marketing.scienceconsulting group, inc.
fouanalytics.com
Site and Media Analytics
June 2020
Augustine Fou, PhD.
212. 203 .7239
2. June 2020 / Page 1marketing.scienceconsulting group, inc.
fouanalytics.com
Who am I?
• I am a digital marketer of 25+ years
• I look at analytics data every day
• I audit campaigns for fraud and other
anomalies that lower performance
• I teach clients how to spot fraud and
make optimizations to improve
outcomes themselves
3. June 2020 / Page 2marketing.scienceconsulting group, inc.
fouanalytics.com
Why did I build it? How is it different?
https://www.linkedin.com/pulse/does-your-analytics-
platform-have-cybersecurity-dr-augustine-fou-/
https://www.linkedin.com/pulse/fouanalytics-
alternative-google-analytics-fraud/
4. June 2020 / Page 3marketing.scienceconsulting group, inc.
fouanalytics.com
Site
Analytics
Media
Analytics
“see fou yourself”
• alternative to Google Analytics
• secure, hardened against attack
• shows all details, no black box
• innovated w/ practitioners
• verify your own media/ads
• secure, hardened against attack
• shows details for decisioning
• recommended optimizations
for #publishers for #marketers
5. June 2020 / Page 4marketing.scienceconsulting group, inc.
fouanalytics.com
Site Analytics
6. June 2020 / Page 5marketing.scienceconsulting group, inc.
fouanalytics.com
Answers these questions…
1. Which visitors to my site are valuable
humans, vs bots (which bots)?
2. Where did they come from (paid media?
organic search? social?)
3. What did they do on the site? Did they
complete a goal/conversion?
4. How do I get more valuable visitors?
7. June 2020 / Page 6marketing.scienceconsulting group, inc.
fouanalytics.com
Definitions
What each of the labels means
• Humans - 3 or more blue flags to confirm
• Some blue flags but not 3 or more
• Can’t label it either red or blue
• Tag was called, but no data was sent back (blocked)
• Tag was not called (not measurable)
• Bot – Search crawler
• Bot – Says its name honestly, (14,000 bot names)
• Some red flags, but not 3 or more
• Bots - 3 or more red flags to confirm
8. June 2020 / Page 7marketing.scienceconsulting group, inc.
fouanalytics.com
Time Series – 1 quarter
Long term trends
9. June 2020 / Page 8marketing.scienceconsulting group, inc.
fouanalytics.com
Time Series – 1 month
Monthly and daily pattern
10. June 2020 / Page 9marketing.scienceconsulting group, inc.
fouanalytics.com
Time Series – 1 week
Weekly pattern
11. June 2020 / Page 10marketing.scienceconsulting group, inc.
fouanalytics.com
Time Series – 1 day
Hourly volumes are visible
12. June 2020 / Page 11marketing.scienceconsulting group, inc.
fouanalytics.com
Locations of human visitors
Where did human visitors come from?
13. June 2020 / Page 12marketing.scienceconsulting group, inc.
fouanalytics.com
Analyze declared bots (orange)
Quickly isolate, analyze what bots are coming to the site
14. June 2020 / Page 13marketing.scienceconsulting group, inc.
fouanalytics.com
Analysis of declared bots
Quickly isolate and analyze what bots are coming to site
• Bots declared
themselves
(lyticsbot,
moatbot)
• Bots came from
data centers
15. June 2020 / Page 14marketing.scienceconsulting group, inc.
fouanalytics.com
Analysis of search bots
Quickly see which specific search bots arrived on the site
• Bots declared
themselves
(Googlebot,
Adsbot, Web
Preview)
• Bots came from
data centers
16. June 2020 / Page 15marketing.scienceconsulting group, inc.
fouanalytics.com
Filtering bots and data centers
Once you know which visitors are bots, you can filter them
On-Site measurement,
bots are still coming
In-Ad measurement, bots
and data centers filtered
11% red
-9% (filtered GIVT
and data centers)
2% red
17. June 2020 / Page 16marketing.scienceconsulting group, inc.
fouanalytics.com
Where do visitors click?
Click patterns on desktop and mobile screens, vs bots
18. June 2020 / Page 17marketing.scienceconsulting group, inc.
fouanalytics.com
Media Analytics
19. June 2020 / Page 18marketing.scienceconsulting group, inc.
fouanalytics.com
Answers these questions…
1. Where were my ads shown, actually
2. Were my ads shown to humans, plus
other characteristics like viewability,
duration, valid interaction, etc.
3. Which domains and apps were fraud or
sub-optimal (turn off in campaign)
20. June 2020 / Page 19marketing.scienceconsulting group, inc.
fouanalytics.com
Full Funnel Measurement
In-Ad
(ad tag)
On-Site
(embed code)
Measure
Ads
Measure
Arrivals
Measure
Conversions
1743 156
A
more humans (blue)
good publishers
On-Site
(embed code)
social
native
video
display
affiliate
21. June 2020 / Page 20marketing.scienceconsulting group, inc.
fouanalytics.com
Detect / reduce fraud in-flight
Launch Week 3 onwardWeeks 1-2
Initial baseline
measurement
Measurement after
first optimization
After eliminating several
“problematic” networks
Stacked percent chart
Blue (human)
Red (bots)
22. June 2020 / Page 21marketing.scienceconsulting group, inc.
fouanalytics.com
Shift dollars to more humans (blue)
Marketer 1
• Blue means humans
• Red means bots
Marketer 2
“recommendation is to increase spend on
sources driving more humans (blue); reduce
spend on sources with more bots (red)”
23. June 2020 / Page 22marketing.scienceconsulting group, inc.
fouanalytics.com
Shift budget to more human sources
Lower quality paid
sources mean higher
cost per human – like
11X higher cost.
Sources of different
quality send widely
different amounts of
humans to landing pages.
24. June 2020 / Page 23marketing.scienceconsulting group, inc.
fouanalytics.com
Where is “red” coming from?
What domains and apps should I consider turning off?
domains mobile apps
Turn off domains and apps that have the most red
25. June 2020 / Page 24marketing.scienceconsulting group, inc.
fouanalytics.com
Systematically reducing bots
• Bots can be programmed to send as much traffic and generate as many impresisons
and clicks as the advertiser wants
By systematically reducing ad
spend to ad networks and sites
that had the highest bots, and
increasing allocation to premium
publishers, the advertiser
increased ad impressions served
to humans, and lowered those
served to bots.
26. June 2020 / Page 25marketing.scienceconsulting group, inc.
fouanalytics.com
Total cost vs unit cost (CPM)
Pay much higher CPM, spend less total, get more outcomes
27. June 2020 / Page 26marketing.scienceconsulting group, inc.
fouanalytics.com
Media Analytics w/
Site & Conversion Tracking
28. June 2020 / Page 27marketing.scienceconsulting group, inc.
fouanalytics.com
Better media = better outcomes
Measure
Ads
Measure
Arrivals
Measure
Conversions
346
1743
5
156
A
B
30X more human
conversion events
• More arrivals
• Better quality
more humans (blue)
good publishers
low-cost media,
ad exchanges
29. June 2020 / Page 28marketing.scienceconsulting group, inc.
fouanalytics.com
Display 4
2,036 humans
human conversion rate
Humans click from ads to site
Site Traffic Conversions
8,482 818
4,216 humans
5%
human conversion rate
14,539 193
225 humans
9%
human conversion rate
2,248 23
168 humans
5%
human conversion rate
1,527 9
Display 3
Display 2
Display 1
Humans
40%
31. June 2020 / Page 30marketing.scienceconsulting group, inc.
fouanalytics.com
Measure conversions, correlate w/ humans
Organic sources
have more humans
(dark blue)
Conversion actions
(calls) show straight line
correlation to humans
32. June 2020 / Page 31marketing.scienceconsulting group, inc.
fouanalytics.com
26,796
1,116
Comparing organic vs paid sources
Marketer’s Site
Traffic
Measure
Conversions
organic
335 humans
66%
paid media
6,698 humans
1,043
222
4X higher
conversion rate
16%
human conversion rate
human conversion rate
33. June 2020 / Page 32marketing.scienceconsulting group, inc.
fouanalytics.com
+54%
Conversion Rate
4,248
1,812,906
= 0.23%
4,248
1,178,389
= 0.36%
Increase Conversion Rates
Make more accurate by subtracting bot traffic from denominator
Subtracted bots
34. June 2020 / Page 33marketing.scienceconsulting group, inc.
fouanalytics.com
+92%
Conversion Rate
4,248
1,812,906
= 0.23%
4,248
942,711
= 0.45%
Increase Conversion Rates
Make more accurate by subtracting bot traffic from denominator
Subtracted bots (red),
and unmeasured (white)
35. June 2020 / Page 34marketing.scienceconsulting group, inc.
fouanalytics.com
How to Install
36. June 2020 / Page 35marketing.scienceconsulting group, inc.
fouanalytics.com
In-ad code for ad impressions
In-Ad
Example from DCM
(copy and paste ad tag)
In-ad tag
In-ad tag
In-ad tag
JS tag in ad creative
37. June 2020 / Page 36marketing.scienceconsulting group, inc.
fouanalytics.com
Approved on Major Networks
38. June 2020 / Page 37marketing.scienceconsulting group, inc.
fouanalytics.com
On-site code for your own website
Website / Landing Pages
(copy and paste JS embed
code into page footer, just
like analytics code)
Example
Google Analytics
BotAnalytics
39. June 2020 / Page 38marketing.scienceconsulting group, inc.
fouanalytics.com
What it Collects
41. June 2020 / Page 40marketing.scienceconsulting group, inc.
fouanalytics.com
Triangulating collected parameters …
• The javascript “tag” collects details
about the browser like the version,
the operating system, screen
resolution, plugins, etc.
• Uses standard javascript APIs which
are set and maintained by W3C
• No personally identifiable
information is collected
• Two or more parameters are
triangulated together to
determine human vs bot via flags
• Users don’t interact with it in any
way – does not log in, enter any
data
• Cookies are never used
• Not all parameters are collected
for every visit
If { useragent = iPhone } but { screen resolution ≠ known resolutions }
If { IP address = data center }
If { plugins = none } If { language = none } If { window = 1 x 1 }
42. June 2020 / Page 41marketing.scienceconsulting group, inc.
fouanalytics.com
Performance Measures, Capacity on AWS
Average 30-40 million requests per hour
(peak 120 million /hr)
Average latency 4 - 5 milliseconds
43. June 2020 / Page 42marketing.scienceconsulting group, inc.
fouanalytics.com
Network timings waterfall - async
• Our js tag is set to async so it is
called after the page finishes
loading (2.33s in red)
• And all write-backs are done
subsequently.