Six months ago, your company rolled out a completely new version of your web app. Product management claims the rollout has been a great success, customer feedback has been positive, and they’re now pushing to sunset the old version. As the analyst, you’ve been asked for a few reports including total visitor sessions for the new version vs. the old version over the last month. You dutifully log in to your web analytics tool and pull some reports, and start building the comparison.
DATA SUMMIT 24 Building Real-Time Pipelines With FLaNK
Inflation: A hidden danger
1. Inflation: A hidden danger
ObservePoint ObservePoint DataAssurance
Inflation is a disease, a dangerous and sometimes fatal disease, a disease that if not checked in time can
destroy a society. — Milton Friedman
Consider this scenario.
Six months ago, your company rolled out a completely new version of your web app. Product management
claims the rollout has been a great success, customer feedback has been positive, and they’re now pushing to
sunset the old version. As the analyst, you’ve been asked for a few reports including total visitor sessions for
the new version vs. the old version over the last month. You dutifully log in to your web analytics tool and pull
some reports, and start building the comparison.
Don’t stake your good reputation on bad data.
2. Inflation: A hidden danger
ObservePoint ObservePoint DataAssurance
It seems that sessions for the old version aren’t as low as they ought to be in comparison to the new version.
Total visits are lower, but only slightly. And a few other key metrics are higher! You look at a few other reports,
and unfortunately, the numbers don’t support Product management’s recommendation.
You present your findings and recommend against sunsetting the old version for now because the metrics
show that user adoption needs work. Product management glares at you from across the table.
You are shocked by the numbers
3. Inflation: A hidden danger
ObservePoint ObservePoint DataAssurance
In a recent omnibus audit including 200 top Internet Retail web sites, ObservePoint found that across the
board, data quality problems are rampant. Missing data, data inflation, and data leakage are experienced by
even the most sophisticated digital marketers.
One lesser-known data quality problem, data inflation, is exactly what it sounds like: digital data that is
reported inaccurately high. Inflation can affect web analytics data such as traffic counts, advertising data such
as clicks and impressions, sales attribution data, or a variety of important metrics. And as the data gets passed
up into other systems, the effects of the inflation are magnified. The cause of digital data inflation is the
instance of duplicate tags.
What’s going on?
4. Inflation: A hidden danger
ObservePoint ObservePoint DataAssurance
When more than one instance of a tag fires on a page, data inflation happens. Each time a tag fires,
something gets counted, and when tag fires more than once, the thing is being counted more than once as
well.
Digital marketers have become much more disciplined about tag placement. In the past, tags might be placed
anywhere on the page – in the header, in the body, in the footer – in both, all three, just anywhere, or maybe
we were lucky if tags made it on the page at all.
But sometimes tags might be coded to multiple places on a page, transposed from other pages, or duplicated
in some other way. Regardless of the process that causes it, once the tag is duplicated, you’ve got data
inflation.
Duplicate tags inflate data
5. Inflation: A hidden danger
ObservePoint ObservePoint DataAssurance
More common today – especially among the advanced digital teams – is the deployment of a Tag Management
System. But no TMS can prevent the old-fashioned deployment of tags directly to pages (either on
purpose or on accident), or control tags that aren’t inside the TMS.
Our omnibus audit uncovered web sites where a TMS was deployed yet duplicate tags existed for some digital
measurement tools. This is a problem, since there might be a temptation to mentally ignore the tags if they’re
not represented inside the TMS.
Due to the sheer number of non-stakeholder code modifications to digital properties all tag deployments erode,
decay, and fall out of compliance with the initial “clean” deployment over time.
Maintaining accurate tagging is ultimately your responsibility as a digital marketer.
Common causes of data inflation
6. Inflation: A hidden danger
ObservePoint ObservePoint DataAssurance
Around ObservePoint, we have a mantra: The only thing worse than no data is bad data. This is a shared
conviction between our founders and every ObservePoint employee.
I should note, the scenario at the beginning of this article is a true story. After the fact, we were asked to audit
the sites in question. We found that the old product pages had 300-500% data inflation caused by duplicate
tags, and the company came to the belief that the company’s new product release was probably a success.
Unfortunately, this knowledge wasn’t available at the moment the data was being used. Executives made a
revenue-impacting decision, based on bad data.
You Must Be Vigilant Against Inflation