A few months ago, I had a quite unclear understanding of what audience measurement is. After been involved in managing marketing for a new local radio station, I first looked at how decisions were made by sellers and buyers in the advertising space market. I was rather disappointed by what I found. To summarize, I would say there are so many errors, bias and dysfunctions in traditional medias audience measurement that it seems it is very risky to take decisions based on this information. Thank you to professionals in this field who answered my questions, who shared their knowledge and opinions with me through my blog and who read back my posts in despite of my ingenuous ignorance. And sorry for my poor English. Tamara

1. My concern about media audience
measurments
Tamara Silina, February 2013

2. A few months ago, I had a quite unclear understanding of what audience
measurement is. After been involved in managing marketing for a new local radio
station, I first looked at how decisions were made by sellers and buyers in the
advertising space market. I was rather disappointed by what I found.
To summarize, I would say there are so many errors, bias and dysfunctions in
traditional medias audience measurement that it seems it is very risky to take
decisions based on this information.
Thank you to professionals in this field who answered my questions, who shared their
knowledge and opinions with me through my blog and who read back my posts in
despite of my ingenuous ignorance.
And sorry for my poor English.
Tamara
Thanks to Orson Welles for the Citizen Kane few images I borrowed.

3. It is not about
creating panic, but to first question: are we
inform… all going to die in
horrible suffering?
Patrick Chapatte, 28 février 2006, Le Temps.

4. What I heard from media
What I have heard about media strategies
What purpose do audience measurements serve?
New technical solutions
What we shall have to face

5. Here you are
5

6. What I have heard about old traditional media strategies
I selected three key ideas about what I was told about old media strategic
analysis of old traditional media groups.
The share of the traditional media (radio, TV, print...) in the ad space market
should decrease because of the inevitable decline of their audience.
Ad expenses should be transferred mainly to digital media and other new
areas of marketing investment.
Their strategy would be to extend as much as possible the period during
which traditional media can still be profitable.
To do this, it would be vital for them not to disturb the factors that keeps their
business especially audience measurement systems.
6

7. What I have heard about new and small traditional media
New entrepreneurs in traditional media business make different analysis:
Today, It is much easier and less expensive to create and operate new
traditional media with new production and broadcasting technologies and
new content sources.
New and small media would be more in synergy with Internet social
networks.
Their potential development would be higher because of their ability to deal
with proximity, community, and generation issues through targeted and
niches strategies.
But, as far as their audience is not measured by the current audience
measurement systems, their development is strongly limited.
7

8. But what is the use of audience measurements?
8

9. What I understood about the purpose of audience measurment
Audience measurement is mainly used in the advertising space market ,to give
a value to , and promote ad space. It also allows content managers to identify
who is listening, watching, reading, seeing what, in order to adjusts their
contents. But it is mainly the first function that determines the configuration of
audience measurement.
For advertisers and advertising agencies, audience measurement allows,
before a campaign, to identify spaces with the best cost / potential impact on
their target. During and after the campaign, it enables to evaluate some
campaigns performances.
9

10. In few words, how are audience measurement made?
Who take decisions? Associations of media, agencies and advertisers decide
which methods and means to implement. They choose, after bidding, research
agencies.
How is it done? For radio and print, telephone or self-administered surveys are
conducted. They may be continuous or by waves. Television audience
measurement use diaries surveys or people meters panels. For outdoor, they use
surveys or GPS tracking.
Some few systems use automatic individual people meters (radio and TV).
Who pays? These surveys and panels are mainly funded by the media who
want their audience to be measured.
10

11. Lie to me
11

12. Problems?
In trying to understand these processes, I was very surprised by malfunctions
that I did not expect to find in one of the first markets in the world economy.
I discovered that most transactions are based on approximate data and
techniques that would have horrified brokers and traders in the fifties.
12

13. 17 key problems
1. Systems do not match the needs of data users,
2. Measures that are not cross - media,
3. Poor definition of the population whose audience must be measured,
4. Weak segmentation and profiling,
5. Irrelevant size and structure of samples,
6. Wrong survey techniques,
7. Bias in recruiting,
8. Question that induce untrue answers,
9. Irrelevant delay and frequency in data collection and delivery,
10. Irrelevant presentation of delivered data,
11. Delayed access to the data that make it obsolete,
12. Non-compliance with specifications,
13. Lax controls,
14. Sidelining of the real experts,
15. Dishonest data manipulation,
16. Resistance of main media to change,
17. Resistance of media research institutes to change.
13

14. 1. Audience measurements don’t match the needs of data users
The first audience measurement function should be to help advertising space
buyers and sellers, to setup space "value".
To optimize and fluidify transactions in a market, the required information for
decision makers should be available by streaming and in real-time, through
consistent tools for consultation, analysis, placement of order and post-
evaluation.
In all structured markets, these functions are integrated into homogeneous,
optimized and controlled systems. In the advertising space market, the
data about programs and commercials monitoring, audience and forecasts,
and the tools to exploit them are provided by heterogeneous suppliers without
global vision of the users real needs, with virtually no control. Each of these
suppliers thinks he is doing its best just by coordinating with others. This type of
'organization' produces a low result.
14

15. What current systems measure, is mainly the exposure of respondents or
observed members of panels, to what media broadcast. But is this concept of
exposure still enough to define the value of the advertising space?
Don’t you think that, for example, involvement, interest, attention,
appreciation, loyalty to programs, would be taken into account by the
advertisers and agencies if that data was available?
Couldn’t we take into account the relationship between the consumers and
the products?
The audience measurements are designed today without a global vision
and without taking into account the real needs of those who determine the
effectiveness of advertising space and who develop the market space.
15

16. 2. No cross media audience measurement
For a potential buyer, the value of an offer advertising space, is related to his
target audience compared to those of all the alternative media and to their
complementarily, for the target coverage.
Media planner's work is not only finding spaces that cost less by contact, but also
identifying space assemblages of complementary media to optimize the
repetition on a specific share of the target. This is possible by spreading messages
through different types of media. Decision makers need to evaluate at the same
time the cost and complementarily of ad spaces of TV channels, radio stations,
newspapers, etc.. This requirement has been explicitly expressed by international
associations of advertisers and, in 2009, by The Coalition for Innovative Media
Measurement (cimmusorg.startlogic.com).
However, each year, new panels and new audience researches are set
up without they taking into account this requirement. It seems that associations
and groups who control audience measurements do not understand the issue.
16

17. 3. Inadequate definition of what is to be measured
More and more media are cross- national. For a radio or a television station
which broadcasts by satellite or on the Internet, the geographic area of its
potential audience is a linguistic territory. For example, Russian-speaking
channels are viewed in fourteen countries and Francophone channels, in forty
countries.
Many advertisers who work in several countries have already internationalized
their branding positioning according to linguistic and cultural territory. Their ad
space purchasing decisions are based on audience in these areas. To do this,
some of them have to conduct their own multi-country measurements.
International media as those who provide both satellite and cable ,have access
to inconsistent and heterogeneous audience measurement data. This is why
their advertising space is under-estimated.
17

18. 4. Irrelevant segmentation and profiling tools
The media audience is fragmented, it is more individual and independent. New
frameworks are needed to understand behavior and segmentation. Despite this,
people who are recruited to participate in surveys and panels are briefly
described and profiled. In fact, in most cases, the minimum criteria is used,
especially for television panels.
Yet the media, advertisers and agencies communicate their targeting criteria
and segmentation. They need segmented data according to these criteria for
their decisions. (http://www.wfanet.org/pdf/med_documents/Media_Charter_FR.pdf. Page 5).
Because they do not have such data, media planners are used to make risky
projections and interpolating.
A paradox: despite the fact that respondents are defined through only basic
criteria, it is generally very difficult and expensive to obtain data extractions using
these criteria.
What could drive those who designed these systems? Did they know that
restricting samples qualification and profiling and reducing access to native
data, would directly affect the media space demand?
18

19. 5. Irrelevant samples size and structure
All parts of a population are not worth the same as advertising targets. We
could therefore expect that the targets who generate most ad expenses,
would be 'covered' by more accurate and more frequent measurements.
But sample surveys and panels are most of the time structured as reduced
models of population, whose audience has to be measured while they should
over-represent segments of the population that are targeted by most
campaigns.
For this reason, a large part of the data is nearly never used.
With this model logic, to get large enough useful sub-samples, it is necessary
that the size of the entire sample is very large, which ultimately would result in
additional - and hardly justifiable- costs.
How is it possible that those who designed these systems, did not ask
themselves what information was really useful? Why were they so anxious to
have very large samples, so unnecessarily representative of the population that
is useless to data users?
19

20. 6. Survey system which does’t match all segments
Those who decide how audience measurement should be always want to use a
monolithic methodology to cover the entire population, all segments, all profiles,
the same way. This is probably in order to be able to compare audience
behavior of different segments. These doesn't match any need of any data user.
It is common sense that no data collection method can match all profiles. Some
are more adequate to some segments and others to other segments.
Those which can match for average profiles are not effective for higher social
categories. For instance, television people meters are not able to measure the
audience of high purchasing power consumers, who, by the way, are subject to
heavy advertising investments…
20

21. 7. Recruitment methods biases
I was lucky to be in touch with some media audience research teams. One of
the things that disturbs them the most, is the very low representativeness of
people who are recruited for these measurements.
One key factor is the low level of acceptance to participate in this kind of
survey or panel. A large part of the samples seems to be very difficult to
recruit. For phone surveys, 90% refuse and 99% do for panels & people meters.
It is very, very far below standards to get representative samples.
Some profiles - especially in the higher social categories – are just impossible to
recruit. This means that some key targets are just not represented in the
samples, even if it was in the scope of work and even if they appear in the
sample description of the research institute.
Contrary to what is sometimes said, no weighing method can correct these
problems.
Unfortunately, samples are widely made of atypical people who cannot
'represent 'real audience
21

22. 8. Questionnaires that produce wrong answers
For radio and print, most audience measurements are using self-administered
questionnaires or are administered by phone or Internet.
For radio, several researches have shown that a large part of people
answering, often don’t know what radio station they are listening to. And a
larger part of them don’t remember what stations they had listened to the day
before . At the same time, the most used information collected by these
surveys is the last day audience.
It is also shown that respondents tend to say the names of stations with a strong
awareness or reputation, rather than to say that they do not remember. This
greatly benefit the most well known radio stations. Some of the respondents
also say the names of the radio stations for which they have sympathy. Similar
phenomena are observed for newspapers and magazines audience.
http://www.fordham.edu/images/undergraduate/communications/audience%20measurement,%20diversity,%20and%201a.pdf
The result is that these survey methods induce wrong answers and more often,
media audiences which are better known ,are overvalued at the expense of
others.
22

23. 9. Dysfunctions due to delays and frequency
Radio and print audience measurement surveys are conducted four or five
times a year in most countries. They need four or six week to be conducted and
data are provided with a delay of several weeks. However, most campaigns last
no more than three weeks.
How could - we justify the quality of the space we sell with such delays? Imagine
a buyer who will receive information on the effectiveness of his campaigns
several months after? Some advertisers have to conduct their own audience
survey to be able to adjust their campaigns.
It is sometimes said by some experts about radio and press audience that
audience vary little. But it is not true, as shown by researches using daily surveys
(Arbitron, Radio audience measurement in New York using PPM).
23

24. 10. Irrelevant data access and delivery
Data supplied by research institutes that produce audience measurements
cannot be accessed through query tools that would cross numerous criteria.
However, such software are available.
According to some professionals, it appears that direct access to data would
make it easy to detect inconsistencies, oversized weighing and too low
representation of some profiles in the samples.
24

25. 11. Information are too old when delivered
In an ordinary market place, the more and the faster information is available,
the stronger is the market active. In most markets, everything is done to get real
time data. That's what I thought until I worked for a radio station.
In most radio stations, like the one I worked for, audience measurements are
made using a telephone survey. About a thousand people are interviewed on
everything they heard the day before. One might say that data every week, is
not so bad and it could stimulate the market. But it is not the case. The data is
processed once per quarter and it takes almost six weeks to have access to the
earliest data.
So by the end of January, I have data from November. I cannot use the data
to justify my new year prices. Advertisers cannot make a decision and they do
not buy my space. Why not treat and deliver data in a timely fit?
25

26. 12. Non-compliance with tenders specifications
The impossibility to reach norms, poorly controlled recruitment
specifications and data adjustments, drive institutes to use solutions that
degrade the quality.
Some profiles, for which recruitment is too difficult or too expensive, are
replaced by people out of the quotas: people of higher socio economic class
are replaced by lower class; people in deep rural areas are replaced by near
city countryside people, or suburban individuals, etc…
Some abusive weighing complements abusive recruitment. Subsamples of
difficult to recruit profiles, are over-weighed well beyond what can be
statistically acceptable. Some people may be counted ten times.
These practices are the main reason for the barriers that are erected to access
individual data from surveys and sorting purposes. They would make them
visible.
We should be wary of audience measurement systems which do not give
access to individual data to any user, even for reason of consistency check.
26

27. 13. Lax controls
In modern markets places such as the Stock Exchange, controlling systems are
very stringent. They focus on the design of systems, their operation, their integrity
and they check absence of abuse in the use of data. Each of these controls is
assigned to a separate institution whose activity is only controlling and auditing.
For audience measurements, the practices are much less stringent. These controls
are organized through committees in which members have participated in the
choice of the institute, or are involved in audience measurement operations.
The assessments are rather assigned to individual experts whose methods may
lack rigor and whose independence is not validated. Often, these experts have
worked in audience measurement leading institutes. In addition, these controls
are very infrequent.
Sometimes experts deliver two versions of their report: one for wide distribution,
with very little criticism, and another for the committee, which is much more
critical. In these cases, it shows who the final client of audience measurements is.
27

28. 14. Real experts sidelining
Experts who are requested by associations, for the design and control of
audience measurement systems, have, most of the time, been involved in
operating this kind of survey or panel. They are not scientists, they are
technicians. They just reproduce what they know, that is to say, the old
systems, and endorse the currents practices.
However, there are researchers who work and publish on these topics. I
found many papers that addressed the problems I have identified. These
researchers are never asked to advise or for technical auditing.
Researchers that I could speak to, are pessimistic about change in
audience measurement systems, because they think the current
organizations that drive them, do not match the new required systems and
they will not be as revolutionary as they should be today.
28

29. 15. Intentional alteration of data
More serious than the dysfunctions which are induced by aberrant
specifications, in some places, data is sometimes intentionally altered, that is to
say, deliberately changed. This happened at the request of media, content
producers or agencies.
Owners of media conglomerates explained to me how they had to pay to be
sure to get good audience figures.
Some engineers who programmed software for media audience measurement
,explained to me how audience of some channels can be routinely boosted by
a discrete weighing.
Moreover, some media planners have detected daily identical ratios for some
time - slots between the main channels, and for long periods.
All these examples are recent.
These practices may exist because the controls are really insufficient.
because the controls are really insufficient.
29

30. 16. Leading medias resistance to change
Groups and committees that lead audiences measurements are often
dominated by mainstream media. They manage to impose themselves
because they have an historical leadership and because they pay more. It is
therefore difficult to oppose their views.
The orientations of these mainstream media are naturally more conservative,
because they believe the relation between share of spend (ad expenditure)
and share of voice (share of audience) and current audience measurement
systems protect their share of voice dominance.
The subscription amount for a media to be measured, is often used by these
mainstream media as a barrier to exclude small media groups.
It is clear that if audience measurement were cheaper, the leadership
of mainstream media would be weakened. That’s why they are fonder of
heavy methodologies (with large samples) and they denigrate innovations that
could reduce cost.
30

31. 17. Leading institutes resistance to change
Audience measurements are highly profitable and not risky for leading research
agencies. In this time of crisis for the marketing research market, they are seen as
a very stable part of the business.
A contract for audience measurements generally last several years, it leaves an
over 30% margin and probability of renewal is high. Few industries have both
such a level and so low a risk.
For television audience measurement, there is a technical barrier to entry which
makes IT virtually impossible to win A tender for a challenger institute.
In some cases, major media are participating in the capital share holding of
institutes who measure their audience. This reinforces their conservatism. In other
market places, such a conflict of interest would be prohibited.
Sometimes, media research institutes are not afraid of conflicts of interest, and
may sell services to the media. "It is as if a manufacturer of thermometers sold
some aspirin." (E. Leser Slate. Http://www.arretsurimages.net/vite.php?id=13605).
Leading institutes will not add pressure to break the status-quo. They would fight
against new technologies that could reduce the cost of media research and
open the door to new competing institutes.
31

32. New technical solutions

33. New available systems
These are the Arbitron PPM, The GfK MediaWatch, Ipsos MediaCell, INVENTIVE
Technologies and maybe others I have not yet identified.
All of them can be used for both television and radio. They are portable and
individual.
Most of them use watermarking technology. They can measure only the
audience of channels whose signal include a special code.
Most are not integrated into overall systems of information and analysis for the
preparation of decision makers (buyers and sellers of ad space).
33

34. Arbitron PPM
TThis Portable People Meter is a small box like a pager. It uses watermarking. Some
versions could provide real time. Versions currently used transfer their data once a
day.
The fact that it is a case to carry always with you, may be a handicap. "It's a bit like
the electronic bracelet of custody" as described by one panelist.
This is a closed system (hard and software), highly protected by patents. Thus, it
can evolve very slowly, more slowly than technologies on which it relies, including
telecom, Internet and recognition signals. For this reason, it gradually loses its
advance.
It has been sold to many concession contracts and has partnerships with major
institutions that have not used it a lot.
www.arbitron.com/portable_people_meters/home.htm.
34

35. GfK MediaWatch
It is also a passive individual mobile people meter, integrated into a watch.
Its main advantage is that it is constantly kept by the panelist.
It does not provided in real time data because it is not connected. The panelist
must daily or weekly connect on a rack to upload data.
A module can integrate the newpapers and magazines audience
measurement (declarative).
It also useS watermarking, in spite the fact Gfk-Telecontrol had pushed far
enough research in recognition of media signals without watermarking.
GfK Telecontrol did not take into account that a watch is a very personal object,
which carries the image and the style of the person who wears it. They
developed quiet an ugly and very recognizable watch. This bias is a strong
handicap for MediaWatch panel recrutement.
It is also a closed system (hardware and software) and therefore it can evolve
only slowly while the technological environment is changing fast.
www.telecontrol.ch/mediawatch.html
35

36. Ipsos MediaCell
This is an app for recognition of sound signals with watermarking, which can be
incorporated in smart phones. It transmits in almost real time.
This device would be ideal but the watermarking algorithms don’t have the
capacity for recognition of low level sound neither in noisy environment
(discussion, engine noise, etc.).
This is not a hardware closed system like the two previously mentioned systems. It
will benefit other phone features such as localization, audience measurement
behavior of mobile Internet. This is one of the systems most likely to change
quickly.
www.ipsos.com/mediact/sites/ipsos.com.mediact/files/Mediacell.pdf
36

37. INVENTIVE Technologies
This system also uses panelists smart phones and tablets. It is the only system
which doesn't use watermarking.
The recognition technology for TV channels and radio stations can distinguish
their signal from strong ambient noises. It can measure the audience of any
channels and any radio stations without limitation. It measures audience share, it
provides audience data in real time (2-3 minutes). It includes a module for
measuring print audience (declarative).
It is integrated into a larger system with advertising monitoring tools and media
planning.
www.inventive-technologies.com
37

38. Digression on watermarking
The use of this technology requires each station / channel that wants to measure
its audience to incorporate a numeric code in the audio signal. It does not
measure audience of unmarked channels and stations.
Audience share, which is known to be the first indicator used for content
audience analysis cannot be measured.
It is almost never mentioned that the watermarking signal recognition is
malfunctioning for the radio in noisy environments (such as in cars, which is one of
the contexts in which the hearing of terrestrial radio is still virtually not challenged
by Internet).
The few independent technical tests conducted on traditional television people
meters (which use watermarking) have an average error of 10%. Tests even rarer
on passive people meters for radio rated more than 40% error. (Joe Pilota
www.imediaconnection.com).
The way marketing research institutes insist on watermarking is surprising when you
know its low performances. The reason for this preference could be that it forces
the media to pay to be measured.
The individual passive audience measurement can be performed without
watermarking and improved performance.
38

39. What to do for my radio?
39

40. What theory says?
”Decision makers prefer alternatives with less uncertainty. Information reduces
uncertainty. So buyers and sellers prefer bids supported by the latest, the most
relevant, the most exhaustive, the most reliable, the most continuous
information and in larger quantities”
C. Shannon (1948) « A Mathematical Theory of Decisions and Communications ».
That’s on of the key reasons why advertising space buyers divest traditional
media: the offer is not supported by this relevant information. Audience
measurements and buying and selling space systems them are handicaps for
the development of this market. It has to change;
40

41. What is said by the most advanced professionnals?
The required system to replace those currently operating is clearly outlined by
the Coalition for Innovative Media Measurement and by experts.
It should measure audience behavior by the same source (single source) and in
real-time, on and by any channel, for television, radio, Internet, print and
outdoor. It should include real time advertising monitoring.
For space buyers, it should be accessed through a new generation of decision-
making software, which would allow to automatically update in real time media
planning.
For space sellers, it would allow to update in real time pricing of space
depending on the audience and demand.
It should be built to easily connect its data to advertising effectiveness measuring
systems.
41

42. My ideal system?
The vision of the ideal system, for my radio, would be cover a wider market than
today to match my real targets (linguistic, geographical, lifestyle...) and other
types of media which match my targets and which could be complementary or
even competitors.
Measures would be analyzed by any advertising and editorial target.
It would include a mechanism for the space market (purchase and sale),
powered by…
Audience measurements: Cross media, reliable, targeted, continuous, available
in real time, including audience share of all media likely to compete with me
(no watermarking).
Monitoring: advertising, cross-media, real-time and integrated to audience data.
Structured data and detailed information on the broadcasts.
Tools to assist decision making for both space buyers (optimizing media plans
and impact assessment) and space sellers (optimizing prices).
It would be really controlled. These checks would be continuously performed by
true control professionals.
The whole system would be optimized, versatile, up to date and inexpensive.
42

43. How to start?
These new systems would probably not be initiated by existing organizations
for the reasons already mentioned (not cross media, resistant to change, etc).
They probably would be initiated by business clubs, cluster committees and in
countries where new professional structures are still open and by trans-
national media.
Once they will begin to operate, even for small panels, they should trigger
questioning, awareness and widespread rapidly.
43

44. What we shall have to face
44

45. What we will have to confront
Resistance of traditional mainstream media: who do not want to change
audience measurement systems because they benefit from the existing ones.
The resistance of some institutes: The current audience measurement systems
are a too important a part of their turnover. Top managers wish to continue this
activity, and for this reason, avoid changes they could not master and / or
which would be less profitable.
The inadequacy of existing professional groups: The new solutions may not be
supported by today professional groups as they are currently organized by
media type and by country. The new structure can form around the new
reference unit: The advertising target. For example, one of these systems could
be developed for the luxury goods market, for pregnant future mothers
market…
45

46. Incompetence: A small portion of players has a global vision of the situation
and real needs. Some do not have the skills to diagnose and others prefer no
change and adopt a "wait and see” attitude.
Denial: some decision makers who actually have the information to understand
the strategic dilemma can not admit the threats it involves and they wish to
hide reality, as did previous decision makers in other sectors (telecom, micro-
computer, vinyl disk, etc.).
The stereotype of the consensus: it is an often heard sentence said by
professionals and which would make you shudder in other industries: “ The
current system is certainly imperfect, but it is consensus." For some players, the
need of a common rule, of a flat consensus is more important than the right
strategy, quality and efficiency.
46

47. Preference for ambiguity: Some buyers and sellers prefer ambiguity which
make impossible the evaluation of the quality of their services. They are
aware of the limitations of their current efficiency, and in some cases,
because their practices are not to the benefit of their clients.
The weight of politics, sometimes: It happened in some countries that
provided audience data were skewed for political reasons.
The attempt to impose watermarking: when the need for change will be
binding for media research institutes, they will highlight systems that reinforce
them in the relations with the media, even if they are not technically the best.
.
47

48. Thanks