Sais.34.1

This issue is provided by
the Johns Hopkins University Press Journals Division
and powered by Project MUSE®

Terms and Conditions of Use
Thank you for purchasing this Electronic J-Issue from the Journals Division of the Johns
Hopkins University Press. We ask that you respect the rights of the copyright holder by
adhering to the following usage guidelines:
This issue is for your personal, noncommercial use only. Individual articles from this J-
Issue may be printed and stored on you personal computer.
You may not redistribute, resell, or license any part of the issue.
You may not post any part of the issue on any web site without the written permission of
the copyright holder.
You may not alter or transform the content in any manner that would violate the rights of
the copyright holder.
Sharing of personal account information, logins, and passwords is not permitted.

1ForewordSAIS Review vol. XXXIV no. 1 (Winter–Spring 2014)
1© 2014 by The Johns Hopkins University Press
Foreword
Following the exposure of the U.S. National Security Administration’s
(NSA) controversial surveillance program, there has been heated debate
surrounding the collection and storage of personal data. Our latest issue of
The SAIS Review of International Affairs, “Policy by Numbers: How Big Data
is Transforming Security, Governance, and Development,” seeks to move
beyond the sensationalism that has accompanied the NSA revelations. We
hope to provide readers a more nuanced perspective on the role of data
in international affairs, with a diverse collection of interviews, essays, and
opinion editorials from scholars, technologists, and policymakers.
We explore the rise of big data, in which governments and profit-
seeking organizations make policies and predictions based upon correla-
tions among massive quantities of data. We examine the trend toward open
data, in which governments provide valuable datasets directly to the public.
We assess the impact of data—positive and negative, international and do-
mestic—on public policy, national security, international development, and
individual well-being.
While the rise of big and open data is associated with promising ap-
plications, there are still vast uncertainties regarding how best to exploit
this technology. We hope that readers from the academic and public policy
communities will feel empowered to enhance their understanding of techni-
cal tools and data analysis, in an age where technological innovation often
outpaces government policy.
We begin with a conversation with Robert Kirkpatrick, Director of
the United Nations Global Pulse Initiative. The UN Global Pulse Initiative
collects and analyzes real-time data to better protect populations from
socioeconomic shocks. Kirkpatrick explores the challenges associated with
big data analytics, the surprising correlations among seemingly unrelated
datasets, and the initiative’s effort to predict food price crises with data
from social media.
Human rights data often impacts policy decisions. The next three
articles explore the opportunities and risks associated with collecting and
analyzing this sensitive information. Megan Price and Patrick Ball use
case studies of violent conflicts in Syria and Iraq to evaluate data-gathering
methodologies in conflict scenarios. They warn that datasets from conflict
scenarios are often subject to bias, and should not be used in isolation
to draw conclusions. Monti Narayan Datta argues that the collection of
quantitative data on modern day slavery has generated discussion in media
and among policymakers on how to mitigate and eradicate slavery. Our
interview with Arch Puddington, Vice President for Research at Freedom
House, discusses worldwide trends in freedom, and the impact of Freedom
House’s annual reports and indices.

2 SAIS Review Winter–Spring 2014
The rise of big and open data has a powerful impact on government
policymaking. Pongkwan Sawasdipakdi frames data as an information
weapon in the context of Thai domestic politics. She examines the govern-
ment’s rice-pledging scheme, and argues that contrasting datasets from the
government and opposition parties are used to gain political power and
credibility. Ian Kalin describes the theory and practice of open data policy
in the United States, and explains how government leaders can replicate
successful open data initiatives. Joel Gurin argues that open government
and open data can improve economic growth, transparency, and citizen
engagement. He also notes the obstacles for implementing open data initia-
tives in developing countries.
How can policymakers craft policies and frameworks that best take
advantage of big data? Given the fast pace of innovation and the slow pace
of policy, Kord Davis discusses how to bridge the gap between policymak-
ers and innovators. He identifies spaces where the public and private sector
can collaborate to produce effective and balanced policy. Aniket Bhushan
argues that the rise of big data and open data has created an opportunity
for disruptive innovation in international affairs. He offers examples related
to real-time macroeconomic analysis, humanitarian response, and poverty
measurement.
Data impacts national security and individual privacy, as well. Chris
Poulin outlines the processes of data collection and analysis, using case
studies from the Arab Spring, medical risk analysis, and his work at the
Durkheim Project, a data analysis initiative that seeks to predict and prevent
veteran suicides. David Rubin, Kim Lynch, Jason Escaravage, and Hillary
Lerner explain how to balance the opposing forces of opportunity and risk,
collective security and individual privacy, and innovation and protection
when using data for national security programs.
Finally, we look to China for lessons on data infrastructure. Eric Hagt
traces the history of China’s satellite navigation system, Beidou, and com-
pares its potential as a tool for development versus domestic and national
security. Margaret Ross statistically analyzes the risk of potential disrup-
tions to the global undersea cable communications network.
We conclude with analyses of influential literature and scholarly re-
search. Ilaria Mazzocco reviews Big Data: A Revolution That Will Transform
How We Live, Work, and Think by Viktor Mayer-Schönberger and Kenneth
Cukier. Bartholomew Thanhauser reviews Evgeny Morozov’s To Save Ev-
erything, Click Here: The Folly of Technological Solutionism.
We would like to thank our advisory board for their guidance in shap-
ing our exploration of data, our excellent editorial staff for their dedication
and persistence, and our authors for their thoughtful work on complex
global challenges. Their combined contributions made the publication of
“Policy by Numbers” possible.
Meghan Kleinsteiber Lauren Caldwell
Editor-in-Chief Senior Editor

3A Conversation with Robert KirkpatrickSAIS Review vol. XXXIV no. 1 (Winter–Spring 2014)
A Conversation with Robert
Kirkpatrick, Director of United
Nations Global Pulse
You are director of United Nations Global Pulse, an initiative to
leverage real-time data and analytics to monitor impacts of inter-
national and local shocks. How did the idea of Global Pulse come
about? What is your mission statement?
The initial idea of Global Pulse came about in the aftermath of the global
financial crisis. There was a recognition that we live in a hyper-connected
world where information moves at the speed of light, and crises and vul-
nerabilities can emerge quickly, but we’re still using two- to three-year-
old statistics to make most policy decisions. It was clear that there were
swathes of people being pushed below the poverty line almost overnight,
and we needed to modernize
our systems and capacities for
absorbing real-time informa-
tion for decision-making.
As a result, United Nations
Secretary-General Ban Ki-moon
established Global Pulse in 2009
to act as an innovation lab and
catalyst for the United Nations.
We bring together global de-
velopment experts, as well as
experts from academia and the
private sector, to explore how
analysis of big data can reveal faster insights about human well-being and
emerging vulnerabilities, in order to better protect populations from hunger,
poverty, and disease.
So Global Pulse’s mission is to accelerate the use of data science for
sustainable development and humanitarian action, to address systemic bar-
riers to adoption, and to cultivate a robust innovation ecosystem.
Robert Kirkpatrick is the director of UN Global Pulse, an initiative of the Executive
Office of the United Nations Secretary-General. The Global Pulse initiative explores
how Big Data and real-time analytics technologies can power a more agile approach to
sustainable development.
We . . . explore how digital data
sources and real-time analytics
technologies can help reveal
insights about human well-being
and emerging vulnerabilities,
in order to better protect
populations from shocks.

As the New York Times wrote in its August 2013 profile of Global Pulse,
the United Nations is often perceived as a “sprawling bureaucracy.”
What makes the Global Pulse team unique? What qualities—personal
and professional—do you seek in a team member?
Global Pulse is unique because we have an “intrapreneurial” approach. It
requires risk-taking and innovation to discover and generate new tools,
techniques, and methodologies to help the UN system and wider community
leverage new sources of real-time information and insights in the service
of humanitarian response and development work. This also requires a real
blend of expertise from within and outside of the UN. Due to the experi-
mental nature of our work, we are set up as a network of labs.
We have multidisciplinary teams working at our Pulse Labs in New
York, Jakarta, and Kampala that include data scientists and analysts, social
scientists, legal experts, and communications and partnerships specialists.
Pulse Lab teams design, scope, and co-create projects with UN agencies
and national institutions that provide sectoral expertise, and with private
sector or academic partners who provide access to data or analytical and
engineering tools.
When building a team, I look for “T-shaped people”—that is, people with
a broad range of skills and a flexible attitude, as well as deep knowledge of
one discipline, whether it is data science, design, partnership management,
or legal and privacy matters.
From what range of sources do you derive the data used for your
analyses? Which datasets do you consider to be the most unique or
surprising? What are the challenges associated with data collection
and analysis?
Global Pulse is interested in trends that reveal something about human
well-being, which can be revealed from data produced by people as they
go about their daily lives (sometimes known as “data exhaust”). Broadly
speaking, we have been exploring two types of data in the Pulse Labs. The
first is data that reflects “what people say,” which includes publicly available
content from the open web, such as tweets, blog posts, news stories, and so
forth. The second is data that reflects “what people do,” which can include
information routinely generated for business intelligence and to optimize
sales in a private sector company. An example of “what people do” data is
anonymized mobile phone traffic information, which can reveal everything
from footfall in a shopping district during rush hour to how a population
migrates after a natural catastrophe.
A dataset that may be surprising is postal data (the traffic and volume
of packages being shipped), which can be used as a proxy for GDP and eco-
nomic activity in a country or region. We are beginning a series of research
projects with the Universal Postal Union (UPU), the United Nations special-
ized agency for the postal sector, to explore this relationship further.
There are several challenges associated with moving this kind of
analysis out of an innovation lab and into practice, including the need to

5A Conversation with Robert Kirkpatrick
build skills and capacity around data science, the formation of sustain-
able partnerships with potential data providers in the private sector, and
identifying where new data and
insights can fit into the planning
and decision-making processes.
And most importantly, we must
take data protection and privacy
norms, policies, and techniques
to a new level to mitigate the po-
tential for misuse. Our mission
to find responsible ways of using
big data for global development
purposes does not include analyzing private or confidential information.
We follow, and advocate for, robust privacy protection principles.
Does Global Pulse focus on certain sectors? If so, why?
Although we can and do work with any part of the UN system that has a de-
velopment problem that data science might contribute to solving, there are
certain areas that are particularly well-suited to big data analysis. This year,
we will focus in particular on public health, including attitudes to health
as expressed on social media, news media, and patterns in anonymized
search data. For example, in partnership with the World Health Organiza-
tion, we are exploring whether early warning of non-communicable disease
risk factors in a country or community could be understood via analysis of
key words in social media data. We continue to look at parental attitudes
to immunizing children as expressed on social media, in order to address
misinformation that stops parents from protecting their children against
preventable diseases.
Another research priority is food security. In Indonesia, our Pulse Lab
Jakarta research team is exploring whether big data can provide insights
about the impacts of food price changes, in order to support the social pro-
tection policies of the government of Indonesia. Other areas of focus this
year include supporting humanitarian action through new data analytics
techniques, finding new ways to measure economic well-being, and using
digital data mining to help shape the priority development agenda that will
replace the Millennium Development Goals after they expire in 2015. Across
all sectors, though, Global Pulse conducts a range of activities to strengthen
the big data for development (BD4D) ecosystem by guiding the development
of regulatory frameworks and technical standards to address data-sharing
and privacy protection challenges. We support an emerging community of
practice to accelerate public sector adoption through advocacy, policy guid-
ance, and technical assistance.
How do you identify and maintain relationships with your private sec-
tor partners?
Private sector partners are incredibly important in helping us leverage big
data as a resource for sustainable development. Using big data responsibly
We must take data protection
and privacy norms, policies,
and techniques to a new level
to mitigate the potential for
misuse.

and effectively requires several different elements, so there are different
areas of expertise, knowledge, and resources we look for when building
partnerships. The Global Pulse network of partners and collaborators in-
cludes forward-thinking private sector companies that are willing to engage
in “data philanthropy,” by granting access to data and technology tools to
the public sector. Our network also includes industry leaders, universities,
research institutes, and non-profit networks of researchers and innovators
who are ready to bring their skills and expertise to bear for advancing the
use of data science across the global development and humanitarian fields.
To establish and maintain these relationships, we have a partnership
manager and a privacy and legal expert, both of whom help guide potential
partners through the process. They work with counterparts in the com-
panies to ensure that safeguards, legal agreements, and data protection
principles are in place. Once collaboration is underway, our research team
will work closely with data analysts in the partner organization to initiate a
project or exploration. Often, the data never leaves the business that owns it;
rather, our data scientists guide the process and then the trends or results
are shared. This modality works well when the data is sensitive.
The experience with our partners, overall, is one of mutual learning.
The Pulse Lab network offers a safe “sandbox” for de-risking this type of
experimentation as we all learn together how the public and private sector
can responsibly harness big data for development.
Could you share a few Global Pulse success stories? Similarly, which
development challenges (such as particular regions or issues) are par-
ticularly difficult to tackle?
There are success stories on the data philanthropy front in which telecom-
munications companies have made anonymized datasets available as part
of a competition or challenge. For example, last year we collaborated with
Orange Telecom to host a “Data for Development Challenge” in which the
company opened up a dataset of anonymized mobile phone data to more
than eighty research teams from around the world to analyze. This research
garnered insights that the international development community can be
inspired by or learn from.
In terms of projects we are carrying out, as I mentioned previously, our
Pulse Lab in Indonesia is conducting research on mining tweets to under-
stand food price crises. Their research has provided new insights into the
very real problem of sudden increases in the price of staple foodstuffs, like
rice prices, pushing families below the poverty line and causing regional
economic instability. Real-time information about these impacts could help
policymakers and governments provide support to families who are suffer-
ing as a result of food price hikes.
Going forward, we plan to conduct further research on social media
analysis for food security and for crowdsourcing food prices, since these are
areas of focus for the government of Indonesia, and will be applicable in
many other parts of the world. Certainly, this is not a solution that can be

7A Conversation with Robert Kirkpatrick
applied universally. Social media analysis is of limited use in countries where
internet penetration is low, and even in regions of a country where the digi-
tal divide is vast.
There are also ana-
lytical challenges
yet to be resolved,
including the ne-
cessity of build-
ing technologies
that can support
diverse local lan-
guages.
We are in the
early days of dis-
covering how big data can be applied to development and humanitarian
contexts, and there are diverse challenges ranging from data access and the
capacity to use real-time data in decision-making to data privacy. These
challenges will be addressed over time.
As Viktor Mayer-Schönberger and Kenneth Cukier wrote, big data is
revolutionizing the way we solve problems. You have noted several ways
that data collection and analysis is helping Global Pulse address global
development challenges. But does the shift toward big data analytics
have drawbacks for the field of international development, as well?
There are risks rather than drawbacks. I hear a lot about the supposed fear
that all development decisions would be made by algorithms. This is not
a realistic fear, but rather a false dichotomy between quantitative use of
data in decision-making and policymakers using qualitative experiences to
decide a course of action. Big data will always be one part of a solution, not
the only solution. Of course, we need research, official statistics, and the
deep knowledge of field workers, communities, and practitioners, but big
data and data science represents a useful addition to the development and
humanitarian worker’s toolbox.
Another common misperception is that real-time data would replace
official statistics, but this assumption is unrealistic, as well. Official statis-
tics will continue to provide high-quality snapshots of progress that can
be benchmarked. But increasingly, between those annual, bi-annual, or
monthly updates, real-time data sources will provide valuable interim feed-
back and indicators. This feedback can enable course-correction when it is
evident that a program isn’t working. And real-time data can reveal shifts
in food-pricing, population changes, or disease outbreaks within a day or
an hour, rather than a month or a year.
We are in the early days of discovering how
big data can be applied to development
and humanitarian contexts, and there
are diverse challenges ranging from data
access and the capacity to use real-time
data in decision-making to data privacy.

Many graduate schools of international affairs, including the Johns
Hopkins School of Advanced International Studies (SAIS), offer cours-
es or concentrations in international development. Do you find that
these programs offer adequately rigorous quantitative or data analysis
requirements? What skills should students develop if they intend to
enter the field of international development?
There is a need for greater skills capacity for data analysis—this is something
that is needed across the board and not only in our field. The international
development practitioner of the future will be someone who is data literate,
and capable of using data analysis to inform his or her understanding and
decision-making. So yes, we’d like to see graduate schools covering data
for development
and the process-
es involved. Just
as the schools of
journalism are
now teaching data
journalism, all
students must un-
derstand how to
identify credible
informative sources, how to perform—or at least understand quantitative
statistical and data analysis, and how to appropriately use data to inform
their judgment. The good news is that I see a lot of appetite for these skills
from current students, so this change is beginning to happen.
The international development practitioner
of the future will be someone who is data
literate, and capable of using data analysis
to inform his or her understanding and
decision-making.

9SAIS Review vol. XXXVI no. 1 (Winter–Spring 2014)
Big Data, Selection Bias, and the
Statistical Patterns of Mortality in
Conflict
Megan Price and Patrick Ball
The notion of “big data” implies very specific technical assumptions. The tools that have made
big data immensely powerful in the private sector depend on having all (or nearly all) of the
possible data. In our experience, these technical assumptions are rarely met with data about
the policy and social world. This paper explores how information is generated about killings
in conflict, and how the process of information generation shapes the statistical patterns in
the observed data. Using case studies from Syria and Iraq, we highlight the ways in which
bias in the observed data could mislead policy. The paper closes with recommendations about
the use of data and analysis in the development of policy.
Introduction
Emerging technology has greatly increased the amount and availability
of data in a wide variety of fields. In particular, the notion of “big data”
has gained popularity in a number of business and industry applications,
enabling companies to track products, measure marketing results, and in
some cases, successfully predict customer behavior.1
These successes have,
understandably, led to excitement about the potential to apply these meth-
ods in an increasing number of disciplines.
Megan Price is the director of research at the Human Rights Data Analysis Group. She
has conducted data analyses for projects in a number of locales including Syria and
Guatemala. She recently served as the lead statistician and head author of two reports
commissioned by the Office of the United Nations High Commissioner of Human
Rights.
Patrick Ball is the executive director of the Human Rights Data Analysis Group.
Beginning in El Salvador in 1991, Patrick has designed technology and conducted
quantitative analyses for truth commissions, non-governmental organizations, domestic
and international criminal tribunals, and United Nations missions. Most recently, he
provided expert testimony in the trial of former de facto President of Guatemala, Gen.
José Efraín Ríos Montt.
The materials contained herein represent the opinions of the authors and editors and should not be
construed to be the view of HRDAG, any of HRDAG’s constituent projects, the HRDAG Board of
Advisers, the donors to HRDAG, or this project.

Although we share this excitement about the potential power of data
analysis, our decades of experience analyzing data about conflict-related
violence motivates us to proceed with caution. The data available to hu-
man rights researchers is fundamentally different from the data available
to business and industry. The difference is whether the data are complete.
In most business processes, an organization has access to all the data: every
item sold in the past twelve months, every customer who clicked through
their website, etc. In the exceptional cases where complete data are unavail-
able, industry analysts are often able to generate a representative sample of
the data of interest.2
In human rights, and more specifically in studies of conflict violence,
we rarely have access to complete data. What we have instead are snapshots
of violence: a few videos of public killings posted to YouTube, a particular
set of events retro-
spectively recorded by
a truth commission,
stories covered in the
local or international
press, protesters’ SMS
messages aggregated
onto a map, or victims’ testimonies recorded by non-governmental human
rights organizations (NGOs) are typical sources. Statistically speaking, these
snapshots are “convenience samples,” and they cover an unknown propor-
tion of the total number of cases of violence.3
It is mathematically difficult,
often impossible, to know how much is undocumented and, consequently,
missing from the sample.
Incompleteness is not a criticism of data—collecting complete or rep-
resentative data under conflict conditions is generally impossible. The chal-
lenge is that researchers and advocates naturally want to address questions
that require either the total number or a representative subset of cases of
violence. How many people have been killed? What proportion was from
a vulnerable population? Were more victims killed last week or this week?
Which perpetrator(s) are committing the majority of the violence? Basing
answers and policy decisions on analyses of partial datasets with unknown,
indeed unknowable, biases can prove to be misleading. These concerns
should not deter researchers from asking questions of data; rather, it should
caution them against basing conclusions on inadequate analyses of raw data.
We conclude by suggesting methods from several quantitative disciplines
to estimate the bias in direct observations.
The Problem of Bias
When people record data about events in the world, the records are almost
always partial; reasons why the observation of violence often misses some or
most of the violence are presented in the examples to follow. Most samples
are partial, and in samples not collected randomly, the patterns of omission
may have structure that influence the patterns observed in the data. For ex-
ample, killings in urban areas may be nearly always reported, while killings
In human rights, and more specifically
in studies of conflict violence, we rarely
have access to complete data.

11Big Data, Selection Bias, and Mortality in Conflict
in rural areas are rarely documented. Thus, the probability of an event be-
ing reported depends on where the event happened. Consequently, analysis
done directly from this data will suggest that violence is primarily urban.
This conclusion is incorrect because the data simply do not include many
(or at least proportionally fewer) cases from the rural areas. In this case, the
analysis is finding a pattern in the documentation that may appear to be a
pattern in true violence—but if analysts are unaware of the documentation
group’s relatively weaker coverage of the rural areas, they can be misled
by the quantitative result. In our experience, even when analysts are aware
of variable coverage in different areas, it is enormously difficult to draw a
meaningful conclusion from a statistical pattern that is affected by bias.
Statisticians call this problem “selection bias” because some events
(in this example, urban ones) are more likely to be “selected” for the sample
than other events (in this example, rural ones). Selection bias can affect
human rights data collection in many ways.4
We use the word “bias” in the
statistical sense, meaning a statistical difference between what is observed
and what is “truth” or reality. “Bias” in this sense is not used to connote
judgment. Rather, the point is to focus attention on empirical, calculable
differences between what is observed and what actually happened.
In this article, we focus on a particular kind of selection bias called
“event size bias.” Event size bias is the variation in the probability that a
given event is reported, related to the size of the event: big events are likely
to be known, small events are less likely to be known. In studies of conflict
violence, this kind of bias arises when events that involve only one victim
are less likely to be documented than events that involve larger groups of
victims. For example, a market bombing may involve the deaths of many
people. The very public nature of the attack means that the event is likely to
attract extensive attention from multiple media organizations. By contrast,
an assassination of a single person, at night, by perpetrators who hide the
victim’s body, may go unreported. The victim’s family may be too afraid to
report the event, and the body may not be discovered until much later, if at
all. These differences in the likelihood of observing information about an
event can skew the available data and result in misleading interpretations
about patterns of violence.5
Case Studies
We present here two examples from relatively well-documented conflicts.
Some analysts have argued that information about conflict-related killings
in Iraq and Syria is complete, or at least sufficient for detailed statistical
analysis. In contrast, our analysis finds that in both cases, the available data
are likely to be systematically biased in ways that are likely to confound
interpretation.
Syria
Many civilian groups are currently carrying out documentation efforts in the
midst of the ongoing conflict in Syria. In early 2012, the United Nations Of-
fice for the High Commissioner for Human Rights (OHCHR) commissioned

the Human Rights Data Analysis Group (HRDAG) to examine datasets from
several of these groups, and in two reports, Price et al. provide in-depth de-
scriptions of these sources.6
In this section, we focus our attention on four
sources—in essence, lists of people killed—which cover the entire length of
the ongoing conflict and which have continued to provide us with updated
records of victims. These sources are the Syrian Center for Statistics and Re-
search7
(CSR-SY), the Syrian Network for Human Rights8
(SNHR), the Syria
Shuhada website9
(SS) and the Violations Documentation Centre10
(VDC).
Figure 1 shows the number of victims documented by each of the four
sources over time within the Syrian governorate of Tartus. The large peak
visible in all four lines in May 2013 corresponds to an alleged massacre in
Banias.11
It appears that all four sources documented some portion of this
event. Many victims were recorded in the alleged massacre, this event was
very well reported, and all four of our sources reflect this event in their lists.
However, three out of the four sources document very little violence occur-
ring before or after May 2013 in Tartus. The fourth source, VDC, shows the
peak of violence in May as the culmination of a year of consistent month-
to-month increases in the number of reported killings.
When interpreting figures such as Figure 1, we should not aim to iden-
tify a single “correct” source. All of these sources are documenting different
snapshots of the violence, and all of them are contributing substantial num-
bers of unique records of victims undocumented by the other sources.12
The
presence of event size bias is detectable in this particular example because
all four of the sources obviously captured a similar event (or set of events)
in May 2013, while at the same time one of those sources captured a very
different subset of events during the preceding months. If we did not have
access to the VDC data, our analysis of conflict violence in Tartus would
incorrectly conclude that the alleged massacre in May 2013 was an isolated
event surrounded by relatively low levels of violence.
The conclusion from Figure 1 should not be that VDC is doing a
“better” job of documenting victims. VDC is clearly capturing some events
that are not captured by the other sources, but there is no way to tell how
many events are not being captured by VDC. From this figure alone we
cannot conclude what other biases may be present in the observed data.
For example, the relatively small peak in February 2012 could be as small
as it seems, or it could be as large as the later peak in May 2013. Without a
method of statistical estimation that uses a probability model to account
for the undocumented events, it is impossible to know.13
To underline this crucial point: despite the availability of a large
amount of data describing violence in Tartus, there is no mathematically
sound method to draw conclusions about the patterns of violence directly
from the data (though it is possible to use the data and statistical models to
estimate how many events are missing). The differences in the four sources
available to us make it possible to detect the event size bias occurring in May
2013, but what other biases might also be present in this observed data and
hidden from view? What new events might a fifth, sixth, or seventh source
document? Are there enough undocumented events such that if they were

13BIG DATA, SELECTION BIAS, AND MORTALITY IN CONFLICT
included, our interpretation of the patterns would change? These are the
crucial questions that must be examined when interpreting perceived pat-
terns in observed data.
Iraq
We detect a subtler form of event size bias in data from the Iraq Body Count
(IBC), which indexes media and other sources that report on violent deaths
in Iraq since the Allied invasion in March 2003.14
Our analysis is motivated
by a recent study by Carpenter et al., which found evidence of substantial
event size bias.15
Their approach was to compare the U.S. military’s “sig-
nificant acts” (SIGACTS) database to the IBC records. As they report, this
comparison showed that “[e]vents that killed more people were far more
likely to appear in both datasets, with 94.1% of events in which ≥20 people
were killed being likely matches, as compared with 17.4% of … killings [that
occurred one at a time].”16
This implies that IBC, SIGACTS, or both, capture
a higher fraction of large events than small events. Carpenter et al. go on
Figure 1. Number of Victims Documented by Four Sources, Over Time, in Tartus

to note that “[t]he possibility that large events, or certain kinds of events
(e.g., car bombs) are overrepresented might allow attribution that one side
in a conflict was more recklessly killing civilians, when in fact, that is just
an artifact of the data collection process.”17
Motivated by this analysis, we considered other ways to examine IBC
records for evidence of potential event size bias. Since IBC aggregates re-
cords from multiple sources, updated IBC data already incorporates many
records from SIGACTS.18
In contrast to the work of Carpenter et al., who
treated IBC and SIGACTS as two separate data sources and conducted their
own independent record linkage between the two sources, we examined only
records in the IBC database, including those labeled as from SIGACTS.
It should be noted that we conducted this analysis on a subset of the
data after filtering out very large events with more than fifty victims. We
made this choice because, on inspection, many of the records with larger
numbers of reported victims are data released in batches by institutions
such as morgues, or incidents aggregated over a period of time, rather than
specific, individual events.
We began by identifying the top one hundred data sources; one or
more of the top one hundred sources cover 99.4 percent of the incidents
in IBC.19
Given these sources, we counted the number of sources (up to
one hundred) for each event. Event size was defined as the mean (rounded
to the nearest integer) of the reported maximum and minimum event size
values. Then the data were divided into three categories: events with one
victim, events with two to five victims, and events with six to fifty victims.
The analysis was performed on these groups.
Figure 2 summarizes our findings. The shading of each bar in Figure
2 indicates the proportion of events of that size reported by one, two, or
three or more sources. For each category of event sizes, most events have
two sources. For events of size one, the second most frequent number of
sources is one, accounting for nearly a third of all events of this size; almost
no single-victim events have three or more sources. The number of events
with three or more sources increases quickly in medium-sized events and
in large events. Relatively few of the largest events are reported by a single
source. Thus there seems to be a relationship between event size and the
number of sources: larger events are captured by more sources. This rein-
forces the finding by Carpenter et al. that larger events are more likely to
be captured by both IBC and SIGACTS. We have generalized this finding to
the top one hundred sources; larger events are more likely to be captured
by multiple sources.
The number of sources covering an event is an indicator of how “inter-
esting” an event is to a community of documentation groups—in this case,
media organizations. The pattern shown in Figure 2 implies that media
sources are more interested in larger events than smaller events. Greater
interest in the larger events implies that larger events are more likely to be
reported (observed) by multiple sources relative to smaller events. Since a
larger proportion of small events are covered by only a single source, it is
likely that more small events are missed, and therefore excluded from IBC.20

As noted by Carpenter et al., the correlation between event attributes
and the likely reporting of those events can result in highly misleading in-
terpretation of apparent patterns in the data. As a relatively neutral example,
analysts might erroneously conclude that most victims in Iraq were killed
in large events, whereas this may actually be an artifact of the data collec-
tion. A potentially more damag-
ing, incorrect conclusion might
be reached if large events are
centered in certain geographic
regions or attributed to certain
perpetrators; in these cases,
reading the raw data directly
would mistake the event size
bias for a true pattern, thereby
misleading the analyst. Inap-
propriate interpretations could result in incorrect decisions regarding
security measures, intervention strategies, and ultimately, accountability.
The correlation between event
attributes and the likely reporting
of those events can result in
highly misleading interpretation
of apparent patterns in the data.
Figure 2. Proportion of Events Covered by One, Two, or Three or More Sources

Discussion
Event size bias is one of many kinds of selection and reporting biases that
are common to human rights data collection. It is important to recall that
we refer here to biases in the statistical sense: a measurable difference be-
tween the observed sample and the underlying population of interest. The
biases that worry us here affect statistics and quantitative analyses; we are
not implying that the political goals of the data collection groups have
influenced their work.
In the context of conflict violence, meaningful statistical analysis
involves comparisons to answer questions such as: Did more violence oc-
cur this month or last month? Were there more victims of ethnicity A or
B? Did the majority of the
violence occur in the north or
the south of the country? The
concern about bias focuses
on how the data collection
process may more effectively
document one month rela-
tive to another, creating the
appearance of a difference
between the months. Unfortu-
nately, the apparent difference is the result of changes in the documentation
process, not real changes in the patterns of violence.
To make sense of such comparisons, the observed data must in some
way be adjusted to represent the true rates. There are a number of methods
for making this adjustment if the observed data were collected at random,
but this is rarely the case. There are relatively few models that can adjust
data that were collected because it was simply available.
In order to compare nonrandom data across categories like months
or regions, the analyst must assume that the rate at which events from
each category are observed is the same. For example, 60 percent of the total
killings were collected in March, and 60 percent of the total killings were
collected in April. This rate is called the coverage rate, and it is unknown,
unless somehow the true number of events were known or estimated. If the
coverage rates for different categories are not the same, the observed data
tell only the story of the documentation; they do not indicate an accurate
pattern. For example, if victims of ethnicity A are killed in large-scale vio-
lent events with many witnesses, while victims of ethnicity B are killed in
targeted, isolated violent events, we may receive more reports of victims of
ethnicity A and erroneously conclude that the violence is targeted at eth-
nicity A. Until we adjust for the event size bias resulting in more reports of
victims of ethnicity A, we cannot draw conclusions about the true relation-
ship between the number of victims from ethnicity A versus B.
There are many other kinds of selection bias. As an example, when rely-
ing on media sources, journalists make decisions about what is considered
newsworthy. Sometimes their decisions may create event size bias, as large
. . . the apparent difference
is the result of changes in the
documentation process, not
real changes in the patterns of
violence.

events are frequently considered newsworthy. But the death of individual,
prominent members of a society are frequently also considered newswor-
thy. Conversely, media “fatigue” may result in under-documentation later
in a conflict, or when other newsworthy stories may limit the amount of
time and space available to cover victims of a specific conflict.21
Many other
characteristics of both the documentation groups and the conflict can result
in these kinds of biases such as logistical or budgetary limitations, trust or
affinity variations within the community, and the security and stability of
the situation on the ground.22
As each of these factors changes, coverage
rates are likely to change as well.
The fundamental reason why biases are so problematic for quantita-
tive analyses is that bias often correlates with other dimensions that are
interesting to analysts, such as trends over time, patterns over space, differ-
ences compared by the victims’ sex, or some other factor. As in the example
of ethnicities A and B above, the event size bias is correlated with the kind
of event. Failing to adjust for the reporting bias leads to the wrong conclu-
sion. As another example, consider the Iraq case described above: If event
size is correlated with the events’ perpetrators, then bias on event size means
bias on perpetrator, and a naïve reading of the data could lead to security
officials trying to solve the wrong security problems. Or, in the Syria case,
if decisions about resource allocation to Tartus were made on the basis of
the observed information, without taking into account the patterns of kill-
ings that were not observed, researchers may have inaccurately concluded
that violence documented in May 2013 represented an isolated event. One
could imagine that such a conclusion could lead to any number of incorrect
decisions: sending aid groups into Tartus under the erroneous assumption
of relative security, or failing to send aid and assistance before or after May
2013, assuming that such resources were more in need elsewhere.
It is important to note that these challenges frequently lack a scientific
solution.23
We do not need to simply capture more data. What we need is
to appropriately recognize and adjust for the biases present in the available
data. Indeed, as indicated in the Iraq example, where multiple media sources
appear to share similar biases, the addition of more data perpetuates and
in some cases amplifies the event size bias.
Detection of, and adjustment for, bias requires statistical estimation.
A wide variety of statistical methods can be used to adjust for bias and es-
timate what is missing from observed data. In our work we favor multiple
systems estimation, which has been developed under the name capture-
recapture in ecology, and used to study a variety of human populations
in research in demography and public health. Analysts more familiar with
traditional survey methods often prefer adjustments based on post-stratifi-
cation or “raking,” each of which involves scaling unrepresentative data to a
known representative sample or population.24
Each method has limitations
and requires assumptions, which may or may not be reasonable, but formal
statistical models provide a way to make those assumptions explicit, and in
some cases, to test whether they are appropriate. Comparisons from raw data
implicitly but necessarily assume that such snapshots are statistically repre-
sentative. This assumption may sometimes be true, but only by coincidence.

Conclusions
Carpenter et al. warn that “press members and scientists alike should be cau-
tious about assuming the completeness and representativeness of tallies for
which no formal evaluation of sensitivity has been conducted. Citing partial
tallies as if they were scientific samples confuses the public, and opens the
press and scholars to being manipulated in the interests of warring parties.”
In a back-of-the-envelope description elsewhere, we have shown that small
variations in coverage rates can lead to an exactly wrong conclusion from
raw data. 25
Groups such as the Iraq Body Count, the Syrian Center for Statistics
and Research, the Syrian Network for Human Rights, the Syria Shuhada
website, and the Violations Documentation Centre collect invaluable data,
and they do so systematically, and with principled discipline. These groups
should continue to collate and share it as a fundamental record of the past.
The data can also be used in qualitative research about specific cases, and in
some circumstances, in statistical models that can adjust for biases.
It is tempting, particularly in emotionally charged research such as
studies of conflict-related violence, to search available data for answers. It is
intuitive to create infographics, to draw maps, and to calculate statistics and
draft graphs to look for patterns in the data. Unfortunately, all people—even
statisticians—tend to draw conclusions even when we know that the data
are inadequate to support comparisons. Weakly founded statistics tend to
mislead the reader.
Statistics, graphs, and maps are seductive because they seem to prom-
ise a solid basis for conclusions. The current obsession with using data to
formulate evidence-based policy increases the pressure to use statistics, even
as new doubts emerge about whether “big data” predictions about social
conditions are accurate.26
When calculations are made in a way that enables
a mathematical foundation for statistical inference, these statistics deliver
on the promise of an objective
measurement in relation to a
specific question. But analysis
with inadequate data is very hard
even for subject matter experts
to interpret. In the worst case,
it offers a falsely precise view,
a view that may be completely
wrong. In the best case, it invites speculation about what’s missing and
what biases are uncontrolled, creating more questions than answers, and
ultimately, a distraction. When policymakers turn to statistical analysis to
address key questions, they must assure that the analysis gives the right
answers.
Statistics, graphs, and maps are
seductive because they seem
to promise a solid basis for
conclusions.

Notes
1
One extreme example includes Target successfully predicting a customer’s pregnancy, as
reported in the New York Times and Forbes. In particular, Target noticed that pregnant women
buy specific kinds of products at regular points in their pregnancy, and the company used
this information to build marketing campaigns.
2
However it is certainly worth noting that even in these contexts sometimes big data are
not big enough and may still be subject to the kinds of biases we worry about in this paper.
See Kate Crawford’s keynote at STRATA and Tim Harford’s recent post on Financial Times
for examples.
3
Specifically, “convenience samples” refer to data that is non-randomly collected, though
collecting such data is rarely convenient.
4
Another common kind of bias that affects human rights data is reporting bias. Whereas se-
lection bias focuses on how the data collection process identifies events to sample, reporting
bias describes how some points become hidden, while others become visible, as a result of
the actions and decisions of the witnesses and interviewees. For an overview of the impact
of selection bias on human rights data collection, see Jule Krüger, Patrick Ball, Megan Price,
and Amelia Hoover Green (2013). “It Doesn’t Add Up: Methodological and Policy Implica-
tions of Conflicting Casualty Data.” In Counting Civilian Casualties: An Introduction to Recording
and Estimating Nonmilitary Deaths in Conflict, ed. by Taylor B. Seybolt, Jay D. Aronson, and
Baruch Fischhoff. Oxford UP.
5
Christian Davenport and Patrick Ball. “Views to a Kill: Exploring the Implications of Source
Selection in the Case of Guatemalan State Terror, 1977–1996.” Journal of Conflict Resolution
46(3): 427–450. 2002.
6
Megan Price, Jeff Klingner, Anas Qtiesh, and Patrick Ball (2013). “Full Updated Statistical
Analysis of Documentation of Killings in the Syrian Arab Republic.” Human Rights Data
Analysis Group, commissioned by the United Nations Office of the High Commissioner
for Human Rights (OHCHR). Megan Price, Jeff Klingner, and Patrick Ball (2013). “Prelimi-
nary Statistical Analysis of Documentation of Killings in the Syrian Arab Republic.” The
Benetech Human Rights Program, commissioned by the United Nations Office of the High
Commissioner for Human Rights (OHCHR).
7
http://www.csr-sy.com
8
http://www.syrianhr.org
9
http://syrianshuhada.com
10
http://www.vdc-sy.info
11
See reports in the LA Times, BBC, and the Independent, among others.
12
Price. et al. 2013.
13
See https://hrdag.org/mse-the-basics/ for the first in a series of blog posts describing
Multiple Systems Estimation (MSE) or Kristian Lum , Megan Emily Price and David Banks
(2013). Applications of Multiple Systems Estimation in Human Rights Research. The Ameri-
can Statistician, 67:4, 191–200. DOI: 10.1080/00031305.2013.821093
14
http://www.iraqbodycount.org
15
Carpenter D, Fuller T, Roberts L. “WikiLeaks and Iraq Body Count: the sum of parts may
not add up to the whole—a comparison of two tallies of Iraqi civilian deaths.” Prehosp Disaster
Med. 2013;28(3):1–7. doi:10.1017/S1049023X13000113
16
Ibid.
17
Ibid.
18
We downloaded the ibc-incidents file on 14 Feb 2014, and processed it using the pandas
package in python.
19
The top 100 sources include, for example, AFP, AL-SHAR, AP, CNN, DPA, KUNA, LAT,
MCCLA, NINA, NYT, REU, VOI, WP, XIN, and US DOD VIA WIKILEAKS.
20
These assumptions can be formalized and tested within the framework of ‘species richness,’
which is a branch of ecology that estimates the number of different types of species within
a geographic area and/or time period of interest using models for data organized in a very
similar way to the IBC’s event records. See Wang, Ji-Ping. “Estimating species richness by a
Poisson-compound gamma model.” Biometrika 97.3 (2010): 727–740.

21
A research question to address this might be: Do media-reported killings in a globally-
interesting conflict like Iraq or Syria decline during periods when other stories attract
interest? Do reported killings decline during the Olympics?
22
Krüger et al. (2013)
23
Bias issues can sometimes be resolved with appropriate statistical models, that is, with
better scientific reasoning about the specific kind of data involved. However, we underline
that bias is not solvable with better technology. Indeed, some of the most severely biased
datasets we have studied are those collected by semi- or fully-automated, highly technologi-
cal methods. Technology tends to increase analytic confusion because it tends to amplify
selection bias.
24
For a description of multiple systems estimation, see Lum et al. 2013. For methods on
missing data in survey research which might be applicable to the adjustment of raw, non-
random data if population-level information is available, see Brick, J. Michael, and Graham
Kalton. “Handling missing data in survey research.” Statistical methods in medical research 5.3
(1996): 215–238. For an overview of species richness models which might be used to estimate
total populations from data organized like the IBC, see op. cit Wang. For an analysis of
sampling issues in “elusive” populations, see Johnston, Lisa G., and Keith Sabin. “Sampling
hard-to-reach populations with respondent driven sampling.” Methodological Innovations
Online 5.2 (2010): 38–48.
25
https://hrdag.org/why-raw-data-doesnt-support-analysis-of-violence/
26
Lazer, David and Kennedy, Ryan and King, Gary and Vespignani, Alessandro, Google Flu
Trends Still Appears Sick: An Evaluation of the 2013–2014 Flu Season (March 13, 2014).
Available at SSRN: http://ssrn.com/abstract=2408560

21SAIS Review vol. XXXIV no. 1 (Winter–Spring 2014)
Using Big Data and Quantitative
Methods to Estimate and Fight
Modern Day Slavery
Monti Narayan Datta
Given the hidden, criminal nature of contemporary slavery, empirically estimating the
proportion of the population enslaved at the national and global level is a challenge. At the
same time, little is understood about what happens to the lives of the survivors of slavery
once they are free. I discuss some data collection methods from two nongovernmental
organizations (NGOs) I have worked with that shed light on these issues. The first NGO,
the Walk Free Foundation, estimates that there are about 30 million enslaved in the world
today. The second NGO, Free the Slaves, employs a longitudinal analysis to chronicle the
lives of survivors. The acquisition and dissemination of such information is crucial because
policymakers and donors sometimes require hard data before committing time, political
will, and resources to the cause.
Unpacking the Problem of Contemporary Slavery
As Kevin Bales of the Wilberforce Institute for the Study of Slavery and
Emancipation explains, “Slavery is the possession and control of a
person in such a way as to significantly deprive that person of his or her
individual liberty, with the intent of exploiting that person through their
use, management, profit, transfer or disposal. Usually this exercise will be
achieved through means
such as violence or threats
of violence, deception and/
or coercion.”1
Thus, at its core,
slavery is a dynamic be-
tween two individuals, the
enslaved and the slave-
holder, in which the slave-
holder has a monopoly of
control and violence upon
Monti Narayan Datta is an assistant professor of political science at the University of
Richmond. His current book project, forthcoming with Cambridge University Press,
focuses on the consequences of anti-Americanism. He is working on several projects
on human trafficking and modern day slavery with Free the Slaves and Chab Dai and
the Walk Free Foundation. Along with Kevin Bales and Fiona David, he is a co-author
of the Global Slavery Index: http://www.globalslaveryindex.org.
. . . at its core, slavery is a dynamic
between two individuals, the
enslaved and the slaveholder,
in which the slaveholder has a
monopoly of control and violence
upon the enslaved.

the enslaved. The slaveholder can coerce the enslaved to perform a number
of abominable acts. This can include: sexual servitude on the streets of New
York City;2
adult labor in the coltan mines of the Congo;3
child slavery in the
shrimp farms of Bangladesh; or forced domestic servitude in the suburbs of
Los Angeles.4
Compounding the matter is that enslaved persons can spend
years—sometimes decades—under such conditions.5
This can sometimes lead
to slavery lasting across several generations. Short of homicide, slavery is
one of the most inhumane crimes one person can commit against another.
In recent years, a number of governments and international govern-
mental organizations have addressed modern day slavery at home and
abroad. In the United States, Congress passed the Victims of Trafficking
and Violence Protection Act (TVPA) in 2000. The TVPA established the
President’s Interagency Task Force to Monitor and Combat Trafficking—a
cabinet-level group whose mission is to coordinate efforts to combat traf-
ficking in persons—led by the U.S. State Department. Since then, the State
Department has produced its annual Trafficking in Persons (TIP) Report,
which has become “the U.S. Government’s principal diplomatic tool to en-
gage foreign governments on human trafficking.”6
Although not without
controversy, the TIP Report has educated many on the sources and impact
of modern day slavery.7
On the global stage, between 2000 and 2001 the United Nations Gen-
eral Assembly adopted three protocols to its Convention against Transna-
tional Organized Crime: (1) the Protocol to Prevent, Suppress and Punish
Trafficking in Persons, especially Women and Children; (2) the Protocol
against the Smuggling of Migrants by Land, Sea, and Air; and (3) the Pro-
tocol against the Illicit Manufacturing and Trafficking in Firearms. With
117 signatory countries, these protocols, known as the Palermo Protocols,
advanced the global discussion not only on what constitutes contemporary
slavery, but also on what the international community can do to mitigate
its spread.
Although some may argue that international agreements like the
Palermo Protocols and documents like the TIP Report matter only margin-
ally,8
others counter they catalyze change.9
Building upon a crest of public
awareness on human trafficking, U.S. President Barack Obama proclaimed
in 2012 at the Clinton Global Initiative, “We are turning the tables on the
traffickers. Just as they are now using technology and the Internet to exploit
their victims, we are going to harness technology to stop them.”10
Although he did not mention it explicitly, President Obama was re-
ferring to the idea of using big data to mitigate contemporary slavery. As
Pulitzer-prize winning journalist Steve Lohr explains, big data is “shorthand
for advancing trends in technology that open the door to a new approach
to understanding the world and making decisions.”11
This typically involves
using software to find trends and patterns in large amounts of aggregated
data from the Internet, sometimes from publicly available data, and other
times from clandestinely obtained data.
Along the lines of utilizing publicly available data, the tech-giant
Google announced in April 2013 a big data partnership with the Polaris

23Big Data and Quantitative Methods to Estimate Modern Day Slavery
Project, an antislavery NGO in Washington, D.C. The partnership, called the
Global Human Trafficking Hotline Network, aims to use data mining soft-
ware to identify human trafficking trends from the hotline that can even-
tually inform “eradication, prevention, and victim protection strategies.”12
Although using big data to fight trafficking is new, the idea has been
demonstrated by scholars like Mark Latonero of the Annenberg Center on
Communication Leadership & Policy at the University of Southern Califor-
nia. Latonero’s team partnered with local law enforcement agencies in Los
Angeles, explored trends in human trafficking on websites like Backpage.
com, and applied this information to target specific traffickers. This was
done by mining data from advertisements for the sexual services of domestic
minors on the adult section of Backpage in the Greater Los Angeles area, and
identifying the phone numbers from those ads that appeared in the greatest
frequencies. With this information, Latonero’s team was able to provide law
enforcement with data linking certain phone numbers to criminal networks.
The U.S. government is also using big data to mine private information
networks, not on the World Wide Web, but on what is called the Deep Web—
that part of the Internet that is not searchable on databases like Google.
The Defense Advanced Research Projects Agency (DARPA), a branch of the
U.S. military, recently launched a program called Memex to hunt criminal
networks on the Deep Web. The first domain DARPA intends to undercover
with this new technology is human trafficking.13
These developments in big data dovetail with a broader discussion
within academia about how social science researchers can apply quantitative
methods to estimate trends in contemporary slavery. Although rigorous,
many studies of modern day
slavery only exist in the pol-
icy and academic communi-
ties, and very few published
works actually employ
quantitative methods. In
a comprehensive review of
the research-based literature
on contemporary slavery,
Elżbieta M. Goździak and
Micah N. Bump of George-
town University found that,
of 218 research-based journal articles, only seven (about 3 percent) were
based on quantitative methods. Without hard data, it can be challenging
for scholars to make generalizable inferences to inform policy.
In this paper, using big data as a backdrop, I discuss some novel quan-
titative methods employed by two NGOs I have worked with that shed light
on contemporary slavery. The first NGO, the Walk Free Foundation, esti-
mates that there are about 30 million enslaved in the world today. The sec-
ond NGO, Free the Slaves, working with its local Indian partner, MSEMVS,
assesses the lives of survivors and how they are reintegrating into society.
The acquisition and dissemination of such information is crucial because
These developments in big data
dovetail with a broader discussion
within academia about how social
science researchers can apply
quantitative methods to estimate
trends in contemporary slavery.

policymakers and donors sometimes require hard data before committing
time, political will, and resources to the cause.
The Walk Free Foundation
Australian philanthropists Andrew and Nicola Forrest established the Walk
Free Foundation (Walk Free)14
three years ago to eradicate contemporary
slavery. After meeting with Microsoft co-founder Bill Gates, Andrew For-
rest was inspired to explore the underpinnings of contemporary slavery
using quantitative methods. As Forrest recounts, “Global modern slavery
is hard to measure, and Bill’s a measure kind of guy,” adding, “in manage-
ment speak, if you can’t measure it, it doesn’t exist.”15
For Forrest, it was
important to inform people in the business and policy worlds of the extent
to which slavery exists, country-by-country, to prompt action. Although
some quantitative assessments of contemporary slavery existed, very little
was publicly available. Forrest sought to collect more precise data to dis-
seminate freely and thus launched a Global Slavery Index (GSI), on which
I have been working since 2012.16
The 2013 GSI ranks 162 of the world’s nations in terms of their level
of contemporary slavery. Methodologically, these rankings are based on
several factors; the most novel is an estimation of the proportion of the
population enslaved in each country. For this measure, the GSI team (led
by Kevin Bales and Fiona David) has drawn upon secondary source data
analysis that Bales pioneered for his book, Disposable People, and later dis-
seminated in Scientific American.17
These secondary sources consisted of a
review of the public record, including materials from published reports from
governments, the investigations of NGOs and international organizations,
and journalistic reports. The GSI team has also drawn upon data from rep-
resentative random sample surveys to extrapolate the prevalence of slavery
for selected comparable countries. Figure 1 illustrates the 2013 GSI data for
the proportion of the population estimated to be enslaved.
In Figure 1 the countries with darker shades indicate a corresponding
higher proportion of enslavement. Some of the countries with the highest
proportions are Haiti (about 2.1 percent of the population enslaved), Mau-
ritania (about 4.0 percent of the population enslaved), Pakistan (about 1.2
percent of the population enslaved), and
India (about 1.1 percent of the popula-
tion enslaved).
Table 1 lists the 2013 GSI data in
terms of the total estimated number of
the enslaved, country-by-country. This is
a novel contribution compared to other
estimates of contemporary slavery. Such information can be useful to busi-
ness people, policymakers, and students who want a more informed under-
standing of where slavery occurs and with what frequency.
Overall, the 2013 GSI estimates about 29.8 million are enslaved among
the 162 countries under study. The country with the least number of esti-
. . . 29.8 million are
enslaved among the 162
countries under study.

mated enslaved in 2013 was Iceland (twenty-two enslaved), and the country
with the greatest number was India (13.9 million enslaved). The standard
deviation (or spread) was extremely large: about 1.2 million enslaved.
One important question is if the GSI has made a difference in the
real world. One way to shed light on this is to explore some of the statis-
tics achieved since the GSI’s launch in October 2013. To date, the GSI has
received over half a million website visits. There have been over thirteen
thousand downloads of the full report, available in English, Arabic, French,
and Spanish. Moreover, there have been over fifteen hundred media reports
about the GSI in over thirty-five countries, including The Economist,18
Die
Standaard,19
La Vanguardia,20
CNN,21
National Public Radio,22
and Time.23
Some of the media reports about the GSI illustrate how it can gener-
ate discussion on an underreported issue. In India, for instance, where the
GSI estimates the greatest number of the enslaved to be, media response
has been strong. The Times of India reported, “Sixty-six years after indepen-
dence, India has the dubious distinction of being home to half the number
of modern-day slaves in the world.”24
Perhaps due to such sentiments, the
Hindustan Times discussed the causes of slavery in India and observed, “Some
of the reasons for high numbers caught in slavery in India are the difficulty
in accessing protections and government entitlements, such as the food
rations card, corruption or non-performance of safety nets (such as the
National Employment Guarantee, primary health care and pensions) and
practices of land grabbing and asset domination by high-caste groups.”25
There is also some evidence that the GSI has begun to influence gov-
ernment policy. In January of this year, building upon the momentum of
the GSI, Andrew Forrest signed a memorandum of understanding (MOU)
Figure 1. Global Slavery Index (GSI)—Proportion of the Population Estimated Enslaved in 2013

Table 1. Global Slavery Index—Estimated Enslaved in 2013
Country Estimated Enslaved Country Estimated Enslaved
Afghanistan 86,089 Lebanon 4,028
Albania 11,372 Lesotho 14,560
Algeria 70,860 Liberia 29,504
Angola 16,767 Libya 17,683
Argentina 35,368 Lithuania 2,909
Armenia 10,678 Luxembourg 69
Australia 3,167 Macedonia 6,226
Austria 1,100 Madagascar 19,184
Azerbaijan 33,439 Malawi 110,391
Bahrain 2,679 Malaysia 25,260
Bangladesh 343,192 Mali 102,240
Barbados 46 Mauritania 151,353
Belarus 11,497 Mauritius 535
Belgium 1,448 Mexico 103,010
Benin 80,371 Moldova 33,325
Bolivia 29,886 Mongolia 4,729
Bosnia and
Herzegovina 13,789 Montenegro 2,234
Botswana 14,298 Morocco 50,593
Brazil 209,622 Mozambique 173,493
Brunei 417 Myanmar 384,037
Bulgaria 27,739 Namibia 15,729
Burkina Faso 114,745 Nepal 258,806
Burundi 71,146 Netherlands 2,180
Cambodia 106,507 New Zealand 495
Cameroon 153,258 Nicaragua 5,798
Canada 5,863 Niger 121,249
Cape Verde 3,688 Nigeria 701,032
Central African
Republic 32,174 Norway 652
Chad 86,329 Oman 5,739
Chile 37,846 Pakistan 2,127,132
China 2,949,243 Panama 548
Colombia 129,923 Papua New Guinea 6,131
Costa Rica 679 Paraguay 19,602
Côte d’Ivoire 156,827 Peru 82,272
Croatia 15,346 Philippines 149,973
Cuba 2,116 Poland 138,619
Czech Republic 37,817 Portugal 1,368
Democratic Republic
of the Congo 462,327 Qatar 4,168
Denmark 727 Republic of the Congo 30,889
Djibouti 2,929 Romania 24,141
Dominican Republic 23,183 Russia 516,217
Ecuador 44,072 Rwanda 80,284
Egypt 69,372 Saudi Arabia 57,504
El Salvador 10,490 Senegal 102,481

Equatorial Guinea 5,453 Serbia 25,981
Eritrea 44,452 Sierra Leone 44,644
Estonia 1,496 Singapore 1,105
Ethiopia 651,110 Slovakia 19,458
Finland 704 Slovenia 7,402
France 8,541 Somalia 73,156
Gabon 13,707 South Africa 44,545
Gambia 14,046 South Korea 10,451
Georgia 16,227 Spain 6,008
Germany 10,646 Sri Lanka 19,267
Ghana 181,038 Sudan 264,518
Greece 1,466 Suriname 1,522
Guatemala 13,194 Swaziland 1,302
Guinea 82,198 Sweden 1,237
Guinea-Bissau 12,186 Switzerland 1,040
Guyana 2,264 Syria 19,234
Haiti 209,165 Tajikistan 23,802
Honduras 7,503 Tanzania 329,503
Hong Kong,
SAR China 1,543 Thailand 472,811
Hungary 35,763 Timor-Leste 1,020
Iceland 22 Togo 48,794
India 13,956,010 Trinidad and Tobago 486
Indonesia 210,970 Tunisia 9,271
Iran 65,312 Turkey 120,201
Iraq 28,252 Turkmenistan 14,711
Ireland 321 Uganda 254,541
Israel 8,096 Ukraine 112,895
Italy 7,919 United Arab Emirates 18,713
Jamaica 2,386 United Kingdom 4,426
Japan 80,032 United States 59,644
Jordan 12,843 Uruguay 9,978
Kazakhstan 46,668 Uzbekistan 166,667
Kenya 37,349 Venezuela 79,629
Kuwait 6,608 Vietnam 248,705
Kyrgyzstan 16,027 Yemen 41,303
Laos 50,440 Zambia 96,175
Latvia 2,040 Zimbabwe 93,749
Source: The Global Slavery Index
with the Pakistani State of Punjab. In the business world, that a government
would sign a deal with a businessman to help eradicate slavery in its own
borders is atypical. Yet Forrest was able to leverage his influence in Pakistan
to encourage a conversation that aims to provide the state of Punjab with
inexpensive coal in exchange for assurances that the government will work
toward the liberation of its own people. 26
Although it is too early to see
how Pakistan will hold up to its promise, this agreement may herald future
MOUs between NGOs like Walk Free and governments that want to mitigate
slavery, and one day even eradicate it.

The GSI may also be influencing heads of state. Former U.S. President
Jimmy Carter references the GSI several times in his new bestselling book,
A Call to Action: Women, Religion, Violence, and Power. And the GSI has been
publicly endorsed by, among others, Hillary Clinton, Gordon Brown, Julia
Gillard, and Tony Blair.27
Free the Slaves
The GSI strives to use big data to count the number of slaves in the world.
Other NGOs have begun to employ longitudinal techniques to chronicle
the lives of survivors of slavery once they are free. One such NGO is Free
the Slaves (FTS), which Kevin Bales, Peggy Callahan, and Jolene Smith co-
founded in 2000 as the sister-organization of Anti-Slavery International (the
oldest international human rights organization in the world).28
Early in its evolution, FTS reasoned that the liberation of any slave
would be beneficial not only for that individual, but also for the commu-
nity, and thus produce a “freedom dividend,” multiplied by each additional
person freed. As FTS explains, “Local communities thrive when formerly
enslaved people start their own businesses; communities begin to flourish as
people come together to organize and watch out for one another; children
go to school—and the benefits extend for generations.” 29
For the past decade, FTS has partnered with different grassroots orga-
nizations in Haiti, India, Nepal, Ghana, the Democratic Republic of Congo,
and Brazil to empower local communities of the enslaved to seek liberation.
In India, FTS has worked with a local grassroots organization called Mina
Sansadham Evam Mahila Vikas Sansthan (MSEMVS).30
Through the ef-
forts of MSEMVS, over 150 villages have eradicated slavery and trafficking
in recent years and many more are beginning to experience liberation in
the North Indian States of Uttar Pradesh and Bihar, two of India’s poorest
states, as Figure 2 highlights.
In addition to empowering people in rural Uttar Pradesh to seek libera-
tion, MSEMVS has been among the first NGOs to begin several longitudinal
studies on the effects, in addition to quantitative studies of the predictive
factors of enslavement. The studies are intended to provide insight into:
(1) whether slavery and trafficking have been eradicated; and (2) whether
the socio-economic conditions of people living in these communities have
improved. I consulted with Free the Slaves at this time, and, along with
Ginny Baumann, Jody Sarich, Austin Choi-Fitzpatrick, and Jessica Leslie,
helped put together a follow-up report for the village of Kukrouthi in Ut-
tar Pradesh.
The follow-up report was conducted among the residents of three
hamlets in Kukrouthi village.31
There were two sources of information: The
first was a set of 120 household level surveys, and the second was a set of
focus group discussions. A total of 929 people were accounted for by the
surveys. The time periods under comparison were 2009 (when the libera-
tion process began) and 2011 (when the process of self-liberation by local
residents was completed).

29BIG DATA AND QUANTITATIVE METHODS TO ESTIMATE MODERN DAY SLAVERY
Some of the key findings between the 2009 and 2011 studies are as
follows, providing credence to FTS’s supposition of there being a “freedom
dividend” after liberation.
Growth in Childhood Education
One important indicator of a freedom dividend in Kukrouthi village is the
number of children in school. The underlying premise is that in free com-
munities children receive better education, which fuels a society’s human
capital. In Kukrouthi, the team from MSEVMS found evidence of significant
growth in childhood education rates. Whereas in 2009 only 69 percent of
the school-aged children were reported to be in school, by 2011, 91 percent
were enrolled, as Figure 3 illustrates.
Figure 2. Uttar Pradesh, North India

30 SAIS Review WINTER–SPRING 2014
Better Nutrition
Another key indicator illuminating the freedom dividend is access to ad-
equate nutrition. As with childhood education, the team from MSEVMS
reported a dramatic increase in the number of families that were able to eat
three meals a day, from 31 percent in 2009 to 71 percent in 2011. This was
more than a 200 percent increase, as Table 2 details.
Table 2. Number of Daily Meals By
Household in Kukrouthi, 2009 and 2011
Number of Meals Year Percentage
Two Meals Per Day 2011 22%
2009 31%
Three Meals Per Day 2011 71%
2009 31%
No Response 2011 8%
2009 3%
Figure 3. Percent Children in School in Kukrouthi, 2009 and 2011

Improved Access to Health Care
Yet another strong indicator of a freedom dividend is access to health care,
even if of rudimentary quality. In 2011, MSEVMS reported that almost the
entire population of Kukrouthi village had access to healthcare. This was
another dramatic increase compared to 2009, when MSEVMS found that
just 52 percent of families received health care treatment. Table 3 provides
a breakdown of this comparison.
Table 3. Comparison of Access to Health Care in
Kukrouthi, 2009 and 2011
Access to Health Care Year Percent
Yes 2011 96%
2009 57%
No 2011 3%
2009 43%
Don’t Know 2011 1%
2009 .
No Response 2011 1%
2009 1%
Improvement in Childhood Vaccinations
Lastly, in 2009, just one-third of children had the proper number of recom-
mended vaccinations (i.e., three vaccinations). By 2011, this had increased
to 90 percent, as Table 4 shows.
Table 4. Comparison of Child Vaccinations in
Kukrouthi, 2009 and 2011
Immunizations Year Percentage
None 2011 .
2009 49%
One 2011 3%
2009 7%
Two 2011 7%
2009 12%
Three 2011 90%
2009 33%

A World Without Slavery
Applying quantitative methods to the study of contemporary slavery could
contribute significantly to shedding more light on the phenomenon. In
collaboration with my colleagues at the Walk Free Foundation, I have
used quantitative methods to estimate the total number of enslaved in the
world today. This, in turn,
has generated discussion
among the media and pol-
icy community on how to
mitigate modern day slav-
ery, with an eye toward its
eradication. With Free the
Slaves and MSEVMS, we
have begun to chronicle systematically how communities can benefit from
freedom. This information provides preliminary evidence to policy makers
that liberating slaves provides a wide range of socioeconomic benefits.
The modern day anti-slavery movement is young. Moving forward, we
need more scholars and policy makers who want to explore what quantita-
tive methods and big data can do for the movement. We are at a point in
the world where everyone agrees that contemporary slavery is a wrong that
must be addressed. The time is ripe for further discussion on how to make
this a reality. I hope we can get there, at least in part, through employing
quantitative methods and exploring big data.
Notes
1
Kevin Bales, The Global Slavery Index, 2013. http://www.globalslaveryindex.org/report/#view-
online
2
For example: http://www.gems-girls.org/get-involved/very-young-girls
3
Congo. https://www.freetheslaves.net/congo
4
CNN Freedom Project, http://thecnnfreedomproject.blogs.cnn.com.
5
For example: Survivors of Slavery Speak Out, http://survivorsofslavery.org
6
Trafficking In Persons Report, http://www.state.gov/j/tip/rls/tiprpt
7
For example: http://www.coha.org/the-trafficking-in-persons-report-who-is-the-united-
states-to-judge
8
For example: John J. Mearsheimer, “The False Promise of International Institutions,” In-
ternational Security, Vol. 19, No. 3 (1995) pp. 5–49.
9
For example: Anne Marie-Slaughter, A New World Order, (Princeton University Press, 2005).
10
Barack Obama, “Remarks by the President to the Clinton Global Initiative,” September 25,
2012. http://www.whitehouse.gov/the-press-office/2012/09/25/remarks-president-clinton-
global-initiative
11
Steve Lohr, “The Age of Big Data,” The New York Times, February 11, 2012. http://
www.nytimes.com/2012/02/12/sunday-review/big-datas-impact-in-the-world.
html?pagewanted=all&_r=0
12
“Polaris Project Launches Global Human Trafficking Hotline Network.” http://www.
polarisproject.org/media-center/news-and-press/press-releases/767-polaris-project-launches-
global-human-trafficking-hotline-network
13
“Darpa Reinventing Search Engines to Fight Crime,” Wired, February 11, 2014. http://
www.wired.co.uk/news/archive/2014-02/11/darpa-memex-human-trafficking
14
The Walk Free Foundation, http://www.walkfreefoundation.org
Applying quantitative methods to the
study of contemporary slavery could
contribute significantly to shedding
more light on the phenomenon.

15
Elisabeth Behrmann, “Gates Helps Australia’s Richest Man in Bid to End Slavery,” Bloom-
berg, April 14, 2013. http://www.bloomberg.com/news/2013-04-10/gates-helps-australia-s-
richest-man-in-bid-to-end-slavery.html
16
The Global Slavery Index, http://www.globalslaveryindex.org
17
Kevin Bales, “The Social Psychology of Modern Slavery,” Scientific American, April 2002.
18
“Dry Bones,” The Economist, October 19, 2013. http://www.economist.com/news/
international/21588105-hateful-practice-deep-roots-still-flourishing-dry-bones
19
“Wereldwijd bijna 29 miljoen slaven [29 million people in slavery worldwide],” De Stan-
daard, October 17, 2013. http://nos.nl/artikel/563375-wereldwijd-bijna-30-miljoen-slaven.
html
20
“Casi 30 millones de personas son esclavos modernos [Almost 30 million people
are modern slaves],” La Vanguardia, October 18, 2013. http://www.lavanguardia.
com/20131018/54391301708/casi-30-millones-de-personas-son-esclavos-modernos-barce-
lona.html
21
Tim Hume, “India, China, Pakistan, Nigeria on Slavery’s List of Shame, Says Report,” CNN,
October 17, 2013. http://www.cnn.com/2013/10/17/world/global-slavery-index
22
Audie Cornish, “Report Estimates 30 Million People in Slavery Worldwide,” National Public
Radio, October 17, 2013, http://www.npr.org/templates/story/story.php?storyId=236407720
23
Nilanjana Bhowmick, “Report: Almost 14 Million Indians Live Like Slaves,” Time, October
17, 2013. http://world.time.com/2013/10/17/report-almost-14-million-indians-live-like-
slaves/
24
“India Has Half the World’s Modern Slaves: Study,” The Times of India, October 18, 2013.
http://timesofindia.indiatimes.com/india/India-has-half-the-worlds-modern-slaves-Study/
articleshow/24313244.cms
25
Abhijit Patnaik, “Modern Slavery Widespread in India,” Hindustan Times, October 17, 2013.
http://www.hindustantimes.com/India-news/NewDelhi/Modern-slavery-widespread-in-
India/Article1-1136431.aspx
26
Dennis Shanahan, “Andrew Forrest Strikes Cheap Coal Deal to End Pakistan Slav-
ery,” The Australian, January 23, 2014, http://www.theaustralian.com.au/business/
mining-energy/andrew-forrest-strikes-cheap-coal-deal-to-end-pakistan-slavery/story-
e6frg9df-1226808181875#
27
The Global Slavery Index. http://www.globalslaveryindex.org/endorsements
28
Anti-Slavery International. http://www.antislavery.org/english
29
FTS In India: Free a Village, Build a Movement. http://www.freetheslaves.net
30
https://www.ashanet.org/projects/project-view.php?p=907
31
Ginny Baumann, et al, “Follow Up Study of Slavery and Poverty In Kukrouthi village, St
Ravidas Nagar District, Uttar Pradesh,” June 2012, unpublished manuscript. Free the Slaves.

35A Conversation with Arch PuddingtonSAIS Review vol. XXXIV no. 1 (Winter–Spring 2014)
A Conversation with Arch
Puddington, Vice President for
Research at Freedom House
Who is the target audience of Freedom House reports?
From the beginning, we have sought to provide analysis that combines
scholarly rigor with a methodology and vocabulary that is accessible to the
general public. Obviously, there is a niche group of policymakers here and
in Europe, as well as journalists, scholars, political activists and dissidents,
who make up our core audience. But our data are also widely used by educa-
tors and students, including at the secondary level.
We have also developed a growing audience among foreign government
officials. This is in large measure due to the important role of democracy
and honest governance in the calculations of international development
agencies, financial institutions, and governments. Especially since Freedom
House findings have been formally incorporated into the foreign assistance
process of the American government, we have experienced a major increase
in communications with foreign diplomats, who want to discuss, or com-
plain about, our conclusions about their countries.
Freedom House’s 2013 Freedom on the Net report examined internet
activism and “increasingly sophisticated restrictions on internet free-
dom” by authoritarian regimes. Based on the report, what opportuni-
ties and obstacles do new technologies offer in promoting freedom?
New technologies offer a significant opportunity to advance democracy.
Throughout the world, online activists and ordinary social media users uti-
lize these tools to organize, lobby, and hold their governments accountable.
Women’s rights groups, free speech advocates, and human rights organiza-
tions have staged successful advocacy campaigns to overturn or prevent the
passage of oppressive laws. In many authoritarian states, such as China,
Saudi Arabia, and Bahrain, exposés by online and citizen journalists reveal-
ing corruption, police abuse, and pollution often force the authorities to
acknowledge the issue, and in some cases, hold the perpetrators accountable.
Unfortunately, the transformative power of digital media is not limited
Arch Puddington is vice president for research at Freedom House. He manages the
publication of Freedom in the World, an annual report assessing global political rights
and civil liberties, and is responsible for the development of new research and advocacy
programs.

to individuals fighting to promote freedom. Technological advances also
bring new tools to censor the web and intimidate citizens who are engaged
in online speech that
is deemed to threaten
the regime, insult the
dominant religion,
or sow social discord.
Authoritarian regimes
monitor the personal
communications of
their citizens for po-
litical reasons, with
the goal of identify-
ing and suppressing
government critics and human rights activists. Such monitoring can have
dire repercussions for the targeted individuals in those countries, including
imprisonment, torture, and even death.
In 2007, Freedom House published a report indicating a “profoundly
disturbing deterioration”—a greater number of countries were becom-
ing less free than were becoming more free. Could you share insights
on this finding?
According to our findings, more countries have experienced declines in
freedom than have experienced gains during each of the last eight years.
This is unprecedented in the forty-one-year history of Freedom in the World.
At the same time, this decline is not in itself a cause for alarm. Many of the
declines represent quite small
setbacks and not a pell-mell
retreat from the gains of pre-
vious decades. Many coun-
tries that embraced democ-
racy over the previous four
decades had little experience
with the institutions of free-
dom, and their adherence to
good government standards
is beginning to fray. Especially in times of relative scarcity, corruption is
emerging as a particular evil, especially as top-to-bottom graft and favorit-
ism erodes popular faith in democratic institutions.
A more serious problem that is reflected in our findings is the durabil-
ity of what we call modern authoritarian regimes. Russia’s Vladimir Putin
and the Chinese Communist Party leadership are the best examples of this
phenomenon, but there are others as well: Aliyev in Azerbaijan, the Iranian
clerics, Correa in Ecuador, the post-Chavez group in Venezuela. Modern
authoritarians preside over countries that are well-integrated into the
global economic and diplomatic systems and often possess energy riches.
Technological advances also bring new
tools to censor the web and intimidate
citizens who are engaged in online
speech that is deemed to threaten the
regime, insult the dominant religion, or
sow social discord.
. . . corruption is emerging as a
particular evil, especially as top-
to-bottom graft and favoritism
erodes popular faith in democratic
institutions.

37A Conversation with Arch Puddington
The leaders are unabashedly antidemocratic and anti-Western. They devote
their energies to the control of the political process, the press, civil society,
and the rule of law. They avoid the excesses and stupidities of communism,
especially in economic policy, but use nuanced and sophisticated methods
to control the levers of power. Modern authoritarianism has emerged over
the past fifteen years, and its practitioners have grown in power and even
international respectability over time. Modern authoritarianism today ranks
as the most worrying threat to freedom around the world.
The most recent Freedom in the World report noted that the number of
electoral democracies has risen, while the distribution of countries
in each of the “free,” “partly free,” and “not free” categories did not
change significantly in comparison to 2012. Why do you think this is
the case?
One way to think of a country with a designation as “free” is as a liberal or
consolidated democracy. In recent years, the number of free countries has
remained steady at eighty-seven to ninety, meaning that approximately 45
percent of the world’s sovereign states enjoy systems that guarantee com-
petitive elections and a broad range of civil liberties. On the other hand, the
number of electoral democracies has oscillated between 115 and 123. There
are thus some thirty countries that can be said to have met internationally
accepted standards for competitive elections but which fall short on other
indicators that measure liberal democracy—press freedom, minority rights,
gender equality, corruption, and so forth.
Freedom in the World ranks Mexico as an electoral democracy but also
places it in the “partly free” category because of the impact of uncontrolled
violence. Indonesia likewise qualifies as an electoral democracy but is ranked
as “partly free” because, among other problems, its government has been
unable to secure the rights of religious minorities. Given that the standards
for gaining a designation as an electoral democracy are less strict than for
achieving designation as a free country, it is not surprising that there is more
movement in and out of the electoral democracy category.
What do you make of recent articles from BBC and al-Jazeera (among
others) calling attention to corruption in the EU? Some have argued
that the quality of electoral democracies in the United States and in
Europe have been deteriorating. What are your thoughts on this as-
sertion? How closely do you think popular indices like those from
Freedom House mirror the reality on the ground?
I’m not overly exercised about the level of corruption in the EU. Every so-
ciety based on money transactions suffers from corruption to one degree
or another. The key here is whether corruption is pervasive, officially toler-
ated, and engaged in by the political leadership. The most damning report
on European corruption was commissioned by the EU itself, and most EU
countries have media which investigate corruption charges and an indepen-

dent judiciary which prosecutes corrupt officials. Europe should be con-
cerned when officials are intervening to prevent the press from uncovering
corrupt acts or prosecutors from bringing charges against officials accused
of graft. All too often accusations of widespread corruption in democracies
are advanced by people of bad faith from countries—Russia and Belarus, for
example—where corruption is a way of life.
As for the United States, there clearly are growing problems with its
political system. Gerrymandering has gotten worse and the new movement
for voter identification has been implemented in ways that suggest efforts
to weaken Democratic candidates. At the same time, the American system
retains a unique dynamism. It remains open to the emergence of new faces
(Barack Obama) and new forces (the Tea Party). Despite its multinational
character, the United States has managed to avoid the emergence of influ-
ential parties or movements that preach racism or xenophobia.
We place considerable effort on capturing these nuances in Freedom in
the World and other reports. Freedom in the World is not a report on governance
per se; we endeavor to reflect the level of freedom an individual experiences
on the ground, and zero in on the threats to freedom whether they come
from the state, terrorists, extremist movements, or other sources. We have
developed a methodology that looks at the broad set of institutions and val-
ues that make up human freedom while providing a flexibility that enables
us to highlight the qualitative differences between one society and another.
How does Freedom House collect the quantitative and qualitative in-
formation used in its reports? How do you extract significant insights
from this information?
We see our principal role as providing analysis, including scores and judg-
ments about democratic performance, to the policymaking community, the
media, and scholars. Our analysts make use of the vast sources of informa-
tion that are available these days, including government reports, the find-
ings of think tanks and NGOs, reports of multilateral institutions, press
accounts, interviews with officials and critics alike, and the many other
sources that have emerged in the data explosion era.
Freedom House is a source for analysis, not data. We see our role as
providing assessments on the state of freedom, identifying the principal
threats to freedom, and showcasing global and regional trends. Using data
from our country analysis, we are able to identify the global and regional
trajectory of freedom, broadly defined, as well as specific elements of free-
dom, such as freedom of expression and press, elections, corruption and
transparency, civil society, and rule of law. We can, in other words, illuminate
which institutions of democracy are most vulnerable to pressure from au-
thoritarian rulers, and which institutions have proved most durable. There
are other organizations that see their mission as providing data on elections,
corruption, assaults on journalists, economic freedom, and so forth. Free-
dom House, by contrast, works to inform the public about the gains and
setbacks in democratic government, civil liberties, and personal freedom.

39A Conversation with Arch Puddington
What do you make of the recent trend in which governments freely
release open data? What are the policy implications of this trend?
Clearly, enhanced transparency is preferable to less openness. My concern is
that some governments will be tempted to fudge or falsify data or decide to
stop publishing informa-
tion when the results are
embarrassing. For some
time now, Argentina has
been publishing inflation
figures that most experts
regard as bogus. After po-
litical attention was drawn
to spiraling crime rates, the
Venezuelan government
stopped publishing statis-
tics on violent crime. These
examples suggest that in the future, as in the past, the data world will be
divided between democracies that almost always publish honest statistics
and other countries whose data may or may not reflect reality.
For democracies, political leaderships will face a new challenge in
explaining the unwelcome news that will inevitably emerge from published
data. More data will mean a more informed citizenry, especially at the elite
level. But it will also mean more pressure on governments to communicate,
often in response to the arguments of demagogues, why unemployment
rates, inequality, traffic accidents, or test scores for children are moving in
the wrong direction.
Authoritarian regimes will have it easier. Their leaders will either quash
uncomfortable facts or distort them. Here it will be essential that interna-
tional financial institutions, transparency think tanks, and the global busi-
ness community weigh in by demanding honest accounting. It is instructive
that Argentina agreed to adjust its inflation figures after pressure from the
IMF.
The field of international affairs
has become more focused on
collecting and analyzing large
quantities of data. What would
you recommend to international
affairs students as the field be-
comes more data-driven?
I would urge students to remember that data and facts can be manipulated
and misused. Serious assessment of a society’s political well-being requires
facts, but it also demands honest interpretation. An overemphasis on data
can distort an analyst’s efforts to understand the true quality of freedom
as thoroughly as can outright bias.
. . . in the future, as in the past, the
data world will be divided between
democracies that almost always
publish honest statistics and other
countries whose data may or may
not reflect reality.
An overemphasis on data can
distort an analyst’s efforts to
understand the true quality of
freedom as thoroughly as can
outright bias.

41Corruption, Transparency, and Apathy in the Western WorldSAIS Review vol. XXXIV no. 1 (Winter–Spring 2014)
Of Note
A Deterioration of Democracy?
Corruption, Transparency, and Apathy in
the Western World
Rachel Ostrow
Arch Puddington, in his interview with the SAIS Review of International
Affairs, expresses a firm belief in the power of interpretation. “An over-
emphasis on data,” he says, “can distort an analyst’s efforts to understand
the true quality of freedom as thoroughly as can outright bias.” Freedom
House’s annual reports on freedom have made data on democracy accessible
for millions of people in the diplomatic, academic, and wider communities.
These analyses have criticized governments throughout the Middle East,
Africa, and Asia for anti-democratic and autocratic methods. However, Free-
dom House’s important work researching authoritarianism and democracy
throughout the world should start to focus once again on its birthplace—the
Western world.
Freedom House, based in the United States (and largely funded by
government agencies such as the State Department and the U.S. Agency
for International Development), could be—and has been—accused of hav-
ing Western biases. A quick look at Freedom in the World 2014 shows that the
United States and Canada, as well as the majority of European nations, are
classified as “free” right up to the Ukrainian border.1
However, several coun-
tries in Europe—as Freedom House rightly notes—have suffered democratic
backsliding. France, Switzerland, and Hungary have all passed laws or gone
through social movements seeking to limit the rights of migrants and ethnic
minorities. These occurrences, though noted in Freedom House’s analysis,
do not seriously affect the calculations within.
Hungary itself is an excellent example of where this analysis has
masked the more sinister undertones within an open democracy. Hungary’s
recent re-election of Viktor Orban—in an election widely seen as free and
fair—can be seen as a backwards turn for Hungarian democracy. As a member
of the right-wing, nationalist Fidesz party, Orban will likely have to make
concessions to the far-right, anti-Semitic, and anti-Roma Jobbik party, which
Rachel Ostrow is a second-year M.A. candidate at the Johns Hopkins University Paul H.
Nitze School of Advanced International Studies (SAIS) concentrating in Russian and
Eurasian Studies. She is Web Editor of The SAIS Review.

Sais.34.1

Recomendados

Recomendados

Más contenido relacionado

La actualidad más candente

La actualidad más candente (20)

Destacado

Destacado (20)

Similar a Sais.34.1

Similar a Sais.34.1 (20)

Más de Booz Allen Hamilton

Más de Booz Allen Hamilton (20)

Sais.34.1