Ellsberg and Heise 2005, extracts from p. 107-114 (WHO-PATH)

1. PHOTO BY HAFM JANSEN
7

2. CHAPTER SEVEN
Developing a Sampling Strategy
Topics covered in this chapter:
Sampling considerations in qualitative studies
Sampling considerations in quantitative research surveys
O ne cannot overemphasize the
importance of developing an appropriate
designing a sample for qualitative or quan-
titative research. It also gives examples of
sample for the type of research design how different strategies have been used to
selected. Although qualitative and quantita- fit the specific needs and circumstances of
tive research use different approaches for research projects.
selecting the individuals or groups to be
studied, in all studies it is crucial to plan SAMPLING
the sampling strategy carefully. Particularly C O N S I D E R AT I O N S I N
in the case of population-based surveys, a Q U A L I TAT I V E S T U D I E S
poorly selected sample may harm the cred-
ibility of a study, even if the rest of the There are no hard and fast rules for sample
study is well executed. sizes in qualitative research. As Hudelson
Qualitative studies generally focus in points out, “The sample size will depend
depth on a relatively small number of on the purpose of the research, the specific
cases selected purposefully. By contrast, research questions to be addressed, what
quantitative methods typically depend on will be useful, what will have credibility,
larger samples selected randomly. These and what can be done with available time
tendencies evolve from the underlying pur- and resources.”2
pose of sampling in the two traditions of In qualitative sampling, the selection of
inquiry. In quantitative research, the goal respondents usually continues until the
of sampling is to maximize how represen- point of redundancy (saturation). This
tative the sample is so as to be able to means that when new interviews no
generalize findings from the sample to a longer yield new information and all
larger population. In qualitative inquiry, the potential sources of variation have been
goal is to select for information richness so adequately explored, sampling may stop.
as to illuminate the questions under study.1 For most qualitative studies, 10 to 30 inter-
This chapter discusses the major issues views and/or 4 to 8 focus groups will suf-
that should be taken into account when fice. Table 7.1 summarizes a number of
A Practical Guide for Researchers and Activists 105

3. CHAPTER SEVEN
TABLE 7.1 TYPES OF SAMPLING STRATEGIES FOR QUALITATIVE STUDIES
Type of Sampling Purpose Example
Intensity sampling To provide rich information from a Interviewing survivors of date rape
few select cases that manifest the to learn more about how coerced
phenomenon intensely (but are not sex affects women’s sexuality.
extreme cases).
Deviant case sampling To learn from highly unusual mani- Interviewing men who do not beat
festations of the phenomenon in their wives in a culture where wife
question. abuse is culturally accepted.
Stratified purposeful sampling To illustrate characteristics of particu- Interviewing different types of serv-
lar subgroups of interest; to facilitate ice providers (police, social work-
comparisons. ers, doctors, clergy) to compare
their attitudes toward and treatment
of abuse victims.
Snowball or chain sampling (Locate To facilitate the identification of Finding commercial sex workers to
one or two key individuals, and hard-to-find cases. interview about experiences of
then ask them to name other likely childhood sexual abuse by getting
informants.) cases referred through friendship
networks.
Maximum variation sampling To document diverse variations; can Researching variations in norms
(Purposely select a wide range of help to identify common patterns about the acceptability of wife beat-
variation on dimensions of interest.) that cut across variations. ing by conducting focus groups
among different sub groups: young
urban women, old urban women,
young rural men, old rural men,
women who have been abused,
women who have not experienced
abuse.
Convenience sampling (Select who- To save time, money, and effort. Forming focus groups based on
ever is easiest, closest, etc.) Information collected generally has who is available that day at the
very low credibility. local community center, rather than
according to clear criteria.
Criterion sampling To investigate in depth a particular Specifically interviewing only
“type” of case; identify all sources abused women who have left their
of variation. partners within the last year in order
to better understand the variety of
factors that spur women to leave.
(From Patton, 1990.3)
different approaches to qualitative sampling. their intimate relationships. They wanted
In qualitative research, the sampling to understand the beliefs and attitudes that
strategy should be selected to help eluci- existed in Nicaraguan culture that sup-
date the question at hand. For example, ported violent behavior toward women.
researchers with the Nicaraguan organiza- More importantly, they wanted to know if
tion Puntos de Encuentro embarked on a there were any “benefits” of nonviolence
project to collect information useful for that could be promoted to encourage men
designing a national media campaign that to reconsider their behavior (Box 5.6).
called on men to renounce violence in Rather than concentrating on collecting
106 Researching Violence Against Women

4. D E V E L O P I N G A S A M P L I N G S T R AT E G Y
information on the norms and attitudes of
“typical” Nicaraguan men, the researchers
decided to use “deviant case” sampling and
concentrate on interviewing men who had
already had a reputation for being nonvio-
lent and renouncing machismo.4 They were
interested in finding out from these men
what benefits, if any, they perceived from
this choice, and what life-course events,
influences, or individuals pushed them in
this direction. The goal was to investigate
what aspirations and life experiences help
create “healthy” intimate partnerships. The
findings were used to design an informa-
tion campaign aimed at recruiting more
men to a nonviolent lifestyle.
SAMPLING
C O N S I D E R AT I O N S
PHOTO BY HAFM JANSEN
I N Q U A N T I TAT I V E
RESEARCH SURVEYS
In contrast to qualitative research, which
generally uses nonprobability or “purpo-
sive” sampling, quantitative research relies violence, the study results A probability or representative
on random sampling of informants. A would be biased towards sample is a group of
probability or “representative” sample is a women who work at home. informants selected from the
group of informants selected from the pop- One way to reduce this partic- population in such a way that
ulation in such a way that the results may ular bias would be to return to the results may be generalized
be generalized to the whole population. homes at night or on week- to the whole population.
When a sample is referred to as ran- ends to increase the likelihood
dom, it means that specific techniques of reaching all women.
have been used to ensure that every indi- The way in which the sample is chosen
vidual who meets certain eligibility criteria affects its generalizability, or the extent to
has an equal probability of being included which the situation found among a particu-
in the study. Failure to adhere to these lar sample at a particular time can be
techniques can introduce error or bias applied more generally. There are many
into the sample, which may lessen the techniques for sampling, each with its own
validity of the study. For example, if a tradeoffs in terms of cost,
household survey on violence only con- effort, and potential to gener- When a sample is referred
ducted interviews during the day, then the ate statistically significant to as random, it means that
respondents most likely to be included in results. Some strategies, such specific statistical techniques
the study would be women who work at as simple random sampling, have been used to ensure that
home, and women who worked outside may not be feasible where every individual who meets
the home would be less likely to be inter- there is little information avail- certain eligibility criteria has
viewed. Since women working outside the able on the population under an equal probability of being
home may have different experiences with study. The following is a brief included in the study.
A Practical Guide for Researchers and Activists 107

5. CHAPTER SEVEN
description of the more common sampling selection of any one individual in no way
techniques used. influences the selection of any other. The
Many people underestimate the chal- word “simple” does not mean that this
lenge of obtaining a well-designed sample. method is any easier, but rather that steps
Mistakes are often made due to confusion are taken to ensure that only chance influ-
over the meaning of the term random ences the selection of respondents.
selection. A random selection does not Random selection can be achieved using a
mean that participants are simply selected lottery method, random number tables
in no particular order. In fact, the tech- (found in many statistical books), or a
niques for obtaining a truly random sample computer program such as Epi Info. To
are quite complex, and inexperienced avoid bias, it is very important to include
researchers should consult an expert in in the sampling frame only individuals who
sampling before proceeding. A well are eligible to be interviewed by criteria
thought-out and tested questionnaire used such as age, sex, or residence. By the same
on a poorly designed sample will still ren- token, if certain individuals are left off the
der meaningless results. original list due to an outdated census that
Random samples are often confused does not include individuals who have
with convenience or quota samples. A recently moved into the population area,
convenience sample is when informants then these omissions could bias the results,
are selected according to who is available, particularly if migration is the result of
in no particular order. In a quota sample crises such as war, natural disasters, or eco-
a fixed number of informants of a certain nomic collapse. In these cases, you will
type are selected. Neither strategy will need to update the sampling list.
result in a random sample appropriate for
survey research. Systematic sampling
In random sampling, each individual or
Simple random sampling household is chosen randomly. In contrast,
This sampling technique involves selection systematic sampling starts at a random
at random from a list of the population, point in the sampling frame, and every nth
known as the sampling frame. If properly person is chosen. For example, if you
conducted, it ensures that each person has want a sample of 100 women from a sam-
an equal and independent chance of pling frame of 5,000 women, then you
being included in the sample. would randomly select a number between
Independence in this case means that the one and 50 to start off the sequence, and
then select every fiftieth woman thereafter.
Both random and systematic sampling
require a full list of the population in
order to make a selection. It is also impor-
tant to know how the list itself was made,
and whether individuals are placed ran-
domly or in some kind of order. If individ-
uals from the same household or with
PHOTO BY HAFM JANSEN
certain characteristics are grouped
together, this may result in a biased sam-
ple in which individuals with these charac-
teristics are either overrepresented or
underrepresented.
108 Researching Violence Against Women

6. D E V E L O P I N G A S A M P L I N G S T R AT E G Y
Stratified sampling
Stratified sampling may be used together
with either simple random sampling or sys-
tematic sampling. This ensures that the
sample is as close as possible to the study
population with regard to certain character-
istics, such as age, sex, ethnicity, or socio-
economic status. In this case, the study
population is classified into strata, or sub-
groups, and then individuals are randomly
PHOTO BY HAFM JANSEN
selected from each stratum. Because strati-
fication involves additional effort, it only
makes sense if the characteristic being
stratified is related to the outcome under
study. For the purpose of analysis it is eas-
ier if the number of individuals selected clusters (such as villages or neighbor- A street map used for
from each stratum is proportional to their hoods). Then a random sample of these locating households in
the Japan WHO study.
actual distribution in the population. (See clusters is drawn for the survey. This is the
Box 7.1 on self-weighting samples.) For first stage of sampling. The second stage
example, in a sample stratified according to may involve either selecting all of the sam-
urban/rural residence, the proportion of pling units (respondents, households) in
rural women in the sample would be the the selected clusters, or selecting a group
same as the proportion of rural women in of sampling units from within the clusters.
the study population. Sometimes more than two stages are
A weighted stratified sample may be required. Thus, one might randomly
preferable when there are some groups choose districts within a province, and
which are proportionately small in the then randomly select villages from the
population, but which are relevant for the selected districts as the second stage.
purpose of the study, such as individuals Individual respondents would be selected
from a certain geographical region or eth- from the clusters as a third stage. At each
nic group. Ensuring that these groups are stage, simple random, systematic, or strati-
adequately represented might require an fied techniques might be used. It is advis-
inordinately large sample size using simple able to consult a statistician if you are
random sampling techniques. In this case, considering a multistage sampling scheme.
it may be appropriate to oversample, or to The advantage of multistage sampling is
select a disproportionately large number of that a sampling frame (e.g., a list of house-
respondents from this stratum. This results holds) is only needed for the selected clus-
in a weighted sample that will have to be ters (villages) rather than for the whole
taken into account in the analysis process. study population. Also, the logistics will be
easier because the sample is restricted to
Multistage and cluster sampling the selected clusters and need not cover
Multistage sampling is often used for the whole study area. An example of a
drawing samples from very large popula- multistage sampling strategy in Peru is
tions covering a large geographical area. It described in Box 7.3.
involves selecting the sample in stages, or The disadvantages of multistage sam-
taking samples from samples. The popula- pling are that the sample size needs to be
tion is first divided into naturally occurring substantially larger than if the sample was
A Practical Guide for Researchers and Activists 109

7. CHAPTER SEVEN
BOX 7.1 SELF-WEIGHTING IN CLUSTER SAMPLES I How sure do you want to be of your
conclusions? Larger sample size gener-
The way in which the sample is chosen greatly influences the usefulness of the ally increases the precision of the results,
resulting estimates. Suppose that there is a district with only two villages:
or the confidence with which one can
I Village A has 4,000 women, of which 800 (20 percent) have been abused.
say that they represent a reliable meas-
I Village B has 800 women, of which 40 (5 percent) have been abused.
ure of the phenomena under study.
The true prevalence of abuse in this district would be calculated as follows:
I What are the characteristics of the
Total cases of violence = (800 + 40) X 100 = 17.5 percent
study population? The more variability
Total number of women (4000 + 800)
there is in the population, the larger the
sample size needed.
However, if we decided to determine the prevalence of abuse in this district
based on a random sample of 100 women from each village, we would find
the following: I How common is the phenomenon
I 20 out of 100 women in Village A reporting abuse. under study? If any of the conditions
I 5 out of 100 women in Village B reporting abuse. you want to measure in your study are
Combining these two figures we would find that 25 out of the 200 women inter- very rare, for example, infant mortality
viewed were abused, which would give us a prevalence of 12.5 percent. or maternal mortality, then you will
need a very large sample size.
What has happened here?
Our sampling procedure led us to an underestimated prevalence because the num-
I What is the purpose of the research?
ber of informants selected from each village was not in proportion to the relative
size of each village. Assuming that we knew the relative sizes of the villages, we The sample size calculation will also
could perform a weighted analysis where the results from Village A would count depend on whether you simply want to
five times as much as those from Village B. However, it is usually preferable to measure the prevalence of a condition
obtain a self-weighting sample. One way to do this would be to select five times
more respondents from Village A than Village B. Another approach is to select the
in a population or whether you want to
villages with probability proportional to size. This means that if you have a list of vil- measure an expected difference
lages, a large village like Village A would be five times more likely to be selected between two groups. Programs such as
for the sample than a village the size of Village B. After the villages were selected, Epi Info contain two different formulas
you could then to select an equal number of respondents from each village. (For an
example of how a self-weighting sample was obtained in Peru, see Box 7.3.) for these two different approaches.
(From Morison, 2000.5) I What kind of statistical analysis will
you use? This underscores the need to
consider how you are going to analyze
selected by simple random sampling. Also, your data from the very beginning. The
it can be more complicated to get a self- sample size must be large enough to
weighting sample. Another difficulty with provide for desired levels of accuracy in
multistage sampling can be defining clus- estimates of prevalence, and to test for
ters if the study area is, for example, a the significance of differences between
large urban area. Sometimes these have different variables.
already been defined for previous censuses
or surveys, but otherwise they have to be I What kind of sampling strategy will
created from a map or based on some be used? Commonly used sample size
other criteria such as school or health cen- formulae and computer packages
ter catchment areas. assume you are using simple random
sampling. If you plan to use multistage
How large a sample do you need? or cluster sampling, you may need to
The ideal sample size for a survey depends increase your sample size to achieve the
on several factors: precision you require. Consider asking a
110 Researching Violence Against Women

8. D E V E L O P I N G A S A M P L I N G S T R AT E G Y
statistician for help in deciding by how BOX 7.2 POPULATION SURVEY USING RANDOM SAMPLING
much the sample size needs to be (STATCALC SAMPLE SIZE AND POWER)
increased.
Population Size 100,000 10,000 10,000 10,000
It is better to collect excellent data from Expected Frequency 30% 30% 20% 30%
fewer respondents than to collect data of Worst Acceptable
Frequency 25% 25% 15% 28%
dubious validity and reliability from many
respondents. Statistical computer packages
Confidence Sample Sample Sample Sample
or mathematical formulas can be used to Level Size Size Size Size
determine sample size for a study. Box 7.2
80% 138 136 104 794
presents a table produced by Epi Info’s
90% 227 222 170 1,244
STATCALC program for ideal sample size
95% 322 313 270 1,678
calculations. This program is available
online at http://www.cdc.gov/epiinfo. 99% 554 528 407 2,583
If your proposed analysis calls for study- 99.9% 901 834 648 3,624
ing particular subgroups of your sample, 99.99% 1,256 1,128 883 4,428
the sample size will need to be expanded
accordingly. For example, to determine the
prevalence of violence, you may need a sample size calcula- How large a sample?
sample of only 300 women. But if you tions. It should also
want to know whether the prevalence of be noted that the This is one of the most common ques-
violence varies by age, education, or socio- sample size will tions asked of statisticians. A frequent
economic group, then you will need a need to be increased but erroneous answer is “as large as
sample size sufficiently large to allow for if a multistage sam- possible” when it instead should be “as
comparisons among these groups. ple is being used. small as possible.”
The initial calculation was made based Because these calcu- It is also important to emphasize that the
on a simple random sample from a study lations can be quite amount of information that can be
population of 100,000 women, where it complex, inexperi- gained from a sample depends on its
was assumed that approximately 30 per- enced researchers absolute size, not upon the sampling
cent of women have experienced violence are urged to consult fraction, or its size as a proportion of
and that a 10 percent margin of error with someone who the population size. It is actually true
would be acceptable (5 percent above and is knowledgeable in that 99 out of one million tells you as
5 percent below). If these assumptions are survey sampling much about the 1 million as 99 out of
actually true, the table indicates that with a techniques. one thousand tells you about the one
sample size of 322 women, one would To explore the thousand.
obtain a 95 percent confidence interval for health consequences
the true prevalence of 25 percent and 35 of violence with (From Persson and Wall, 2003.6)
percent. Note, however, that if the esti- greater precision, and
mates used for sample size calculations are to compare the occurrence of violence in
very inaccurate then the required precision different sites within each country, the
may not be obtained. WHO VAW study uses a multistage sam-
The table also shows that differences in pling strategy aiming for 3,000 interviews
the size of the study population do not in two sites; 1,500 in the capital city and
greatly influence sample size, whereas 1,500 in a province. However, to end up
changes in the expected frequency and with 1,500 completed interviews, it is usu-
particularly the level of precision that is ally necessary to increase the estimated
needed can have an enormous effect on sample size by 10-20 percent to account
A Practical Guide for Researchers and Activists 111