1. Civic Education in Developing Nations:
Does it Work, and How?
Gary Langer, Julie Phelan and Gregory Holyk
Langer Research Associates
info@langerresearch.com
World Association for Public Opinion Research
May 14, 2013
2. STEP overview
Support to the Electoral Process
USAID-funded program of civic education sessions to
encourage political/civic engagement in Afghanistan,
directed by Counterpart International
Key subject areas: principles of democracy, civic rights,
rule of law, structure of government, rights of women
and the disabled
More than 17,000 sessions in all 34 provinces, 2009-2011
More than 3.3 million participants
Support via large-scale media campaign, chiefly original
radio programming on civil society issues
3. Our task
Assignment: Assess effectiveness of revised program
In effect in 12 provinces since July 2010
61,900 sessions with 1.56 million participants
4. Research questions
Has participation in STEP increased Afghan citizens’
awareness or knowledge of politics and government?
Has it impacted their engagement in civic affairs?
Has it affected their evaluation of the role of
democracy in Afghan society, trust in government or
understanding of civic values?
Has it impacted attitudes regarding the role and rights
of women and the disabled in Afghan society?
Were there any community-level impacts?
What predicts STEP programming success?
5. First stop: Literature review
Extensive review of existing literature (50+ papers) on
the effectiveness of civic education programs
Conclusions informed sample and questionnaire
design and approaches to data analysis
6. Lit. review findings
Identified critical elements of citizens’ relationship
with government and civil society:
Engagement with political and civic institutions
Values that reflect norms of civil society
Political efficacy, meaning a sense that one’s personal
involvement with politics is worthwhile and can
produce desired effects
Knowledge or familiarity with government structures
and processes
Support for democratic ideals and recognition of the
rights of others
7. Lit. review conclusions
Civic education has a measurable effect on key
outcomes when participants attend multiple sessions
using participatory methods led by high-quality
teachers
Ingrained values, especially tolerance, are particularly
difficult to influence
Cultural and contextual factors (including economic
conditions, living conditions and security) are
important
8. Design challenges and resources
Challenges:
No built in pre-/post-test or control/treatment protocols
No database of participants for recontact purposes
Available resources:
List of STEP communities, including number of sessions
and number of participants
List of community-level program organizers
9. Design solutions
Select STEP communities PPS to participation rates
Randomly sample 10 adults per community for community
sample (N = 687)
Contact STEP organizer in each community, obtain recalled list
of STEP participants, randomly select and interview for
participant sample (N = 700)
Randomly select a non-STEP community in each district in
which a STEP community was sampled; randomly sample 10
adults in each for non-STEP community sample (N = 681)
Obtain convenience samples of 130-160 disabled individuals in
each of these three groups
10. Analytical approach
Subject results to general linear modeling
(ANCOVA) in which respondent characteristics –
e.g., age, sex, education, ethnicity, region,
economic status and living conditions such as
security – are held constant in evaluating the
effect of STEP exposure across the sample types
11. Field work
Carried out by ACSOR, Aug. 19-Sept. 9, 2011
Average 33-minute face-to-face interviews
118 female interviewers (for female respondents), 127
male interviewers (for male respondents)
RR 70 percent for the STEP community sample, 73
percent for the non-STEP community sample
15 percent back-checked; additional QC in data
processing
12. Results: Overview I
Counterpart-STEP produced measurable, positive effects on its
participants’ political and civic engagement.
Compared with the non-treatment sample, participants were
much more likely to be interested in politics and government, to
have worked to solve a local problem and to be registered to
vote.
Effects remain statistically significant when controlled for sex,
age, education, employment, ethnicity, region, living conditions
and household economic status.
Two strongest predictors of effects are local living conditions and
participants’ ratings of the quality of the STEP sessions they
attended, including teaching materials, topics and instructors.
13. Results: Overview II
Few carry-over impacts on broader community
But some. STEP community residents were more likely
than those in non-STEP communities to express trust in
gov’t leaders, say the gov’t is doing all it can to provide
services and see voting as an individual responsibility
Major impact on civic and political engagement among
female participants, e.g. in interest in politics and gov’t,
being registered to vote and feeling politically empowered
Disabled participants more likely to express interest in
politics, to want to influence how government works and to
intend to vote in future elections
14.
15.
16.
17.
18.
19.
20.
21.
22.
23.
24.
25.
26.
27.
28.
29.
30.
31. Conclusions
Counterpart-STEP produced measurable, positive outcomes
in civic orientation and political and community
involvement, in modeling with controls
Top predictors of civic engagement among STEP participants
are STEP session quality and local conditions; therefore:
Not just any civic education program will do. Use of active
teaching techniques, hand-outs and high-quality
instructors all are critical factors in predicting
effectiveness
Local conditions such as security, economic conditions
and the availability of basic services are an important
precursor to effects. Basic needs must be met before civic
programming can have an impact
32. Recommendations
Civic education programs should assess the readiness of target
communities to focus beyond basic living conditions
Programs should include the elements shown to impact program
quality (e.g., visual teaching materials, active teaching styles)
and should regularly assess class and instructor quality
Planning for assessment at the design and implementation stage
(e.g., pre/post-test) would enhance impact evaluations
The relative weakness of community-level effects suggests a
greater focus on personal interventions
A focus on comprehension rather than rote knowledge is
advised; engagement improved irrespective of increased
knowledge
33. Thank you!
Gary Langer, Julie Phelan and Gregory Holyk
Langer Research Associates
info@langerresearch.com
World Association for Public Opinion Research
May 14, 2013