1. When will public officials
listen?
A vignette experiment on the effects of
input legitimacy on public officials’
willingness to use public participation
Koen Migchelbrink
PGI Lunch Seminar – May 8th, 2018
2. Public Governance Institute
Contents
• Doctoral Research
o CITADEL
o Public Officials’ Willingness to Use citizen inputs
• Vignette study
o Input legitimacy
o Survey experiments with vignettes
o Fielding
o Main results
3. Public Governance Institute
Doctorate – Funding
Empowering Citizens to
TrAnsform European PubLic
Administrations
• Project Number: 726755
• Total budget: 3,6 million
• 11 project partners from 4 EU
member states
www.citadel-h2020.eu
4. Public Governance Institute
Doctoral research – Willingness
• Public administration and
citizen inputs
• Public officials are key
• Are public officials willing
to use citizen inputs?
o Sub-optimal policy making?
o Costs?
o Loss of control?
o Representative?
5. Public Governance Institute
Doctoral Research I - Representativeness
Descriptive representativeness
- Participatory elite
- Same handful of already known
citizens
Turnout
- Typically below 10%
- Sometimes only 1 or 2
6. Public Governance Institute
Vignette study - RQ
What is the effect of the democratic input factors of public
participation on the willingness of public officials to use public
participation in administrative decision-making?
H1: Higher input legitimacy = more willing public officials
H2: Higher input legitimacy = higher anticipated quality of policy outcomes.
H3: Higher input legitimacy = higher perceived target audience support.
7. Public Governance Institute
Vignette study - Theory
Democratic Legitimacy
approach
o Legitimacy = Compliance?
o Input legitimacy
o Output legitimacy
8. Public Governance Institute
Vignette study - Operationalization
• Dependent variables
o Willingness to use public inputs
o Perceived quality of policy outcomes
o Target audience support for the policy outcomes
• Independent Variables (manipulations)
o Turnout (more/less than expected)
o Participant representativeness (representative
group/usual suspects)
9. Public Governance Institute
Vignette study - Method
• Survey experiment
o Vignettes (N = 10)
o Full-factorial (2 𝑘)
o Within-subjects design
o Online (Qualtrics)
Bur/Dem
Attitudes
Vignettes
Parti/Cit
Attitudes Background
Turn. Repr.
High Vig1 Vig2
Low Vig3 Vig4 Control
Vig 5
11. Public Governance Institute
Vignette study - Sampling
• Population
o Grade A & B
• Sample (RR = 41,8%)
o Total population sampling
o Data cleaning
• Insufficient progress
• Manipulation checks
• Speeders
o Attrition?
12. Public Governance Institute
Vignette study – Randomization
• Replication of overall
sample characteristics in
vignette samples
• Parameters:
o Chi2 Test of Independence
• Gender
• Administrative level
• Policy domains
o Wilcoxon signed rank test
• Age
13. Public Governance Institute
Vignette study – Method of analysis
o OLS
o cluster-robust standard
errors
o Dummy-coded
o control-centered
y1 = β0vig1 + β1vig2 +
β2vig3 + β4vig5
y2 = β0vig6 + β1vig7 +
β2vig8 + β3vig9 + β4vig10
CITADEL is focused on transforming the public sector to make more efficient, inclusive and citizen-centric public services that identify/capture new or unsatisfied needs more quickly and satisfy them more effectively and in an inclusive way, providing also guidelines and features to support new processes. To achieve its objectives, the CITADEL ecosystem will combine and promote a set of technologies (e.g., semantics, mobile, analytics, sentiment analysis, open linked data) to both empower PA to improve its offering and the engagement of citizens and other subjects (i.e., the private sector), as well as foster cooperation among the PA and the users of its services.
Focus on the gap, almost no research into what public officials think about public participation in what is there domain: public officials.
Input legitimacy
participatory quality/responsiveness of the decision making process
Output legitimacy:
Problem-solving capacity of the policy outputs
Democratic Legitimacy approach (Scharpf, 1970, 1997, 1999, 2003)
Legitimacy=compliance
Our population exists of public officials with the administrative and legal competences to decide on the design and implementation of policies. We select these participants based on their administrative grade (A and B), which in turn are based on educational attainment (academic master of (vocational)bachelor). The sample frame (provided by City of Antwerp staff, included 2128 individuals).
We opted for a total population sampling strategy and purposefully included the entire population in our sampling frame. We choose this strategy because the population was relatively small an readily accessible to us. The downsides are of statistical-philosophical nature which I will not discuss here.
Data cleaning: we cleaned our data based on three iterations. First we excluded all respondents without sufficient progress. (150 dropped out after the first battery, 60 after the second battery, 65 after the response interruption set, 40 after the last two random vignettes.). Second, we removed the respondents who answered more than one manipulation check wrong (Qs: age, political preference, educational attainment). Finally, we removed speeders: responses with an impossible fast response time (mean< 25 sec),
Vignettes were conducted between feb1 and feb 21st.
We assessed attrition by conducting balance tests (dropouts vs. remainers) and found no significant results. This does not provide security but reduces insecurity,
We tested the success of randomization by assessing how well our randomization procedure succeeded in replicating the general sample characteristics into the specific vignette samples. If the relative differences on the selected parameters are insignificant than the samples are balanced on those parameters.
We used chi2 test of independence to test for significant differences between the ratio’s of gender, administrative level, and policy domains (categorical data, unequal samples)
And we used a Wilcoxon signed rank test (interval-ratio level, non-normal distributed) to assess the differences in age distributions.
Because of the successful randomization, the estimation of the treatment effects is straightforward. We applied a simple additive ols model with cluster robust standard errors. The vignettes are dummy coded and centered around the control vignette. This way, the control is the intercept and the difference from the intercept of each coefficient is the acerage treatment effect for each vignette.