This document discusses the development of a new survey tool for official statistics in Austria. It aims to enable multipurpose surveys using modular questionnaire design and different mixed-mode data collection designs, including a new web-based "CAWI" mode. Key challenges include developing reasonable module grouping and links while maintaining overall survey context, and ensuring comparability between modes while introducing new collection methods like CAWI. The new system aims to launch in the 2014/15 Household Budget Survey and be in place for the 2016 Labor Force Survey.

1. www.statistik.at We provide information
Development and Implementation
of a Mixed-Mode Multipurpose
Survey-Tool for Official Statistics
Marc Plate
Social Statistics
Rome
15 May 2014

2. www.statistik.at slide 2 | 28 May 2014
Development of a new Survey-Tool
Status Quo:
CATI and CAPI using BLAISE,
Mixed-Mode: Mainly alternating modes for Panels
160 CAPI Interviewer
60 CATI Interviewer
Up to 200.000 Interviews, in up to 20 Surveys per year
New IT-System to reach these goals:
• Enable Multipurpose Surveys
• Enable different kinds of Mixed-Mode Designs
• Enable new mode “CAWI”
Planned first use: Household Budget Survey 2014/15 then LFS 2016

3. www.statistik.at slide 3 | 28 May 2014
Enable Multipurpose Design
Vision: Modularize the statistical production. Questionnaires are composed by combining
modules
Official Statistics have the need to design surveys more flexibel
Q1
Q2
Q4
Q5
Q6
Q7
Module 1
Module 2
Module 3
eQuestionnaire
…
Module-Repository
CAWI
CATI
CAPI
PAPI

4. www.statistik.at slide 4 | 28 May 2014
The Module Repository - an Example
Module: Expenditures for
Education and Training
Submodule : Housing
Expenditures connected to
formal education
Submodule : Expenditures for
other (non formal or
occupational) education
Submodule: Expenditures for
formal education
Submodule : Expenditures for
occupational training
Module: Formal Education
Submodule: Highest level of
education attained
Module: Occupational Education
Submodule: Type of
occupational education
Submodule: Place of education
Module-Owner
Module-Owner

5. www.statistik.at slide 5 | 28 May 2014
Enable Mixed-Mode Design
Vision: Make use of “classic” and “new” ways of data collection (CATI, PAPI, CAPI, CAWI) and
supply the possibility of combining modes in a concurrent, sequential or longitudinal
mixed-mode design
• We need the best balance between standardized ,automated and respondent oriented
case-management
• We need a combination of modes to achieve highest data quality
Centralized System for
planning, conducting
and monitoring field-
phase

6. www.statistik.at slide 6 | 28 May 2014
Monitoring the Field Phase – an example
Activity Performance Indicator Evaluation Criteria Corrective action
Timing − median time per
page/item
− median time for
completion
per respondent
− above a certain
level
− above or below
a certain level
− check wording, layout
… of the item
− consider removing the
entry
Undesired
response
patterns
− rate of satisfycing patterns
per screen/grid
− rate of satisfycing patterns
per respondent
− above a certain
level
− above a certain
level
− check wording, layout
… of screen/grid
− consider removing
entry
Error/
prompt
messaging
− rate of error/prompt
messages per item
− rate of error/prompt
messages per respondent
− above a certain
level
− above a certain
level
− check wording, layout
… of item
− inspect answers of
respondent
Response
corrections
− rate of response
corrections
per item
− above a certain
level
− check wording, layout
… of the item
Recruiting − structure of referals - some referals
with frequency
smaller than
expected
− check for errors,
consider alternative
recruiting approach
Lozar-Manfreda & Belek 2013

7. www.statistik.at slide 7 | 28 May 2014
Websurvey – The Vision
More and more Respondents raise the wish to answer a survey online
Vision: Offer a web-questionnaire so well designed, that it works for the general
population without the need of any personal contact.
Websurvey
On- and Offline
Functionality
Highest data security
Webportal
List of Questiontypes
Innovative Plausibility Checks
Innovative Explanatory notes
Paradata
Integration of Lookuplists

8. www.statistik.at slide 8 | 28 May 2014

9. www.statistik.at slide 9 | 28 May 2014
Multipurpose Design / Modularisation
The Challenges
Substantial Challenges
• Reasonable grouping of questions
• How to make links between modules
• …
Organisational Challenges
• Reorganisation of responsibilities into modul-responsible, topic-responsible,
questionnaire-responsible Persons.
• Strong communications between departments
• Module-responsible Person has to design „perfect“ questions
• Project Team of one survey is not reasponsible for all questions anymore
How can modularity be accomplished without loosing sight of the overall context?
What methodological implication arise, when questionnaires are designed part by part?

10. www.statistik.at slide 10 | 28 May 2014
Mixed-Mode Design / CAWI
The Challenges
Known methodological issues of Mixed-Mode desing, particularly
• Compareability between modes
• Break in time series
• How to get R to use web when we „only“ know adress and name
• CAWI: Does loosing the interviewer mean loosing quality assurance? (Losing
control over who is answering, Losing means of collecting feedback on
questions)
• „New“ Issue: Paradata
Need for constant Evaluation of questionnaire for all modes
.

11. www.statistik.at slide 11 | 28 May 2014
Please address queries to:
Marc Plate
Contact information:
Guglgasse 13, 1110 Vienna
phone: +43 (1) 71128-7972
Marc.Plate@statistik.gv.at
Thank you very much
for your attention