Shanghai International Program for Development Evaluation Training Asia-Pacific Finance and Development Center; 200 Panlong Road-Shanghai, October 14, 2008
Selecting and Constructing Data Collection Instruments 选择和构建数据收集工具
1. Selecting and Constructing Data Collection Instruments 选择和构建数据收集工具 Shanghai International Program for Development Evaluation Training Asia-Pacific Finance and Development Center 200 Panlong Road-Shanghai, October 14, 2008 Linda Morra-Imas
37. Advantage/Challenge: Available Data 优点 / 挑战:现有数据 Advantages 优点 Often less expensive and faster than collecting the original data yourself 与自己收集原始数据相比成本较低速度较快 Challenges 挑战 There may be coding errors or other problems. Data may not be exactly what is needed. You may have difficulty getting access. You have to verify validity and reliability of data 可能存在编码错误或其它问题。数据可能不是需要的确切数据。你在获取数据方面可能有困难。你必须核实数据的有效性和可靠性。
38.
39. Ways to Observe 观察方法 Unobtrusive 非介入式 No one knows you are observing 没有人知道你正在进行观察 Participant 参与式 You actually participate in the activity 你实际上参与到活动当中去了 Obtrusive 介入式 The people being observed know you are there to observe them 正在被观察的人知道你在观察他们
40.
41. Advantages and Challenges: Observation 观察的优点与挑战 Advantages 优点 Collects data on actual vs. self- reported behavior or perceptions. It is real-time vs. retrospective 收集有关实际行为或理解的数据与自报告数据作比较,实时数据与回顾数据的对比 Challenges 挑战 Observer bias, potentially unreliable; interpretation and coding challenges; sampling can be a problem; can be labor intensive ; low responses rate 观察者的偏见,可能不可靠;解释和编码方面的挑战;抽样可能是个问题;要花费大量的劳力;反应率低
42.
43.
44.
45. Structured vs. Semi-structured Surveys 结构化调查与半结构化调查的对比 Structured 结构化 harder to develop 设计较难 easier to complete 完成较容易 easier to analyze 分析较容易 more efficient when working with large numbers 当在处理大量调查对象时效率更高 Semi-structured 半结构化 a little easier to develop 设计较为容易 labor intensive to conduct 实施起来耗费较多劳力 harder to analyze but provide a rich source of data 分析较为困难,但提供了丰富的数据 subject to bias in interpreting 在解释时会有偏差 / 受偏见的影响 burdensome for people to complete as a self-administrated questionnaire 给完成自填式问卷的人造成负担
46. Advantages and Challenges of Surveys 调查的优点与挑战 Advantages 优点 Best when you want to know what people think, believe, or perceive, only they can tell you that 当你想要知道人们在想什么、相信什么、感知什么的时候,这是最好的方法,只有这些方法能够告诉你 Challenges 挑战 People may not accurately recall their behavior or may be reluctant to reveal their behavior if it is illegal or stigmatized. What people think they do or say they do is not the same as measuring what they actually do . 人们可能无法准确地回忆起他们的行为,或者如果行为是违法的或受污蔑的,他们可能不愿意暴露这些行为。人们 认为他们做了什么 或 说他们做了什么 与测量他们 实际上做了什么 并不一样
47.
48.
49.
50.
51.
52.
53.
54. Focus Group Process 专题组过程 Step Process 步骤 程序 1 Introduce the focus group meeting 介绍专题组会议 2 Have the participants introduce themselves 让参与者进行自我介绍 3 Present the first question, it should be easy, an ice-breaker 提出第一个问题,问题应该简单,起到 “ 破冰者 ” 的作用 4 Ask the main questions 提问主要问题 5 Ask the last (summary) questions 提问最后一个(总结性)问题 6 Ask if there are other comments or questions 问一下是否还有其它的说明和问题 7 Write-up impressions, major issues and points of discussion 整理对讨论的印象、讨论的主要议题和内容
55. Advantages and Challenges of Focus Groups 专题组的优点和挑战 Advantages 优点 Relatively quick and easy, may take less staff time than in-depth, in-person interviews; provides flexibility to make changes in process and question; ability to explore different perspectives; it can be fun 相对快速和容易,可能比深入的亲自面谈要节省工作人员的时间;在过程和问题中可以进行灵活的改变;能够探讨不同的看法;可能较为有趣 Challenges 挑战 Analysis is time consuming, participants might be different from rest of population; risk of bias in interpreting data; risk of group being influenced by moderator or dominant members 分析很耗时,参与者可能与其余人群不同;在解释说明数据时存在偏见的风险;存在小组会受到主持人或权威成员影响的风险
56.
57. Guidelines for Diaries or Journals 日记或日志方法使用指南 www.dadangsolihin.com Step Process 步骤 过程 1 Recruit people face-to-face 面对面地招募参与者 • encourage participation, appeal to altruism, assure confidentiality, provide incentive 鼓励参与,引起对利他主义的兴趣,确保机密性,提出激励措施 2 Provide a booklet to each participant 向每人提供一本小册子 • cover page with clear instructions, definitions, example 封面要写明清晰的指示说明、定义和示例 • short memory-joggers, calendar 简短的备忘录 , 日历 3 Consider the time-period for collecting data 考虑收集数据的时间周期 • if too long, may become burdensome 如果时间太长,可能成为负担 • if too short may miss the behavior or event 如果时间太短,可能错过一些行为或事件
58.
59. Advantages and Challenges of Diaries, Journals … 日记、日志的优点和挑战 Advantages 优点 Rich data can capture the details that might be quickly forgotten over time 丰富的数据能够获知细节信息,而这些细节可能很快就会被忘掉 Good for information on how people use time 对于获得有关人们是如何使用时间的信息很有用 Helps in collecting sensitive information 有助于收集敏感信息 Supplements interviews 是对面谈的一种补充 Challenges 挑战 Requires literacy 要求有读写能力 May change behavior 可能会改变行为 Data may be incomplete or inaccurate 数据可能不完整或不精确 Poor handwriting, difficult to understand phrases 字迹潦草,理解困难
60.
61.
62. Advantages and Challenges of Expert Judgment 专家评判的优点和挑战 Advantages 优点 Fast, relatively inexpensive 快速、成本相对较低 Communication of final results can give credibility 交流最终结果能够表明可信性 Challenges 挑战 Weak for impact evaluation 对于影响评价 而言显得不足 May be based mostly on perceptions 可能大多以感知为基础 Worth of data is only as good as the perceived credibility of the experts 数据的价值仅仅与专家感觉的可信性一样好
63.
64. Advantages and Challenges of Delphi Technique 德尔菲法的优点及挑战 www.dadangsolihin.com Advantages 优点 Inexpensive 便宜 Conducive to independent thinking 有助于独立思考 Allows sharing of information 允许信息共享 Challenges 挑战 Judgments of a selected group only 只有选定小组的意见 Tendency to eliminate extreme positions 有消除极端立场的趋势 Time consuming and requires skill in communication 耗时且需要沟通技巧 Requires adequate time and participant commitment 需要充足的时间以及参与者的努力
65.
66. Advantages and Challenges of Citizen Report Cards 公民报告卡的优势及挑战 www.dadangsolihin.com Advantages 优势 Mixes focus groups and questionnaire data collection 混合了专题组及问卷调查的数据收集 Increases response rates 增加了回应率 Simple communication 简单的交流 Challenges 挑战 Local conditions must be conducive 必须具备有利的当地条件 Requires a large sample 需要一个大样本 Lack of predictability in how different players respond 不同的参与者如何回应,对此缺乏预测
82. Developing an Interview 设计一个面谈 Step Procedure 步骤 程序 1 Define the purpose of the interview. Link your purpose to the evaluation objectives 明确面谈的目的。将你的目的与评价 目标联系起来 2 Decide on open-ended or close-ended questions 决定用开放式问题或封闭式问题 3 Draft interview questions and sequence the questions 草拟面谈问题并对其进行排序 4 Prepare an Introduction and Closure for the interview 准备一份面谈简介和结束语 5 Prepare to record responses 准备好记录问题的回答结果 6 Pre-test the instrument 对面谈中使用的工具进行预先测试