This document provides an overview of chapters 9-11 from the textbook "Program Evaluation: Alternative Approaches and Practical Guidelines". It discusses key topics such as cultural competence in evaluation, mainstreaming evaluation and building evaluation capacity within organizations, comparative analysis of different evaluation approaches, and clarifying the evaluation request and responsibilities. Presentations and discussion questions are provided for each chapter to engage learners in applying the concepts.
2. • Welcome to our presentation on Chapters 9-11
discussing Considerations, Analysis of Approaches, and
Guidelines for evaluations.
• Content for each section will be delivered in multiple
media based on the most effective means for
presentation.
• Please review the summary content and respond to the
discussion questions at the end of each section using the
Blackboard discussion threads.
5. • Two factors that influence evaluation practice, but transcend
particular evaluation approaches:
• the need to build cultural competence
• evaluation capacity building or mainstreaming evaluation
• We need to give consideration to the context of our evaluations
because they take place in both an organizational context as well
as a broader cultural context. These contexts influence, or
should influence, our evaluation choices.
6. “A systematic, responsive inquiry that is actively cognizant,
understanding, and appreciative of the cultural context in
which the evaluation takes place; that frames and articulates the
epistemology of the evaluative endeavor; that employs
culturally and contextually appropriate methodology; and that
uses stakeholder-generated, interpretive means to arrive at the
results and further use of the results” (SenGupta, Hopson, &
Thompson-Robinson, 2004, p. 13).
7. • The issue: As evaluators, we often conduct
evaluations within cultural contexts that differ
from our own.
8. Why is it important?
• To identify the needs of stakeholders
• To consider multiple perspectives of program success
• To increase the legitimacy of the evaluation to all stakeholders
• To increase the usefulness of the results
How can we achieve it?
• Self-examination
• Awareness of your own cultural norms
• Make inclusion a priority
• Quiet observation, respectful interactions, and reflection
9. Example: A pipeline company assigns an internal
evaluator from head office to launch an evaluation of the
company’s community investment program. This requires
cultural competence in recognizing the context of each
rural community and responding appropriately to each
stakeholder’s needs.
10. • Evaluations within an organizational context have impact beyond programs,
but on the organization itself.
• Process Use:
• “Individual changes in thinking and behavior, and program or
organizational changes in procedures and culture, that occur among
those involved in evaluation as a result of the learning that occurs
during the evaluation process” (Patton, 1997c, p. 90).
Process Use
• Learning • Decisions Organization
Evaluation • Analysis Results • Changes
11. • The recognition of this impact lead to the movement to mainstream
evaluation in organizations.
• Mainstreaming Evaluation:
• “The process of making evaluation an integral part of an organization’s
everyday operations. Instead of being put aside in the margins of work,
evaluation becomes a routine part of the organization’s work ethic if it is
mainstreamed. It is part of the culture and job responsibilities at all levels
of the organization” (Sanders, 2002, p. 254).
Evaluati Workflow Workflow
on
Evaluati
on
Non-
Mainstreamed
Mainstreamed
12. • In parallel to mainstreaming,
others have maintained a focus
on evaluation capacity building
(ECB) in organizations.
Evaluation Capacity Building (ECB)
“A context-dependent, intentional action system of guided processes and
practices for bringing about and sustaining a state of affairs in which
quality program evaluation and its appropriate uses are ordinary and
ongoing practices within and/or between one or more
organizations/programs/sites” (Stockdill, Baizerman, & Compton, 2002, p.
8).
13. • Evaluations in organizations have two forms:
• project-based, designed to provide information on the program or
policy being evaluated
• ongoing, with the evaluator working to sustain an environment
conducive to evaluation and its use within the organization
14. Strategies for building evaluation capacity and
mainstreaming:
• Stakeholder Involvement in an Evaluation
• Coaching/Mentoring
• Technology
• Written Materials
• Training
• Communities of Practice
• Meetings
15. • Example: A financial trade company recognizes a need
to evolve their programs over time to remain competitive
within their sector. Yet as opposed to maintaining an
evaluation group separate from the internal programs and
processes to be evaluated, senior leadership pushes to
mainstream the evaluation processes so that each job
role and workgroup is involved in program improvement.
16. Limitations of mainstreaming evaluation and building
evaluation capacity:
• Need to build competence and skills within the
organization
• Danger of reducing quality of formal evaluations
• Organization members are not evaluation professionals
17. 1. Cultural Competence: refer to example 1 on slide 9
You are the lead evaluator for the pipeline company’s Community
Investment program evaluation. Develop a strategy for how you will
enact cultural competence in deal with the regional contexts within
your evaluation. Use the text for examples and guidelines.
2. Mainstreaming Evaluation: refer to example 2 on slide
15
You are a senior leader at the financial trade organization. Develop a
strategy for how you will mainstream evaluation. Use the text as well
as the Preskill presentation on slide 14 (specifically 10:14 – 14:30)
for examples and guidelines.
Please post your responses in the designated Chapter 9 thread on
Blackboard.
18. A Comparative Analysis of Approaches
Presented by: Carolina Sanchez-Lopez & Reynaldo Lopez
19. • Please view the following Prezi for a summary of Chapter
10:
Chapter 10: A Comparative Analysis of Approaches Prezi
http://prezi.com/7sdnxf_99ake/a-comparative-analysis-of-
approaches/?utm_source=share&utm_campaign=shareprezi&utm_medium=email
20. 1. Should we attempt to synthesize the various approaches into one?
What would be the advantages of doing so?
2. Adams Elementary School has started a volunteer program in which
parents are encouraged to help out in the classroom. The goal of the
program is to not only provide the teacher with assistance, but also to
get parents more involved in the school and their children’s education.
The principal hopes to boost the learning of the children who are
achieving below grade level by getting their parents more involved in
their children’s education through volunteer efforts in the classroom.
Contrast using a decision-oriented, participant-oriented, and program-
oriented approach.
Please post your responses in the designated Chapter 10 thread on
Blackboard.
22. • Please view the following Prezi for a summary of Chapter
11:
Chapter 11: Clarifying the Evaluation Request &
Responsibilities
http://prezi.com/recommend/5q1wxgdcqfsh
23. • In getting down to some solid planning for an evaluation – what
concerns must one address?
• When might you refuse to do an evaluation? What types of reasons
exist not to evaluate?
• What types of research results can not be used?
• What reasons are given for the creation of evaluability assessment?
How does a program become evaluable?
Please post your responses in the designated Chapter 11 thread on
Blackboard.
24. • Fitzpatrick, J.L., Sanders, J.R., & Worthen, B.R. (2011). Program Evaluation: Alternative
Approaches and Practical Guidelines (4th Ed.). Pearson. Allyn & Bacon.
• Patton, M. Q. (1997c). Utilization-focused evaluation: The new century text. Thousand Oaks, CA:
Sage.
• Preskill, H. Effective Strategies for Facilitating Evaluation Capacity Building. [video] Retrieved from
http://www.youtube.com/watch?v=uW-D9K0Ohl0
• Sanders, J. (2002). Presidential address: On mainstreaming evaluation. American Journal of
Evaluation, 23(3), 253-259.
• Stockdill, S. H., Braizerman, M., & Compton, D. W. (2002). Toward a definition of the ECB process:
A conversation with the ECB lliterature. In D.W. Compton, M. Braizerman, & S. H. Stockdill (Eds.),
The art, craft, and science of evaluation capacity building. New Directions for Evaluation, No. 93, 7-
26. San Francisco: Jossey-Bass.
• SenGupta, S., Hopson, R., & Thompson-Robinson, M. (2004). Cultural competence in evaluation:
An overview. In M. Thompson-Robinson, R. Hopson, & S. SenGupta(Eds.), In search of cultural
competence in evaluation: Toward principles and practices. New Directions for Evaluation No. 102,
5-20. San Francisco: Jossey-Bass.
• Trevisan, M. And Huang, Y. (2003) Evaluability Assessment: A Primer, Practical Assessment,
Research & Evaluation. Retrieved from http://pareonline.net/getvn.asp?v=8&n=20
Images retrieved from iStock Royalty Free Images
