Presentation with Kathleen Kelsy for the American Evaluation Association, October 2012

1. Using a DevelopmentalUsing a Developmental
Approach to EvaluatingApproach to Evaluating
Innovative ExtensionInnovative Extension
ProgramsPrograms
Kathleen Kelsey, Grape CoP
Oklahoma State University
Sarah Baughman, Network Literacy CoP
eXtension

2. 1. Ongoing development or
adapting an intervention to new
conditions
2. Adapting effective general
principles to a new context
3. Developing a rapid response to
a major change
4. Pre-formative development of a
potentially scalable innovation or
getting an intervention ready for
a summative evaluation
5. Major systems change and
cross-scale evaluation to
provide feedback about how the
intervention is unfolding and
how it may need to be adapted
for broader application.
(p. 21-22)

4. PlanningPlanning
Why Developmental Approach?Why Developmental Approach?
Grapes
• Collaborative approach of
community of practice
• Exploration of what is being
developed and what is
emerging
• Sustainability or what’s next
Network Literacy
• CoP leaders looking for a
more “flexible” evaluation
process that allowed for
outcomes to emerge and
change
• Ultimately looking for system
change
• Social media significant part
of programming requiring
rapid response

5. Finding an EvaluatorFinding an Evaluator
Why Us?Why Us?
Grapes Network Literacy
• Staff evaluator for larger
MFLN
• Willingness to try different
approach

6. Working with the evaluatorsWorking with the evaluators

7. Data CollectionData Collection

8. ReportingReporting
Grapes
• Informal
• Continuous
• Included formal annual
reports for grant
Network Literacy
• Combination of informal
reporting through
conversations with
leadership team
• Formal reporting to MFLN
and Department of Defense
on monthly basis

9. Cycling Lessons LearnedCycling Lessons Learned
• Critical friend role
Network Literacy
• Incorporates the informal
and formal reporting into
leadership team work
• Challenge to get
membership and leadership
of CoP to take action
• Heavy investment of time
and effort

10. Where we are now and what weWhere we are now and what we
might changemight change
• Year 3 for Grape CoP, year 2 for
Network Literacy
• NetLit still emerging based partly on
challenges reaching target DoD
audience
• Grapes ready to begin formative
evaluation