Presentation delivered to graduate class Principles of Extension. Much of the material generated in this lecture were from the extension, logic model, scholarship of engagement were taken from the University of Wisconsin-Extension, Program Development and Evaluation program. http://www.uwex.edu/ces/pdande/evaluation/evallogicmodel.html

1. Extension and Engagement for the21stCentury Land Grant Institution The Logic Model and the Scholarship of Engagement Steven E. Newman, Ph.D., A.A.F. Greenhouse Crops Extension Specialist and Professor of Floriculture

2. Traditional Extension Model Outreach and Service Client based Outreach delivery Continuing education Laboratory testing Field days Youth development Ag consultation Food and nutrition

3. The Scholarship of Engagement What is the scholarship of engagement? Is it service? Service is often seen as somehow outside the "real" work of scholars.

4. The Scholarship of Engagement What is the scholarship of engagement? Is it service? Service is often seen as somehow outside the "real" work of scholars. Faculty members can extend their intellectual curiosity into their service activities bringing together their teaching, research, and service in a synergistic way, to the benefit of each aspect of their work and the benefit of those with whom they work.

5. The Scholarship of Engagement What is the scholarship of engagement? Is it service? Is it legitimate scholarship?

6. The Scholarship of Engagement What is the scholarship of engagement? Is it service? Is it legitimate scholarship? To make faculty service more legitimate, the institution must treat outreach and service activities as scholarly activities in the same way that research always has been and teaching is increasingly being treated. When faculty and administrators finally embrace a scholarship of engagement and acknowledge the important role of service in both the internal and external functioning and health of the campus, then faculty can begin to experience integrated academic lives.

7. The Scholarship of Engagement Engagement provides a conduit of information to flow from units not normally considered part of the outreach mission of an institution. This must include: Stakeholders Consumers All faculty Students

8. Evaluation of Engagement

9. The Logic Model

10. The Logic Model Five Components: Resources or inputs Program activities Outputs Outcomes Impacts

11. A logic model is… A depiction of a program showing what the program will do and what it is to accomplish. A series of “if-then” relationships that, if implemented as intended, lead to the desired outcomes The core of program planning and evaluation University of Wisconsin-Extension, Program Development and Evaluation

12. Simplest form INPUTS OUTPUTS OUTCOMES University of Wisconsin-Extension, Program Development and Evaluation

13. Logic models can be applied to: a small program, a process (i.e. a team working together), a large, multi-component program, or even to an organization or business. University of Wisconsin-Extension, Program Development and Evaluation

14. LOGIC the principles of reasoning reasonable the relationship of elements to each other and a whole MODEL small object representing another, often larger object (represents reality, isn’t reality) preliminary pattern serving as a plan tentative description of a system or theory that accounts for all its known properties The American Heritage Dictionary, 2nd Ed University of Wisconsin-Extension, Program Development and Evaluation

15. “If you don’t know where you are going, how are you gonna’ know when you get there?” Yogi Berra Where are you going? How will you get there? What will show that you’ve arrived? University of Wisconsin-Extension, Program Development and Evaluation

16. Many people say a logic model is a road map University of Wisconsin-Extension, Program Development and Evaluation

17. Logic model may also be called… Theory of change Program action Model of change Conceptual map Outcome map Program logic University of Wisconsin-Extension, Program Development and Evaluation

18. What gets measured gets done If you don’t measure results, you can’t tell success from failure If you can’t see success, you can’t reward it If you can’t reward success, you’re probably rewarding failure If you can’t see success, you can’t learn from it If you can’t recognize failure, you can’t correct it. If you can demonstrate results, you can win public support. Reinventing Government, Osborne and Gaebler, 1992 Accountability era University of Wisconsin-Extension, Program Development and Evaluation

19. What logic model is not… A theory Reality An evaluation model or method It is a framework for describing the relationships between investments, activities, and results. It provides a common approach for integrating planning, implementation, evaluation and reporting. University of Wisconsin-Extension, Program Development and Evaluation

20. A bit of history Dates to late 1960’s Current accountability demands; logic model in widespread use Public Sector - GPRA Non-Profit Sector Private Sector International Agencies Evaluation University of Wisconsin-Extension, Program Development and Evaluation

21. Why the hype? What’s the benefit? Focus on and be accountable for what matters – OUTCOMES Provides common language Makes assumptions EXPLICIT Supports continuous improvement Promotes communications University of Wisconsin-Extension, Program Development and Evaluation

22. Logic modelling is a way of thinking…not just a pretty graphic “We build the road and the road builds us.” -Sri Lankan saying University of Wisconsin-Extension, Program Development and Evaluation

23. Everyday example H E A D A C H E Feel better Getpills Takepills Situation INPUTS OUTPUTS OUTCOMES University of Wisconsin-Extension, Program Development and Evaluation

24. Everyday example H U N G R Y Feel better Getfood Eat food University of Wisconsin-Extension, Program Development and Evaluation

25. Every day logic model – Family Vacation Family Members Drive to state park Family members learn about each other; family bonds; family has a good time Budget Set up camp Car Cook, play, talk, laugh, hike Camping Equipment OUTPUTS OUTCOMES INPUTS University of Wisconsin-Extension, Program Development and Evaluation

26. Assumptions Assumptions underlie much of what we do. It is often these underlying assumptions that hinder success or produce less-than-expected results. One benefit of logic modeling is that it helps us make our assumptions explicit. University of Wisconsin-Extension, Program Development and Evaluation

27. Assumptions The beliefs we have about the program, the participants, and how the program will work. Includes ideas about: the problem or existing situation program operations expected outcomes and benefits the participants and how they learn, behave, their motivations resources staff external environment: influences the knowledge base etc. University of Wisconsin-Extension, Program Development and Evaluation

28. Assumptions As you left the house today and came to this workshop, what were some of your assumptions about the day? Why is it important that we think about assumptions? University of Wisconsin-Extension, Program Development and Evaluation

29. A youth financial literacy program Teens establish sound financial habits Teens make better decisions about the use of money Teens gain knowledge and skills in money management A high school financial planning program – 7 unit curriculum - is developed and delivered in high schools Partners invest resources University of Wisconsin-Extension, Program Development and Evaluation

30. Business Counseling Example Improved business performance These owners gain knowledge and change practices resulting in A variety of educational activities are provided to business owners who participate Agency invests time and resources University of Wisconsin-Extension, Program Development and Evaluation

31. Parent Education Program – Logic model SITUATION: During a county needs assessment, majority of parents reported that they were having difficulty parenting and felt stressed as a result OUTCOMES INPUTS OUTPUTS Parents increase knowledge of child dev Assess parent ed programs Parents identify appropriate actions to take Reduced stress Staff Parents of 3-10 year olds attend Parents better understanding their own parenting style Improved child-parent relations Design- deliver evidence-based program of 8 sessions Money Parents use effective parenting practices Partners Parents gain skills in new ways to parent Research Facilitate support groups Parents gain confidence in their abilities University of Wisconsin-Extension, Program Development and Evaluation

32. Example: Water quality University of Wisconsin-Extension, Program Development and Evaluation

34. Group work

35. Practice

36. Q and ATrainer Funds Equipment Research base Training curriculum Grantees Improved planning Improved evaluation Accountable here University of Wisconsin-Extension, Program Development and Evaluation

37. If-then relationships Underlying a logic model is a series of ‘if-then’ relationships that express the program’s theory of change IF then IF then IF then IF then IF then University of Wisconsin-Extension, Program Development and Evaluation

38. Theory of change “A theory of change is a description of how and why a set of activities – be they part of a highly focused program or a comprehensive initiative – are expected to lead to early, intermediate, and long-term outcomes over a specified period.” (Anderson, 2000) University of Wisconsin-Extension, Program Development and Evaluation

39. Logical chain of connections showing what the program is to accomplish INPUTS OUTPUTS OUTCOMES Activities Participation Short Medium Long-term Program investments What we do Who we reach What we invest What results University of Wisconsin-Extension, Program Development and Evaluation

40. How will activities lead to desired outcomes? A series of if-then relationships Tutoring Program Example IF then IF then IF then IF then IF then They will move to next grade level on time We invest time and money We can provide tutoring 3 hrs/week for 1 school year to 50 children They will get better grades They will learn and improve their skills Students struggling academical-ly can be tutored University of Wisconsin-Extension, Program Development and Evaluation

41. Don’t forget the arrows Arrows and feedback loops show the links between inputs, outputs and outcomes Arrows depict the underlying causal connections University of Wisconsin-Extension, Program Development and Evaluation

42. Simplest form of logic model INPUTS OUTPUTS OUTCOMES University of Wisconsin-Extension, Program Development and Evaluation

43. A bit more detail INPUTS OUTPUTS OUTCOMES Activities Participation Short Medium Long-term Program investments What we invest What we do Who we reach What results SO WHAT?? What is the VALUE? University of Wisconsin-Extension, Program Development and Evaluation

44. Fully detailed logic model University of Wisconsin-Extension, Program Development and Evaluation

45. Defining the Situation: Critical first step in logic model development What problematic condition exists that demands a programmatic response? Why does it exist? For whom does it exist? Who has a stake in the problem? What can be changed? If incorrectly understood and diagnosed, everything that flows from it will be wrong. Factors affecting problems: protective factors; risk factors Review research, evidence, knowledge-base Traps: Assuming we know cause: symptoms vs. root causes. Framing a problem as a need where need is actually a program or service. “Communities need leadership training” Precludes discussion of nature of the problem: what is the problem? Whose problem? Leads one to value provision of the service as the result – is the service provided or not? University of Wisconsin-Extension, Program Development and Evaluation

46. University of Wisconsin-Extension, Program Development and Evaluation

47. Satisfaction University of Wisconsin-Extension, Program Development and Evaluation

48. C H A I N OF O U T C O M E S University of Wisconsin-Extension, Program Development and Evaluation

49. Tend not be included in a logic model graphic: Situational statement Priorities List of assumptions List of external factors Evaluation methods University of Wisconsin-Extension, Program Development and Evaluation

50. Actions Changes in behaviors and practices Learning Changes in knowledge, attitudes, skills, aspirations Reactions Degree of satisfaction with program; level of interest; feelings toward activities, educational methods Participation Number and characteristics of people reached; frequency and intensity of contact Hierarchy of effects Social-economic-environmental improvements Source: Bennett and Rockwell, 1995, Targeting Outcomes of Programs University of Wisconsin-Extension, Program Development and Evaluation

51. Language: What do you mean by… Goal = Impact Impact = Long-term outcome Objectives (participant focused) = Outcomes Activities = Outputs Outputs may signify “tangible” accomplishments as a result of activities; products University of Wisconsin-Extension, Program Development and Evaluation

52. Goal – outcome definition Goal represents a general, big-picture statement of desired results. “We find that it is useful to think of goals as the answer to the question ‘What are issues that you would like the program to address?’ (e.g., the goal of the program is to address existing community laws and norms about ATOD use) and outcomes as the answer to: ‘What changes do you want to occur because of your program?’ (e.g., the outcome of the program will be to increase the number of community residents who believe teenaged smoking is dangerous).” (Western CAPT) University of Wisconsin-Extension, Program Development and Evaluation

53. Outputs vs.outcomes Example: Number of patients discharged from state mental hospital is an output. Percentage of discharged who are capable of living independently is an outcome Not how many worms the bird feeds its young, but how well the fledgling flies (United Way of America, 1999) University of Wisconsin-Extension, Program Development and Evaluation

54. Logic Model… Represents intention, is not reality Focuses on expected outcomes Challenge of causal attribution Many factors influence process and outcomes Doesn’t address: Are we doing the right thing? Limitations University of Wisconsin-Extension, Program Development and Evaluation

55. From Poister, 2003 University of Wisconsin-Extension, Program Development and Evaluation

56. Logic model with indicators for Outputs and Outcomes Outputs Outcomes Farmers practice new techniques Farm profitability increases Program implemented Targeted farmers Farmers learn Number of workshops held Quality of workshops Number and percent reporting increased profits; amount of increase Number and percent of farmers attending Number and percent who increase knowledge Number and percent who practice new techniques University of Wisconsin-Extension, Program Development and Evaluation

57. Typical activity indicators to track Amount of products, services delivered #/type of customers/clients served Timeliness of service provision Accessibility and convenience of service Location; hours of operation; staff availability Accuracy, adequacy, relevance of assistance Courteousness Customer satisfaction # of clients served # of consultations # of workshops held # of attendees # of referrals Quality of service University of Wisconsin-Extension, Program Development and Evaluation

58. Methods of data collection SOURCES OF INFORMATION Existing data Program records, attendance logs, etc Pictures, charts, maps, pictorial records Program participants Others: key informants, nonparticipants, proponents, critics, staff, collaborators, funders, etc. DATA COLLECTION METHODS Survey Interview Test Observation Group techniques Case study Photography Document review Expert or peer review University of Wisconsin-Extension, Program Development and Evaluation

59. Engagement Engagement is the partnership of university knowledge and resources with those of the public and private sectors to enrich scholarship, research, and creative activity; enhance curriculum, teaching and learning; prepare educated, engaged citizens; strengthen democratic values and civic responsibility; address critical societal issues; and contribute to the public good.

60. Benchmarks of Engagement Evidence of Institutional Commitment to Engagement Evidence of Institutional Resource Commitments to Engagement Evidence that Students are Involved in Engagement and Outreach Activities Evidence that Faculty and Staff are Engaged with External Constituents

61. Benchmarks of Engagement Evidence that Institutions are Engaged with their Communities Evidence of Assessing the Impact and Outcomes of Engagement Evidence of Resource/Revenue Opportunities Generated through Engagement

62. Assessment of Engagement Faculty Evaluation Clear goals

63. Assessment of Engagement Faculty Evaluation Clear goals Adequate preparation

64. Assessment of Engagement Faculty Evaluation Clear goals Adequate preparation Appropriate methods

65. Assessment of Engagement Faculty Evaluation Clear goals Adequate preparation Appropriate methods Significant results

66. Assessment of Engagement Faculty Evaluation Clear goals Adequate preparation Appropriate methods Significant results Effective presentation

67. Assessment of Engagement Faculty Evaluation Clear goals Adequate preparation Appropriate methods Significant results Effective presentation Reflective critique

68. What is required of you? Background of significant scholarship

69. What is required of you? Background of significant scholarship Diagnostic skills

70. What is required of you? Background of significant scholarship Diagnostic skills Use or development of creative and focused methodologies

71. What is required of you? Background of significant scholarship Diagnostic skills Use or development of creative and focused methodologies Information organization and media skills

72. What is required of you? Background of significant scholarship Diagnostic skills Use or development of creative and focused methodologies Information organization and media skills, and Written and oral skills in interpreting as well as presenting information