Se ha denunciado esta presentación.
Utilizamos tu perfil de LinkedIn y tus datos de actividad para personalizar los anuncios y mostrarte publicidad más relevante. Puedes cambiar tus preferencias de publicidad en cualquier momento.

TTIPEC: Monitoring and Evaluation (Session 1)

Powerpoint on monitoring and evaluation for session 1 of the TTIPEC Ghana workshop

Libros relacionados

Gratis con una prueba de 30 días de Scribd

Ver todo
  • Inicia sesión para ver los comentarios

TTIPEC: Monitoring and Evaluation (Session 1)

  1. 1. Monitoring & Evaluation TIPS & TRICKS TTI PEC Anglophone Africa Regional Peer Learning Workshop Coconut Grove Regency Hotel | Accra, Ghana Wednesday, July 16th, 2014 Dr. Mo Adefeso-Olateju and Rebecca Oluwatoyin Doherty
  2. 2. AGENDA Introductions Name, Organization, Brief Description of Project & Phase Objectives Reflective Activities Introduction to monitoring & evaluation Developing an M&E System Practical challenges with M&E Conducting Project Evaluations Conclusion
  3. 3. OBJECTIVES ! ! To provide hands-on support in developing and utilizing M&E tools The session will be focused on practical support and will be as interactive as possible A brief overview presentation on previous M presentations and materials shared at TTI events Introduces potential tools and resources t assess outcomes
  4. 4. REFLECT The first time I heard M&E was _____________________________. ! I t h o u g h t M & E w a s _____________________________. ! I later discovered that M&E was _____________________________. ! N o w I u se M&E in my work by _____________________________. ! Today, I am interested in working on a/an (planned, ongoing, or completed) __________ project. ! I d id my baseline sur vey in/on _________________________________. Designing & settingup the M&E syste Gathering managinginformatio Reflectin critically to improve actio Communicating reporting result Key parts of a M&E system
  5. 5. M & E and Project Planning: “THE CHICKEN, OR THE EGG?” SHOULD WE HAVE M&E PERFORMANCE INDICATORS BEFORE A WORK PLAN? SHOULD WE DEVELOP AN M&E PLAN BEFORE THE PROJECT PLAN? WHAT DO WE DO WITH THE RESULTS OF OUR MONITORING?
  6. 6. M&E IN THOUGHT AND PRACTICE TRUE or FALSE: I always conduct a needs assessment before conceiving a project _________________. ! I should always conduct a needs assessment before conceiving a project _________________. ! I start thinking about stakeholder participation at the onset of my project planning _________________. ! My Monitoring/Evaluation/M&E is always participatory in process and results ___________________. ! We always reflect on the information that we collect for M&E _____________________. Please explain how. ! M&E seems to be more of a measurement tool/activity than a planning or evaluative activity.   ! Whilst measurement (through well-developed performance indicators) is critical to M&E, M&E is a much broader, organization-wide and learning process.  
  7. 7. M&E: A SNAPSHOT OF OUR GROUP PARTICIPANTS KNOW WANT TO KNOW LEARNED - Gloria Eboh, African Heritage Institution (AfriHeritage) - Sola Oluwadare, African Heritage Institution (AfriHeritage) I would like to do this by being able to analyze the data gotten from the M&E tools accurately - Analyzing the M&E results; - How to improve on the M&E tools I am using currently. Simone Fugar. Institute of Economic Affairs, Ghana - Adopt better online monitoring tools; - Knowledge of the most efficient online monitoring tools - Drusilla David, Centre for the Study of the Economies of Africa, Abuja, Nigeria. - Anumudu Peace Onyinyechukwu, Centre for the Study of the Economies of Africa. - CSEA uses the Impact Log as an M&E tool. We will like to improve the log to enable us - capture in detail the activities of the centre - outcome of its research findings - improve on the impact log to enable it measure its impact to its target audience, policymakers and stakeholders to determine if its messages are been used in the policymaking sphere - to enhance development based outcomes in the long,medium or short term as the case may be. - How to use other performance indicators to measure and capture specific communication activities adequately and regularly. - How to measure outcomes from different information products and how their impact and influence stands. - How to strengthen our monitoring and evaluation skills by incorporating other tools and methods - as well as capacity building in tool use to ensure regular data capturing. - Vicentia Kotia, - I use an adapted version of the impact log sheet, a simple excel sheet - No topic in particular but I'd love to learn and get better at M&E - Hands on training on the use of simple but effective M&E tools - Job Eronmhonsele, & Peter Ugholeh. Centre for Population and Environmental Development (CPED) Benin City, Nigeria - I developed to track on-going activities and outputs Just a framework on the website for knowing the number of site visitors within a given period, - and a CHART (still undergoing development) to help monitor CPED invitation to participate and make presentations in conferences and workshops around the globe. - We would like to have a robust M & E tool so as to track visitors and get feedback - as well as know which of the publications appeal to visitors. - We would like to learn how to use Google Analytic; Hoot suite; Twitter Counter
  8. 8. Food for Thought Your project plan should have been from the M&E perspective. E.g. Consultations with the target population informs the needs assessment and the project as the most feasible/desirable/affordable solution. NATURE OF THE PROJECT/BRIEF DESCRIPTION M&E should also be a part of your work plan. E.g. at this stage of the project, this is what we expect to see M&E affects program design because of your means of collecting data Needs Assessment; Assets Assessment/Capacity Assessment Start with an M&E Plan. A good M&E Plan will help a good project…or at least, side-by-side Let us re-do our work plan. Have we laid the foundation for a bungalow? And now trying to build a duplex? Result is a change in behavior. Measure success by outcomes, and not outputs. We often celebrate the result of outputs, and not outcomes or impact. Output is what you put out; however, results should be measured by the success of people (Dr. Chuks Eresia-Eke)
  9. 9. SHORT VIDEO ! https://www.youtube.com/watch? v=MUSzePBTpAY
  10. 10. Introduction to monitoring and evaluation
  11. 11. ! Monitoring: ! the regular collection and analysis of data/information ! communication to strengthen project partnership. ! Evaluation: ! occasional assessment of the overall value and progress of a project Common definitions of M&E
  12. 12. Looking critically at M&E Monitoring = process of collecting and interpreting data to provide management and stakeholders with evidence of the extent of progress towards achieving the objectives from the implementation of a plan ! ! ! Evaluation = process of determining whether a particular plan was well designed (was it relevant, did it have a significant impact and was it sustainable?) and well implemented (was it effectively and efficiently implemented?) → leads to lessons learned for decisions
  13. 13. Changing the M&E Perspective ! M&E is mainly for the external funding body ! Focus on activity & financial reporting ! Lots of data & little analysis ! Little learning ! Little stakeholder participation ! M&E is about highly qualified technical experts conducting statistical studies ! M&E is boring and not very useful for project staff/beneficiaries Shifting Paradigms, methodologies, approaches & tools ■ M&E is for everybody involved in the initiative ■ M&E helps in understanding impact & reasons for success/failure ■ Analysis & critical reflection to improve the initiative ■ Learning is a key function of M&E ■ People processes – lots of stakeholder participation ■ M&E is active, interesting and useful to all involved in an initiative
  14. 14. Developing an M&E System
  15. 15. What is an M&E system? • An M&E system is • A set of planning, information gathering and synthesis, • reflection and reporting processes, • along with necessary supporting conditions and capacities required for the M&E outputs to make a valuable contribution to project decision-making and learning.
  16. 16. Setting up a M&E system ! Practical Steps 1. Define the purpose and scope 2. Decide what to monitor & evaluate 3. Plan information gathering & organisation 4. Plan critical reflection processes & events 5. Plan for quality communication & reporting 6. Plan for the necessary conditions & capacities
  17. 17. Key parts of a M&E system Designing & settingup theM&Esystem Gathering and managinginformation Reflecting critically to improve action Communicating and reportingresults
  18. 18. M&E as an integral part of project management The project strategy! (what will be achieved and how it will be achieved) Detailed operational plan Project outputs, outcomes and impacts The M&E! system! ! ! ! ! Continualrefinement Communicating and! reporting results Reflecting critically! to improve action Gathering and! managing information Developing! the M&E system Fielddata informati on Basis for Basis for Leads to … Mutual refinements Improvements through M&E
  19. 19. Integrated Learning Processes in M&E - Illustration jan marfeb apr sep novoct decmay julyjune aug NGO review meeting Project mgt coord comm meeting Inter-agency steering committee meeting Special review on extension activities Quarterly report Half-yearly & yearly report WORKSHEET
  20. 20. Reflection processes & events - Example WORKSHEET
  21. 21. Results framework = framework used to assess performance ▪ Expectations ▪ Indicators ▪ Sources of information ▪ Baselines and targets + Reports for planning
  22. 22. The 3 Spheres 
 (Tsui et al 2014)
  23. 23. Results chain RESULTS Inputs ! Activities ! Outputs ! Outcomes ! Impact Short-term Medium-term Long-termResources Processes Within control of the organization Partial Org. Control No Org. Control Sphere of Control: Advocacy, Tactics, Inputs, Activities, Outputs Sphere of Influence: Changes in behavior, relationships, attitudes, agenda Sphere of Concern: Policy & Political Context WORKSHEET
  24. 24. Results chain – Example WORKSHEET
  25. 25. Practical challenges with monitoring and evaluation
  26. 26. Insufficient Consideration of M&E at Initial Project Stages ! Unintentionally, M&E is often set to fail during the project design ! No clear direction is given in the project design on how M&E system will be developed. ! No adequate budget for M&E ! No sufficient time and expertise allocated to M&E
  27. 27. Failure to align Phases of M&E with Phases of the Project WORKSHEET
  28. 28. Challenges in defining indicators ! More often indicators are stated and left at the level of a logical framework. ! No attempt is made to develop an indicator matrix that gives the details ! See example………………FAWE Uganda, Asante [Number with reference to logframe] [Name of indicator] Definition Precise definition [=Formula] What it measures [A narrative explanation of the importance of the indicator] Frequency [For example, monthly quarterly, annually, every two years etc] Disaggregation [For example, by grade, sex, school type, geographical area etc] Source Notes Responsibility [Who is responsible in the M&E Unit for collecting the information]
  29. 29. Developing Effective Indicators ! ! Indicator = measure 
 (of performance, progress, condition) ! Properties of good indicator: ▪ Specific = leaves no ambiguity ▪ Measurable = can be compared ▪ Attainable = a source readily exists ▪ Relevant = measures intended progress ▪ Time-bound = refers to specific time
  30. 30. Indicators, baseline and targets (& Planning) OUTCOME ACTIVITY OUTPUT Short- to medium-term Medium-term Medium- to long-term IMPACT OBJECTIVE Design training to increase community involvement Training of SMCmembers on community involvement SMC members are actively involved in school management A school supported by the community helps increase attendance Increasing attendance will yield lower repetition and higher completion Higher completion will improve learning INDICATOR Number of SMC guidebooks printed Number of SMCs where at least 3 members have been trained Percentage of SMCs which are actively involved in school management Attendance rate Repetition rate Percentage of pupils who master specified competencies BASELINE O 10% 10% 75% 8% 30% TARGET 5000 50% by year X 25% by year X 80% by year X 6%by year X 35% by year X WORKSHEET
  31. 31. The importance of monitoring outcomes, and not only outputs – Practical way of conducting evaluations • Considerations • Processes
  32. 32. Evaluation ! A systematic (and as objective as possible) examination of a planned, on-going or completed project. ! It aims to answer specific management questions and to judge the overall value of a project and supply lessons learned to improve future actions, planning & decision making.
  33. 33. (Some) Purposes of Evaluation ! Learning: Generating new insights/knowledge in order to contribute to improved action in the future ! Accountability: Ensuring that an initiative is answerable to its beneficiaries, partners & donors for effective and efficient delivery and for appropriate use of funds.
  34. 34. Different types of project/programme evaluations ! Mid term evaluations (Formative/process evaluations) – To provide feedback on the implementation process ! End of project evaluation (Summative) – To assess outcomes, impacts, sustainability, external utility & lessons learnt ! Self assessments – Internally guided & controlled. To gather information and provide reflection, discussion, debate & improvements **The above could be internal/external – depending on who is commissioning and/or taking a lead in undertaking them
  35. 35. Key steps in a project evaluation Source: IUCN 2004
  36. 36. Common evaluation questions The Situation to Improve Problems and Visions Planned (Goal, Purpose, Outputs and Activities) Inputs Activities Actual Outputs 4. IMPACT 2. Efficiency 1. Relevance 3. Effectiveness 5. Sustainability
  37. 37. Evaluation questions 1. Relevance - Was/is the project a good idea given the situation to improve? Was the logic of the project correct? Why or Why Not? 2. Effectiveness - Have the planned results been achieved? Why or Why Not 3. Efficiency - Have resources been used in the best possible way? Why or Why Not? 4. Impact - To what extent has the project contributed towards its longer term goals? Why or Why Not? Have there been any unanticipated positive or negative consequences of the project? Why did they arise? 5. Sustainability - Will there be continued positive impacts as a result of the project once it has finished? Why or Why Not?
  38. 38. References & Resources ! • Barnett, C. and R. Gregorowski (2013). ‘Learning about Theories of Change for the Monitoring and Evaluation of Research Uptake.’ IDS Practice Paper In Brief 14. Publisher IDS. (http://www.ids.ac.uk/publication/learning-about-theories-of-change-for-the-monitoring- and-evaluation-of-research-uptake) • Batchelor, S. (2012) Information Eco-systems of Policy Actors. https://www.ids.ac.uk/ publication/information-ecosystems-of-policy-actors-reviewing-the-landscape • Beynon, P., Chapoy, C., Gaarder, M. & E. Masset (2012). ‘What difference does a policy brief make?’ IDS, 3ie and NORAD. (http://www.3ieimpact.org/en/evaluation/policy-influence/ policy/) • Doemeland, D. & J. Trevino (2014). ‘Which World Bank reports are widely read?’ • Outcome Mapping: Learning Community http://www.outcomemapping.ca • Stachowiak, S. 2007. Pathways for Change: 6 Theories about How Policy Change Happens. Organizational Research Services. • Sumner, A. et al. (2011). ‘What shapes research impact on policy? Understanding research uptake in sexual and reproductive health policy processes in resource poor contexts.’ Health Research Policy & Systems. Vol. 9. • Tsui et al (2014). ‘Monitoring and Evaluation of Policy Influence and Advocacy.’ ODI Working Paper.
  39. 39. Thank You! Questions/Comments/Feedback
  40. 40. For further inquiries, please contact: Dr. Modupe Adefeso-Olateju, mo.olateju@tepcentre.com Rebecca O. Doherty, doherty.rebecca@gmail.com THANK YOU

×