Lifecycle of a Service Project (Part 5): Evaluation and Promotion
•Descargar como PPTX, PDF•
2 recomendaciones•2,432 vistas
1) The document discusses evaluating and promoting Rotary service projects. It emphasizes that evaluation should begin at the project design phase and continue throughout implementation. 2) Effective evaluation plans have clear and relevant measures that are tied to the project's anticipated impacts. They also dedicate resources to monitoring and evaluation activities. 3) Promoting projects is important to share Rotary's story and impact. Evaluation data provides evidence of success and lessons learned that can be used in promotion. Rotary resources like newsletters, social media, and websites can help with promotion.
Recomendados
Recomendados
Más contenido relacionado
La actualidad más candente
La actualidad más candente (13)
Similar a Lifecycle of a Service Project (Part 5): Evaluation and Promotion
Similar a Lifecycle of a Service Project (Part 5): Evaluation and Promotion (20)
Más de Rotary International
Más de Rotary International (20)
Último
Último (20)
Lifecycle of a Service Project (Part 5): Evaluation and Promotion
- 1. Lifecycle of a Service Project (Part 5): Evaluation and Promotion Rotary International 20 May 2014 #Connect4Good
- 2. May 2014 WELCOME TO THE WEBINAR #Connect4Good Sheena Lilly Coordinator, Regional Membership Plans Membership Development Adam Arents Supervisor, Learning Technology Learning and Development
- 3. May 2014 During today’s webinar you will: • Learn more about why evaluation is important to project success • Hear tools and tips for how to evaluate you project • Learn how to effectively promote your project LEARNING OBJECTIVES #Connect4Good
- 4. May 2014 PARTICIPATING IN THE WEBINAR #Connect4Good Getting connected to audio Use the Audio pod to select Use Telephone - or - Use Mic & Speakers * To improve sound quality, please close all unnecessary programs such as email, MS Office, etc. If you have a cellular device, please move it away from your computer.
- 5. May 2014 LIFECYCLE OF A SERVICE PROJECT #Connect4Good
- 6. May 2014 LIFECYCLE OF A SERVICE PROJECT #Connect4Good
- 7. May 2014 EVALUATING YOUR PROJECT What is Monitoring & Evaluation, and why is it important? – Monitoring is review that takes place during a project – Evaluation is a timely and comprehensive look at the project during implementation and after the project is completed – Why does it matter?
- 8. May 2014 EVALUATING YOUR PROJECT When does monitoring and evaluation start? – Monitoring and evaluation truly begins at the project design phase – Determine what you want to do and how you can have an impact through a community needs assessment • Then figure out how you can measure it
- 9. May 2014 EVALUATING YOUR PROJECT What does Rotary expect? – Funded – • Reports must include total number of beneficiaries plus one standard measure selected at the time of application • Up to 10% of budget allowed for monitoring and evaluation costs – Unfunded – • There are no formal requirements, but it is still a best practice to collect and report monitoring and evaluation data
- 10. May 2014 EVALUATING YOUR PROJECT What does a good monitoring and evaluation plan look like? 1. Less is more 2. Measures match core project components 3. Measures tie directly with the project’s anticipated impact 4. Mixture of types of measures 5. Resources are dedicated to monitoring and evaluation activities
- 11. May 2014 PROMOTING YOUR PROJECT Why should we promote our project? How is evaluation connected to promotion? – Promoting your project is your opportunity to tell Rotary’s story – Use your monitoring and evaluation data to document success and provide tangible proof of your accomplishments
- 12. May 2014 PROMOTING YOUR PROJECT What resources can help me document and promote my project? • Fellow Rotarians/club and district officers • Club and district newsletters • Rotary Public Image Coordinators • Rotary systems • Brand Center • Rotary Showcase • Social media
- 13. May 2014 POLL
- 14. MAY 2014 Meet our panelists
- 15. May 2014 MEET OUR PANELIST #Connect4Good Tristam Johnson Rotary Club of Brattleboro Sunrise, Vermont, USA District 7870
- 16. May 2014 MEET OUR PANELIST #Connect4Good Michael Barrington Rotary Club of Concord, California, USA District 5160
- 17. MAY 2014 Evaluation and promotion in your projects
- 18. May 2014 TELL US ABOUT YOUR EXPERIENCE #Connect4Good Monitoring Evaluation Timing Takes places continually throughout project implementation Assess the entire cycle Purpose Represents an integral part of project management Reviews achievements of the project and examines if the plan was the best one to achieve the expected outcomes Focuses on project implementation comparing actual deliverables with what was planned Measures achievements plus positive/negative and unintended effects Looks for lessons learned from both success and failure; looks for best practices that can be applied elsewhere Facilitator Usually done by those directly involved in the project Can be conducted by an independent outsider who can be impartial in consulting with project staff Practical differences between monitoring and evaluation
- 19. May 2014 TELL US ABOUT YOUR EXPERIENCE #Connect4Good
- 20. May 2014 TELL US ABOUT YOUR EXPERIENCE #Connect4Good
- 21. May 2014 TELL US ABOUT YOUR EXPERIENCE #Connect4Good
- 22. May 2014 TELL US ABOUT YOUR EXPERIENCE #Connect4Good
- 24. May 2014 TELL US ABOUT YOUR EXPERIENCE #Connect4Good
- 25. May 2014 TELL US ABOUT YOUR EXPERIENCE #Connect4Good
- 26. May 2014 TELL US ABOUT YOUR EXPERIENCE #Connect4Good
- 27. May 2014 TELL US ABOUT YOUR EXPERIENCE #Connect4Good
- 28. May 2014 TELL US ABOUT YOUR EXPERIENCE #Connect4Good
- 29. May 2014 TELL US ABOUT YOUR EXPERIENCE #Connect4Good
- 30. May 2014 TELL US ABOUT YOUR EXPERIENCE #Connect4Good
- 31. May 2014 ADDITIONAL ROTARY RESOURCES #Connect4Good Rotary.org resources • Rotary Club Central • Showcase • Rotary Grants • Brand Center • Global grant materials – Grant Management Manual – Global Grant Monitoring and Evaluation Plan Supplement
- 32. MAY 2014 Your service project tips #Connect4Good
- 34. MAY 2014 Thank you for attending today’s webinar! Register for upcoming webinars and view recordings here: www.rotary.org/webinars
Notas del editor
- SHEENA: My name is Sheena Lilly and I am fromMembership Development. I will be co-moderating today’s session along with Adam Arents from Learning and Development. You may remember us if you joined us for the earlier webinars in this series.ADAM: Hi everyone.As Sheena mentioned, I’m Adam and I support webinars and e-learning. We’re happy to be here again to discuss how you can evaluate and promote your service project.
- SHEENA:Today’s webinar will help you produce sustainable service initiatives. We will highlight strategies other Rotarians have used in their service projects, offer best practices, and guide you to Rotary resources that can help you with your service project.As many of you know, this is the fifth in our series. You can find details about this webinar series and links to recordings for each of the previous four sessions on the Webinars page at ROTARY.ORG/WEBINARS. We’ll include a link to this page, and all of the resources we mentioned today, in a follow-up email after the webinar. All participants will also receive a link to the recording of this webinar. ADAM:Today you can expect to:Learn more about why evaluation is important to project successHear tools and tips for how to evaluate you projectLearn how to effectively promote your projectWe also want to help our attendees learn from each other during this webinar. So during our time together, we will ask you to share your tips for monitoring, evaluation and promotion by typing in the question pane, and we’ll also ask you to vote in a few polls. We encourage you to participate actively in the webinar. Send us your questions and comments either through the question pane or on Twitter using the hashtag #Connect4Good. We’ll be sharing some of your questions and comments as they come in. Now we’ll take a moment to show you how you can participate.
- SHEENAYou each have your own control panel in the upper right corner of your screen that looks similar to the one here. Use the orange arrow to open or close your control panel.ADAM: For today’s webinar we have __ registrants from __ countries joining us. To maintain the highest sound quality possible, all but our presenters will be muted during the webinar. Please submit questions or comments to our panelists and to Rotary staff by using the question pane on your control panel.You can also use the question pane if you’re having technical difficulties. Just describe the problem you are having in the question pane, and a staff member will assist you right away.Now, let’s take a moment to practice. Please use your question pane to type in one or two words to describe how you currently evaluate and promote your projects.****** Comment on answers coming into the questions pane
- SHEENA:We mentioned earlier that every service project is unique. However, there’s one thing service projects have in common, which is that they tend to share a common “lifecycle”. On your screen is a graphic that represents the lifecycle of a service project.Each webinar in this series touched on different aspects of these key areas, and gave you best practices and tips that you can implement right away, no matter where you are in the lifecycle of YOUR project.
- SHEENA:Today, we’re going to focus on evaluation and promotion. Remember to keep your project goals in mind as you consider your plans for how to evaluate accomplishing these goals and how to promote your successes. Take your time determining which strategies and tools are most appropriate for documenting the success of each of your goals.
- ADAM:In the world of service projects, you rarely hear people talk about evaluation without also hearing the word “monitoring” associated with it. We typically just call it “M&E” and expect it be part of the project lifecycle, but for many it is a new concept. So what is monitoring and evaluation, and why is it so important in the project lifecycle?Monitoring is regular review of the project milestones and accomplishments while a project is taking place.Evaluation is a comprehensive review of the project accomplishments and impact during and at the end of the project, and for a period of time after the project is completed. Today’s webinar will focus more on project evaluation, but it is important to mention that monitoring should also be a central part of your overall evaluation plans. Regular project monitoring and evaluation are critical to understanding the successes and weaknesses of your project. You can design even better projects in the future by understanding what works and what doesn’t. And you can create a strong positive public image for your club in your community, and for Rotary around the world, by documenting your success.
- SHEENA:If you joined us for the second webinar in the series, you’ll remember that monitoring and evaluation actually begins at the project design phase. When you are in the project planning phase, discussing community needs and thinking about options for your project, you will begin to see areas where your club can make an impact. These will become your project goals and shape the implementation plan. Monitoring and evaluation are simply the follow up and analysis of your success toward your original project goals.For example, your club is interested in working with a local school. Through discussion with the community, you identify the need for additional materials within the school, but also a need for teachers to be trained on a current teaching methodology that may address some of the gaps in the students’ performance. Your evaluation plan may include a description of the type of materials, who got them, and how did these materials benefit them. Teachers trained in a new methodology may be asked if the training was effective, if they were able to apply it, if they are able to measure improvements in student performance. If you are ambitious and the teachers support this, you can collect a base line on current grades before your project and then collect grades for a comparison once the project has been completed if the teachers feel this might fairly reflect the positive impacts of the training.
- SHEENA:Rotary’s expectations vary depending on whether the project is considered funded or unfunded by the Rotary Foundation. In either case, it is a best practice to establish regular monitoring and evaluation as part of your project lifecycle.For Rotary-funded projects, typically global grants, project sponsors are required to select a standard measure at the time they complete the application. These measures are pre-populated into the application once an area of focus is selected, and an application cannot be submitted without a measure. Though they should be tracked throughout the project, they are only reported at the final report. Additionally, project sponsors are required to provide the total number of direct beneficiaries in the final report as well. Understanding that this evaluation may be time consuming and challenging for some clubs, Rotary allows up to ten percent of the project budget to be allocated to project monitoring and evaluation by an outside source.For projects that do not directly receive funding from Rotary, project sponsors are not required to track or report evaluation data. However, it is still a best practice to collect this information during and after your project.
- ADAM:Now that we’ve covered what a monitoring and evaluation plan is, we can explore further into what a good monitoring and evaluation plan looks like. In general, there are five key principles to establishing a good monitoring and evaluation plan:1. Less is more, particularly if you are just starting out in monitoring and evaluation. Monitoring is simply active involvement in the project’s implementation, making sure that the objectives are being met on the established schedule; evaluation is checking your success against those objectives. Stick to basic, easy to count measures, such as total number of direct beneficiaries, number of teachers trained, or number of wells dug, or even designing your own publication to track your projects. These tangible items are the easiest to track achievement on. Just make sure you do not double count!2. Make sure the measures track your progress against your main project objectives. A good monitoring and evaluation plan does not seek to track everything, but rather to highlight success in a few key areas – perhaps more importantly, the most successful projects and monitoring and evaluation plans keep a modest scope! SHEENA:3. Your proposed measures should tie directly with your project’s anticipated impact. Using the school example we discussed earlier, it may be interesting to show that attendance increased as a result of your project, but if your goal was to train teachers and provide materials, it may not be necessary to track attendance for this specific project. 4. If you are more experienced in monitoring and evaluation, the measures should be diverse. Counting activities and participants is a good place to start, but does not necessarily show the quality of the project being implemented. Measures that track increased knowledge, increased capacity for work or income generation, or documented decreased incidence of water-borne diseases are examples of quality measures.5. A good monitoring and evaluation plan, particularly for a Rotary Foundation funded project, will include financial resources specifically dedicated to monitoring and evaluation. Rotary does not expect Rotarians to become experts in monitoring and evaluation, just as Rotary does not expect Rotarians to become experts in drilling wells for a water project. It’s perfectly acceptable – even encouraged – to find local experts to assist you in collecting this information. In many cases, this may be a cooperating organization that you are partnering with. As long as you have ready access to the information and are actively part of the planning and implementation process, use the experts as needed!
- ADAM:Evaluation is just one of our topics for today. Now we’ll focus on promoting your project. For many clubs, promoting a project is second nature. However, the connection between monitoring, evaluation, and promotion may be a little less obvious. Through regular monitoring and evaluation of your project, you begin to develop promotional materials. By documenting your accomplishments on an ongoing basis, you have the information needed to effectively promote your project.Your project is part of Rotary’s story. Promoting your club’s projects becomes part of a larger conversation about the good Rotarians do around the world. Having documentation such as evaluation data, pictures, beneficiary testimonials provides a rich picture of your project’s impact.
- SHEENA:Just like with evaluation, promotion does not have to be difficult, and every good project should have a promotion plan as well as an implementation and evaluation plan. If you don’t know where to start, Rotary has many resources that can give you ideas for how to promote your project. You can look for local resources, such as club and district public relations chairs. These individuals may already have contacts with local media outlets that can help promote your work. At the zone level, the Rotary Public Image Coordinators can provide the same support.Rotary’s brand center, launched earlier this year, provides tools and templates you can use to create visual images for your project outside the Rotary world. You can promote your project to fellow Rotarians through Rotary Showcase. In fact, information input into Rotary Club Central can automatically be sent to Showcase for promotion!Additionally, Rotary created another webinar on this very topic, called “Share Your Impact! Best Practices for Telling Your Service Story.” This webinar provides tools and ideas for how you can create effective promotional materials. We also have a number of webinar recordings on how to take advantage of social media to promote your humanitarian endeavors.Finally, there are many, many online resources available for promoting your project. Club and district websites and newsletters, as well as social media sites like Facebook, Instagram, Twitter, Google+, and LinkedIn, all may be opportunities to share your project’s successes.
- SHEENA:Before we continue with the webinar, you have a chance to participate in our first poll of the day. We want to know your level of comfort with measurement and evaluating your project.When it comes to setting measures and evaluation, my club:Has never heard of monitoring and evaluationIs still figuring out global grantsHas created a few measures for our current projectHas well established, long-term evaluation plans in placePLEASE SELECT ALL OF THE OPTIONS ON YOUR SCREEN THAT APPLY TO YOU:[DISCUSS POLL RESULTS]
- SHEENA: Thanks for participating in the poll. Now let’s meet our panelists.
- SHEENA:Our first panelist is Tristam Johnson. Tristam is a member of the Rotary Club of Brattleboro Sunrise in District 7870. After too many years in tourism in the Rocky Mountains, Tristam successfully secured a Masters in International and Intercultural Management and spent 16 years working on community development projects in nine countries of Latin America. His project types included health, education, micro-finance, small business, economic development, local governance, and institutional growth. Tristam managed the awarding and tracking of grants as well as the monitoring and evaluation components that allowed clear tracking of progress and impacts. He has now spent easily 20 years on project development, always keenly interested in identifying the measurable impacts that could demonstrate success. Tristam has even had experience with Matching Grants and 3H Grants. He now serves as a member of RI’s Cadre of Technical Advisors reviewing project formulation, implementation, and evaluation.Welcome, Tristam!TRISTAM:Thank you for having me!
- SHEENA:Our second panelist is Michael Barrington. Michael has been a Rotarian for sixteen years and a Cadre Technical Advisor for twelve. He is the Past President of the Rotary Club of Concord, California, a multiple Paul Harris fellow, the Club Foundation Chair and the Chair of International Projects for District 5160. Michael manages his own business providing professional monitoring and evaluation, Statistical Analysis, Impact Assessment and Process Reengineering services to major NGOs worldwide. He has worked in more than 30 countries. He has a Ph.D. in Organizational Management and speaks six languages. He wants his Rotary epitaph to be: “I tried to change the world one life at a time.” It’s great to have you here, Mike!MIKE:Thank you!
- ADAM:Tristam and Mike, let’s talk about your projects and your experiences with evaluation and promotion.
- ADAM:Mike, how would you describe the difference between monitoring and evaluation in practical terms?MIKE:In general for a club designing an international project, emphasis should be placed on building in a solid monitoring plan to ensure that the project is on track in meeting its goals and objectives. This is critical to the project since it will enable the club both to review each stage of development and where necessary fine tune, recalibrate or make a significant adjustment. With close monitoring any modifications required are much easier to make. It also means that there will be few if any surprises once the project is completed and the evaluation takes place.The evaluation stage is where a detailed objective review of every aspect of the project will be examined especially the actual outcomes versus those initially proposed. It will look at lessons learned and ask questions such as “was this plan the best one to achieve the stated outcomes and are there lessons to be learned from the initiation, management and completion of the project. And sometimes a project can take unforeseen turns.In Eastern Russia I was asked to evaluate a large immunization project giving shots for mumps, measles and rubella to thousands of children. It should have taken three years and took almost five. Both the international and host clubs worked extremely diligently on the planning of this project and throughout the lifecycle. However the attendant circumstances turned the project upside down. UNICEF worked with the Rotarians and provided very affordable vaccine which was delivered by ship to Vladivostok. On arrival t was immediately impounded since it was discovered that the vaccine was manufactured in South Africa and Russia was restricting all trade with that country. After months of negotiating it was agreed that samples should be sent to Moscow to be tested by a Russian laboratory. It took almost a year to get the results and another three months to get the vaccine released. Once released the Russian Rotarians had to rework all of their now outdated implementation plans and prepared a massive media campaign: radio, press, TV etc. Eventually the intensive immunization process began. One month into the campaign a call came from Moscow “to cease and desist.” News of the campaign had reached the international press and they were now asking why the Russian government did not vaccinate its children. Another long delay took place until a compromise was worked out: The Rotarians would administer shots for Measles and Mumps and a Russian medical team would deliver Rubella. Even with the most careful project planning it is hard to imagine that these kinds of challenges and setbacks could have been foreseen.
- SHEENA:Tristam, why do you think monitoring and evaluation is important?TRISTAM:Evaluation feeds critical information to project designers, managers, and funders both during and at the close of a project. A project evolves based on assumptions about a need and then gets developed so that it can address that need. Sometimes the initial assumption is “ground-truthed”; verified by those who will benefit, but sometimes not. The project will include goals and objectives explicitly stated or implied. It will offer a suggestion of the end results and also the path or strategy that will lead to those results. Lastly, timing of the different steps, stages, or strategies of the project may be described illustrating how each intersects with the other or leads logically sequentially to the end result.Evaluation during the project, clear, objective, sincere evaluation, can review each of these core elements, and help the management team to make timely adjustments that allow accomplish of the goals. An evaluation at the end of the project is vital for creating credible relationships between the funder and the management team. The results may clearly justify the investment. A final evaluation can also illuminate to the project beneficiaries next steps, refinements, or expansion. Evaluation at this stage also permits identifying weak points, which in turn can lead to a far better sequel. Best of all, evaluation should be continuous consistent and objective to render the best results.
- ADAM:So, then, Tristam, how have you incorporated monitoring and evaluation into your projects in the past?TRISTAM:The monitoring process carefully watches that the unfolding of the project follows the design that was developed, agreed upon, and supported. Evaluation looks at the results of planned efforts to see if they are creating the hoped for results, that each successfully laid the groundwork for the subsequent, and that the impact indicators are in fact accomplishing the goal.I have spent a lot of up front time developing a project in order to have a very clear project process, a very clear sequence of steps, a clear resource investment plan, and easily identifiable impact indicators that can demonstrate the value of the project and that it can accomplish its stated goal(s). When using resources, yours or someone else’s, it is particularly vital to ensure that those resources are not being wasted. One project I worked on from Bogotá, Colombia, focused on job generation. The prime $5million project proposal very clearly defined the number of direct and indirect jobs that would be generated. My task then was to make sure every contract for work included an obligation to track job generation and to report the numbers on a quarterly basis. I then followed up with each contract recipient ensuring that they reported the numbers and did so in a systematic consistent fashion.
- SHEENA:And Tristam, how has evaluation helped you plan future projects?TRISTAM:An interesting experience with micro-credit projects in Bogotá allowed me to alter the loan repayment plan. For micro-credit projects each micro-entrepreneur offers a business plan and then agrees to repay the loan on a weekly, bi-weekly, or monthly basis, depending. A young fellow set up a fruit & yogurt mobile cart, non-motorized, in front of a school and did quite well during the active school year, but made next to nothing when the kids were on vacation. Evaluation in this case allowed me to re-structure the loan pay-back obligation taking into account the cyclical nature of the business. This experience then made me far more aware of those variables we often take for granted so that I deliberately look for variables that may have an impact.Evaluation also helped alter the fundamental assumptions that created the core of the 3H Grant the launched Pure Water for the World. I learned from a pediatrician that kids with intestinal parasites from drinking contaminated water needed about 6 months with clean water before it made any logical sense to offer a treatment to flush out the parasites; the benefit of doing it sooner could be too easily lost.
- ADAM:Switching gears a little bit, Mike, why is promotion important for projects?MIKE:All of our International Projects are promoted in several ways: We have a large mobile display unit that we wheel out at every meeting. It contains text and photos of each project. We meet in a hotel so everybody who enters or leaves the hotel gets to see it. If there are other events taking place in the hotel its amazing how much attention it attracts. It is on display every Friday from 11:30 until 1:30.We have a designated club member who is also a member of the BOD with the title of PR. It is his/her responsibility to continue to develop and nurture strong relationships with the local press. As a result we regularly have articles in the local press and are mentioned on local cable TV.We have an excellent Club web site where most of our activities are publicized.Our club has a very well designed and organized recruitment process. When we have a potential new member each Committee Chair is invited to sit with the candidates(s) and the Membership Director in a special, informal meeting and present, in my case, what International projects we have in process. I normally have a couple of my team with me since we usually have four or five international projects running simultaneously. It really is ‘internal marketing’ and a great way to demonstrate Service Above Self in a very concrete manner. Its amazing how this takes off and people tell their friends about the works of Rotary.We have been drilling water wells in the Sahel region of North West Burkina Faso for the past four years. What started as single club project developed into a project by eight local Rotary Clubs. Since then it has attracted interest all over our District and more than 40 clubs stated that they wished to participate. We have received requests from twenty villages for a water well but we don’t have the money to do this. The wells are deep, have to be drilled and cost about $16,000 US. Because of the interest in the project I invited the International Chairs from eight local clubs to form a committee where we would work together and I would educate them regarding Global Grants and the mysteries of TRF. The object also was to focus not only on Global Grants but on attracting corporate funds and private foundation funds funds from outside of Rotary. Since my recent visit we have decided that once we have completed our current objectives of five wells, instead of drilling wells we will attempt to raise sufficient funds to purchase a drilling rig and ship it out to Burkina. This is a very expensive undertaking but it is already attracting a lot of attention from some major corporations including Coca cola and several US companies that specialize in building/selling rigs. I am focusing totally on this project by contacting corporations and foundations, providing them with photos and short videos to try and spike their interest: so we shall have to see how it develops. Our International Committee makes every effort to promote the work of Rotary. We are invited by other clubs to make presentations and we ourselves ask clubs if we can present. In addition we receive requests from other organizations outside of Rotary e.g . Soroptimists, SIRS, Church groups etc. who want to hear about our projects. Usually they have read about us in the press, our web site or by word of mouth. I usually make about 12/15 presentations a year and my colleagues make about 5/10. For us it is all about promotion.
- SHEENA:Now, let’s approach project design and evaluation from a different perspective. Both of you serve on The Rotary Foundation’s Cadre of Technical Advisors, reviewing grant applications prior to approval, or conducting project site visits during implementation or after the project is closed. Because this role gives them a larger view into what types of successes and challenges Rotary clubs face, we’re going to spend a little time talking about their experiences and perspectives as Cadre members.
- ADAM:Tristam, how has your experience as a Cadre member shaped your view of monitoring and evaluation?TRISTAM:I have now evaluated four projects for RI and numerous others for international contractors. My experience as a Cadre member has made me see the educational value of helping Rotarians understand the multi-faceted benefits that can result from evaluation. Moreover, in order to evaluate, those responsible for a project need to have a very clear image of the results of their project. This must be so clear that they can easily identify simple easily trackable impact indicators that can be gathered on a regular basis, the gathering process monitored for consistency, and the indicators reviewed for trends and progress. Gathering indicators from the start, compared to a baseline, allows for timely realization of strength or weakness in the project and effective adjustment. Who wouldn’t want that?
- SHEENA:Mike, can you tell us about your recent monitoring and evaluation experiences in South America?MIKE:
- ADAM:Tristam, based on your experiences thus far, what would you say are the most common issues you see in applications?TRISTAM:Outcomes are not specific and measurable..implementation strategies are not clearly thought out, logical & sequential..goals are too broad..monitoring is not clearly defined.. contingencies are missing…the simple easy impact indicators are often missingADAM:So how do you suggest people address them?TRISTAM:I’ve been involved with Rotary projects and grants since the late 90’s, Matching Grants & 3H Grants, and I have seen a very impressive shift from an informal application process with minimal accountability to processes that are rigorous and accountability is now demanding and clear. In addition, I now see many resources and connections a Rotarian can take advantage of to tighten an application, make it a stronger submission, and to create a clear project management plan. Also, this webinar introduces 2 of us who are willing to respond to questions, to be helpful.
- SHEENA:Both of you have extensive project experience outside Rotary as well. So, when talking project planning and evaluation in general, what do you see as the most common challenges? Tristam, we’ll start with you. TRISTAM:I believe that projects start from the heart, a response to a vital need, and then the project action plan gets designed without a clear and very specific focus on the purpose, the target of RI funds and Rotarian efforts. Instead, the project moves in generalities. From this perspective, action steps and resource commitments seem to leap over obvious logical moments to check in with project core assumptions and project beneficiaries. Sometimes there is a data baseline missing that establishes the current situation against which impacts and progress can be measured.SHEENA:And for an evaluation plan?TRISTAM:An evaluation plan ought to be created as a tool to keep project success moving forward. To that end,The hoped for impacts of the project should be easily defined, easily collected, and easily measuredThe impact indicators should be collected at regular intervals in a consistent mannerThese indicators will tell a “story” about actual progress as compared to hoped for progressProject management team can take the results of the “story” and in a timely way adjust, if necessary, to keep progress positive and successful
- ADAM:And to take that one step further – with so many variables and so many things out of a club’s control in any project - what happens when projects go awry from the original plan or are not as successful as the club anticipated? Mike, can you share your experiences working with clubs on this issue?MIKE:This is a very difficult topic. In cases where the project was not fully completed, I have found - that clubs were concerned and feared there would be serious repercussions from TRF! They were very reluctant to talk about it. I conducted a visit for a large Grant that was in its sixth year, for a project that should have been competed in three. My role was to evaluate the project and if possible find solutions so that the Grant could be closed. When my visit was announced by TRF, I received an email from the club president telling simply that I should not come! Within the club the whole project had created a split resulting in several members leaving including the principal team member for the project. There was also a deep fracture and total breakdown in communication between the host and the international club that involved a whole district. During my visit, I saw that the project suffered because of:Over reaching: it was too complex and outside the scope of Rotary volunteers.Lack of Rotary expertise in project managementLack of a realistic budget that took into consideration ‘inflation’ and the tremendous competition for local construction materials. The logistics were complicated as the project site was located a great distance from the host club and there were no roads leading into the beneficiary community. There was no consideration given to the fact that no construction could take place during the monsoon season.There was no money left to rectify or complete any more construction.The solution: What was done was done!There were a few issues that could be addressed by the club, so we focused on what they could do well in a short time frame. They needed to accept their mistakes and this could only be done by helping the club admit and own its failures without rehashing and resurrecting all of the pain they had gone through. My simple approach with them was that “if we do not study our history then we are doomed to repeat it.” I went through this process with all of the club members present and did it quickly.I then took more time to focus on the positive and enabled them to see that in spite of everything they actually had achieved quite a lot. There were X numbers of families now housed that otherwise would not have shelter. They had involved many communities and obtained sweat equity from people who had never heard of Rotary previously.They now had a wealth and variety of experiences that would serve the club in its future projectsThey had learned that a choice of international partner is more than just to provide funding. There has to be unequivocal communication and mutual support between both sponsors.I showed them that for the future in very complex and large grants it is essential to have a plan in place indicating how disagreements or misunderstandings can be resolved. (Easier said than done!).
- SHEENA: And finally,Mike, if you could give advice to someone who is just starting out in evaluation, what would it be?MIKE:In looking to the future the Rotary Foundation is now seeking statistical data from some projects with a view to not just measuring outcomes and sustainability but also to examining whether there are elements that could be used and replicated in other regions or countries. I might suggest to a person starting out:Learn everything you can about the meaning of evaluation….What is Primary data?What is Secondary data?What types of data are needed?Qualitative (visual or words)Quantitative (numerical)Participatory & Non Participatory information gathering. How will the process of information gathering be used by the project?Main methodology: Individual interviewsGroup discussionObservationStructured surveysQuestionnaires. Etc.All of the above are tools that could be used by any evaluator. Prior to your visit study not only the material provided by TRF but read around your subject. I normally carry a 3 ring binder which I refer to as my ‘bible’, containing all kinds of information. Don’t be afraid to ask any questions. I prepare all of my basic questions beforehand even though I may have to modify them on site or possibly not even use some of them.Try to interview people other than direct beneficiaries and get their view of the project. Through discussion with the host clubs I try to generate lists of people I would like to interview. This might be augmented once I arrive on site.Above all be prepared to find that the project has not be delivered exactly and to the letter as prescribed in the original grant.And finally be ready find that after all of your hard work and preparation, your carefully planned visit and scheduled interviews may need to be re-arranged as you are actually working.
- ADAM:And from your perspective, Tristam?TRISTAM:Keep it simple; don’t try to create or buy or involve software tools that are too professional. Excel is an outstanding data management tool, easy to use, universally available, and exportable.Look for very specific short term impacts that quickly reveal success.Remember that most projects began from the heart, which most often means improving the human condition, the quality of life. What part of your project can create that improvement and what indicator will help you tell that story?To evaluate the success of a project, you need to define what success looks like. If clean water is the project focus, the number of units installed is less meaningful than a reduction in the frequency of diarrhea, even better, per a specific cohort.If the project is improving medical clinics, define the immediate positive community impact that can attest to the importance of improved clinics.Evaluation is never meant to judge people or their actions. It is simply a tool that can help everyone stay on track, shift if necessary to alternative strategies, modify outcomes and redesign resource commitments.
- SHEENA:Thank you to our panelists today, Mike and Tristam, for your stories and experience!There are many other resources available on rotary.org to help you with project planning, evaluation, and promotion. If you have not already done so, you’ll need to create a MyRotary account to access the information. Creating an account is easy – if you’re an active Rotarian, all you need is your email address, your member ID, and your club name. If you’re not a Rotarian, you can still create an account and access a lot of great information.Once you’re registered, you can access things such as:Rotary Club CentralShowcaseRotary GrantsBrand Center And grant-specific training materials
- ADAM: We’ve heard a lot about Tristam and Mike’s experiences with evaluation and promotion. Now we want to hear from you – please share your tips for evaluation and promotion with the audience. Use the question pane to type in your tips. Again, you can also share your tips on Twitter using the hashtag #Connect4Good. [READ SOME TIPS THAT THE AUDIENCE SUBMITTED THROUGH THE QUESTION PANE]SHEENA: (After a few minutes of questions have gone by) Keep those tips coming. We’ll continue reading some of these during the last part of the webinar.
- SHEENA: While we’re more of your service project tips are coming in, let’s go to your questions. We’ve already received a lot of great questions, and we will answer as many of them as we can over the next few minutes. Anythat we can’t respond to during this webinar will be answered by email afterward.The first question comes from ______. [Read questions, flagged in RED, and direct to the appropriate panelist for response.][Intersperse questions with Service Project Tips, these will be flagged in YELLOW.]ADAM:We’re nearing the end of our time for today’s session. Thank you for attending. We appreciate your participation! You can also find a recording of today’s webinar, and the previous webinars, on the same page – just click “On Demand”. After today’s webinar ends, you will see a survey on your screen. Please take a minute to complete the survey, since we will use your feedback to help us develop the rest of the webinars in this series.
- ADAM: Feel free to share the recording with others who may have missed today’s webinar.Within a few days, you’ll receive an email with a link to a recording of this webinar, along with information about some of the resources mentioned during today’s presentation. Another thank you to our panelists, Tristam and Mike, and all of our participants. We hope to see you at a webinar in the future!