Welcome to part three of evaluation for RFAs, RFPs and RFQs. After designing your evaluation, you need to manage it and use the results to improve the performance of your program or your individual practice in some way. I find it helpful to look at managing an evaluation across five components.
First, you need to initiate the evaluation by either selecting and engaging internal staff, or by contracting or hiring external staff. This is as simple or as complicated as you make it, and depending on where the money for your program came from, you may have specific requirements for this activity outlined by your donor.
Second is the inception phase. In this phase, the individuals who will be performing the evaluation refine the evaluation questions, develop the tools that will be used to collect data, put together the analytical framework and the protocols used for analyzing the data and build the schedule that will be used to manage the evaluation. The inception phase usually includes an inception report that is ultimately used as the foundation or guide for the work that the evaluation team will perform.
The third phase is the data collection phase. This is where the bulk of the time and energy is spent. Data collection could be as simple as one person conducting interviews with a small group of stakeholders or it could be as complex as performing a large-scale survey across an entire country in multiple communities and everything in between. If you have designed your evaluation correctly, the data collection, or fieldwork, should be no more or no less than what is required to collect the data necessary to answer your evaluation questions.
The fourth phase is analyzing the data you’ve collected. In reality, the analysis of data by your evaluation team will likely start as soon as the fieldwork begins. Especially in evaluations that rely heavily on qualitative data, researchers will instinctively make determinations, form opinions and develop insights as they speak with more and more people. This is been my experience when performing evaluations over the last decade at ISG, and I always find it satisfying how my opinion, understanding and determination about how performance can be improved evolves over the data collection phase. This can be quite substantial. For example, in an evaluation I performed for the Swiss government a few years ago on a local foundation, my evaluation partner and I began the data collection process with the initial sense that the organization was highly ineffective and essentially irrelevant. However by the end of the data collection process, after talking with dozens of people, our collective opinion had changed 180° and we ultimately reported to our client that we believed the work of the foundation should not only be continued, but expanded. Analysis does not only happen on the fly, however, and should follow a rigorous standard protocol to ensure that all data is treated the same way.
The final stage of managing an evaluation is reporting on what you have learned. In my opinion, this is where many evaluations fall flat. At the end of an evaluation 99 times out of 100 you produce a report. A long, dry, research-driven, data-heavy report. If we are honest with each other, a report that very few people will read, and even fewer will take action on or share with others. So the challenge you have is to report on your findings in a way that is visual, engaging, bite-sized, sexy and digestible for the attention-starved, social media-driven world we live in.
Once you’ve completed an evaluation, you want to be able to use that material to improve your performance, or the performance of the project are working on. One way you can use your evaluation is to adjust your programming and provide information into the feedback loop you maintain as a manager to make sure your project is always being improved on. Another way to use an evaluation is to contribute your findings to the greater aid and development community by publishing results and sharing for data and materials. A third way to use your evaluation is to look at future delivery. By seeing how you performed in your current project you can apply these lessons to a new project for a different client, or use the lessons to develop a new product or service that will leverage what you’ve learned.
In the next section of this training I’ll be discussing some of the best practices we have developed over the last 10 years at ISG in designing, managing and using evaluations. Thank you again for watching this training on evaluation for RFAs, RFPs and RFQs. Remember, if you have any questions at all please contact us training@Aidpreneur.com