Evaluate

Best practice standards, a menu of resources, and step by step guidance for DMERL (Design, Monitoring, Evaluation, Research & Learning) in each phase of the project management cycle.

Why Evaluate?

The Evaluation phase is critically important to any humanitarian program team or organization. Just as the Design phase is foundational for project impact, the Evaluation phase works to understand how we are progressing towards impact and whether we are achieving it. Evaluation enables you to track progress, examine possible changes in the context, explore impacts of the program(s) to date and consider potential course correction (for those programs still ongoing), or using the lessons and recommendations for future programs or strategies.

Evaluating enables projects to:

Learn from past mistakes or gaps.
Build on past successes.
Be informed by the voices of a diverse set of stakeholders.

Based on your Evaluation, you can decide what to do next, either in the project lifetime or for future projects. The Management Response Plan (MRP) is a planning document that is used to facilitate these conversations about what actions to take following an evaluation, so that your organization can improve its performance over time.

Best Practice Standards

This DMERL Framework shares two Best Practice Standards for the Evaluate phase of DMERL in the Project Management Cycle.

Best Practice Standard

All American Red Cross projects must be evaluated using a common evaluation framework or justification documented as to why not.

Best Practice Standard

Every evaluation has a Management Response Plan (MRP) addressing evaluation results and they are acted upon and documented.

Key Resources on Evaluation in DMERL

October 20, 2013. Binga District, Zimbabwe. As part of its Building Resilient African Communities (BRACES) program, the American Red Cross is working with the Zimbabwe Red Cross to implement disaster risk reduction efforts featuring water, sanitation and hygiene projects as well as clean cookstoves. Before projects begin, volunteers from the Zimbabwe Red Cross go house to house surveying residents to determine what needs are the greatest. Here, Volunteer Sikhathazile Mahlangu interviews resident Samuna Gabwa. Photo by Niki Clark/American Red Cross

Guidelines:

IFRC Evaluation Framework

The IFRC Evaluation Framework guides how evaluations are “planned, managed and utilized” by the Secretariat. The Framework provides specific guidance on when to conduct a mid-term, final or real-time evaluation and re-affirms the importance of the OECD DAC criteria as guiding evaluation questions for evaluations of humanitarian assistance and international development. The Framework also provides step by step guidance on planning and managing an evaluation as well as information about disseminating evaluations in the RC/RC movement through the evaluation database. Click here.

IFRC Project/programme monitoring and evaluation (M&E) guide

Kaufman Foundation Guidance On Evaluation Reports

This data visualization tool on presenting evaluation reports was developed in collaboration with Stephanie Evergreen and represents a practical how to note on how to make more concise, effective reports. It is the antidote to the TLDR (Too Long Didn’t Read) problem that Stephanie Evergreen has made famous at evaluation conferences and in blog posts and guides the user to take an approach aligned with the 1-4-25 page suggested format for reports (one-page handout, four-page executive summary and 25-page full report with annexes). Click here.

Management Response Plan (MRP) Guidance Note

Tools:

Indaba Participatory Video Toolkit

International Services Evaluation Design Framework Tool

The International Services Evaluation Design Framework is a tool to provide a frame for developing evaluation questions related to the OECD criteria. This tool provides a common learning framework for International Services to ensure that evaluations collect evidence that is comparable across countries and provides global insights and trends. The tool also helps stakeholders and decision makers to prioritize evaluation questions to that the studies can deep dive into specific learning areas and provide meaningful findings and recommendations. Click here.

K. Kellogg’s Foundation Evaluation Toolkit

Management Response Plan Template

Most Significant Change (MSC)

Terms Of Reference Template

USAID Evaluation Toolkit

General Resources:

ALNAP Website

Better Evaluation's Manager's Guide to Evaluation

American Evaluation Association Website And AEA’s Guiding Principles

The American Evaluation Association is a professional association of evaluators devoted to the application and exploration of program evaluation, personnel evaluation, technology, and many other forms of evaluation. The AEA website is an excellent resource for foundational materials on the profession including AEA’s guiding principles, guidance one evaluator competencies as well as home to blog postings on many evaluation topics. Click here.

IFRC Library of evaluations

OECD DAC Criteria For Evaluation

The OECD DAC Network on Evaluation Development developed a set of criteria or higher-level evaluation questions in 1991 and they have played an influential role in the practice of evaluation design in the humanitarian and international development context since then. As of December 2019, the criteria have been updated to include an additional criterion on the coherence of the intervention. They are also meant to be normative and influence the nature of humanitarian interventions: according to OECD DAC, all interventions should be relevant to the context, coherent with other interventions, achieve their objectives, deliver results in an efficient way, and have positive impacts that last. Click here.

The Evaluation Center At Western Michigan University

How to Evaluate a Humanitarian Project

Step 1 - Plan and Design Evaluation based on organizational and donor standards

For American Red Cross projects:

Standard 9: All projects must be evaluated using a common evaluation framework, or justification documented as to why not.

For American Red Cross, evaluations should be designed with technical support from headquarters and in collaboration with an Evaluation Advisory Group.

For humanitarian projects funded by USAID/BHA:

USAID requires every project to be evaluated. Bureau for Humanitarian Affairs (BHA) requires an evaluation for any project that is 18 months or longer. Additionally, BHA requires the evaluation of every project in a country where American Red Cross has previously implemented a BHA award but not yet conducted an evaluation.

Step 2 - Do data collection and analysis, and prepare and present results

End-line data should be collected to assess the changes at the end of the project. It should be collected as per the MEL Plan and then documented. Evaluations often go beyond end-line data to include qualitative in-depth analysis of the project. Results should be shared both in writing and verbally at the draft stage for feedback from the project team. Final results should be delivered in a way that is accessible to all stakeholders. In keeping with accountability standards, learning should be shared affected populations involved in the project.

Step 3 - Act on Findings and Recommendations

All evaluations should be followed by a Management Response Plan (MRP) that is tracked by the evaluation manager. The MRP should include the organization's response to findings and recommendations. The MRP should also include concrete actions to response to evaluation recommendations, as well as the team or person responsible and a timeline.

Step 4 - Disseminate learning and evaluation findings

A dissemination plan for Evaluation reports and findings is an important part of any Evaluation and the initial plan should be outlined during the development of the Terms of Reference. Your dissemination plan should focus on maximizing the utilization of the findings across a wide-range of decision-makers.

Dissemination plans can include plans for sharing written products as well as verbal presentations or workshops to communicate Evaluation results. Make sure that your learning review and evaluation reports are concise and accessible in presentation. Consider using multimedia communications tools such as podcasts, PowerPoint slide decks, infographics, and summary documents, if relevant.

Knowledge Management & Learning at the Center of Evaluation

Learning is inherent to the Evaluation phase. One way for a humanitarian organization to incorporate learning from evaluation into the project cycle is to revisit prior After Action Review (AAR) reports to curate lessons when a new disaster strikes. This ensures a cycle of continuous improvement in operations. Likewise, evaluations can be used to inform new program design or to adjust existing project implementation.