Now and again IEU receives questions on various evaluation aspects.
Since some of these questions and related answers may be of interest as well as guidance and assistance to you, we have prepared a list of the most common ones.
Feedback is however very important to us, should you therefore think of any further issues/questions to be added, please contact us at firstname.lastname@example.org.
Reasons for Evaluation, Evaluation Criteria, Evaluation Types, Evaluation Budget, Project revision clearance by IEU, Participatory Self-Evaluation, Core Learning Partners (CLPs), Evaluation ToR, Evaluators, Evaluation Reports and Follow-up Action, Evaluation Training.
My office is successful, I attract a lot of funds and I have been hiring many new staff members, why should I still evaluate?
Without effective planning, monitoring and evaluation it is impossible to objectively measure success of a project/programme, to judge if work is going in the right direction and to plan future efforts for improvement.
Evaluation further enables continuous learning and performance improvement through the implementation of evaluation recommendations, consideration of evaluation lessons learned and their use for design of new strategies, programmes and projects.
Evaluation also assures accountability by reporting on UNODC activities to the Governing Bodies. By providing objective evidence of what UNODC has achieved and what changes have been produced using the resources provided, evaluation is aimed more specifically at accounting for the use of resources and for the results produced to all UNODC stakeholders, including staff and management, donors, Member States, target groups and implementing partners.
What are the Evaluation Criteria in UNODC?
Evaluation criteria are important to provide an overarching framework for an assessment and define the evaluation questions. The UN commonly uses and adapts the evaluation criteria of the Organization for Economic Cooperation and Development's Development Assistance Committee (OECD-DAC) to evaluate its interventions: Relevance, Efficiency, Effectiveness, Impact and Sustainability.
Additional criteria, such as Human Rights, Gender, Participation, Design, Partnerships and Cooperation or Innovation, are also commonly used to customize evaluations and adapt to the specific needs of the evaluation.
For more information, see What is Evaluation, Evaluation Handbook (Chapter 1), OECD-DAC Criteria for Evaluating Development Assistance as well as United Nations Evaluation Group (UNEG).
What type of evaluation should I conduct?
Staff in Headquarters, as well as in the Field Offices, manage Independent Project Evaluations, while IEU manages and conducts In-Depth Evaluations. Participatory Self-Evaluations are entirely undertaken by Project Managers.
Whether an Independent Project Evaluation or a Participatory Self-Evaluation is undertaken depends on the criteria and on the purpose of the evaluation. For more information, please see Criteria for Independent Project Evaluations and Participatory Self-Evaluations .
Depending on the evaluation purpose, Independent Project Evaluations are undertaken at mid-term of the project ( mid-term evaluations) and/or shortly before the end of the project ( final evaluations). Projects lasting four years or more must undergo at the very least a mid-term evaluation after two years and a final evaluation .
For more information, please see Criteria for Independent Project Evaluations and Participatory Self-Evaluations, Evaluation Step by Step and Evaluation Handbook (Chapter 1).
How much money should I allocate for evaluation and where should it be saved?
Every evaluation is different depending on the complexity and scope of the project, however, experience shows that 2-3% of the overall budget of the project is a good estimate for an Independent Project Evaluation, where a minimum of US$ 30,000, to cover transport, DSA, fees, translations, presentation to stakeholders, logistics etc, is recommended. Consultation with IEU is proposed for the individual projects.
A Participatory Self-Evaluation takes the form of either a stakeholders' response modality (requires no budget) or a workshop, with a recommended minimum of US$20,000.
Funding for evaluation costs should be reserved and allocated under the budget line 5700 at the design stage.
For more information, please see Criteria for Independent Project Evaluations and Participatory Self-Evaluations and Guidelines for Participatory Self-Evaluations
What criteria have to be included in a project submission or a project revision in terms of Evaluation to ensure the Independent Evaluation Unit will clear it?
In accordance with the new project document template in ProFi, the paragraph on evaluation should include details on the following:
of evaluation and
rationale for this choice,
usage of evaluation findings from prior evaluations,
roles and responsibilities and
budget (specify under budget line 5700 as well as in paragraph).
For more information, please see Evaluation Quality Criteria Checklist for Project Documents.
Why has the self-evaluation process become participatory?
The evaluation is conducted in a participatory manner by those who are entrusted with the design and delivery of a project and by those who are directly involved in or benefiting from the project. This means that stakeholders have to be closely involved in the process.
Project Managers use Participatory Self-Evaluations as a tool to reflect in a structured way on why the results were achieved or not and what can be done for improvement.
The purpose is twofold: learning and accountability.
For more information, please see Guidelines for Participatory Self-Evaluations.
Which forms does the PSE take to ensure participation?
Stakeholders' involvement in Participatory Self-Evaluations can take the following two forms
1. Stakeholders' written response to the Participatory Self-Evaluation report drafted by the Project Manager. Stakeholders state whether they are in agreement with the Participatory Self-Evaluation against the evaluation criteria (design, relevance, efficiency, partnerships, gender and human rights, effectiveness and sustainability).
2. Workshops where stakeholders along with the project team discuss face to face the Participatory Self-Evaluation report drafted by the Project Manager and provide a common assessment against evaluation criteria. In this case, a minimum of USD 20,000 have to be allocated to budget line 5700 at the beginning of the project, in accordance with UNODC's evaluation guidelines.
What are Core Learning Partners (CLP's)?
While maintaining independence, evaluation is carried out based on a participatory approach, which seeks the views and assessments of all evaluation parties. These parties are represented in the Core Learning Partners (CLPs), who are the key stakeholders of the subject evaluated (project, programme, policy etc.) with an interest in the evaluation. The CLPs work closely with the Project Manager and/or the Evaluation Manager to guide the evaluation process.
Evaluation Stakeholders in the UNODC context include staff from relevant Units and Sections of UNODC Headquarters and Field Offices, beneficiary Governments, implementing partners, donors, project beneficiaries and other actors involved in activities related to the programme or project evaluated (Non-Governmental Organisations, United Nations organizations, other international organizations, research institutes etc.), as appropriate
Who should participate to the elaboration of the Terms of Reference for an Evaluation?
Terms of Reference serve as a "contract" between UNODC and the Evaluator(s).
For Independent Project Evaluations, Project Managers are in charge of drafting the ToR, in cooperation with relevant Units and Sections at Headquarters and in Field Office. The ToR must be shared with all key stakeholders, i.e. "Core Learning Partners" (CLP). The Independent Evaluation Unit (IEU) offers guidance during the drafting process, and further reviews, comments and clears the ToR.
For more information, please see Guidelines for Evaluation ToR.
I cannot find an evaluation consultant with both the relevant skills in evaluation and in the subject area of the project, how can I proceed ?
It is recommended that evaluations be conducted by an evaluation team that consists of a lead consultant with expertise in evaluation and a supporting consultant with expertise in the subject area of the project to be evaluated. If an evaluation team is not feasible, it is recommended that priority be given to consultants with evaluation expertise as conducting an evaluation requires specific skills and knowledge.
If the project to be evaluated proves that there is a strong need for specific technical expertise, we leave it to the discretion of the Project Manager for final selection of the consultant and recommend that the consultant work closely with IEU so that the evaluation process and final report meet the quality and standards set by IEU as well as UNEG.
Should I opt for a national or an international evaluator?
It depends on the scope and needs of your evaluation. For example, national evaluators tend to have more knowledge of the region and can provide expertise on the local context; on the other hand, international evaluators may have more expertise in evaluation and consequently are more expensive. It is recommended that the evaluation team consists of both a national and an international consultant.
Why do I need to formulate evaluation questions before I even have an evaluator?
The evaluation questions serve to provide a framework for the evaluation to guide the evaluator. These questions allow the Project Manager and Core Learning Partners (CLPs) the opportunity to reflect on the project and to address any specific aspects of the project that they feel may be of significance or priority.
My Evaluation Report is already published on IEU's website, this means I am totally done with the evaluation process, right?
The evaluation process does not end when the evaluation report is complete. In fact, learning and active use of knowledge generated from the evaluation is the most important element of the evaluation exercise.
As part of the organization's commitment to transparency, evaluation reports are to be disseminated to relevant stakeholders and all participants of the evaluation process and a dissemination workshop may be organized.
Appropriate evaluation follow-up mechanisms are further required in order to ensure that evaluation recommendations are properly utilized and implemented in a timely fashion, and that Organization's future activities take into account the results of previous evaluations.
The Project Manager is required to fill in the Evaluation Follow-up Plan (EFP) for the Implementation of Evaluation Recommendations within one month after issurance of the final evaluation report. as well update it on a yearly basis.
The evaluation recommendations will further be added to the online recommendations tracking system and made accessible to the general public on Intranet.
For more information, please see Evaluation Handbook.
I feel like I would need to be trained in Evaluation, what should I do?
Become an Evaluation Focal Point! For more information, please see Evaluation Focal Points ToR
Visit the World Bank IPDET training website (www.ipdet.org).
Conduct a needs assessment in your region and contact us at IEU (email@example.com) to get information on the next IEU training in your region.
Speak about your needs to the Evaluation Focal Point in your office/section and try to elaborate a concept note for a training in your region - subject to availability, funding might be allocated from IEU.
Add a Question
My question is not on the list, where can I submit it?
Feedback is very important to us and should you think of any further issues/questions to be added, please contact us at firstname.lastname@example.org.