Programmes and projects need to be designed with evaluation in mind as evaluation is an essential part of an intervention’s life cycle. See also Evaluation in the Project/Programme Cycle. When formulating a new project/programme or a revision, the Project/Programme Manager should: 1) Review evaluation results from previous interventions in order to inform the design of the new programme/project; 2) Review evaluation plans in order to ensure coordination and coherence of evaluation activities; and, 3) Carefully plan the timing of the evaluation and ensure sufficient funding is reserved. See also Evaluation Step by Step.
Moreover, during this phase it is necessary to commit capacity, time and budget to ensure a Human Rights (HR) and Gender Equality (GE)–responsive evaluation process. Mainstreaming Human Rights and Gender Equality, UNEG Guidance - Integrating Human Rights and Gender in Evaluation. See also Human Rights and Gender Equality
All programme and project documents, including any revisions, are required to include a section on evaluation. The Independent Evaluation Section (IES) reviews and clears all documents in the Direct Approval process as well as provides comments for the UNODC Programme Review Committee (PRC). See also Evaluation in the Project/Programme Cycle.
To avoid last minute changes, pre-consultation with IES is recommended well before the project document/project revision is submitted for clearance.The following details must be included in the paragraph on evaluation (Template for the evaluation paragraph in project documents; (Spanish); Template for the evaluation paragraph in project revisions; (Spanish)): 1) Type of evaluation; 2) quarter and year to initiate the evaluation; 3) budget (calculated with the Evaluation budget matrix) 4) use of evaluation results; 6) prior evaluations; and, 7) whether a cluster evaluation is planned. See Guidelines for Cluster Evaluation (requires pre-consultation with IES).
The administrative process starts 8 months prior to the start of an Independent Project Evaluation (IPE) (Flowchart Independent Project Evaluation) and 10-12 months of an In-depth Evaluation (IDE) (Flowchart In-depth Evaluation). The Manager 1) informs IES and the key stakeholders of the evaluation; 2) discusses the modality and timing of the evaluation with IES; 3) confirms that pre-reserved funds are available; and, 4) initiates the IPE in the on-line evaluation management application Unite Evaluations. See Guidance on how to log in and initiate the evaluation. (IES initiates an IDE).
See also Evaluation during Covid-19 Crisis (Respuesta de IES) and Covid-19 Guidance for Managers and Evaluators
After consultation with IES, the Manager drafts the ToR and submits through
Unite Evaluations. Guidelines
Evaluation ToR, Template Evaluation
ToR
The ToR outline the purpose and the key questions to be answered; time frame and deliverables; as well as stakeholders and the use of the evaluation results.
Evaluation Handbook (Guide for project/programme managers and evaluators);
Evaluation Handbook (Practical guide to carry out high quality evaluations)
The Manager identifies qualified candidates for the evaluation team in line with guidance by IES, with IES reviewing and clearing any proposed candidates before the recruitment can be initiated. An IPE includes a minimum of one lead evaluator with extensive evaluation expertise and one expert in the substantive topic to be evaluated. One of the team members should also have expertise in HR and GE-responsive evaluation approaches. For an IDE, additional team members/experts are required.
Once the evaluation team is recruited, the desk review phase starts, with the Manager providing access to all desk review materials. The evaluation team reviews the materials and submits an inception report through Unite Evaluations to IES for review.
Guidelines Inception Report, Template Inception Report (Directrices para los informes de inicio).
The inception report includes e.g. the elaborated evaluation methodology with data collection tools and instruments, as well as a work plan with timelines and roles and responsibilities.
Data is collected by the evaluation team, using various tools and methods, ensuring a mixed-methods and HR and GE-responsive approach, with data often collected remotely via phone or web-based interviews, e-mail or surveys. The Manager provides logistical support with travel agendas and interview schedules, as well as any additional background materials as required.
See also Evaluation during Covid-19 Crisis (Respuesta de IES) and Covid-19 Guidance for Managers and Evaluators
After the data collection and analysis, the evaluation team drafts the evaluation report (Guidelines Evaluation Reports, Template Evaluation Report (IPE);Template Evaluation Report (IDE)). The draft report is submitted through Unite Evaluations to IES for review, followed by a review by the Manager and the key stakeholders for any factual errors.
The lead evaluator provides the final report with a 2-page summary Brief (Template Evaluation Brief (IPE); Template Evaluation Brief (IDE)) and presents the final evaluation results to the key stakeholders.
The evaluation process does not end with the receipt of the final evaluation report. In the
light of the recommendations of the final evaluation report, Programme/Project Managers discuss and liaise
with relevant parties, including senior management, other UNODC sections, donors, etc. and prepare a Management Response (MR).
The MR is included in a table in the final evaluation report, indicating whether each recommendation is accepted/partially
accepted or rejected. For an IDE, a written narrative (approximately 1–2 pages) is further added to the final report.
Managers are also responsible for the development of the Evaluation Follow-up Plan (EFP) on the implementation of the evaluation recommendations. The EFP is prepared in the on-line evaluation management application Unite Evaluations and is reviewed and cleared by IES. See Guidance for the drafting and monitoring of the Follow-up Plan. The Manager is required to update the follow-up plan once a year. IES further reports on the implementation of recommendations to the Executive Director and Member States on an annual basis.
The evaluation report is further disseminated by Programme/Project Manager for learning and improved UNODC programming. IES shares evaluation results with Member States and UNODC's Executive Director, as well as publishes Evaluation Reports and Evaluation Briefs on its website. All UNODC Evaluation Reports further undergo an external quality assessment (Quality Assessment of Independent Project and In-Depth Evaluation Reports). See the Evaluation Quality Assessment Report 2019 and the 2-page Summary.
