See below the key steps of an evaluation at UNODC, including roles and responsibilities of IES, project/programme managers and the evaluation team.
REVISED UNODC EVALUATION POLICYSee further the website with IES Guidelines and Templates, all also included on this page, as well as the website Unite Evaluations, which is the evaluation management tool used for all phases and processes of UNODC evaluations. Learn how to initiate an evaluation in the Unite Evaluations User Manual (English; Spanish).
See also the Toolkit for Evaluating Interventions on Preventing and Countering Crime and Terrorism; Brief (2-pager).
EVAUATION AT THE DESIGN STAGE OF A PROJECT
Projects and programmes need to be designed with evaluation in mind as evaluation is an essential part of their life cycle. For more information, see the revised UNODC Evaluation Policy.
When formulating a new project/programme or a revision, Managers should: 1) Use prior evaluation results and recommendations for continuous improvement and learning; 2) Review existing evaluation plans to ensure coordination and coherence; 3) Carefully plan and update the timing of future evaluations; 4) Ensure that sufficient funding is reserved for evaluation (and available when it is time to initiate the evaluation) as calculated with the Evaluation Budget Matrix (new) (English; Spanish); and, 5) Consult with IES.
APPROVAL OF EVALUATION PLAN OF A PROJECT
Project/programme documents or revisions thereof must include an accurate and updated Evaluation Plan, which is reviewed and cleared by IES. To avoid last minute changes, pre-consultation with IES is recommended. Accurate data is further required for UNODC's annual Evaluation Plans.
The Evaluation Plan Template (English; Spanish) for IPMR and the project document requires the following details: 1) Type of evaluation (mid-term or final); 2) Quarter and year to initiate the evaluation(s); 3) Budget for the evaluation(s) as calculated with the Evaluation Budget Matrix (new) (English; Spanish). 4) Use of evaluation results; and, 6) Prior evaluations that have informed the design of the project/programme.
INITIATION
Preparatory work starts at least 7 months before the planned finalisation of an Independent Project Evaluation (IPE) and 10-12 months for an In-depth Evaluation (IDE). See the Flowchart for an IPE (new) (English; Spanish) or for an IDE.
The Manager consults with IES and initiates the IPE in Unite Evaluations (IES initiates an IDE). See Unite Evaluations User Manual (English); Spanish).
Please note: Evaluations with insufficient budget and/or unrealistic timeframe will not be undertaken. If IES has further not been informed on the revised timing of an evaluation, IES capacity may not be sufficient, whereby it may need to be postponed.
TERMS OF REFERENCE
After consultation with IES on the modality and timing of the evaluation, the Manager drafts the ToR, using the Evaluation ToR Template for an IPE (new) (English; Spanish) or for an IDE, focusing on the questions and criteria that are most useful.
The Manager submits the draft ToR through Unite Evaluations for IES' review, comments and subsequent clearance.
Please note: IES shares the cleared revised draft ToR with the identified Core Learning Partners (CLPs) for review and comments.
At this stage, the Manager also starts compiling the desk review materials.
RECRUITMENT
The Manager identifies and provides CVs of potential Substantive Experts (that meet the requirements of the ToR) to IES for review. IES provides guidance and proposes candidates for the lead Evaluation Expert.
Please note: IES clearance is required before the recruitment process may be initiated. The Manager is responsible for gathering and contracting the evaluation team.
Minimum number for an IPE: 1 lead Evaluation Expert and 1 Substantive Expert in the subject area. At least 1 in the evaluation team should have expertise in human rights and gender equality. An IDE requires additional experts.
INCEPTION REPORT
The Manager provides desk review materials and holds a kick-off meeting with the evaluation team (inviting IES).
The evaluation team conducts an extensive desk review and initial analysis, develops the evaluation methodological approach and data collection instruments and, submits a draft inception report, using the Evaluation Inception Report Template for an IPE (new); (English; Spanish) or for an IDE through Unite Evaluations for IES' review, comments and subsequent clearance.
Please note: IES clearance is required before the field mission/data collection phase may be initiated.
DATA COLLECTION
The evaluation team collects data through interviews (virtual or in person), observations, site visits, focus groups etc. as per the Inception Report. See also COVID-19 Guidance Note for Managers and Evaluators
The Manager provides logistical support, including travel arrangements, set up of interviews, independent interpretation, coordination with respective offices etc. to the evaluation team.
The evaluation team analyses and triangulates the collected data in a systematic manner and debriefs the Manager on its preliminary observations from the data collection.
EVALUATION REPORT
The evaluation team uses the Evaluation Report Template for an IPE (new (English; Spanish) or for an IDE and submits a Draft Report through Unite Evaluations for IES' review.
After IES' pre-clearance, the Manager reviews for any factual errors, followed by CLPs' review and comments.
The evaluation team submits the Final Report, a 2-page Brief, using the Evaluation Brief Template for an IPE (English; Spanish) or for an IDE and PowerPoint slides for IES' review. After IES' clearance, including of the Management Response, the lead evaluator presents the final results.
As a final step of an evaluation, the Manager discusses and liaises with relevant internal and external stakeholders (Senior Management, UNODC Offices/Sections/Units, donors etc.) in order to develop an Evaluation Follow-up Plan (EFP) and a Management Response (MR).
Please note: Completing the EFP and MR is mandatory and a pre-requisite for finalization of the evaluation process and clearance of the Final Evaluation Report by IES. For more information see Evaluation Follow-up and Use of Results at UNODC, as well as the Evaluation Follow-up Plan and Management Response Guidelines (English; Spanish) and Template (English; Spanish).
The Manager disseminates the Final Evaluation Report for learning and improved UNODC programming. IES shares evaluation results with Member States and UNODC's Executive Director, as well as publishes Evaluation Reports and Evaluation Briefs on its website. IES further commissions independent external quality assessments (EQAs) of all UNODC evaluation reports to ensure they provide credible information for evidence-based programming. See all EQA Reports and 2-page Summaries.
