Evaluation Handbook

I. C. Types of Evaluation, Part I

 

Different types of evaluation are undertaken within UNODC. Staff in Headquarters, as well as in the Field Offices, manage, in coordination with IEU, Independent Project Evaluations and Cluster Evaluations, while IEU leads and manages In-depth Evaluations. Participatory Self-Evaluations are entirely undertaken by Project Managers. Joint evaluations within the United Nations System are also gaining in importance as means of improving coordination and harmonization as well as assessing the effectiveness of the United Nations.

1. Evaluation managed by Project Managers

All UNODC projects require an evaluation in relation to their own objectives. Whether an Independent Project Evaluation or a Participatory Self-Evaluation is undertaken depends on the criteria listed below and on the purpose of the evaluation. Cluster Evaluations can also be envisaged under certain conditions. IEU should be consulted prior to initiating any of the above mentioned evaluations.

a) Independent Project Evaluations

An Independent Project Evaluation is an evaluation of an individual project designed to achieve specific objectives within specified resources, in an adopted time span and following an established plan of action, often within the framework of a broader programme. The basis of evaluation should be built into the project document. Independent Project Evaluations are managed by Project Managers, guided and quality controlled by IEU and undertaken by external evaluators.

i. Criteria

Independent Project Evaluations are required for all projects. The only exceptions that can be made are set out in paragraph below with Participatory Self-Evaluations. However, the following types of projects must undergo an Independent Project Evaluation:

-    projects of a pilot nature;
- projects of particular relevance (e.g. innovation and extraordinary success promising to deliver lessons learned and best practices);
- projects at risk;
- projects with an excessive duration (50 % longer than originally planned);
- projects for which Member States or donors requested an Independent Project Evaluation.

ii. Roles and Responsibilities

Independent Project Evaluations are initiated and managed by Project Managers, and conducted by independent external evaluator(s). Independent Project Evaluations must be conducted in accordance with UNODC evaluation guidelines and templates, as well as UNEG Norms and Standards

IEU does not directly participate in or undertake Independent Project Evaluations. Its role is one of quality assurance and support of Project Managers throughout the process. IEU clears (i) the TOR, (ii) the evaluation methodology, (iii) the selection of evaluation consultants, and (iv) the draft/final reports, before posting on UNODC website.

For further information on roles and responsibilities please see Chapter II, Section F.

IEU provides a systematic quality assessment of Independent Evaluation reports based on pre-established criteria. The template is designed for IEU Evaluation Officers, as a tool for quality oversight of project-level evaluations ( for further information please see Chapter IV, Section F).

iii. Timing

Depending on the evaluation purpose, Independent Project Evaluations are undertaken at mid-term of the project (mid-term evaluations) and/or shortly before the end of the project (final evaluations).

Mid-term evaluations aim at improving future performance of the project. While final evaluations aim at determining the extent to which objectives were achieved and at contributing to future programming, policy making and overall organizational learning.

Projects lasting four years or more must undergo at the very least a mid-term evaluation after two years and a final evaluation .

A mid-term evaluation is an evaluation performed half-way through the implementation of a project or programme with the aim to assess whether the performance indicators and targets have been reached and whether sufficient progress has been made for the objectives to be achieved on time and within the available resources, and it recommends corrective measures if needed. It can also be conducted as part of a larger evaluation initiative.

A final evaluation is an evaluation performed shortly before the end of a project (or of a project's phase) to ensure evaluation capacity. The aim is to determine the extent to which planned and unplanned objectives and outcomes were achieved, to identify the factors of success or failure, to assess the sustainability of the benefits generated, and to draw conclusions that may inform future programming, policy making and overall organizational learning.

iv. Planning

Independent Project Evaluations need to be initiated 4 to 6 months in advance, depending on the size of the project. Evaluations are to be completed during the life span of the project to ensure appropriate evaluation capacity and to avoid project extensions.

Please see Chapter III, Section A for further guidance on the overall duration of an evaluation process.

v. Budget

In order to guarantee sufficient funds for Independent Project Evaluations, a minimum of US$ 30,000 (including transport, DSA and fee) should be reserved from the project budget and allocated under the budget line 5700 at the design stage.

An amount below US$ 30,000 requires IEU consultation and approval.

It is understood that evaluation costs vary depending on the complexity and the scope of the project, the evaluation team composition (number of evaluators, national or international consultants) and the field missions to be undertaken.

vi. Follow-up

Project Managers are responsible for tracking the implementation of evaluation recommendations by filling in the Evaluation Follow-up Plan (EFP) within one month after issuance of the evaluation report Project Managers, and alternatively, their supervisors, ensure monitoring of the progress of the EFP and update it regularly.

Then, IEU reports annually to the Executive Director and to UNODC management on the implementation of evaluation recommendations.

Please refer to Chapter IV, Section C. for further information on EFPs and follow-up to evaluation recommendations.

The planning and management of Independent Project Evaluations is presented in greater detail in Chapters II and III which provide step-by-step guidance.

b) Cluster Evaluations

A cluster evaluation is an evaluation of a set of related projects which aims, i.a., at identifying commonalities and synergies across projects as well as determining the progress made towards the programming objective, e.g. subprogramme objective of a regional programme.

Cluster Evaluations look across a group of projects to identify commonalities and synergies that together take on greater significance.

i. Criteria

Whenever meaningful, projects should be aggregated by theme, sub-programme or any other relevant unit of analysis to ensure evaluation cohesion within the evaluation portfolio.

ii. Roles and Responsibilities

Cluster Evaluations are managed by Project Managers but are conducted by independent external evaluator(s), under guidance of and in cooperation with IEU.

Cluster Evaluations could qualify as In-depth Evaluations upon consultation with IEU. In this case, IEU takes the lead in the evaluation

2. Participatory Self-Evaluations (PSE)

A Participatory Self-Evaluation is an evaluation carried out by (i) those who are entrusted with the design and delivery of a project and (ii) those who are directly involved in or benefitting from the project.

a) Criteria

Participatory Self-Evaluations are recommended for projects that started after 30 June 2010 with an overall budget below US$ 1,000,000 or for projects that started before 30 June 2010 with an overall budget below US$ 500,000. Exceptions to this rule can be made if an Independent Project Evaluation is required by a Member State or a donor or if the project has a pilot character.

Project Managers are responsible for selecting the appropriate evaluation modality and for consulting with IEU in this regard.

Summary Table of Evaluation Criteria

Starting Date Overall Budget Type of Evaluation
before 30 June 2010   <  US$ 500,000 Participatory Self-Evaluation
before 30 June 2010 = or > US$ 500,000 Independent Project Evaluation
after 30 June 2010 < US$ 1,000,000 Participatory Self-Evaluation
after 30 June 2010 = or > US$ 1,000,000   Independent Project Evaluation

For further information on PSE, please see Chapter V.

_________________________________________________________

 
IEU Home
Evaluation Handbook Home
Table of Contents
Acronyms
Foreword
Introduction
Chapter I: Defining Core Concepts
Chapter I Tools:
Chapter II: Planning an Evaluation at the Design Stage
Chapter III: Managing an Independent Project Evaluation
Chapter IV: Undertaking an In-depth Evaluation
Chapter V: Undertaking a Participatory Self-Evaluation
Chapter VI: Using the Evaluation
Annexes
Annex I: Evaluation Glossary
Annex II: UNEG Norms
Annex III: UNEG Standards
Download as pdf
Chapter I: Defining Core Concepts
Feedback
Feedback