United Nations Office on Drugs and Crime

Evaluation Handbook

I. D. Evaluation Criteria

This section describes the evaluation criteria developed by the Development Assistance Committee of the Organisation for Economic Co-operation and Development (OECD-DAC). In addition to the DAC criteria, other evaluation criteria could be developed (such as design, partnerships and cooperation, and innovation) and cross-cutting issues (such as Gender and Human Rights) could be addressed in evaluations.

Depending on the size of the programme or project, the stage at which the evaluation is taking place and the time and budget available, an evaluation might focus on some of the evaluation criteria, while putting less emphasis on others. All of this is to be specified in the project or programme document ( please see Chapter II) and/or in the evaluation terms of reference (TOR) ( please see Chapter III.

Each of the criteria translates into a different set of questions to be answered by the evaluation. Please see in the Chapter I Tools: Table with evaluation criteria definitions and corresponding sample of evaluation questions.

1. DAC Criteria

Evaluation criteria are important to provide an overarching framework for an assessment and define the evaluation questions. The United Nations commonly uses and adapts the evaluation criteria of the Organization for Economic Cooperation and Development's Development Assistance Committee (OECD-DAC) to evaluate its interventions: relevance, efficiency, effectiveness, impact and sustainability.

Relevance is the extent to which the objectives of a project are continuously consistent with recipients' needs, UNODC mandate and overarching strategies and policies.

Efficiency is a measure of how resources/inputs (funds, expertise, time, etc.) are converted into outputs.

Effectiveness is the extent to which a project or programme achieves its objectives and outcomes.

Impact is the positive and negative, primary and secondary long-term economic, environmental, social change(s) produced or likely to be produced by a project, directly or indirectly, intended or unintended, after the project was implemented.

Sustainability is concerned with measuring whether the benefits of a project or programme are likely to continue after its termination. Projects need to be environmentally as well as financially sustainable.

2. Additional Criteria

Additional criteria are also commonly used to customize evaluations and adapt to the specific needs of the evaluation. Indeed, UNODC evaluations use the additional criteria of design, partnerships and cooperation, and innovation.

Design of a project or programme measures the extent to which:

- The logical framework approach was adopted, with measurable expected Performance Indicators at the country and regional levels, outcomes and outputs, performance indicators, including gender equality and human rights, targets, risks, mitigation measures and assumptions.

- An appropriate participatory needs assessment and context analysis took place.

Partnerships and cooperation is a measure of the level of UNODC cooperation with partners and implementing partners (e.g. donors, NGOs, Governments, other UN agencies etc.), through:

-	The extent to which partnerships have been sought and established, and synergies been created in the delivery of assistance
-	The extent to which there was effective coordination among partners
-	The extent to which partnerships' responsibilities were fully and effectively discharged
-	The extent to which partnerships' inputs were of quality and provided in a timely manner
-	The extent to which the project or programme contributes to the One UN, UNDAF, and other UN system-wide coordination mechanisms (e.g. participation in UN Country Team) and the extent to which UNODC participation in UN activities influences its performance.

Innovation is the extent to which a project or a programme initiates efficient and effective innovative practices.

To define additional evaluation criteria, the following guiding questions should be answered:

→	To what extent does the criterion inform the purpose of the evaluation?
→	How much and what kinds of information do potential users need?
→	Should there be equal focus on each of the criteria or will some information be more useful?
→	Is the criterion a useful or appropriate measure for the particular evaluation?
→	Which criterion will produce the most useful information given available resources?

3. Cross-cutting Issues: Gender and Human Rights

Carrying out gender equality and human rights responsive evaluations implies a shift in what to evaluate, how to evaluate, and why to evaluate.

To align with United Nations priorities and the Office of Internal Oversight Services (OIOS) recommendations [16], cross-cutting issues, such as gender equality and human rights, should also be addressed in all evaluations through:

-	Mainstreaming in the evaluation criteria ( for further information on how to mainstream gender equality and human rights into the evaluation criteria please see Chapter I Tools).
-	Mainstreaming in the evaluation process: selection of the evaluation team, participation of stakeholders, selection of the appropriate evaluation methods, collection and analysis of data, preparation and dissemination of the evaluation report ( for further information on how to mainstream gender equality and human rights into the evaluation process please see chapter II, III and IV).

_________________________________

[16] OIOS Inspection of Programme Level Monitoring and Evaluation (M&E) of the UNODC (IED-09-006, 5 October 2009) concluded that gender mainstreaming is not consistent in the results framework and absent in the evaluation policy. OIOS recommended the following regarding gender mainstreaming:

1. To ensure that subprogrammes translate gender into systematic reviews and analyses, UNODC should deploy an integrated approach to developing and incorporating gender dimensions throughout its results framework.

2. To ensure that subprogrammes incorporate gender dimensions into their M&E processes, UNODC should include in the Evaluation Policy the necessary guidance for staff to mainstream gender dimensions into M&E processes, including but not limited to data collection, analyses, reporting, and programmatic decision-making processes.

IEU Home

Evaluation Handbook Home

Table of Contents

Acronyms

Foreword

Introduction

Chapter I: Defining Core Concepts