Evaluation Handbook

I. A. Evaluation and related Concepts

 

This section describes what evaluation is and what it is not (evaluation vs. monitoring, inspection, audit, review and research), what the evaluation guiding principles are and where evaluation stands in the project/cycle management.

1. What Evaluation is

Evaluation is the systematic and objective assessment of an on-going or completed project, programme or policy, its design, implementation and results. The aim is to determine the relevance and fulfilment of objectives, efficiency, effectiveness, impact and sustainability [1].

2. What Evaluation is not


a) Evaluation versus Monitoring

Monitoring is a continuing function that uses systematic collection of data on specified indicators to provide management and the main stakeholders of an ongoing project or programme with indications of the extent of progress towards achievement of goals and objectives.
Monitoring is Evaluation is
An internal function carried out by PM An external and independent function carried out by IEU and external consultants
An objective exercise An objective exercise
A continuous exercise A single exercise
A measurement of progress A measurement of results

The importance of monitoring and its interactions with evaluations are addressed in: Chapter II, Section A, paragraph 3.

b) Evaluation versus Inspection, Audit, Review and Research

Like monitoring and evaluation, inspection, audit, review and research functions are oversight activities, but they each have a distinct focus and role and should not be confused with monitoring and evaluation.

Inspection is a general examination of an organizational unit, issue or practice to ascertain the extent it adheres to normative standards, good practices or other criteria and to make recommendations for improvement or corrective action. It is often performed when there is a perceived risk of non-compliance.

Audit is an assessment of the adequacy of management controls to ensure the economical and efficient use of resources; the safeguarding of assets; the reliability of financial and other information; the compliance with regulations, rules and established policies; the effectiveness of risk management; and the adequacy of organizational structures, systems and processes. Evaluation is more closely linked to Managing for Development Results and learning, while audit mainly focuses on compliance.

Reviews, such as rapid assessments and peer reviews, are distinct from evaluation and more closely associated with monitoring. They are periodic or ad hoc, often light assessments of the performance of an initiative and do not apply the due process of evaluation or rigor in methodology. Reviews tend to emphasize operational issues. Unlike evaluations conducted by independent evaluators, reviews are often conducted by those internal to the subject or the commissioning organization.

Research is a systematic examination completed to develop or contribute to knowledge of a particular topic. Research can often feed information into evaluations, other assessments or decision making processes.

3. What the Evaluation Guiding Principles are

Based on the Norms and Standards for Evaluation in the United Nations System published by the United Nations Evaluation Group [2] the guiding principles of any evaluation are the following:

-     Independence, objectivity and impartiality to give credibility and legitimacy to evidence-based evaluation results;
- Utility-focused evaluations to guarantee ownership by addressing the needs of stakeholders in projects and programmes;
- Transparency and consultation. Participation enhances learning, since the reflection of different interests, needs and perceptions empowers all parties involved;
- Confidence. Evaluators must respect the rights of institutions and individuals to provide information in confidence and to verify statements attributed to them;
- Timely incorporation of findings, recommendations and lessons into the policies, strategies and decision-making processes of UNODC;
- Professional integrity of evaluators as well as in evaluation procedures, highlighting any limitations or uncertainties of interpretation within the evaluation,
- Evaluation must be sensitive to the beliefs and customs of local social and cultural environments and must be conducted legally and with due regard to the welfare of those involved in the evaluation, as well as those affected by its findings,
- In line with the Universal Declaration of Human Rights, evaluators must be sensitive to and address issues of discrimination and gender inequality. Principles of Human Rights and Gender Equality apply to the composition of the evaluation team as well as to the lens through which data is being analysed [3],
- Evaluation should not reflect personal interests. Evaluators are required to disclose any potential conflict of interest and familiarise themselves with the UNEG Code of Conduct for Evaluation in the UN System. Before undertaking evaluation work within the UN system [4], each evaluator completes a declaration of interest form (please see Chapter III, Section C and D) which should be attached to the Terms of Reference of the evaluator,
- Where evaluators find evidence of wrong-doing or unethical conduct, they are obliged to report it to the proper oversight authority.

4. Where Evaluation stands in the Project-cycle Management

Programmes and projects are planned and implemented following a sequence that begins with the identification of ideas in consultation with stakeholders. Project Manager and relevant Offices and Units then formulate the project/programme document. The project document is then reviewed against specific quality criteria for approval either by the Programme Review Committee (PRC) or by the Field Representative (direct approval process).

The role of the approval process is to ensure quality, appropriateness, coherence and feasibility of UNODC operational activities, in conformity with UNODC mandate, policies, procedural requirements, rules and regulations as well as with geographic and thematic strategies. Once approved, projects/programmes are implemented, monitored and evaluated.

The project/programme cycle includes the following stages in which evaluation plays a role:

a) Strategy Setting - Identification Phase

The identification of projects/programmes should be based on research and assessments, evaluations, and internal high-level consensus.

b) Project/Programme Development - Design and Approval Phase

Project/Programme Managers plan and budget for an evaluation into the design of projects/programmes in order to (i) objectively measure success of a project/programme, (ii) judge if work is going in the right direction and (iii) plan future efforts for improvement.

To ensure proper funding of evaluation activities, projects and programmes build the resource requirements in their funding agreements with the donors and in their budgets.

Project Managers should consider the evaluation quality criteria for design prepared by IEU and available in the Chapter 1 Tools.

Also, Project Managers should consider using evaluation recommendations and lessons learned for the design of new strategies, programmes and projects.

Before the project or programme has started, an evaluability assessment may take place as well. Such an assessment is an early review of projects/programmes to ascertain that results are verifiable and to decide whether the project/programme could be implemented as such, modified or even stopped. Please see Chapter II, Section B for further details on evaluability assessments.

c) Resource Mobilisation

The resource mobilisation activity comprises the identification and reservation of programme and project funding, in cooperation with potential donors. It is a cross-organizational multi-phased activity that begins at the identification stage and extends through the formulation, the implementation and the evaluation stages where negotiation with potential donors and multilateral funds could take place.

Evaluation results may serve as a fund raising instrument to assure donors of UNODC adequate delivery.

d) Implementation

During the project implementation phase, a monitoring system is in place, performance indicators are used and baselines are set up to ensure evaluability of the projects and programmes. These are pre-requisites to evaluations.

Depending on the size and budget of the project, mid-term and final evaluations can be performed. Mid-term evaluations are performed half-way through the implementation of a project/programme. Mid-term evaluations are used for reflection and potential corrective action(s). They provide important findings and recommendations that should be considered for further improvement of the project/programme's implementation.

Final evaluations are conducted shortly before the end of a project or programme. When projects/programmes are already closed there is often no capacity within UNODC to conduct the evaluation. Final evaluations are therefore to be undertaken during the life span of a project or programme and should take place before filling in the Project Completion Report.

In conclusion, as evaluation is integral part of the project cycle:

-     Evaluation should be championed by Senior Management and understood as an institutional responsibility;
- Evaluation findings should be systematically communicated and used to improve the institution's performance (at all levels). Cluster Evaluations and Regional Programme Evaluations are particularly informative of the institution's performance in a particular thematic area or in a particular region;
- Appropriate funds have to be set aside;
- Evaluation should be carried out systematically for all projects and programmes.

For further information on mid-term and final evaluations please see Chapter I, Section C.

_________________________________

[1] Additional evaluation criteria could be used depending on the evaluation purpose: please see Chapter I, Section D.

[2] Please see UNEG website: http://www.uneval.org/normsandstandards/index.jsp?doc_cat_source_id=4

[3] Please see UNEG website: http://www.uneval.org/papersandpubs/documentdetail.jsp?doc_id=980

[4] Please see UNEG website: http://www.unevaluation.org/unegcodeofconduct

_________________________________________________________

 
IEU Home
Evaluation Handbook Home
Table of Contents
Acronyms
Foreword
Introduction
Chapter I: Defining Core Concepts
Chapter I Tools:
Chapter II: Planning an Evaluation at the Design Stage
Chapter III: Managing an Independent Project Evaluation
Chapter IV Undertaking an In-depth Evaluation
Chapter V: Undertaking a Participatory Self-Evaluation
Chapter VI: Using the Evaluation
Annexes
Annex I: Evaluation Glossary
Annex II: UNEG Norms
Annex III: UNEG Standards
Download as pdf
Chapter I: Defining Core Concepts
Feedback
Feedback