Evaluation Handbook

I. E. Evaluation Stakeholders

 

Clearly defined governance and management structures, including evaluation stakeholders' roles and responsibilities, are crucial for efficient evaluation. This section outlines evaluation stakeholders' roles and responsibilities.

Please refer to the matrices on roles and responsibilities in Chapter II Tools for further information.

1. UNODC Independent Evaluation Unit (IEU)

It is the responsibility of IEU to respond to the commitment of the United Nations Evaluation Group (UNEG) in professionalizing the evaluation function and promoting an evaluation culture within UNODC for the purposes of accountability and continuous learning.

The role of IEU in the evaluation process depends on the type of evaluation.

Role of IEU by Type of Evaluation:

The role of IEU in In-depth Evaluations is one of leader. IEU staff is responsible for managing and conducting In-Depth Evaluations in collaboration with external independent evaluators or respective external firms.

The role of IEU in relation to Independent Project Evaluations and Cluster Evaluations is one of quality assurance and support throughout the evaluation process. IEU does not directly participate in or undertake Independent Project Evaluations.

For Participatory Self-Evaluations, IEU provides guidance and clearance at the major stages of the process.

IEU provides guidance on any activity within UNODC that relates to evaluation. This includes not only guidance on Participatory Self-Evaluations, Independent Project Evaluations and Cluster Evaluations, but also advisory role in the design of UNODC projects/programmes to ensure inclusion of appropriate evaluation provisions.

2. Project Managers

In the case of Independent Project Evaluations and Cluster Evaluations, Project Managers assume the day-to-day responsibility for managing the evaluation, including its funding, and serve as a central person connecting evaluation stakeholders. They also ensure compliance with UNODC Evaluation Policy and Guidelines.

IEU supports Project Managers by providing quality assurance of the evaluation process and deliverables. IEU is to be consulted to obtain guidance, methodological support and final clearance of the ToR, the selection of consultants and the final evaluation report. Please refer to the matrices on roles and responsibilities in the Chapter II Tools.

In the case of Participatory Self-Evaluation, the Project Manager is solely responsible for managing the process and assuring the quality of the evaluation deliverables. However, IEU reserves the right to provide feedback and comments as it clears the main steps of the process, including the final report before its posting on ProFi.

3. Evaluation Team

In order to promote transparency and ensure the maximum objectivity of evaluations, UNODC relies on external evaluators selected on the basis of their competence, independence and integrity. Evaluators must be selected by means of a transparent process.

To ensure a professional evaluation, evaluators are guided by the Norms and Standards for evaluation in the United Nations system produced by the United Nations Evaluation Group.

An evaluation team consists of a team leader, and depending on the size and complexity of the project to be evaluated, of inter-disciplinary national and international team members with evaluation expertise.

IEU generally recommends at least two evaluators per evaluation, one international and one national, comprising complementary skills and abilities.

For further information on the evaluation team, please refer to Chapter III, Section D.

4. Core Learning Partners (CLPs)

While maintaining independence, evaluation is carried out based on a participatory approach, which seeks the views and assessments of all stakeholders. The latter are represented in the Core Learning Partnership [17] (CLP) which IEU suggests forming.

The Core Learning Partnership (CLP) encourages a participatory evaluation approach by allowing its members to participate in and provide feedback on key steps of the evaluation process. CLP members are the key stakeholders of the subject evaluated (project, programme, policy etc.) who have an interest in the evaluation. The CLP works closely with the Project Manager and/or the IEU Evaluation Manager to guide the evaluation process.

A key lesson learned from evaluation practice is that the higher the degree of stakeholders' participation throughout the evaluation, the stronger their commitment to the process, their ownership of the subsequent evaluation results, and their willingness to implement the recommendations. Lessons can also be shared more quickly as they emerge during the evaluation itself.

Establishing a Core Learning Partnership creates a common understanding among key stakeholders regarding the evaluation purpose and approach, the future use of the evaluation, and ensures consensus on the focus to be adopted and the methodology to be used. Furthermore the formation of such a partnership helps to ensure regular, structured and formal two-way communication during an often long and complex process.

The intensive involvement of the CLP also ensures an opportunity for its members to understand the lessons emerging from the evaluation and, in some cases, to start applying those lessons even before the evaluation report is published. Once the final report is circulated, the members of the CLP may also act as advocates for the recommendations, thereby speeding up the spread of good practices.

Within the CLP, all stakeholders need to be treated with sensitivity. It is of particular importance to ensure that the voices of women and other vulnerable groups are heard and recognized during consultations.

In the case of Independent Project Evaluations the members of the CLP might be project staff, representatives of the project or programme implementers and partners, and project or programme beneficiaries.

The CLP should be established at the planning stage of the evaluation, and its members ought to be invited to comment on the draft Terms of Reference (ToR) and refine evaluation questions, if needed. The CLP provides support and insights throughout the evaluation process, as applicable. At the end of a field mission, the evaluation team could submit its preliminary findings to the members of the CLP and invite them to give feedback. It is advisable to organize a specific meeting for this purpose (in addition to a session involving all stakeholders if necessary). The discussion with the CLP allows evaluators to discuss the concerns raised in the draft report. This process also helps to avoid or at least reduce possible resistance to conclusions and recommendations. Once the data collection and analysis process is finalised members of the CLP will be invited to provide feedback on the draft evaluation report.

CLP Roles and Responsibilities:

-     Comment on ToR and refine evaluation questions, if needed.
- Propose evaluators, if willing to.
- Take note of evaluator(s) proposed and selected.
- Provide support and insights throughout the evaluation process, as applicable.
- Provide feedback on preliminary findings.
- Comment on draft report.
- Take note of the final report.
- Attend presentation of results (if possible).
- Agree on implementation of recommendations.

In the case where there is already an existing mechanism and structure (e.g. a steering group, programme or project board or thematic group, tripartite review) to ensure an adequate level of engagement and ownership by stakeholders and partners, members of such boards and additional key stakeholders for a particular evaluation can constitute the CLP. As long as an existing structure allows for an adequate level of stakeholder participation throughout the evaluation process, there is no need to create a new structure.

If such a structure does not exist, a mapping exercise should be carried out to identify key stakeholders for a particular evaluation. In crisis settings, a formal functional structure is unlikely to exist. When creating one in such circumstances, it is important to ensure representation is balanced, so that one particular group of people will not be dominant in the structure, which can heighten existing tensions amongst different groups of people or individuals.

5. Other Stakeholders

Other evaluation stakeholders beyond the CLP may still be involved as key informants in the evaluation process as deemed necessary.

Evaluation Stakeholders in the UNODC context include staff from relevant Units and Sections of UNODC Headquarters and Field Offices, beneficiary Governments, implementing partners, donors, project beneficiaries and other actors involved in activities related to the programme or project evaluated (Non-Governmental Organisations, United Nations organizations, other international organizations, research institutes etc.), as appropriate.

6. Evaluation Focal Points in the Field

In order to strengthen UNODC evaluation culture, ensure adequate evaluation servicing and reduce distances between Headquarters and Field Offices, IEU partners with Field Offices.

With the aim to responding in a timely and efficient manner to evaluation needs in each region, IEU is supporting an evaluation network of evaluation focal point in each Regional Office. Evaluation focal points are a critical link as they act as intermediaries between IEU and Project Managers in the Field.

Evaluation focal points are expected to have an overview of regional evaluation activities and ensure smooth evaluation processes. They liaise on a regular basis with IEU, collect queries related to evaluation from Project Managers, advise Project Managers as regards evaluation processes, collect and follow-up on evaluation plans implementation, identify evaluation capacity in the region, assist in creating a roster of evaluators in the region.

Their role is particularly relevant in Independent Project Evaluations with the long-term view of decentralizing them.

7. Observers

Any party to the evaluation may nominate observers to the evaluation team as long as observation does not interfere with evaluation activities and does not have an effect on evaluation activities' outcomes. The expenditures for such participation are borne by the observer.

_________________________________

[17] Core Learning Partnerships are already operating successfully in many UN agencies, including the United Nations Children's Fund (UNICEF) and the International Fund for Agricultural Development (IFAD).


 
IEU Home
Evaluation Handbook Home
Table of Contents
Acronyms
Foreword
Introduction
Chapter I: Defining Core Concepts
Chapter I Tools:
Chapter II: Planning an Evaluation at the Design Stage
Chapter III: Managing an Independent Project Evaluation
Chapter IV: Undertaking an In-depth Evaluation
Chapter V: Undertaking a Participatory Self-Evaluation
Chapter VI: Using the Evaluation
Annexes
Annex I: Evaluation Glossary
Annex II: UNEG Norms
Annex III: UNEG Standards
Download as pdf
Chapter I: Defining Core Concepts
Feedback
Feedback