Evaluation Handbook

V. C. Guidelines for Producing Participatory Self-Evaluation Reports

 

1. By obtaining Written Responses from Stakeholders

a.     The Project Manager (PM) initiates the Participatory Self-Evaluation (PSE) process;
b. PM and/or HQ focal point, if any, identifies representative(s) for each stakeholders' groups  [5];
c. PM and/or HQ focal point, if any, informs representatives identified on the PSE process and on their role (see above);
d. PM and/or HQ focal point, if any, identifies UNODC staff who should be granted view access to the PSE process (Directors, Field Representatives, project team members etc.);
e. PM and/or HQ focal point, if any, fills in the Template in ProFi or in the Chapter V Tools ( see see Step 1 boxes of the PSE Template located in the Chapter V Tools);
f. PM and/or HQ focal point, if any, submits completed PSE report to IEU via ProFi or by email;
g. IEU clears or provides comments, if any;
h. If requested, PM and/or HQ focal point, if any, revises the PSE report based on IEU comments and submits it to IEU again;
i. IEU sends the PSE report drafted by the Project Manager to the stakeholders identified;
j. The stakeholders use the PSE report as a basis for producing their own assessment of the project. They indicate whether they (dis)agree with the PSE report produced by the PM by filling in the PSE Template generated by ProFi or located in the Chapter V Tools  (see Step 2 boxes of the PSE template located in the Chapter V Tools) ;
k. The stakeholders provide their responses to IEU by email;
l. IEU receives the various responses from the stakeholders;
m. IEU posts the stakeholders' responses in ProFi to ensure confidentiality and anonymity [6];
n. Once the stakeholders have provided their responses, the PM and/or the HQ focal point, if any, drafts (i) the executive summary (without altering stakeholders' responses) and (ii) a common Self-Evaluation Follow-up Plan (SEFP) in ProFi or by filling in Step3 boxes of the PSE Template located in the Chapter V Tools. The summary and the SEFP should reflect the common assessment of management and stakeholders against the evaluation criteria.
Any strong disagreement between the Project Manager and the stakeholders in the assessment should be clearly stated in the Executive Summary to ensure transparency in the process
o.    PM requests clearance from IEU;
p. IEU clears or provides comments;
q. If requested, PM and/or HQ focal point, if any, revises the PSE report based on IEU comments and submits it to IEU again;
r . Once cleared by IEU, the final report (Executive Summary, Lessons Learned and SEFP) is automatically posted in ProFi;
s. Under the responsibility of the supervisor, PM and/or HQ focal point, if any, monitors the progress of the Self-Evaluation Follow-up Plan (SEFP) and updates it every regularly in ProFi;
t. IEU follows up on the status of implementation of the recommendations on an annual basis.

The below table provides information about the ProFi access granted to UNODC staff at various stages of the PSE process. This information relates only to the steps undertaken in the ProFi application.

STAKEHOLDERS' RESPONSE
STEP 1
REPORT STATUS: DRAFT
Create the PSE report PM only
Edit the PSE report PM and/or HQ focal point
View the PSE report IEU, PM, HQ focal point, readers, project team members
Request IEU clearance for the PSE report PM and/or HQ focal point
REPORT STATUS: SENT TO IEU
Edit the PSE report No one
View the PSE report IEU, PM, HQ focal point, readers, project team members
Send back to PM and HQ focal point with IEU comments IEU
View IEU comments IEU, PM, HQ focal point, readers, project team members
Edit IEU comments IEU
STEP 2
REPORT STATUS: CLEARED BY IEU
Edit the PSE report No one
View the PSE report IEU, PM, HQ focal point, readers, project team members
Create the PSE report for stakeholders IEU
STEP 3
REPORT STATUS: WITH PROJECT COORDINATOR FOR SUMMARY
Edit the responses No one
Edit the PSE report No one
View the responses IEU, PM, HQ focal point, readers, project team members
View the PSE report IEU, PM, HQ focal point, readers, project team members
Edit Executive Summary, lessons learned and SEFP PM and/or HQ focal point
View Executive Summary, lessons learned and SEFP IEU, PM, HQ focal point, readers, project team members
Request IEU clearance for the Executive Summary, lessons learned and SEFP PM and/or HQ focal point
Send back to PM and HQ Focal Point with IEU comments IEU
View IEU comments IEU, PM, HQ focal point, readers, project team members
Edit IEU comments IEU
REPORT STATUS: CLEARED BY IEU
View Executive Summary, lessons learned and SEFP All UNODC
Edit Executive Summary and lessons learned No one
Edit SEFP PM and/or HQ focal point, and their respective supervisors
View the whole process, including stakeholders' responses IEU, PM, HQ focal point, readers, project team members

2. By Conducting a Workshop

a.    The Project Manager (PM) initiates the PSE process;
b. PM and/or HQ focal point, if any, reviews the Guidance on conducting a PSE workshop (see below;
c. PM and/or HQ focal point, if any, identifies representative(s) for each stakeholders' groups [7];
d. PM and/or HQ focal point, if any, informs representatives identified on the PSE process and on their role (see above;
e. PM and/or HQ focal point, if any, identifies UNODC staff who should be granted view access to the PSE process (Directors, Field Representatives, project team members etc;
f. PM and/or HQ focal point, if any, fills in the Template in ProFi or in the Chapter V Tools (see Step 1 boxes of the PSE Template located in the Chapter V Tools);
g. PM and/or HQ focal point, if any, submits completed PSE report to IEU through ProFi or by email;
h. IEU clears or provides comments, if any;
i. If requested, PM and/or HQ focal point, if any, revises the PSE report based on IEU comments and submits it to IEU again;
j. PM and/or HQ focal point, if any, organizes the PSE workshop with the stakeholders' representatives to discuss the PSE report prepared in advance by him/her;
k. Following the workshop, PM and/or the HQ focal point, if any, drafts (i) the executive summary (without altering stakeholders' responses) and (ii) a common Self-Evaluation Follow-up Plan (SEFP) in ProFi or by filling in Step 3 boxes of the PSE Template located in the Chapter V Tools. The summary and the SEFP should reflect the common assessment of management and stakeholders against the evaluation criteria.
Any strong disagreement between the Project Manager and the stakeholders in the assessment should be clearly stated in the Executive Summary to ensure transparency in the process.
l.     Within one month following the workshop, PM and/or the HQ focal point, if any, requests IEU clearance;
m. IEU clears or provides comments;
n. If requested, PM and/or HQ focal point, if any, revises the PSE report based on IEU comments and submits it to IEU again;
o. Once cleared by IEU, the final report (Executive Summary, Lessons Learned and SEFP) is automatically posted in ProFi;
p. Under the responsibility of the supervisor, PM and/or HQ focal point, if any, monitors the progress of the Self-Evaluation Follow-up Plan (SEFP) and updates it regularly in ProFi;
q. IEU follows up on the status of implementation of the recommendations on an annual basis.

The below table provides information about the ProFi access granted to UNODC staff at various stages of the PSE process. This information relates only to the steps undertaken in the ProFi application.

WORKSHOP
STEP 1
REPORT STATUS: DRAFT
Create the PSE report PM only
Edit the PSE report PM and/or HQ focal point
View the PSE report IEU, PM, HQ focal point, readers, project team members
Request IEU clearance for the PSE report PM and/or HQ focal point
REPORT STATUS: SENT TO IEU
Edit the PSE report No one
View the PSE report IEU, PM, HQ focal point, readers, project team members
Send back to PM and HQ focal point with IEU comments IEU
View IEU comments IEU, PM, HQ focal point, readers, project team members
Edit IEU comments IEU
STEP 2
REPORT STATUS: APPROVED FOR WORKSHOP
Edit the PSE report No one
View the PSE report IEU, PM, HQ focal point, readers, project team members
STEP 3
REPORT STATUS: WITH PROJECT COORDINATOR FOR SUMMARY
Edit Executive Summary, lessons learned and SEFP PM and/or HQ focal point
View Executive Summary, lessons learned and SEFP IEU, PM, HQ focal point, readers, project team members
Request IEU clearance for the Executive Summary, lessons learned and SEFP PM and/or HQ focal point
Send back to PM and HQ Focal Point with IEU comments IEU
View IEU comments IEU, PM, HQ focal point, readers, project team members
Edit IEU comments IEU
REPORT STATUS: CLEARED BY IEU
View Executive Summary, lessons learned and SEFP All UNODC
Edit Executive Summary and lessons learned No one
Edit SEFP PM and/or HQ focal point, and their respective supervisors

Guidance on Conducting a Workshop

Purpose of the Workshop

The purpose of such a workshop is to collect stakeholders' views and to jointly assess the performance of the project against evaluation criteria (design, relevance, efficiency, partnerships and coordination, gender and human rights, effectiveness and sustainability). The output of the workshop is the Participatory Self-Evaluation report reflecting management and stakeholders common understanding of the project's achievements against expected objectives contained in the project document.

Planning

The Participatory Self-Evaluation should be planned at least three months in advance. The main steps of the process are the collection of the relevant data, the consultation with stakeholders, the drafting of the Participatory Self-Evaluation report and the organisation of the workshop.

A participatory workshop requires a substantial time commitment, especially its preparation phase, which involves consultation with stakeholders and identification of their representatives who will participate in the process.

When planning a participatory workshop, it is vital that stakeholders are consulted early on and their role in the process discussed.

Invitations, along with the Self-Evaluation report drafted by the Project Manager, should be issued at least a month in advance to ensure availability of participants and representation of stakeholders. Project Managers may be bound by stakeholders' time availability.

It is essential that participants have enough time to read the report drafted by the Project Manager before the workshop. Also, it is recommended to obtain initial feedback on the draft report before the formal workshop begins in order to ensure constructive discussions during the workshop.

Budgeting

It should be kept in mind that a minimum of USD 20,000 should be reserved for the organisation of the participatory workshop. The size of this event would vary depending on the size of the project. However, the Organization would have to account for some typical items in the budgeting:

-     venue;
- participants' travel and DSA;
- materials/stationery;
- translation services;
- refreshments;
- any other logistics costs;
- facilitator's fee, if any.

In case the budget is not available to gather all stakeholders in the same venue, opinions of stakeholders could still be heard by recording their views in small workshops in their respective countries or through phone interviews and questionnaires.

Participants

No more than 25 participants per workshop should be invited in order to allow for constructive discussions. Participants could be divided into smaller groups for discussions if deemed necessary.

Facilitating

The Project Manager should ensure that discussions are relevant, participatory and constructive, and that outputs (Participatory Self-Evaluation report and self-evaluation follow-up plan) are useful.

_________________________________

[5] Stakeholders' groups are those that can contribute to and would benefit from coordinating/drafting the stakeholders' response to the self-evaluation report (e.g. UNODC staff, target groups, counterparts, donors, executing agencies, partner organizations, including other UN agencies involved, NGOs, research institutions, universities etc.). In Independent Project Evaluations, this group is also referred as Core Learning Partners.

[6] To ensure confidentiality and anonymity, only IEU, the Project Coordinator and the project team can view the stakeholders' responses. In addition, only IEU can see from whom the stakeholders' responses come from. Indeed, the information provided should not be traced back to stakeholders.

[7] Stakeholders' groups are those that can contribute to and would benefit from coordinating/drafting the stakeholders' response to the self-evaluation report (e.g. UNODC staff, target groups, counterparts, donors, executing agencies, partner organizations, including other UN agencies involved, NGOs, research institutions, universities etc.).

________________________________________________________

 

 
IEU Home
Evaluation Handbook Home
Table of Contents
Acronyms
Foreword
Introduction
Chapter I: Defining Core Concepts
Chapter II: Planning an Evaluation at the Design Stage
Chapter III: Managing an Independent Project Evaluation

Chapter IV: Undertaking an In-depth Evaluation

Chapter V: Undertaking a Participatory Self-Evaluation

Chapter V Tools:
Chapter VI: Using the Evaluation
Annexes
Annex I: Evaluation Glossary
Annex II: UNEG Norms
Annex III: UNEG Standards
Download as pdf
Chapter V: Undertaking a  Participatory Self-Evaluation
Feedback
Feedback