1. Selection of Evaluators

In order to promote transparency and ensure the maximum objectivity of evaluations, UNODC relies on external evaluators selected on the basis of their competence, independence and integrity.

Evaluators are selected by means of a transparent process involving IEU and the Project Manager.

In all cases, IEU reviews the curricula vitae of the proposed candidates against the criteria set down in the Terms of Reference and may:

(a) Have no objection to the recommended candidates;

(b) Suggest candidates other than those recommended; or

(c) Request new candidates.

2. Competencies

Competencies are identified in the ToR by the Project Manager, in consultation with IEU.

To avoid conflict of interest and undue pressure, evaluations ought to be conducted by evaluators who have had no prior involvement in the design or implementation of the project or programme to be evaluated. These are usually independent consultants, external to UNODC.

Evaluators must have no vested interest and have the full freedom to conduct impartially their evaluative work, without potential negative effects on their career development. They must be able to express their opinion in a free manner.

3. Conflict of Interest

When selecting the evaluation team, the Project Manager and IEU must ensure that there is no conflict of interest. A Declaration of interest (see Chapter III tools) should be signed by the Evaluator(s).

The following potential sources of conflicts of interest could be identified:

a) Conflict of interest due to past engagement

IEU will not assign consultants to the evaluation of projects or programmes in which they have had prior involvement in the design, implementation, decision-making or financing stages. Typical examples of prior involvement include the inception, formulation, appraisal, supervision, support mission, or any other design or support activity for projects, programmes, corporate processes or policies to be examined by the evaluation.

b) Conflict of interest due to potential future involvement

IEU also applies a ceiling to the percentage of work that a consultant can perform for UNODC. In general, IEU will not recruit consultants with an UNODC work history that exceeds 25 per cent of their total work history. In addition, when consultants are recruited through a firm (reimbursable loan), a ceiling of 35 per cent of the overall total work history will be applied to the firm or institution in question. Further restrictions apply according to the task to be performed and are explained below.

Work history refers to professional experience, including consultancies, employment by UNODC as a staff member, as temporary staff or the equivalent.

c) Conflict of interest due to involvement in multiple assignments

Some restrictions are also placed on concurrent and future employment of IEU consultants: they are to have no parallel assignments within UNODC during IEU contract period, and they should agree not to work with the division or department concerned by the evaluation for a period of six months after the expiration of IEU contract.

Other potential sources of conflict of interest that are not covered by the above provisions should be assessed on a case-by-case basis by the evaluation officers concerned in consultation with their supervisor [3].

If a conflict of interest is uncovered or arises during the evaluation, the organization should determine whether the evaluator should be dismissed or the evaluation terminated.

4. Composition

The composition of the evaluation team is identified in the ToR by the Project Manager, in consultation with IEU.

The composition of the evaluation team should be gender balanced, geographically diverse and include professionals from the countries or regions concerned [4].

Getting the gender balance right in an evaluation team is important as female evaluators may have easier access to women and can encourage them to participate in discussions and express their views. However, even more important than the sex of the evaluators is their knowledge and commitment to gender issues. It is better to have on the team a gender-sensitive man who has received training on gender issues than a gender-insensitive woman.

An evaluation team consists of a team leader, and depending on the size and complexity of the project evaluated, of inter-disciplinary national and international team members with evaluation expertise. When needed, a translator could be part of the evaluation team.

For small-scale projects, one evaluator supported by the Project Manager might be able to undertake the whole evaluation on his or her own.

IEU generally recommends at least two evaluators per evaluation, one international and one national where they have complementary skills and abilities. However, this cannot always be accommodated. Given the scope of most Independent Project Evaluations, the evaluation can only be conducted by one evaluator with both technical and evaluation expertise. In this case, although technical expertise as regards the subject evaluated is important, priority should be given to a consultant with evaluation expertise. In addition, there should be distinction on whether the evaluator is a national or international recruit.

There are many benefits to hiring a national versus an international and vice versa. The rationale for hiring a national evaluator is that s/he allows for competencies that perhaps an international evaluator may not have (e.g. knowledge of local culture, fluency in local dialects, understanding of social norms, etc). Furthermore, involving national evaluators helps to strengthen the national evaluation capacity. Conversely, the rationale for hiring an international evaluator is that s/he may have more experience in specific evaluation tools and methods which would be of great benefit to any evaluation allowing for perhaps increased reliability of the findings and better learning, etc.

5. Roles and Responsibilities

The evaluation team leader acts as the primary liaison with the Project Manager in the respective Units and Sections at Headquarters and Field Offices or with the Evaluation Manager in IEU. The evaluation team leader bears the primary responsibility for the evaluation, coordinates the input provided by the various team members and ensures the timely undertaking of the evaluation and a smooth evaluation process. This requires technical expertise, evaluation skills and experience, as well as good interpersonal, management, facilitation, writing and presentation skills. The specific tasks of the team leader ought to be reflected in the Terms of Reference.

Each team member is responsible for a certain part of the evaluation exercise, including the report writing. Usually the following tasks are shared: conducting some of the data collection and drafting parts of the evaluation report, including findings, conclusions and recommendations. The ToR ought to provide a detailed description of the qualifications required for each team member and the deliverables expected.

6. Qualifications and Skills

The qualifications and skills of the evaluation team are identified in the ToR by the Project Manager, in consultation with IEU.

Independent evaluation consultants are selected for their evaluation and technical skills and experience and their local or country knowledge, depending on the specificity of each project. They must also have an understanding and experience of evaluation concepts, techniques and ethics, and be able to work as part of an international team. Whenever possible, local consultants should also be included since they are familiar with the local context and speak the local language. The involvement of local consultants also contributes towards building local evaluation capacity.

When a team of several consultants is needed, the team leader plays a key role in getting the team to work together in a smooth and efficient way. Besides having relevant evaluation and technical skills the team leader should therefore have effective management, interpersonal, facilitation, writing and presentation skills, as the team leader's working style may also affect the acceptance of evaluation results.

7. Joint Evaluation

There are several ways to approach the selection of evaluators for a Joint Evaluation.

In some cases, the approach taken to the selection of evaluators may need to correspond to the funding modality.

_________________________________

[2] IFAD Evaluation Manual

[3] IFAD Evaluation Manual

[4] UNEG Norms and Standards, 2005

[5] UNDP Evaluation Handbook

________________________________________________________