Evaluation and COVID-19

The COVID-19 pandemic posed unforeseen challenges in conducting evaluations and providing evaluative services. IES however quickly adapted and invested resources into developing innovative methods and related guidance, moving to digital responses, ensuring finalization of evaluations and meeting of accountability needs of stakeholders and Member States. 

GUIDANCE NOTE FOR MANAGERS AND EVALUATORS

The COVID-19 Guidance Note for Managers and Evaluators contains a decision matrix that shows managers on how to adjust to COVID-19 in ongoing and planned evaluations. Moreover, there is guidance for evaluators on how to tailor evaluation methodologies, ensuring that no one is left behind.  See some essential details below from the Guidance Note. 

CONSIDERATIONS WHEN PLANNING AN EVALUATION

a)  ensure a do no harm approach; b) adhere to UNEG Ethical Guidelines for Evaluation; c) avoid overburdening and/or off-loading work onto field officesd) safeguard the quality standards of evaluative work including representation of all stakeholder groups, leaving no one behind; e) only be undertaken when there is a clear plan for utility and the needs for the evaluation results outweigh the risks and costs associated. 
For more information, see the Evaluation Planning Decision Tree (p. 3) in the Guidance Note

CONSIDERATIONS FOR ONGOING EVALUATIONS

a) communicate on the information needs and any changes from the original ToR based on COVID-19; b) share any relevant data on COVID-19 responses and or impact to the programme; c) remain open to innovative data collection tools and methods; d) actively engage with IES and the evaluation team; e) ensure an evaluation design and final report that meet the evidence needs; f) ensure adequate representation, leaving no one behind. 

For more information, see the Evaluation Management Decision Tree (p. 5) in the Guidance Note 

POTENTIAL RISKS FOR EVALUATIONS

a) bias against under-represented groups; b) lower quantity and quality of data; c) inability to conduct on-site-data collection; d) limited access to field sites; e) limited availability of national evaluators; f) limited availability of field office staff and counterparts; g) health-related risks; h) hesitance to novel methods

For more information, see Table 1 on Risks and Mitigation Measures (p. 7) in the Guidance Note 

REMOTE DATA COLLECTION OPTIONS

a) on-line interviews per e.g. Skype, Microsoft Teams etcor per phone; b) surveys via mobile phone, email, online tools; c) on-line discussion platforms; d) video calls; e) remote observation via on-line feed or video footage; f) web scrapping; g) web search data analysis; h) crowd sourcing.

For more information, see Table 2 on Evaluation Data Collection Options, including limitations and requirements (p. 9) in the Guidance Note 

IES RESPONSE TO COVID-19

As a response to COVID-19, IES adapted its approaches and identified solutions to supplement data collection and analysis to increase the utility of evaluation results. IES also invested heavily in enhancing remote data collection methods and in new technology to ensure continued robust methodology in all UNODC evaluations. See IES Response to COVID-19 Crisis (Respuesta de IES COVID-19). 

In June 2020, IES also held two webinars for managers and project staff involved in evaluations on "Planning and undertaking evaluations in UNODC during the COVID-19 crisis". Read more in IES Web Story: Evaluation in the Time of COVID-19.

GUIDANCE OF OTHER EVALUATION FUNCTIONS