EVALUATION OF UNHCR-LED INITIATIVES TO END STATELESSNESS – FINAL REPORT – VOLUME I Collection Method Coverage and participation Notes Global and regional key informant interviews 27 HQ staff 26 regional bureau staff 18 external partners and experts Semi-structured interviews based on key evaluation questions, follow-up consultations with many key stakeholders Interim report Interim data collection report covering global and regional data analysis Interim report presented to Evaluation Reference Group and group of internal stakeholders for discussion and feedback Country and regional case studies 4 country case studies, 3 regional case studies. Total of 56 external partners and experts, and 43 country office staff Critical moments reflection workshops with 4 country offices to identify activities, outcomes and influencing factors; followed by individual and group interviews and document review. Regional cases based on interviews and document review with regional staff and partners, selected country office staff Regional engagement groups (REG) REGs for Africa, Asia and MENA comprised of diverse set of external stakeholders and UNHCR regional staff. 31 total participants Two sessions held for each REG in Africa and Asia to consider key regional drivers of statelessness, activities, successes and lessons learned. MENA REGs included group discussions with civil society organizations and sub-regional groups of government actors Sensemaking and regional validation workshops Consolidated case study data and global preliminary findings. Regional consolidation of case data and impressions Three sensemaking sessions held with the Evaluation Service and Statelessness Section to discuss global data and preliminary findings. 4 regional validation workshops held to present data and impressions to UNHCR regional and country staff 19. The evaluation team has coordinated its work with the team leading the concurrent PRM evaluation from Resonance. Based on a confidentiality agreement signed between Resonance and UNHCR, the two evaluation teams have shared some data and preliminary analysis and conducted a few joint interviews to minimize duplication for key informants. 20. A more detailed explanation of the methodology is included in Annex 3. The full list of stakeholders consulted by the evaluation team is contained in Annex 6 and the bibliography is in Annex 7. An explanation of the data sets used for the financial and RBM system analysis is contained in Annex 8. Financial and RBM data used in this analysis comes from the UNHCR Focus system. Budget and expenditure figures were used from the UNHCR Management System Renewal Project (MSRP) financial system to ensure the most updated source. Financial data covers 2012 to 2019 inclusive, as prior to 2012 detailed figures were not broken down by Pillar. RBM data covers the period 2015-2019, as prior to 2015 some key variables for the analysis were not available. Key considerations and limitations 21. The evaluation followed the UNEG Code of Conduct for Evaluation 12, the UNEG Norms and Standards for Evaluation13 including standard 3.2 on ethics, and the UNEG Ethical Guidelines for Evaluations.14 The ethical principles in the UNEG guidelines set forth obligations for evaluators including independence, impartiality, credibility, conflict of interest, honesty and integrity, and accountability. The principles also include obligations to participants related to respect for dignity and diversity, the rights of individuals or groups in the evaluation, confidentiality and avoidance of harm. During the inception phase, the evaluation team assessed the ethical risks that could arise as a result of switching to remote forms of data collection as well as potential risks in the context of Covid-19. Following an assessment by the Global Ethics Committee, the overall methodology was endorsed but it was decided that the potential ethical risks outweighed the benefits of conducting consultations with persons of concern, where even remote interviews could place people at risk since most would need to travel to a central location for connectivity. As such, these consultations did not occur. To minimize this gap, the evaluation interviewed civil society groups that serve and advocate for these populations, including some stateless staff and members, given the value of their perspectives from working directly with persons of concern, and as a 12 13 14 United Nations Evaluation Group, “UNEG Code of Conduct for Evaluation in the UN System”, March 2008. United Nations Evaluation Group, “UNEG Norms and Standards for Evaluation”, June 2016. United Nations Evaluation Group, “UNEG Ethical Guidelines for Evaluation”, March 2008. UNHCR / May, 2021 7