DATA COLLECTION
The Evaluation Team collected primary data from November 2020–February 2021. Due to COVID-19
international travel restrictions, the team conducted data collection virtually over Zoom or Microsoft
Teams, with group facilitation assisted by other technology solutions, including SurveyMonkey, Miro, and
Google Docs. Snowball sampling and purposive sampling from document reviews was used to identify
interviewees and, during the data gathering phase, the Evaluation Team interviewed or engaged a total of
165 individuals, with the breakdown of the total interviews highlighted in Graph 1, below.
Informants represented a wide range of stakeholders,
including those who were both directly funded by PRM
and some that were linked to PRM funding. Individual
and group interviews included PRM, UNHCR, members
of the UN family, U.S. Embassy staff, Women’s Refugee
Commission (WRC), development actors, International
Non-Governmental Organizations (INGOs), and civil
society actors (CSOs). A list of the organizations
interviewed or engaged in global interviews is included in
Annex J.
Graph 1: Reach of Interviews
Dominican Republic
34
Thailand
35
Nepal
38
Global
58
The only government official interviewed was in Thailand. In all three countries, the Evaluation Team
interviewed stateless or formerly stateless persons. In addition, in December 2020 the Evaluation Team
facilitated a virtual Strengths, Weaknesses, Opportunities and Threats analysis with UNHCR’s
Statelessness Section in Geneva. The Evaluation Team also administered an (anonymous) online survey to
PRM’s Refugee Coordinators (RefCoords), which was accompanied by an online focus group discussion
(FGD).
The Evaluation Team recorded and transcribed all the evaluation primary research to ensure accuracy and
in accordance with the evaluation’s design and analysis approach. In each interview, evaluators obtained
informed consent orally. In cases when an interpreter was engaged, such as in Thailand and Nepal, the
interpreter signed a non-disclosure agreement.
Following data collection, all interview transcripts were uploaded to Dedoose (a qualitative research
software) for analysis. The Evaluation Team conducted content and thematic analysis to identify patterns,
as well as sentiment analysis to determine key perceptions and to find areas of concurrence across
interviewees. Findings were organized by EQs and are presented by EQ in the next sections of this report.
LIMITATIONS
The evaluation’s main limitations are related to: (1) conducting a fully remote and virtual evaluation; (2)
reporting bias and (3) generalizability. Despite the pivot to a fully remote evaluation, the Evaluation Team
did not face major challenges engaging CSOs or stateless people through remote means in each of the three
countries, which was raised as a concern during the Inception Phase. However, the inability to travel and
‘see’ programming in action or contexts is a limitation of a fully virtual evaluation. The Evaluation Team
had to instead rely on descriptions and explanations of programming and challenges, which is sufficient but
does not allow for participant observation. Reporting bias is a concern for evaluations that place a heavy
emphasis on consulting with implementing partners, as they can be “incentivized” to share positive
outcomes only. The potential bias was mitigated by the Evaluation Team engaging with a wide array of
organizations and stakeholders, including those outside of PRM’s scope of influence (i.e., academics,
CSOs, stateless persons, non PRM funded INGOs, etc.). Reporting bias was also limited through the
evaluation’s methodologies, including outcome harvesting and participatory workshops that focused on
building consensus, as well as through qualitative analysis coding of all interviews using a standard
approach. The limitation to the evaluation around generalizability is that the countries selected are unique
and programming strategies are not consistent across the board. The evaluation also faced some challenges
assessing bilateral diplomatic efforts, given the sensitivity of statelessness in some contexts and the inability
3