START UP IN DE P T H RANDOMIZED CONTROLLED TRIALS Randomized controlled trials (RCTs) involve comparing two randomly selected groups of people, one of which receives assistance (“the treatment group”) and one of which does not (“the control group”). Differences in outcomes between the groups can be attributed to the assistance itself. RCTs provide the strongest possible evidence of whether assistance works. RCTs are difficult to implement in a paralegal setting, as it is unethical to turn clients away. However, projects can do things like selecting at random which geographic localities they will serve first. If the project does not have enough staff to assist everybody who asks for help, using a lottery to determine who to help and keeping track of people who do not receive help can create a control group. Even when an RCT is not possible, understanding the logic of RCTs can help you make clearer and more convincing statements about the impacts of a project. You might ask whether clients differ from the broader population—for example, are they wealthier/poorer? More empowered/educated? Considering how your caseload differs from the community as a whole can help clarify whether the program might assist others, how it might need to be changed to assist others, and what claims you can make about its impacts with what level of certainty. 58 | Data typically distinguishes closed cases from open ones, although if the time it takes to receive documents is long, interim monitoring may be wise, both because it allows you to measure impact before cases can be closed and because it may identify problems in your approach.  ee 9.10 Closing a Case for how to determine when a case should S be closed. This is generally when clients have received documents, although some outcomes come after this. Of course, it is important to avoid bias in gathering data. Approaches to avoiding bias include diversity-sensitive questions, for example asking about the different experiences of men, boys, women, and girls, and taking into account the gender, language, status, and so on, of the evaluator and the client. See also 5.3 Collecting, Managing, and Using Data. Namati has used the following questions in order to create interviews and surveys to measure the value of their paralegal program in relation to their key goals: Individual empowerment •D  o clients feel that they have been empowered through their work with a paralegal on their legal identity case? Does this experience of empowerment differ for clients who successfully acquired an identity document versus those who did not? •W  hat, if anything, do clients report that they can do now they weren’t able to do before? •D  o clients know laws and their rights related to nationality 12 months after a case opened? •D  o clients who work with a paralegal exhibit greater confidence about rights and freedom of movement than community members who went through the application process alone? •A  re clients using newly acquired documents to seek and access services or other concrete benefits? Interviewers ask about children’s enrollment in school, freedom of movement, and access to financial services, government health services, formal employment, and voting. •W  hy didn’t people apply for documentation on their own before approaching a paralegal? •D  id paralegals mobilize people to apply for documents who say they wouldn’t have otherwise? Effectiveness of paralegal case processes • Where do clients see room for improvement in the case process? •H  ow has interacting with the government with a paralegal been different from interacting with the government without a paralegal? •W  hat aspects of working with paralegals did clients perceive as most and least empowering? A C O M M U N I T Y- B A S E D P R A C T I T I O N E R ’ S G U I D E : D O C U M E N T I N G C I T I Z E N S H I P A N D OT H E R F O R M S O F L E G A L I D E N T I T Y