Evaluation works by making comparisons. A common approach is to
compare how clients’ lives change over time, for example by collecting
data on various results of interest when they begin working with a
paralegal (known as baseline data), after they receive documentation,
and ideally some time later, when important outcomes linked to legal
status may have occurred, such as acquiring land titles or enrolling in
school. This approach requires the collection of data at multiple points in
time, via surveys or other structured data collection processes. Take time
during the program design phase to think through how and when you
will go about collecting data, and how you will evaluate it. This is vital for
understanding how and whether your program works.
If a less formal approach to monitoring and evaluation is adopted,
it is still important to think carefully about how you can take stock of
and review your work periodically. Channeling lessons learned about
performance and impact back into program planning is an essential
element of the program cycle, and must involve the communities your
program serves and bring their feedback on board.
CASE STUDY
ADRA Thailand takes all possible
steps to track the number of
stateless people obtaining
Thai nationality. Project staff
regularly follow up with district
offices, stateless individuals,
and heads of villages to seek
this information. However,
ADRA does not receive direct
notifications and therefore has
to recognize the limitations of
its data.
Some programs have dedicated monitoring and evaluation staff who
perform this task, while others ask paralegals to do it. Academics and
external impact evaluation organizations .such as Innovations for Poverty
Action, Evidence in Governance and Politics, and the World Bank’s
Development Impact Evaluation can also perform this function. These
experts assist in developing research questions and evaluation strategies
and generally support more complex methods than in-house monitoring
and evaluation can. On the other hand, external partners have their own
needs. If you do not want public scrutiny of the results, you should not
partner with an entity that plans to publish academic papers or other
reports of their examination of your program.
Common methods of gathering data for monitoring and evaluations include:
•S
urveys of clients and the wider community. These can be
administered at initial consultations, in meetings with officials and
community leaders, and after clients have received services.
•F
ollow-up calls and site visits to seek clients’ opinions, observe
proceedings, and learn of ID document outcomes.
•F
ocus groups in which clients or other members of the community
answer questions about their experiences and perceptions.
• Anonymous suggestion boxes.
•Q
ualitative research projects. Such projects would typically require
outside evaluators and might examine such questions as the effects of
paralegal service on empowerment. (See the sample questions below.)
•C
ase management databases allowing real-time tracking of statistics
across a caseload, such as Salesforce, CaseBox or Wingu. See 5.3
Collecting, Managing, and Using Data for a discussion of databases.
• Comparing data gathered with outcomes among the non-client
population. This might involve comparing clients’ results with existing
population data.
DISCRIMINATION
EMPOWERMENT
SUSTAINABILITY
DATA
RELATED RESOURCES
(EXTERNAL)
RELATED SECTIONS
(WITHIN)
A C O M M U N I T Y- B A S E D P R A C T I T I O N E R ’ S G U I D E : D O C U M E N T I N G C I T I Z E N S H I P A N D OT H E R F O R M S O F L E G A L I D E N T I T Y
|
57