How to do rigorous impact evaluation
February 24th brought together NGOs in Ghana for a training in impact evaluation methodology. Staff from IPA prepared a workshop at the request of partners involved in an evaluation of Aflatoun - a social and financial curriculum intended to produce growth in financial knowledge, savings, and self-esteem among children ages 6 to 14. The program is being assessed in 45 schools in Ghana this year, and implementing partners SNV asked for an introduction to the theory behind the randomized controlled trial in which they are taking part.
The concept caught the interest of colleagues from an array of sectors - from those working in school meals to those addressing the effects of climate change in Ghana, and the training was expanded. In the end, IPA staff led a group of 20 participants through an introduction to the work of rigorous impact evaluation.
Presentations alternated with group work sessions allowed attendees to design their own studies and to experiment with approaches to randomizing interventions. Here were some novel research questions that came out of this session:
Does the time of day at which a school meal is provided affect education outcomes like attendance and performance (i.e., will children learn better if they eat earlier in the day, or are later meals an incentive to attend/pay attention)?
Will targeting husbands of pregnant women with information about prenatal care increase visits to clinics? If so, what is the best information to give husbands - information about health or information about the cost of care?
The response of attendees also raises the question of training design. The small number of participants who had previous contact with the workshop facilitators created a more intimate atmosphere that may have helped to make attendees feel comfortable to participate and to stay engaged throughout the day. The available context of a shared, ongoing evaluation (Aflatoun) also provided readily familiar examples that served to illuminate difficult concepts.
There is a lot of work to do in refining workshop design and making training avaiable, but the positive result in Ghana is evidence of a deep enthusiasm among local practitioners for conducting rigorous and meaningful evaluations to inform and improve their work.