Editor’s note: This is part two in a series on what IPA has learned about making sure evidence gets used, and what we hope to do more of in the future.  See part one, laying out our new strategic ambition, here.

What Nava Ashraf, Professor of Economics at the London School of Economics and Lead Academic for IGC Zambia, and others have coined “co-creation of evidence” is becoming a cornerstone of IPA’s strategy. We’re naming as a priority this collaborative process of researchers and stakeholders generating knowledge together in order to ensure evidence is applicable to real world situations. This entails an authentic commitment from all partners—researchers, implementers, policymakers—to ensure that they mutually build each other’s capacity to drive knowledge creation through field research and policy engagement.

A preschool teacher in the Quality Preschool for Ghana program instructs students.

Here are a few specific ways in which we’ve tried to pursue this genuine co-creation of research around the world.

Spend the extra time and effort to develop projects centered on policy engagement

The essence of co-creation is research that responds to policymakers’ and practitioners’ real needs. IPA encourages co-creation of research opportunities that are win-win-win situations for the researcher, the policy partner, and the funder. A typical case of co-creation is one that answers questions that are important to all these parties and creates a public good that others can leverage for greater impact. In other cases, it is the intervention itself that is negotiated and co-created.

For example, the Strengthening Accountability to Reach All Students project, a large scale evaluation assessing the impact of extra teaching targeting (targeted instruction) to catch up kids who are behind in Ghana’s public school system, was developed under the framework that all stakeholders share ownership of the project. A critical priority for all partners, including the primary investigators Adrienne M. Lucas, Sabrin Beg, and Anne Fitzpatrick, was that the project would be delivered through existing school structures, to facilitate its sustainability beyond the evaluation. During the project design phase, partners discussed the intervention components together in-depth, and identified which government division was best suited to implement each part of the project based on their existing mandates. This was all documented and agreed upon through a Terms of Reference document, which brought clarity and created a shared purpose.

The co-created video intervention informing students about the returns to education took the form of a four-part telenovela.

In another case, IPA, researchers—Christopher Neilson, Francisco Gallego, and Oswaldo Molina—and educators spent months developing a video intervention informing Peruvian students about the returns to education—what they might earn if they finished secondary school, pursued a certain career path, or dropped out. When the intervention was successful and scale up became a possibility, the Ministry was quite amenable to expanding the program, we expect especially because we developed and rolled out the videos together. Crucially, although the staff of the Ministry were no longer those who had originally developed the videos together with the researchers, the institutional trail of co-ownership outlasted the staff turnover and greatly eased the path to scale. The fact that the Ministry was an owner of the initiative—it wasn’t any one person’s pet project—helped it outlast political turnover.

The fact that the Ministry was an owner of the initiative—it wasn’t any one person’s pet project—helped it outlast political turnover.

Maintain stakeholder engagement throughout an entire research project

Co-creating evidence goes beyond the articulation of a research project or intervention. It also means keeping stakeholders engaged throughout the entire project, building their buy-in. In IPA’s experience, this increases the chances that partners continue to be interested and invested in the project, which increases the chances that they will apply the research results to their work.

For example, while working towards the evaluation and eventual scale-up of a teacher training intervention in Ghana’s preschool sector, IPA convened a QP4G Steering Committee that met quarterly. Several members of the Ghana Education Service (GES), National Teaching Council, National Nursery Teacher Training Center (NNTTC, a division under GES responsible for training early childhood teachers), and UNICEF participated in steering committee activities. These included field visit to the study’s schools, reviewing baseline and midline briefs, and providing input into the eventual scaling plan.

The QP4G evaluation, led by researchers Sharon Wolf, J. Lawrence Aber, and Jere R. Behrman, became one of the first impact evaluations in sub-Saharan Africa to show that in-service teacher training can improve preschool quality and school readiness. With IPA’s support, NNTTC is in the process of incorporating the QP4G training into their regular teacher training curriculum. (Hear more about the project from NPR.)  

Build capacity over time

Some of the most impactful work is done through building the capacity of those who make policy decisions to use rigorous evidence to inform those decisions. Capacity building can be done organically through the process of evidence co-creation. Our partners in Ghana, for example, have been leaders in Ghana’s education sector for a couple of decades, and have partnered with IPA on several research projects. Their collaboration has been invaluable for IPA: their knowledge of education in Ghana, how policymaking works, pedagogy, curriculum development, etc., informed several research projects. While academic studies typically appear in individual papers, the programs they’re studying are typically part of a broad and evolving suite of programs and policies. Understanding the research’s place in the longer term ecosystem helps it speak to what came before and what we hope will come after (and hopefully better).

While academic studies typically appear in individual papers, the programs they’re studying are typically part of a broad and evolving suite of programs and policies.

And through that collaboration, our collective knowledge about what it takes to produce rigorous evidence, how evidence can inform education policy, and specific pieces of evidence in education in Ghana grew as well. In the summer of 2018, at Ghana’s education sector annual planning meeting, one of these partners presented the evidence on targeted instruction and advocated for implementing targeted instruction, in addition to basing future education strategies on rigorous evidence. These partners have been key to the growing culture of evidence-informed policymaking in education in Ghana, and we think partnering with IPA has helped spur this growth.

Prepare partners for bad (and good) news

We talk a lot about creating win-win-win scenarios and building buy-in with our partners. However, we also acknowledge that partners who co-create evidence with IPA almost inevitably run a risk. What happens if we evaluate their program, invest a lot of time in the research process, and it turns out their program does not have the impact they thought it would? Government partners almost certainly risk political backlash (from within government or from constituents), and NGOs might alienate the funders they rely on, if they find out the programs they’ve been paying for haven’t been working.

What happens if we evaluate their program, invest a lot of time in the research process, and it turns out their program does not have the impact they thought it would?

We’ve learned over time to work up front to understand partners’ constraints and help them manage the risk that agreeing to an evaluation can bring. One way to do this is by helping them think through scenarios long before the results come out. Whether the news is good or bad—how will they learn? How will they respond? Garnering this pre-commitment is a crucial step to leveraging co-creation of evidence for policy impact, as it encourages partners to plan for impact in advance.

One great example is Sabre Education, which worked with IPA, the Ghana Education Service, and researcher Sharon Wolf to evaluate its Fast Track Transformational Teacher Training Program. Results showed that although the program functioned as intended by significantly improving student-teachers’ implementation and knowledge of the national kindergarten curriculum, these impacts faded once newly qualified teachers were posted at their new schools. These findings propelled Sabre Education to adjust their program to address the challenges teachers faced at their new schools.

As the research community continues to be rightfully pushed towards research fit for impact, co-creation needs to become more than just something that we do in name – but something we put in practice every day. We know all too well how challenging this is, but we have also seen that the extra effort helps move research papers from the shelf to the forefront of policy conversations and decisions.