Combining Evidence Generation and Technical Support for Effective Systems Transformation

Type(s) of RFE services: Direct Data Collection, Path to Scale, Technical Assistance
Main partner(s): Transforming Education in Cocoa Communities (TRECC), Côte d’Ivoire
Timeline: 2016-2021
One-sentence description: IPA’s Right-Fit Evidence Unit (RFE) provides data and evidence to the Transforming Education in Cocoa Communities (TRECC) initiative to support decision-making around scaling projects. In parallel, RFE supports project implementers in successfully creating monitoring, evaluation, and learning (MEL) systems that allow real-time learning and can be used to constantly improve the effectiveness of their programs.

The Challenge

The Transforming Education in Cocoa Communities (TRECC) initiative aims to improve the living conditions of all children and youth in Côte d’Ivoire through quality education. TRECC adapts successful interventions —focusing on primary education and early childhood development (ECD)—to the Côte d’Ivoire context and helps them move towards scale.

Interventions supported by TRECC go through a series of iterations as they work towards scale, starting with a pilot phase and aiming to be eventually adopted and implemented by the national government. Hence, it is critical for TRECC and its partners to approach their work with a willingness to apply the lessons learned as they test their programs in the field. High-quality MEL systems are key to collecting those lessons. It is also crucial that the MEL systems are appropriately tuned to answer the questions relevant for each partner's phase in the path to scale. These questions may include: Does the project respond to the needs in Côte d’Ivoire? Is the implementer delivering the project at the expected level of quality? Are effective systems in place to guarantee the project’s sustainability beyond the initial phases? Is the government interested and have enough resources to eventually scale up the project? When is it appropriate to bring a project from the pilot phase to scale? What indicators should we monitor to maximize learning and accountability at each phase?

The Engagement

RFE supports TRECC in response to its need for a learning coach to help its partners successfully carry out their interventions at scale, and its need for a critical friend to help evaluate partners’ progress in that path. Therefore, the engagement is composed of two main activities:

1. Providing MEL technical assistance to implementing partners and government bodies, so the they can produce and use data-based evidence to improve their interventions. During four years of collaboration, the RFE team participated in all TRECC projects’ operational committees, bringing together all stakeholders, emphasizing the importance of a learning-oriented approach and providing support to achieve it. In addition, we conducted activities to strengthen each organization’s MEL-related capacity. As each program has its unique theory of change and set of priority learning questions, RFE tailored the support to each organization.
2. Conducting independent data collection to provide reliable evidence to inform programmatic and financing decisions by the Government and the cocoa and chocolate companies co-financing the intervention alongside the Jacobs Foundation and Bernard van Leer Foundation. To this end, RFE relies on the "path to scale" as the guiding framework for decision-making. The main advantage of this framework is that it encourages innovation by providing a clear process for programs to be iteratively tested and proven before being scaled. In addition, the process allows the time and evidence necessary for government engagement and ownership.

In the first years of the collaboration, we worked mostly on the independent data collection piece as requested by donors. Since 2020, we switched our focus to providing MEL technical support to the organizations and the government, to accompany the scale-up of the most promising interventions.

The Results

Over the past four years of collaboration, we observed a major change in the use of data and evidence to make decisions or improve the programs.

Feedback from donors—cocoa and chocolate companies, the Jacobs Foundation, and the Bernard van Leer Foundation—shows that the “path to scale” framework and the independent evaluations helped them make informed decisions about projects’ next steps: “We used IPA’s independent evaluation reports to decide whether to extend or terminate projects. It is a concrete example of the utility and relevance of IPA support.” Helen Keller International and the Ministry of Health of Côte d’Ivoire piloted a parenting education program. The independent evaluation of RFE showed mixed results: the intervention improved parents’ knowledge on several good parenting practices, but there were challenges with home visits and the early stimulation component of the curriculum. Therefore, RFE recommended an adjustment phase. This new iteration not only allowed greater learning and improvement along the way, but it also gave sufficient time for the Ministry of Health to take more ownership of the intervention.

For the implementing partners, working with IPA meant accurate and reliable insights to continuously improve projects’ management. Indeed, independent data collection from IPA’s experienced team about several implementation process aspects (for example, classroom observations, participation in training, etc.) led to course correction and fine-tuning in the implementation. After each data collection, the data is used to produce a report whose findings are shared and discussed with all stakeholders to find explanations and solutions to the observed issues. In one case, IPA’s independent data revealed the existence of bad relationships between a teacher responsible for a bridging class and his direct supervisor. The latter was new and did not know much about the pedagogical approach applied in the bridging class. The concerted solution was to give limited pedagogical roles to supervisors during their initial year in the project. This simple solution ensures good collaboration between rural animators and directors in all bridge classes, regardless of the experience with the project.

Finally, technical assistance from IPA’s local RFE team allowed partners to have better and higher-quality data and helped reinforce its MEL capacity and culture. The technical support from IPA ranges from workshop facilitation for the creation of MEL plans that align with the CART principles (Credibility, Actionability, Responsibility, and Transportability), to advice on MEL activities through regular check-in meetings. This support not only ensures partners focus on actionable indicators which can help decision-making, but also empowers them gradually to operationalize the MEL plan themselves. That is the reason one interlocutor likened IPA to a swimming coach: “IPA is like a swimming coach. They help us in keeping our line and following the direction and maintaining the pace we set at the beginning” – Project Manager, TRECC implementing partner.

Overall, implementers were very satisfied with the support provided by IPA. On a one-to-ten scale, 75 percent of implementers gave 9 or 10 scores on the question “How likely is it that you would recommend Right-Fit Evidence services to a friend or colleague?” This translates to “world class” performance (achieved by organizations such as Apple or Google) on the benchmark “Net Promoter Score” rating. The MEL officers and the managers reported that RFE helped them to improve their MEL systems (in particular, the quality of the data they collect) and increase the interest of the project managers in this data. Governmental partners—which were later included in the process—also acknowledged the quality of the workshops led by IPA.

TRECC partners, led by the Jacobs Foundation, and the Government of Côte d’Ivoire are preparing to strengthen their partnerships to scale the most successful solutions through government systems. Feedback shows that partners are keen on continuing to leverage IPA expertise to help the Government succeed in the scaling process.