Abstract

COVID-19 forced students across the world to move from in-person to remote schooling, presenting particularly acute challenges for children in low- and middle-income countries living in households with limited resources and internet access. Researchers worked with IPA, the Inter-American Development Bank (IDB), and Costa Rica’s Ministry of Public Education to develop and evaluate a text message campaign for parents designed to support learning for preschool-aged students during the shift to remote instruction. After 15 weeks, children in households who received the campaign performed better on cognitive assessments, an effect driven by increased parental involvement in children’s education.

Policy Issue

School closures resulting from COVID-19 forced students across the world to move suddenly from in-person to remote learning. This shift had especially stark consequences for very young children in low- and middle-income countries. Students of all ages in these settings often live in households with limited resources and internet connectivity, and very young students also require a large amount of support from parents to access and use resources for remote learning. Evidence suggests that these households  have also faced acute economic and psychological challenges during the pandemic.^[1] As a result, interventions focused on parental support and well-being may be effective strategies to improve student learning during the pandemic. Some prior research has found online strategies for supporting student learning during the pandemic to be effective in both high-income^[2] and low- and middle-income^[3] countries, but less is known about how to support learning on very young students.

Context of the Evaluation

Costa Rica closed its schools in response to COVID-19 on March 12, 2020. Soon afterward, the Ministry of Public Education launched a remote learning program called Aprendo en Casa (AeC), which aimed to share educational content with the largest proportion of households possible through a combination of print and web materials and television and radio broadcasts. Teachers generally communicated with parents through phone calls and WhatsApp groups.

According to data from this study’s initial survey, 98 percent of teachers established these communication channels with families; 68 percent reported being able to communicate with all students’ families, while a further 25 percent communicated with over half. Parents faced several challenges: 57 percent reported that they needed more help supporting their children’s remote education, while 73 percent of households reported losing at least part of their income and 65 percent reporting a lost job. Approximately 89 percent of parents in the initial survey presented at least one symptom of mental health challenges.

Details of the Intervention

Researchers worked with IPA, the Inter-American Development Bank (IDB), and the Ministry of Public Education to develop and conduct a randomized evaluation of a text message program designed to support student learning in households with four- and five-year-old preschool students in Costa Rica.

The text message campaign consisted of a series of 3-4 weekly messages sent over a span of 15 weeks. The messages suggested simple numeracy and literacy activities that parents could do with their children, encouragement to carry out these activities, and weekly motivational messages. To address the psychological toll of the pandemic and provide a positive learning environment, the campaign also included messages about positive parenting, time management, and healthy habits.

To evaluate the impacts of the campaign on student learning and household well-being, the research team conducted a two-stage randomized evaluation:

• In the first stage, 691 participating parental networks on WhatsApp were randomly assigned either to an intervention group that received the text messages (338 networks), or a comparison group that continued with “business as usual” (353 networks).
• In the second stage, parents within the intervention group were randomly assigned either to receive the text messages or not. This allowed researchers to measure whether parents who did not receive the messages, but were in networks with those who did, experienced any of the impacts of the messages indirectly by comparing them with those in networks where nobody received them. 

Researchers conducted initial surveys at the beginning of the intervention, and follow-up surveys after 15 weeks at its conclusion. To measure student learning during the pandemic, the research team adapted learning assessments that measure students’ cognitive skills typically carried out in person into a version that could be conducted over the phone (Hernández-Agramonte et al, 2021).^[4] Researchers also collected two rounds of survey data on parental investment in education and household characteristics with online surveys.

Results and Policy Lessons

After 15 weeks, students in households that received the text message campaign demonstrated higher cognitive skills than those that did not, due to increased parental involvement in education in these households. The program was cost-effective, delivering promising results for education policy in low- and middle-income countries during school disruptions.

Student learning outcomes: Students in households receiving the text message campaign had scores on cognitive assessments that were 0.11-0.12 standard deviations higher than those in the comparison group. The improvements were mostly in numeracy skills like counting, adding, and subtracting.

Parental involvement: The improvements in student learning were driven by increased parental engagement with their children’s education. Parents who received the text messages had 0.24 standard deviations more involvement in the activities recommended by the campaign. Parents who received the campaign also demonstrated an increased amount of belief in the abilities of their children. They did not, however, report an increased belief in their own abilities, nor a higher engagement with the government remote education program, suggesting that the result was mainly driven by increased involvement.

Demand for the program: The campaign was popular, with 94 percent of parents who received it saying they were interested in the text messages continuing. Parents who received the text messages were also more likely to support continuing the AeC distance learning program.

Within-network impacts: In networks where some households received the text messages, but others did not, the benefits of the messages did not extend to the children that did not receive them. Survey data suggests that households tended not to share the messages with teachers or one another.

Cost-effectiveness: The program was highly cost-effective, producing on average a 0.01 standard deviation increase in learning in a student for every US$0.08 spent—a considerably lower cost than other interventions aiming to increase early numeracy skills in Latin America.^[5]

Sources