Abstract

How can we improve students' educational achievement? Can radio instruction successfully support distance learning, particularly in settings such as the COVID-19 pandemic? In rural Peru, researchers measured whether remote support from educational coaches for caregivers during MateWasi — a radio program with interactive math lessons for preschool-aged children— impacted their children’s math learning outcomes. Results show that remote support increased the involvement of caregivers in their children’s educational development. In turn, children’s math learning improved by 0.12 standard deviations on an index that measured outcomes including oral counting, spatial ability, comparing of quantities, and solving word problems related to addition and subtraction.

Policy Issue 

At the peak of the COVID-19 pandemic in April 2020, nearly 1.6 billion K-12 students worldwide were participating in remote learning. By October 2021, 64 countries had either fully or partially closed schools.[1] School closures had a significant impact on learning development for students, particularly in low- and middle-income countries (LMICs). It is estimated that students from these countries are between nine and fifteen months behind where they would have been without the pandemic.[2]

Evidence suggests that radio instructional programs that support teachers in the classroom have the potential to increase learning outcomes — even amid school closures.[3] Evidence also suggests that remote coaching to caregivers to become involved in their children’s education has positive effects on learning outcomes.[4] Furthermore, Ministries of Education may not usually have interventions to address learning loss that occurs during summer vacations. Consequently, remote coaching to caregivers to become involved in their children’s education constitutes an area of opportunity for generating positive effects on learning outcomes. This evaluation connected these issues by evaluating whether remote coaching for caregivers during a radio-based math instruction program during the summer impacts math learning outcomes for preschool children in Peru.

Context of the Evaluation 

In March 2020, schools in Peru transitioned to remote learning for the 2020 academic year. This lasted through the 2021 academic year,[5] leading Peru to have one of the longest periods of school closures in the world.[6] To address the academic challenges facing students during the school closures, IPA and the Inter-American Development Bank (IDB) — with the technical support of Peru’s Ministry of Education and financial support of Old Dart Foundation (ODF) — developed the “MateWasi - Aprendiendo en Familia (Learning as a Family)” program. The program was built on the success of the Tikichuela program in Paraguay, which uses interactive radio instruction as a means to improve science learning outcomes for students.

To set the foundations for future learning, MateWasi was designed for the preschool age. It was implemented during summer vacation (January to March 2021) in Peru’s San Martin region where radio instruction became widespread during pandemic-related school closures, and where the levels of math learning outcomes are among the lowest in the country. For ten weeks, forty 15-minute mathematics lessons were broadcast on public radio to prepare the children for first grade. These lessons included concepts on counting; simple math operations with symbols like addition; measuring; and understanding, recognizing, and drawing different shapes.

Details of the Intervention 

Researchers measured whether remote support from educational coaches to caregivers during the “MateWasi - Aprendiendo en Familia” mathematics radio program impacted their children’s math learning outcomes.

Caregivers received weekly calls and text messages from education coaches who guided them through caregiving topics and teaching activities related to math to do with their children. These caregivers also received a package of materials with educational worksheets and guides to supplement the radio lessons. Because MateWasi was broadcast on public radio, all households could access and listen to the lessons.

A total of 1,065 households with at least one child of preschool age (4 to 6) participated in the intervention. The households were divided randomly into the following groups:

• MateWasi + remote coaching: 533 households had a caregiver receive remote coaching from education coaches during the MateWasi program.
• Comparison group: 532 households did not have a caregiver receive remote coaching during the MateWasi program.

Researchers conducted a baseline survey and a final survey ten days after the completion of the intervention. In addition, researchers used the Early Grade Mathematical Assessment and Measuring Early Learning Quality and Outcomes tests to measure learning outcomes for the children.

Results and Policy Lessons 

Remote coaching increased the involvement of caregivers in their children’s educational development. This in turn improved children’s math learning by 0.12 standard deviations on an index that measured outcomes including oral counting, spatial ability, comparing of quantities, and solving word problems related to addition and subtraction. The magnitude of learning improvement is consistent with positive outcomes from similar interventions in other settings.

Coaching effects on caregiver involvement: Remote coaching increased the probability of caregivers engaging in math-related activities with their children by 13 percentage points (a 17 percent increase over caregivers who did not receive coaching). Remote coaching also increased the number of days per week a caregiver spent doing math-related activities with their children from 3.4 days per week to nearly 4 days per week (a 17 percent increase over caregivers who did not receive remote coaching).

Remote coaching increased the likelihood of caregivers playing mathematics games with their children by 24 percentage points. It also increased the probability of caregivers tutoring their children by nine percentage points and tutoring their children in math-specific subjects by twelve percentage points.

Learning effects based on caregiver education level: The impacts on child learning were primarily driven by caregivers with less education than caregivers with higher education. This could be because remote coaching provided support for caregivers who were less likely to provide at-home educational resources for their kids.

Learning effects based on child’s gender: Caregiver involvement in MateWasi had greater impacts on boys than on girls. The program increased the likelihood of participating in mathematics activities by 21 percentage points for boys compared to 5 percentage points for girls. In addition, caregivers of boys were more likely to be engaged in their children’s learning than caregivers of girls.

Sources

[1] Muñoz-Najar, Alberto, Alison Gilberto, Amer Hasan, Cristóbal Cobo, Joao Pedro Azevedo, and Maryam Akmal. "Remote Learning during COVID-19: Lessons from Today." Principles for Tomorrow (2021).

[2] McKinsey Staff, “How COVID-19 caused a global learning crisis,” McKinsey & Company, April 4, 2022, https://www.mckinsey.com/industries/education/our-insights/how-covid-19-caused-a-global-learning-crisis

[3] Crawfurd, Lee, David K. Evans, Susannah Hares, and Justin Sandefur. Teaching and testing by phone in a pandemic. No. 591. Center for Global Development, 2021.

Lichand, Guilherme, Carlos Alberto Dória, Onício Leal Neto, and João Cossi. "The impacts of remote learning in secondary education: Evidence from Brazil during the pandemic." (2021).

Angrist, Noam, Peter Bergman, and Moitshepi Matsheng. School’s out: Experimental evidence on limiting learning loss using “low-tech” in a pandemic. No. w28205. National Bureau of Economic Research, 2020.

[4] Barrera, Oscar, Karen Macours, Patrick Premand, and Renos Vakis. "Texting parents about early child development." (2020).

Berlinski, Samuel, Matias Busso, Taryn Dinkelman, and Claudia Martínez. Reducing parent-school information gaps and improving education outcomes: Evidence from high-frequency text messages. No. w28581. National Bureau of Economic Research, 2021.

[5] Näslund-Hadley, Emma, Juan Manuel Hernández Agramonte, Carolina Méndez, and Fernando Fernandez. "Remote caregiver Coaching in Preschool Mathematics: Evidence from Peru." (2022).

[6] Simeon Tegel, “As Peru opens from pandemic, nearly all schools remain closed,” AlJazeera, October 29, 2021, https://www.aljazeera.com/news/2021/10/29/as-peru-opens-from-pandemic-nearly-all-schools-remain-closed