Evaluating Remedial, Inquiry-Based Science Education in Lima, Peru
Abstract
When young learners are performing below average in school, how can we close the learning gap? In this study, researchers partnered with Universidad Cayetano Heredia to design and evaluate the impact of a program aimed at helping low-performing students—mostly from low-income urban households—master basic skills in science. Through an innovative, remedial, inquiry-based approach, in which students engage in hands-on practical work, tutors and students met after school once a week for sixteen weeks during the school year. Despite low levels of participation, the program overall significantly improved students’ test scores by .12 standard deviations, although gains were all concentrated among boys.
Policy Issue
Context of the Evaluation
Peruvian students consistently underperform in math and science. In the Second Regional Comparative Education Study (SERCE), for instance, Peruvian students scored below the Latin American average in both subjects3. In third grade mathematics, more than half of the student population was below basic proficiency levels4. In the 2012 application of the Program for International Student Assessment (PISA), Peruvian students ranked last, among the 65 participating countries, in both math and natural science5. About a third of all participating Peruvian students placed in the lowest proficiency level in science, meaning they had not mastered even the most basic skills. These assessments suggest that Peruvian students lack critical abilities, such as analyzing and synthesizing information, applying new knowledge in real-life settings, and reasoning.
In an effort to improve science teaching and learning, Peru’s Ministry of Education has been collaborating with the Universidad Cayetano Heredia since 2010 on diverse initiatives to identify the best solutions for comprehensive K-College reform.
Universidad Cayetano Heredia, the main implementation partner, has a long track record developing and implementing educational initiatives, and is invested in pursuing rigorous research to help improve learning and achievement in Peruvian schools.
Details of the Intervention
To test if remedial, inquiry-based, science education—taught by trained science tutors—improves students’ science test-score performance, researchers carried out a randomized evaluation in which half of the lowest performing students in participating schools in metropolitan Lima received a remedial science program. Specifically, the program targeted third-grade children whose prior science achievement placed them in the bottom half of their school. Approximately 1,200 students in 48 randomly chosen public schools in Lima participated in the study.
In the spring of 2014, third grade students from selected schools took a baseline diagnostic science test. Of those who placed in the lower half of the school distribution, half were randomly selected to receive tutoring through a lottery that stratified by school and gender. The remaining half of the lower-scoring students continued with business-as-usual school activities.
The first stage of the program consisted of tutor recruitment and pre-service training. Prior to the start of the school year, Universidad Cayetano Heredia selected potential tutors—preferably women that could serve as role models for girls—who were either science schoolteachers or college students majoring in pedagogy. The tutors participated in a 20-hour introductory training course.
The second stage of the program was the remedial instruction itself. Tutoring groups were composed of five to six students per tutor/school. Tutors met with students once a week for 16 weeks at each school after regular hours between mid-June and November of 2014. Typical sessions began with a challenge/question. Tutors guided students in the formulation of hypotheses, design of experiments, and discussion of their findings. Students were then encouraged to formulate preliminary answers based on prior knowledge, acquire new information through experimentation and reading, re-structure prior knowledge, establish conclusions and apply the new knowledge to unfamiliar situations.
At the end of the tutoring program, students took a test similar to the one used at baseline.
Results and Policy Lessons
Overall, participation in the remedial instruction program was low -- students attended an average of five to six remedial sessions, or about 40 percent of the total number of sessions initially scheduled. This was equivalent to only a four to five percent increase in total science instruction time over the school year.
Despite low participation, students assigned to the remedial sessions scored .12 standard deviations higher on average on a science test after the program. However, these improvements were concentrated exclusively among boys, for whom gains were .22 standard deviations, while there was no statistically significant effect for girls. Moreover, there were no improvements among the lowest-performing 10 percent of students.
Boys and girls attended the same number of remedial sessions on average, so the unequal effect for boys was not driven by participation. Instead, observation of classrooms suggests boys may have received preferential treatment from tutors over girls.
These findings suggest that low-performing students can learn through inquiry-based pedagogical approaches. The targeted, inquiry-based remedial science education model could easily be expanded to provide intensive academic support at a large scale for students who fall behind. Since the tutors are local and the training is short, the project would be straightforward to replicate.
However, to bring this remedial science education model to scale, two important challenges remain. First, a challenge remains to identify instructional models that help mitigate instructor stereotypes and attitudes with regards to gender and science. Second, the overall effectiveness of the remedial education model was achieved in spite of low student participation. Participation may improve by more clearly disseminating the program and promoting its benefits among parents and students. Since many students are either economically active or provide help at home by taking care of younger siblings, a more flexible schedule could also help improve participation.
Sources
1. Banerjee, Abhijit, Shawn Cole, Esther Duflo, and Leigh Linden. Remedying education: Evidence from two randomized experiments in India. No. w11904. National Bureau of Economic Research, 2005.
2. Kremer, Michael, and Alaka Holla. "Improving education in the developing world: what have we learned from randomized evaluations?." Annual Review of Economics 1 (2009): 513.
3. Office of Evaluation and Oversight. “The Pedagogy of Science and Environment: Experimental Evidence from Peru.” September 12, 2012. p.1
4. Ibid.
5. Program for International Student Assessment. “PISA 2012 Results in Focus.”