Textbooks and Test Scores in Kenya
Policy Issue
Education in developing countries is often hindered by large class sizes and low teaching quality. Over the past decade many developing countries have expanded primary school access, energized by initiatives such as the United Nations Millennium Development Goals, which call for achieving universal primary education by 2015. But increased quantity has not always been matched with improved quality, and only 60 percent of appropriately aged children gain the skills necessary to attend secondary school worldwide. In sub-Saharan Africa, that number drops to only a quarter.1 There is a widespread belief that the provision of textbooks can substantially improve educational outcomes in developing countries, but there is little empirical evidence as to whether providing textbooks alone can overcome the effects of other systemic problems such as high teacher-student ratios and frequent absenteeism.
Context of the Evaluation
In Kenya, 50 percent of families live below the poverty line, and high teacher absenteeism is reported in schools.2 Nevertheless the overwhelming majority of children start primary school. But some schools appear to promote only strong students to grade 8 in order to maintain high average scores on secondary school entrance exams, and those not promoted often drop out. Many students also drop out in earlier grades – 35 percent of the surveyed students in grade 3 dropped within the next three years.
Primary school is provided at a low cost, with some expenses, such as textbooks, paid by parents. Because of this cost, schools in Busia and Teso usually have textbooks for teachers to use, but few textbooks for children. At the start of this study, 80 percent of students in the sample were in classrooms with less than one English textbook for every 20 students, and the analogous figures for math and science textbooks were 78 percent and 89 percent, respectively. Government officials and NGOs are hopeful that textbook provision will improve test scores, encouraging higher grade attainment and overall improved educational outcomes.
Details of the Intervention
In 1995, International Child Support (ICS) began a program to improve primary education in Kenya’s Busia and Teso districts by providing additional official government textbooks. J-PAL affiliates, in an effort to determine the impact which textbooks have on students, teachers, and overall learning, evaluated this program. ICS randomly divided 100 needy, low performing schools into four even groups. In each year, program schools were compared to those not yet receiving the program.
Program Design
| Distribution of Textbooks | School Year | Schools |
Group 1 | Math books provided to grades 3, 4, 5, 6 and 7 English provided to 3, 5 and 7 Science and Agriculture provided to grade 8 | 1996 | 25 |
Group 2 | Grant equal to US$2.65 per student, 43% of which was spent on textbooks | 1997 | 25 |
Group 3 | Similar grant in 1998 school year | 1998 | 25 |
Group 4 | Similar grant in 2000 school year | 2000 | 25 |
Sharing textbooks is common in Kenya, and two or three students typically share a workspace. Hence, a 60 percent textbook per pupil ratio was provided in English and science, and a 50 percent ratio was provided in math, giving nearly all students shared access to a textbook. A pre-test in the form of a district exam was administered to all schools before the textbooks were distributed, and district exam scores were collected at the end of each of the subsequent program years for comparison data. Classroom activities, attendance and dropout rates were also monitored.
Results and Policy Lessons
This study found no evidence that textbook provision increased average test scores, or that it reduced either grade repetition or dropout rates. Textbooks increased progression to secondary school for eighth graders but did not reduce grade repetition or raise attendance in lower grades. This provides evidence for the hypothesis that the program mostly benefited strong students, since only strong students reach grade 8 and have a hope to progress to secondary school, while many students in the lower grades were actually unable to read the textbooks. Students in grade 8, a selective and academically strong group, were 5 percentage points more likely to enter secondary school in the second program year than comparison school students.
Kenyan students are extremely heterogeneous in their family background, preparation for schooling, and economic circumstances. The Kenyan curriculum remains oriented to elites with language of instruction in English, most students' third language. Textbooks are meant to pair with this curriculum, arguably better suited toward elite students and ill-suited for academically weaker students. The evidence from this evaluation supports such a picture of the Kenyan education system, and suggests that better suited materials might produce achievement gains in a wider section of the population.
Sources
1 UNICEF, “Basic Education and Gender Equality,” http://www.unicef.org/girlseducation/index_bigpicture.html
2 As of 2000. CIA World Fact Book, “Kenya,” https://www.cia.gov/library/publications/the-world-factbook/geos/ke.html