This post summarizes findings from a group of studies in education that were presented on March 2, 2012 at a half day event hosted by Innovations for Poverty Action (IPA) and the University of California’s Center for Evaluation for Global Action (CEGA). Please forgive any errors or omissions. Cross-posted from Philanthropy Action.

Read about the other presentations from this conference in Part One and Part Three.

Leigh Linden from the University of Texas at Austin opened the second session of the IPA/CEGA event on education with the promise of results from two studies he conducted in India on the impact of computer-aided instruction. Computer-aided instruction refers to the use of computers in the classroom for a range of purposes, from self-directed lessons to practice in core skills.

Linden first summarized what is already known about the impact of computer-aided instruction. In brief, the research results have been mixed. In one study, computer-aided instruction led to a significant decrease in skills; in another to a significant increase. On balance, Linden says that the evidence has shown positive, if muted, increases in reading and math skills from computer aided instruction, but that it is difficult to generalize since existing studies vary dramatically in the programs they test, the students they assess, the teaching environments and other highly variable factors.

On to the evidence: Linden first presented findings from a project he conducted with Gyan Shala, a network of private schools in India. These schools were well-structured with a highly specified curriculum. In a baseline test intended to assess math skills, Gyan Shala students massively outscored their peers attending other local schools.

Gyan Shala students in this project were randomly selected to receive an hour of special computer-aided instruction for the purposes of practicing math skills. Students were divided into two groups: a “pull out” group that received computer time during school hours; and a second group that received computer time after school hours.

Linden found that the pull out program resulted in a significant average decline in skills over the study period. The pull out group saw a .7 standard deviation decrease in skills compared with their peers who did not receive computer time. The effect varied depending on the ability of the student at the beginning of the project. The best students in the school saw some decrease in skills, but far less than the least successful students, whose skill loss was dramatic. The children who received computer-aided instruction after school hours saw the reverse effect. The best-performing students did not benefit much, but the worse performing students benefited significantly.

These results may cause many teachers unions to celebrate, as they suggest that instruction really matters, especially for struggling students. Computers cannot replace instructional time, even in India, where teacher and student absenteeism is high.

Linden’s second study measured the impact of computer-aided instruction in English language learning. In this second study, Indian public schools were given tablet computers similar to the LeapFrog. The tablets had an English language application that children could follow at their own pace. Some schools also received flashcard activities that the teachers did with the children. Linden divided the students into four randomly assigned cohorts. One got to use the tablets only; one got to do the flashcard activities only; some got to use both tablets and flashcards; some got nothing.

Linden’s results again varied depending on the skill level of the child. All three cohorts showed average improvements. But the best performing students did much better using the tablet computers at their own pace, and gained nothing from the flashcard activities. In contrast, the less successful students saw the greatest benefit from doing flashcard activities led by the teacher, and gained nothing from the tablets. Again, for the lowest performers teachers really matter.

Curiously, students also showed improvements in math—a subject not addressed by the computer-aided technology. Linden concludes from interviewing students that the technology and/or the flashcard activities allowed the students to finish their English-language lessons faster, so the teachers were using the extra time for other instructional areas.

These findings reinforce an important idea for social interventions: context matters. Linden expressed his concern that many social interventions are planned with the assumption that they might help and won’t hurt, but the evidence shows that they can hurt. All the students who had their instructional time limited in order to take advantage of computer-aided instruction saw their skills decline.

Someone should tell Nicolas Negroponte, founder of One Laptop Per Child (OLPC).