There is a pithy adage that every simple question has a complicated answer.In the context of development economics, this is meant to be a warning that silver bullets rarely exist; indeed, only a limited number of interventions are shown to have a proven impact in multiple contexts.Clean Cookstoves have been touted as an intervention that has a dramatic effect on a variety of metrics. From a recent post on the New York Times' India Ink blog:

In September 2010, U.S. Secretary of State Hillary Rodham Clinton announced the creation of the Global Alliance for Clean Cookstoves…their ambitious goal, “100 by 20,” is to get 100 million homes to adopt clean stoves by the year 2020. They’ve even secured the actress Julia Roberts as a global ambassador.

The Global Alliance isn’t modest about its goal, nor is it about the claims it makes on behalf of clean stoves. According to its web site, a clean stove reduces child pneumonia by 50 percent; saves the equivalent of one to two tons of carbon dioxide per year; and leads to savings in fuel costs that allows a stove to pay for itself.

The post goes on to discuss the findings from one of the few impact evaluations conducted on clean cookstoves, conducted by Rema Hanna, Esther Duflo and Michael Greenstone. From the paper's abstract:

While we find a meaningful reduction in smoke inhalation in the first year, there is no effect over longer time horizons. We find no evidence of improvements in lung functioning or health and there is no change in fuel consumption (and presumably greenhouse gas emissions). The difference between the laboratory and this study’s field findings appears to result from households’ revealed low valuation of the stoves.

What are we to make of these results? Advocates of clean cookstoves certainly have some reason to be disappointed, given the absence of measurable health impacts on a given set of health metrics in the context of the study (rural Orissa, India). However, it is worth keeping in mind that the study was conducted within a certain context that is not necessarily representative of other communities that could benefit from clean cookstoves. Thus, the randomized controlled trial in Orissa does not preclude the possibility of positive health effects in a different context, although it does force us to adjust our expectations. In her communication with the author of the blog post, Professor Hanna had the following to say:

This isn’t to say that indoor air pollution is not a problem, or that an improved cooking stove cannot be part of the solution. But rather, we just don’t have enough evidence that the stoves systematically improve health, particularly under real world conditions where people do not regularly use the stoves, and if they do, the use often does not perfectly follow the manufacturer’s instructions.

How we frame the findings of the study--and indeed, the results of randomized controlled trials in general--affects the way we think about them. It would be incorrect to say that the study proved that clean cookstoves do not have positive health impacts; rather, the study could not disprove that clean cookstoves do not have positive health impacts. In other words, when one considers that this is one study in one environment, the biggest takeaway is that more research is in order before we definitively pronounce judgment on the health effects of clean cookstove adoption.

In an ideal world, we would go from here towards conducting additional impact evaluations on this topic. However, as Professor Hanna suggests, with finite resources and a host of alternatives available to the development enterprise, organizations like the Global Alliance would be well served by funding health programs with stronger evidence behind them as well. 

To scale-up a program with insufficient evidence behind it is to put the cart before the horse. With sufficient research however, today's moot program may well be tomorrow's proven impact.