Introduction to Rapid-Fire Operational Testing for Social Programs
One key message of the Goldilocks Initiative is that impact evaluation is not for everyone. Yet, even when measuring impact is not feasible, social enterprises and non-profits can still answer important questions about their programs using rigorous measurement techniques.
One of these is techniques rapid-fire testing: randomized trials that compare the effect of related interventions on a single, immediate (or short-term) outcome. This method is used to test operational issues and aims to influence immediate outcomes, such as product take-up, program enrollment, loan repayment, and attendance, among others. In rapid-fire tests, participants are randomized into different treatment groups (and sometimes, but not necessarily, a pure control group) and exposed to variations in a program’s design or message.
The outcome of interest (usually program take-up or use) is measured and compared across treatment and control groups. Often outcomes are measured administratively, so that there is no large survey undertaking necessary in order to gather the data. For example, tests may use data from a financial institution that the institution would gather anyhow (deposits, loan repayments), from a store on sales, or from online tests tools such as Google Analytics and Optimizely, which facilitate both the test and the data collection.