What should NGOs do when they can't run an RCT?
The last few weeks have been a whirlwind. I recently applied to transition from my current position on the Rainwater Storage Device evaluation in Kamwenge to a new IPA project in Kampala. The project is a little different than the traditional line of work for IPA, namely rigorous impact evaluations, instead we plan to work with NGOs to support and improve upon their Monitoring & Evaluation (M&E) systems. Two weeks ago I got the new job, last weekend was spent looking for an apartment in Kampala, and the past four days were spent running around Kamwenge putting things in order before my departure. Yesterday I packed up my life and hitched a ride with a government truck headed into Kampala.
And with that I said goodbye to Kamwenge. While this is a transition I wanted, and am very excited for, it was oddly strange to leave. Many of the things I loved about Kamwenge in the first six months lost their glow in the spring (eg: doing my laundry by hand on Sunday mornings went from a refreshing chance for some mental housekeeping to a monotonous chore). At first the pace of work kept me preoccupied but this spring, with a more regular work schedule, I definitely hit the doldrums from time to time. I had all the time in the world to read and cook and play soccer. If I had decided to I could have set myself to studying the local language or volunteering in one of the nearby schools. I never did either of those things; partly because I wound up not being there for much of the last three months, but mostly because I wasn’t comfortable initiating a commitment I couldn’t keep. In the end I had three good friends in Kamwenge. They are all guys who are young, smart, and eager to succeed and I’ll miss seeing them around town and learning from them. While most of the town recognized me as the mzungu who lived “up at Balaam’s place”, to them I was still just that, another mzungu. And after another year in Kamwenge, or even three, I’m not sure that would have changed.
One thing sank in this spring: in being the only outsider in a community it’s easy to forget that there was another before me and there will be an endless stream of others that follow. For the first several months last summer I was frequently called Stuart around town. Each time I corrected them, No, Stuart was a Canadian man and he left Kamwenge several months ago. Frequently I also had to explain that not only were we not brothers, I had in fact never met the man. Similarly I imagine that anyone who shows up in town this summer will hear my name from time to time. We outsiders operate much like a revolving door, turning over in time with the projects we come to implement.
I am very grateful to have had the opportunity to live and work in a rural setting. I don’t think I would have been content if my first position had been in a developing capital. That said, I am ready to live in Kampala, ready to have colleagues in an office and share an apartment with a roommate, ready to have the regular amenities of a city (supermarkets with perishable items, sandwich shops, tennis courts, live music, decent internet, etc.), and the daily interaction with other expats.
Professionally, I’m excited for the new challenge. The M&E project is still very much an abstract idea: NGOs often want or need to evaluate the impact their projects have on beneficiaries. IPA does really great impact evaluations but the methods we employ are sometimes simply not feasible for certain NGOs and are not appropriate for certain types of projects (eg: early pilots). That said, excluding a long and expensive RCT style evaluation, what is the best method for NGOs that want to evaluate their projects? What data should they collect? And, ideally, how should they interpret and use the data they collect? We don’t know all the answers to these questions and therein lies the challenge of this new initiative. Personally the appeal of the project is this idea that we can help bridge the gap between rigorous evaluations and actionable M&E work. Hopefully the partnerships that we form and the systems we propose will build upon current best practices and help NGOs better understand their current projects and use this knowledge to implement better projects in the future.