TulaSalud: Challenges of Measuring Impact for a Mobile Health program

Rapid expansion of mobile technology across developing countries has ushered in the emergence of the “mobile health” sector, or mHealth, which refers to the use of mobile technology in medical care. TulaSalud is an NGO based in Guatemala that implements an mHealth program that aims to improve health services for rural populations. Founded in 2008 with support from the Tula Foundation, TulaSalud operates in close partnership with the Ministry of Health in Alta Verapaz, one of the poorest regions in the country. Alta Verapaz is characterized by chronic malnutrition and high rates of maternal and infant mortality.

TulaSalud uses a mobile application to help community health workers (CHWs) collect patient information and provide basic healthcare services. Health outcomes in Alta Verapaz have improved considerably since TulaSalud introduced the program, but the precise contribution of the mHealth platform to this progress is uncertain.

This case study focuses on the important role of feedback data in designing and implementing an mHealth program that supplements existing healthcare services, and in making changes to improve program implementation.

TulaSalud serves as a good example of a program that uses data to make operational decisions and improve its operations. To build on the current approach, we recommend adding impact evaluations to test how variations in the platform and training of CHWs improve their performance. The efficacy of practices TulaSalud promotes is documented in medical research, and improving the training of CHWs and their ability to use the system is likely to be a more relevant question for TulaSalud (and less burdensome operationally) than an assessment of the platform on health outcomes.

Lessons for Others

1. Developing a responsible, actionable mobile system requires significant up-front investment.

Be prepared for a significant investment of resources and frequent iteration to create the system and the strategy for monitoring it. In addition to design, data collection, processing, and analysis, backoffice work is often required to check data quality and make sure that the data can be used and employed in a timely manner.

2. Gathering user feedback is a critical feature of the monitoring and evaluation system.

User feedback data can help optimize and streamline an mHealth system and ensure the actionability of the data it generates.

3. When using proven strategies, focus evaluation resources on creating new operational evidence.

If an approach is already proven to work, consider using an impact evaluation to answer important operational questions instead, such as how to improve program delivery.