Measuring the Effectiveness of Online Skills Training for Female Microenterprises in Guatemala and Mexico
Abstract
Live online skills training courses have the potential to reach more people at lower costs than in-person training. However, evidence is limited on whether these platforms can deliver similar or higher impacts to microentrepreneurs as in-person, classroom-based programs can. In Guatemala and Mexico, researchers conducted a randomized evaluation to measure the impact and cost-effectiveness of a live online entrepreneurship training program for female microenterprises during the COVID-19 pandemic. The live online training led to improved business practices and higher sales after two months, but these effects disappeared after six months.
Policy Issue
Online programs have increased in recent years as a wider reaching and less costly alternative to in-person programs. As COVID-19 distancing policies disrupted social gatherings, in-person programs became infeasible and subsequently transitioned to online settings. However, entrepreneurs in low- and middle-income countries may encounter barriers such as limited access to technology required to attend the sessions. In addition, large online courses in high-income country settings have experienced high dropout rates.
Context of the Evaluation
At the time of study, female-led enterprises accounted for more than half of Mexican microenterprises and 40 percent of Guatemalan microenterprises. The COVID-19 pandemic had differing effects for both countries’ economies in 2020, with Mexico’s GDP shrinking 4.5 percent, while Guatemala’s GDP shrunk by 1.8 percent. To support female microenterprises during this period, the Mexican NGO CREA launched the “Strengthen Your Business” program and advertised it primarily on social media as a free online course. The program covered topics related to resilience, costing, prices, marketing, e-commerce, and making their business plan. By the time the training program had taken place, however, the most severe period of shutdowns was over and firms were starting to operate and recover.
Entrepreneurs self-selected into “Strengthen Your Business”, and they tended to be younger and more educated than the average female microentrepreneurs in Mexico. On average, these enterprises had 1.5 employees, reported monthly profits of US$100, and were in operation for four years.
Details of the Intervention
Researchers partnered with IPA and CREA to conduct a randomized evaluation measuring the impact of CREA’s "Strengthen Your Business” live online entrepreneurship training program on female microentrepreneurs’ business practices and performance in Mexico and Guatemala. Researchers also evaluated the cost-effectiveness of the program.
A total of 2,208 female microenterprises from across Guatemala and all states in Mexico participated in the evaluation. Researchers randomly assigned 1513 microenterprises to receive live, online training on entrepreneurship topics determined by CREA or topics they selected collectively. The other 695 microenterprises served as a comparison group, receiving a self-paced online training option in which they could access the slides and webinar content of the CREA-determined course platform.
Researchers conducted a follow-up survey two months after training started to measure the short-term impacts of the program, including adoption of recommended business practices and business performance measured by profits and sales. Researchers conducted a second follow-up survey six to eight months after to evaluate whether impacts persisted.
Results and Policy Lessons
The live online training led to positive impacts for microenterprises after two months, including an 8.8 percent increase in recommended business practices and a 24 percent increase in monthly sales. After six months, the positive effects disappeared.
Business Practices: Trained enterprises increased their performance on an index reflecting their adoption of recommended business practices by 5.4 percentage points (a 10.8 percent increase over the comparison group mean of 50.2 percent). This effect was driven by an increase in planning practices and accounting practices, with no impact on marketing practices. In the six-month follow-up, the positive effect disappeared. Over time, microentrepreneurs who took the live online course dropped business practices while microentrepreneurs in the comparison group adopted new ones.
Sales and Profits: After two months, the increased take-up of recommended business practices from the live online training led to a MXN 4,113 (US$240 at the time of study) increase in monthly sales, a 24 percent increase over the comparison group mean of MXN 17,023 (US$933). However, the positive effects on sales disappeared by the six-month follow-up as participants in the intervention group dropped their newly learned business practices and participants in the comparison group started to adopt them. Researchers could not detect an impact on profits.
Cost-effectiveness: In Mexico, live online training was 19 percent less expensive than in-person training (US$50 per participant in Mexico compared to US$62). The costs of live online training were not too different from the costs of in-person training because personnel accounted for most of the program costs, while class sizes were limited to 15-20 entrepreneurs in order to facilitate similar start dates and session times.
Ultimately, researchers found that the live online classes were indeed viable platforms to implement training, even if they were not as cheap or delivered lasting impacts. For future programs, they recommended that class sizes should be doubled and led by the program’s most effective trainers to maximize impact while nearly halving training staff costs.
