Abstract

The growth of digital financial services has been accompanied by an increase in digital fraud. This raises certain questions: How can consumers effectively raise concerns and seek help? Can analyzing social media posts and customer support artificial conversations (chatbots) provide greater insight into consumers’ experiences and ways to prevent fraud? In the Philippines, researchers partnered with the Central Bank to analyze complaint data from its new consumer assistance chatbot and social media posts directed to financial service providers. The study aims to better understand the differences in the types of complaints between chatbot and social media data and generate recommendations to improve the chatbot.

Policy Issue

Over the past decade, digital finance contributed significantly to the 35 percent fall of the unbanked population globally,^[1] However, new risks are also emerging, such as fraud targeting mobile money accounts, or debt stress from high-cost digital consumer lending^[2]. For example, in 2021, suspected digital fraud attempts in financial services increased by 149 percent globally, as compared to 2020^[3].

To address this situation, digital service providers are taking advantage of new technologies, such as social media and smart chatbots, to create alternative channels for consumers to raise concerns and seek assistance. The data generated by these technologies offer a unique opportunity for real-time monitoring of consumer experiences, with the potential to provide valuable insights to authorities and regulators seeking to combat fraud. But regulators tend to lack the capacity to conduct in-depth analysis of high-volume data. Innovations for Poverty Action (IPA) has piloted systematic tools for leveraging providers’ customer support information and social media data, but achieving widespread adoption will require more research on how to maximize data use to protect consumers.

Context of the Evaluation

In 2019, almost 70 percent of adult Filipinos were unbanked^[4]. However, digital finance is growing rapidly in the Philippines. In 2020, over 4 million new electronic financial accounts were created and digital payments made up as much as 20 percent of all transactions^[5]. Many consumers entering the digital financial market are inexperienced with formal financial services and particularly vulnerable to risks.

The Central Bank of the Philippines, Bangko Sentral ng Pilipinas (BSP), launched its consumer assistance chatbot in July 2020, to provide more accessible, timely, and efficient complaints resolutions. The chatbot can handle queries and complaints from consumers through the chatbot, its website, SMS, and Facebook. This chatbot is part of BSP’s efforts to increase digital finance access while protecting users from fraud and other risks. In addition to launching the chatbot, BSP is using social media to capture the concerns raised by those who choose not to file formal complaints and offer a more user-friendly way to engage.

Details of the Intervention

This study is not a randomized controlled trial.

Researchers are analyzing chatbot data from July 2020 to June 2021 to identify the most common complaints, the sentiments of the complaints (positive, negative, neutral), the profile of complainers (gender, location), as well as the chatbot’s performance, using key metrics like goal completion rate, user satisfaction rate, session time and conversation volume.

In addition, researchers are extracting data from Facebook, Twitter, and Google Reviews pertaining to the BSP and more than 100 financial service providers (FSPs) for the period of August 2020 to August 2021. The data includes public posts on these institutions’ own pages or feeds, mentions of these institutions in public posts by other users, and the comments under these posts. Google Reviews data includes comments and ratings under the applications of BSP and the selected FSPs. Similar to the chatbot analysis, researchers are studying the common topics, sentiment, and segmentation in the social media data. In addition, they are analyzing the interaction between providers and consumers, such as response rate, and its correlation with other factors such as topics and attitudes toward the institutions.

The study aims to better understand the differences in the types of complaints, sentiments, and segmentations between chatbot and social media data, and generate recommendations for the improvement of the chatbot's complaint system. Improving the chatbot may increase the likelihood that complaints are addressed efficiently, increase access to digital finance, and offer a more user-friendly way to raise concerns.

Results and Policy Lessons

Research ongoing; results forthcoming.

Sources