Abstract

The capacity of states to effectively allocate resources and serve the public good depends on the quantity and quality of information they have. This information is often collected through decentralized processes and requests to other public or private entities. But what happens when these entities do not respond? In Colombia, IPA worked with the Inspector Attorney General’s Office (PGN) to implement a low-cost strategy for collecting more and better information about transparency practices. Drawing from behavioral insights, the research team found that sending direct communication to entities legally obliged to share information rather than those responsible for higher-level oversight increases the average response rate by 5.1 percentage points. The study equips the PGN with a strategy for more effective data collection and therefore more information to comply with its public mandate.

Policy Issue

A critical task of the state is the collection of data to monitor and guide public management. This information ranges from data collection on individuals through the census or vital registration to information on state functions and outcomes and is used to both enforce laws and allocate resources. A lack of sufficient or accurate reporting can impede informed policymaking.

Collecting state data through requests to public and private entities requires clear communication of needs, relevance, and deadlines. Even when requests are clear, compliance and accuracy are often low. What prevents entities from providing mandatory information and what can facilitate better information sharing? Understanding the barriers to information transfer will allow the PGN to enact low-cost strategies to ensure compliance with the 2014 National Law of Transparency and Access to Public Information.

Context of the Evaluation

The Inspector Attorney General’s Office (PGN) in Colombia is responsible for monitoring the use of public resources and the behavior of public officers. The PGN’s Office for the Defense of Public Heritage, Transparency and Integrity (Transparency Office) oversees compliance with the 2014 National Law of Transparency and Access to Public Information. This Law defines which public or private entities or persons must make public and accessible all information related to their functions, objectives, and performance. The Law also states that all citizens have the right to freely access this information. The  public and private entities required to comply are divided into three categories: i) traditional obligated subjects (public entities at the national and local levels), ii) non-traditional obligated subjects (private entities that manage public resources or offer public services) and, iii) political parties or movements and significant groups of citizens.

To monitor compliance with the Law and guarantee citizens’ right to information, the Transparency Office created and launched the Index of Transparency and Access to Information (ITA) matrix in 2019. This matrix serves as one aspect of PGN efforts to combat corruption and misuse of public resources. The data collection instrument contains a series of approximately 200 questions that entities must self-report annually. However, the initial response rate of the ITA matrix was very low. Only 29 percent of subjects filled out the matrix, with a rate of just 3.3 percent among non-traditional subjects. For that year, the Office sent communications to sector heads—entities that supervise private or public organizations in a specific sector—and entrusted them with the task of notifying subjects about their obligation to fill out the ITA matrix. No direct communication was sent to the entities responsible for reporting.

Details of the Intervention

To address the low rate of response to the ITA matrix, the PGN worked with IPA through the ProcuraLAB to design, implement, and test a low-cost intervention to increase the response rate to the ITA matrix, gain knowledge about the barriers to efficient data transfer, and provide the Transparency Office with more information on ensuring compliance with the Transparency Law.

Researchers defined a series of hypotheses about the barriers impeding information transfer. These included a lack of understanding by subjects about their obligation under the Transparency Law, administrative overload limiting capacity for timely response, limited capabilities or understanding of the matrix, and a lack of incentives or awareness of oversight from the PGN.

Based on these hypotheses, the research team designed an intervention in 2020 that changed how the PGN communicates by sending direct emails to obligated subjects instead of relying on the sector heads. A sample of 12,053 obligated subjects were randomly assigned to only receive communication through the sector head or to receive a direct message. Researchers varied the content and frequency of the direct communications to address the different hypothesized barriers for information sharing.

The direct emails included 16 combinations of four types of messages: information about obligations to complete the ITA matrix, information about retrospective compliance in 2019, notice of potential review of results from the  2020 measurement, and training resources for completing the ITA matrix. The communications were sent the week before the ITA matrix opened on September 15, 2020 and reminders were sent on October 1, 2020. Researchers measured impacts of the communications using the responses recorded in the ITA webpage during the 2020 measurement and a 2019 obligated subjects census, both provided by the Transparency Office.

Results and Policy Lessons

Preliminary results show that direct communication was effective in boosting the ITA response rate. Receiving the direct communication from the PGN with information about the obligation to comply with the ITA matrix increased the probability of filling out the matrix by 5.1 percentage points. Receiving a reminder led to a response rate 8.4 percentage points higher than those who relied on sector heads only. The effect was greater for non-traditional subjects who registered the lowest response rate in 2019 and for obliged subjects who did not fill out the matrix in 2019. In absolute terms, 1,888 entities that did not fill out the matrix in 2019 completed the measurement in 2020.

The increase in response was mainly driven by direct communication with or without a follow-up reminder. There is no evidence to confirm that content about oversight or training resources influenced the response rate. This suggests that the main barriers to effective data collection are associated with the communication strategy employed by the PGN and insufficient information from subjects about their obligation, rather than lack of capacity to complete the matrix or knowledge about either PGN’s role in oversight or compliance with the Transparency Law. To follow up on this finding, the research team is conducting an audit of the accuracy of responses by validating self-reported assessments of transparency practices. This component of the measurement will assess how direct communication impacts the quality and accuracy of the data provided in the 2020 measurement.

The results provide the PGN with rigorous quantitative evidence that can be used to support its decision-making on how to perform a decentralized request for data. The study also equips the PGN with evidence on the main barriers for effective data collection and low-cost mechanisms that can be used to promote compliance with the National Transparency Law.

 *Results are preliminary and may change after further data collection and/or analysis.