Increasing timely reporting by local governments through to-do lists

What was the challenge?

As part of the American Rescue Plan (ARP), the Department of the Treasury (“Treasury”) is administering the State and Local Fiscal Recovery Funds program (SLFRF), which provides $350 billion in funding for eligible state, local, territorial, and Tribal governments to support their response to and recovery from the COVID-19 public health emergency. Approximately 27,000 small cities and towns – referred to as non-entitlement units (NEUs) of local government – were allocated $19.5 billion of this funding.¹ Treasury seeks to support NEUs to meet their reporting requirements in order to promote transparency, responsibility, and equity in the use of funds.²

In order to submit the required spending reports, recipients must first register (“sign up”) on Treasury’s American Rescue Plan (ARP) Reporting and Compliance Portal. To ensure that NEUs know how to complete their reporting requirements on time, Treasury communications should clearly and concisely lay out the steps NEUs need to take.

What did we do?

We collaborated with Treasury to redesign emails to encourage signing up on Treasury’s portal. We made two changes to the email: 1) including step-by-step instructions at the beginning of the email (“to-do list”); and 2) including a “call to action” in the subject lines (“call to action”). These additions quickly educate recipients on the necessary actions they need to undertake and the required steps.

What did we learn?

We find statistically significant evidence that both the to-do list and call to action treatments increase portal signup. The emails were sent on February 10th, well ahead of the April 30th reporting deadline. One day after the email was sent, the rate of signup was one percentage point higher for those who got the to-do list treatment compared to those who did not. Two weeks later this difference was even larger. Portal signup was 26.9% among those who got the to-do list treatment and 23.8% among those who did not, a statistically significant increase of 3.1 percentage points. This represents a 13% increase in signups relative to the group that did not receive the to-do list email.

The call to action treatment was also effective at increasing signup, though less so. One day following the email, the estimated effect is very close to zero and is statistically insignificant. After two weeks, portal signup was 26.1% among those who got the call to action treatment and 24.6% among those who did not, a statistically significant increase of 1.5 percentage points. This represents a 6% increase in signups relative to the group that did not receive the call to action email.

Figure 1. Numbers of NEUs signed up on Treasury’s ARP Reporting and Compliance Portal

In March, Treasury sent the same randomized email again to a larger set of NEUs (N=23,911). As an exploratory analysis (not included in the pre-analysis plan), we estimated the impact of this email on NEUs’ submission of spending reports – the outcome that Treasury intended to influence by having NEUs sign up on the portal. Effects were again positive and statistically significant. Two weeks after the second email was sent, report submissions were 3.0 percentage points higher among the NEUs who got the to-do list treatment than those who did not and report submissions were 1.2 percentage points higher for the NEUs who got the call to action treatment compared to those that did not. This represents an increase in submission of 20% and 7% relative to the groups that did not get the to-do list treatment and call to action treatment, respectively.

Figure 2. Numbers of NEUs that submitted spending reports on Treasury’s ARP Reporting and Compliance Portal

Key takeaways

We found that simplifying instructions through a bulleted to-do list increased responsiveness to actions requested by a federal agency. We estimate that this small change, which came at no additional cost to the agency, could have increased signups by hundreds of NEUs if included in all emails. Furthermore, while the estimated impact of a call to action in the subject line of the emails is weaker, it’s still important to note that the subject line of an email matters.

OES and Treasury worked quickly to generate these insights ahead of the April 30 deadline for spending report submissions by NEUs. The speed with which we were able to generate results meant that Treasury could incorporate findings into planned campaigns and is now including to-do lists in its reporting emails to SLFRF recipients.

Notes:

1. Non-entitlement units of local government (NEUs), defined in section 603(g)(5) of the Social Security Act, as added by section 9901 of the American Rescue Plan Act of 2021, are local governments typically serving populations of less than 50,000. NEUs include cities, villages, towns, townships, or other types of local governments.
2. U.S. Department of the Treasury, Coronavirus State and Local Fiscal Recovery Funds.

Verify the upload date of our analysis plan on GitHub.