alt text

alt text


View the PDF Version

Agency priority

The U.S. Department of Homeland Security’s U.S. Citizenship and Immigration Services (USCIS) aims to increase customer satisfaction by encouraging the effective and efficient use of USCIS’s online services and contact centers as part of their mission to uphold America’s promise as a nation of welcome and possibility with fairness, integrity, and respect for all they serve. Increasing the number of customers who use USCIS’ online portal, myUSCIS, is a key agency priority, as part of their efforts to reduce the paperwork burden on customers and agency staff.

Program change description

USCIS has an existing outreach program that identifies and provides outreach to customers who have submitted a paper version of a form that was available online. Customers may underutilize online tools for a variety of reasons, including a lack of information, a (mis)belief that such tools are not secure, or a perception that using online tools is difficult or cumbersome.1 Evidence on the effectiveness of low-cost, informational messages for reducing administrative burdens such as these is mixed.2

As part of their existing outreach efforts, USCIS sends a one-time email to customers who have recently filed a form by mail. The email encourages them to create an online account and link their paper-filed case to their account, and provides instructions for doing so.

In collaboration with USCIS, we designed two alternate email messages. The first emphasized the benefits of using an online account to track one’s case status in real time, drawing on prior research that shows making the work performed by government appear more salient (known as “operational transparency”) can increase their trust and engagement.3 The second, grounded in literature on the effectiveness of highlighting social norms to motivate behavior, emphasized that millions of other customers had already signed up for online accounts. Both emails employed best- practices in designing effective communications, including simplified language, shorter overall message length relative to the status quo message, and a clear call to action. As with the status quo email, both new emails were sent once in the month following the form filing date.

Evaluation design

The program change was evaluated with an individual level randomized evaluation. We randomized 261,398 customers to one of three groups: 1. the status quo group, assigned to receive the USCIS’s existing outreach email (n=86,469); 2. the operational transparency group, assigned to receive the simplified email that emphasized the benefits of using online accounts to track one’s case status (n=87,702); and 3. the social norms group, assigned to receive the simplified email that emphasized that many other customers had already signed up for online accounts (n=87, 227). Customers were clustered at the email address level, and each email was randomly assigned to a message group. Enrollment into the intervention occurred once per month over the course of three months (May, July, and August 2023).

Analysis of existing data

We intended to analyze the impact of randomized outreach using existing USCIS data on the creation of online accounts and the linkage of paper-filed cases to online accounts.

Results

We were not able to complete the analysis as planned. In cases where data are not available or the evaluation did not provide comparable comparison groups, we do not report results. In this case, the evaluation team learned that customers who received outreach already had online accounts, but needed to activate them, leading to a mismatch between the needed action and the originally intended behavior.

Implications

This evaluation demonstrated the feasibility of embedding a rigorous evaluation into ongoing outreach efforts at USCIS. It also enhanced cross-team collaboration within USCIS, identified areas in which ongoing informational outreach could be modified to be better aligned with organizational processes, and suggested new avenues for evaluating engagement with online services in the future.

Notes:

  1. Herd, P., & Moynihan, D. P. (2019). Administrative Burden: Policymaking by Other Means. Russell Sage Foundation. https://doi.org/10.7758/9781610448789.
  2. See e.g., Elizabeth Linos, Allen Prohofsky, Aparna Ramesh, Jesse Rothstein, and Matt Unrath. “Can Nudges Increase Take-up of the EITC?,” American Economic Journal: Economic Policy, Volume 14, No. 4, November 2022, Pages 432-452. https://www.aeaweb.org/articles?id=10.1257/pol.20200603; Amy Finkelstein, Matthew J Notowidigdo, Take-Up and Targeting: Experimental Evidence from SNAP, The Quarterly Journal of Economics, Volume 134, Issue 3, August 2019, Pages 1505–1556, https://doi.org/10.1093/qje/qjz013.