alt text

alt text


View the PDF Version

Key findings

We found that sending a one-time text message did not increase the rate of online filing for Permanent Resident Card (“Green Card”) renewal forms relative to sending no text message. Further, a modified text message emphasizing that online filing allows for greater and easier process transparency did not change the rate of online filing relative to the text message used prior to the evaluation.

Agency priority

The U.S. Department of Homeland Security’s U.S. Citizenship and Immigration Services (USCIS) aims to increase customer satisfaction by encouraging the effective and efficient use of USCIS’s online services and contact centers as part of their mission to uphold America’s promise as a nation of welcome and possibility with fairness, integrity, and respect for all they serve. Increasing the number of customers who use USCIS’ online portal is a key agency priority as part of their efforts to reduce the paperwork burden on customers and agency staff.1 USCIS has identified lawful permanent residents (LPRs) as one key target group to increase use of online services. Because Green Cards must be renewed every 10 years, and USCIS’ online offerings are relatively new, many LPRs may not be aware that they can file their renewal application online. Despite the online availability of Form I-90 [Application to Replace Permanent Resident Card (Green Card)] since 2015, online filing rates hovered at an average of approximately 60% over the two years preceding this evaluation, according to internal agency data.

Program change description

USCIS offers many of their forms and services through their online portal, myUSCIS. However, many customers do not maximize their use of this service. Customers may underutilize online tools for a variety of reasons, including a lack of information, a (mis)belief that such tools are not secure, or a perception that using online tools is difficult or cumbersome.2 Evidence on the impact of low-cost, informational messages for reducing these administrative burdens is mixed.3

As part of their broader outreach efforts to promote online services, USCIS sends text messages to LPRs when their Green Card expiration date is six, four, and two months away. This outreach reminds recipients that the Form I-90 is available online.

We worked with USCIS to develop a modified text message that informed recipients that the Form I-90 is available online and emphasized that filing online allows customers to more easily track their case status. Prior research has demonstrated that an organization can increase customers’ trust and satisfaction with its services by increasing operational transparency — providing the customer with a window into the work performed during the course of services, rather than only the end result.4

Evaluation design

The program change was evaluated with a randomized controlled trial between March and August 2023. We randomized 59,288 LPRs into three conditions: 1. the control group which did not receive a text message (n=19,854); 2. the status quo group who received the agency’s pre-existing message that read, “Renewing a Green Card is now easier than ever with online filing. To learn more, visit uscis.gov/i-90” (n=19,774); and 3. the modified message group assigned to the newly designed text message reading, “Renewing a Green Card with online filing allows you to track your case status in real time. To learn more, visit uscis.gov/i-90” (n=19,660).

Customers were clustered at the phone-number level, and each phone number cluster was randomly assigned to a message group. Enrollment into the evaluation occurred once monthly over the course of five consecutive months. Only customers whose Green Cards were set to expire two months from the enrollment month (e.g., expiring in July if enrolled in May) were included in the sample. As such, all customers in the sample had previously received reminders at six and four months prior to their filing deadline as part of the agency’s usual outreach program.

Analysis of existing data

Our primary outcome of interest was the customer’s filing modality (online or by mail) for their Form I-90. We analyzed Form I-90 filing data from March 1, 2023 and March 20, 2024. Because our primary outcome is filing modality, our analytic sample is limited to customers who filed a Form I-90 during the outcome period. The dataset also included information about individual characteristics — including age, race, and gender — that enabled a more precise estimate of the messages’ effectiveness.5

Results

Overall, 26,372 customers (or 44.48% of the randomized sample) filed a Form I-90 during the outcome period. This filing rate was lower than USCIS had initially anticipated. There are a few potential reasons for this. First, some customers in our sample had filed a Form I-90 prior to randomization. 6 Second, some customers in our sample applied for naturalization (via a Form N-400), which may have influenced their decision to also file a Form I-90. 7 As a result, our sample was smaller than anticipated at the outset of the evaluation and, thus, we were unable to detect treatment effects with the necessary precision.

Among the customers who filed a Form I-90, we found no significant difference in the likelihood of filing online between the customers assigned to receive either text message (67.11%) and those assigned to the control group (67.44%, p = 0.60, 95% CI [-0.015, 0.009]). Further, we find no significant difference in the likelihood of filing online between customers assigned to the status quo group (66.68% online filing rate) or the modified message group (67.54%) and those assigned to the no-message control group (status quo: p = 0.689, 95% CI [-0.021, 0.007]; modified: p = 0.863, 95% CI [-0.013, 0.015]) or from each other (p = 0.689, 95% CI [-0.005, 0.023]).

In our exploratory analyses, we found no evidence that informational outreach affects the time it takes for a customer to file their Form I-90. Assignment to receive any text message did not impact time-to-file, relative to receiving no message. (Relatedly, we find no significant difference in time-to-file by message type.8 Similarly, we found no evidence that assignment to a text message group impacted whether the customer’s case was processed by the time the data were retrieved in March 2024, relative to the control group (p = 0.846, 95% CI [-0.010, 0.008]). Finally, we conducted exploratory analyses to examine whether the effect of informational text outreach differed for particular groups of customers, specifically older adults (age 64+), those born in Hispanic countries, and those who had assistance filing their forms (i.e., an attorney or preparer). USCIS’s historical data indicated that these groups were the least likely to use online services.

Indeed, we do find that these groups are much less likely to file online. In our sample, older adults (aged 64 years and older) filed online at a rate of 60.67% compared to 69.58% among non-seniors (p< 0.001, 95% CI [-0.103, -0.074]), 58.65% of customers from a Hispanic country filed online compared to 79.28% of customers from a non-Hispanic country (p < 0.001, 95% CI [-0.211, -0.186]), and 35.54% of those who receive assistance from an attorney or preparer filed online compared to 79.47% of customers who received no formal assistance (p < 0.001, 95% CI [-0.450, - 0.422]). However, we do not find any evidence that the impact of informational text outreach or type of message was different for these groups than for the full sample.9

Figure 1. The message condition had no effect on customers’ likelihood of filing a Form I-90 online Bar chart showing that the message condition had no effect on customers’ likelihood of filing a Form I-90 online

Implications

The results suggest that sending outreach messages about online services two months before customers’ Green Card expiration dates did not significantly affect online filing rates. This could be due to the phrasing, modality, or timing, or it could be that the behavioral barriers customers face to using online services are too high to be influenced by light-touch methods. Eliminating ineffective outreach could result in savings, given the costs involved with sending text messages. However, our evaluation was limited by a smaller than expected sample size, so further evidence is needed to corroborate these findings. We were able to highlight groups for whom online filing rates lag behind the average customer, suggesting future avenues for more targeted outreach efforts. Future work could also be done to assess the optimal combination of outreach while balancing cost implications.

Notes:

  1. U.S. Citizenship and Immigration Services. FYs 2023-2026 Strategic Plan (2022), https://www.uscis.gov/sites/default/files/document/reports/StrategicPlanFY23.pdf.
  2. Herd, P., & Moynihan, D. P. (2019). Administrative Burden: Policymaking by Other Means. Russell Sage Foundation. https://doi.org/10.7758/9781610448789.
  3. See e.g., Elizabeth Linos, Allen Prohofsky, Aparna Ramesh, Jesse Rothstein, and Matt Unrath. “Can Nudges Increase Take-up of the EITC?,” American Economic Journal: Economic Policy, Volume 14, No. 4, November 2022, Pages 432-452. https://www.aeaweb.org/articles?id=10.1257/pol.20200603; Amy Finkelstein, Matthew J Notowidigdo, Take-Up and Targeting: Experimental Evidence from SNAP, The Quarterly Journal of Economics, Volume 134, Issue 3, August 2019, Pages 1505–1556, https://doi.org/10.1093/qje/qjz013.
  4. Ryan W. Buell, Ethan Porter, Michael I. Norton. Surfacing the submerged State: Operational Transparency Increases Trust in and Engagement with Government. Manufacturing & Service Operations Management, Volume 23, Issue 4, July-August 2021, Pages 745-1004. https://doi.org/10.1287/msom.2020.0877.
  5. Unless noted otherwise, all of the analysis reported in this abstract was prespecified in an analysis plan, which can be found at oes.gsa.gov.
  6. 2,908 (4.90%) customers filed a Form I-90 prior to their randomization date. This was balanced across treatment conditions [F(2, 56315) = 0.315, p = 0.730].
  7. 3,545 (5.98%) customers filed a Form N400, also balanced across conditions [F(2, 56315) = 2.798, p = 0.061].
  8. Any message vs. control: Hazard ratio (HR) 1.009, p = 0.496; 95% CI [0.983, 1.036]; status quo vs. control: HR 1.018, p = 0.240, 95% CI [0.988, 1.049]; modified vs. control: HR 1.000, p = 0.9997, 95% CI [0.970, 1.031].
  9. Seniors: Status quo vs control p = 0.322, 95% CI [-0.047, 0.015], modified vs control p = 0.210, 95% CI [-0.011, 0.051]; Hispanic country of birth: status quo vs control p = 0.657, 95% CI [-0.021, 0.033], modified vs control p = 0.937, 95% CI [-0.29, 0.026]; Formal assistance: status quo vs control p = 0.908, 95% CI [-0.026, 0.029], modified vs control p = 0.952, 95% CI [-0.027, 0.030].