Hands typing on laptop

Hands typing on laptop


View the PDF Version

Key findings

Providing applicants with a more structured way to report their income from multiple jobs did not meaningfully affect application completion rates in a “digital assister” SNAP application.1 However, applicants in the structured income reporting condition reported more income on average and, as a result, were 3.5 percentage points more likely to report income above the threshold for estimated SNAP eligibility.

Agency priority

The Supplemental Nutrition Assistance Program (SNAP) is America’s largest nutrition program, providing an average of nearly $200 a month in food assistance to 1 in 8 Americans.2 SNAP is a U.S. Department of Agriculture (USDA) program that is administered by state agencies. To receive these benefits, individuals apply on behalf of their household with their local county or state SNAP office. SNAP applications can often be completed online for ease of access, but even online applications can be long and difficult to complete. Administrative burdens experienced by applicants throughout the benefits application process may prevent them from applying for and receiving valuable financial support.3

Eliminating barriers in access to SNAP is a priority of USDA. Reducing burdens in forms and applications is one pathway towards improving the delivery of federal services.

Program change description

Code for America (CfA, our collaborator for this work) created a digital assister to help SNAP applicants apply for benefits more easily in one state administering SNAP. In the business-as-usual digital assister, SNAP applicants with income from multiple jobs are asked to report their total household income from the last 30 days in a single, open text box.

In an analysis of six months of baseline data (from September 2023 to February 2024), we found that SNAP applicants whose households receive income from multiple jobs were 12.6 percentage points less likely to submit the application compared to those with income from only one job. Applicants with income from multiple jobs may face unique barriers with unstructured income reporting, such as not knowing exact income amounts from each job, and having to sum their income across different sources and pay schedules.4,5

We worked with CfA to design an alternative income reporting option for applicants with multiple jobs that provides more structure for applicants navigating the income reporting process. Our primary aim was to make it easier for applicants to report their income by splitting an otherwise difficult task into smaller components. Additionally, providing more structure to income reporting can reduce errors that come from rounding or omitting some sources of income.6

Evaluation design

The evaluation ran from June 6th – August 15th, 2024. All SNAP applicants who made it to the income reporting module during the study and who reported having more than one job (N=20,549) were randomly assigned to one of two income reporting options.7

In the unstructured reporting condition (n=10,179), applicants were asked to report their total household income from the last 30 days in a single text box. In the structured reporting condition (n=10,370), applicants were asked to report the income from each job for each earner in their household. For example, in a household with two earners who each have two jobs, applicants would start with one earner, enter the income from each of that person’s jobs, and then do the same for the second earner. For each job, applicants would select the pay frequency and enter the amount earned per pay period. Applicants in the structured reporting condition also had the option to estimate the income of each earner in the household if they preferred not to enter their income from each job.

Analysis of existing data

CfA collects all responses to its SNAP digital assister, along with indicators of condition assignment, time spent on the application, and whether an applicant ultimately submitted their application. The primary outcome of interest was whether the applicant submitted the application.

Results

We found that application submission rates were lower in the structured reporting condition (57.9%) compared with the unstructured condition (58.9%), but this difference was not statistically significant (p=.135, 95% CI [-2.38, 0.32]).8

We ran additional analyses comparing the unstructured and structured reporting conditions on a set of exploratory outcomes. Applicants in the structured reporting condition reported an average of $223.90 more income (p<.001, 95% CI [153.62, 294.19]). These applicants were also 3.5 percentage points more likely to report income that was over the threshold for SNAP eligibility (p<.001, 95% CI [0.02, 0.05]).9We were not able to determine whether there was a difference in approvals for SNAP benefits between the unstructured and structured reporting conditions.10

The exploratory analyses also indicate that application submission rates among some subgroups of applicants differed between the structured and unstructured reporting conditions. Applicants with only one household member and applicants who used the digital assister on a mobile device were significantly less likely to submit the application when they were in the structured reporting condition compared with the unstructured reporting condition. Among applicants with more than one household member and applicants who used a desktop device, there was no difference between the structured and unstructured reporting conditions.

Implications

Our results indicate that the structure of income reporting in a digital assister does not have a meaningful overall effect on application submission rates for applicants in households with multiple jobs. This suggests that other barriers such as gathering and verifying income information make it difficult for these applicants to complete federal forms and benefits applications. For example, applicants may not know the exact earnings of other members of their household, or may have a hard time keeping track of earnings records from multiple jobs, making it difficult for them to complete the application — regardless of how income is reported.

Although assignment to the structured income reporting condition did not appear to affect submission rates on average, it did lead applicants to report more income. As a result, applicants in the structured reporting condition were more likely to be over the digital assister threshold for SNAP eligibility. This may be due to the structured reporting condition leading people to more accurately and fully report their income, resulting in more accurate determinations of ineligibility. On the other hand, the structured reporting condition may have led people to inadvertently over-report their income, resulting in incorrect determinations of ineligibility. Future evaluations that are able to leverage verified income data would better be able to determine whether structured income reporting options can increase reporting accuracy.

Notes:

  1. A digital assister is a tool that builds an easier-to-use interface on top of an existing form, somewhat akin to a graphical user interface.
  2. “Program Data Featured Reports, Fiscal Year 2023” U.S. Department of Agriculture Food and Nutrition Service, accessed July 18, 2024. https://www.fns.usda.gov/pd/overview.
  3. Moynihan, Donald, Pamela Herd, and Hope Harvey. “Administrative Burden: Learning, Psychological, and Compliance Costs in Citizen-State Interactions.” Journal of Public Administration Research and Theory 25, no. 1 (2015): 43 – 69.
  4. Marquis, Kent, and Jeffrey Moore. 1990. “Measurement Errors in SIPP Program Reports.” Working Paper SIPP-WP-113. U.S. Census Bureau. https://www.census.gov/library/working-papers/1990/demo/SIPP-WP-113.html.
  5. Hurd, Michael, F. Thomas Juster, and James P. Smith. 2003. “Enhancing the Quality of Data on Income: Recent Innovations from the HRS.” Journal of Human Resources XXXVIII (3): 758–72. https://doi.org/10.3368/jhr.XXXVIII.3.758.
  6. Wu, Qiong, and Liping Gu. “Comparing single-and multiple-question designs of measuring family income in China family panel studies.” Sociological Methods & Research 53.2 (2024): 872-897.
  7. This included applicants who themselves had income from more than one job and/or who had income from more than one member of their household.
  8. For our confirmatory analysis, we ran an unadjusted model as pre-specified in our analysis plan. The results hold in a Lin-adjusted model with a pre-specified list of covariates.
  9. Data on income and predicted eligibility are only available for applicants who made it to each of these stages of the application, meaning that these estimates do not represent the full sample and may be affected by selection bias. Notably, completion of these sections of the application does not differ between conditions.
  10. We received county approval data from CfA indicating whether or not applicants who submitted applications via the digital assister were ultimately approved or denied for SNAP. This county dataset was missing approval data for many clients, and we were unable to determine the reason for this missingness. We ran multiple models with different subsets of applicants (e.g., including all applicants, restricting the sample to applicants who were in the approval dataset) and did not find any significant estimates of the treatment effect. We may receive additional data to follow up on whether the treatment affected ultimate SNAP approval.