Increasing participation in Ticket To Work through redesigned mailers


View the PDF Version

Key findings

Redesigned mailings did not increase Ticket assignments, but had a small impact on Helpline calls.

Target a priority outcome

The Social Security Administration’s Ticket to Work (TTW) program is a federally funded program that provides employment services for people ages 18 through 64 who receive Social Security disability benefits and want to work.1 TTW aims to provide beneficiaries the choices, opportunities, and support they need to enter the workforce and maintain employment with the goal of becoming economically self-supporting over time. Employment service providers — Vocational Rehabilitation agencies and authorized Employment Networks — offer support such as career counseling, job search assistance, and job training, and are paid by the program when their TTW clients meet employment-related outcomes. Program participation remains low: only an estimated 5% or less of eligible beneficiaries have “assigned a Ticket” (made an agreement with an employment service provider) since program inception.2 This evaluation helps address SSA’s Fiscal Year 2023 Evaluation Plan Ticket to Work Optimization project, which poses the evaluation question: “What is the impact of improved communications with disability beneficiaries who are eligible to participate in the Ticket to Work (TTW) program on program participation?”

Translate evidence-based insights

Typically, SSA mails three notices to Social Security Disability Insurance (SSDI) beneficiaries and Supplemental Security Income (SSI) recipients with disabilities about the TTW program. These notices are sent approximately two months after their initial disability benefit award, at the 12-month anniversary of their initial award, and at the 36-month anniversary. The notices include standard language about the TTW program and a paper Ticket titled “Your Ticket to Work,” which provides an additional page with TTW information.

In collaboration with SSA, we conducted a behavioral diagnosis that identified several potential barriers to TTW participation. We produced a behavioral journey map that shows the steps taken by a TTW applicant, identifies potential behavioral barriers along the journey, and maps potential evidence-based interventions to address those barriers.

We then designed an intervention consisting of redesigned mailers (notices and cardstock Tickets) to address the multiple behavioral barriers that may have factored into the low level of participation, including a fear of benefit loss and uncertainty about options.3 Our intervention did not address structural barriers identified in the behavioral map, such as limited service availability. This intervention aimed to increase the number of beneficiaries who “assign a Ticket” with employment service providers.

We redesigned the standard TTW notice to provide less information more concisely (simplification), provide beneficiaries a clearer outline of action steps (implementation prompts), highlight the benefits of TTW participation up front (salience), emphasize that beneficiaries already qualify (endowment effect), and treat beneficiaries as ready to return to work (positive identity priming).4 We made similar revisions to the paper Ticket, replacing it with a more durable and eye-catching cardstock Ticket.5 Its format — small and easily posted on a refrigerator or bulletin board — also makes key information about the program easier to save and access later.

Embed evaluation

In three individual-level randomized trials, we evaluated changes to TTW notices including their timing, the type sent, and the language used across a combined sample of 934,611 beneficiaries. Treatment assignment was based on the two terminal digits of a person’s Social Securitynumber (SSN).

The evaluation was conducted over 18 months, from September 2020 through February 2022, and involved three simultaneous randomized control trials with distinct populations:

1. Startup Cohort: Startup cohort (newly eligible for TTW; n = 282,026)6

2. One Year Cohort: 1-year anniversary (already contacted once; n = 302,981)

3. Two Year Cohort: 2-year anniversary (already contacted twice and not contacted under business as usual; n = 349,604)

The evaluation has a 2 x 2 factorial design for the Startup and One Year Cohorts, where we varied both the notice (redesigned or original notice) and the cardstock Ticket (included or not). Beneficiaries were assigned with equal probability across the four treatment groups.

For the Two Year Cohort, we replaced the three-year notice with a redesigned notice at two years post-award, and beneficiaries were assigned to either a treatment group (receiving both the redesigned notice and the cardstock Ticket) or a control group (no mailing) with roughly equal probability.

Analyze using existing data

The effectiveness of the revised notices were measured by Ticket assignments and Helpline calls. We used SSA call-center records and data from SSA’s Completed Determination Record (also known as the “Disability Control File”) to calculate Ticket assignments and Helpline calls within 9 months after mailings were sent. We used SSA’s Characteristic Extract Record and Disabled Beneficiary and Dependents Extract for information on beneficiary characteristics.

Figure 1. Baseline Ticket assignment and Helpline call rates are low Baseline Ticket assignment and Helpline call rates are low

Results

The redesigned notice did not have statistically significant effects on Ticket assignments after 9 months: a 0.05 percentage point increase in the Startup Cohort (p = 0.31, 95% CI [-0.04, 0.13]) and a 0.02 percentage point increase in the Two Year Cohort (p = 0.68, 95% CI [-0.06, 0.10]). Yet, it did have modest, statistically significant effects on Helpline calls: a 0.28 percentage point increase in the Startup Cohort (p = <0.001, 95% CI [0.18, 0.38]) and a 0.36 percentage point increase in the Two Year Cohort (p = <0.001, 95% CI [0.28, 0.45]). These represent meaningful increases given baseline rates of 1.82% and 1.37%, respectively.

The cardstock Ticket did not have statistically significant effects on Ticket assignments after 9 months: a 0.02 percentage point decrease in the Startup Cohort (p = 0.62, 95% CI [-0.11, 0.07]) and 0.05 percentage point increase in the One Year Cohort (p = 0.20, 95% CI [-0.03, 0.13]). It also had insignificant effects on Helpline calls: a 0.01 percentage point increase in the Startup Cohort (p = 0.83, 95% CI [-0.09, 0.11]) and a 0.04 percentage point decrease (p = 0.38, 95% CI [-0.13, 0.05]) in the Two Year Cohort.

Sending a mailing two years after initial benefit award did not have statistically significant effects on Ticket assignments after 9 months: a 0.05 percentage point increase (p = 0.24, 95% CI [-0.03, 0.13]). However, it had a statistically significant effect on Helpline calls: a 0.73 percentage point increase (p < 0.001, 95% CI [0.65, 0.80]), relative to a baseline of 0.54 percentage points.7

Figure 2. Redesigned mailings did not increase Ticket assignments, but had a small impact on Helpline calls Redesigned mailings did not increase Ticket assignments, but had a small impact on Helpline calls

Build evidence

SSA will continue using the redesigned notice when sharing information about TTW with SSDI and SSI beneficiaries 2, 12, and 36 months after their initial disability award (about 570,000 people annually). SSA may not invest in the cardstock Ticket in the future, given that it did not improve either outcome and is more expensive to distribute.

This evaluation points towards opportunities for further research, such as exploring why people might call the Helpline but not take up the program, exploring the impacts of redesigned TTW mailings over longer follow-up periods, or exploring how the economic impact of public emergencies such as the COVID-19 pandemic affects outreach programs. One of our pre-registered exploratory analyses suggests that sending the Two Year Cohort mailing may have increased Ticket assignments among those in counties with very high unemployment, while increasing the effect on Helpline calls.

Notes:

  1. I.e., individuals receiving Social Security Disability Insurance and/or Supplemental Security Income benefits based on disability.
  2. U.S. Government Accountability Office. “Ticket to Work Helped Some Participants, but Overpayments Increased Program Costs.” Report to Congressional Committees (2021).
  3. Mathematica. Characteristics, Employment, and Sources of Support Among Working-Age SSI and DI Beneficiaries. April 30th, 2009.
  4. Office of Evaluation Sciences. How to design effective communications: What has OES learned? Bhargava, S., & Manoli, D. (2015). Psychological frictions and the incomplete take-up of social benefits: Evidence from an IRS field experiment. American Economic Review, 105(11), 3489-3529.
  5. These cardstock Tickets were included in the original TTW mailings, but were eliminated due to budget constraints. The alternative (status quo) is a paper Ticket including more text.
  6. These sample sizes incorporate exclusions due to unanticipated missing data and logistical issues. Analyses that handle those issues differently yield similar findings.
  7. All statistically significant results we report remain so after applying our pre-registered multiple testing correction.