A pair of communications do not change FAFSA completion rates

Graduation with a skyline view

Graduation with a skyline view

What was the challenge?

Part of the U.S. Department of Housing and Urban Development’s (HUD) strategic plan is to raise awareness about higher education and financial aid opportunities to HUD-assisted residents. Completing the Free Application for Federal Student Aid (FAFSA) is the first step to accessing Federal and other forms of aid for post-secondary education. HUD partnered with the Office of Evaluation Sciences (OES) in 2017 to design and evaluate a package of informational communications which encouraged young adults residing in public housing in Seattle and King County to complete the FAFSA.

What was the program change?

HUD and OES designed and sent a bundle of two letters to address the basic informational gaps that may exist. The letters gave general information about the FAFSA and simplified the process for starting the application to two easy steps. The second letter attempted to dispel common misconceptions about out-of-pocket costs of college by showing the total cost of tuition and fees once a full Pell Grant was taken into account and used a social norm to indicate that most public housing residents were eligible for a Pell Grant.

How did the evaluation work?

All 17-24 year old residents in residing in public housing in the Seattle Housing Authority (SHA) and King County Housing Authority (KCHA) were mailed both letters between February and April 2017. The evidence-based insight was evaluated with a non-experimental, augmented synthetic control method. The augmented synthetic control method creates a comparison unit by weighting other public housing authorities (PHAs) in the comparison group in a way that best matches the characteristics of the treated PHAs. The estimated effect is calculated as the difference between the FAFSA completion rate of the (combined) treated units and the FAFSA completion rate of the weighted synthetic comparison unit. Residents of similar age from other large PHAs were used to create the comparison group.

What was the impact?

The results suggest the communications did not change FAFSA completion rates in the 2017 or 2018 academic years. The average FAFSA completion rate in SHA and KCHA was 47.4 percent and 46.2 percent for the 2017 and 2018 academic years, respectively. Sending the two letters is estimated to have increased FAFSA completion by 3.4 percentage points in the 2017 academic year and 3.6 percentage points in 2018 academic year over the synthetic comparison unit, but the estimates cannot be distinguished from zero. Robustness checks using different model specifications yield substantively similar results of a small, statistically insignificant, positive effect.

image

Verify the upload date of our Analysis Plan on GitHub.

Year

2019

Status

Complete

Project Type

Impact evaluation of program change

Agency

Housing and Urban Development

Domain

Education

Resources

View Analysis Plan (PDF) View Abstract (PDF)