This is the first part of a two-part discussion about how OES work improves the federal customer experience. An abbreviated version of this post appears on the Performance.gov blog.

Shannon currently sees her doctor at the Naval Medical Center Camp Lejeune. Keeping track of her communications with her doctor, making sure she doesn’t miss important lab results or messages about her care, is one more to-do when she has a full time job as a Marine Logistics/Mobility Chief and mom of three.

Military healthcare clinics provide secure online messaging to allow patients like Shannon to communicate with providers, get lab results, and renew prescriptions, but many patients haven’t enrolled yet in the messaging system. The actual design of how we communicate with people about available benefits and services matters a lot. Designing experiences that enable individuals to access federal services intuitively is part of what the broader Federal customer experience movement is hoping to achieve.

Working with federal agency collaborators, OES proposes and evaluates program changes that incorporate these evidence-based insights. Program changes can be as small as modifying the wording in a reminder letter, but are intended to address agency priorities by improving the likelihood that people complete forms, apply for programs, or submit required documentation. In this case, the Defense Health Agency (DHA) wanted it to be easier for Shannon to connect with her doctor, but it was important to let Shannon know that the secure messaging service existed and make the enrollment process as frictionless as possible.

In 2017, OES and DHA designed an improved approach to inform patients like Shannon about the secure messaging service and make it easier for them to enroll. The approach had three components: providers made a personal appeal about registration during patient visits, on-site staff assisted patients with signup, and staff gave a checkout sheet to patients with registration instructions to complete later.

image
This figure shows patient registration for secure messaging at the Naval Medical Center Camp Lejeune Family Medicine Clinic (which implemented the changes described here) compared to registration at seven other, demographically similar military treatment facilities selected by DHA over six and a half months.

OES aims to not only apply evidence-based insights, but also to evaluate the impact of these changes. Whenever possible, we aim to design an impact evaluation to evaluate the program change, but also employ other rigorous methods. With the DHA collaboration, it wasn’t possible to randomly assign some patients to the intervention and others to business as usual, so OES compared patient messaging registration at the Naval Medical Center Camp Lejeune Family Medicine Clinic, which made the changes, to registration at seven similar military treatment facilities.¹ Registrations were five times higher at Camp Lejeune, suggesting that we had correctly identified bottlenecks to patients enrolling in secure messaging.

We know that “ease” and “efficiency” (how long an individual expects something to take versus how long it actually takes) are core drivers of overall experience measures like satisfaction and even trust in the federal entity an individual is interacting with. In this case, OES changes were intended to make it easier for Shannon to connect with her doctor and manage her care. Hopefully, these changes contributed to higher satisfaction on the part of Camp Lejeune patients.

OES applies and evaluates evidence-based insights in many contexts other than healthcare. An example from 2015 shows how evidence can be collected and applied iteratively in a short time frame.

In this project OES worked with the Department of Education’s office of Federal Student Aid (FSA) to help student loan borrowers. Federal student loan borrowers who miss many payments are transferred to a private collections agency and face high financial penalties. To avoid these penalties, FSA allows borrowers to exit default by making a series of income-based payments, which can be as low as $5.

OES and FSA used an iterative impact evaluation to select the most effective of several email subject lines and the most effective message content to advertise the loan rehabilitation agreement. In the third round of the impact evaluation, almost a quarter of borrowers who had been randomly selected to be sent an email with the subject line “We’ve Scheduled Your Appointment” opened the email, an improvement of about 5 percentage points from the previous subject line. And, borrowers who had been randomly selected to be sent an email that suggested a time to phone the FSA call center were more likely to call within 3 weeks than those who received an email with no appointment time.

Both the DHA and FSA projects demonstrate that applying insights from the social and behavioral sciences can measurably improve the experience of interacting with government.