Building evidence

Integrating Health Care Services in Nigeria

A feasibility pilot to increase access to voluntary family planning services.

Timely Wage Reporting Among SSI Recipients

A reminder encouraged SSI recipients to self-report changes in wages earlier, potentially reducing overpayment.

Increasing Vaccine Uptake Through Performance Feedback

Analysis Plan Registration

Characterizing Nonresponse Bias in the American Housing Survey (AHS)

Analysis Plan Registration

How to Design Effective Communications - What has OES learned?

Increasing utilization of family planning services in Mozambique through an SMS Intervention

Analysis Plan Registration

Using Proactive Communication to Increase College Enrollment for Post-9/11 GI Bill Beneficiaries

A general proactive email, encouraging increased postsecondary enrollment, influenced the enrollment intensity of Post-9/11 GI Bill users

Increasing School Attendance in Seattle

Proactive communications do not change attendance outcomes

Using Social Norms to Decrease Energy Use in Public Housing

Letters with feedback on energy use and comparisons to efficient neighbors did not reduce energy use among public housing residents

Increasing Vaccine Uptake Among Veterans at the Atlanta VA Health Care System

A modified clinical reminder did not increase patient vaccine uptake.

Increasing Survey Response to Improve USAGov Email Content

A timeline making government operations transparent increased survey response and an early message of gratitude decreased survey response.

Increasing Flu Vaccine Uptake Among Veterans at the New York Harbor VA

Emails incorporating behavioral insights were not effective in increasing flu shot uptake.

Decreasing Energy Costs in Federally Assisted Housing

Sending flyers with energy-saving tips did not change energy use.

Increasing Flu Vaccine Uptake Among Veterans at the St. Cloud VA

Postcards incorporating behavioral insights were not more effective than a basic informational postcard.

Matching Government Surplus Property with Interested Buyers

Customized emails reduced no-bids and increased overall revenue on gsaauctions.gov.

Increasing Vaccine Uptake Among Seniors

Sending postcard reminders increases vaccination rates among elderly

Increasing Immunization Compliance in Schools

Data visualization for school-specific immunization compliance reports did not impact compliance rates.

Testing Variations of Maternal Immunization Messages

Variations of an ad did not lead to detectable differences in site visits.

Encouraging Flu Vaccine Uptake Among Pregnant Patients at Duke

Targeted email outreach via an electronic medical record messaging system did not increase uptake of the flu vaccine among pregnant women.

Increasing Flexible Spending Account (FSA) Participation Among Government Employees

Targeted email outreach to GSA employees did not increase enrollment in Flexible Spending Accounts.

Project SOAR

Analysis Plan Registration

Increasing Use of Patient Generated Health Data through Patient Reminders

Sending reminder messages focused on provider accountability increased patients’ use of electronic blood glucose flow sheets.

Increasing Use of Patient Generated Health Data through Provider Encouragement

A virtual provider training and encouragement increased use of electronic blood glucose flow sheets.

Increasing Veteran Health Care Benefits Enrollment

Emails to transitioning servicemembers about VA health care benefits did not affect enrollment.

Increasing AHS Response Rates

A second advance letter does not significantly improve outcomes.

Increasing TSP Enrollment among USPS Employees

Testing mailings to increase USPS retirement enrollment and contributions

Increasing SSI Uptake Among a Potentially Eligible Population

Letters to adults 65 or older increased SSI awards by 340%.

Increasing ART Adherence and Retention in Care for HIV Positive Women in Ethiopia

Phone calls delivered to HIV+ women did not increase ART adherence.

Increasing FAFSA Completion Among Public Housing Residents: Seattle and King County

A pair of communications do not change FAFSA completion rates

Increasing FAFSA Completion Among Public Housing Residents: NYCHA

A series of communications do not change FAFSA completion rates

Increasing FAFSA Renewal Rates

Reminder emails sent to current students led to an increase of approximately 250,000 FAFSA submissions.

Preventing Malaria During Pregnancy

Redesigned referral process did not increase antenatal care (ANC) attendance for pregnant women.

Identifying Students for the Education for Homeless Children and Youth Program

Identifying Homeless Students for Educational Support Services

Increasing Pre- and Post-Natal Care Uptake Among Medicaid-Eligible Mothers

Home Visiting Uptake Among Medicaid-Eligible Mothers.

Reducing Overprescribing of Quetiapine in Medicare Part D

Peer comparison letters for high prescribers of quetiapine reduce prescription volume and improve guideline conformity of prescription fills.

Encouraging Action Among Students At Risk to Withdraw

Reminding students to reapply for financial aid or begin loan repayment decreased postsecondary enrollment by one percentage point.

Improving Employment Services for UI Claimants in Oregon

Requiring personal employment plans did not change the employment rate

Increasing Voluntary Submission of Demographics Information with USAJOBS Applications

The combination of default settings and simplified language increases voluntary submission of demographics information by 19%.

Prompting Decisions on Retirement Saving

Encouraging higher TSP contribution rates during new employee orientation showed no detectable effect on the likelihood of contributing 5% or more.

Improving Capacity of Federal Managers through Project Management

A 16-week training course for Federal Supervisors shows potential to improve organizational culture and performance.

Encouraging Microloan Applications from Women-Owned Farms

A single direct mailer had no statistically reliable effect on the likelihood that female farmers who received it would apply for microloans.

Increasing Military Patient Use of Secure Messaging through Email Reminders

Emails sent after an appointment-based intervention had no statistically reliable impact on patient registration for secure messaging.

Increasing Military Patient Use of Secure Messaging through Personal Appeal and Assistance

Provider encouragement and on-the-spot assistance increased patient registrations for secure messaging.

Improving Re-entry to Reduce Recidivism

Empowering the re-entry population to thrive in their communities by developing a handbook that articulates concrete steps.

Increasing TSP Enrollment Rates through Active Choice

Presenting TSP enrollment as an active choice during in-processing increased enrollment by 6-7%.

Increasing Newsletter Subscription Rates of Active Duty Servicemembers

Clear communications emphasizing active choice increase subscription rates.

Encouraging Engagement of Military Caregivers

Emails to caregivers of ill or wounded servicemembers increased awareness of available benefits.

Increasing FAFSA Completion by HUD-Assisted Youth

Sending letters to HUD-assisted youth had no significant effect on FAFSA completion rates.

Increasing Enrollment in the Small Business Administration's Learning Center

Simplifying a registration form increased learning course completion and data collection.

Improving Voter Participation in Local USDA Elections

Sending redesigned ballot materials to eligible farmers increased voter participation by 2.9%.

Decreasing Non-responses to NSLP Income Verification Requests

Simplifying the income verification letter for National School Lunch Program recipients increased response rate by 2.1%.

National School Lunch Program Eligibility Verification

Timely communication with beneficiaries has mixed effects on response rates.

Increasing Public Understanding of Climate Change Indicators

Simplifying visual graphics increases comprehension of climate change indicators.

Increasing Tax Filing Among Lower-Income Individuals

Sending postcards effectively increased uptake of Earned Income Tax Credit among eligible population.

Increasing Health Insurance Enrollment Among Uninsured Americans

Modified emails encourage open rates about health insurance enrollment.

Encouraging Medicare Beneficiaries to Self-Identify as ACO Patients

Sending a letter to Medicare beneficiaries increased voluntary alignment to Accountable Care Organizations.

Increasing Patient Use of Health IT

Revising patient instructions following appointments led to 9.98% increase in online health portal activation.

Facilitating Individual Action to Build Stronger Community-Police Relationships

Materials enable citizens to facilitate conversations and action planning around better community policing.

Increasing Student Loan Rehabilitation Rates for Defaulted Borrowers

Emphasizing the negative consequences of loan default increases entry into loan rehabilitation programs.

Increasing IDR Re-certification Among Student Borrowers

Including required payment amount increased IDR recertification by 8.4%.

Increasing IDR Applications Among Eligible Student Loan Borrowers - Scale Up

Email scale-up resulted in more than 6,000 borrowers signing up for income driven repayment plans.

Increasing IDR Applications Among Eligible Student Loan Borrowers through Targeted Messages

Emails increase application submissions for income driven repayment plans.

Increasing Access to PSLF Benefits Among Eligible Population

Emails increased Public Service Loan Forgiveness form submissions three-fold without increasing the error rate.

Increasing Military Retiree Reactivation of Online MyPay Accounts

Emails increased myPay account reactivation by military retirees.

Encouraging Flu Vaccine Uptake among Medicare Beneficiaries

Letters sent to Medicare beneficiaries increased uptake of the flu vaccine.

Communicating Employment Supports to Denied Disability Insurance Applicants

Letters to denied disability applicants about employment support programs did not reduce appeals.

Reducing Summer Melt

Eight personalized text messages to low-income students resulted in a 6% increase in college enrollment.

Reducing Self-reporting Errors by IFF Form Users

Adding the signature box resulted in an extra $1.59 million paid to government by vendors.

Industrial Funding Fee Reports Update

A confirmation prompt reduced financial self-reporting errors initially, but the effect did not persist in subsequent periods.

Increasing Post-Grace Period Loan Payments by Student Borrowers

Emails sent to student loan borrowers at risk of default increased payments.

Increasing Satisfaction Survey Response Rate from Federal Tenants - Time of Day

Emails sent at lunchtime saw 1.5% more Federal employees click through to a workplace survey.

Increasing Satisfaction Survey Response Rate from Federal Tenants - Subject Lines and Day of Week

Emails with subject lines that included the word "please" increased Federal employees responses to a workplace survey.

Increasing Satisfaction Survey Response Rate from Federal Tenants - Calendar Invite

Emails including a calendar invite reminder led to increased survey responses from Federal employees.

Increasing Chapter 36 Benefits Uptake by Veterans

Emails informing veterans about education and career benefits increased applications.

Reducing Inappropriate Prescribing of Schedule II Substances in Medicare Part D

Sending letters to potential overprescribers did not influence Schedule II prescribing activities.

Encouraging Account Transfers by Securities Holders

Scheduling appointments to transfer securities increased call-in rates by 2.37%.

Increasing Farmer Microloan Applications

A new letter increases farmers' access to credit by 22%.

Increasing Online TSP Enrollment by Servicemembers

A single email to military Servicemembers nearly doubled the rate of savings plan enrollment.

Increasing Servicemember On-Base TSP Enrollment

Explaining benefits of TSP during in-processing doubled the number of Servicemembers enrolling.

Increasing Duplex Printing Among Federal Workforce

A digital pop up significantly lowers office-wide printing costs.

Increasing Health Insurance Enrollment Among Uninsured Americans

A letter increased health insurance enrollment by 7.15%.

Increasing IDR Applications Among Delinquent Student Borrowers

Sending emails to delinquent borrowers positively impacted IDR application completion.

Increasing Servicemember Roth TSP Re-Enrollment

One email enables 22% more Servicemembers to re-enroll in their Thrift Savings Plans.

Increasing Debt Repayment by Delinquent Lenders

A modified letter sent to delinquent borrowers increased utilization of online payments.