Below are resources we’ve developed through our work building evidence government-wide.
Evaluation Project Methods
Because results from OES tests impact the lives of millions of Americans, the quality of our work is of paramount importance. We follow the OES Evaluation Policy and six steps to ensure our findings are relevant and reliable.
Pre-Registration as a Tool for Strengthening Federal Evaluation
In order to ensure that evaluation findings are reliable and that statistical results are well founded, it is essential that evaluators commit to specific design choices and analytic methods in advance. By making these details publicly available – a practice known as preregistration – we promote transparency and reduce the risk of inadvertently tailoring methods to obtain certain results or selectively reporting positive results. This guidance paper describes the importance and benefits of preregistration and addresses concerns that Federal evaluators might have.
How to Use Unexpected and Null Results
Recent research shows that null results in federal evaluations are more common than we think, and occur for a variety of reasons. When agencies share both expected and unexpected results, we can learn about what programs work, what effect sizes are realistic, and improve Federal evaluations. This post dispels misconceptions about null results and highlights different uses and lessons from null results.
Unexpected Results Handout
Intergovernmental Personnel Act (IPA) Toolkit
OES uses the Intergovernmental Personnel Act (IPA) Mobility Program to bring on unique talent to support agency priorities and evidence needs. The purpose of this toolkit is to provide internal guidance on how OES has utilized the IPA Mobility Program in the past to fill highly technical positions in a quick and simple way where there might otherwise be capacity gaps. Personnel with IPAs at OES have served in roles as fellows, academic affiliates, and statistical methods support.
OES IPA Toolkit
How to Design Effective Communications
The United States government sends hundreds of millions of letters and emails to its citizens every year. These communications serve as a critical touchpoint by which citizens engage with our government programs and impact outcomes such as health insurance coverage, education enrollment and financing, tax compliance and benefits take-up, retirement security, and employment. Designing effective communications is therefore vital to ensuring that our government programs reach the people they are intended to serve. Since 2015, OES has completed more than 50 communication-specific interventions and evaluations with more than a dozen Federal agencies. Here’s what these tests have revealed about how to design effective communications.
Comms Design Tips