OES develops resources to help agencies learn from leading evaluation practices.

Effect Size and Evaluation: The Basics

An impact evaluation aims to detect and measure the effect of a program or policy on a priority outcome. To plan for an evaluation, we need to decide how large or small an effect we want to be able to detect. This important decision will influence all aspects of evaluation planning, including budget, operations, duration, and sample. This resource explains what effect sizes are and their importance in designing an evaluation.

Effect Size Guide

Evidence Reviews to Support Evidence-Based Policymaking

The Foundations for Evidence-Based Policymaking Act of 2018 (the Evidence Act) directs Federal agencies to develop evidence to support policymaking. A crucial component of developing evidence is understanding what evidence already exists. This helps ensure that key learnings are incorporated into new and existing programming, and that the resources available for evidence-building activities are targeted towards areas where there are bigger evidence gaps. This resource introduces a framework for how to conduct a review of existing evidence, and provides additional resources for those seeking to conduct more systematic reviews.

Evidence Reviews Guide

Evaluation Working Group Case Study

GSA formed an Evaluation Working Group (EWG) in 2020 to provide feedback on the agency’s FY22 Annual Evaluation Plan and other Evidence Act deliverables. Meeting three times between April and June, the group helped develop criteria for significant evaluations and drafts of the Annual Evaluation Plan and GSA Evaluation Policy. The case study summary describes the EWG’s purpose and how they went about their work, as well as provides materials and agendas for forming similar groups at other agencies.

Evaluation Working Group Case Study

Appendix: Planning Materials for an Evaluation Working Group

Preregistration as a Tool for Strengthening Federal Evaluation

In order to ensure that evaluation findings are reliable and that statistical results are well founded, it is essential that evaluators commit to specific design choices and analytic methods in advance. By making these details publicly available – a practice known as preregistration – we promote transparency and reduce the risk of inadvertently tailoring methods to obtain certain results or selectively reporting positive results. This guidance paper describes the importance and benefits of preregistration and addresses concerns that Federal evaluators might have.

Preregistration guide

How to Use Unexpected and Null Results

Recent research shows that null results in federal evaluations are more common than we think, and occur for a variety of reasons. When agencies share both expected and unexpected results, we can learn about what programs work, what effect sizes are realistic, and improve Federal evaluations. This post dispels misconceptions about null results and highlights different uses and lessons from null results.

Unexpected results guide