OES Evaluation Process & Policy
Calculating Standard Errors Guide
This guidance paper describes OES’s preferred method for calculating parametric standard errors for OLS regressions — in particular, the reasons for using HC2 standard errors — and how to calculate them in R and Stata. OES often analyzes the results of RCTs by estimating a parametric statistical model — typically an ordinary least squares (OLS) regression — where one of the parameters represents the effect of a treatment or intervention. In order to infer whether this treatment effect is statistically significant, we typically depend on an estimate of the standard error associated with this parameter.
Multiple Comparison Adjustment Guide
This guidance paper describes OES's approach to adjusting for multiple comparisons. When evaluators run multiple statistical tests -- for example, looking at multiple possible outcomes of a program or intervention, or testing multiple versions of an intervention -- it is important to account for this when reporting and interpreting the results.
Preregistration in External Registries Guide
This guidance paper describes the importance and benefits of preregistration and addresses concerns that Federal evaluators might have. In order to ensure that evaluation findings are reliable and that statistical results are well founded, it is essential that evaluators commit to specific design choices and analytic methods in advance. By making these details publicly available -- a practice known as preregistration -- we promote transparency and reduce the risk of inadvertently tailoring methods to obtain certain results or selectively reporting positive results.