Describing candidate assessments and hiring outcomes at DOI
Exploring means and proportions of recruitment success, time-to-hire, and candidate characteristics for hiring efforts at the Department of the Interior
Person walking on a street with briefcase
What was the challenge?
Executive Order 13932 calls for the increased use of skills- and competency-based assessments in the federal hiring process and requires agencies to use alternative methods to assess applicants’ qualifications for federal jobs.
The Department of Interior (DOI) began implementing EO 13932 in 2021 by using one of two types of candidate assessment methods: (1) subject-matter experts (SMEs) to manually evaluate candidate resumes; or (2) USA Hire scored assessments of general competencies. These assessment methods were coupled with questionnaires typically used for candidates to self-assess their technical competencies.
What did we do?
A descriptive study can provide a baseline understanding of DOI’s implementation of EO-compliant assessments and how key hiring outcomes vary across the population of hiring actions. We compiled and analyzed administrative data to explore means and proportions of outcome variables of interest. The evaluation was designed to examine the use of different assessment tools used by DOI and key hiring outcomes over the past two years, and answer three research questions:
- How are candidate assessments being implemented at DOI?
- How long does the hiring process take and how often are positions filled for each candidate assessment method?
- Does the representation of female, people of color, and Veteran candidates change at key stages of the recruitment process within each assessment method?
We compiled data on hiring actions and applications to job vacancies at DOI from July 2020 to July 2022 to provide a description of the utilization of each of the three assessment methods and summarize key outcomes of the recruitment process. These outcomes include certification time, post-certification time to selection, recruitment success, and the representation of candidates at different stages of the recruitment process.
Figure 1. Implementation of hiring assessments across DOI
What did we learn?
How are candidate assessments being implemented at DOI?
DOI began implementing EO 13932 in January 2021. After January 2021, the use of self-assessments decreased in frequency as the department adopted the multiple hurdle approach by using manual and USA Hire assessments. Use of the three assessment methods over time varies across the five DOI bureaus included in the study.
How long does the hiring process take and how often are positions filled for each candidate assessment method?
The time from the close of vacancy announcements to the cert list being issued averages about 12 days for self-assessments, 27 days for manual assessments, and 15 days for USA Hire assessments.
The majority of the recruitment time is accounted for in the post-certification period for all assessment methods. The average time from the cert list being issued to selection is 37 days for self-assessments, 34 days for manual assessments, and 39 days for USA Hire assessments.
Overall, recruitment success for DOI ranges between 60% and 70%. Self-report assessments have the highest recruitment rates with 72% of hiring actions resulting in a selection, followed by hiring actions using USA Hire (67%) and those using manual assessments (62%).
Changes in representation of applicant characteristics varies across assessment methods and candidate characteristic groups:
Self-assessments: The representation of women and non-White or Hispanic applicants decreased at each stage of recruitment process. Representation of Veterans increased at the cert list stage, but decreased at the selection stage.
Manual assessments: The representation of women increased between the application and selection stages, while the representation of non-White or Hispanic applicants decreased during the recruitment process. The representation of Veterans increased at the cert list and selection stage compared with the application stage.
USA Hire assessments: The proportion of women applicants increased between the application and cert list stages, and the proportion of non-White or Hispanic applicants decreased between those stages. Representation of Veterans increased between the application and selection stages.
Future exploration could examine the impact of candidate assessment tools on the hiring process through a randomized evaluation. This type of study could provide insights into whether any differences in hiring outcomes are likely due to the use of one assessment method over another, rather than being driven by hiring officials’ preferences or other parts of the hiring process.
Verify the upload date of the Analysis Plan on GitHub.