Government-wide Pulse Survey

What is the Employee Voice Initiative?

In October 2021, the federal government launched a pilot pulse survey initiative, which invited federal employees to share their thoughts via 3-4 questions to help inform the administration’s actions on how best to support the federal workforce. This pilot was a collaborative effort of the President’s Management Council (PMC), together with the Office of Management and Budget, the Office of Personnel Management, and the General Services Administration. In total, the pilot initiative included three pulse survey rounds, each approximately two to three months apart. These surveys were sent to approximately two million civilian employees of the 24 CFO Act agencies. The pulse surveys covered three themes: employee engagement, equity and inclusion, and the post-COVID reentry process. Each round also included embedded survey experiments to build evidence on methods of increasing response rates as well as on differences in employee mindsets.

Pulse 1 Survey Experiment Results

The first pulse survey round, launched in October 2021, included two embedded experiments aimed at evaluating differences in response rate by (1) messenger, and (2) survey theme.

Response rates by messenger

In October 2021, civilian employees of the 24 CFO Act agencies received an email inviting them to participate in the first pulse survey of the Employee Voice Initiative. Prior to sending the email invitations, all employees were randomly assigned to receive an email:

• Signed by the Deputy Director for Management at OMB, or
• Signed by their agency’s President’s Management Council representative.

Emails signed by the Deputy Director for Management at OMB included the OMB logo at the top; emails signed by agency PMC members included the agency logo at the top of the message. The content of both emails was identical, and included a short description of the initiative and a link to complete the survey.

We evaluated differences in survey response rates by messenger in the five days after sending the initial email invitation. This analysis was conducted for 23 of the 24 agencies (N = 1,320,587) included in the sample because the initial email invitations for one agency did not reach its employees. We found that emails signed by agency PMC members yielded a 0.1 percentage point higher response rate–a small, but statistically significant, increase (p = .05; 95% CI [0.00, 0.002]). The results suggest that choosing a known messenger may be one way to marginally increase response rates in future government survey efforts.

Response rates by survey theme

Prior to the first pulse round, all employees were also randomly assigned to one of the three survey themes: employee engagement, equity and inclusion, or reentry. The survey questions differed across themes, but each survey included 3-4 questions relevant to its theme. The email invitation included one sentence corresponding with the survey theme, but other features of the email–such as the link and the main text–did not differ by survey theme.

We evaluated differences in survey response rates by theme in the five days after sending the initial email invitation. This analysis was also conducted for 23 of the 24 agencies (N = 1,320,587) included in the sample. Overall, 16.0% of invited employees responded to the reentry survey, compared to 15.3% for the employee engagement survey, and 13.9% for the equity and inclusion survey. These differences in response rate are all statistically significant (F = 399.92, p < .001). The differential response rates by survey theme warrant further research to better understand why employees were less likely to respond to the equity and inclusion survey, and how to promote engagement in the future.

This analysis focuses on response in the first five days following the initial email invitation. Since the survey was open for nine days, however, the final response rates are higher (see more here). Nevertheless, the overall trends in response by survey theme remain the same.

Verify the upload date of our analysis plan on GitHub.

Pulse 2 Survey Experiment Results

The second pulse survey, launched in February 2022, included an embedded experiment in the reentry survey. All employees who were randomly assigned to the reentry survey theme prior to pulse 1 again received an invitation to complete the reentry survey in pulse 2. Like the first pulse survey, the second pulse also included three questions, although questions differed between the two rounds.

Everyone who started the reentry survey in the second pulse round was randomly assigned to see one of two questions:

1. If I found a job elsewhere with more workplace flexibilities or remote options, I would take it.
2. If I found a job elsewhere with more pay or better benefits, I would take it.

Both questions were measured on a five-point scale from “strongly disagree” to “strongly agree.” We evaluated differences in agreement between the two questions among 126,540 respondents who answered one of the two questions. We found that, on average, respondents were more likely to express that they would take a job with more pay/benefits than one with more flexibility. Average agreement with “I would take another job with more flexibility/remote options” was 3.56 on a five-point scale, in which 5 reflects “strongly agree.” In comparison, average agreement with “I would take another job with better pay/benefits” was 3.70 on a five-point scale. This difference of 0.14 points is statistically significant (p < .001; 95% CI [-0.15, -0.12]). The results from this pilot suggest that pay and benefits may influence expressed willingness to change jobs more than workplace flexibility. However, it is important to note that these results are specific to the context in which the survey experiment was conducted–a period of unprecedented workplace flexibility and economic uncertainty. Whether these results would replicate in a context with less workplace flexibility or greater economic security is unknown.

Verify the upload date of our analysis plan on GitHub.

Pulse 3 Survey Experiment Results

The third pulse survey was launched in March 2022 and included an embedded experiment in the equity survey. As in the first and second surveys, employees who were initially randomly assigned to the equity survey theme prior to Pulse 1 received an email invitation to participate in the third equity survey. The survey included three questions, and every employee who began the equity survey was randomly assigned to see one of two versions of the last question:

1. Employees like me are given the support to succeed here.
2. Employees with different backgrounds are given the support to succeed here.

Both questions were measured on a five-point scale from “strongly disagree” to “strongly agree,” where 5 reflects “strongly agree” in our analysis.

In total, 113,606 employees responded to the equity survey, for a response rate of 17.5%.¹ Differences in agreement between the two question versions were evaluated for four groupings: White versus non-White employees, Black/African-American versus non-Black employees, Hispanic/Latino versus non-Hispanic/Latino employees, and female versus male employees.²

Overall, respondents agreed more strongly that “employees from different backgrounds” are given the support to succeed than similar respondents agreed with “employees like me” are given the support to succeed (3.79 vs. 3.54 on a 5-point scale, or a 7% increase, p < 0.001; 95% CI [-0.27, -0.24]; N = 100,519).³ To analyze demographic differences in employees’ perceptions of support at work, the levels of agreement with the two versions of the question were compared in four separate analyses: White vs. non-White, Black vs. non-Black, Latino/Hispanic vs. non-Latino/Hispanic, and male vs. female employees. The results of this experiment within each of these subgroups, as well as among the total sample, is shown in Table 1.

Table 1. Perceptions of Support for Federal Employees

Sample	Agreement that “Employees like me” are given support to succeed (1-5 scale)	Agreement that “Employees from different backgrounds” are given support to succeed (1-5 scale)	Percent increase in agreement for “Employees from different backgrounds” over “Employees like me” baseline
White	3.56	3.89	9%
Non-White	3.49	3.59	3%
Black	3.47	3.48	0%
Non-Black	3.54	3.84	8%
Latino/Hispanic	3.51	3.62	3%
Non-Latino/ non-Hispanic	3.54	3.81	8%
Male	3.56	3.87	9%
Female	3.51	3.71	6%
Total Sample	3.54	3.79	7%

Of note, the difference in White employees’ perceptions of support for employees of different backgrounds versus employees like themselves was significantly larger than among non-White employees: among White employees, the average level of agreement with the “different backgrounds” frame was 3.89, compared to 3.56 with the “employees like me” frame—9% higher. This suggests that White employees perceive employees of different backgrounds to have more support than employees like themselves. Among non-White employees, the average level of agreement with the “different backgrounds” frame was 3.59, compared to 3.49 with the “employees like me” frame—3% higher (p < 0.001; 95% CI [-0.264, -0.188]; N = 100,519).

Conversely, Black and Latino/Hispanic employees perceived a smaller gap between the level of support given to employees like themselves and employees of different backgrounds than non-Black and non-Latino/Hispanic employees, respectively. Among Black employees, there was a 0.3% difference in perceived support for employees from different backgrounds versus employees like themselves: the average level of agreement with the “different backgrounds” frame was 3.48 versus 3.47 with the “employees like me” frame. In comparison, the difference in perceived support for employees from different backgrounds versus employees like themselves was 8% higher among non-Black employees: the average agreement with the “different backgrounds” frame was 3.84, compared to 3.54 with the “employees like me” frame (p < 0.001; 95% CI [0.228, 0.337]; N = 100,519).

Similarly, this gap in perceived support was 3% higher among Latino/Hispanic employees but 8% higher among non-Latino/Hispanic employees (p < 0.001; 95% CI [0.090, 0.233]; N = 100,519). Among Latino/Hispanic employees, the average agreement with the “different backgrounds” frame was 3.62, compared to 3.51 with the “employees like me” frame. Meanwhile among non-Latino/Hispanic employees, average agreement with the “different backgrounds” frame was 3.81, compared to 3.54 with the “employees like me” frame.

Women also perceived a smaller difference in the levels of support for employees of different backgrounds compared to employees like themselves than men (p < 0.001; 95% CI [0.081, 0.152]; N = 100,519). Among women, the average agreement for employees from different backgrounds was 3.71, compared to 3.51 for employees like themselves (6% higher), whereas among men, the average agreement for employees from different backgrounds was 3.87, compared to 3.56 for employees like themselves ( 9% higher).

Taken together, the results suggest that White employees and men believe that higher levels of support exist for employees from diverse backgrounds than for employees like themselves. Meanwhile Black, Hispanic, and female employees perceive that more similar levels of support are available for themselves and for employees from different backgrounds. This misalignment in the perceptions of the support available to employees from diverse backgrounds could have implications for workplace diversity, equity, and inclusion efforts and points to areas for further research.

Verify the upload date of our analysis plan on GitHub.

View Abstract (PDF)

¹ 16,009 employees were excluded from the analysis, because they received a different survey theme from the earlier evaluations due to an error that occurred when matching employees to the survey theme they received in past evaluations. Including these employees in the analysis does not meaningfully change the results.

² The category designations for race, ethnicity, and gender reported here match the labels used in the OPM EHRI database.

³ Note: Models included inverse probability weighting, and those with a likelihood of response above the 95% percentile were trimmed from the sample, which affected 13,087 observations. Analyses on the overall difference between the “employees like me” frame and “different backgrounds” frame were not included in the published analysis plan, but are provided for comparison.