Pilot initiative informs government on how to best support the Federal workforce

What is the Government-wide Pulse Survey?

In October 2021, the federal government launched a pilot pulse survey initiative, which invites Federal employees to share their thoughts via 3-4 questions to help inform the Administration’s actions on how best to support the Federal workforce. This pilot is a collaborative effort of the President’s Management Council, together with the Office of Management and Budget, the Office of Personnel Management, and the General Services Administration.

In total, the pilot initiative will include three pulse survey rounds, each approximately two to three months apart. These surveys will be sent to approximately two million civilian employees of the 24 CFO Act agencies. The pulse surveys cover three themes: employee engagement, equity and inclusion, and the reentry process. Each round also includes embedded survey experiments to build evidence on the effective implementation of large-scale pulse surveys.


Pulse Survey 1 Analysis Plan (PDF)

Pulse 1 Survey Experiment Results

The first pulse survey round, launched in October 2021, included two embedded experiments aimed at evaluating differences in response rate by (1) messenger, and (2) survey theme.

Response rates by messenger

In October 2021, civilian employees of the 24 CFO Act agencies received an email inviting them to participate in the first pulse survey of the Government-wide Pulse Survey initiative. Prior to sending the email invitations, all employees were randomly assigned to receive an email:

  • Signed by the Deputy Director for Management at OMB, or
  • Signed by their agency’s President’s Management Council representative.

Emails signed by the Deputy Director for Management at OMB included the OMB logo at the top; emails signed by agency PMC members included the agency logo at the top of the message. The content of both emails was identical, and included a short description of the initiative and a link to complete the survey.

We evaluated differences in survey response rates by messenger in the five days after sending the initial email invitation. This analysis was conducted for 23 of the 24 agencies (N = 1,320,587) included in the sample because the initial email invitations for one agency did not reach its employees. We found that emails signed by agency PMC members yielded a 0.1 percentage point higher response rate–a small, but statistically significant increase (p = 0.05; 95% CI [0.00, 0.002]).

Response rates by survey theme

Prior to the first pulse round, all employees were also randomly assigned to one of the three survey themes: engagement, equity and inclusion, or reentry. The survey questions differed across themes, but each survey included 3-4 questions relevant to its theme. The email invitation included one sentence corresponding with the survey theme, but other features of the email–such as the link and the main text–did not differ by survey theme.

We evaluated differences in survey response rates by theme in the five days after sending the initial email invitation. This analysis was also conducted for 23 of the 24 agencies (N = 1,320,587) included in the sample. Overall, 16.0% of invited employees responded to the reentry survey, compared to 15.3% for the engagement survey, and 13.9% for the equity and inclusion survey. These differences in response rate are all statistically significant (F = 399.92, p < .001).

This analysis focuses on response in the first five days following the initial email invitation. Since the survey was open for nine days, however, the final response rates are higher (see more here). Nevertheless, the overall trends in response by survey theme remain the same.


Pulse Survey 2 Analysis Plan (PDF)

Pulse 2 Survey Experiment Results

The second pulse survey, launched in February 2022, included an embedded experiment in the reentry survey. All employees who were randomly assigned to the reentry survey theme prior to pulse 1 again received an invitation to complete the reentry survey in pulse 2. Like the first pulse survey, the second pulse also included three questions, although questions differed between the two rounds.

Everyone who started the reentry survey in the second pulse round was randomly assigned to see one of two questions:

  1. If I found a job elsewhere with more workplace flexibilities or remote options, I would take it.
  2. If I found a job elsewhere with more pay or better benefits, I would take it.

Both questions were measured on a five-point scale from “strongly disagree” to “strongly agree.” We evaluated differences in agreement between the two questions among 126,540 respondents who answered one of the two questions. We found that, on average, respondents were more likely to express that they would take a job with more pay/benefits than one with more flexibility. Average agreement with “I would take another job with more flexibility/remote options” was 3.56 on a five-point scale. In comparison, average agreement with “I would take another job with better pay/benefits” was 3.70 on a five-point scale. This difference of 0.14 pp is statistically significant (p < .001; 95% CI [-0.15, -0.12]).


Pulse Survey 3 Analysis Plan (PDF)

Pulse 3 Survey Experiment Results

Results forthcoming

Year

2022

Agency

General Services Administration, Office of Management and Budget, Office of Personnel Management

Domain

Government Operations