Person holding an iphone

Person holding an iphone


View the PDF Version

Key findings

We randomized child care providers to receive either a message that simply notified them that they were enrolled in a text message program (“standard message”) or a message that explained why they were enrolled in the text message program (“transparent message”). We find no difference in rates of opting out of receipt of further messages across the groups. However, a large volume of delivery failures and low opt-out rates in both groups limited our ability to detect small differences in opt-out rates between these groups. Only a third of the text messages sent to providers were successfully delivered (1,096 out of 3,268 texts). Among providers who had text messages successfully delivered, less than 3% opted out of the default option to receive future text messages — reinforcing the evidence on the effectiveness of defaults.

Agency priority

As individuals increasingly rely on digital technologies to navigate their day-to-day lives, government agencies have made it a priority to incorporate them into their service delivery models.1 The Department of Health and Human Services’ Administration for Children and Families and Minnesota’s Department of Human Services (MN DHS) used digital outreach in the form of text messages as part of a larger effort to deliver information and reminders about their Child Care Stabilization Base Grant (CCSBG) program to eligible child care providers. The CCSBG program was funded through a $24 billion American Rescue Plan (ARP) allocation attempting to stabilize the child care labor market in response to the COVID-19 pandemic. MN DHS implemented the CCSBG program in Minnesota and awarded providers grants of up to $400 per full-time equivalent employee each month.2

MN DHS was required by a Federal Communications Commission ruling to provide text message recipients with the option to opt out of text messages.3 MN DHS sought ways to increase the share of providers who continued to receive messages so they could keep providers informed on priority program updates.

Program change description

Research shows that individuals are more likely to agree to an option when it is set as a default.4 Default options are initiated automatically and do not require additional steps to pursue; individuals who do not want the default option can take action to opt out of it.5 Because it takes more effort to opt out of a default than to go along with one, defaults are powerful tools in promoting desired behaviors.6 We worked with MN DHS to leverage the power of defaults in their text message outreach by enrolling providers into their texting program by default.

How the default option is presented may influence the likelihood that an individual opts out of that choice. Transparent defaults draw attention to the fact that a default option has been selected and explain how the default might impact someone who agrees to it. In doing so, transparent defaults may make default options seem more trustworthy and compelling.7,8,9

Evaluation design

In this evaluation, we compared the effectiveness of standard default messages and transparent default messages on providers’ decisions to opt out of MN DHS’ texting program. We randomized the version of the opt-out message that MN DHS would send to 3,374 childcare providers (associated with 3,268 phone numbers), who were identified in state licensure data as likely eligible for CCSBG. Three days after the February CCSBG application period opened (February 17, 2023), MN DHS sent half the providers in the evaluation (1,633 provider phone numbers) the transparent message and the other half (1,635 providers phone numbers) the standard message. The standard message (Image 1) informed providers that MN DHS could share information about CCSBG over text and included a link to opt out of these messages. The transparent message (Image 2) included the standard message and an additional explanation for why providers had been enrolled to receive MN DHS’ text messages.

Image 1. Standard text message sent to childcare providers Standard text message sent to childcare providers

Image 2. Transparent text message sent to childcare providers Transparent text message sent to childcare providers

Analysis of existing data

Data from MN DHS’ text message delivery service were used to compare opt-out rates between the standard and transparent message groups.10 The data include information on whether the text message was delivered and, if so, whether the provider opted out of future messages. If the text message was not delivered, the text message service shared information on whether the failure was due to the provider being unreachable or because of throttling.11

Results

We find no difference in opt-out rates between the standard and transparent message groups. The opt-out rate was 1.0% in the standard message group and 0.9% in the transparent message group, a null effect of 0.1% (p = 0.72, 95% CI [ -0.78, 0.53]). However, a large volume of delivery failures, differences in throttle rates across the standard and transparent message groups, and low opt-out rates could affect our ability to detect small differences in opt-out rates between these groups.12 These factors could also bias our estimates of the effect of the transparent message. Implementation issues with text message delivery meant that only a third of the text messages were delivered to providers. In the standard message group, 34.4% of the text messages were delivered successfully, compared to 32.7% in the transparent message group, a difference of 1.7% (p=0.31, 95% CI [-4.9, 1.5]), which was not statistically significant. Although successful delivery of messages did not differ between the groups, there were statistically significant differences in throttle rates; the percentage of providers whose texts were throttled was 47.1% among those in the standard message group and 50.7% among those in the transparent message group, an effect of 3.6 percentage points (p = 0.04, 95% CI [0.2, 7.0]). This may indicate underlying differences in the types of people whose messages were delivered, even though overall receipt did not statistically differ between the groups.

Opt-out rates were low overall. Less than 1% of providers (0.9%) in the evaluation opted out of future messages. Among providers who had successfully delivered messages the opt-out rate was 2.7%.

Figure 1. Transparent defaults did not affect opt-out rates or successful delivery of messages, but throttling was greater in the transparent message group Bar chart showing that transparent defaults did not affect opt-out rates or successful delivery of messages, but throttling was greater in the transparent message group

Implications

Although text messages can be an effective tool to communicate with large numbers of individuals, throttling and other factors can affect whether outgoing text messages reach their intended audience.13 Agencies can make evidence-informed decisions to improve the effectiveness and integrity of their text message programs by collecting data on the failure rates and delivery status of sent messages.

The low opt-out rate among providers who had messages delivered corroborates a well-established finding that individuals conform to defaults.14 Asking individuals to opt out of text messages rather than opt into them can increase subscription rates, enabling the sharing of information on important public benefits. While we did not find evidence that providing an explanation for a default impacts opt-out rates, we cannot rule out this possibility since throttle rates differed between the transparent message and standard message groups.

Notes:

  1. Joseph R. Biden Jr. Executive Order. “Transforming Federal Customer Experience and Service Delivery To Rebuild Trust in Government, Executive Order 14058 of December 13, 2021 .” Federal Register Vol.86, no. 239 (Thursday, December 16, 2021): 71357-71366.
  2. Between February and June 2023, the CCSBG amount varied from $205 to $400 per full-time equivalent employee.
  3. Strengthening the Ability of Consumers To Stop Robocalls, 47 CFR 64.1200 (2024)
  4. Jachimowicz, J. M., Duncan, S., Weber, E. U., & Johnson, E. J. (2019). When and why defaults influence decisions: A meta-analysis of default effects. Behavioural Public Policy, 3(2), 159-186.
  5. Thaler, Richard H., and Cass R. Sunstein. 2008. Nudge: Improving decisions about health, wealth, and happiness. New Haven, CT: Yale Univ. Press.
  6. Paunov, Y., Wänke, M., & Vogel, T. (2019). Ethical defaults: which transparency components can increase the effectiveness of default nudges?. Social Influence, 14(3-4), 104-116.
  7. Bruns, H., Kantorowicz-Reznichenko, E., Klement, K., Jonsson, M. L., & Rahali, B. (2018). Can nudges be transparent and yet effective?. Journal of Economic Psychology, 65, 41-59.
  8. Loewenstein, G., Bryce, C., Hagmann, D., & Rajpal, S. (2015). Warning: You are about to be nudged. Behavioral Science & Policy, 1(1), 35-42.
  9. Paunov, Y., Wänke, M., & Vogel, T. (2019). Transparency effects on policy compliance: Disclosing how defaults work can enhance their effectiveness. Behavioural Public Policy, 3(2), 187-208.
  10. Providers could opt out of text messages at any time. We pre-specified that we would analyze the effects of the opt-out text messages until texts were sent encouraging providers to apply (February 21, 2023), but we use outcomes from when the data were pulled on February 28, 2023 since dates for when opt- outs occurred are not available. See the analysis plan.
  11. Throttling refers to text messages that were restricted due to the volume or character count of the texts exceeding a limit set by the delivery provider. Throttling can cause delays in message delivery and even prevent delivery altogether.
  12. Due to the large number of messages that were not received, we deviated from the analysis plan in two ways and consider the results of this analysis to be exploratory rather than confirmatory. First, we included all providers in the analysis other than those with duplicate phone numbers (rather than dropping non-recipients, as we had pre-specified) because Due to the large number of messages that were not received, we deviated from the analysis plan in two ways and consider the results of this analysis to be exploratory rather than confirmatory. First, we included all providers in the analysis other than those with duplicate phone numbers (rather than dropping non-recipients, as we had pre-specified) because receipt of text messages may have been correlated with treatment. If providers in the transparent default group were less likely to receive a text message than providers in the standard default group, then these providers would have had fewer opportunities to opt out in the first place; potentially biasing our estimate of the transparent default text’s effect. Second, we performed an additional analysis examining the effect of treatment on delivery status to assess this possibility.
  13. A2P 10DLC refers to a system in the U.S. that allows businesses to send Application-to-Person (A2P) messaging via standard 10-digit long code (10DLC) phone numbers. A2P 10DLC provides better delivery quality and lower filtering risk than long code SMS, using the same phone numbers.
  14. Jachimowicz, J. M et al., When and why defaults influence decisions: A meta-analysis of default effects, 159-186.