A recent randomized evaluation between the Office of Evaluation Sciences and the White River Junction Veterans Affairs (VA) Medical Center found that reminder cards can increase the return of unused prescription opioid pills. The project built on the Cash for Your Stash program, which allowed patients to return unused opioid pills to the VA pharmacy to receive $5 per returned pill.

As part of this evaluation, OES compared the cost of the Cash for Your Stash program to the cost including OES-designed reminder cards. We found that the reminder cards increased the return of unused pills, and reduced the cost per pill returned: when using reminder cards, the total program cost per pill returned was $12.57. When no reminder cards were used, the total program cost per pill returned was higher at $15.66.

In order to build evidence on impact and identify cost effective program improvements across government, we think it is valuable to be transparent about how cost estimates are produced. In this post, we detail the methodology and assumptions to produce these cost estimates, using a project example. (See our previous blog post on what it costs to incorporate behaviorally informed interventions).

What did this analysis involve?

We can break our process of costing the program into four main steps:

1. Tabulating the costs at the original site: In collaboration with our partner at the White River Junction VA Medical Center, we used the “ingredient method” of costing¹ to gather program costs by listing all of the “ingredients” necessary to implement the program, estimating the unit costs of each item, and determining how much of each ingredient was needed. We broke down all costs for an average year of the program, for both the basic Cash for Your Stash program, and the additional reminder card:

  • Recurring supply costs ($3,417 per year for the basic Cash for Your Stash program): Physical items patients received as part of the program, such as stickers on the pill bottles or the cash incentives for pill return.
  • Personnel costs ($2,867 per year for the basic Cash for Your Stash program): Salaries and benefits of full-time staff for the percent time spent on the program.
  • Fixed costs for the first year ($17,499 for the basic Cash for Your Stash program): A starter pack for the receptacle needed for pill disposal and the cost of the security camera posted nearby.

2. Incorporating treatment effects to show cost-effectiveness: We presented costs in terms of the cost per pill returned, rather than the total program cost. This involved incorporating data on the number of returned pills in the basic Cash for Your Stash program, and the version of the program with reminder cards. Importantly, while the reminder cards increased the number of pills returned, they also increased other program costs such as the total incentives paid out to patients. To estimate the cost per pill returned, we divided the cost by the number of pills returned per year. For instance, for the basic Cash for Your Stash program: ($3,417 + $2,867 + $17,499) / 433 pills returned per year = $55 per pill returned in the first year of the program.²

3. Bounding the results to account for differences in sites or treatment effects: The implementation costs (e.g., staffing, new equipment) won’t be the same at every clinic, and we don’t always know exactly how much something costs in a particular year. It also may be that the treatments are more or less effective in different situations or over different time periods. So, to account for this uncertainty, we estimated different bounds according to the CHEERS reporting guidelines for health economic evaluations: we came up with a best estimate for the cost that was observed in practice, as well as upper and lower bounds based on what we might expect in different scenarios with different conditions.

4. Sharing the estimates: In our final materials we reported the cost estimate based on implementation costs observed in practice (rather than the upper or lower bound). We also consulted with our partner at the White River Junction VA Medical Center and decided to report only recurring staff and supply costs and not fixed costs, to reflect what program operations would look like after the program was up and running. The recurring costs change somewhat when making this assumption. For instance, when a clinic has already used the liners in the receptacle starter pack, they would purchase more receptacle liners than they would in the first year of the program when they had the starter pack on hand. The cost per pill returned when the program is up and running works out to be:($3,906 + $2,867) / 433 pills returned per year = $15.66 per pill returned.

We found that the Cash for Your Stash program with reminder cards cost $12.57 per pill returned per year. By comparison, our calculations above show that when no reminder cards were used, the cost per pill returned was $15.66. In other words, the reminder cards increased the return of unused pills and reduced the cost per pill returned by $3.09.

For other VA facilities considering implementing this program, we estimate that the recurring costs with reminder cards may range from $9.79 to $19.55 per pill returned.

Notes

  1. Dhaliwal, Iqbal, Esther Duflo, Rachel Glennerster, and Caitlin Tulloch. “Comparative cost-effectiveness analysis to inform policy in developing countries: a general framework with applications for education.” Education policy in developing countries 17 (2013): 285-338, https://economics.mit.edu/sites/default/files/2022-08/cost-effectiveness-1.pdf.
  2. Rounding errors may occur.