On Balance: Benefit-Cost Analysts Step Forward to Inform COVID-19 Policymaking
03/30/2020
By: Craig Thornton
Being in the SBCA is great because it connects me with so many very smart people who care about public policy and benefit-cost analysis. This value was illustrated by the rapid rate at which the SBCA network began to exchange ideas and estimates about the benefits and costs of interventions to address the COVID-19 situation. As these ideas start to work their way into the more general media, I wanted to share a few observations with readers of On Balance about the challenges facing our community in conducting and interpreting these benefit-cost analyses.
First, given how little we know about COVID-19, the first set of analyses will be preliminary and reflect a higher level of imprecision than analyses of diseases and interventions where we have more history. Nevertheless, these initial responses are essential for informing public policy. Policy makers are going to have to make decisions. They should have the benefit of the best available analyses, even if those analyses will get more accurate as the underlying estimates and assumptions used to produce them. The key here is to inform current decision-making and to then update the analyses as more evidence becomes available in order to inform future decision making.
Second, analysts and decision-makers have to be humble. COVID-19 is going to affect virtually every aspect of social activity, and benefit and cost analyses are going to capture only some of those effects. Thus, analysts need to be careful to indicate the activities on which they focus and those that they ignore. For example, estimates of the overall effect social distancing policies have on the U.S. economy can greatly assist decision makers. The fact that such big-picture analyses gloss over things like how the effects are distributed among residents should be noted, but need not take away from the value they provide by clarifying some of the policy implications. Decision makers need to use the information the first-round studies provide while also trying to keep track of the many factors where there are still many unknowns.
Analysts also have to be clear about the uncertainty inherent in their analyses. In order to make some sense of the complex reality, analysts are going to make assumptions about the specific nature of interventions, their effectiveness in changing behaviors and the spread of the virus, and what would have happened in the absence of the interventions. At this stage of the pandemic, it will be hard to judge the accuracy of these various assumptions. The use of sensitivity tests (or as time permits the use of more sophisticated measures of uncertainty) can help decision makers understand which assumptions crucially affect the conclusions. Similarly, placing various assumptions in the context of what we know from earlier pandemics or large economic shocks can help decision makers understand whether the key assumptions are reasonable.
Third, I want to note that the Society’s journal, Journal of Benefit-Cost Analysis has published several studies that are particularly relevant to the current policy debates. Here are four studies that offer very useful advice about how to conduct or assess benefit-cost analyses related to COVID-19. They are available through the Journal’s website:
As a starting point, I want to highlight the excellent analysis by Linda Thunström, Stephen Newbold, David Finnoff, Madison Ashworth, and Jason Shogren: “The Benefits and Costs of Flattening the Curve for COVID-19.” That study, which can serve as a model for future work on this issue, finds that social distancing measures that substantially reduce contact among individuals can generate large net benefits to the United States, even if those measures exacerbate the short-term economic slowdown caused by the virus. [As an aside, one of these authors, Jason Shogren, is scheduled to speak at the SBCA 2020 European Conference, scheduled for October 13-14 in Stockholm. We at SBCA hope that the virus situation will be sufficiently resolved by then to permit us to hear more about his thinking.]
Some other studies or comments that I have found useful include:
Craig Thornton is the current President of the Society for Benefit-Cost Analysis. This was written as an individual member; not in an official capacity for the Society.
By: Craig ThorntonBeing in the SBCA is great because it connects me with so many very smart people who care about public policy and benefit-cost analysis. This value was illustrated by the rapid rate at which the SBCA network began to exchange ideas and estimates about the benefits and costs of interventions to address the COVID-19 situation. As these ideas start to work their way into the more general media, I wanted to share a few observations with readers of On Balance about the challenges facing our community in conducting and interpreting these benefit-cost analyses.
First, given how little we know about COVID-19, the first set of analyses will be preliminary and reflect a higher level of imprecision than analyses of diseases and interventions where we have more history. Nevertheless, these initial responses are essential for informing public policy. Policy makers are going to have to make decisions. They should have the benefit of the best available analyses, even if those analyses will get more accurate as the underlying estimates and assumptions used to produce them. The key here is to inform current decision-making and to then update the analyses as more evidence becomes available in order to inform future decision making.
Second, analysts and decision-makers have to be humble. COVID-19 is going to affect virtually every aspect of social activity, and benefit and cost analyses are going to capture only some of those effects. Thus, analysts need to be careful to indicate the activities on which they focus and those that they ignore. For example, estimates of the overall effect social distancing policies have on the U.S. economy can greatly assist decision makers. The fact that such big-picture analyses gloss over things like how the effects are distributed among residents should be noted, but need not take away from the value they provide by clarifying some of the policy implications. Decision makers need to use the information the first-round studies provide while also trying to keep track of the many factors where there are still many unknowns.
Analysts also have to be clear about the uncertainty inherent in their analyses. In order to make some sense of the complex reality, analysts are going to make assumptions about the specific nature of interventions, their effectiveness in changing behaviors and the spread of the virus, and what would have happened in the absence of the interventions. At this stage of the pandemic, it will be hard to judge the accuracy of these various assumptions. The use of sensitivity tests (or as time permits the use of more sophisticated measures of uncertainty) can help decision makers understand which assumptions crucially affect the conclusions. Similarly, placing various assumptions in the context of what we know from earlier pandemics or large economic shocks can help decision makers understand whether the key assumptions are reasonable.
Third, I want to note that the Society’s journal, Journal of Benefit-Cost Analysis has published several studies that are particularly relevant to the current policy debates. Here are four studies that offer very useful advice about how to conduct or assess benefit-cost analyses related to COVID-19. They are available through the Journal’s website:
- “Towards Principles and Standards for the Benefit-Cost Analysis of Safety,” by Scott Farrow and W. Kip Viscusi, Journal of Benefit-Cost Analysis, Volume 2, Issue 3, 2011, Article 5
- “Skills of the trade: valuing health risk reductions in benefit-cost analysis,” by Lisa A. Robinson and James K. Hammitt, Journal of Benefit-Cost Analysis, 2013; 4(1): 107–130
- “Some Pitfalls of Practical Benefit-Cost Analysis,” by Clark Nardinelli, Journal of Benefit-Cost Analysis, 2018; 9(3):519–530.
- “Dynamic Benefit-Cost Analysis for Uncertain Futures,” by Susan E. Dudley, Daniel R. Pérez , Brian F. Mannix and Christopher Carrigan, Journal of Benefit-Cost Analysis, 2019; 10(2):206–225.
As a starting point, I want to highlight the excellent analysis by Linda Thunström, Stephen Newbold, David Finnoff, Madison Ashworth, and Jason Shogren: “The Benefits and Costs of Flattening the Curve for COVID-19.” That study, which can serve as a model for future work on this issue, finds that social distancing measures that substantially reduce contact among individuals can generate large net benefits to the United States, even if those measures exacerbate the short-term economic slowdown caused by the virus. [As an aside, one of these authors, Jason Shogren, is scheduled to speak at the SBCA 2020 European Conference, scheduled for October 13-14 in Stockholm. We at SBCA hope that the virus situation will be sufficiently resolved by then to permit us to hear more about his thinking.]
Some other studies or comments that I have found useful include:
- “Does Social Distancing Matter?” by Michael Greenstone and Vishan Nigam, Working Paper No. 2020-26, March 2020
- "Impact of non-pharmaceutical interventions (NPIs) to reduce COVID-19 mortality and healthcare demand." By Ferguson, Neil M., et al. March 16, 2020, London: Imperial College COVID-19 Response Team.
- “The Macroeconomics of Epidemics!” by Martin S. Eichenbaumy, Sergio Rebeloz, and Mathias Trabandtx, March 23, 2020
- “The Loss from Pandemic Influenza Risk,“ by Victoria Y Fan, Dean T Jamison, and Lawrence H Summers.
- “Pandemics: Risks, Impacts, and Mitigation,” by Nita Madhav, Ben Oppenheim, Mark Gallivan, Prime Mulembakani, Edward Rubin, and Nathan Wolfe
- “This Time the Numbers Show We Can’t Be Too Careful,” by Cass Sunstein, Bloomberg Opinion, March 26, 2020, 12:30 PM EDT
Craig Thornton is the current President of the Society for Benefit-Cost Analysis. This was written as an individual member; not in an official capacity for the Society.
Post a new comment