2017 Conference - Session 3

Session 3 - Thursday, March 16, 2:00 - 3:30pm

A.3: International Approaches to Analyzing Benefits and Costs of Chemical Risk Policies (Roundtable)

Chair: Susan Dudley, The George Washington University

Governments around the world rely on benefit-cost analysis in one form or other to develop and evaluate policies aimed at reducing their citizens’ exposure to chemical risks. This roundtable panel of experts will share practices and experiences associated with analyzing benefits and costs of chemical risk policies in Europe and North America.


Nils Axel Braathen, OECD
Jessica Coria, University of Gothenburg
Joe Devlin, Environment & Climate Change Canada
Matti Vainio, European Chemicals Agency

B.3: Value of a Statistical Life: Recent Advances

Chair: Thomas Kniesner, Claremont Graduate University

Discussant: Al McGartland, U.S. Environmental Protection Agency


1. Is Survival a Luxury Good? Income Elasticity and the Value per Statistical Life; James K. Hammitt*, Harvard University

The value of changes in mortality risk is conventionally estimated by the marginal rate of substitution between income and mortality risk—the value per statistical life (VSL). The income elasticity of VSL is important for estimating how the value of mortality risk varies with time (for evaluating programs with long-lived effects) and across populations with different income levels (for evaluating programs with international consequences). Previous estimates of income elasticity based on meta-analysis of wage-differential studies and cross-sectional comparisons in stated-preference studies suggest values of approximately 0.5 while international comparisons and extrapolation from estimates of the coefficient of relative risk aversion imply values between 0.5 and 2 or more. We present new estimates based on a 16 year series of wage-differential estimates in Taiwan. Between 1982 and 1997, estimated VSL increased eight-fold while per capita GNP increased two and half times and the occupational fatality rate in manufacturing and service industries decreased by half. Comparison of VSL with GNP per capita implies the income elasticity is between 2 and 3, but controlling for changes in endogenous job risk and worker characteristics yields estimates between 0.6 and 0.9.

2. Best Estimate Selection Bias of Estimates of the Value of a Statistical Life; W. Kip Viscusi*, Vanderbilt University

Assessments of key economic parameters frequently rely on the best estimate or a meta-analysis of the “best-set” of estimates of the parameter from the literature. Government agencies’ choices of the VSL for policy evaluations typically rely on the “best-set” estimates of the VSL. But the selection of the best estimates by either the article authors or policymakers may involve judgments of what magnitudes of VSL estimates that are viewed as being reasonable, thus inducing selection biases. This paper’s meta-regression analysis considers over 1,000 VSL estimates from 68 studies. The all-set sample consists of all reported estimates of the value of a statistical life (VSL), while the best-set sample consists of the best estimates from these studies. This article finds statistically significant selection biases in each case, but much greater biases for the best-set sample. “Best estimate selection bias” exacerbates the problems associated with existing publication selection biases. Estimates based on the Census of Fatal Occupational Injuries (CFOI) are considerably greater than for other studies even after correcting for publication selection effects. The bias-corrected estimates of VSL for the all-set USA sample is $9.6 million. Rather than focusing on a best-set sample, policymakers can avoid the selection biases by choosing the VSL based using the estimates from their preferred specification from an all-set analysis.

3. Estimating the Value of a Statistical Life in (the Likely Many) Cases of Reference Dependence; Jack Knetsch*, Simon Fraser University

Estimates of the value of statistical lives (VSL) are based on the values that people place on a small change in the probability of death. Particularly when these assessments are based on surveys, the assessments are commonly people's willingness to pay for a decrease in the risk of death (the WTP measure). The resulting VSL estimates are, in practice, assumed to be equally applicable for all changes in the risks of death -- ones that impose losses or reductions of losses, as well as ones that provide gains.

However, changes resulting in losses and reductions of losses are more accurately assessed with the WTA measure of the minimum amounts people would require to accept an increase, or to forego a reduction, in the risk of death. To the extent that assessments that use the WTP measure differ from those that are based on the WTA measure, the inappropriate choice of measure will impose a potentially important bias to the results of the analysis and the guidance which is provided.

The present study was conducted to provide more direct tests of possible disparities between people's valuations of small positive and negative changes in the probability of death. Further, the study examined the possible extent to which positive changes of a reduction in the risks of death were regarded by people as being a gain, and therefore best assessed with the WTP measure, or a reduction of a loss and more appropriately assessed with the WTA measure.

The findings strongly suggest large disparities between gain and loss measures. They also indicate that many positive changes are regarded as reductions of losses rather than gains, which the WTP measures would likely understate. Together, these findings have likely implications for estimates derived from wage-risk observations as well as those elicited from contingent valuation surveys.

C.3: Issues in BCA of Energy and the Environment

Chair: Ann Wolverton, U.S. Environmental Protection Agency

Discussant: Fran Sussman, Independent Consultant


1. Consumer Willingness to Pay for Vehicle Attributes: What Do We Know?; Gloria Helfand*, U.S. Environmental Protection Agency

A full benefit-cost analysis seeks to take into account impacts on consumer welfare as well as financial benefits and costs. Tighter standards for vehicle greenhouse gas emissions and fuel economy have raised questions about possible effects on other vehicle attributes, such as noise, safety, comfort, or performance. Assessing the effects of these impacts on consumer welfare requires estimates of consumer willingness to pay (WTP) for the foregone benefits. This paper evaluates WTP estimates for a range of vehicle characteristics from academic literature. We use 52 U.S.-focused papers since 1995 with sufficient data to calculate WTP values for various vehicle attributes. We identify over 150 individual characteristics included in these papers, which we consolidate into 15 general categories: comfort, fuel availability, fuel costs, fuel type, incentives, model availability, non-fuel operating costs, performance, pollution, prestige, range, reliability, safety, size, and vehicle type. We then calculate, for each observation, WTP values and their ranges for those characteristics, based on the coefficients and data reported in the papers. In addition to mean WTP estimates, we present uncertainty estimates around each WTP value, based either on standard errors of the estimated coefficients or the standard deviations in random coefficient models. We also examine the implications of heterogeneous consumer characteristics (e.g., different levels of income, household size, and other factors). Findings suggest large variation in estimates of WTP values, both within and across studies. This variation may result in part because of methodological difficulties in estimating how attributes affect consumer vehicle choices, such as omitted variables, collinearity, and the use of proxies. We discuss the implications of this variation in WTP estimates for estimating impacts on consumer welfare due to changes in fuel efficiency technology.

2. Re-Searching for Hidden Costs: Producer Heterogeneity and Adoption of Fuel-Saving Vehicle Technologies; Hsing-Hsiang Huang*, Oak Ridge Institute for Science and Education

Hidden costs – undesirable aspects of the new technologies – have been suggested as reasons for the “energy paradox”: markets are slow to take advantage of cost-effective opportunities for energy efficiency in the light-duty vehicles market. Though quantitative evidence about the existence of hidden costs is limited, a recent study by Helfand et al. (2016) did not find systematic evidence of hidden costs associated with the use of fuel-saving technologies. We extend Helfand et al. (2016)’s study by exploiting a pooled dataset of professional auto reviews for consumers for model-year 2014 and 2015 vehicles. The preliminary results show that, for the technologies examined, reviews with positive evaluations significantly outnumber those with negative evaluations. The conclusion of Helfand et al. (2016) holds after we control for potential confounders, such as year, year-by-reviewer, and year-by-automaker fixed effects. Overall, the use of fuel-saving technologies does not increase the probability of getting a negative evaluation of operational characteristics. In addition, there is heterogeneity across automakers in the relationships between some technologies and operational characteristics. For instance, for the start-stop technology in the pooled data, 50 percent and 36 percent of the evaluations are negative for Subaru and BMW, respectively, while Chevrolet, Ford, Honda, and Toyota have zero negative evaluations. The heterogeneity may be due to quality of implementation of the technology across automakers. The heterogeneity is much smaller for some technologies, such as full electric and mass reduction, which have 0 to 17 percent and 0 to 8 percent of negative evaluations for all the automakers reviewed, respectively. It implies automakers have been and might be able to implement fuel-saving technologies without imposing hidden costs. Consistent with the findings of Helfand et al. (2016), the results do not provide evidence for hidden costs as the explanation of the energy paradox.

3. Benefit-Cost Analysis: Problems with Product Failures; Arthur Fraas*, Resources for the Future; and Sofie Miller*, The George Washington University

Federal regulation in the energy, environmental and product safety area often require the adoption of new technologies. However, the incorporation of a new technology in products does not necessarily go smoothly as evidenced by several examples of notable product failures (e.g., Whirlpool clothes washers, Subaru engines). In such cases, the costs and benefits to consumers may vary significantly from initial agency estimates. This presentation would consider several case studies. These case studies would review regulatory agency treatment of the potential for product failures, outline the nature of the product failure, and discuss the effectiveness of market and legal institutional responses to product failures. Ex post responses include manufacturer warranties, market information channels like Consumer Reports, and legal remedies like class action lawsuits. Finally, we would consider the extent to which product failures deserve further consideration in the rulemaking process and in both prospective and retrospective benefit-cost analyses.

D.3: Regulatory Review and Benefit-Cost Analysis in the States (Roundtable)

Chair: Stuart Shapiro, Rutgers University

Much of the debate on the role of benefit-cost analysis in policy-making has centered on the regulatory process at the federal level. This debate has gone on for 36 years and shows little signs of abating. Meanwhile, many of the fifty states have begun to take steps to incorporate economic analysis into their regulatory decisions. As is often the case at the state level, the process for developing regulations takes many different forms and the use of benefit-cost analysis and its level of incorporation into the regulatory process varies across states.

This panel will consider the basis and implementation of requirements for various forms of analysis at the state level. It is made up of four practitioners from states which have implemented regulatory review and cost-benefit analysis. The states have taken different approaches to their implementations and are in different stages of implementation.

Each panelist will provide an overview of the role of analysis in their state. We will also explore the challenges associated with analysis and regulatory review at the state level, and lessons which may be useful to other states, and BCA practitioners across many levels of government. Each panelist will speak for 15 minutes and there will be a half hour for questions and discussion.


Larry Getzler, Virginia Department of Planning and Budget
Anca Grozav, North Carolina Office of State Budget and Management
David Sumner, Pennsylvania Independent Regulatory Review Commission
Ben Witherell, New Jersey Department of Environmental Protection

E.3: BCA Applications in Transportation and Infrastructure

Chair: Deborah Aiken, U.S. Department of Transportation

Discussant: David Luskin, U.S. Department of Transportation


1. Information Saves Lives: An Impact Evaluation of Automobile Crash Tests; Damien Sheehan-Connor*, Wesleyan University

Imperfect information about product quality can lead to market failure, creating the potential that a policy of information provision can improve welfare. Consumers are unlikely to be able to accurately assess motor vehicle safety with the consequence that automakers in a competitive market will produce vehicles of sub-optimal safety. Third-party information providers, whether private or public, may ameliorate this sort of problem. In an effort to do this, the Insurance Institute for Highway Safety (IIHS) began performing frontal offset crash tests in 1995 and side impact crash tests in 2003 on many vehicles sold in the United States. This paper finds strong evidence that manufacturers responded to the tests by producing safer vehicles and that consumers respond by increasing purchases of those vehicles rated as safest. An impact analysis is performed to quantify the safety benefits attributable to this shift toward safer vehicles. Significant changes in the ability of a vehicle model to withstand a crash are most easily made when that model undergoes a substantial redesign, which typically occurs every five to eight years. This paper includes vehicle redesign year in a richly controlled model of impacts on driver fatality and compares outcomes in vehicles redesigned just before versus just after implementation of the IIHS crash testing programs. The key findings are that the frontal crash test program was associated with a reduction in fatality risk for drivers involved in severe frontal, but not side, collisions whereas the side crash test program was associated with a reduction in driver fatality risk in side, but not frontal, collisions. A lower bound estimate of the health benefits attributable to the programs is a reduction of 1,800 fatalities in 2013 while the overall economic benefits are estimated at a minimum of $2,400 per vehicle.

2. Estimation of Logistics Cost Savings for a Multi-Jurisdictional Freight Rail Project using a BCA Framework; Saravanya Sankarakumaraswamy*, University of Washington

This work applied a novel approach to logistics cost estimation for multijurisdictional freight rail projects based on a case study conducted as part of research sponsored by National Cooperative Freight Research Program Project. This discusses the application of the methodology to the Heartland Corridor. The approach adopted relies on publicly available open data sources to support Benefit-Cost Analysis (BCA) in order to address problems associated with the availability of input data and a more representative measure of logistics cost. These data sources include Freight Analysis Framework freight movements’ data between states, the Surface Transportation Board’s Uniform Railroad Costing System (URCS) data and R-1 reports for Norfolk Southern, and data on Class 1 rail track miles operated from American Association of Railroad. These sources are combined to provide commodity and rail specific logistics costs comprising of transport, inventory, loading and unloading costs and damage costs. As such this allows a better comprehensive breakdown of the cost of shipping between origin-destination pairs, than the conventional approach of focusing only on inventory costs. The Heartland Corridor project’s intermodal terminals and line-haul improvements are expected to produce diversion from trucks to rail apart from capacity enhancement through double stacked trains. Hence we have discussed the modal diversion from trucks to rail and estimated the amount of potential diversion and the associated savings in logistics cost. The diversion benefits were accounted by estimating the cost savings for new users apart from the benefits accounted for existing users. The main contributions of this work can be listed as the following: (1) demonstrate the use of open source and publicly available data in BCA (2) demonstrate the choice of appropriate data sources among the different options (3) estimate logistics cost savings by accounting for modal diversion from freight investments and (4) demonstrate the cost savings estimation for multi-jurisdictional freight rail projects.

3. Addressing Uncertainty in the Estimation of the Benefits and Costs of Complex Transportation Infrastructure Projects; Kenneth Kuhn*, RAND Corporation

This research describes and applies a methodology for addressing uncertainty inherent in the estimation of benefits and costs of transportation infrastructure projects. The methodology involves transparency, the identification of sources of uncertainty, sensitivity testing including purposely pessimistic scenarios, Monte Carlo simulation, and Robust Decision Making. Time and resources will be limited in practice, but it is important for practitioners to understand the strengths of the weaknesses of each of the mentioned techniques. The methodology describes how to categorize sources of uncertainty. For example, a parameter may be a source of epistemic uncertainty, aleatory uncertainty, and/or deep uncertainty. If uncertainty could be eliminated or reduced by collecting additional data, then the parameter creates epistemic uncertainty. Freight and passenger values of time are sources of epistemic uncertainty in transportation. Classification of sources of uncertainty helps analysts devise mitigation strategies. For instance, sensitivity testing and Robust Decision Making are established ways for dealing with sources of deep uncertainty. This presentation includes sample results from the application of the techniques to a case study based on the Heartland Corridor project. The Heartland Corridor is a multimodal, multijurisdictional freight transportation infrastructure project, a particularly challenging type of project to evaluate. The project reduced the distance and time required for freight movement in the Mid-Atlantic and Midwest regions. The project benefited new users of facilities that were improved or constructed. A model of diversion was required to estimate associated benefits. States benefited from reduced tractor-trailer traffic on roads in the region and, thus, reduced required pavement maintenance. The public at large also benefited from reduced tractor-trailer traffic, through reduced safety and environmental externality costs. These and other benefits and costs were evaluated. The complete results highlight the presence of substantial uncertainty and risk of wrongheaded analysis and/or recommendation.

F.3: Issues in BCA of Research, Development, and Innovation

Chair: R. Jeffrey Lewis*, ExxonMobil Biomedical Sciences, Inc.

Discussant: Kathleen Miller, U.S. Food and Drug Administration


1. Human Capital Benefits of Big Science: Evidence from Early Career Researchers at CERN; Massimo Florio*, University of Milan

Is there a social benefit of Big Science arising from increasing human capital? High-energy physics laboratories at forefront of science are breeding ground of ideas and skills. More than 36,000 students and post-docs will be involved in research until 2025 at the Large Hadron Collider (LHC) at CERN mainly through international collaborations, which include also non-member states as the US and other countries. To what extent do these early career researchers value in money terms the skills they have acquired? Do they expect that their learning experience will have an impact on their professional future? By drawing from earlier literature on experiential learning and salary expectations, we have designed an in-depth survey involving two samples, one of current and the other one of former students at LHC, the latter now employed in various jobs, including industry and finance. We want, particularly, to compare the expectations of current students with the assessment of those who have completed their PhD. Results from ordered logistic regression models suggest that the experiential learning at LHC positively correlates with both current and former students’ salary expectations. Those already employed clearly confirm the expectations of current students. The results are driven by the self-assessment of the skills acquired. Respondents put a price tag on their experience at LHC, a ‘salary premium’ ranging from 5% to 12% compared with what they would have expected for their career without such an experience at CERN. The expected perceived cumulative human capital social benefit from training at the LHC is expected to be around 6 billion Euro over the period 1993-2025.

2. Benefit Cost Analysis of Research, Development and Innovation (RDI) Infrastructures - Strategy for Europe 2020; Kristina Gogic*, Office of the Croatian Ombudsman

The research, development and innovation (RDI) infrastructure is the generic name for investment projects that are designed and operated according to very different specifications, in some cases their features are unique and cannot be analyzed with the same degree of standardization of methods as, for example, in railways or in water, for which there are several decades of evaluation experience and a large library of appraisal documents. Benefit Cost Analysis of RDI infrastructures is a new field and the project proposer should be aware that, at the same time, it requires a solid understanding of the principles of BCA, professional experience in project evaluation in different areas and a very flexible practical approach tailored to the specific project under appraisal. While the target groups of other infrastructures are relatively well identified, e.g. passengers for high-speed rail or residents in an urban area for solid waste management, the multifaceted nature of RDI is such that many types of direct and indirect target groups are involved, from business to the general public. Each of them has standing in the BCA and this makes the evaluation of infrastructures a particularly complex task. It is expected that over the planning period (2014.-2020.) a portfolio of BCA of RDI infrastructures will be gradually built within the Member States, following the high priority given to research and innovation for the European Union growth strategy. Many research and innovation projects face difficulties in securing finance, despite being fundamentally good projects.

3. The Inclusion of Economic Variables in Case Studies of Biomedical Research Impact; Sue Hamann*, National Institutes of Health

Science evaluators are increasingly asked to include economic variables and econometric analyses in their evaluation of the outcomes and impacts of federally funded research. Cost studies, particularly cost benefit analysis and cost utility analysis, are frequently used to assess the costs and economic outcomes of medical prevention and treatment interventions; however, taking a step backward and linking federally funded biomedical research to an implemented specific health intervention and outcome is not yet, and may never be, standard. Interest in quantifying the impact of federally funded research on markets and industries is also growing. Much administrative data on federal research expenditures, research topics, and published research findings is available, but linking such data to discoveries and downstream interventions is challenging. Case studies, including those with economic analyses, are being explored as one method of tracing federally funded biomedical research from basic science to public health improvement and other societal benefits.

At the 2016 SBCA Annual Conference, we presented the results of evaluability assessments to determine readiness to conduct economic analysis for several oral health research programs. We also presented a preliminary framework of information types and sources that would be necessary and sufficient for economic analysis, as well as one method for determining the quality of economic analysis. We received valuable feedback at the conference, which has been included in our continued design of the framework. In parallel, policy analysts at the National Institutes of Health (NIH) have articulated a case study method for producing and analyzing evidence of the impact of selected federal research programs (e.g., molecular medicine, vaccines, neuro-stimulation technologies, and longitudinal cardiovascular disease study). For the 2017 SBCA Annual Conference, we propose to assess these case studies against our framework and the quality indicators. This work is an important step in the development of the Science of Science Policy.