SBCA 2017 Workshop Offerings

Wednesday, March 15, 2017

George Washington University Marvin Center, Washington, D.C.

SBCA will be offering the following pre-conference professional development workshops*:

1. Retrospective Benefit-Cost Analysis

2. Using Experts to Estimate Parameter Values in BCA

4. Causal Analytics for Benefit-Cost Analysts: What Effects do Policies Cause?

* Workshops are subject to cancellation if they do not meet minimum registration thresholds. Please note that Workshop 3 has been canceled. 

A description of each workshop, including the preliminary agenda and list of instructors, as well as the registration fees, is provided below.

 


Retrospective Benefit-Cost Analysis

Organizer: Jennifer Baxter, Industrial Economics, Incorporated

Description

While prospective benefit-cost analysis is a well-established component of the regulatory development process in the United States and other developed economies, retrospective analysis is less commonly undertaken. Yet such analysis provides important insights into how to best improve existing regulations, as well as into how to improve the conduct of prospective analysis. Retrospective analysis is now strongly encouraged under Executive Order 13563 for significant federal regulations, and is increasingly advocated in many other policy contexts in the United States and internationally.

Many presume that retrospective analysis will be more accurate than prospective analysis, assuming that analysts can simply sum the incurred costs and benefits. In reality, retrospective analysis is very challenging and often highly uncertain. The most difficult step is disentangling the incremental effects of the policy from the effects of other factors that influence current conditions, so as to compare outcomes in the relevant setting with the policy against counterfactual (and hypothetical) outcomes in the same setting had the policy never been adopted.

This workshop brings together four leading experts with diverse perspectives to discuss both the institutional context for these analyses and their conduct. It is targeted on both those interested in conducting these analyses and those interested in better understanding the strengths and limitations of analyses they review. Prior to the workshop, participants will receive a list of optional readings. The workshop itself will consist of a series of presentations with ample time for discussion. There are no prerequisites or requirements for participation.

Preliminary Agenda (subject to change)

9:00 – 9:45 Introduction & Institutional Frameworks and Processes (Cary Coglianese, University of Pennsylvania)
9:45 - 10:30 Fundamentals of Conducting Retrospective Benefit-Cost Analysis (Jennifer Baxter, Industrial Economics, Incorporated)
10:30 - 10:45 Break (with refreshments)
10:45 – 11:30 Insights from Program Evaluation (David Bernstein, Westat)
11:30 – 12:15 Example(s) of Recent Analyses (Maureen Cropper, Resources for the Future and University of Maryland)
12:15 - 12:30 Wrap-up (Jennifer Baxter)

Registration

  • Early registration (before January 31): $250
  • Regular Registration (from February 1 through March 14): $330
  • On-Site Registration (from March 15 on): $410

Back to top


Using Experts to Estimate Parameter Values in BCA

Organizer: Aylin Sertkaya, Eastern Research Group

Description

Decision makers must frequently rely on data or information that is incomplete or inadequate in one way or another. Judgment, often from experts and occasionally from non-experts, then plays a critical role in the interpretation and characterization of those data as well as in the completion of information gaps. Further, OMB expects agencies to examine relevant uncertainties in benefit-cost analyses of especially complex rules that exceed the $1 billion annual threshold using simulation models and/or expert judgment (OMB Circular A-4). But how experts or other stakeholders are selected and their judgments elicited matters – they can also strongly influence the opinions obtained and the analysis on which they rely.

Several approaches to eliciting expert judgments have evolved. The workshop will cover topics ranging from expert recruitment, elicitation protocol design, different elicitation techniques (e.g., individual elicitations, Delphi method, nominal group technique, etc.) to aggregation methods for combining opinions of multiple experts. The role of expert elicitation and its limitations, problems, and risks in policy analysis will also be addressed. The workshop will include presentation of two case studies that will include a discussion of the selection process; elicitation protocol development, elicitation technique utilized, and the various issues that arose before, during, and after the elicitation process and the manner in which they were resolved. The class will conclude with a hands-on exercise where participants will learn about the importance of pre-elicitation training for calibrating experts.

Participation in this workshop will benefit economists, policy analysts and decision analysts. There are no prerequisites or requirements for attending this workshop.

Preliminary Agenda (subject to change)

1:30 - 1:45 Introduction to workshop-Motivation for having this workshop and its relevance for benefit-cost analyses (Aylin Sertkaya, ERG)
1:45 - 2:15 A quick guide to eliciting opinions for decision-making – Introduction to the process as a whole (Aylin Sertkaya, ERG)
2:15 - 3:00 Case Study 1: Pilot Expert Elicitation Study of Uncertainty in VSL When Applied in an Air Pollution Context (Henry Roman, Industrial Economics)
3:00 - 3:15

Break (with refreshments)

3:15 - 4:00 Case Study 2: Effectiveness of Pre- & Post Harvest Practices in Reducing Salmonella Contamination Risk in Fresh and Fresh-Cut Tomatoes (Cristina McLaughlin, FDA)
4:00 - 5:00 Hands-on Exercise: Effect of Training on Expert Judgment (Frank Hearl, NIOSH)

Registration

  • Early registration (before January 31): $250
  • Regular Registration (from February 1 through March 14): $330
  • On-Site Registration (from March 15 on): $410

Back to top


Causal Analytics for Benefit-Cost Analysts: What Effects do Policies Cause?

Organizer: Tony Cox, Cox Associates

Description

Benefit-cost analysis (BCA) prescribes taking actions that cause total benefits larger than their total costs, but accurate prediction of benefits and costs is technically challenging, as is retrospective evaluation of effects. Effects caused by a policy may depend on what else happens, raising the challenge of attributing effects to one cause among many. Actions often have uncertain consequences, given realistically incomplete and imperfect knowledge and understanding of how the world works. Causal laws are often best modeled as probabilistic, and the relevant probabilities may not initially be known or agreed to. The psychology of causal reasoning is also notoriously misleading. The net result is that BCA must often rely on uncertain causal models of costs and benefits inferred from complex data with multiple interacting variables and imperfect, partly missing data. Despite these challenges, drawing valid causal conclusions from such data is crucial for predicting and evaluating effects of policies. 

This workshop introduces current machine-learning and computational statistics methods, implemented in free R packages, that can help to do so by discovering, testing, and validating causal models from observational data. The workshop will cover the principles underlying causal discovery and modeling algorithms, such as that (a) Causes provide unique information about their effects; (b) Changes in causes help to predict and explain subsequent changes in their effects; and (c) Information flows from causes to their effects over time. 

Participants will learn to apply information-based methods of causal inference to support BCA modeling while acknowledging and characterizing realistic uncertainties based on data. Practical illustrations will be given using a Causal Analytics Toolkit (CAT) that allows Excel users to apply advanced R packages to identify and quantify potential causal relationships in data. No previous knowledge of R or causal analysis is required.

The workshop will be taught by Tony Cox. 

Preliminary Agenda (subject to change)

1:30 - 2:00 Types of causality: Associational/attributive (Hill), counterfactual, computational, predictive, manipulative
2:00 - 2:30 Uncertain and probabilistic causality in tort law liability, insurance, biology, economics, complex systems physics, and engineering 
2:30 - 3:00 Principles and methods for causal analysis: Inference, explanation, and prediction
3:00 - 3:15 Break (with refreshments)
3:15 - 3:45 Information-based algorithms for causal discovery and inference
3:45 - 4:15 Making the algorithms useful: R packages and the Causal Analytics Toolkit (CAT)
4:15 - 5:00 Example applications: Toxicology, epidemiology, and policy analysis

Registration

  • Early registration (before January 31): $350
  • Regular Registration (from February 1 through March 14): $430
  • On-Site Registration (from March 15 on): $510

Back to top


Questions?

Please contact SBCA at SBCAinfo@uw.edu.

Refund Policy

Any cancellation requests must be received via telephone or by email by 11:59 p.m. PST on January 31, 2017 to be eligible for a refund of the entire registration fee. Cancellation requests received after 11:59 p.m. PST on January 31, 2017 and before 11:59 p.m. PST on February 21, 2017 will be subject to a $75 cancellation fee. After 11:59 p.m. PST on February 21, the registrant shall be liable for the full registration fee. Refunds will not be given for no-shows. Email inquiries to sbcainfo@uw.edu. 

Back to top