Mobile Menu
Back to Blog

On Balance: Forming Covid-19 Policy under Uncertainty

By Charles F. Manski, Northwestern University

In a recent paper in the Journal of Benefit-Cost Analysis (Manski, 2020), I observed that formation of COVID-19 policy must cope with many uncertainties about the nature of the disease, the dynamics of the pandemic, and behavioral responses. I noted that these uncertainties have been well-recognized qualitatively but not satisfactorily characterized quantitatively. I argued that credible measurement of uncertainties would improve prediction of policy impacts and promote reasonable policy decisions.
Incredible Certitude in Epidemiological and Macroeconomic Modeling

Epidemiological models of disease dynamics, sometimes combined with models of macroeconomic dynamics, have been used to reach conclusions about optimal COVID-19 policy. However, researchers have done little to appraise the realism of their models, nor to quantify uncertainties. Hence, I find little basis to trust the policy prescriptions that have been put forward.

Epidemiological modelers have sought to determine COVID-19 policy that would be optimal from a public health perspective if specified models of disease dynamics were accurate and public health were measured in specified ways. However, epidemiological modeling has only considered impacts on health. Policy assessment should consider the full health, economic, and social impacts of alternative options. Recognizing this, macroeconomists have sought to expand the scope of optimal policy analysis by joining epidemiological models with models of macroeconomic dynamics and by specifying welfare functions that consider both public health and economic outcomes.

A serious underlying problem in both epidemiological and macroeconomic modeling has been the dearth of evidence available to inform model specification and estimation. Studies of disease and macroeconomic dynamics are largely unable to perform randomized trials. Modeling necessarily relies on observational data, which are difficult to interpret. Lacking much evidence, epidemiologists and macroeconomists have developed models that are sophisticated from mathematical and computational perspectives but that have little empirical grounding. These modeling efforts may perhaps be useful if interpreted cautiously as computational experiments studying policy making in hypothetical worlds. However, their relevance to the real world is unclear.

I have persistently argued for forthright communication of uncertainty in research that aims to inform public policy (Manski, 2019). I have criticized the prevalent practice of policy analysis with incredible certitude. Exact predictions of policy outcomes are routine. Expressions of uncertainty are rare. Yet predictions often are fragile, resting on unsupported assumptions and limited data. Expressing certitude is not credible. Incredible certitude has been prevalent in both epidemiological and economic modeling.

There is an urgent need for epidemiologists and economists to join forces to develop credible integrated assessment models of epidemics. Even with the best intentions, this will take considerable time. There is some reason to hope that epidemiologists and economists may be able to communicate with one another because they share a common language for mathematical modeling of dynamic processes. However, each group has in the past exhibited considerable insularity, which may impede collaboration. Moreover, neither discipline has shown much willingness to face up to uncertainty when developing and applying models.

Adaptive Policy Diversification

I think it misguided to make policy that is optimal in hypothetical scenarios but potentially much less than optimal in reality. It is more prudent to approach policy as a problem in decision making under uncertainty. Facing up to uncertainty, one recognizes that it is not possible to guarantee choice of optimal policies.

While one cannot guarantee optimality under uncertainty, one may still make decisions that are reasonable in well-defined respects. I have suggested adaptive diversification of COVID-19 policy. Adaptive policy diversification was proposed in Manski (2009, 2013). Akin to financial diversification, policy is diversified if a planner randomly assigns treatment units (persons or locations) to different policies. At a point in time, diversification avoids gross errors in policy making. Over time it yields new evidence about policy impacts, as in a randomized trial. As evidence accumulates, a planner can revise the fraction of treatment units assigned to each policy in accord with the available knowledge. This idea is the ideal form of adaptive diversification.

Explicitly random assignment of policies may not be feasible in practice. Nevertheless, it may be possible to vary policy across time or place to approximate adaptive diversification. Justice Brandeis suggested something of this sort close to a century ago. In a famous dissent on a Supreme Court case, he referred to the states as the “laboratories of democracy.”

To illustrate, consider the choice between suppression and mitigation of COVID-19. Suppression may be the better policy if it were known that this policy has strong positive health impacts and only small negative economic impacts. On the other hand, mitigation may be the better policy if suppression has weak positive health impacts and large negative economic impacts. Policy diversification, with some locations implementing suppression and others implementing mitigation, gives up the ideal of optimality in order to protect against making a gross error in policy choice.

When diversifying, what fraction of locations should implement each policy option under consideration? This depends on the welfare function that society uses to evaluate options and on the uncertainties that afflict prediction of policy impacts. Manski (2009) studied adaptive diversification when social welfare is utilitarian, and a planner uses a simple dynamic version of the minimax-regret criterion to cope with uncertainty. The result is a simple diversification rule. I think it would be productive to specify an appropriate welfare function, characterize the relevant uncertainties, and adapt this analysis to diversify COVID-19 policy.


Manski, C. (2009), “Diversified Treatment under Ambiguity,” International Economic Review 50, 1013-1041.

Manski, C. (2013), Public Policy in an Uncertain World, Cambridge, MA: Harvard University Press.

Manski, C. (2019) “Communicating Uncertainty in Policy Analysis,” Proceedings of the National Academy of Sciences, 116, 7634-7641.

Manski, C. (2020), “Forming COVID-19 Policy under Uncertainty,” Journal of Benefit-Cost Analysis, 11, 341-356.
Post a new comment
Refresh Image