Our Knowledge of the World is Often Not Simple: Policymakers Should Not Duck that Fact, But Should Deal with It

Although I share some of Bolger and Rowe’s reservations about the use of “seed questions” as a means of assessing the quality of judgments provided by experts and developing weights with which to combine their subjective judgments, I do not accept their argument that “for policy making a single representation of the uncertain quantity, and related probability, is commonly needed.” A single representation is certainly commonly desired by decisionmakers whose life is simplified if they can find Senator Muskie’s hypothetical “onehanded scientist.” But, the world is frequently not that simple, and policymakers need to acknowledge that fact and deal with it, not hide behind some mathematical procedure that masks important disagreements. Bolger and Rowe did add a footnote to their assertion, noting that combining may be inappropriate when it “leads to an average that does not properly represent the views of any experts, or when judgments form input to a very non-linear model.” Those qualifications deserve elaboration. On “an average that does not properly represent the views of any experts,” consider Fig. 1, which shows a result from an expert elicitation my colleagues and I conducted just under a decade ago.(1) We asked a set of ocean science experts to provide probabilistic judgments about the possibility that, as the planet warms due to climate change, the Atlantic Meridional Overturning Circulation (AMOC, sometimes termed the “ocean conveyer belt”) will shut down, as data from sediment cores demonstrates it