Burning embers: towards more transparent and robust climate-change risk assessments


  • 1.

    Lenton, T. M. et al. Climate tipping points — too risky to bet against. Nature 575, 592–595 (2019).


    Google Scholar
     

  • 2.

    United Nations. United Nations Framework Convention on Climate Change (1992).

  • 3.

    Garner, G., Reed, P. & Keller, K. Climate risk management requires explicit representation of societal trade-offs. Clim. Change 134, 713–723 (2016).


    Google Scholar
     

  • 4.

    Nordhaus, W. D. Managing the Global Commons: The Economics of Climate Change (MIT Press, 1994).

  • 5.

    Leemans, R. & Vellinga, P. The scientific motivation of the internationally agreed ‘well below 2 °C’ climate protection target: a historical perspective. Curr. Opin. Environ. Sustain. 26–27, 134–142 (2017).


    Google Scholar
     

  • 6.

    Fischlin, A., Ji, Z., Vladu, F. & Bisiaux, A. Report on the Structured Expert Dialogue on the 2013–2015 Review of the United Nations Framework Convention on Climate Change (UNFCCC). Final Report FCCC/SB/2015/INF.1 (UNFCCC Secretariat, 2015).

  • 7.

    Smith, J. et al. in Climate Change 2001: Impacts, Adaptation, and Vulnerability. Contribution of Working Group II to the Third Assessment Report of the Intergovernmental Panel on Climate Change (IPCC) Ch. 19 (eds. McCarthy, J. J., Canziani, O. F., Leary, N. A., Dokken, D. J. & White, K. S.) 913–967 (Cambridge Univ. Press, 2001).

  • 8.

    McCarthy J. J., Canziani O., Leary N. A., Dokken D. J. & White K. S. (eds) Climate Change 2001: Impacts, Adaptation, and Vulnerability. Contribution of Working Group II to the Third Assessment Report of the Intergovernmental Panel on Climate Change (IPCC) (Cambridge Univ. Press, 2001).

  • 9.

    O’Neill, B. C. et al. IPCC reasons for concern regarding climate change risks. Nat. Clim. Change 7, 28–37 (2017).


    Google Scholar
     

  • 10.

    Matthews, J. B. R. in Global Warming of 1.5°C. An IPCC Special Report on the Impacts of Global Warming of 1.5°C Above Pre-industrial Levels and Related Global Greenhouse Gas Emission Pathways, in the Context of Strengthening the Global Response to the Threat of Climate Change, Sustainable Development, and Efforts to Eradicate Poverty (eds Masson-Delmotte, V. et al.) 539–562 (IPCC, 2018).

  • 11.

    Paris Agreement. United Nations Treaty Collection. United Nations https://treaties.un.org/doc/Treaties/2016/02/20160215%2006-03%20PM/Ch_XXVII-7-d.pdf (2016).

  • 12.

    Intergovernmental Panel on Climate Change (IPCC). Chapter Outline of the Working Group II Contribution to the IPCC Sixth Assessment Report (AR6). 46th Session of the IPCC (IPCC, 2017).

  • 13.

    Mahony, M. Climate change and the geographies of objectivity: the case of the IPCC’s burning embers diagram. Trans. Inst. Br. Geogr. 40, 153–167 (2015).


    Google Scholar
     

  • 14.

    Oppenheimer, M., Little, C. M. & Cooke, R. M. Expert judgement and uncertainty quantification for climate change. Nat. Clim. Change 6, 445–451 (2016).


    Google Scholar
     

  • 15.

    Budescu, D. V., Por, H. H. & Broomell, S. B. Effective communication of uncertainty in the IPCC reports. Clim. Change 113, 181–200 (2012).


    Google Scholar
     

  • 16.

    Swart, R., Bernstein, L., Ha-Duong, M. & Petersen, A. Agreeing to disagree: uncertainty management in assessing climate change, impacts and responses by the IPCC. Clim. Change 92, 1–29 (2009).


    Google Scholar
     

  • 17.

    Socolow, R. H. High-consequence outcomes and internal disagreements: tell us more, please. Clim. Change 108, 775–790 (2011).


    Google Scholar
     

  • 18.

    Adler, C. E. & Hirsch Hadorn, G. The IPCC and treatment of uncertainties: topics and sources of dissensus. Wiley Interdiscip. Rev. Clim. Change 5, 663–676 (2014).


    Google Scholar
     

  • 19.

    Sutton, R. T. ESD Ideas: A simple proposal to improve the contribution of IPCC WGI to the assessment and communication of climate change risks. Earth Syst. Dyn. 9, 1155–1158 (2018).


    Google Scholar
     

  • 20.

    Yohe, G. & Oppenheimer, M. Evaluation, characterization, and communication of uncertainty by the Intergovernmental Panel on Climate Change — an introductory essay. Clim. Change 108, 629–639 (2011).


    Google Scholar
     

  • 21.

    Moss, R. H. & Schneider, S. H. in Guidance Papers on the Cross Cutting Issues of the Third Assessment Report of the IPCC (eds Pachauri, R., Taniguchi, T. & Tanaka, K.) 33–51 (World Meteorological Organization, 2000).

  • 22.

    Manning, M. et al. (eds) Describing Scientific Uncertainties in Climate Change to Support Analysis of Risk and of Options. Report of IPCC Workshop 11–13 (IPCC, 2004).

  • 23.

    Mastrandrea, M. D. et al. Guidance Note for Lead Authors of the IPCC Fifth Assessment Report on Consistent Treatment of Uncertainties (IPCC, 2010).

  • 24.

    Intergovernmental Panel on Climate Change (IPCC). Climate Change and Land: An IPCC Special Report on Climate Change, Desertification, Land Degradation, Sustainable Land Management, Food Security, and Greenhouse Gas Fluxes in Terrestrial Ecosystems (eds Shukla, P. R. et al) (IPCC, 2019).

  • 25.

    Intergovernmental Panel on Climate Change (IPCC). in IPCC Special Report on the Ocean and Cryosphere in a Changing Climate (eds Pörtner, H.-O. et al.) (IPCC, 2019).

  • 26.

    Jones, J. & Hunter, D. Qualitative research: consensus methods for medical and health services research. BMJ 311, 376–380 (1995).


    Google Scholar
     

  • 27.

    Peel, A. et al. Use of expert judgement across NICE guidance-making programmes: A review of current processes and suitability of existing tools to support the use of expert elicitation. Appl. Health Econ. Health Policy 16, 819–836 (2018).


    Google Scholar
     

  • 28.

    Waggoner, J., Carline, J. D. & Durning, S. J. Is there a consensus on consensus methodology? Descriptions and recommendations for future consensus research. Acad. Med. 91, 663–668 (2016).


    Google Scholar
     

  • 29.

    Butler, A. J., Thomas, M. K. & Pintar, K. D. M. Systematic review of expert elicitation methods as a tool for source attribution of enteric illness. Foodborne Pathog. Dis. 12, 367–382 (2015).


    Google Scholar
     

  • 30.

    Morgan, M. G. Use (and abuse) of expert elicitation in support of decision making for public policy. Proc. Natl Acad. Sci. USA 111, 7176–7184 (2014).


    Google Scholar
     

  • 31.

    Aspinall, W. P. & Cooke, R. M. in Quantifying scientific uncertainty from expert judgement elicitation. In Risk and Uncertainty Assessment for Natural Hazards (eds Rougier, J., Sparks, S. & Hill, L. J.) 64-99 (Cambridge Univ. Press, 2011).

  • 32.

    Graefe, A. & Armstrong, J. S. Comparing face-to-face meetings, nominal groups, Delphi and prediction markets on an estimation task. Int. J. Forecast. 27, 183–195 (2011).


    Google Scholar
     

  • 33.

    Black, N. et al. Consensus development methods: a review of best practice in creating clinical guidelines. J. Health Serv. Res. Policy 4, 236–248 (1999).


    Google Scholar
     

  • 34.

    Tol, R. S. J. Equitable cost-benefit analysis of climate change policies. Ecol. Econ. 36, 71–85 (2001).


    Google Scholar
     

  • 35.

    Tol, R. S. J. Estimates of the damage costs of climate change: part 1: benchmark estimates. Environ. Resour. Econ. 21, 47–73 (2002).


    Google Scholar
     

  • 36.

    Tol, R. S. J. Estimates of the damage costs of climate change: Part II. Dynamic estimates. Environ. Resour. Econ. 21, 135–160 (2002).


    Google Scholar
     

  • 37.

    Intergovernmental Panel on Climate Change (IPCC). Climate Change 2001: Synthesis report (Cambridge Univ. Press, 2001).

  • 38.

    Caddy, J. F. Limit reference points, traffic lights, and holistic approaches to fisheries management with minimal stock assessment input. Fish. Res. 56, 133–137 (2002).


    Google Scholar
     

  • 39.

    Wilson, D. C. & Pascoe, S. Chapter 13 Delivering complex scientific advice to multiple stakeholders. Dev. Aquacult. Fish. Sci. 36, 329–353 (2006).


    Google Scholar
     

  • 40.

    Bystrom, A. B., Naranjo-Madrigal, H. & Wehrtmann, I. S. Recomendaciones de manejo basadas en indicadores para la pesca artesanal con líneas de fondo en Costa Rica, Centroamérica. Rev. Biol. Trop. 65, 475–493 (2017).


    Google Scholar
     

  • 41.

    Simms, R. A. et al. Development of maternity dashboards across a UK health region; current practice, continuing problems. Eur. J. Obstet. Gynecol. Reprod. Biol. 170, 119–124 (2013).


    Google Scholar
     

  • 42.

    Donnelly, L. & Harrison, M. Geomorphological and geoforensic interpretation of maps, aerial imagery, conditions of diggability and the colour-coded RAG prioritization system in searches for criminal burials. Geol. Soc. Spec. Publ. 384, 174–194 (2013).


    Google Scholar
     

  • 43.

    Ruffell, A. & McAllister, S. RAG system for the management forensic and archaeological searches of burial grounds. Int. J. Archaeol. 3, 1–8 (2015).


    Google Scholar
     

  • 44.

    Mahony, M. & Hulme, M. The colour of risk: An exploration of the IPCC’s “burning embers” diagram. Spont. Gen. 6, 75–89 (2012).


    Google Scholar
     

  • 45.

    Vellinga, P. & Swart, R. The greenhouse marathon: a proposal for a global strategy. Clim. Change 18, vii–xii (1991).


    Google Scholar
     

  • 46.

    Schneider, B. Burning worlds of cartography: a critical approach to climate cosmograms of the Anthropocene. Geo Geogr. Env. 3, e00027 (2016).


    Google Scholar
     

  • 47.

    White, K. S. et al. Technical summary in Climate Change 2001: Impacts, Adaptation, and Vulnerability. Contribution of Working Group II to the Third Assessment Report of the Intergovernmental Panel on Climate Change (eds McCarthy, J. J., Canziani, O. F., Leary, N. A. & Dokken, D. J.) 19–73 (Cambridge Univ. Press, 2001).

  • 48.

    Mastrandrea, M. D. & Schneider, S. H. Probabilistic integrated assessment of “dangerous” climate change. Science 304, 571–575 (2004).


    Google Scholar
     

  • 49.

    Gattuso, J. P. et al. Contrasting futures for ocean and society from different anthropogenic CO2 emissions scenarios. Science 349, aac4722 (2015).


    Google Scholar
     

  • 50.

    Magnan, A. K. et al. Implications of the Paris agreement for the ocean. Nat. Clim. Change 6, 732–735 (2016).


    Google Scholar
     

  • 51.

    Fischlin, A. Berücksichtigen wir in der Klimapolitik genügend Sicherheitsmargen? Do we have sufficient safety margins in climate policy? GAIA Ecol. Perspect. Sci. Soc. 18, 193–199 (2009).


    Google Scholar
     

  • 52.

    Schneider, S. H. et al. in Climate Change 2007: Impacts, Adaptation and Vulnerability. Contribution of Working Group II to the Fourth Assessment Report of the Intergovernmental Panel on Climate Change (IPCC) Ch. 19 (eds Parry, M. L., Canziani, O. F., Palutikof, J. P., van der Linden, P. J. & Hanson, C. E.) 779–810 (Cambridge Univ. Press, 2007).

  • 53.

    Smith, J. B. et al. Assessing dangerous climate change through an update of the Intergovernmental Panel on Climate Change (IPCC) “reasons for concern”. Proc. Natl Acad. Sci. USA 106, 4133–4137 (2009).


    Google Scholar
     

  • 54.

    Intergovernmental Panel on Climate Change (IPCC). in Climate Change 2007: The Physical Science Basis. Contribution of Working Group I to the Fourth Assessment Report of the Intergovernmental Panel on Climate Change (IPCC) (eds Solomon, S. et al.) 1–18 (Cambridge Univ. Press, 2007).

  • 55.

    Intergovernmental Panel on Climate Change (IPCC) in Climate Change 2007: Impacts, Adaptation and Vulnerability. Contribution of Working Group II to the Fourth Assessment Report of the Intergovernmental Panel on Climate Change (IPCC) (eds Parry, M. L., Canziani, O. F., Palutikof, J. P., van der Linden, P. J. & Hanson, C. E.) 7–22 (Cambridge Univ. Press, 2007).

  • 56.

    Parry, M. L., Canziani, O. F., Palutikof, J. P., van der Linden, P. J. & Hanson, C. E. in Climate Change 2007: Impacts, Adaptation and Vulnerability. Contribution of Working Group II to the Fourth Assessment Report of the Intergovernmental Panel on Climate Change (IPCC) (eds Parry, M. L., Canziani, O. F., Palutikof, J. P., van der Linden, P. J. & Hanson, C. E.) 843–868 (Cambridge Univ. Press, 2007).

  • 57.

    Fischlin, A. et al. in Climate Change 2007: Impacts, Adaptation and Vulnerability. Contribution of Working Group II to the Fourth Assessment Report of the Intergovernmental Panel on Climate Change (IPCC) Ch. 4 (eds Parry, M. L., Canziani, O. F., Palutikof, J. P., van der Linden, P. J. & Hanson, C. E.) 211–272 (Cambridge Univ. Press, 2007).

  • 58.

    Hennessy, K. et al. in Climate Change 2007: Impacts, Adaptation and Vulnerability. Contribution of Working Group II to the Fourth Assessment Report of the Intergovernmental Panel on Climate Change (IPCC) Ch. 11 (eds Parry, M. L., Canziani, O. F., Palutikof, J. P., van der Linden, P. J. & Hanson, C. E.) 507–540 (Cambridge Univ. Press, 2007).

  • 59.

    Intergovernmental Panel on Climate Change (IPCC). in Climate Change 2007: Impacts, Adaptation and Vulnerability. Contribution of Working Group II to the Fourth Assessment Report of the Intergovernmental Panel on Climate Change (IPCC) (eds Parry, M. L., Canziani, O. F., Palutikof, J. P., van der Linden, P. J. & Hanson, C. E.) 23–78 (Cambridge Univ. Press, 2007).

  • 60.

    Oppenheimer, M. et al. in Climate Change 2014: Impacts, Adaptation, and Vulnerability. Part A: Global and Sectoral Aspects. Contribution of Working Group II to the Fifth Assessment Report of the Intergovernmental Panel on Climate Change (IPCC) Ch. 19 (eds Field, C. B. et al.) 1039–1099 (Cambridge University Press, 2014).

  • 61.

    Intergovernmental Panel on Climate Change (IPCC). Climate Change 2014: Synthesis Report. Contribution of Working Groups I, II and III to the Fifth Assessment Report of the Intergovernmental Panel on Climate Change (eds Core Writing Team, Pachauri, R. K. and Meyer, L. A.) (IPCC, 2014).

  • 62.

    Field, C. B. et al. Managing the Risks of Extreme Events and Disasters to Advance Climate Change Adaptation: Special Report of the Intergovernmental Panel on Climate Change (Cambridge Univ. Press, 2012).

  • 63.

    Intergovernmental Panel on Climate Change (IPCC). in Managing the Risks of Extreme Events and Disasters to Advance Climate Change Adaptation. A Special Report of Working Groups I and II of the Intergovernmental Panel on Climate Change (eds Field, C. B. et al) 1–19 (Cambridge Univ. Press, 2012).

  • 64.

    Hoegh-Guldberg, O. et al. in Global Warming of 1.5°C. An IPCC Special Report on the Impacts of Global Warming of 1.5 °C Above Pre-industrial Levels and Related Global Greenhouse Gas Emission Pathways, in the Context of Strengthening the Global Response to the Threat of Climate Change, Sustainable Development, and Efforts to Eradicate Poverty Ch. 3 (eds Masson-Delmotte, V. et al.) 175–311 (IPCC, 2018).

  • 65.

    Palazzo, A. et al. Linking regional stakeholder scenarios and shared socioeconomic pathways: quantified West African food and climate futures in a global context. Glob. Environ. Change 45, 227–242 (2017).


    Google Scholar
     

  • 66.

    Byers, E. et al. Global exposure and vulnerability to multi-sector development and climate change hotspots. Environ. Res. Lett. 13, 055012 (2018).


    Google Scholar
     

  • 67.

    Hanasaki, N. et al. A global water scarcity assessment under Shared Socio-economic Pathways – Part 2: Water availability and scarcity. Hydrol. Earth Syst. Sci. 17, 2393–2413 (2013).


    Google Scholar
     

  • 68.

    Hanasaki, N. et al. A global water scarcity assessment under Shared Socio-economic Pathways – Part 1: Water use. Hydrol. Earth Syst. Sci. 17, 2375–2391 (2013).


    Google Scholar
     

  • 69.

    Oppenheimer, M. et al. in IPCC Special Report on the Ocean and Cryosphere in a Changing Climate Ch. 4 (eds Pörtner, H.-O. et al.) 321–445 (IPCC, 2019).

  • 70.

    World Meteorological Organization. United In Science: High-Level Synthesis Report of Latest Climate Science Information Convened by the Science Advisory Group of the UN Climate Action Summit 2019 (World Meteorological Organization, 2019).

  • 71.

    Robinson, A., Calov, R. & Ganopolski, A. Multistability and critical thresholds of the Greenland ice sheet. Nat. Clim. Change 2, 429–432 (2012).


    Google Scholar
     

  • 72.

    Levermann, A. et al. The multimillennial sea-level commitment of global warming. Proc. Natl Acad. Sci. USA 110, 13745–13750 (2013).


    Google Scholar
     

  • 73.

    Levermann, A. et al. Projecting Antarctic ice discharge using response functions from SeaRISE ice-sheet models. Earth Syst. Dyn. 5, 271–293 (2014).


    Google Scholar
     

  • 74.

    Golledge, N. R. et al. The multi-millennial Antarctic commitment to future sea-level rise. Nature 526, 421–425 (2015).


    Google Scholar
     

  • 75.

    Fürst, J. J., Goelzer, H. & Huybrechts, P. Ice-dynamic projections of the Greenland ice sheet in response to atmospheric and oceanic warming. Cryosphere 9, 1039–1062 (2015).


    Google Scholar
     

  • 76.

    Pattyn, F. et al. The Greenland and Antarctic ice sheets under 1.5 °C global warming. Nat. Clim. Change 8, 1053–1061 (2018).


    Google Scholar
     

  • 77.

    Pörtner, H.-O. et al. in Climate Change 2014: Impacts, Adaptation, and Vulnerability. Part A: Global and Sectoral Aspects. Contribution of Working Group II to the Fifth Assessment Report of the Intergovernmental Panel on Climate Change (IPCC) Ch. 9 (eds Field, C. B. et al.) 411–484 (Cambridge Univ. Press, 2014).

  • 78.

    National Institute for Health and Care Excellence (NICE). Developing NICE Guidelines: The Manual. Process and Methods Guides (NICE, 2014).

  • 79.

    National Institute for Health and Care Excellence (NICE). Guide to the Methods of Technology Appraisal 2013 (NICE, 2013).

  • 80.

    Stafford, A. C., Bindoff, I. K., Tenni, P. C., Peterson, G. M. & Doran, C. M. A methodological framework for estimating the clinical and economic value of community pharmacists’ clinical interventions using expert opinion. J. Clin. Pharm. Ther. 37, 378–385 (2012).


    Google Scholar
     

  • 81.

    Khodyakov, D. et al. Conducting online expert panels: a feasibility and experimental replicability study. BMC Med. Res. Methodol. 11, 174 (2011).


    Google Scholar
     

  • 82.

    Sinha, I. P., Smyth, R. L. & Williamson, P. R. Using the Delphi technique to determine which outcomes to measure in clinical trials: recommendations for the future based on a systematic review of existing studies. PLoS Med. 8, e1000393 (2011).


    Google Scholar
     

  • 83.

    Moher, D., Schulz, K. F., Simera, I. & Altman, D. G. Guidance for developers of health research reporting guidelines. PLoS Med. 7, e1000217 (2010).


    Google Scholar
     

  • 84.

    Ferri, C. P. et al. Global prevalence of dementia: a Delphi consensus study. Lancet 366, 2112–2117 (2005).


    Google Scholar
     

  • 85.

    Cresswell, K. M. et al. Global research priorities to better understand the burden of iatrogenic harm in primary care: an international Delphi exercise. PLoS Med. 10, e1001554 (2013).


    Google Scholar
     

  • 86.

    Hasson, F., Keeney, S. & McKenna, H. Research guidelines for the Delphi survey technique. J. Adv. Nurs. 32, 1008–1015 (2000).


    Google Scholar
     

  • 87.

    Shrader-Frechette, K. Experts in Uncertainty: Opinion and Subjective Probability in Science. Ethics Vol. 103 (ed. Cooke, R. M.) 599–601 (Oxford Univ. Press, 1993).

  • 88.

    Ferguson, J. H. The NIH consensus development program: the evolution of guidelines. Int. J. Technol. Assess. Health Care 12, 460–474 (1996).


    Google Scholar
     

  • 89.

    Dalkey, N. & Helmer, O. An experimental application of the DELPHI method to the use of experts. Manag. Sci. 9, 458–467 (1963).


    Google Scholar
     

  • 90.

    Delbecq, A. L. & Van de Ven, A. H. A group process model for problem identification and program planning. J. Appl. Behav. Sci. 7, 466–492 (1971).


    Google Scholar
     

  • 91.

    Dias, L. C., Morton, A. & Quigley, J. in Elicitation: the Science and Art of Structuring Judgement (eds Dias, L. C., Morton, A. & Quigley, J.) 1–14 (Springer, 2017).

  • 92.

    James, D. & Warren-Forward, H. Research methods for formal consensus development. Nurse Res. 22, 35–40 (2015).


    Google Scholar
     

  • 93.

    Clemen, R. T. & Winkler, R. L. Combining probability distributions from experts in risk analysis. Risk Anal. 19, 187–203 (1999).


    Google Scholar
     

  • 94.

    European Food Safety Authority. Guidance on expert knowledge elicitation in food and feed safety risk assessment. EFSA J. 12, 3734 (2014).


    Google Scholar
     

  • 95.

    Walker, K. D., Evans, J. S. & MacIntosh, D. Use of expert judgment in exposure assessment. Part I. Characterization of personal exposure to benzene. J. Expo. Sci. Environ. Epidemiol. 11, 308–322 (2001).


    Google Scholar
     

  • 96.

    McCormack, C. G. et al. Key impacts of climate engineering on biodiversity and ecosystems, with priorities for future research. J. Integr. Environ. Sci. 13, 103–128 (2016).


    Google Scholar
     

  • 97.

    Griscom, B. W. et al. Natural climate solutions. Proc. Natl Acad. Sci. USA 114, 11645–11650 (2017).


    Google Scholar
     

  • 98.

    Mach, K. J. et al. Climate as a risk factor for armed conflict. Nature 571, 193–197 (2019).


    Google Scholar
     

  • 99.

    Bamber, J. L. & Aspinall, W. P. An expert judgement assessment of future sea level rise from the ice sheets. Nat. Clim. Change 3, 424–427 (2013).


    Google Scholar
     

  • 100.

    Mukherjee, N. et al. The Delphi technique in ecology and biological conservation: Applications and guidelines. Methods Ecol. Evol. 6, 1097–1109 (2015).


    Google Scholar
     

  • 101.

    Bhave, A. G., Conway, D., Dessai, S. & Stainforth, D. A. Water resource planning under future climate and socioeconomic uncertainty in the Cauvery River Basin in Karnataka, India. Water Resour. Res. 54, 708–728 (2018).


    Google Scholar
     

  • 102.

    Zommers, Z. & Alverson, K. in Resilience: The Science of Adaptation to Climate Change (eds Zommers, Z. & Alverson, K.). 329–336 (Elsevier, 2018).

  • 103.

    Berrang-Ford, L., Pearce, T. & Ford, J. D. Systematic review approaches for climate change adaptation research. Reg. Environ. Change 15, 755–769 (2015).


    Google Scholar
     

  • 104.

    Zickfeld, K., Morgan, M. G., Frame, D. J. & Keith, D. W. Expert judgments about transient climate response to alternative future trajectories of radiative forcing. Proc. Natl Acad. Sci. USA 107, 12451–12456 (2010).


    Google Scholar
     

  • 105.

    Kriegler, E., Hall, J. W., Held, H., Dawson, R. & Schellnhuber, H. J. Imprecise probability assessment of tipping points in the climate system. Proc. Natl Acad. Sci. USA 106, 5041–5046 (2009).


    Google Scholar
     

  • 106.

    Bantel, K. A. Comprehensiveness of strategic planning: the importance of heterogeneity of a top team. Psychol. Rep. 73, 35–49 (1993).


    Google Scholar
     

  • 107.

    Bolger, F. & Wright, G. Improving the Delphi process: lessons from social psychological research. Technol. Forecast. Soc. Change 78, 1500–1513 (2011).


    Google Scholar
     

  • 108.

    Hurlbert, M. et al. in Climate Change and Land: An IPCC Special Report on Climate Change, Desertification, Land Degradation, Sustainable Land Management, Food Security, and Greenhouse Gas Fluxes in Terrestrial Ecosystems Ch. 7 (eds Shukla, P. R. et al.) 673–800 (IPCC, 2019).

  • 109.

    Jairath, N. & Weinstein, J. The Delphi methodology (part two): a useful administrative approach. Can. J. Nurs. Adm. 7, 7–20 (1994).


    Google Scholar
     

  • 110.

    Bolger, F., Stranieri, A., Wright, G. & Yearwood, J. Does the Delphi process lead to increased accuracy in group-based judgmental forecasts or does it simply induce consensus amongst judgmental forecasters? Technol. Forecast. Soc. Change 78, 1671–1680 (2011).


    Google Scholar
     

  • 111.

    Singh, G. G. et al. Group elicitations yield more consistent, yet more uncertain experts in understanding risks to ecosystem services in New Zealand bays. PLoS One 12, e0190326 (2017).


    Google Scholar
     

  • 112.

    Speirs-Bridge, A. et al. Reducing overconfidence in the interval judgments of experts. Risk Anal. 30, 512–523 (2010).


    Google Scholar
     

  • 113.

    Sperber, D., Mortimer, D., Lorgelly, P. & Berlowitz, D. An expert on every street corner? Methods for eliciting distributions in geographically dispersed opinion pools. Value Health 16, 434–437 (2013).


    Google Scholar
     

  • 114.

    Adger, W. N., Brown, I. & Surminski, S. Advances in risk assessment for climate change adaptation policy. Philos. Trans. R. Soc. A Math. Phys. Eng. Sci. 376, 20180106 (2018).


    Google Scholar
     

  • 115.

    Yohe, G. “Reasons for concern” (about climate change) in the United States. Clim. Change 99, 295–302 (2010).


    Google Scholar
     

  • 116.

    Warszawski, L. et al. The inter-sectoral impact model intercomparison project (ISI–MIP): Project framework. Proc. Natl Acad. Sci. USA 111, 3228–3232 (2014).


    Google Scholar
     

  • 117.

    Frieler, K. et al. Assessing the impacts of 1.5°C global warming – simulation protocol of the Inter-Sectoral Impact Model Intercomparison Project (ISIMIP2b). Geosci. Model Dev. 10, 4321–4345 (2017).


    Google Scholar
     

  • 118.

    Sharpe, S. Telling the boiling frog what he needs to know: why climate change risks should be plotted as probability over time. Geosci. Commun. 2, 95–100 (2019).


    Google Scholar
     

  • 119.

    Colson, A. R. & Cooke, R. M. Expert elicitation: using the classical model to validate experts’ judgments. Rev. Environ. Econ. Policy 12, 113–132 (2018).


    Google Scholar
     

  • 120.

    Cooke, R., Mendel, M. & Thijs, W. Calibration and information in expert resolution; a classical approach. Automatica 24, 87–93 (1988).


    Google Scholar
     

  • 121.

    Field, C. B. et al. (eds) Climate Change 2014: Impacts, Adaptation, and Vulnerability. Part A: Global and Sectoral Aspects. Contribution of Working Group II to the Fifth Assessment Report of the Intergovernmental Panel on Climate Change (IPCC) (Cambridge Univ. Press, 2014).



  • Source link

    Leave a Reply

    Your email address will not be published. Required fields are marked *