Decision-Making for Managing Climate-Related Risks: Unpacking the Decision Process to Avoid “Trial-and-Error” Responses

Abstract

We provide an overview of decision support tools and methods that are available for managing climate-related risks and for delivering adaptation and resilience options and solutions. The importance of understanding political, socio-economic and cultural contexts and the decision processes that these tools support is emphasized. No tool or method is universally suited to all circumstances. Some decision processes are structured with formal governance requirements; while others are less so. In all cases, discussions and interactions with stakeholders and other players will have formal and informal aspects. We categorize decision support tools in several broad ways with the aim of helping decision makers and their advisors select tools that are appropriate to their culture, resources and other circumstances. The assessment examines the constraints and methodological assumptions that need be considered.

Introduction

Climate change only differs from other risk management problems by the fact there is only one Earth and a number of the risks are existential to life on the planet. But it is still about managing risk, for which there is an immense body of literature and decades of experience to draw upon; the wheel need not be reinvented. Risks (Simpson et al., 2021) can be directly related to greenhouse gas emissions, such as the risk related to exceeding an average increase in air temperature since preindustrial times of 1.5°C, or indirectly related to that, such as health outcomes from a warming climate (Vanos et al., 2020) (see Box 1). Further, climate risks may also relate to risks from actions used to ameliorate other risks.

Figure 1

Deciding on actions to ameliorate climate-related risks is a very human process; many psychological factors may impact on the cognitive and deliberative processes of individuals and organizations (Orlove et al., 2020). These factors play important roles in making sense of the problems to be tackled and in the final decisions on what actions may be taken. They can also influence choices (decisions) on the types of methods to employ to inform decision-making. Actual methods, tools and processes for making decisions on climate change are not often discussed, except in a macro sense, when talking about how to manage climate change, as if the solution and the problem are inextricably linked and seemingly obvious. Yet managing risks, particularly indirect ones, may be achieved through many different pathways, many of which may involve great uncertainty, and which will have varying benefits, costs and effectiveness for ameliorating risk. A decision-maker may potentially have leanings to one or a few of many options, depending on their own preferences, which may not be in the interests of everyone. Moreover, individual steps to manage climate risks will have varying degrees of reversibility, potentially locking in future pathways. How can errors of judgement be reduced, and poor outcomes avoided as far as possible?

A climate management process is more likely to require iterative approaches over many years, if not decades, because a number of risks are expected to emerge in the future and actions are needed now in order to take effect to reduce risk before the future arrives. While “trial-and-error” processes may be a necessary option for managing emergency responses to extreme events, it is not an option for timely mitigation intended for limiting global mean temperatures to not rising above 2°C (IPCC, 2018). Nor is it an option under many circumstances where individuals, communities and sectors may be seriously disadvantaged by a proposed action. So what decision methods are available that could be brought to bear in resolving and deciding on (locally, regionally, globally) the best courses of action to tackle climate change, adapt to the challenges that are unable to be avoided, and enable the greatest chance for climate recovery and resilience of natural and human systems? More importantly and bearing in mind the human influences in process and outcomes, how might the complexity of the interactions of different risks (Simpson et al., 2021) be unpicked and made relatively straightforward in order to appraise how best to tackle the problem with limited resources, and many competing interests and perspectives?

In effect, risk management under climate change is the same as managing a nested “control system” (Box 1). The outer system is Earth's climate, with inner systems in Earth's regions, and progressing further inward toward locations (cities; or, in nature, ecosystems), and specific instances (houses, households and individuals; forests, glades and colonies). At whatever level, the principles of a control system can apply. It is not just a top-down process, defined as measures or regulations adopted by governments or corporate bodies, but includes bottom-up processes where actions can be taken by individuals or collectives of individuals with a mind to “think globally-act locally”. There is an interaction within and between these levels to achieve an effective control system at any level. Decision-makers need to be aware of these interactions in order to provide effective responses; the degree to which the interactions need to be made explicit will depend on the scale of response being considered. Also, they need to be aware that, unlike a readily-understood control system such as a thermostat, adjustments (or iterations) will be needed along the way to correct the trajectory, rather than allow overshoots and a need for subsequent correction and restoration.

Decision processes do not have a uniform structure. The circumstances surrounding decisions may differ in many ways. There may be a range of uncertainties involved, differing in character and scale (see Box 1, Figure 1, Table 1). For climate change, the issues may relate to protecting a small locality or community or perhaps adapting a region or continent; and the potential consequences of the decision may vary from something quite moderate up to very significant, perhaps existential. The people involved can vary from a single decision-maker to several decision-makers, or in the climate change context often local or national governments with a plethora of stakeholders. The objectives may be unclear at the outset, often contentious among the decision makers and stakeholders. Experts may disagree in their advice, and data to resolve their differences may be sparse. The formal governance structures which dictate who should decide, and their responsibilities, authorities and accountabilities, can constrain the decision-making considerably, including the formal interactions with stakeholders. At the same time, availability of social and more traditional media ensures that informal debate among all players – decision makers, stakeholders, experts, and decision analysts – and the public will take place beyond the formal decision process creating expectations and even bottom-up actions, yet also providing a lot of relevant and useful information. Against such a breadth of circumstances, it is inconceivable that a single decision-making tool will be suitable for all cases: although some proponents of a method or software application might suggest otherwise in their marketing. People seeking to manage responses to the climate challenges amongst a portfolio of other challenges can find linking tools to tasks within their context a challenge in itself.

Here, we recognize that decision-making for managing climate-related risks is most likely to be unorganized, unstructured and, in simple terms, messy at the outset. However, many risks need to be managed by a collective of people, using processes that enable collective and repeatable outcomes, whether they be through communities, businesses and industries, civil society groups, and governments. The more often the processes are predictable and repeatable, the more they can be used by others in similar contexts. We lean to decision analytic approaches that can navigate the complex nature of risk management and make explicit the nature and background of a decision.

Our aim is to provide an overview for policy-makers on what tools may be useful to support decision-making in managing climate-related risks, recognizing the complexity of the issues both in the physical world and the socio-political world in which the decisions have to be made. In doing so we also provide a guided literature review, both directly and through many of our citations. Decision-making (hereafter DM when used as an adjective) literature specifically oriented toward “climate change” is sparse (Figure 2). We separate DM literature from other literatures that may relate to the causes and drivers of the phenomena that underpin understanding the risks; we seek literature specifically related to the decision process. A broader literature on decision-making tools is used in this overview, with links to experience in their application to climate-related DM. While we do not undertake an exhaustive and systematic review, we have covered sufficient breadth for the reader to find examples of the application of the main DM tools and techniques available.

Figure 2

The first part of our overview relates to framing decision-making. We identify the components of making decisions to respond to climate risks in a timely manner; some components of decision-making may have greater importance than others depending on the context for the decision-makers. By doing so, we aim to provide a framework in which the context for decision-making can be better understood and the tools better utilized. The second part is about the decision process, presenting phases of and approaches to the process. In particular, we then catalog several decision-making tools in ways that should enable individuals, communities, organizations and government departments and agencies identify a small number that may suit their needs in relation to climate change adaption and mitigation. We aim to help problem-solvers and their advisors become “intelligent customers” of decision analysis. We make no claims that our advice is objective in any sense; any catalog requires judgement to classify each item. However, we hope that we have written this paper in a way that catalyses a “pause for thought” so that users will better understand the various ways to make a decision utilizing a suite of available methods and tools. Lastly, we provide some insights from our experience on the road ahead for managing the climate challenges.

Framing Decision-Making

Approaches to Making Decisions

The majority of our decision-making is informal, barely structured with little explicit deliberation and made in ways of which we may be barely conscious; of course, the majority of our decision-making concerns things such as when to eat lunch or the route to take across a station concourse. Consideration of the consequences of different outcomes may be barely noticeable. The degree to which risks may be considered depends on how “lucky one feels”, which is an important motivator as to whether systematic approaches are used (risk aversion) or not (risk tolerant). In this case, risk aversion is less about fear and avoidance but more about determining that the consequence of a risk being realized cannot be tolerated. More systematic approaches may be something like needing to get to a meeting on time and considering the timeliness of different options for routes across the city. For significant decisions and in a professional context, we usually seek to make our decisions more formally, more “rationally” (“blind luck” is not an option), and in an auditable way, perhaps supported by some form of decision analysis and evidence. In groups we deliberate and seek to resolve differences of views. In organizations and governments there are formal governance rules and constitutions determining the decision processes, authorities and accountabilities; but alongside these, informal discussions inevitably take place, influencing the outcome.

That any form of decision analysis necessarily imposes some form of consistency and rationality upon the explicit modeling of the objectives and uncertainties is often not appreciated. Moreover, the consistency and rationality assumed by some approaches may contradict those assumed by others, making the use of some tools incompatible with the use of others. Thus users need to check that they not only understand but agree with the assumptions underpinning those methods that they adopt, or they may be misled by or misuse the results. A very important aspect that distinguishes methodologies of decision analysis is whether they seek to be objective or instead render subjective judgements explicit, taking into consideration diverse values and uncertainties. That said, the application of any method, whether its assumptions fit with the users' perspectives or not, stimulates discussion and focuses attention on understanding the issues, and that alone can be enormously beneficial.

Informal decision-making may not naturally satisfy many of the assumptions made by a decision analytic method. The simplistic response is that informal decision-making is about securing an outcome that may not be easily justified, and that formal decision-making embodies principles to ensure that important decisions are made soundly and rationally. However, in practice, informality and formality run side by side and can be more harmonious, making it not quite so easy to make such a ready distinction (Hodgkinson and Starbuck, 2008; Gregory et al., 2012; French and Argyris, 2018). At the individual level, Kahneman, Tversky and many others have investigated the differences between informal and formal decision making (Kahneman and Tversky, 1974; Morton and Fasolo, 2009; Kahneman, 2011; Montibeller and Winterfeldt, 2015). For many years this work was discussed under the general heading of heuristics and biases, recognizing that informal decision-making uses “quick and dirty” heuristics to make choices, but at the risk of biasing choices on average away from what various principles of rationality would suggest. More recently, the terminology has changed to talking about:

Whether there is a true dichotomy here is moot and there are many other subtleties discussed in the literature (Shleifer, 2012; Evans and Stanovich, 2013). However, this simple distinction is sufficient for our purposes. Decision analysis seeks to encourage System 2 Thinking, helping decision-makers, their advisors and stakeholders each individually think through and reflect on the issues. However, it is easy in discussions and specifically in articulating probability and value judgements, to slip into System 1 Thinking. Better methodologies and tools have elicitation processes for nudging and challenging participants to think carefully and explicitly when giving judgements, but weaker ones simply take the responses and build them into the analysis.

Most decisions are made by groups and there are informal and formal aspects to their interactions and deliberation. Sometimes a decision is reached by simple discussion and consensus, or maybe an informal vote. Other times, “horse-trading” and other agreements can connect decisions (“… and you vote with me next time”). Less democratically, there may be a dominant leader who influences agreement with their views. Business, organizational and political/government decision making are bound by more formal governance structures and constitutions, which prescribe who can take part, what interactions are allowed, how stakeholders may have their voice heard, voting systems, etc. In many societies, decision-making has become more inclusive with stakeholders and the public being consulted formally (Bayley, 2008; Renn, 2008; Rios Insua and French, 2010). This is particularly true in the environmental domain in which many modern techniques of stakeholder engagement, public participation and deliberative democracy have been developed (Beierle and Cayford, 2002; Gregory et al., 2012). Alongside such inclusive deliberations, inevitably informal discussions are also influential. In businesses and organizations, these may be no more than “water-cooler” conversations; but the advent of social media has allowed much wider, often very influential discussion to take place for all types and scales of decision-making. To parallel the distinction between System 1 and 2 Thinking, French and Argyris (2018) have introduced the terminology:

Decision analysis is aimed primarily at supporting System 2 Societal Deliberation helping those charged with the responsibility of taking the decision to do so in an informed, auditable and explicit way. It should, however, recognize the information sources provided by System 1 Societal deliberation such as social media, from which the decision-makers can learn about stakeholder values and other perspectives on the issues, thus ensuring that they are aware of breadth and depth of the issues that they face. Indeed, decision analysis can help in using its tools to communicate the decision-makers' reasoning to their stakeholders, particularly in the formulation and implementation stages of decision-making (French et al., 2005; Morton et al., 2009). In more inclusive decision-making, decision analysis can articulate discussions between decision-makers, stakeholders and experts, drawing the System 1 Societal Deliberations into the formal System 2 process (Mustajoki et al., 2004; Gregory et al., 2012; French and Argyris, 2018). It is important to recognize that broad processes which support this transition to System 2 Thinking and Social Deliberation are context dependent and depend on the skills of the analysis teams rather than something that can be achieved in an almost mechanical way alongside the modeling, computations and analysis. Effective decision analysis requires many more diverse skills than some mathematical and algorithmic introductions to decision analysis suggest.

Modeling and Decision Analysis

Decision analysis requires two forms of modeling. First, there is a need to model the external context and the physical issues being addressed; in our case, some climate change impacts. Such modeling is descriptive of the context and can be validated empirically if there are data available, though in many examples of risk management and mitigation, preventive actions are needed before full data may become available. Such models are constructs of two or more entities and the relationships and influences between them. In managing risks, the model may simply be: “if we choose to undertake this action to ameliorate the unacceptable risk, then these consequences will arise because of these reasons.” This model may be founded on the professional judgement of decision makers and their experts or more empirically-based. Options for actions may be further elaborated by alternative beliefs or observations relating actions, the system being managed, and the consequences. The model can be made more robust by assembling knowledge relating to:

• What are the drivers of the risk and how might it be ameliorated?

• What makes the risk unacceptable?

• How specific does the action need to be described in order to fully understand the consequences?

• How does the action interact with the system to deliver consequences?

These questions may be explored with heuristic models, network (pathway) models, statistical models, dynamic mathematical models or a mixture of types, depending on the available knowledge and data. Some approaches to decision analysis are limited to specific types of models, while others are more flexible. The robustness of a model for decision making is determined by the degree to which an action will be systematically chosen for the task and correctly ameliorate risks as expected. Box 2 illustrates the development of a model highlighting some connections of different parts of the management and Earth Systems in managing the risks of damage from flooding.

Figure 3

The second form of modeling related to the decision-makers' and stakeholders' beliefs, values and objectives. These are more subjective and do not allow empirical validation. Moreover, the modeling is not descriptive in the sense that beliefs, values and objectives exist fully and explicitly before the modeling process begins. Rather the process of elicitation helps the participants reflect on what they are truly seeking to achieve and constructs the detailed objectives for the analysis (Keeney, 1992; Lichtenstein and Slovic, 2006; French, 2021). This form of modeling is particularly focused on helping the participants move from System 1 Thinking toward System 2 Thinking as it generally introduces rationality conditions that help their values become more consistent. Such modeling is known as prescriptive rather than descriptive modeling. Comparing and deliberating on different prescriptive models may also be important in Social Deliberation if some stakeholders hold to dogma that conflicts with current established approaches. While good decision-making depends on sound empirical description of the context, the process of reaching a decision and consensual acceptance of the selected course of action are not necessarily helped by effectively informing some of the stakeholders that they are “simply wrong” (French and Argyris, 2018), so the deliberations around prescriptive models need to be carefully and sensitively facilitated.

There is much emphasis currently on evidence-based decision-making and we would certainly echo this, but with a careful interpretation. Evidence and the knowledge that it supports is often encoded in descriptive models. We only have direct observations about the past. Decision-making is about planning for the future and so to use observations, we must make judgements about its relevance to the future: do we believe that things will continue in this way or that? Moreover, in many cases relating to risk management, what evidence we have is partial, if indeed we have sufficient data to claim any validated evidence at all. This inevitably means that there is a tension between scientific advisors, who want more time to accumulate and validate evidence, and the decision-makers, who need to make urgent decisions to mitigate risks. Decision analysts need to appreciate this tension in managing the deliberations between decision makers, their scientific experts and stakeholders.

A tension exists between observations, evidence, models about the future and effective risk management related to climate change; what constitutes evidence in climate-related risk management? In the last three decades, the Drivers-Pressures-State change-Impact-Response (DPSIR) framework has grown to underpin evidence-based management (Patrício et al., 2016). At its heart, is the need to attribute change to a driving cause; attribution of climate change to greenhouse gas emmissions from human activity has been a central theme of the Intergovernmental Panel on Climate Change (Bindoff et al., 2013). The application of DPSIR is usually at small spatial scales (10,000 km² at most) (Patrício et al., 2016) with a view that impacts can be detected and, once detected, restoration would be possible within a similar time frame. Such an approach implies that failure to not have a significant impact can be easily detected and rectified. In climate change, this is equivalent to accepting that an overshoot of a target global mean temperature would not be a disaster and that the climate can be restored before disastrous effects would arise. Yet, we know that the effects of greenhouse gases emitted now will take many decades before their effects will be diminished. Managing this risk requires decisions on actions well in advance of observations demonstrating impacts. Timeliness for action in risk management is an important factor not usually associated with DPSIR analyses. Uncertainty in both descriptive and prescriptive modeling increases the risks of failure. Evidence, therefore, needs to comprise not only observations of the state of the system but consider their power for detecting change and attributing it to the causes, as well as the degree to which models can help manage future risks.

Lastly, we are aware that the prescriptive modeling and analysis used to support decision-making can be interpreted naively as algorithmic computations on simple, often linear models without concern for the wider processes that these calculations support. To be frank, one of us has seen many naïve analyses performed by quantitatively adept scientists which have not truly supported the socio-political processes that surround any complex decision and which consequently have not really informed the decision-makers and stakeholders. Effective decision analysis uses the prescriptive modeling to articulate the deliberations between the participants building a shared understanding of the issues and each other's perceptions and values. It is through that shared understanding that a decision emerges, not simply from the maximization of some objective function. So in the next sections we emphasize many such “softer” socio-political issues that need to be reflected upon and brought into the deliberations in developing appropriate decision analyses for a set of issues.

Contextual Issues

Decisions are affected by many contextual factors beyond the formality and constraints of the processes used (see French and Geldermann, 2005; Hodgkinson and Starbuck, 2008; French et al., 2009 for a wider discussion of contextual issues) and can be explicitly incorporated in the analyses leading to and supporting decisions. Some important issues to consider are:

• The “decision-maker” may be an individual, a group sharing responsibility, a community, an organization or other legal entity, a government or, in some senses, by society itself.

• The degree of “system understanding” of the risk to be managed, including the root causes and drivers of the risk (cause and effect), and the way the risk and its consequent effects manifest and encoded by the descriptive model of the system.

• The range of possible options for managing risk may vary between a few alternatives and effectively an infinite number.

• There can be many other players involved: stakeholders who will share in the potential impacts in diverse ways; experts who can advise on possible actions, other risks and consequences; and the decision analysts who develop the decision modeling and use this to help articulate deliberation.

• Culture is an important element, especially in relation to the recognition of subjectivity in the discussions and whether this should be modeled and made explicit. Stakeholders' response to risk and uncertainty has many cultural influences (Hofstede, 1984; Thompson et al., 1990; Douglas, 1992). Since climate change is a global issue, variation in local culture can mean that a decision tool and approach appropriate to a set of issues in one region may be inappropriate for seemingly the same issues in another.

• The time and other resources available before a decision must be made can constrain the range of investigations and modeling used.

• The range and depth of uncertainties involved are very important (see Subsection Types of Uncertainties).

• The values and objectives driving the decision may differ between options too. In the private sector, the profit motive and shareholder value may not override other criteria, but financial objectives have high weight; whereas the public sector have more altruistic objectives reflecting responsibilities to, e.g., the public, maintaining society and the environment (see Subsection Values and Objectives).

Box 2 develops these concepts as part of a control system, using a simplified example of managing the risks associated with flooding based on a model of how the management and Earth system might work. The contextual issues that arise in mitigating and adapting to climate change are important in determining suitable decision processes and tools. Firstly, the threatened impacts are global, but with many regional and continental differences in the scale and type of impact. Different areas of societal and business activity will be affected differently. A wide range of stakeholder interests will need to be considered in almost all cases, including those of humanity itself. There are ethical and moral issues to consider alongside more prosaic objectives. The breadth of uncertainties, many of which are deep and difficult to assess, is huge. While less so than in previous decades, climate science is controversial, and consensus will be hard to achieve across all stakeholders.

Types of Uncertainties

Situations of Uncertainty

The breadth of uncertainties can be overwhelming. Courses of action are broadly constrained by the knowledge readily available and the familiarity present in a situation. Identifying the general situation with respect to uncertainty will direct DM planning from the beginning and facilitate communication on what is required.

Cynefin is a way of categorizing decision contexts (“spaces”) according to the decision-makers' and their experts' knowledge of cause and effect and hence their ability to model the system (Snowden, 2002; French, 2013) (see also the community of practice—https://www.cognitive-edge.com/). If a context is known or knowable, then it will be possible usually to build sophisticated models and make sound predictions; but if the context is complex and chaotic only the simplest of models will be possible. Courtney (2003) and others have characterized uncertainty simply on the quality and precision of models that can be built and developed, a very similar categorization to Cynefin. We have chosen to go with the Cynefin formalism since it seems clearer to us to think of knowledge of cause and effect in general terms, rather than when knowledge is specifically expressed as a formal model.

Cynefin recognizes four broad cases (Figure 4):

Figure 4

There is a fifth area in Figure 4, relating to disordered contexts, i.e., those contexts which have yet to be classified. While disordered contexts may be important in other applications of Cynefin, one of the first tasks in problem formulation is to understand and classify the context so the disordered area quickly becomes irrelevant to decision analysis. Moving from the Chaotic Space through the Complex and Knowable Spaces to the Known Space, our knowledge and understanding move from deep uncertainty to certainty. Epistemology from sense-making through inference to full knowledge can be described very simply against the backdrop of Cynefin (French, 2013). Various decision analytic techniques are available for the Known, Knowable and Complex Spaces, but in the Chaotic Space decision-making is a matter of trial and error; or, if there is time, defer any decision and investigate the situation to see if one can learn enough to move the context into the Complex Space. An outcome of this approach is to provide greater openness as to the scale of the problem relative to the knowledge base, which then promotes more obvious courses of action.

Types of Uncertainty

Decision-makers can face many forms of uncertainty. Many typologies have been developed to describe these, each focusing on one or more characteristics (Knight, 1921; Berkeley and Humphreys, 1982; French, 1995b; Paté-Cornell, 1996; French et al., 2020). Discussions of uncertainties often focus on the external world within which the problem is being faced, and for which we describe three types of uncertainties—stochastic, epistemic (structural) and analytical. We also describe two important uncertainties internal to the decision process—ambiguity and values—relating to the decision-makers' and stakeholders' perceptions and valuations of the world. These five uncertainties are:

Decision tools and processes can address all five uncertainties, though many concentrate on just one or two. For instance, confining attention to ambiguity and value uncertainties and ignoring stochastic and epistemic uncertainties can help focus discussion sufficiently to clarify goals and objectives and support deliberation between wider stakeholder groups and decision-makers. Partitioning and classifying the components of any uncertainty into these five wide categories is a matter of judgement; but the process of doing so catalyses discussion and helps ensure that all uncertainties are noted in any analysis, even if some are subsequently ignored in order to focus on others.

Any of these uncertainties can be too deep to be modeled and analyzed or resolved by deliberation within the time and resources available for a decision. This may happen because data are very sparse, expert disagreements very wide or, in the case of value uncertainties, ethical issues extremely complex and controversial. In such cases, however the uncertainties should be dealt with in principle; the depth of disagreement between experts and stakeholders, the lack of data, and the need to make a decision relatively quickly mean that in practice methods that can deal with deep uncertainties will need to be adopted until the uncertainties can be resolved (Walker et al., 2013; Marchau et al., 2019; French, 2020). We discuss this further below.

Table 2 helps relates these different types of uncertainties to some of the challenges that arise in facing up to a climate change issue. For instance, consider specific hazards (row 2). Uncertainties may concern (i) the frequency with which an extreme weather condition occurs; (ii) how large a change in the weather extremes will occur; (iii) how well we can predict the weather pattern; (iv) what the goals and objectives would be in adapting to the new pattern and (v) how serious the effects would be in terms of these goals and objectives.

Uncertainty does not just relate to what might happen (i.e. stochastic, epistemic and analytical uncertainties); but also to how well potential impacts can be described and valued (i.e. ambiguity and value uncertainties). This can be true at organizational and governmental levels as much as for individuals; and may be particularly the case when the scale of an issue in space or time is large, as is usual in climate change contexts. Issues that extend over regions or countries or over long timespans have strong tendencies to be set in complex socio-political and economic contexts in which values are uncertain and hotly debated, making them complex or even chaotic contexts for decision-making, however straightforward a technical solution might seem.

The balance between how particular decision analyses address uncertainties relating to the external world and those relating to the values driving the decision making is important. Some analyses partially ignore uncertainties relating to the former in order to focus on conflicts in the values held by different stakeholders and help structure debate; others build very sophisticated models of the external world to predict potential consequences, but in doing so lose transparency and risk becoming untrustworthy black boxes to many stakeholders. There are no methods which guarantee to balance such conflicts and provide a oath through such complexities, but skilled decision analysts have the professional facilitation skills that can help find a resolution (Phillips, 2007; French et al., 2009).

Values and Objectives

Decision making is driven by values, by what the decision-makers want to achieve. Values are necessarily subjective, but in societies that seek to avoid explicit subjectivity in their decision-making, economics and financial theory provide ways of costing many climate change impacts in a broadly objective manner; but there are some “intangible” impacts such as the loss of a historic site or natural estate, or the cultural impact of moving communities that are difficult to cost. The ability to assess intangible impacts, albeit subjectively, is one of the characteristics that distinguish different schools of decision analysis.

Although we have discussed contextual issues and uncertainties first, good decision making in practice follows value-focused thinking. Keeney (1992) describes this as “first deciding what you want then figuring out how to get it.” This runs counter to the more usual alternative-focused approach: namely first identifying some alternatives, then deciding between them. However, value-focused decision making is more creative, not being confined to a set of pre-defined alternatives. Moreover, being aware of the objectives of a decision analysis at the outset means that analytic effort can be focused on what matters, avoiding irrelevancies and providing the means by which any options can be evaluated for their contribution to a solution. In particular, since climate change, environmental and economic models can be very computationally expensive to analyse, value-focused thinking can direct effort to calculating what matters; the scale of the effort required to address the problem can be more easily identified.

Economic and financial methods provide one way of exploring values and objectives: e.g. cost-benefit methods seek for each alternative to evaluate the total cost of implementation and consider it relative to the cost of potential impacts that the alternative ameliorates (Boardman et al., 2017; OECD, 2018). More generally, value and utility methods offer ways of assessing both tangible and intangible costs and benefits, albeit relying more of subjective or, at least, less objective inputs (Keeney, 1992; Keeney and Raiffa, 1993; Bedford et al., 2005).

In passing, we note that many perspectives on rational decision-making separate the Science from the Values that need to be balanced in making a choice. By “Science” we mean the knowledge and investigations that can be brought to bear on resolving the issue and addressing the uncertainties. By “Values”, we mean the decision-makers' objectives that the ultimate choice seeks to meet. Of course, in any democratic society addressing climate change issues, the decision-makers should draw stakeholders' objectives into the ones that they use in the analysis (Renn, 2008; Rios Insua and French, 2010).

The Decision-Making Process

In control systems, a decision process will have in mind to update the controls in a regular feedback process. Simple control systems will have only one control with a pre-determined target for adjusting the control based on a measurement of the state of the system – as in the thermostat in our example. More diffuse controllers such as for the climate system, may use many different actions including new actions as differences between the state of the system and the target are measured, considered and responded to. If a risk has only been identified for the first time, then part of the decision process in this first instance will be to determine not only the actions to ameliorate the risk but also whether, and when, to assess in the future the success, failure or other impacts of the actions and whether subsequent adjustments or new actions are needed.

Phases

Almost every writer on decision analysis has summarized the decision-making process as a cyclic iteration of several phases (see the many citations to decision analysis in this paper). Here we use three broad phases, which are adequate for our purposes:

• Sense-Making and Modeling: Before any auditable, rational decision-making can begin, it is necessary to identify issues, values, objectives, uncertainties, stakeholders, possible actions and their consequences, engage with stakeholders and consult experts as needed, and determine the scope and boundaries of the subsequent analyses. Only when substantial progress has been made on these, is it possible to build a quantitative model and conduct any analysis. This is also a time when the interaction between different risks and decision processes can be mapped, the relative importance of each identified, and the need for integration in their ongoing management.

• Analyzing and Exploring: Once a model is built and/or appropriate existing data and knowledge services are identified, exploration and analyses are undertaken in relation to the study's objectives, options and generally building an understanding. Sensitivity and robustness analyses may – should – supplement the decision analysis, setting bounds on some of the residual uncertainty. During the process, the model and information should be validated as much as possible against available data and the decision-makers', experts' and stakeholders' perceptions. The detail and application of this phase is very much dependent on the Cynefin context in which the problem starts out, and, if needed, how much time may be available to move the problem from one context to the next.

• Interpreting and Implementing: The results and guidance offered by the analysis need to be interpreted into real world actions. This requires that the decision-makers and analysts make a judgement whether the analysis is adequate or, in technical terminology, requisite for the decision, guiding them to a consensus on the way forward (French et al., 2009). They need to judge whether the model, the analysis and the conclusions are fit for their purposes. Once made, they will also need to communicate the decision to stakeholders and implement the actions.

Figure 5 relates the three phases to the use of data from the real world, and choosing from available options to meet the policy objectives. The left-hand side of the graphic corresponds to the discussions, deliberations, analyses and studies that support the decision making. The right-hand side relates to the real world, which is always too complex to be perfectly modeled or analyzed. We emphasize that the real world includes not just relevant changes in climate, but human society, the environment, business, industry and agriculture and all the systems that need be considered in developing policies in mitigation and adaption.

Figure 5

Generally, the three phases of decision-making proceed from the top to the bottom of the graphic and are indicated by the bulleted lists, but we recognize that analyses, discussions and deliberation will iterate backwords and forwards as understanding of the issue grows. The “decision-maker” at the bottom of the graphic is to be understood as the person or, more likely, group, who are responsible and accountable for the decision under the appropriate governance structure. We emphasize that the interactions of this decision-maker with the real world include appropriate consultations and engagement with stakeholders.

This apparently linear approach from problem to decision implies risk management is organized, that all risks are identified and the processes set in train are carried out with some order, including the monitoring of success. Yet this is obviously rarely the case. Invariably, the process iterates within and between phases as thinking about one issue catalyses further thoughts about other issues or reflections during one phase indicate that other issues should have been considered in an earlier one. Further, problems may be latent, arising at seemingly random times, decisions postponed, and attention of scarce resources diverted to other purposes some way during the process. Moreover, many risks will be interrelated, and will be dealt with on differing timelines and urgencies. Being mindful of these relationships between risks and between their management process can help reduce tensions between them, take advantage of synergies in activities and processes, and avoid inadvertent negative consequences between risks.

Simplistically, decision analytic studies tend to be conducted in one of two modes (Franco and Montibeller, 2010).

• The expert mode in which the analysts work away from the decision-makers, experts and stakeholders, consulting them individually or groups as necessary to gather information. Such studies are common in addressing problems in the Cynefin Known and Knowable spaces. Because such problems occur commonly, well-structured models are relatively easy to build. The analysts' task is mainly to run sophisticated computer codes to explore and analyse the system.

• The facilitated modeling mode in which analysts and decision-makers, accompanied maybe by some experts and stakeholders, meet in one or more workshops to “solve” the problem. Such studies are common in tackling contexts lying in the Cynefin Complex and Chaotic spaces. Initially the emphasis is on understanding the perceptions of the group on what is happening and identifying possible strategies that may be taken up in response, and on the values that will drive their decision-making. Later, quantitative models are built in the presence of the group to capture these and numerical inputs elicited for those quantities that cannot be inferred from “objective” data. The group see and explore the analysis together, before deciding on a course of action.

This rough dichotomy is an oversimplification; many studies involve elements of both. Large projects dealing with complex issues, and integrating across related risks, may begin with several facilitated workshops to explore and identify issues, creating a series of questions. These questions are then explored through sophisticated modeling studies carried out in the expert mode. Later, there may be a return to facilitated modeling to share what has been learnt and evaluate possible strategies, providing guidance to the decision-makers. Some or all of the workshops might be conducted as face-to-face events or remotely (Coakes et al., 2002; French et al., 2009; Nunamaker et al., 2014; Pyrko et al., 2019).

Approaches to Decision Analysis

Decision analyses comprise many families of techniques, some with sufficient philosophical and methodological underpinnings to be called a “school”; while others are more collections of techniques with enough common qualities to be grouped together. We have categorized seven broad classes of techniques that support decision making and give details of each approach in Table 3, identifying how they relate to the general considerations in our earlier discussion.

Bayesian Methods

Bayesian methods provide a structured approach to assembling information around the consequences of choices, either by modeling, analysis of multiple scenarios or structuring deliberation. They can address all types of uncertainties, and are underpinned by axiomatic theoretical bases and powerful computational methods. These most assuredly form a school built on a coherent set of assumptions and philosophical perspectives. Methods can draw in both hard data and expert knowledge weighing them together appropriately. They use the same Bayesian statistical approaches that lie at the heart of many machine learning and artificial intelligence algorithms. Intuitive, graphical interfaces such as decision trees, belief nets and influence diagrams make the methods relatively transparent. Bayesian methods emphasize the auditable, building of consensus. They make explicit the biases (subjectivity) of information, stakeholders and the decision-maker. Traditionally, Bayesian methods use probability to represent uncertainty, multi-attribute utility functions to represent preference and then maximize expected utility to identify an “optimal” decision. As such they apply in the Known and Knowable Spaces. However, the use of multiple scenarios, sensitivity analysis and exploratory decision conferences enable the methods to be applied in the Complex Space (Keeney and Raiffa, 1993; French and Rios Insua, 2000; Smith, 2010; Howard and Abbas, 2016; French, 2020; Workman et al., 2021) (See Table 3a for further details).

Decision-Making Under Deep Uncertainty

Deep uncertainty relates to circumstances in which data are too sparse, experts in too much disagreement or time is too short to model the uncertainty. As such, decision-making under deep uncertainty (DMDU) methods are focused on working in the Complex Space. Approaches here emphasize robustness (“no regrets” options) and the use of scenarios, and often link well with scenario-focused robust Bayesian studies. Indeed, DMDU studies draw in many other approaches to decision analysis, using them to identify robust rather than optimal strategies. DMDU analyses can help decision makers to think contingently and build a more wide-ranging recognition of the risks (Walker et al., 2013; Maier et al., 2016; Marchau et al., 2019; French, 2020; Workman et al., 2021) (See Table 3b for further details).

Decision Process and Risk Management Tools

The process of decision-making can be very complex, extending over time and involving many parties. A range of tools and techniques have grown up to help manage the decision-making process and support risk management and the implementation of the chosen strategy. Some tools organize data and analyses, often being built on a geographic information system. Others manage processes, organizing workflows. Some have inevitably expanded in function to support decision-making itself, even though their primary focus might be on, say, implementation and monitoring risks. They apply in all the Cynefin Spaces. Such tools are closely related to knowledge management systems; knowledge management processes and decision process management differ more in terminology than in substance (Dalkir, 2005; French et al., 2009; Jashapara, 2011) (See Table 3c for further details).

Economic and Financial Approaches

Many of the tools involved in analyzing decisions stem from economic theory and accounting practices: e.g., cost benefit analysis, which seeks to price out all aspects of the consequence of a strategy, or real options theory, which seeks to value financial investments allowing for their risks and the contingent buying and selling. Such methods are perceived as objective when dealing with tangibles, but are more controversial in their valuing of intangibles. Since these methods model uncertainties with probabilities and then work with expectations, they share much in common with Bayesian methods. However, many applications of cost-benefit analysis omit any detailed treatment of uncertainty. Because of the detailed data requirements of these methods, their application is limited to the Known and Knowable Spaces, though there have been some investigations of using real options in the face of deep uncertainty (Neely and de Neufville, 2001; Bedford et al., 2005; Pearce et al., 2006; Hallegatte et al., 2013; Buurman and Babovic, 2016; Boardman et al., 2017) (See Table 3d for further details).

Interval Methods

Because of concerns that the statistical accuracy of some data is unknown and that decision-makers and experts cannot make numerical judgements accurately, analyses have been suggested which accept ranges for numerical inputs. While avoiding accuracy issues, weakening the arithmetic also may weaken other foundational assumptions, including some basic principles of rationality. Different types of uncertainty can often be confused, and the analyses can contradict basic probability theory. Interval models of semantic, and imprecision can be useful in exploring ambiguity and value uncertainty, though modeling rather than resolving such uncertainties does not necessary help in decision-making. Some interval methods can be thought of more as sensitivity techniques applied to other decision analytic approaches. Typical approaches here relate to the fuzzy or possibility theory, and evidential reasoning. Interval methods can be applied in the Known, Knowable and Complex Spaces (Shafer, 1976; French, 1984, 1995a; Pedrycz et al., 2011; Xu, 2012) (See Table 3e for further details).

Multi-Criteria Decision Analysis

A term covering many approaches: indeed, Bayesian, DMDU and interval methods are sometimes considered multi-criteria decision analyses (MCDA). Some MCDA seek an optimal or best strategy; others form partial rankings, eliminating weak strategies but not discriminating fully between the better ones. Many MCDA methods eschew dealing with uncertainties and focus on modeling and exploring conflicting objectives and balancing these. Some methods have a rather pragmatic basis, although the European School of Multi-Criteria Decision Aid have much firmer philosophical foundations. There are MCDA methods that are appropriate to each of the Known, Knowable and Complex Spaces, though any method may be limited to just one of these spaces. MCDA techniques are especially useful in working with senior decision-makers in setting policy and broad objectives, and in processes of stakeholder engagement (Roy, 1996; Roy and Vanderpooten, 1996; Belton and Stewart, 2002; Bouyssou et al., 2006; Zopounidis and Pardalos, 2010; Velasquez and Hester, 2013; Korhonen and Wallenius, 2020) (See Tables 3f,g for further details).

Soft Elicitation

Soft elicitation, also known as problem structuring, is the process of asking problem owners, experts and stakeholders for the knowledge, perceptions, beliefs, uncertainties and values that a model needs to embody before being populated with numbers. Methods here help in problem formulation, structuring understanding: e.g., cognitive maps, soft OR, soft systems, prompts such as PESTLE and other qualitative tools. The output of soft elicitation can lead to the building of sophisticated quantitative models; and can also structure communications and deliberations with stakeholders. Exploratory data analysis and visual analytics are also relevant. Soft elicitation is, rather obviously, focused on the sense-making and modeling phase of decision making, but it also has enormous advantages in setting the frame for communication between all parties and thus applies in all three phases. Also there are many cases in which the clarity brought by framing the issues well has obviated the need for formal quantitative analysis. These techniques are useful in all of the Cynefin Spaces, though they come to the fore in the Complex and Chaotic Spaces In which sense really needs to be made (Rosenhead and Mingers, 2001; Checkland, 2013; Steed et al., 2013; Bendoly and Clark, 2016; Pyrko et al., 2019; French, 2021) (See Table 3h for further details).

Identifying Decision-Making Tools Appropriate to a Problem

No “one-size-fits-all” tool is available for managing every climate risk or, indeed, managing the same risk but in different contexts, urgencies or availabilities of resources. This section aims to provide a means by which a climate risk manager may appraise the value of different analytic techniques for their situation. We encourage a prospective user of these techniques to consider the nature of the control system they are dealing with, such as described in the box, the Cynefin context in which they find themselves, and the types of uncertainties most conspicuous in their case. Table 3 can then be used to assess the appropriateness, or not, of different groups of techniques described above. The Table lists the various forms of decision analysis, indicating how they manage uncertainties, how they may be used in the different Cynefin contexts, how they fit into the different phases of decision-making and the resources needed in each use. In order to dig deeper into whether an approach may be suitable, citations are given to relevant literature to support our comments. In addition, we cite some relevant case studies in the application of the tools to climate-related risk management. We make no claims of exhaustiveness, and recognize that in identifying these characteristics we are making many subjective choices, but we hope that they offer a constructive guide into the literature that may help problem-owners and analysts find tools potentially valuable in their context.

While once-intractable, Bayesian Methods have made huge strides becoming computationally tractable and transparent to non-specialist users since the last century (Edwards et al., 2007). Moreover, developments in elicitation can be used to address behavioral and cognitive issues that can bias judgemental inputs (Dias et al., 2018; Turkman et al., 2019; Hanea et al., 2020). Many of the other methods evolved before these advances. Thus, Bayesian ideas should not be dismissed on those grounds; the main issue in using them is that they are explicitly subjective, emphasizing transparency, consensus, impartiality, and correspondence to observable reality instead of objectivity (Gelman and Hennig, 2017). Different cultures recognize and value subjectivity and objectivity differently. Some demand that subjective judgements are recognized explicitly, while others only acknowledge objective issues explicitly.

Decisions are based on analyses of the knowledge and information at hand to the decision-maker. The context of the decision process described above influences what can be done in each phase of decision making. Figure 6 illustrates how knowledge and uncertainty of the different subsystems in the example control system for managing the risk of consequences of flooding (Figure 3) influence the Cynefin space that the management problem may fall within, as well as indicating the analytic techniques that may be available.

Figure 6

Decision analyses used to support decision-making on climate-related risks shown as case studies in Table 3 were assessed for the circumstances in which they were used. The first dimension of the assessment was the geo-political scale to which the decision was intended to apply—household (or individual), community (village or neighborhood), city (including the greater city jurisdiction), sub-national region (a state, province), nation, trans-national regions (within a continent), international (through global agreements, organizations and the like). This scale differs from the type of body making the decisions, which is reflective of whether the outcome is intended as top-down, autonomous, or bottom-up. Here, a top-down decision is one that applies from a body that is autonomous at higher geopolitical scale to lower scales, whereas a bottom up decision is one made by a body autonomous at lower scales intended to influencing high scale outcomes.

The second dimension relates to the contribution of the technique to decision outcomes. These contributions relate to phases in the decision process but, as described previously, may not be implemented in a set sequence. The types of contributions include:

The results of this assessment are shown in Figure 7. Evidence of the basis of actual decisions and whether decision-analytic techniques were used to support the making of those decisions is difficult to find in the peer-reviewed literature. Most of the case studies were related to theoretical studies with realistic data, reflecting that most literature on climate change is about scenarios and the consequences of those scenarios. Many fewer studies address the actual decision processes of managing climate-related risks. Moreover, the spectrum of different types of contribution to the decision process seem more focussed at subnational/national levels.

Figure 7

Concluding Remarks

Climate change brings many profound challenges and with them a need to manage a gamut of risks, ranging in scale from very local to global and severity from a relatively simple need to adapt to existential. Addressing these will involve many people, many decision-makers, stakeholders and experts. Some situations may have time and resources for acquiring data, opinions and to test options; others need urgent actions. In consequence, there are many decisions to be made and a great need for modeling and analysis to support these decisions.

In this paper, we have sought to guide policy makers, their advisors and the broader climate change community (scientists, NGOs, advocacy groups) into the literature on decision analysis and the range of tools available to support decision-making. We have sought to emphasize the complexity of decision-making, particularly in the context of time-constrained risk management. We have presented existing approaches and decision analytic tools in a way that we believe will help policy makers find methods that are appropriate to their circumstances. We hope that our paper stimulates their recognition of the complexities involved in the decision-making and at the same time offers constructive suggestions to help develop appropriate decision analyses.

Publisher's Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

• 1

  ÅströmH.Friis HansenP.GarrèL.Arnbjerg-NielsenK. (2014). An influence diagram for urban flood risk assessment through pluvial flood hazards under non-stationary conditions. J. Water Clim. Change.5, 276–286. 10.2166/wcc.2014.103

  • CrossRef
  • Google Scholar

• 2

  AckermannF. (2012). Problem structuring methods ‘in the Dock': arguing the case for Soft OR. Eur. J. Operat. Res.219, 652–658. 10.1016/j.ejor.2011.11.014

  • CrossRef
  • Google Scholar

• 3

  AlexeeffS. E.PfisterG. G.NychkaD. (2016). A Bayesian model for quantifying the change in mortality associated with future ozone exposures under climate change. Biometrics.72, 281–288. 10.1111/biom.12383

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 4

  AtkinsonG.BraathenN. A.GroomB.MouratoS. (2018). Cost-Benefit Analysis and the Environment: Further Developments and Policy Use.Paris, France: OECD Publishing.

  • Google Scholar

• 5

  BakerE.SolakS. (2011). Climate change and optimal energy technology RandD policy. Eur. J. Operat. Res.213, 442–454. 10.1016/j.ejor.2011.03.046

  • CrossRef
  • Google Scholar

• 6

  BarnettV. (1999). Comparative Statistical Inference. Chichester: John Wiley and Sons. 10.1002/9780470316955

  • CrossRef
  • Google Scholar

• 7

  BatishaA. F. (2015). Implementing fuzzy decision making technique in analyzing the Nile Delta resilience to climate change. Alexandria Eng. J.54, 1043–1056. 10.1016/j.aej.2015.05.019

  • CrossRef
  • Google Scholar

• 8

  BayleyC. (2008). Public Participation, in MelnickE.L.EverittB.S. eds. Encyclopedia of Quantitative Risk Analysis and Assessment, (Chichester: John Wiley and Sons) p. 1383–1391. 10.1002/9780470061596.risk0535

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 9

  BedfordT.FrenchS.AthertonE. (2005). Supporting ALARP decision-making by cost benefit analysis and multi-attribute utility theory. J. Risk Res.8, 207–223. 10.1080/1366987042000192408

  • CrossRef
  • Google Scholar

• 10

  BehzadianM.KazemzadehR. B.AlbadviA.AghdasiM. (2010). PROMETHEE: a comprehensive literature review on methodologies and applications. Eur. J. Oper. Res.200, 198–215. 10.1016/j.ejor.2009.01.021

  • CrossRef
  • Google Scholar

• 11

  BeierleT.CayfordJ. (2002). “Democracy in Practice: Public Participation in Environmental Decisions”. Washington, DC: Routledge.

  • Pubmed Abstract
  • Google Scholar

• 12

  BellM. L.HobbsB. F.ElliottE. M.EllisH.RobinsonZ. (2001). An evaluation of multi-criteria methods in integrated assessment of climate policy. J. Multi-Criteria Decis. Anal.10, 229–256. 10.1002/mcda.305

  • CrossRef
  • Google Scholar

• 13

  BeltonV.StewartT. J. (2002). “Multiple Criteria Decision Analysis: an Integrated Approach”, Boston, MA: Kluwer Academic Press. 10.1007/978-1-4615-1495-4

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 14

  BendolyE.ClarkS. (2016). Visual Analytics for Management: Translational Science and Applications in Practice,”New York, NY: Taylor and Francis. 10.4324/9781315640891

  • CrossRef
  • Google Scholar

• 15

  BerkeleyD.HumphreysP. C. (1982). Structuring decision problems and the 'bias heuristic'. Psychol. Bull.50, 201–252. 10.1016/0001-6918(82)90042-7

  • CrossRef
  • Google Scholar

• 16

  BiehlL. L.ZhaoL.SongC. X.PanzaC. G. (2017). Cyberinfrastructure for the collaborative development of U2U decision support tools. Climate Risk Management15, 90–108. 10.1016/j.crm.2016.10.003

  • CrossRef
  • Google Scholar

• 17

  BindoffN. L.StottP. A.AchutaRaoK. M.AllenM. R.GillettN.GutzlerD.et al. (2013). Detection and Attribution of Climate Change: from Global to Regional, in Climate Change 2013: The Physical Science Basis. Contribution of Working Group I to the Fifth Assessment Report of the Intergovernmental Panel on Climate Change, eds. StockerT. F.QinD.PlattnerG. K.TignorM.AllenS. K.BoschungJ.et al. (Cambridge, New York, NY: Cambridge University Press) p. 867–952.

  • Google Scholar

• 18

  BoardmanA. E.GreenbergD. H.ViningA. R.WeimerD. L. (2017). Cost-Benefit Analysis: Concepts and Practice.Cambridge: Cambridge University Press. 10.1017/9781108235594

  • CrossRef
  • Google Scholar

• 19

  BonczekR. H.HolsappleC. W.WhinstonA. B. (2014). Foundations of Decision Support Systems.New York, NY: Academic Press.

  • Google Scholar

• 20

  BosomworthK.LeithP.HarwoodA.WallisP. J. (2017). What's the problem in adaptation pathways planning? The potential of a diagnostic problem-structuring approach. Environ. Sci. Policy.76, 23–28. 10.1016/j.envsci.2017.06.007

  • CrossRef
  • Google Scholar

• 21

  BouyssouD.Jacquet-LagrèzeE.PernyP.SlowińskiR.VanderpootenD.VinckeP. (2002). Aiding Decisions With Multiple Criteria: Essays in Honor of Bernard Roy. New York, NY: Springer.

  • Google Scholar

• 22

  BouyssouD.MarchantT.PirlotM.TsoukiasA.VinckeP. (2006). Evaluation and Decision Models with Multiple Criteria: Stepping Stones for the Analyst.New York, NY: Springer.

  • Google Scholar

• 23

  BrownC.SteinschneiderS.RayP.WiS.BasdekasL.YatesD. (2019). Decision Scaling (DS): decision support for climate change, in Decision Making under Deep Uncertainty (Cham: Springer), 255–287. 10.1007/978-3-030-05252-2_12

  • CrossRef
  • Google Scholar

• 24

  BursteinF.HolsappleC.W. (2008). Handbook on Decision Support Systems (in two volumes).Berlin: Springer Verlag. 10.1007/978-3-540-48713-5

  • CrossRef
  • Google Scholar

• 25

  ButlerJ.BohenskyE.SuadnyaW.YanuartatiY.HandayaniT.HabibiP.et al. (2016). Scenario planning to leap-frog the sustainable development goals: an adaptation pathways approach. Clim. Risk Manage.12, 83–99. 10.1016/j.crm.2015.11.003

  • CrossRef
  • Google Scholar

• 26

  BuurmanJ.BabovicV. (2016). Adaptation Pathways and Real Options Analysis: An approach to deep uncertainty in climate change adaptation policies. Policy Soc.35, 137–150. 10.1016/j.polsoc.2016.05.002

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 27

  CatenacciM.GiupponiC. (2013). Integrated assessment of sea-level rise adaptation strategies using a Bayesian decision network approach. Environ. Model. Softw.44, 87–100. 10.1016/j.envsoft.2012.10.010

  • CrossRef
  • Google Scholar

• 28

  ChecklandP. (2013). Soft systems methodology, in Encyclopedia of Operations Research and Management Science (Boston, MA: Springer), 1430–1436. 10.1007/978-1-4419-1153-7_971

  • CrossRef
  • Google Scholar

• 29

  ChristensenR.JohnsonW.BranscumA.HansonT. (2011). Bayesian Ideas and Data Analysis: An Introduction for Scientists and Statisticians. Boca Raton: CRC press. 10.1201/9781439894798

  • CrossRef
  • Google Scholar

• 30

  CoakesE.WillisD.ClarkeS. (2002). Knowledge Management in the SocioTechnical World.London: Springer Verlag. 10.1007/978-1-4471-0187-1

  • CrossRef
  • Google Scholar

• 31

  CourtneyH. (2003). Decision-driven scenarios for assessing four levels of uncertainty. Strat. Leaders.31, 14–22. 10.1108/10878570310455015

  • CrossRef
  • Google Scholar

• 32

  DalkirK. (2005). Knowledge Management in Theory and Practice.Burlington, MA: Elsevier Butterworth-Heinemann.

  • Google Scholar

• 33

  de BruinK.DellinkR.RuijsA.BolwidtL.van BuurenA.GravelandJ.et al. (2009). Adapting to climate change in The Netherlands: an inventory of climate adaptation options and ranking of alternatives. Clim. Change.95, 23–45. 10.1007/s10584-009-9576-4

  • CrossRef
  • Google Scholar

• 34

  de RuigL. T.BarnardP. L.BotzenW. W.GrifmanP.HartJ. F.de MoelH.et al. (2019). An economic evaluation of adaptation pathways in coastal mega cities: an illustration for Los Angeles. Sci. Total Environ.678, 647–659. 10.1016/j.scitotenv.2019.04.308

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 35

  De SmetY.LidouhK. (2013). An introduction to multicriteria decision aid: The PROMETHEE and GAIA methods, in European Business Intelligence Summer School, eds AufaureM. A.ZimányiE. (Berlin: Springer), 150–176. 10.1007/978-3-642-36318-4_7

  • CrossRef
  • Google Scholar

• 36

  DiasL.MortonA.QuigleyJ. (2018). Elicitation: The Science and Art of Structuring Judgement. Cham: Springer.

  • Google Scholar

• 37

  DouglasM. (1992). Risk and Blame: Essays in Cultural Theory.London: Routledge.

  • Google Scholar

• 38

  DurbachI. N. (2014). Outranking under uncertainty using scenarios. Eur. J. Oper. Res.232, 98–108. 10.1016/j.ejor.2013.06.041

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 39

  DurbachI. N.StewartT. J. (2020). Probability and beyond: including uncertainties in decision analysis, in Behavioral Operational Research, eds WhiteL.KuncM.BurgerK.MalpassJ. (Cham: Springer), 75–91. 10.1007/978-3-030-25405-6_5

  • CrossRef
  • Google Scholar

• 40

  EdwardsW.MilesR. F.Von WinterfeldtD. (2007). Advances in Decision Analysis: from Foundations to Applications. Cambridge: Cambridge University Press. 10.1017/CBO9780511611308

  • CrossRef
  • Google Scholar

• 41

  El-ZeinA.TonmoyF. N. (2015). Assessment of vulnerability to climate change using a multi-criteria outranking approach with application to heat stress in Sydney. Ecol. Indicat.48, 207–217. 10.1016/j.ecolind.2014.08.012

  • CrossRef
  • Google Scholar

• 42

  EvansJ. S. B.StanovichK. E. (2013). Dual-process theories of higher cognition: advancing the debate. J. Perspect. Psychol. Sci.8, 223–241. 10.1177/1745691612460685

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 43

  FeketeJ.-D.PrimetR. (2016). Progressive analytics: a computation paradigm for exploratory data analysis. arXiv[Preprint]. arXiv:1607.05162. 10.48550/arXiv.1607.05162

  • CrossRef
  • Google Scholar

• 44

  FigueiraJ. R.MousseauV.RoyB. (2016). ELECTRE methods, in Multiple Criteria Decision Analysis, eds GrecoS.EhrgottM.FigueiraJ. (New York, NY: Springer), 155–185. 10.1007/978-1-4939-3094-4_5

  • CrossRef
  • Google Scholar

• 45

  FischhoffB. (2015). The realities of risk-cost-benefit analysis. Science350, aaa6516. 10.1126/science.aaa6516

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 46

  FrancoL. A.MontibellerG. (2010). Facilitated modelling in operational research. Eur. J. Oper. Res.205, 489–500. 10.1016/j.ejor.2009.09.030

  • CrossRef
  • Google Scholar

• 47

  FrenchS. (1984). Fuzzy decision analysis: some criticisms, in Fuzzy Sets and Decision Analysis, eds ZimmermannH. J.ZadehL. A.GainesB. R. (Amsterdam: North Holland).

  • Google Scholar

• 48

  FrenchS. (1995a). An introduction to decision theory and prescriptive decision analysis. IMA J. Mathem. Appl. Bus. Ind.6, 239–247. 10.1093/imaman/6.2.239

  • CrossRef
  • Google Scholar

• 49

  FrenchS. (1995b). Uncertainty and imprecision: modelling and analysis. J. Operat. Res. Soc.46, 70–79. 10.1057/jors.1995.8

  • CrossRef
  • Google Scholar

• 50

  FrenchS. (2003). Modelling, making inferences and making decisions: the roles of sensitivity analysis. TOP.11, 229–252. 10.1007/BF02579043

  • CrossRef
  • Google Scholar

• 51

  FrenchS. (2013). Cynefin, Statistics and Decision Analysis. J. Operat. Res. Soc.64, 547–561. 10.1057/jors.2012.23

  • CrossRef
  • Google Scholar

• 52

  FrenchS. (2020). Axiomatising the bayesian paradigm in parallel small worlds. Operat. Res.10.1287/opre.2019.1896. [Epub ahead of print].

  • CrossRef
  • Google Scholar

• 53

  FrenchS. (2021). From soft to hard elicitation. J. Operat. Res. Soc.1–17. 10.1080/01605682.2021.1907244. [Epub ahead of print].

  • CrossRef
  • Google Scholar

• 54

  FrenchS.ArgyrisN. (2018). Decision analysis and political processes. Decis. Analy.15, 208–222. 10.1287/deca.2018.0374

  • CrossRef
  • Google Scholar

• 55

  FrenchS.GeldermannJ. (2005). The varied contexts of environmental decision problems and their implications for decision support. Environ. Sci. Policy.8, 378–391. 10.1016/j.envsci.2005.04.008

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 56

  FrenchS.HaywoodS.OughtonD. H.TurcanuC. (2020). Different types of uncertainty in nuclear emergency management. Radioprotection55, S175–S180. 10.1051/radiopro/2020029

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 57

  FrenchS.MauleA. J.MythenG. (2005). Soft Modelling in Risk Communication and Management: Examples in Handling Food Risk. J. Operat. Res. Soc.56, 879–888. 10.1057/palgrave.jors.2601901

  • CrossRef
  • Google Scholar

• 58

  FrenchS.MauleA. J.PapamichailK. N. (2009). Decision Behaviour, Analysis and Support.Cambridge: Cambridge University Press. 10.1017/CBO9780511609947

  • CrossRef
  • Google Scholar

• 59

  FrenchS.Rios InsuaD. (2000). Statistical Decision Theory.London: Arnold.

  • Google Scholar

• 60

  GelmanA. (2003). A Bayesian formulation of exploratory data analysis and goodness-of-fit testing. Int. Statist. Rev.71, 369–382. 10.1111/j.1751-5823.2003.tb00203.x

  • CrossRef
  • Google Scholar

• 61

  GelmanA.CarlinJ. B.SternH. S.DunsonD. B.VehtariA.RubinD. B. (2013). Bayesian Data Analysis.London: Chapman and Hall. 10.1201/b16018

  • CrossRef
  • Google Scholar

• 62

  GelmanA.HennigC. (2017). Beyond subjective and objective in statistics. J. R. Statis. Soc A180, 967–1033. 10.1111/rssa.12276

  • CrossRef
  • Google Scholar

• 63

  GervásioH.Da SilvaL. S. (2012). A probabilistic decision-making approach for the sustainable assessment of infrastructures. Expert Syst. Applic.39, 7121–7131. 10.1016/j.eswa.2012.01.032

  • CrossRef
  • Google Scholar

• 64

  GilbuenaR.KawamuraA.MedinaR.NakagawaN.AmaguchiH. (2013). Environmental impact assessment using a utility-based recursive evidential reasoning approach for structural flood mitigation measures in Metro Manila, Philippines. J. Environ. Manage.131, 92–102. 10.1016/j.jenvman.2013.09.020

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 65

  GoodwinP.WrightG. (2014). Decision Analysis for Management Judgement.Chichester: John Wiley and Sons.

  • Google Scholar

• 66

  GovindanK.JepsenM. B. (2016). ELECTRE: A comprehensive literature review on methodologies and applications. Eur. J. Oper. Res.250, 1–29. 10.1016/j.ejor.2015.07.019

  • CrossRef
  • Google Scholar

• 67

  GregoryR. S.FailingL.HarstoneM.LongG.McDanielsT.OhlsonD. (2012). Structured Decision Making: A Practical Guide to Environmental Management Choices.Chichester: Wiley-Blackwell. 10.1002/9781444398557

  • CrossRef
  • Google Scholar

• 68

  GrovesD. G.Molina-PerezE.BloomE.FischbachJ. R. (2019). Robust Decision Making (RDM): Application to Water Planning and Climate Policy, in Decision Making under Deep Uncertainty, eds MarchauV.WalkerW.BloemenP.PopperS. (Cham: Springer), 135–163. 10.1007/978-3-030-05252-2_7

  • CrossRef
  • Google Scholar

• 69

  GunawanS.AzarmS. (2005). Multi-objective robust optimization using a sensitivity region concept. Struct. Multidiscipl. Optimiz.29, 50–60. 10.1007/s00158-004-0450-8

  • CrossRef
  • Google Scholar

• 70

  GutjahrW. J.PichlerA. (2016). Stochastic multi-objective optimization: a survey on non-scalarizing methods. Ann. Operat. Res.236, 475–499. 10.1007/s10479-013-1369-5

  • CrossRef
  • Google Scholar

• 71

  HallJ. W.LempertR. J.KellerK.HackbarthA.MijereC.McInerneyD. J. (2012). Robust climate policies under uncertainty: a comparison of robust decision making and info-gap methods. Risk Analy. Int. J.32, 1657–1672. 10.1111/j.1539-6924.2012.01802.x

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 72

  HallegatteS.ShahA.LempertR.BrownC.GillS. (2013). Investment decision making under deep uncertainty-application to climate change. Policy Research Working Papers. The World Bank, P. 41. 10.1596/1813-9450-6193

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 73

  HaneaA.NaneG. F.BedfordT.FrenchS. (2020). Expert Judgement in Risk and Decision Analysis.Switzerland: Springer. 10.1007/978-3-030-46474-5

  • CrossRef
  • Google Scholar

• 74

  HaqueA. N. (2016). Application of multi-criteria analysis on climate adaptation assessment in the context of least developed countries. J. Multi-Criteria Decis. Analy.23, 210–224. 10.1002/mcda.1571

  • CrossRef
  • Google Scholar

• 75

  HennigP.OsborneM. A.GirolamiM. (2015). Probabilistic numerics and uncertainty in computations. Proc. R. Soc. A: R. Soc.471, 20150142. 10.1098/rspa.2015.0142

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 76

  HodgkinsonG.StarbuckW. (eds.). (2008). The Oxford Handbook of Organizational Decision Making. Oxford: Oxford University Press. 10.1093/oxfordhb/9780199290468.001.0001

  • CrossRef
  • Google Scholar

• 77

  HofstedeG. (1984). Cultural Consequences.Beverley Hills, CA: Sage.

  • Google Scholar

• 78

  HolsappleC.SenaM.WagnerW. (2019). The perceived success of ERP systems for decision support. Inf. Technol. Manage.20, 1–7. 10.1007/s10799-017-0285-9

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 79

  HowardR. A.AbbasA. E. (2016). Foundations of Decision Analysis.Harlow: Pearson Ed.

  • Google Scholar

• 80

  HowellS.StarkA.NewtonD.PaxsonD.CarvusM.PereiraJ. (2001). Real Options: Evaluating Corporate Investment Opportunities in a Dynamic World.Harlow: FT Prentice Hall.

  • Google Scholar

• 81

  HydeK.MaierH. R.ColbyC. (2003). Incorporating uncertainty in the PROMETHEE MCDA method. J. Multi-Criter. Decis. Analy.12, 245–259. 10.1002/mcda.361

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 82

  IoossB.SaltelliA. (2017). Introduction to sensitivity analysis, in Handbook of Uncertainty Quantification. (Cham: Springer). 10.1007/978-3-319-12385-1_31

  • CrossRef
  • Google Scholar

• 83

  IPCC (2018). Global Warming of 1.5°C. An IPCC Special Report on the impacts of global warming of 1.5°C above pre-industrial levels and related global greenhouse gas emission pathways, in the context of strengthening the global response to the threat of climate change,sustainable development, and efforts to eradicate poverty.

  • Google Scholar

• 84

  JägerW.ChristieE.HaneaA.den HeijerC.SpencerT. (2018). A Bayesian network approach for coastal risk analysis and decision making. Coastal Eng.134, 48–61. 10.1016/j.coastaleng.2017.05.004

  • CrossRef
  • Google Scholar

• 85

  JashaparaA. (2011). Knowledge Management: An Ingetrated Approach.Harlow, UK: FT Prentice Hall.

  • Google Scholar

• 86

  JeffreysH. (1961). Theory of Probability.Oxford: Oxford University Press.

  • Google Scholar

• 87

  KahnemanD. (2011). Thinking, Fast and Slow.London: Penguin, Allen Lane.

  • Google Scholar

• 88

  KahnemanD.TverskyA. (1974). Judgement under uncertainty: heuristics and biases. Science185, 1124–1131. 10.1126/science.185.4157.1124

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 89

  KeeneyR. L. (1992). Value-Focused Thinking: a Path to Creative Decision Making. Cambridge: Harvard University Press.

  • Google Scholar

• 90

  KeeneyR. L.RaiffaH. (1993). Decisions With Multiple Objectives: Preferences and Value Trade-offs. Cambridge: Cambridge University Press. 10.1017/CBO9781139174084

  • CrossRef
  • Google Scholar

• 91

  KimY.ChungE.-S. (2013). Fuzzy VIKOR approach for assessing the vulnerability of the water supply to climate change and variability in South Korea. Appl. Mathem Modell.37, 9419–9430. 10.1016/j.apm.2013.04.040

  • CrossRef
  • Google Scholar

• 92

  KlamrothK.KnowlesJ. D.RudolphG.WiecekM. M. (2018). Personalized Multiobjective Optimization: An Analytics Perspective (Dagstuhl Seminar 18031). Schloss Dagstuhl, Wadern, Germany.

  • Google Scholar

• 93

  KnightF. H. (1921). Risk, Uncertainty and Profit.Boston, MA: Hart, Schaffner and Marx; Houghton Mifflin Company.

  • Google Scholar

• 94

  KonidariP.MavrakisD. (2007). A multi-criteria evaluation method for climate change mitigation policy instruments. Energy Policy35, 6235–6257. 10.1016/j.enpol.2007.07.007

  • CrossRef
  • Google Scholar

• 95

  KorhonenP. J.WalleniusJ. (2020). Making Better Decisions. Cham: Springer. 10.1007/978-3-030-49459-9

  • CrossRef
  • Google Scholar

• 96

  KumarA.SahB.SinghA. R.DengY.HeX.KumarP.et al. (2017). A review of multi criteria decision making (MCDM) towards sustainable renewable energy development. Renew. Sustain. Energy Rev.69, 596–609. 10.1016/j.rser.2016.11.191

  • CrossRef
  • Google Scholar

• 97

  LempertR. J.GrovesD. G. (2010). Identifying and evaluating robust adaptive policy responses to climate change for water management agencies in the American west. Technol. Forec. Soc. Change77, 960–974. 10.1016/j.techfore.2010.04.007

  • CrossRef
  • Google Scholar

• 98

  LichtensteinS.SlovicP. (2006). The Construction of Preference.Cambridge: Cambridge University Press. 10.1017/CBO9780511618031

  • CrossRef
  • Google Scholar

• 99

  MaierH. R.GuillaumeJ. H. A.van DeldenH.RiddellG. A.HaasnootM.KwakkelJ. H. (2016). An uncertain future, deep uncertainty, scenarios, robustness and adaptation: how do they fit together?Environ. Model. Softw.81, 154–164. 10.1016/j.envsoft.2016.03.014

  • CrossRef
  • Google Scholar

• 100

  ManochaN.BabovicV. (2017). Development and valuation of adaptation pathways for storm water management infrastructure. Environ. Sci. Policy.77, 86–97. 10.1016/j.envsci.2017.08.001

  • CrossRef
  • Google Scholar

• 101

  MarchauV.WalkerW.E.BloemenP.PopperS. (2019). Decision Making under Deep Uncertainty.Springer. 10.1007/978-3-030-05252-2

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 102

  Markl-HummelL.GeldermannJ. (2014). A local-level, multiple criteria decision aid for climate protection. EURO J. Decis. Proces.2, 121–152. 10.1007/s40070-013-0011-8

  • CrossRef
  • Google Scholar

• 103

  MarttunenM.LienertJ.BeltonV. (2017). Structuring problems for multi-criteria decision analysis in practice: a literature review of method combinations. Eur. J. Oper. Res.263, 1–17. 10.1016/j.ejor.2017.04.041

  • CrossRef
  • Google Scholar

• 104

  MassinghamP. (2010). Knowledge risk management: a framework. J. Knowl. Manage.14, 464–485. 10.1108/13673271011050166

  • CrossRef
  • Google Scholar

• 105

  Melbourne-ThomasJ.ConstableA.WotherspoonS.RaymondB. (2013). Testing paradigms of ecosystem change under climate warming in Antarctica. PLoS ONE.8, e55093. 10.1371/journal.pone.0055093

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 106

  MichailidouA. V.VlachokostasC.MoussiopoulosN. (2016). Interactions between climate change and the tourism sector: Multiple-criteria decision analysis to assess mitigation and adaptation options in tourism areas. Tour. Manage.55, 1–12. 10.1016/j.tourman.2016.01.010

  • CrossRef
  • Google Scholar

• 107

  MontibellerG.WinterfeldtD. (2015). Cognitive and motivational biases in decision and risk analysis. Risk Analy. 35, 1230–51. 10.1111/risa.12360

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 108

  MorganB. J. (2008). Applied Stochastic Modelling, 2nd Edn. Boca Raton, FL: CRC Press. 10.1201/b17188

  • CrossRef
  • Google Scholar

• 109

  MortonA.AiroldiM.PhillipsL. D. (2009). Nuclear risk management on stage: a decision analysis perspective on the UK's Committee on Radioactive Waste Management. Risk Analy.29, 764–779. 10.1111/j.1539-6924.2008.01192.x

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 110

  MortonA.FasoloB. (2009). Behavioural decision theory for multi-criteria decision analysis: a guided tour 60(2): 268-275. J. Operat. Res. Soc.60, 268–275. 10.1057/palgrave.jors.2602550

  • CrossRef
  • Google Scholar

• 111

  MustajokiJ.HämäläinenR. P.MarttunenM. (2004). Participatory multi-criteria decision analysis with Web-Hipre: a case of lake regulation policy. Environ. Model. Softw.19, 537–547. 10.1016/j.envsoft.2003.07.002

  • CrossRef
  • Google Scholar

• 112

  NeelyJ. E.de NeufvilleR. (2001). Hybrid real options valuation of risky product development projects. Int. J. Technol Policy Manage.1, 29–46. 10.1504/IJTPM.2001.001743

  • CrossRef
  • Google Scholar

• 113

  NunamakerJ. F.BriggsR. O.RomanoN. C. R.Jr (2014). Collaboration Systems: Concept, Value, and Use.New York: Routledge. 10.4324/9781315705569

  • CrossRef
  • Google Scholar

• 114

  OECD. (2018). Cost-Benefit Analysis and the Environment. oecd.org: Organisation for Economic Co-operation and Development. 10.1787/9789264085169-en

  • CrossRef
  • Google Scholar

• 115

  OrloveB.ShwomR.MarkowitzE.CheongS.-M. (2020). Climate decision-making. Ann. Rev. Environ. Resour.45, 271–303. 10.1146/annurev-environ-012320-085130

  • CrossRef
  • Google Scholar

• 116

  PapathanasiouJ.PloskasN.LindenI. (2016). Real-World Decision Support Systems: Case Studies.Springer. 10.1007/978-3-319-43916-7

  • CrossRef
  • Google Scholar

• 117

  PardingK. M.DoblerA.McSweeneyC. F.LandgrenO. A.BenestadR.ErlandsenH. B.et al. (2020). GCMeval–An interactive tool for evaluation and selection of climate model ensembles. Clim. Serv.18, 100167. 10.1016/j.cliser.2020.100167

  • CrossRef
  • Google Scholar

• 118

  ParkS. E.MarshallN. A.JakkuE.DowdA. M.HowdenS. M.MendhamE.et al. (2012). Informing adaptation responses to climate change through theories of transformation. Global Environ. Change.22, 115–126. 10.1016/j.gloenvcha.2011.10.003

  • CrossRef
  • Google Scholar

• 119

  Paté-CornellM. E. (1996). Uncertainties in risk analysis: Six levels of treatment. Reliabil. Eng. Syst Safet.54, 95–111. 10.1016/S0951-8320(96)00067-1

  • CrossRef
  • Google Scholar

• 120

  PatrícioJ.ElliottM.MazikK.PapadopoulouK.-N.SmithC. J. (2016). DPSIR—two decades of trying to develop a unifying framework for marine environmental management?Front. Marine Sci.3, 177. 10.3389/fmars.2016.00177

  • CrossRef
  • Google Scholar

• 121

  PearceD.AtkinsonG.MouratoS. (2006). Cost-benefit Analysis and the Environment: Recent Developments. Organisation for Economic Co-operation and development.

  • Google Scholar

• 122

  PedryczW.EkelP.ParreirasR. (2011). Fuzzy multicriteria decision-making: models, methods and applications. New York, NY: John Wiley and Sons. 10.1002/9780470974032

  • CrossRef
  • Google Scholar

• 123

  PhanT. D.SmartJ. C.Stewart-KosterB.SahinO.HadwenW. L.DinhL. T.et al. (2019). Applications of bayesian networks as decision support tools for water resource management under climate change and socio-economic stressors: a critical appraisal. Water11, 2642. 10.3390/w11122642

  • CrossRef
  • Google Scholar

• 124

  PhillipsL. D. (2007). Decision Conferencing, in Advances in Decision Analysis: From Foundations to Applications, eds von WinterfeldtD.MilesR. F.JrEdwardsW. (Cambridge: Cambridge University Press) p. 375–399. 10.1017/CBO9780511611308.020

  • CrossRef
  • Google Scholar

• 125

  ProberS. M.ColloffM. J.AbelN.CrimpS.DohertyM. D.DunlopM.et al. (2017). Informing climate adaptation pathways in multi-use woodland landscapes using the values-rules-knowledge framework. Agric. Ecosyst. Environ.241, 39–53. 10.1016/j.agee.2017.02.021

  • CrossRef
  • Google Scholar

• 126

  PyrkoI.EdenC.HowickS. (2019). Knowledge acquisition using group support systems. Group Decis. Negot.28, 233–253. 10.1007/s10726-019-09614-9

  • CrossRef
  • Google Scholar

• 127

  RazT.MichaelE. (2001). Use and benefits of tools for project risk management. Int. J. Project Manage.19, 9–17. 10.1016/S0263-7863(99)00036-8

  • CrossRef
  • Google Scholar

• 128

  ReillyT.ClemenR. T. (2013). Making Hard Decisions with Decision Tools.Boston, MA: South Western College Publishing.

  • Google Scholar

• 129

  RennO. (2008). Risk Governance.London: Earthscan. 10.1007/978-1-4020-6799-0

  • CrossRef
  • Google Scholar

• 130

  RichardsR.San,óM.RoikoA.CarterR. W.BusseyM.MatthewsJ.et al. (2013). Bayesian belief modeling of climate change impacts for informing regional adaptation options. Environ. Model. Softw.44, 113–121. 10.1016/j.envsoft.2012.07.008

  • CrossRef
  • Google Scholar

• 131

  RichardsR.SanoM.SahinO. (2016). Exploring climate change adaptive capacity of surf life saving in Australia using Bayesian belief networks. Ocean Coastal Manage.120, 148–159. 10.1016/j.ocecoaman.2015.11.007

  • CrossRef
  • Google Scholar

• 132

  Rios InsuaD. (1990). Sensitivity Analysis in Multi-Objective Decision Making.Berlin: Springer Verlag. 10.1007/978-3-642-51656-6

  • CrossRef
  • Google Scholar

• 133

  Rios InsuaD. (1999). Sensitivity Analysis in MCDA.

  • Google Scholar

• 134

  Rios InsuaD.FrenchS. (2010). Democracy: a Group Decision and Negotiation Perspective.Dordrecht: Springer. 10.1007/978-90-481-9045-4

  • CrossRef
  • Google Scholar

• 135

  Rios InsuaD.RuggeriF. (2000). Robust Bayesian Analysis.New York: Springer-Verlag. 10.1007/978-1-4612-1306-2

  • CrossRef
  • Google Scholar

• 136

  RosenheadJ.MingersJ. (2001). Rational Analysis for a Problematic World Revisited.Chichester: John Wiley and Sons.

  • Google Scholar

• 137

  RoyB. (1996). Multi-Criteria Modelling for Decision Aiding. Dordrecht: Kluwer Academic Publishers. 10.1007/978-1-4757-2500-1

  • CrossRef
  • Google Scholar

• 138

  RoyB.VanderpootenD. (1996). The European School of MCDA: emergence, basic features, and current works. J. Multi-Criteria Decis. Analy.5, 22–36. 10.1002/(SICI)1099-1360(199603)5:1<22::AID-MCDA93>3.0.CO;2-F

  • CrossRef
  • Google Scholar

• 139

  SaarikoskiH.MustajokiJ.BartonD. N.GenelettiD.LangemeyerJ.Gomez-BaggethunE.et al. (2016). Multi-Criteria Decision Analysis and Cost-Benefit Analysis: Comparing alternative frameworks for integrated valuation of ecosystem services. Ecosyst. Serv.22, 238–249. 10.1016/j.ecoser.2016.10.014

  • CrossRef
  • Google Scholar

• 140

  SaatyT. L. (1980). The Analytical Hierarchy Process. New York: McGraw-Hill. 10.21236/ADA214804

  • CrossRef
  • Google Scholar

• 141

  SauterV. L. (2014). Decision Support Systems for Business Intelligence. John Wiley and Sons.

  • Pubmed Abstract
  • Google Scholar

• 142

  ShaferG. (1976). A Mathematical Theory of Evidence.Princeton university press. 10.1515/9780691214696

  • CrossRef
  • Google Scholar

• 143

  ShawD.FrancoA.WestcombeM. (2006). Special issue: problem structuring methods I: new directions in a problematic world. J. Operat. Res. Soc. 57, 757–758. 10.1057/palgrave.jors.2602193

  • CrossRef
  • Google Scholar

• 144

  ShawD.FrancoA.WestcombeM. (2007). Special issue: problem structuring methods II: Taking problem structuring methods forward. J. Operat. Res. Soc. 58, 545–546. 10.1057/palgrave.jors.2602366

  • CrossRef
  • Google Scholar

• 145

  ShleiferA. (2012). Psychologists at the gate: a review of daniel kahneman's “thinking, fast and slow. J. Econ. Literat.50, 1080–1091. 10.1257/jel.50.4.1080

  • CrossRef
  • Google Scholar

• 146

  SimpsonN. P.MachK. J.ConstableA.HessJ.HogarthR.HowdenM.et al. (2021). A framework for complex climate change risk assessment. One Earth.4, 489–501. 10.1016/j.oneear.2021.03.005

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 147

  SmithJ. Q. (2010). Bayesian Decision Analysis: Principles and Practice.Cambridge: Cambridge University Press. 10.1017/CBO9780511779237

  • CrossRef
  • Google Scholar

• 148

  SnowdenD. (2002). Complex acts of knowing - paradox and descriptive self-awareness. J. Knowl. Manage.6, 100–111. 10.1108/13673270210424639

  • CrossRef
  • Google Scholar

• 149

  SperottoA.MolinaJ.TorresanS.CrittoA.Pulido-VelazquezM.MarcominiA. (2019). A Bayesian Networks approach for the assessment of climate change impacts on nutrients loading. Environ. Sci. Policy.100, 21–36. 10.1016/j.envsci.2019.06.004

  • CrossRef
  • Google Scholar

• 150

  SperottoA.MolinaJ.-L.TorresanS.CrittoA.MarcominiA. (2017). Reviewing Bayesian Networks potentials for climate change impacts assessment and management: A multi-risk perspective. J. Environ. Manage.202, 320–331. 10.1016/j.jenvman.2017.07.044

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 151

  SpiegelhalterD. (2019). The Art of Statistics: How to Learn From Data. Basic Books.

  • Google Scholar

• 152

  SteedC. A.RicciutoD. M.ShipmanG.SmithB.ThorntonP. E.WangD.et al. (2013). Big data visual analytics for exploratory earth system simulation analysis. Comput. Geosci.61, 71–82. 10.1016/j.cageo.2013.07.025

  • CrossRef
  • Google Scholar

• 153

  StewartT. J.FrenchS.RiosJ. (2013). Integration of multicriteria decision analysis and scenario planning. Omega.41, 679–688. 10.1016/j.omega.2012.09.003

  • CrossRef
  • Google Scholar

• 154

  StreimikieneD.BalezentisT. (2013). Multi-objective ranking of climate change mitigation policies and measures in Lithuania. Renew. Sustain. Energy Rev.18, 144–153. 10.1016/j.rser.2012.09.040

  • CrossRef
  • Google Scholar

• 155

  SymstadA. J.FisichelliN. A.MillerB. W.RowlandE.SchuurmanG. W. (2017). Multiple methods for multiple futures: Integrating qualitative scenario planning and quantitative simulation modeling for natural resource decision making. Clim. Risk Manage.17, 78–91. 10.1016/j.crm.2017.07.002

  • CrossRef
  • Google Scholar

• 156

  TabariH. (2020). Climate change impact on flood and extreme precipitation increases with water availability. Sci. Rep.10, 13768. 10.1038/s41598-020-70816-2

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 157

  TanerM. Ü.RayP.BrownC. (2017). Robustness-based evaluation of hydropower infrastructure design under climate change. Clim. Risk Manage.18, 34–50. 10.1016/j.crm.2017.08.002

  • CrossRef
  • Google Scholar

• 158

  ThompsonM.EllisR.WildavskyA. (1990). Cultural Theory.Boulder: Westview Print.

  • Google Scholar

• 159

  TurkmanM. A. A.PaulinoC. D.MüllerP. (2019). Computational Bayesian Statistics: An Introduction.Cambridge: Cambridge University Press. 10.1017/9781108646185

  • CrossRef
  • Google Scholar

• 160

  TzengG.-H.HuangJ.-J. (2011). Multiple Attribute Decision Making: Methods and Applications. CRC press. 10.1201/b11032

  • CrossRef
  • Google Scholar

• 161

  VanosJ. K.BaldwinJ. W.JayO.EbiK. L. (2020). Simplicity lacks robustness when projecting heat-health outcomes in a changing climate. Nat. Commun.11, 6079. 10.1038/s41467-020-19994-1

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 162

  VelasquezM.HesterP. T. (2013). An analysis of multi-criteria decision making methods. Int. J. Operat. Res.10, 56–66.

  • Pubmed Abstract
  • Google Scholar

• 163

  WalkerW. E.LempertR. J.KwakkelJ. H. (2013). Deep uncertainty, in Encyclopedia of operations research and management science, eds. GassS.Fu.M.C. (New York: Springer) 395–402. 10.1007/978-1-4419-1153-7_1140

  • CrossRef
  • Google Scholar

• 164

  WeaverC. P.LempertR. J.BrownC.HallJ. A.RevellD.SarewitzD. (2013). Improving the contribution of climate model information to decision making: the value and demands of robust decision frameworks. Wiley Interdiscipl. Rev.4, 39–60. 10.1002/wcc.202

  • CrossRef
  • Google Scholar

• 165

  WorkmanM.DarchG.DooleyK.LomaxG.MaltbyJ.PollittH. (2021). Climate policy decision making in contexts of deep uncertainty - from optimisation to robustness. Environ. Sci Policy.120, 127–137. 10.1016/j.envsci.2021.03.002

  • CrossRef
  • Google Scholar

• 166

  XenariosS.PolatidisH. (2015). Alleviating climate change impacts in rural Bangladesh: a PROMETHEE outranking-based approach for prioritizing agricultural interventions. Environ. Develop. Sustain.17, 963–985. 10.1007/s10668-014-9583-0

  • CrossRef
  • Google Scholar

• 167

  XuD.-L. (2012). An introduction and survey of the evidential reasoning approach for multiple criteria decision analysis. Ann Operat. Res.195, 163–187. 10.1007/s10479-011-0945-9

  • CrossRef
  • Google Scholar

• 168

  YangZ.NgA. K.LeeP. T.-W.WangT.QuZ.RodriguesV. S.et al. (2018). Risk and cost evaluation of port adaptation measures to climate change impacts. Transp. Res. Part D: Transp. Environ.61, 444–458. 10.1016/j.trd.2017.03.004

  • CrossRef
  • Google Scholar

• 169

  ZhangM.-J.WangY.-M.LiL.-H.ChenS.-Q. (2017). A general evidential reasoning algorithm for multi-attribute decision analysis under interval uncertainty. Eur. J. Oper. Res.257, 1005–1015. 10.1016/j.ejor.2016.08.028

  • CrossRef
  • Google Scholar

• 170

  ZopounidisC.PardalosP. M. (2010). Handbook of Multicriteria Analysis.Berlin: Springer. 10.1007/978-3-540-92828-7

  • CrossRef
  • Google Scholar