Decisions can be divided into two basic elements: (1) the decision-making process and (2) the decision outcome (Keuper, 2004). A bounded rationality model of decision-making assumes that decision-makers cannot obtain complete and perfect information; their knowledge of all options, their cognitive abilities, and the time available are limited (Simon, 1956; Orquin and Mueller Loose, 2013). For this reason, decision-makers must break down the decision-making problem and prioritize certain information or apply specific strategies to reach a decision (Kahneman, 2011; Orquin and Mueller Loose, 2013). In the context of visualizations, Padilla et al. (2018) more recently have proposed two types of decision-making with visualizations, one that is fast and easy (type I) and one that is slower and more effortful (type II). Type I decisions occur if the visualization matches common preconceptions (e.g., “dark is more” on maps; Garlandini and Fabrikant, 2009) and processing is relatively simple, resulting in a fast and correct decision. Type II decisions occur if the visualization requires additional effort. Additional effort could be due to a variety of reasons, such as inconsistencies with the viewer's preexisting knowledge, a lack of knowledge about uncertainty, or difficulty integrating multiple sources of information. In these cases, processing is effortful and requires more time and working-memory resources to arrive at a correct decision.

Decisions can be difficult without uncertainties, but when uncertainties are involved in decisions, they can become many times more complicated. For decisions with uncertainties, each option can lead to different outcomes, with the results arriving with different probabilities (Joslyn and LeClerc, 2013). We argue this may result in more effortful type II processing, even if representations of uncertainty match pre-existing knowledge about graphs. Since the assessment and estimation of probabilities is extremely complex, decision-makers often resort to simpler procedures in order to assess and weigh different options against each other. Such simplifications are called heuristics, or strategies (Tversky and Kahneman, 1974). While heuristics are often useful in simplifying a very complex problem, they can also lead to serious, systematic errors (Padilla et al., 2018). For example, in the context of uncertainty visualization, this can take the form of an inside-outside heuristic, where decision-makers have no sense of risk for data located outside the borders of a hurricane forecast's “cone of uncertainty”—which indicates a 66% confidence interval of potential forecast tracks—even though those areas are still at risk of being affected by the hurricane (Cox et al., 2013; Ruginski et al., 2016; Padilla et al., 2017).

Empirical studies on uncertainty visualization comprehension often contain complex tasks in which not “only” values have to be retrieved or compared from a map, but which require a thorough analysis of the primary data as well as the uncertainties. In concrete terms, participants often have to decide which option is the most optimal from a selection of options, sometimes based on multiple sources of information (Kinkeldey et al., 2015b; Kübler et al., 2019). The effects of uncertainty visualizations on the decision can be analyzed in two ways. In the objective assessment of the effects, it is analyzed whether more correct decisions are made as a result of the uncertainty visualizations, whether the decision-makers arrive at a decision more quickly or whether the decisions differ in other ways (Kinkeldey et al., 2015a; Padilla et al., 2018). The objective assessment, especially regarding decision accuracy, can be challenging, depending on the study design, due to the “right decision, wrong result” problem. The problem says that under uncertainty a right decision (the most rational decision based on task-relevant information) can lead to a wrong result due to uncertainty (Cheong et al., 2016, 2020). In the subjective assessment of the effects, it is analyzed whether the uncertainty visualizations change the attitude of the decision-makers (e.g., their risk attitude) or their decision security. Subjective assessments are usually based on self-assessments of the participants (Kinkeldey et al., 2015a).

Overall, results are mixed concerning the effects of uncertainty visualizations: uncertainty visualizations can result in more accurate decisions (Andre and Cutler, 1998; Greis et al., 2018), but accuracy can also remain unchanged compared to decisions made without uncertainty (Leitner and Buttenfield, 2000; Riveiro et al., 2014). Results are likely mixed since visualization choice has a large effect on comprehension, as specific encodings of uncertainty visualizations have been shown to be more effective than others at facilitating accurate judgments and reducing misconceptions (Cheong et al., 2016; Ruginski et al., 2016; Hullman et al., 2018). Others have argued that individuals take specific decision strategies when presented with uncertainty, which may or may not change accuracy. For example, Hope and Hunter (2007b) found that participants chose significantly more regions with low uncertainty, often although these regions should not have been chosen due to the low soil suitability class or other rational considerations. The authors justify this behavior with the concept of loss aversion by Kahneman and Tversky (1979). The concept states that, for example, in a bet with the same chance of losing and winning, the participants suffer more from a possible loss than they are happy about the possible win, which is why they reject the bet. In terms of a decision made with map-based uncertainty, this means that the participants estimate the potential losses due to high uncertainty to be higher than the potential gains, and therefore do not select locations with high uncertainty (Hope and Hunter, 2007b).

Even if an optimal uncertainty visualization technique is chosen, gathering information for a decision and cognitively processing the information is a time-consuming process. Decision-makers therefore almost always have internal or external time constraints in real-world decisions made with uncertainty. While time limits place internal or external constraints on decision-making, time pressure, on the other hand, only exists if the time limitation triggers a feeling of stress (Ordóñez and Benson, 1997). Time pressure is therefore a subjective (often negative) reaction to a time limit associated with strong arousal (Svenson and Edland, 1987). In uncertainty visualization research, however, the aspect of time pressure has not received extensive research. With the exception of a few studies (e.g., Riveiro et al., 2014; Cheong et al., 2016), the participants in uncertainty decision-making studies typically have as much time as needed for their decisions, which does not always correspond to real-world decision-making scenarios.

The current study therefore focuses on the influence of time pressure, given that time pressure can affect decision-making processes and outcomes (Hwang, 1994) and is important for real-world scenarios under time pressure, such as emergency search and rescue operations (Wilkening and Fabrikant, 2011) and evacuation decisions (Cao et al., 2016; Cheong et al., 2016). Results are mixed as to the effects of time pressure on decision-making. Time pressure can facilitate decision-making by increasing engagement or arousal but hinders decision-making when time pressure is extreme and induces stress (Hwang, 1994; Ordóñez et al., 2015). Negative effects of time pressure on decision performance are often characterized by speed-accuracy trade-offs. Speed accuracy trade-offs occur when faster decisions result in less correct decisions (Ahituv et al., 1998; Wilkening and Fabrikant, 2011; Kyllonen and Zu, 2016; Cheong et al., 2020). These trade-offs may occur due to increasing task difficulty (Hwang, 1994; Crescenzi et al., 2016; Padilla et al., 2019) or speed-confidence trade-offs, where decision confidence decreases with increased time pressure (Maule and Edland, 1997; Wilkening, 2012). Further, time pressure results in users reporting a lesser amount of satisfaction overall with their task performance (Crescenzi et al., 2013, 2016).

Even if time pressure does not alter decision outcomes, it can alter decision strategies and information search. For example, individuals under time pressure tend to constrain their search and alter decision-making strategies. Time pressure can have a negative effect on decision performance because decision-makers use different decision strategies to reduce cognitive stress (Hwang, 1994). These strategies are heuristics, as are the strategies for decisions under uncertainty. Research suggests that individuals process information more quickly (e.g., more effective or scattered search per unit of time) and filter information (Maule and Edland, 1997; Maule et al., 2000). Generally, decision-makers under time pressure use fewer but more important attributes, give greater weight to negative or task-irrelevant aspects, take fewer risks, and consider fewer options than decision-makers with unlimited time (Ahituv et al., 1998; Forster and Lavie, 2008).

While the effects of time-pressure on decision-making seem to be largely negative, very limited work has been done analyzing the effects of time pressure on geographic uncertainty data, especially when individuals must weigh multiple equally important sources of map information. To our knowledge, Wilkening and Fabrikant (2011) and Cheong et al. (2016) are the only studies that have studied the effects of time pressure on map-based decisions with uncertainty, even though individuals must make important, quick decisions with uncertainty every day. Yet it is critical to have a better understanding of how decision-making might change in these high pressure, information-dense situations, since decisions can have life-altering consequences in contexts such as search-and-rescue. Therefore, the current study will implement a search-and-rescue landing site selection task with uncertainty information, similar to Wilkening and Fabrikant (2011).

Wilkening and Fabrikant (2011) investigated the influence of time pressure on map-based decisions using various map representations of slope. In the study, the participants had to decide under low, medium or high time pressure, depending on the task, which of the six marked helicopter landing sites had a slope inclination of <14% and were therefore safe for a helicopter landing. Slopes were visualized using four different methods: (1) contour lines only, (2) contour lines plus light hill shading, (3) contour lines plus dark hill shading, and (4) contour lines plus colored slope classes. Wilkening and Fabrikant (2011) found that the participants performed significantly better with the slope color hue map (4) than with all other maps (see also Wilkening, 2012). Individuals also felt that this map was the safest way to make their decision. Participants performed best at medium time pressure, followed by coping under low time pressure. The worst performance was achieved by the participants under great time pressure. Further, when individuals had to make evacuation decisions under time pressure with uncertainty visualizations of forest fires, Cheong et al. (2016) found that individuals made the most accurate judgments under time pressure when uncertainty was encoded as color hue when compared to text, borders (simple solid and dashed lines), color value, and transparency. Due to the results of these studies, our study implements a diverging green-purple color hue to encode slope information. We chose to overlay uncertainty as a texture given texture's appropriateness to visualize uncertainty with a diverging color scale (Brewer, 2016; see section Preliminary Study Stimuli for more). The decision of texture type was driven by a preliminary study (section Preliminary Study), prior to our main study of interest (section Main Study).

Broadly the current study aims to determine whether and how map-based decisions under different conditions (without and under time pressure; with and without uncertainty) affect decision-making processes. To this end, we combine multiple measurements (accuracy, decision strategy, response time, decision confidence, self-assessed stress, eye-tracking) in order to determine where trade-offs occur in a search-and-rescue decision-making scenario and gain more targeted insights into how individuals process, perceive, and decide when viewing uncertainty visualizations. In order to achieve this goal, we propose the following research questions and associated hypotheses:

(1) How does uncertainty visualization affect complex map-based decisions made with and without time pressure?

H1a: Uncertainty visualizations will influence map-based decisions regardless of time pressure. This hypothesis is based on the previously mentioned research showing that uncertainty visualizations have an influence on map-based decisions without time pressure (e.g., Wilkening and Fabrikant, 2011; Hullman et al., 2018). In order to test this hypothesis, the results of complex map-based decisions are assessed both objectively (accuracy, response time) and subjectively (decision confidence). We expect uncertainty to be treated differently than certain data irrespective of time pressure due to its well-documented effects on decision-making. Specifically, we expect individuals to have less confidence in uncertainty information, take longer to answer due to more effortful decision-making and be less accurate than without uncertainty information.

A competing hypothesis, H1b, is that time pressure systematically changes the effect of uncertainty on decisions. This hypothesis based on the idea that uncertainty visualization comprehension is more effortful (type II decisions; Padilla et al., 2018). If this is the case, we would expect that time pressure will result in more errors with uncertainty, since time pressure would constrain the additional time necessary to arrive at an accurate decision. At the same time, we would expect that the control group takes more time to view uncertainty visualizations, relying less on fast heuristics, thus resulting in equal performance with and without uncertainty in the control group.

(2) How do uncertainty visualizations affect decision-making processes, such as visual search, with or without time pressure?

H2: Uncertainty visualizations influence the decision-making processes that precede decisions regardless of time pressure. Any effect of uncertainty on decision outcomes may be attributable to changes in the decision-making processes. The current hypothesis is based on the fact that decision research has shown that heuristics are used for decisions under uncertainties, which is why the decision-making process changes (Kahneman, 2011; Padilla et al., 2015; Ruginski et al., 2016). Eye tracking and decision strategies will be used to answer research question 2 and to test the related hypothesis. Specifically, we will determine whether gaze time for areas of interest (AOIs) in the display changes based on uncertainty and time pressure, and whether response strategy changes based on uncertainty or time pressure.

(3) How does decision-making with or without time pressure differ?

H3: Decisions made with time pressure will be faster, but less accurate, compared to decisions made without time pressure. This hypothesis is based on decision research, which has shown that time pressure can have negative effects on decisions when it induces stress in participants. In addition, it was shown that under time pressure very specific decision-making strategies are applied, which is why it is assumed that the decision-making processes differ. To test the hypothesis, we will utilize decision accuracy, response time, and eye tracker data. Specifically, we think that individuals with time pressure will answer more quickly, but less accurately, showing a classic speed-accuracy trade-off, compared to individuals making decisions without time pressure.

The first part of the study consisted of a personality questionnaire, the second part consisted of eight map-based decision tasks, and the third part consisted of questions about the scenario and uncertainty visualizations.

Participants answered dot stimuli the fastest [mean (M) = 33.9, standard deviation (SD) = 10.0], no uncertainty stimuli less fast (M = 36.1, SD = 10.6), and hatch stimuli the slowest (see Figure 2, M = 42.9, SD = 14.5).

Nested non-parametric Wilcoxon-Mann-Whitney tests for group differences using 10,000 sample Monte Carlo distribution approximations implemented with the coin package in R (Zeileis et al., 2008) showed that the mean values were not significantly different between dots and hatches (z = −1.94, p = 0.05), between dots and no uncertainty (z = −0.72, p = 0.48), or between hatches and no uncertainty (z = 1.5, p = 0.13). Still, the dots method was chosen for the primary study given the fastest mean response time, lowest variability in response time, and more positive feedback in the written follow-up amongst all visualizations.

Without depiction of slope uncertainty, participants selected landing spots that were closer to the person to be rescued, but not necessarily at locations with ideal slope conditions. For decisions supported by visualized slope uncertainty, participants opted for the safer landing location, but further away as necessary from the person to be rescued. Safer in this case means that participants who were not under time constraints chose to land the helicopter at a location, which they knew with more certainty would be suitable for landing.

From a theoretical point of view, the choice of Category 8 landing sites was not a bad decision even with uncertainty. Since uncertainties meant that the slope gradients could be in the previous or next class, participants had to assume that the slope gradients would be or remain unchanged in the previous or next class with a probability of 33.3%. For the Category 8 landing sites, this means that the slope inclinations with uncertainties of 33.3% each are “bad,” “acceptable,” or “ideal.” The probability that the slope inclination of a Category 8 landing site is suitable for landing despite uncertainties is therefore 66.6%, which is greater than the probability that the landing site is unsuitable. Since site 8 is closer to the target person than the landing sites of Category 7, it would be a marginally equal choice to the landing site of Category 7, which had a probability of 100%, depending on how participants weighted information. Nevertheless, the participants more often chose the landing sites of Category 7 for decisions when uncertainty was present in the maps.

We argue that this decision strategy can be explained by the phenomenon of loss aversion. This decision strategy originated from a study by Kahneman and Tversky (1979), which shows that in a bet with the same chance of loss and profit, the participants suffer more from loss than rejoice over the profit, which is why they reject the bet. Since we observed this strategy when participants were confronted with uncertainty regardless of time pressure, it could be that uncertainty leads to loss aversion decisions more broadly and could be a defining feature of decisions made with uncertainty. Is there other evidence that uncertainty provokes these kinds of decisions?

Loss aversion was observed by Hope and Hunter (2007b), in which the participants had to decide on a new airport location based on a soil suitability map. In this study, participants tended to avoid choosing areas with greater uncertainty, since they viewed potentially low land suitability as a greater loss than the potential gain of high land suitability. Therefore, uncertain locations were viewed as risky and avoided more frequency. In addition to Hope and Hunter (2007b), Cheong et al. (2016) also noted that loss aversion probably played a role in forest fire evacuation decisions. Kübler (2016) shows, however, that loss aversion does not play a role in all decisions with uncertainty visualizations. In her study, the participants more frequently chose houses in insecure regions and thus, according to Kübler (2016), estimated the possible profit of a house in a zone with uncertainties higher than the possible loss, contrary to the concept of loss aversion.

Why might loss aversion play a role in some decisions with uncertainty visualizations, but not others? It is conceivable that the risk attitude of the participants may have an influence on whether they estimate possible gains from uncertainties to be less than possible losses. It is also possible that the decision scenario and the nature of the risks will determine whether decision-makers overestimate the potential losses due to uncertainties. Minimizing losses of life (evacuation) and maximizing profits (real estate) are two very different contexts with different stakes. More work should be done determining which specific data domain contexts this effect occurs (such as evacuation, real estate, population, epidemiology, etc.; see section Limitations and Future Directions for more).

Since uncertainty visualizations affect decisions without time pressure, we expected uncertainty to affect decision accuracy irrespective of time pressure (Deitrick and Edsall, 2006; Riveiro et al., 2014; Kinkeldey et al., 2015a). Still, we considered the alternative hypothesis that decisions with uncertainty might change dependent on time pressure (Padilla et al., 2018). We found that the uncertainty visualizations led to the participants of the control group taking significantly longer to reach a decision, even though it did not benefit their accuracy. This result could be due to the high accuracy overall across all experimental conditions (89%) suggesting that the task may have not required the cognitive effort necessary to provoke type II decision-making, even if decisions were made more slowly (Padilla et al., 2018). Future work could further investigate this effect by introducing a dual-task—such as remembering a sequence of numbers or words—which more directly manipulates cognitive workload (Padilla et al., 2019). This result could also be due to the fact that time pressure did not seem to result in additional stress (see section Limitations and Future Directions for more).

This finding is not necessarily supported by past literature—in many cases uncertainty visualizations do not influence decision time (Leitner and Buttenfield, 2000; Riveiro et al., 2014; Kübler, 2016). In one case, uncertainty visualizations even led to shorter decision times (Andre and Cutler, 1998). In the current study, the participants had to solve multi-criteria decision tasks, similar to Riveiro et al. (2014) and Kübler (2016). The discrepancy between the results here and those from the literature thus likely have a different reason than the complexity of the decision-making tasks.

Hope and Hunter (2007a) noted that uncertainty visualization techniques can have an extremely significant influence on decisions. Kübler (2016) was also able to demonstrate that different uncertainty visualization techniques can lead to significantly different decision-making times. It is therefore conceivable that the discrepancy to the existing literature is due to the uncertainty visualization technique applied here, because none of the authors mentioned used only a texture as uncertainty visualization, as it was done in this study. Kübler (2016) used three different techniques, one of which was texture based. In this study, however, blurring led to longer decision times, but not texture. As explained in the research context, Cheong et al. (2016) compared the influence of six different uncertainty visualization techniques on decisions. One of these techniques was a point-based texture (Johannsen et al., 2018). They found that participants with the texture and two other techniques took slightly longer to reach a decision than with the other three visualization techniques. In the existing literature, there is therefore no clear evidence that mainly texture-based uncertainty visualizations lead to longer decision times, which is why the extended decision time in this study is not clearly attributable to the uncertainty visualization technique used. We still hope future work assesses similar decision tasks using alternative encodings of uncertainty (see section Limitations and Future Directions for more).

Our eye-tracking results provide some insight into how participants utilize extra time during decision-making with uncertainty. Specifically, participants tend to look at the map and decision-relevant map legend more often. It appears that slope classifications were looked at more often during decisions with uncertainty and especially when there was no time pressure. This took away from time to lookup legend information that was decision-relevant, but certain, such as the distance and obstacle information. Since eye tracking has hardly been applied in research on spatial uncertainty visualizations as in research on general information visualization (e.g., Goldberg and Helfman, 2010, 2011; Toker et al., 2017), it is difficult to place these results into a larger context. To our knowledge, Brus et al. (2012) are among the very few who also used the eye-tracking method to investigate how participants make decisions with uncertainty maps. They found that the legend of uncertainty visualizations in particular attracted the attention of the participants, which is why they came to the conclusion that it is particularly important to depict the uncertainties in a legend. This phenomenon was also observed in our study. But since the additional gaze time with these areas during uncertainty was not associated with greater decision accuracy (individuals with time pressure performed equally well), participants may have gained the information necessary to make an informed decision by a certain timepoint. Another interpretation is that individuals became more efficient with their visual search behavior under time pressure, or perhaps implemented strategies to make faster decisions by memorizing the map legends in advance of the task. Additional research is necessary to further disentangle how changes in attentional processes affect visualization decision outcomes.

As shown in section Time Pressure Manipulation Check, the SSSQ did not reveal any significant changes in self-assessed worry, stress, or engagement during the decision task. This can imply two things. On the one hand, this result could mean that the time limit used did not result in stress; on the other hand, it could be that the participants felt very well-stressed and under time pressure, but the SSSQ was not the appropriate means to measure this stress. Since time pressure only exists when a time limit triggers a feeling of stress (Ordóñez and Benson, 1997), it is important that our study induced stress in participants. Several studies have shown that the SSSQ is a sensitive indicator of changes in personal sensation and responds sensitively to a wide range of stressors (such as the time limit in the current study) and changes in task complexity (Helton, 2004; Helton and Näswall, 2015). Given that time pressure resulted in equal accuracy and decision strategies and did not affect SSSQ scores, we believe it is likely that participants simply did not feel stressed as a result of the manipulation, or that the task was too easy given a possible ceiling effect of accuracy. We suggest future work that constrains time limits further and implements converging measures of emotional response, such as skin conductance and pupillometry (Bradley et al., 2008). Another approach to make the scenario more realistic or stressful could include providing explicit feedback to participants or “gamifying” the task, which affect engagement, arousal, and performance (Hamari et al., 2014; FeldmanHall et al., 2016; Wichary et al., 2016). This is especially important to address since stress can change risk and reward salience (Mather and Lighthall, 2012). Further work should explore adding explicit feedback to uncertainty visualization decisions under stress.

Further, while our study sought to make claims about uncertainty visualization generally, we only studied one type of uncertainty visualization (texture), which was overlaid on slope data binned according to color hue. There is an entire line of research devoted to color choice and the benefits and pitfalls of binning color hue, saturation, and contrast in visualizations (Iliinsky and Steele, 2011; Padilla et al., 2016; Bujack et al., 2017; Szafir, 2017). For example, some have argued for utilizing value-suppressing methods of depicting uncertainty, where uncertainty constraints both color hue and saturation, as opposed to a separately overlaid texture or bivariate color map (Correll et al., 2018). Therefore, we cannot necessarily generalize our results to other non-texture, non-color-based methods of data and uncertainty visualization, or methods of uncertainty that are depicted using gradients, rather than a binary yes/no classification. We hope that future research explores how the effects of uncertainty on decision-making differ in scenarios where uncertainty is depicted with color only, texture only, and other combinations of visual encoding channels (e.g., where hue encodes data value and saturation encodes uncertainty around that value).

We did not test our display with experts familiar with our display, such as helicopter pilots, who likely utilize existing knowledge to inform their decisions and may have completed the task differently from non-experts. In addition, to make the task manageable, we did not include all possible variables that may have affected pilot decisions, such as visibility. Future work should assess these displays with experts and include all relevant decision variables to determine if these results are generalizable to actual use contexts.

Similarly, our study was limited to a cartographic display in order to simulate a stressful decision-making scenario with geographic data. However, work shows that contextual variables, such as narrative framings and data domain, can affect visualization interpretation (Hullman and Diakopoulos, 2011; Correll and Gleicher, 2014; Hullman et al., 2018). These contexts have also been shown to affect behavior and decision-making under conditions of risk and uncertainty (Kübler et al., 2019). Further, the effectiveness of visualizations can differ based on the task, such as simple value extraction, value comparison, or comparison based on multiple data sources with uncertainty, as was the case in our study (Padilla et al., 2016; Kim and Heer, 2018; Dimara et al., 2020). We therefore implore researchers to further examine decision-making with geographic uncertainty data, including studies that directly compare the same underlying map data with different task framings.

Last, uncertainty can have many meanings and we only provided brief training with uncertainty to non-experts, which may have limited individuals understanding of uncertainty. Different types of uncertainty result in differing interpretations and misinterpretations (Hullman, 2016). For example, inferential uncertainty from statistical models depicted with confidence intervals can lead to overestimations of effects compared to outcome uncertainty depicted with prediction intervals (Hofman et al., 2020). One potential solution to minimizing uncertainty misinterpretations in 2D data is by presenting hypothetical outcome plots, which explicitly depict uncertainty in a frequency framework (Kale et al., 2019). Others have argued that raising user awareness of uncertainty can reduce trust in the data, especially if the source of uncertainty is unclear. Reduced trust may also explain the loss aversion strategy (Sacha et al., 2015). Future work should utilize different definitions, types, and framings of uncertainty to further assess the generalizability of our results, and determine whether shifts in awareness or trust change how users utilize uncertainty information.