Uncertainty and cognitive control
- 1 Institute of Psychological Sciences, University of Leeds, Leeds, UK
- 2 Wolfson Research Institute, University of Durham, Durham, UK
A growing trend of neuroimaging, behavioral, and computational research has investigated the topic of outcome uncertainty in decision-making. Although evidence to date indicates that humans are very effective in learning to adapt to uncertain situations, the nature of the specific cognitive processes involved in the adaptation to uncertainty are still a matter of debate. In this article, we reviewed evidence suggesting that cognitive control processes are at the heart of uncertainty in decision-making contexts. Available evidence suggests that: (1) There is a strong conceptual overlap between the constructs of uncertainty and cognitive control; (2) There is a remarkable overlap between the neural networks associated with uncertainty and the brain networks subserving cognitive control; (3) The perception and estimation of uncertainty might play a key role in monitoring processes and the evaluation of the “need for control”; (4) Potential interactions between uncertainty and cognitive control might play a significant role in several affective disorders.
Decision-making during uncertainty is classically defined as a situation that has limited or incalculable information about the predicted outcomes of behavior (Huettel et al., 2005). Uncertainty is considered a key dimension of everyday behavior that has a significant influence on decision-making (Yoshida and Ishii, 2006), and important links with emotion and psychopathological disorders (Holaway et al., 2006; Boelen and Reijntjes, 2009). Although uncertainty has long been a topic of scientific interest (see Bertelson and Boons, 1960), it has become an object of intense investigation in more recent years, inspiring a growing number of behavioral, neuroimaging, and computational studies.
Most of the available experimental studies referring to the concept of uncertainty have taken place within the realm of research on decision-making processes. More specifically, these studies usually employ tasks in which participants have to learn to predict future outcomes on the basis of past outcomes that often take the form of positive and/or negative reinforcements. Uncertainty in these paradigms is usually manipulated by the variation of the predictive power of past outcomes1. Much of this research has focused on statistical models of learning such as Bayesian or Reinforcement learning models (Daw et al., 2005; Kording and Wolpert, 2006; Yoshida and Ishii, 2006; Krynski and Tenenbaum, 2007). Most evidence to date suggests that humans are able to effectively handle uncertainties in the environment to predict future events and make appropriate decisions (Volz et al., 2003; Hsu et al., 2005; Yu and Dayan, 2005; Kording and Wolpert, 2006; Yoshida and Ishii, 2006; Behrens et al., 2007).
However, although these approaches have been generally successful in providing formal algorithms fitting human behavior in uncertain conditions (Steyvers et al., 2003; Chater et al., 2006; Yoshida and Ishii, 2006; Nassar et al., 2010), these models tend to be largely agnostic regarding the specific cognitive mechanisms recruited for this successful adaptation. In particular, with a few exceptions (e.g., Daw et al., 2005), it is unclear if implicit learning processes are sufficient to adapt to uncertainty or if high level cognitive control processes are also involved. Yet, the possibility that cognitive control processes (or executive function) are involved in overcoming uncertainty has often been proposed (e.g., Huettel et al., 2005). In addition, uncertainty may also be to some extent tacitly present in many cognitive control models and tasks (see Botvinick et al., 2001; Miller and Cohen, 2001). However, the field of research on uncertainty and the field of research on cognitive control processes have evolved mostly independently and attempts to integrate them are rare. The goal of this article is to explicitly address the potential relationships between the constructs of uncertainty and cognitive control. We will first outline the conceptual overlaps between uncertainty and cognitive control. Next, we will review a large body of neuroimaging evidence suggesting an overlap between brain areas involved in processing uncertainty and the neural network subserving cognitive control. Then, we will review evidence suggesting that uncertainty has a significant influence on a key component of cognitive control – monitoring processes. Finally, we will review a body of evidence suggesting that the construct of “Intolerance of Uncertainty” (IOU) is linked to affective disorders widely known to be associated with deficits in cognitive control.
Theoretical Overlaps between Uncertainty and Cognitive Control
Uncertainty is generally seen as a realization that our beliefs and representations of the world are unable to accurately predict future events in our environment. In behavioral and cognitive sciences, uncertainty has mainly been defined within the scope of decision-making and therefore refers to a difficulty to predict events that are the consequences of our actions (Volz et al., 2003; Hsu et al., 2005; Huettel et al., 2005; Yu and Dayan, 2005). Uncertainty can present itself under many forms. For instance, an influential theory (Yu and Dayan, 2005) proposed that there are two fundamental types of uncertainty: expected and unexpected uncertainty. The former relates to environments in which available information is a weak predictor of future events. However, this unreliability is stable and known by agents in the environment. For instance, driving in a city where most drivers consistently display a poor compliance of traffic rules is a typical example of expected uncertainty: the rules are weak predictors of how drivers behave (and of how they will react to our own behavior), and this unreliability is known and relatively stable. In contrast, unexpected uncertainty refers to a situation in which fundamental changes in the environment invalidate past predictions. For instance, pilots of aeroplanes in cruise at high altitude experience this type of uncertainty when they encounter turbulence that had not been forecast.
Empirical literature on uncertainty has used experimental paradigms that can be subsumed under these two categories. A large number of studies induced uncertainty by lowering the predictive power of task cues (e.g., Bertelson and Boons, 1960; Scheffers and Coles, 2000; Pailing and Segalowitz, 2004; Huettel et al., 2005). Typically in these studies, participants are shown cues which are probabilistic predictors of a given outcome (e.g., a square that predicts the occurrence of a reward for 80% of test trials). Uncertainty in these paradigms is attained by lowering the accuracy of the prediction (e.g., a square that predicts rewards only for 60 or 50% of test trials). Importantly, in many of these paradigms the unreliability of cues remains stable during long blocks of trials (e.g., Huettel et al., 2005), which approximates the concept of expected uncertainty. In contrast, other studies manipulated the volatility of task rules (e.g., Behrens et al., 2007), in which uncertainty is achieved by dynamically changing stimulus–response rules across trials. For instance, if rule A involves reward outcomes for 80% of squares and rewards for 30% of circles, volatility will be achieved if this rule frequently alternates with a rule B that would involve the inverse reward probabilities of rule A. A stable (and thus “certain”) condition would involve a long sequence of trials in which only one of the rules is true. Manipulating uncertainty through volatility of rules approximates Yu and Dayan’s (2005) concept of unexpected uncertainty, as it introduces fundamental changes in the environment that invalidate past predictions.
The common feature of these different categories of experimental manipulations is that they create states that signal the need to actively regulate our representations of the environment, in order to obtain better predictions and therefore achieve better adaptation. This regulation can include a suppression and replacement of current representations, or adjustments of current representations that reflect the level of predictive unreliability of the environment (Daw et al., 2005; Yu and Dayan, 2005).
Cognitive control, also often referred to as executive function, usually denotes a category of processes that are implemented when automatic schemata are not sufficient for successful adaptation to the environment. These processes tend to be conscious2, attention-demanding, and they involve a flexible coordination of several cognitive processes in order to attain a specific goal (Atkinson and Shiffrin, 1968; Baddeley, 1986, 2003; Norman and Shallice, 1986; Baddeley and Della Sala, 1996; Miller and Cohen, 2001; Braver et al., 2007).
For instance, experienced car drivers utilize a number of over-trained automatic schemata on a daily basis (e.g., how to negotiate roundabouts and crossroads, how to operate the gearbox, etc.). However, if US or continental European drivers have to drive in a right-hand drive country (e.g., UK or Australia), then these automatic schemata are no longer adapted and a series of conscious, attention-demanding processes have to be implemented on a sustained basis. The driver will have to actively inhibit previously learned automatic behaviors and might have to consistently maintain the new traffic rules that have to be followed in working memory (WM). This example illustrates a typical implementation of at least two canonical cognitive control processes: inhibition of dominant responses and active maintenance of relevant information in WM stores (Miyake, 2001; Braver and Ruge, 2006). It is easy to imagine other cognitive control tasks that these drivers might have to carry out: dual tasks, controlled encoding, and retrieval from episodic memory of new driving rules, updating of rules in WM, etc.
Research on cognitive control has flourished in the last 25 years, and enormous progress has been attained in characterizing different subprocesses or dimensions of cognitive control (Miyake, 2001; Braver et al., 2007), as well as their neural substrates (e.g., Botvinick et al., 2001; Miller and Cohen, 2001; Braver and Ruge, 2006; Sakai, 2008). The question of what “triggers” cognitive control implementation, i.e., what determines that a system has to switch from an automatic toward a controlled processing mode, has remained for a long time unexplored and relegated to a “black box” (Baddeley, 2003). However, a relatively recent trend of research has explored the hypothesis that specific systems are devoted to the detection of the “need for control.” Specifically, a number of models have suggested the existence of brain systems devoted to monitoring the need for control and responsible for sending relevant “trigger” signals to other systems responsible for implementing control processes (Botvinick et al., 2001; Davis and Whalen, 2001; Sander et al., 2003; Kerns et al., 2004; Schaefer et al., 2006; Yeung and Cohen, 2006; Schaefer and Gray, 2007). A number of neuroimaging studies have provided clues on the neural network involved in these functions [see below the discussion about the anterior cingulate cortex (ACC), orbital frontal cortex (OFC), and amygdala].
Several hypotheses have been provided regarding the exact contextual conditions that would signal the need for control: detection of errors (Yeung and Cohen, 2006; Braver et al., 2007), changes in the motivational value (or goal-relevance) of the environment (Sander et al., 2003; Schaefer et al., 2006; Baddeley, 2007), conflict between competing responses (Botvinick et al., 2001), and changes in the prediction of error likelihood (Brown and Braver, 2005). Importantly, the common feature of these hypotheses is a conflict between a current representation used to guide behavior (and often accessed in an automatic mode) and environmental conditions in which expected outcomes are unlikely to be optimally achieved.
Points of Intersection between Uncertainty and Cognitive Control
From the considerations above, it becomes clear that the constructs of uncertainty and cognitive control have at least two fundamental points of intersection:
(1) The definition of uncertainty has a remarkable similarity with the conditions usually thought to signal the need for control. In both cases, there is a mismatch between available schemata and the actual environment that leads to suboptimal outcomes (errors, conflict between responses, cognitive cost etc.).
(2) This mismatch is a signal that demands the implementation of processes that will actively adjust the representations guiding behavior so that a more efficient adaptation can be achieved.
Despite these similarities, it may be the case that humans do not need cognitive control to resolve uncertainty. It is entirely conceivable that implicit statistical learning could be sufficient to achieve a successful adaptation to many forms of uncertainty. We will hereafter review available research that can potentially provide clues regarding the possibility that cognitive control is recruited during uncertain decision-making.
Functional Neuroimaging of Uncertainty: Evidence of Overlapping Patterns of Brain Activity between Decision-Making under Uncertainty and Cognitive Control Tasks
Neuroimaging studies using mainly functional magnetic resonance imaging (fMRI) have provided one of the most prolific bodies of scientific research on uncertainty in recent years. Available evidence suggests that a distributed network of brain areas is systematically involved in tasks manipulating uncertainty. Next, we examine some of the key components of this network and its overlap with the network of brain areas known to be associated to cognitive control tasks.
Dorsolateral Prefrontal Cortex
Traditionally, the dorsolateral prefrontal cortex (DLPFC) is thought to play a crucial role in WM and in the active maintenance of current task goals (Goldman-Rakic, 1995). Extensive neuroimaging and neuropsychological evidence shows that the DLPFC is strongly implicated in cognitive control tasks. For instance, it is involved in the retention of information during WM tasks and may store active representations that are necessary for adaptive decision-making performance (see Curtis and D’Esposito, 2003).The DLPFC has also been implicated in response selection in the presence of interfering or conflicting information (Rowe et al., 2000; Botvinick et al., 2001; Hadland et al., 2001; Schumacher and D’Esposito, 2002; Schumacher et al., 2003), control adjustments (Durston et al., 2003), the formulation and execution of plans of action (Fuster et al., 2000), top-down attentional control (Milham et al., 2003), and in maintaining attentional demands of the task (MacDonald et al., 2000). In addition, the DLPFC is also involved in task switching (Hyafil et al., 2009; Savine and Braver, 2010). Overall, several studies and models suggest that the DLPFC is involved in the implementation of control processes in order to resolve conflict, in contrast with other areas thought to be involved in the detection of conflict (Carter and Van Veen, 2007). This interpretation would be consistent with the fact that the DLPFC has preferential connections with motor system structures which may be central to how the PFC exerts control over behavior (Miller and Cohen, 2001). A wealth of additional studies have explored the links between the DLPFC and cognitive control, and reviewing them would be beyond the scope of this article. We refer the readers to authoritative reviews on this topic for more details (Miller and Cohen, 2001; Braver and Ruge, 2006; Carter and Van Veen, 2007; Mansouri et al., 2009).
Recent research suggests that the DLPFC may also play an important role in decision-making under uncertainty. Particularly, Huettel et al. (2005) found increased hemodynamic activity in the DLPFC during decision-making under uncertainty. The authors presented participants with eight different types of stimuli (varying amounts of triangles and circles). Participants were asked to make high or low confidence judgments about the likelihood of the feedback stimulus being the same as their selection. If the stimuli included all triangles or all circles, the chance of feedback stimuli having the same shape would be 80%, thus the uncertainty would be low at 20%. These authors varied the proportion of shapes presented as stimuli so that selections could be made in 20, 27.5, 35, 42.5, or 50% uncertainty. As uncertainty increased, so did activation in the DLPFC; which they suggest reflects the DLPFC’s role in uncertainty resolution in the form of short-term modification of stimulus–response contingencies, a process that might rely on high level cognitive control processes (Huettel et al., 2005). These representations of stimulus–response contingencies are thought to guide our everyday decisions through underlying rules in which we learn that a specific association between a stimulus (S) and a response (R) is linked with a positive or negative outcome (O) (De Wit and Dickinson, 2009). Interestingly, evidence from probabilistic learning tasks such as the weather prediction task (Gluck and Bower, 1988; Gluck et al., 2002) suggests that the involvement of the DLPFC in probabilistic learning might reflect implicit, non-declarative processes rather than conscious control processes (Poldrack et al., 1999; Weickert et al., 2002). However, recent evidence indicates that the DLPFC is not necessary to non-declarative learning in these tasks (Rushby et al., 2011).
Overall, these findings suggest a role for the DLPFC in modifying S–R–O contingencies to enable adaptive decision-making in the face of uncertainty and that activation in the DLPFC is dependent on predictability of S–R–O contingencies (Paulus et al., 2002).Therefore, predictability and thus certainty, may be represented in the DLPFC through maintaining representations of the reinforcement history (previously encountered S–R–O rules) to select an “optimal strategy” (Paulus et al., 2002). It could therefore be speculated that the DLPFC is involved in the active maintenance and manipulation in WM of a history of past contingencies in order to facilitate the prediction of future outcomes. Nevertheless, there is some inconsistency across studies with evidence indicating that the DLPFC activation increases as decision stimuli become more unexpected (Huettel et al., 2002) and others that suggest that the DLPFC is more active during low error rates, i.e., greater predictability (Paulus et al., 2002).
Posterior Parietal Cortex
Parietal areas and the posterior parietal cortex (PPC) in particular have often been associated to cognitive control tasks (Braver and Ruge, 2006). The PPC has been implicated in an array of cognitive control processes including task switching (Sohn et al., 2000) and movement planning (Snyder et al., 1997). The PPC is also thought to be important for sensory–motor integration that might encode movement intentions (Andersen and Buneo, 2002). Indeed, neurons within the PPC of non-human primates provide codes for decision variables such as probability and expected value when making a decision between potential movement intentions (Platt and Glimcher, 1999).
Along with the DLPFC, Huettel et al. (2005) also found that the resolution of uncertainty was linked with the PPC. These authors suggest that the PPC may also be involved in short-term response selection processes such as modification of S–R–O contingencies during uncertainty. More long term resolution of uncertainty may rely on S–R–O contingency development, i.e., learning adaptive S–R–O rules which might be mediated by areas of the medial frontal cortex (Huettel et al., 2005). Indeed, Volz et al. (2003, 2004) also demonstrated medial frontal cortex and PPC activation increases with increasing uncertainty. In their study, Volz et al. (2003) showed participants pairs of pictures showing comic figures which were systematically associated with a particular reward probability (either 0.6, 0.7, 0.8, 0.9, or 1.0), and these associations were consistent throughout the experiment. As uncertainty increased, posterior parietal areas known to underlie WM functions also increased. It has been suggested that this type of result could reflect the role of the PPC in actively maintaining in WM all S–R–O rules that are valid in an experiment (Bunge et al., 2002).
Anterior Cingulate Cortex
The Anterior Cingulate Cortex (ACC) is one of the most prominent neural substrates identified in cognitive control processing (see Botvinick et al., 2001; Braver and Ruge, 2006). For instance, the ACC has been found to be involved in many classical cognitive control tasks such as the Stroop task (Pardo et al., 1990), the Flanker task (Botvinick et al., 1999), N-back task (Harvey et al., 2005), and stop-signal tasks (Brown and Braver, 2005), amongst others. Evidence also suggests that the ACC is involved in monitoring cognitive conflicts (MacDonald et al., 2000; Botvinick et al., 2001) and error detection (Ullsperger and Von Cramon, 2003). Certain electrophysiological components known to be linked to cognitive control (and to conflict monitoring in particular) are thought to be generated by ACC sources. For instance, the N2 and the Error-related negativity (ERN/Ne) event-related brain potentials (ERP) components are thought to originate in the ACC (Dehaene et al., 1994; Dikman and Allen, 2000; Gehring and Knight, 2000; Debener et al., 2005) and are likely to reflect the cognitive demands of situations involving a high level of conflict between competing responses (Yeung and Nieuwenhuis, 2009). Importantly, the ACC is widely thought to be part of a network of brain structures specialized in the detection of environmental conditions signaling the need for the implementation of cognitive control, and responsible for sending “triggers” to other systems specialized in the actual implementation of control (Botvinick et al., 2001; Kerns et al., 2004; Brown and Braver, 2005).
Activity in the ACC has also been demonstrated to correlate with levels of uncertainty (Stern et al., 2010). In their task, Stern et al. (2010) showed participants four sequential draws of cards taken from two decks. Deck A which contained 80% red cards and 20% blue cards and Deck B which contained 20% red cards and 80% blue cards. Participants had to identify the deck from which the cards had been drawn. Uncertainty was manipulated by the frequency of cards presented with the same color in the sequence of four cards presented to the participants. For instance, four blue cards was the highest certainty condition, whereas a combination of two red and two blue cards was associated to maximum uncertainty. The authors found that increased uncertainty was associated with increases in ACC activity. According to the authors, the ACC was specifically involved in “evidence accumulation” whereas the OFC was involved in the execution of the decision following the four sequential draws of cards (Stern et al., 2010). Increased activity in the ACC has also been found to be related to outcome uncertainty and uncertainty-related physiological arousal (Critchley et al., 2001). In their study, Critchley et al. (2001) gave participants a two choice decision-making task, in which a cue (a card) was predictive of the value of a “feedback card.” Participants had to decide from the initial card if the feedback card would be higher or lower than the initial card. Uncertainty was manipulated by the degree to which the initial card was a reliable predictor of the feedback card. The participants then experienced a delay period before the outcome (gain/loss) was presented. Critchley et al. (2001) found that the ACC was modulated by both outcome uncertainty and anticipatory delay. In addition, the authors found a distinct region of the ACC that was commonly modulated by both uncertainty and physiological arousal.
The ACC has also been associated with hypothesis testing in uncertain contexts (Elliott and Dolan, 1998). Specifically, Elliott and Dolan (1998) presented participants with a series of pairs of checkerboards with different spatial configurations of black and white squares in which participants were asked to try to work out a rule governing which of each pair was correct (feedback was set at an uncertain 50% correct level). The task requirement was to generate a hypothesis (i.e., top left-hand corner filled) and then test and update it based on information about which checkerboard was correct. Hypothesis testing and making a choice was associated with different foci within the anterior cingulate. Whilst the dorsal ACC was involved in complex hypothesis testing, the ventral ACC was associated with implementing a choice. This suggests different roles of the ACC in purely cognitive aspects of processing (hypothesis testing) and evaluative processes related to the emotional consequences of a choice (Elliott and Dolan, 1998.) Indeed, the ventral region of the ACC is interconnected with limbic regions such as the amygdala which is widely thought to underlie emotional processes (Bracht et al., 2009).
Finally, the ACC has also been linked to volatility, i.e., uncertainty created by frequent changes in S–R–O contingencies (Behrens et al., 2007). In Behrens et al.’s (2007) study, subjects carried out a one-armed bandit task in which they had to choose between blue and green stimuli. Subjects underwent trials where the probability of a blue outcome was 75% (a certain/stable environment) and trials where reward probabilities switched between 80% blue and 80% green every 30 or 40 trials (an uncertain/volatile environment). Behrens et al.’s (2007) results suggest that ACC activity might reflect a Bayesian estimate of the environment’s volatility during a monitoring stage, i.e., when outcomes are being evaluated in order to regulate current beliefs about the underlying stimulus–response contingencies of the environment. The model of Behrens et al. (2007) also suggests that the ACC might encode how much influence feedback should give to subsequent decisions, with more recent outcomes being more salient in uncertain contexts (Rushworth and Behrens, 2008).
The available findings about the involvement of the ACC in decision-making under uncertainty and in cognitive control tasks provide an interesting parallel. On the one hand, studies and models from the realm of cognitive control research suggest that the ACC might be involved in the detection of the contextual conditions signaling the need for control. On the other hand, data from the field of decision-making processes suggest that the ACC might be involved in the computation of the level of uncertainty of current SRO representations (Behrens et al., 2007). An integration of these findings could tentatively suggest that both traditions of research are tapping overlapping processes, in the sense that uncertainty could be easily seen as a condition signaling the need for control. However, results from the field of decision-making under uncertainty (and in particular the study from Behrens et al., 2007) might be providing a more formal description of the computational nature of one of the potential mechanisms that can trigger the implementation of cognitive control. Nevertheless, at this stage, these considerations remain hypothetical, as there is to our knowledge no evidence that clearly demonstrates that the estimation of uncertainty, as described by Behrens et al. (2007), necessarily lead to the implementation of cognitive control, or that the adaptation to uncertainty in these studies relied on processes other than implicit learning.
The Orbito-Frontal Cortex (OFC) has classically been linked to the perception of reward and punishment feedback, including anticipation and receipt of feedback, and is usually thought to be involved in the representation of stimulus–reward relationships (Rolls, 1996; O’Doherty et al., 2003). The OFC has also become a brain area of interest for cognitive control when it was discovered that this region not only detects valenced outcomes but it also probably uses this information to bias future behavioral choices (O’Doherty et al., 2001, 2003). Therefore, it has been suggested that the OFC might also be involved in processing environmental conditions signaling the need for behavioral control (O’Doherty et al., 2003; Braver and Ruge, 2006).
More recently, the OFC has also been shown to differentiate between different levels of uncertainty (Hsu et al., 2005). In this study, Hsu et al. (2005) manipulated uncertainty by creating situations in which participants had to make choices under ambiguity. For instance, one of the conditions included placing a bet on the color of a card (e.g., blue or red) to be drawn from a deck without knowing the actual proportions of blue and red cards of the deck. In control conditions, relevant information was available prior to the choice (i.e., participants were told the exact proportions of red and blue cards in the deck). They found that the level of uncertainty correlated with activation of the OFC and the amygdala. These authors suggest that a network involving the OFC and the amygdala would reflect the operations of a “vigilance”/evaluation system dedicated to evaluate uncertainty and bias behavioral adaptation as a function of uncertainty levels. They further support this idea through a behavioral study on patients with OFC lesions, finding that OFC lesions impair the ability to distinguish between degrees of uncertainty (Hsu et al., 2005).
Related results suggest that the OFC enables acquisition of S–R–O contingencies, whereby patients with OFC lesions show deficits in shifting behavior when the underlying S–R–O rule is volatile (Rolls, 2000). This is consistent with data showing that OFC activation is associated with hypothesis generation of S–R–O rules (Goel and Dolan, 2000) and the detection of change in reward contingencies (O’Doherty et al., 2003). In addition, a number of other studies have also shown that brain activity in the OFC increases with increasing outcome uncertainty (Critchley et al., 2001) and reward uncertainty (Tobler et al., 2006). Finally, activity in the OFC also increases as the probabilistic stimulus–response contingencies become more complex, suggesting that the OFC may reflect additional requirements of dealing with uncertainty (Elliott et al., 1999). However, Stern et al. (2010) suggest that activation of the OFC by uncertainty is absent when there is no risk, suggesting that the OFC does not detect cognitive uncertainty per se, but instead responds to uncertainty only when there are incentive consequences.
In summary, available research clearly shows that the OFC is sensitive to several types of manipulation of uncertainty. It is unclear at this stage as to what specific role the OFC plays in the processing of uncertainty. However, research in the field of cognitive control would indicate that the OFC might be involved in tracking valenced information signaling the need for behavioral control. It could therefore be speculated that the OFC detects several forms of outcome uncertainty because it equates them to motivationally relevant contextual cues reflecting the need for control.
The amygdala is widely known to be associated to a series of emotional processes, and it has been famously associated to fear and anxiety (LeDoux, 1996), although it is now known that the amygdala is also involved in a wider array of affective processes including positive affect (Zald, 2003). A recent research trend has also found evidence that the amygdala plays an important role in cognitive control. For instance, Schaefer et al. (2006) have shown that increases in amygdala activity significantly predicted an improvement of performance in a canonical executive task, the N-back WM task, across two different experiments. More recent studies have found similar evidence of a relationship between the amygdala and cognitive control processes (Nishijo et al., 2008; Ousdal et al., 2008; Yun et al., 2010; Chiew and Braver, 2011), which adds to earlier evidence in both human and animal research (for a review, see Schaefer and Gray, 2007). A common theory to account for the role of the amygdala in cognitive control posits that the amygdala is responsible for evaluating changes in the motivational value (or goal-relevance) of the environment, and when these changes exceed a certain threshold, biasing signals are sent to cognitive and motor systems in order to facilitate adaptation (Davis and Whalen, 2001; Sander et al., 2003; Schaefer et al., 2006; Schaefer and Gray, 2007). It is thought that this mechanism can be involved in the implementation of cognitive control, but also in the generation of emotional responses (Schaefer et al., 2006).
As indicated by the findings of Hsu et al. (2005) described above, the amygdala also appears to be linked to the evaluation of contextual uncertainty. Similar conclusions have been reached in animal studies (for a review, see Rosen and Donley, 2006). In particular, Rosen and Donley (2006) suggest that the amygdala may be particularly linked to uncertain situations characterized by unexpected changes in the environment, which would relate to the concept of unexpected uncertainty (Yu and Dayan, 2005), and to the construct of rule volatility (Behrens et al., 2007). The amygdala is also activated under internally driven uncertainty, i.e., self reported levels of uncertainty (Zaretsky et al., 2010), which suggests that the amygdala is activated when sufficient data regarding a potential threat in the surroundings is lacking (Whalen et al., 1998; Zaretsky et al., 2010). For instance, Zaretsky et al. (2010) asked participants to report their perceived certainty regarding the level of danger associated to ambiguous face stimuli. They found that amygdala activity increases were associated with an increase in perceived uncertainty. Consistent with this idea, Davis and Whalen (2001) have suggested a model in which the amygdala is responsible for increasing levels of vigilance in response to uncertain stimuli or situations. This increase in vigilance levels would bias a series of control systems which would in turn facilitate general adaptation to the initial uncertain context.
The similarities between models of the role of the amygdala in uncertainty and models of the role of the amygdala in cognitive control are obvious. In both cases, fundamental changes in the goal-relevance of a situation are thought to trigger a response that signals the need to implement a controlled adaptive response. This mechanism shares similarities with the role often attributed to the ACC in cognitive control and reinforces the thesis that uncertainty plays a significant potential role in “triggering” the implementation of cognitive control.
Uncertainty and Cognitive Control
Whilst many of the aforementioned studies have shown overlaps between regions implicated in uncertainty processing and cognitive control, a recent study by Bhanji et al. (2010) attempted to investigate the dissociations between these two processes. In this study, the researchers manipulated task difficulty, assumed to index the recruitment of cognitive control resources and choice certainty. For each trial, participants were presented with a cue (containing numbers) and had to predict whether a feedback screen would include a number higher or lower than the cue. Outcomes ranged from small ($0.2) to large ($1) gains or losses for correct and incorrect answers respectively. In “Gambling” blocks, participants had to “guess” the value of the feedback. Uncertainty was manipulated through biasing the probabilities of positive/negative outcomes linked to specific number cues. Low choice certainty included cues of 2, 3, 8, and 9 whilst high certainty included cues 4, 5, 6, and 7. In “Rule” blocks, participants had to estimate the value of the feedback following previously learned rules that could be simple or complex. The authors found that the amygdala and the ventromedial prefrontal cortex were less active in choice uncertainty compared to choice certainty, and the insula was more involved in uncertainty compared to certainty. Next, the authors also found that the lateral PFC and ACC areas were related to rule complexity. While this study is a worthwhile attempt to differentiate uncertainty from processes related to cognitive control, some of its results can be explained by an alternative account. For instance, in this study, choice certainty was intrinsically linked with a higher frequency of trial-to-trial positive outcomes (monetary gains). Therefore, the involvement of the amygdala and VMPFC in choice certainty could well be the result of an anticipation of reward, as acknowledged by the authors.
In summary, available evidence from functional neuroimaging shows a strong overlap between the neural networks of uncertainty processing and cognitive control. Specifically, uncertain contexts appear to activate a typical “cognitive control network” including lateral PFC areas, parietal areas and the ACC. In addition, uncertainty is also linked to brain systems thought to be responsible for monitoring the need to implement top-down control (ACC and amygdala). A tentative explanation could posit that different forms of uncertainty share common features that monitoring systems (probably linked to the ACC and the amygdala) are programmed to detect and interpret as signals that top-down control needs to be implemented. Next, biasing signals could be sent to other systems more directly linked to the actual implementation of cognitive control processes (e.g., WM, inhibition, etc.). This explanation would be compatible with existing models of the dynamics of cognitive control and monitoring processes (Botvinick et al., 2001; Yeung and Cohen, 2006), as well as with current models of amygdala function (Davis and Whalen, 2001; Sander et al., 2003; Schaefer et al., 2006; Ousdal et al., 2008). In the next section, we discuss in more detail the potential links between uncertainty and monitoring processes.
Uncertainty and Monitoring
As suggested in the previous sections, uncertainty might play a role in a key component of cognitive control – monitoring processes. Monitoring usually refers to a set of processes that evaluate the need to implement or adjust top-down control, and this process is often defined within the scope of the specific monitoring of the outcomes of ongoing behavior (Botvinick et al., 2001; Kerns et al., 2004; Yeung and Cohen, 2006; Brown, 2009). In this section we review evidence supporting the idea of a relationship between decision-making under uncertainty and monitoring processes.
First, relevant evidence can be found in the field of complex dynamic control (CDC) tasks. CDC tasks are complex tasks often enacting realistic scenarios with frequent applications in ergonomics and human factors (e.g., flight and driving simulators, virtual markets, virtual problem solving such as forest fire-fight scenarios, etc.). Evidence from CDC tasks suggests that uncertainty is associated to an increase in the frequency of monitoring behaviors and in the amount of attention allocated to the monitoring of decision outcomes (Osman, 2010). For instance, Mosier et al. (2007) investigated the performance of airline pilots in a controlled flight simulator context. Several scenarios were used which required monitoring (visual search of decision feedback information in control panels) and control and execution (diagnosis and decision) processes. When participants were presented with incongruent or conflicting information, presented in an unpredictable manner (e.g., system failures), diagnosis took significantly longer, and pilots spent more time in monitoring behaviors than when situations and information were stable and information congruent. Similarly, Metzger and Parasuraman (2005) examined the effects of an automated system on performance of air traffic controllers in monitoring free-maneuvering aircraft across two contexts. In one context, participants were provided with a 100% reliable automated system to aid them in a conflict detection task and in the second they were provided an imperfect monitoring aid in which the reliability of the automated system was uncertain. Metzger and Parasuraman (2005) observed that under the uncertain system, traffic controllers had a greater likelihood of detecting conflicts, which can indicate that greater attentional resources were allocated to conflict monitoring when the context was uncertain. Consistent with this interpretation, Diez et al. (2001) measured eye tracking data from pilots during their interaction with a Boeing 747 flight simulator, and in particular during monitoring of decision outcomes (through events in the control panel). A distinction was made between information related to stable vs. uncertain aspects of the flight (for instance, altitude, position and speed are variable and uncertain). Results showed that eye fixations were longer for indicators of uncertain flight-related information compared to stable information. In addition, subsequent recall was higher for uncertain in comparison to stable information.
However, there might be an upper limit to the amount of uncertainty that can be handled by outcome monitoring (Xie and Guo, 2000). Sarter et al. (2007) measured behavioral and eye tracking data from pilots on a 1-h flight simulation under extremely challenging events, where participants experienced expected and unexpected changes in automation modes. In these highly uncertain environments, participant’s fixation rates on unexpected changes were low and completely failed to monitor these changes 43% of the time in comparison to a failure rate of 29% in expected automation changes. This uncertainty monitoring bias often comes with a cost, as increasing amounts of uncertainty make it difficult to effectively process relevant feedback (Atkins et al., 2002; Patrick and James, 2004; Bredereke and Lankenau, 2005; Gao and Lee, 2006; Mosier et al., 2007).
Second, evidence of a relationship between uncertainty and monitoring processes is also provided by research using Event-related potentials (ERPs). Outcome monitoring processes are investigated in this field mainly through two brain potentials: the ERN and the feedback-related negativity (FRN; Falkenstein et al., 2000; Pailing and Segalowitz, 2004; Hajcak et al., 2006; Sailer et al., 2010).
The ERN is an early frontocentral negativity time-locked to an incorrect response (Gehring et al., 1990; Falkenstein et al., 1991) and is believed to be generated in the ACC (Dehaene et al., 1994; Dikman and Allen, 2000; Gehring and Knight, 2000). Competing theories postulate that the ERN reflects the activity of a generic error detection (Dehaene et al., 1994) or a response conflict system (Carter et al., 1998). Whilst errors are thought to be processed differently to correct responses (Gehring and Knight, 2000), it has been suggested that uncertainty may cause errors to go undetected because of misrepresentation of the correct response (Pailing and Segalowitz, 2004) and thus may reduce the amplitude of the ERN (Falkenstein et al., 2000; Coles et al., 2001). In a study by Scheffers and Coles (2000), participants completed a two choice reaction time task under degraded stimulus conditions, designed to induce stimulus uncertainty. After response to each trial, participants were asked to rate their perceived accuracy on the trial. Results showed that errors due to premature responses (where errors were accurately identified as errors immediately after the response) had large ERN amplitudes; however, errors caused by stimulus uncertainty (errors where there was uncertainty about the error) had a reduction in ERN amplitude. Similar conclusions were reached by Pailing and Segalowitz (2004), who used letter and tone discrimination tasks in which uncertainty was manipulated by varying the amount of information provided by pre-stimulus cues. Results showed a significant decrease in the ERN amplitude for uncertain conditions. Importantly, this effect was enhanced in conditions in which attention was depleted by a concurrent task. These results suggest that under conditions of increased uncertainty about performance, errors, and correct trials are more likely to be judged or perceived in a similar manner.
The FRN is an early negativity time-locked to the delivery of feedback in decision-making paradigms. The FRN is typically larger following feedback signaling negative outcomes (e.g., “losses”) compared to positive outcomes (“wins”; Miltner et al., 1997; Gehring and Willoughby, 2002; Holroyd and Coles, 2002; Holroyd et al., 2003; Yeung and Sanfey, 2004; Nieuwenhuis et al., 2005; Hajcak et al., 2006). However, more recent studies have shown that feedback of positive valence can also elicit FRN (Yeung and Sanfey, 2004; Hajcak et al., 2005; Muller et al., 2005; Yeung et al., 2005). It has been suggested that the FRN reflects a general performance monitoring system that is activated by violations in expectancy (Nieuwenhuis et al., 2004; Oliveira et al., 2007).
The FRN has also been implicated in the processing of outcome certainty (Hajcak et al., 2005; Cohen et al., 2007). For instance, Cohen et al. (2007) used a probabilistic reinforcement task to investigate the effect of outcome probabilities on the FRN. In this study, uncertainty was operationalized through the manipulation of reward probabilities across blocks in a two choice decision task. High uncertainty in win trials (where reward probability was 25%) resulted in larger and more sustained FRN amplitude, in comparison to more certain (50 and 75% reward probability) positive outcomes. Similar results on FRN amplitude come from a recent study by Moser and Simons (2009). Participants performed a two choice gambling task in which they were asked to predict the outcome of their choice prior to selection and after selection. Here, FRN amplitude was highest when participants were uncertain of the outcome and changed their mind from an initial prediction of “lose” to post choice prediction of “win” (see also Sailer et al., 2010 for similar conclusions). In addition, the FRN may also be related to control processes, with research suggesting that this component reflects the activity of a reinforcement learning system which is used to adjust subsequent behavior (Hajcak et al., 2006; Cohen et al., 2007) and is also linked with attentional mechanisms (Moser and Simons, 2009) and WM demands (Suchan et al., 2005).
In summary, reviewed evidence from CDC tasks and electrophysiological research suggests strong links between uncertainty and monitoring processes: uncertainty increases the frequency of monitoring behaviors and the amount of attention allocated to outcome monitoring. Next, uncertainty also modulates brain potentials reflecting the processing of decision outcomes: uncertainty is associated to larger FRNs and reduced ERNs.
Intolerance of Uncertainty and Psychopathology
Intolerance of Uncertainty
It is widely acknowledged that psychiatric populations exhibit dysfunctional decision-making (Ladouceur et al., 2000). Interestingly, it is now becoming apparent that psychiatric disorders can also be linked with deficits in coping with uncertainty. It has been suggested that an inability to cope with uncertainty may even act as a driving force behind a number of behaviors and cognitions (e.g., worry, obsessions, compulsions, hypervigilance) associated with various anxiety disorders (Holaway et al., 2006). IOU is a construct defined as “a tendency to react negatively on an emotional, cognitive, and behavioral level to uncertain situations and events” (Heimberg et al., 2004, p. 143). IOU has been demonstrated in general anxiety disorder (GAD; Garber et al., 1980; Andrews and Borkovec, 1988; Dugas et al., 1998; Covin et al., 2008; Simmons et al., 2008; Boelen and Reijntjes, 2009), obsessive–compulsive disorder (OCD; Steketee et al., 1998), Schizophrenia (Broome et al., 2007; Dudley et al., 2011), and Eating Disorders (Konstantellou and Reynolds, 2010; Sternheim et al., 2011). Here, we will review the evidence that demonstrates an important role for IOU in a number of disorders known to be associated with deficits in cognitive control tasks.
General Anxiety Disorder
General anxiety disorder is a disorder characterized by excessive and persistent worrying (Dugas et al., 1998) and has been used to highlight the importance of uncertainty in psychiatric disorders (Paulus, 2007). IOU has been postulated to play a central role in anxiety (Krohne, 1989). For instance, measures of IOU can differentiate between clinical and healthy populations in GAD (Buhr and Dugas, 2002) and can predict changes in anxious symptoms (Dugas et al., 2009). Importantly, high worriers show a clear deficit in decision-making under uncertain situations with longer reaction times (see Ladouceur et al., 2000) and more information seeking behavior (Gibbs-Gallagher et al., 2003). Further, IOU has been associated with greater recall and threatening interpretations of ambiguous information (Dugas et al., 2005) and greater attempts to reduce uncertainty (Ladouceur et al., 2000). Worriers take longer to make category judgments than non-worriers, particularly when confronted with uncertain (ambiguous) stimuli (Metzger et al., 1990; Tallis et al., 1991). Raghunathan and Pham (1999) found that anxious decision makers preferred to choose high probability rewards linked to low monetary gains instead of more infrequent but much larger gains. This finding illustrates the tendency of anxious individuals to opt for safer and less uncertain choices, even at a significant cost.
Patients with psychosis, or at risk of psychosis, display a difficulty in tolerating uncertainty (e.g., Broome et al., 2007). Strauss et al. (2011) claim that reduced reward-seeking behavior in Schizophrenia is critically related to the extent to which patients make exploratory choices when they are uncertain. “Jumping to conclusions” (JTC), a frequent symptom in schizophrenia, has also been found to be associated with the construct of IOU (Freeman et al., 2008). Moritz et al. (2011) have suggested the idea that JTC in schizophrenic patients may contribute to well-being since quick decision-making decreases doubt and uncertainty. Indeed, JTC may be more evident in individuals who find it difficult to tolerate ambiguity (Colbert and Peters, 2002). This suggests a need to reduce uncertainty in schizophrenia, similar to that seen in other psychopathologies. Interestingly, a similar phenomenon is associated to IOU in GAD individuals. High IOU was indeed associated with faster decision-making in anxious individuals when subjects were faced with an outcome delay and thus waiting in a state of uncertainty (Luhmann et al., 2011). Further, an important theory of uncertainty processing (Yu and Dayan, 2005), has been linked to delusions and hallucinations associated with schizophrenia. Specifically, Patel et al. (2010) have suggested that hallucinations may be linked to neurotransmitters involved in the interaction between top-down and bottom-up processing where hallucinations could be experienced due to over-processing of top-down signals resulting in uncertainty in information processing.
One of the characteristics of OCD is pathological doubt, which is most clearly evident among patients with checking rituals (Rachman and Hodgson, 1980; Rasmussen and Eisen, 1992). Interestingly, uncertainty extends to long term memory of past events, as OCD individuals tend to be uncertain about their memory for checked events (Van Den Hout and Kindt, 2004). More directly, Tolin et al. (2003) found that repeating rituals were associated with IOU. IOU in OCD patients may reflect the belief on the part of OCD patients that they lack sufficient coping or problem solving skills to effectively manage threatening situations that have the potential to evoke discomfort and negative emotionality (Holaway et al., 2006). Many of the traits associated with OCD can be viewed as the products of control exerted within the emotional, interpersonal, cognitive, and behavioral domains (Gibbs-Gallagher et al., 2003). Difficulty in decision-making in OCD is characterized by a cautious approach to information categorization, frequent requests for information repetition, and doubts about the correctness of decisions (Obsessive Compulsive Cognitions Working Group, 1997).
Cognitive Control and Psychopathology
Importantly, patients with these psychopathological disorders not only show an IOU but also exhibit deficits in classical cognitive control tasks. For instance, OCD patients usually show an impaired performance on tests of executive function. Indeed, OCD patients show performance deficits on measures of response inhibition (Aycicegi et al., 2003). Particularly, OCD subjects exhibit higher interference costs in the Stroop task (Hartston and Swerdlow, 1999), making significantly more errors and slower reaction times on the interference trials of the Stroop test (Martinot et al., 1990; Bannon et al., 2002). It has been suggested that OCD subjects exhibit deficits in behavioral and cognitive inhibition (Enright and Beech, 1990; Rosenberg et al., 1997), which might underlie the repetitive symptomatic behaviors of the disorder, such as compulsions and obsessions (Bannon et al., 2002). Deficits among patients with OCD may also be associated with co-morbidity with depressive symptoms and schizotypal personality features (Aycicegi et al., 2003). Schizophrenia is also usually characterized by deficits in executive function (Liddle and Morris, 1991). For instance, schizophrenic patients perform worse in NoGo conditions in a Go/NoGo tasks suggesting a failure of response inhibition in schizophrenic patients (Weisbrod et al., 2000). Deficits in cognitive control have also been observed in patients with high anxiety (Wood et al., 2001). For instance, these deficits include dysfunctional response inhibition (Gernsbacher et al., 1990) with more attention paid to threatening distracters in the Stroop task (Mathews et al., 1990). High levels of anxiety have also been shown to limit WM capacity (Eysenck and Calvo, 1992).
Neural Processing of Uncertainty in Patients with Affective Disorders
Together, these results imply that disorders such as GAD, OCD, and Schizophrenia show deficits in both uncertainty processing and cognitive control. In addition, these abnormalities are coupled with dysfunctional neural processing. Indeed, hyperactivity of frontal–amygdala limbic regions suggests that greater IOU is associated with an elevated affective response to uncertainty (Krain et al., 2008). Here, we will review neurological evidence for deficits in neural substrates that are known to underlie cognitive control and uncertainty processing.
Schizophrenic patients have been shown to activate frontal areas insufficiently (Berman et al., 1992) with patients showing deficits in response inhibition that are coupled with disruption of the frontal P300 ERP amplitude, which may also indicate dysfunction of frontal neuronal circuits (Weisbrod et al., 2000). Interestingly, a fronto-parietal dysregulation has been observed in schizophrenic patients where the assessment of uncertainty is linked to decreased activation of the medial prefrontal cortex and an increased activation in the parietal cortex (Paulus et al., 2002). Further, schizophrenic patients show a deficit in reward-related probabilistic trial-and-error learning (Koch et al., 2010). This impairment was associated with the inability to reduce processing resources in association with increasing predictability, suggesting that patients exhibit a deficit in neural processing in response to uncertainty. Indeed, patients showed altered activation patterns in mainly frontal, cingulate, and striatal brain areas; regions known to be involved in uncertainty processing in healthy volunteers during feedback- or reward-based probabilistic learning (Fiorillo et al., 2003; Koch et al., 2008; Schlosser et al., 2009). These regions are also strongly implicated in decision-making, performance monitoring, and cognitive control (Volz et al., 2003; Zysset et al., 2006; Koch and Preuschoff, 2007; Koch et al., 2008).
General anxiety disorder subjects show increased activation in the ACC in response to infrequent errors, suggesting the propensity to be more engaged in error-related processing, which might then increase anticipation of adverse outcomes and contribute to fearfulness and avoidance of future conflict processing (Paulus et al., 2004). Indeed, hyperactivity in the ACC, which is chiefly involved in both uncertainty and cognitive control (see previous sections), may be a key feature of anxiety disorders (Paulus, 2007). Further, dysfunctional ACC activity has been observed during performance of an emotion-word Stroop task in anxious individuals (Engels et al., 2007) and with post-traumatic stress disorder (PTSD; Bremner et al., 2004; Etkin and Wager, 2007). Indeed, the degree of activity in the dorsal (“cognitive”) region of the ACC has been said to be a predictor of the level of self reported negative affect across individuals during reappraisal (Ochsner et al., 2002). These findings suggest that cognitive control regions play an important role in reappraisal and controlling emotional responses (Banich et al., 2009).
In addition, GAD patients high in IOU have shown less activation in fronto-median areas (Schienle et al., 2010), a region thought to be involved in uncertainty processing (Volz et al., 2003). Schienle et al. (2010) have suggested that this finding might reflect a potential deficit of systems devoted to cognitive coping and preparatory actions in individuals who suffer from IOU (Schienle et al., 2010).
Finally, OCD symptoms are said to be caused by abnormalities in fronto-striatal circuitry (Insel, 1992). Particularly, OCD is thought to be linked to a dysfunction in the circuits connecting the basal ganglia to the OFC, which would produce a number of cognitive and motor abnormalities (Wise et al., 1989; Alexander et al., 1990). The OFC has directly been implicated in patients with OCD (Rauch et al., 1994, 2002). As discussed previously, the OFC is known to be involved in certain aspects of cognitive control and it also appears to be involved in uncertainty processing (Hsu et al., 2005). It has been proposed that the deficit of OCD patients in the OFC might reflect an altered processing of reward history and valuation of options due to a relative disconnection between the dorsolateral, orbitofrontal, and anterior cingulate cortices with limbic regions (especially the amygdala) and with the basal ganglia (Holaway et al., 2006).
To date, relatively few studies have directly addressed the link between IOU and cognitive control in patient populations. However, a study by Broome et al. (2007) attempted to link IOU with deficits in cognitive control in individual at high risk of developing psychosis. In this study, patients were shown two jars of colored beads, informed of the relative proportions of beads in each and then told that they would be shown a series of beads drawn from one of the jars. They are then asked, on the basis of the observed sequence, to judge which jar is the source of the beads, and to be “as certain as possible.” The ability to hold information about bead color online was assessed using an adaptation of the digit span task that used a string of different colored beads. The authors found that IOU was negatively correlated with WM performance (assessed by the colored beads task) and positively correlated with JTC.
In summary, reviewed evidence indicates that individual differences in how we deal with uncertainty are linked to a series of psychopathological disorders known to be associated with deficits in cognitive control. Next, IOU seems to be linked to altered patterns of brain activity in systems involved in both uncertainty and cognitive control (Schienle et al., 2010). Finally, there is evidence that IOU might also be linked to an impairment in WM, a typical sub-process of cognitive control (Broome et al., 2007).
Synthesis and Conclusion
The evidence reviewed in this article shows that available research provides a wealth of both indirect and direct evidence of potential and even intrinsic relationships between uncertainty and cognitive control. First, functional neuroimaging studies investigating uncertainty have uncovered a neural network that has a remarkable overlap with brain networks usually associated with cognitive control tasks. In particular, a network involving lateral PFC areas, parietal cortex, and the ACC seems to be constantly activated for decision-making tasks in which outcome uncertainty is manipulated and also in a wide range of classical cognitive control tasks. Importantly, neuroimaging studies of uncertainty have also unveiled behavioral patterns that are consistent with the recruitment of controlled processes, such as an increase of response time in uncertain conditions (an effect that had already been shown in a vast body of behavioral research, see for instance Bertelson and Boons, 1960).
Second, a number of behavioral and electrophysiological studies indicates a strong relationship between uncertainty and a key component of cognitive control – outcome monitoring. In particular, it appears that highly uncertain environments tend to increase the recruitment of monitoring processes. In addition, studies on the ERN and FRN brain potentials suggests that systems devoted to processing the valence of decision outcomes are hyper-activated during uncertainty, and that correct and incorrect decisions are more difficult to discriminate in such contexts. This pattern of effects could indicate that uncertainty is a state that triggers an enhanced activity of systems dedicated to the monitoring of the outcomes of our actions. A tentative explanation for these effects could be that when there is a realization that the outcomes of our decisions are difficult to predict, an enhanced processing of outcomes might be an adaptive solution. This strategy can potentially facilitate the accurate detection of changes in the underlying rules in the environment and lead to a more successful adaptation. Interestingly, monitoring processes are often seen as a trigger for the implementation of cognitive control processes, by providing a mechanism that evaluates the “need for control” of a given situation. In this sense, the construct of uncertainty could be seen as a summary of the contextual conditions that are the causal antecedents of the implementation of cognitive control, rather than a set of processes that would be independent from cognitive control. This explanation would be consistent with the fact that many of the overlapping activations in neuroimaging studies from the field of cognitive control and the field of decision-making under uncertainty, point toward a network of brain areas thought to be involved in the detection of contextual cues signaling the need for control (ACC, OFC, and amygdala), and in the utilization of this information to bias behavior (see the section on functional neuroimaging). Future studies will be needed to explicitly examine whether and how manipulating levels of outcome uncertainty can trigger the implementation of cognitive control.
Third, although there is evidence that uncertainty is strongly linked to the monitoring stage of cognitive control, only indirect published evidence exists of the potential role of cognitive control in the resolution of uncertainty in post-monitoring stages. The involvement of lateral PFC areas in conditions of increasing outcome uncertainty may suggest that systems devoted to the implementation of cognitive control are used to resolve uncertainty, as reviewed in the neuroimaging section. This idea is also supported by behavioral patterns observed during uncertainty that are consistent with the recruitment of cognitive control (such as increased RT during high uncertainty). Overall, the data and models reviewed in this article suggest a two-way transaction between outcome uncertainty and cognitive control: outcome uncertainty can be seen as a state that triggers the implementation of cognitive control and in turn one of the main functions of cognitive control processes would be to facilitate learning about uncertain S–R–O mappings. It is important to stress that this possibility does not rule out that implicit learning processes also play a role in the adaptation to uncertain environments. Further research will be needed to investigate the extent to which cognitive control contributes to the resolution of uncertainty above and beyond implicit learning processes. For instance, future research should focus on paradigms in which uncertain decision-making is combined with concurrent tasks tapping cognitive control (e.g., Bhanji et al., 2010, provide an example of such paradigms).
Fourth, an emerging literature in psychopathology indicates that individual differences in IOU are linked to a series of affective disorders known to be associated with deficits in cognitive control, such as OCD and GAD. In addition, some evidence (Broome et al., 2007) indicates that IOU is also correlated with lower performance in cognitive control tasks. This body of evidence tentatively suggests that a number of affective disorders may be characterized by an inability to adapt to uncertainty, and that this deficit might be linked to lower efficiency in cognitive control function. However, further research will be needed to characterize more precisely the potential causal links between uncertainty, cognitive control and affective disorders. In addition, an interesting potential area of development in the field of IOU could be the development of cognitive training techniques targeted at decision-making under uncertainty. Cognitive and attentional training techniques seem to be providing promising results in clinical populations (See et al., 2009). Testing training schedules for clinical populations aimed at using cognitive control in order to cope more effectively with uncertainty might inform us about the potential causal links between IOU, uncertainty and cognitive control.
Finally, it has to be noted that the fundamental distinction between expected and unexpected uncertainty does not appear to have led to systematic differences in brain activity. Paradigms tapping these two concepts (e.g., volatility studies for unexpected uncertainty and reward probability manipulations/ambiguity studies for expected uncertainty) have been associated to approximately similar brain networks in functional imaging studies. Future research will be necessary to refine the techniques used to investigate the neural correlates of these two types of uncertainty, and also to specify further the taxonomy of different forms of uncertainty.
Conflict of Interest Statement
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Alexandre Schaefer is supported by the UK Biotechnology and Biological Sciences Research Council (BBSRC) and Amy R. Bland is supported by the Economic and Social Research Council (ESRC).
- ^This form of uncertainty can be referred to as “outcome uncertainty,” which has to be differentiated from the concept of “task uncertainty,” which refers to uncertainty about the nature of the task to be performed in the next trial in multi-trial task-switching paradigms (e.g., Rubin and Meiran, 2005). The present review focuses essentially on outcome uncertainty.
- ^However, there is an ongoing debate on whether cognitive control is necessarily conscious (e.g., Sumner and Husain, 2008; Van Gaal et al., 2010, 2011).
Alexander, G. E., Crutcher, M. D., and Delong, M. R. (1990). Basal ganglia-thalamocortical circuits: parallel substrates for motor, oculomotor, prefrontal and limbic functions. Prog. Brain Res. 85, 119–146.
Atkinson, R. C., and Shiffrin, R. M. (1968). “Human memory: a proposed system and its control processes,” in The Psychology of Learning and Motivation, eds K. W. Spence, and J. T. Spence (New York: Academic Press) 2, 89–195.
Aycicegi, A., Dinn, W. M., Harris, C. L., and Erkmen, H. (2003). Neuropsychological function in obsessive-compulsive disorder: effects of comorbid conditions on task performance. Eur. Psychiatry 18, 241–248.
Banich, M. T., Mackiewicz, K. L., Depue, B. E., Whitmer, A. J., Miller, G. A., and Heller, W. (2009). Cognitive control mechanisms, emotion and memory: a neural perspective with implications for psychopathology. Neurosci. Biobehav. Rev. 33, 613–630.
Berman, K. F., Torrey, E. F., Daniel, D. G., and Weinberger, D. R. (1992). Regional cerebral blood flow in monozygotic twins discordant and concordant for schizophrenia. Arch. Gen. Psychiatry 49, 927–934.
Bhanji, J. P., Beer, J. S., and Bunge, S. A. (2010). Taking a gamble or playing by the rules: dissociable prefrontal systems implicated in probabilistic versus deterministic rule-based decisions. Neuroimage 49, 1810–1819.
Bracht, T., Tüscher, O., Schnell, S., Kreher, B., Rüsch, N., Glauche, V., Lieb, K., Ebert, D., Il’yasov, K. A., Hennig, J., Weiller, C., Van Elst, L. T., and Saur, D. (2009). Extraction of prefronto-amygdalar pathways by combining probability maps. Psychiatry Res. 174, 217–222.
Braver, T. S., Gray, J. R., and Burgess, G. C. (2007). “Explaining the many varieties of working memory variation: dual mechanisms of cognitive control,” in Variation in Working Memory, eds A. R. A. Conway, C. Jarrold, M. J. Kane, A. Miyake, and J. N. Towse (Oxford: Oxford University Press), 76–106.
Bremner, J. D., Vermetten, E., Vythilingam, M., Afzal, N., Schmahl, C., Elzinga, B., and Charney, D. S. (2004). Neural correlates of the classic color and emotional stroop in women with abuse-related posttraumatic stress disorder. Biol. Psychiatry 55, 612–620.
Broome, M. R., Johns, L. C., Valli, I., Woolley, J. B., Tabraham, P., Brett, C., Valmaggia, L., Peters, E., Garety, P. A., and Mcguire, P. K. (2007). Delusion formation and reasoning biases in those at clinical high risk for psychosis. Br. J. Psychiatry Suppl. 51, s38–s42.
Carter, C. S., Braver, T. S., Barch, D. M., Botvinick, M. M., Noll, D., and Cohen, J. D. (1998). Anterior cingulate cortex, error detection, and the online monitoring of performance. Science 280, 747–749.
Coles, M. G. H., Scheffers, M. K., and Holroyd, C. B. (2001). Why is there an ERN/Ne on correct trials? Response representations, stimulus-related components, and the theory of error-processing. Biol. Psychol. 56, 173–189.
Debener, S., Ullsperger, M., Siegel, M., Fiehler, K., Von Cramon, D. Y., and Engel, A. K. (2005). Trial-by-trial coupling of concurrent electroencephalogram and functional magnetic resonance imaging identifies the dynamics of performance monitoring. J. Neurosci. 25, 11730–11737.
Diez, M., Boehm-Davis, D. A., Holt, R. W., Pinney, M. E., Hansberger, J. T., and Schoppek, W. (2001). “Tracking pilot interactions with flight management systems through eye movements,” in Proceedings of the 11th International Symposium on Aviation Psychology (Columbus: Ohio State University).
Dudley, K. J., Li, X., Kobor, M. S., Kippin, T. E., and Bredy, T. W. (2011). Epigenetic mechanisms mediating vulnerability and resilience to psychiatric disorders. Neurosci. Biobehav. Rev. 35, 1544–1551.
Dugas, M., Francis, K., and Bouchard, S. (2009). Cognitive behavioural therapy and applied relaxation for generalized anxiety disorder: a time series analysis of change in worry and somatic anxiety. Cogn. Behav. Ther. 38, 29–41.
Dugas, M. J., Marchand, A., and Ladouceur, R. (2005). Further validation of a cognitive-behavioral model of generalized anxiety disorder: diagnostic and symptom specificity. J. Anxiety Disord. 19, 329–343.
Durston, S., Davidson, M. C., Thomas, K. M., Worden, M. S., Tottenham, N., Martinez, A., Watts, R., Ulug, A. M., and Casey, B. J. (2003). Parametric manipulation of conflict and response competition using rapid mixed-trial event-related fMRI. Neuroimage 20, 2135–2141.
Engels, A. S., Heller, W., Mohanty, A., Herrington, J. D., Banich, M. T., Webb, A. G., and Miller, G. A. (2007). Specificity of regional brain activity in anxiety types during emotion processing. Psychophysiology 44, 352–363.
Etkin, A., and Wager, T. D. (2007). Functional neuroimaging of anxiety: a meta-analysis of emotional processing in ptsd, social anxiety disorder, and specific phobia. Am. J. Psychiatry 164, 1476–1488.
Falkenstein, M., Hohnsbein, J., Hoormann, J., and Blanke, L. (1991). Effects of crossmodal divided attention on late ERP components. II. Error processing in choice reaction tasks. Electroencephalogr. Clin. Neurophysiol. 78, 447–455.
Garber, J., Miller, S. M., and Abramson, L. Y. (1980). “On the distinction between anxiety states and depression: perceived control, certainty, and probability of goal attainment,” in Human Helplessness: Theory and Applications, eds J. Garber, and M. E. P. Seligman (New York: Academic Press), 131–172.
Gibbs-Gallagher, N., South, S. C., and Oltmanns, T. F. (2003). Attentional coping style in obsessive-compulsive personality disorder: a test of the intolerance of uncertainty hypothesis. Pers. Individ. Dif. 34, 41–57.
Hadland, K. A., Rushworth, M. F. S., Passingham, R. E., Jahanshahi, M., and Rothwell, J. C. (2001). Interference with performance of a response selection task that has no working memory component: an rTMS comparison of the dorsolateral prefrontal and medial frontal cortex. J. Cogn. Neurosci. 13, 1097–1108.
Harvey, P. O., Fossati, P., Pochon, J. B., Levy, R., Lebastard, G., Lehéricy, S., Allilaire, J. F., and Dubois, B. (2005). Cognitive control and brain resources in major depression: an fMRI study using the n-back task. Neuroimage 26, 860–869.
Holaway, R. M., Heimberg, R. G., and Coles, M. E. (2006). A comparison of intolerance of uncertainty in analogue obsessive-compulsive disorder and generalized anxiety disorder. J. Anxiety Disord. 20, 158–174.
Koch, K., Schachtzabel, C., Wagner, G., Reichenbach, J. R. R., Sauer, H., and Schlosser, R. (2008). The neural correlates of reward-related trial-and-error learning: an fMRI study with a probabilistic learning task. Learn. Mem. 15, 728–732.
Krain, A. L., Gotimer, K., Hefton, S., Ernst, M., Castellanos, F. X., Pine, D. S., and Milham, M. P. (2008). A functional magnetic resonance imaging investigation of uncertainty in adolescents with anxiety disorders. Biol. Psychiatry 63, 563–568.
MacDonald, A. W., Cohen, J. D., Stenger, V. A., and Carter, C. S. (2000). Dissociating the role of the dorsolateral prefrontal and anterior cingulate cortex in cognitive control. Science 288, 1835–1838.
Martinot, J., Hardy, P., Feline, A., Huret, J., Mazoyer, B., Attar-Levy, D., Pappata, S., and Syrota, A. (1990). Left prefrontal glucose hypometabolism in the depressed state: a confirmation. Am. J. Psychiatry 147, 1313–1317.
Milham, M. P., Banich, M. T., Claus, E. D., and Cohen, N. J. (2003). Practice-related effects demonstrate complementary roles of anterior cingulate and prefrontal cortices in attentional control. Neuroimage 18, 483–493.
Miltner, W. H. R., Braun, C. H., and Coles, M. G. H. (1997). Event-related brain potentials following incorrect feedback in a time-estimation task: evidence for a neural system for error detection. J. Cogn. Neurosci. 9, 788–798.
Moritz, S., Kerstan, A., Veckenstedt, R., Randjbar, S., Vitzthum, F., Schmidt, C., Heise, M., and Woodward, T. S. (2011). Further evidence for the efficacy of a metacognitive group training in schizophrenia. Behav. Res. Ther. 49, 151–157.
Müller, S. V., Möller, J., Rodriguez-Fornells, A., and Münte, T. F. (2005). Brain potentials related to self-generated and external information used for performance monitoring. Clin. Neurophysiol. 116, 63–74.
Nassar, M. R., Wilson, R. C., Heasly, B., and Gold, J. I. (2010). An approximately bayesian delta-rule model explains the dynamics of belief updating in a changing environment. J. Neurosci. 30, 12366–12378.
Nieuwenhuis, S., Heslenfeld, D. J., Von Geusau, N. J., Mars, R. B., Holroyd, C. B., and Yeung, N. (2005). Activity in human reward-sensitive brain areas is strongly context dependent. Neuroimage 25, 1302–1309.
Norman, D., and Shallice, T. (1986). “Attention to action: willed and automatic control of behavior,” in Consciousness and Self-Regulation: Advances in Research and Theory IV, eds R. Davidson, R. Schwartz, and D. Shapiro (New York, NY: Plenum Press) 1–18.
O’Doherty, J., Critchley, H., Deichmann, R., and Dolan, R. J. (2003). Dissociating valence of outcome from behavioral control in human orbital and ventral prefrontal cortices. J. Neurosci. 23, 7931–7939.
Oliveira, F. T. P., Mcdonald, J. J., and Goodman, D. (2007). Performance monitoring in the anterior cingulate is not all error related: expectancy deviation and the representation of action-outcome associations. J. Cogn. Neurosci. 19, 1994–2004.
Ousdal, O. T., Jensen, J., Server, A., Hariri, A. R., Nakstad, P. H., and Andreassen, O. A. (2008). The human amygdala is involved in general behavioral relevance detection: evidence from an event-related functional magnetic resonance imaging Go-NoGo task. Neuroscience 156, 450–455.
Pardo, J. V., Pardo, P. J., Janer, K. W., and Raichle, M. E. (1990). The anterior cingulate cortex mediates processing selection in the Stroop attentional conflict paradigm. Proc. Natl. Acad. Sci. U.S.A. 87, 256–259.
Patel, S., Attard, A., Jacobsen, P., and Shergill, S. (2010). Acetylcholinesterase inhibitors (AChEI’s) for the treatment of visual hallucinations in schizophrenia: a case report. BMC Psychiatry 10, 68. doi: 10.1186/1471-244X-10-68
Paulus, M. P., Feinstein, J. S., Tapert, S. F., and Liu, T. T. (2004). Trend detection via temporal difference model predicts inferior prefrontal cortex activation during acquisition of advantageous action selection. Neuroimage 21, 733–743.
Paulus, M. P., Hozack, N., Frank, L., and Brown, G. G. (2002). Error rate and outcome predictability affect neural activation in prefrontal cortex and anterior cingulate during decision-making. Neuroimage 15, 836–846.
Rauch, S. L., Shin, L. M., Dougherty, D. D., Alpert, N. M., Fischman, A. J., and Jenike, M. A. (2002). Predictors of fluvoxamine response in contamination-related obsessive compulsive disorder: a PET symptom provocation study. Neuropsychopharmacology 27, 782–791.
Rosenberg, D. R., Keshavan, M. S., O’Hearn, K. M., Dick, E. L., Bagwell, W. W., Seymour, A. B., Montrose, D. M., Pierri, J. N., and Birmaher, B. (1997). Frontostriatal measurement in treatment-naive children with obsessive-compulsive disorder. Arch. Gen. Psychiatry 54, 824–830.
Sarter, N. B., Mumaw, R. J., and Wickens, C. D. (2007). Pilots’ monitoring strategies and performance on automated flight decks: an empirical study combining behavioral and eye-tracking data. Hum. Factors 49, 347–357.
Schaefer, A., Braver, T. S., Reynolds, J. R., Burgess, G. C., Yarkoni, T., and Gray, J. R. (2006). Individual differences in amygdala activity predict response speed during working memory. J. Neurosci. 26, 10120–10128.
Scheffers, M. K., and Coles, M. G. (2000). Performance monitoring in a confusing world: error-related brain activity, judgments of response accuracy, and types of errors. J. Exp. Psychol. Hum. Percept. Perform. 26, 141–151.
Schlosser, R. G., Nenadic, I., Wagner, G., Zysset, S., Koch, K., and Sauer, H. (2009). Dopaminergic modulation of brain systems subserving decision making under uncertainty: a study with fMRI and methylphenidate challenge. Synapse 63, 429–442.
Schumacher, E. H., and D’Esposito, M. (2002). Neural implementation of response selection in humans as revealed by localized effects of stimulus–response compatibility on brain activation. Hum. Brain Mapp. 17, 193–201.
See, J., MacLeod, C., and Bridle, R. (2009). The reduction of anxiety vulnerability through the modification of attentional bias: a real-world study using a home-based cognitive bias modification procedure. J. Abnorm. Psychol. 118, 65–75.
Sohn, M. H., Ursu, S., Anderson, J. R., Stenger, V. A., and Carter, C. S. (2000). The role of prefrontal cortex and posterior parietal cortex in task switching. Proc. Natl. Acad. Sci. U.S.A. 97, 13448–13453.
Sternheim, L., Startup, H., and Schmidt, U. (2011). An experimental exploration of behavioral and cognitive-emotional aspects of intolerance of uncertainty in eating disorder patients. J. Anxiety Disord. 25, 806–812.
Strauss, G. P., Frank, M. J., Waltz, J. A., Kasanova, Z., Herbener, E. S., and Gold, J. M. (2011). Deficits in positive reinforcement learning and uncertainty-driven exploration are associated with distinct aspects of negative symptoms in schizophrenia. Biol. Psychiatry 69, 424–431.
Ullsperger, M., and Von Cramon, D. Y. (2003). Error monitoring using external feedback: specific roles of the habenular complex, the reward system, and the cingulate motor area revealed by functional magnetic resonance imaging. J. Neurosci. 23, 4308–4314.
Van Gaal, S., Scholte, H. S., Lamme, V. A. F., Fahrenfort, J. J., and Ridderinkhof, K. R. (2011). Pre-SMA gray-matter density predicts individual differences in action selection in the face of conscious and unconscious response conflict. J. Cogn. Neurosci. 23, 382–390.
Weickert, T. W., Terrazas, A., Bigelow, L. B., Malley, J. D., Hyde, T., Egan, M. F., Weinberger, D. R., and Goldberg, T. E. (2002). Habit and skill learning in schizophrenia: evidence of normal striatal processing with abnormal cortical input. Learn. Mem. 9, 430–442.
Whalen, P. J., Rauch, S. L., Etcoff, N. L., Mcinerney, S. C., Lee, M. B., and Jenike, M. A. (1998). Masked Presentations of emotional facial expressions modulate amygdala activity without explicit knowledge. J. Neurosci. 18, 411–418.
Zaretsky, M., Mendelsohn, A., Mintz, M., and Hendler, T. (2010). In the eye of the beholder: internally driven uncertainty of danger recruits the amygdala and dorsomedial prefrontal cortex. J. Cogn. Neurosci. 22, 2263–2275.
Keywords: uncertainty, cognitive control, executive function, emotion, volatility, ambiguity, monitoring, motivation
Citation: Mushtaq F, Bland AR and Schaefer A (2011) Uncertainty and cognitive control. Front. Psychology 2:249. doi: 10.3389/fpsyg.2011.00249
Received: 01 July 2011; Accepted: 11 September 2011;
Published online: 03 October 2011.
Edited by:Gilles Pourtois, University of Ghent, Belgium
Reviewed by:Birgit Stürmer, Humboldt Universität Berlin, Germany
Ruth M. Krebs, Duke University, USA
Copyright: © 2011 Mushtaq, Bland and Schaefer. This is an open-access article subject to a non-exclusive license between the authors and Frontiers Media SA, which permits use, distribution and reproduction in other forums, provided the original authors and source are credited and other Frontiers conditions are complied with.
*Correspondence: Alexandre Schaefer, Wolfson Research Institute, Durham University, Queen’s Campus, Stockton-on-Tees TS17 6BH, UK. e-mail: firstname.lastname@example.org