Abstract
The ability to learn contingencies between actions and outcomes in a dynamic environment is critical for flexible, adaptive behavior. Goal-directed actions adapt to changes in action-outcome contingencies as well as to changes in the reward-value of the outcome. When networks involved in reward processing and contingency learning are maladaptive, this fundamental ability can be lost, with detrimental consequences for decision-making. Impaired decision-making is a core feature in a number of psychiatric disorders, ranging from depression to schizophrenia. The argument can be developed, therefore, that seemingly disparate symptoms across psychiatric disorders can be explained by dysfunction within common decision-making circuitry. From this perspective, gaining a better understanding of the neural processes involved in goal-directed action, will allow a comparison of deficits observed across traditional diagnostic boundaries within a unified theoretical framework. This review describes the key processes and neural circuits involved in goal-directed decision-making using evidence from animal studies and human neuroimaging. Select studies are discussed to outline what we currently know about causal judgments regarding actions and their consequences, action-related reward evaluation, and, most importantly, how these processes are integrated in goal-directed learning and performance. Finally, we look at how adaptive decision-making is impaired across a range of psychiatric disorders and how deepening our understanding of this circuitry may offer insights into phenotypes and more targeted interventions.
Goal-directed action and its relevance to psychiatry
Flexible behavior is fundamental for adapting to a changing environment. In this context, learning the consequences of an action and the value of those consequences are critical precursors for choosing the best course of action. Impairment in either process, or a failure to integrate them with action selection, leads to aberrant decision-making, with detrimental consequences for achieving goals and real-world functioning. Dysfunctional decision-making is common across a range of psychiatric disorders, and indeed, it has been argued that many psychiatric symptoms are associated with dysfunction in either learning or reward circuitry (cf. Nestler and Carlezon, 2006; Martin-Soelch et al., 2007). Determining how the brain supports each step in achieving flexible, goal-directed behavior is, therefore, not only a major goal of decision neuroscience, but may also provide valuable insight into the neurobiology and attendant functional disabilities associated with psychiatric illness.
Decades of research in associative learning have provided key insights into the behavioral and biological processes that mediate goal-directed action. One advantage of this approach has been the development of testable structural and functional hypotheses, and the invention of critical behavioral paradigms specifically to assess predictions from these hypotheses. We argue that this approach provides a unique opportunity to systemically explore the decision-making deficits commonly observed in clinical populations, and allows for the classification of a variety of decision-making impairments within a common framework. In this review, we first describe the psychological determinants of goal-directed behavior, and the evidence for how these processes map onto specific neural circuits. We will then use this framework to assess how these processes may be affected in common symptoms within three clinical disorders: schizophrenia, attention-deficit hyperactivity disorder (ADHD), and depression. Behavioral and neurobiological heterogeneities exist within traditional disorder classifications, as well as commonalities across diagnostic boundaries. We argue that knowledge of specific decision-making processes and their neural bases may provide a unifying framework, using which we can classify deficits across psychiatric disorders to produce a functionally–and biologically -driven understanding of psychopathology.
What is goal-directed action?
Formally, goal-directed action reflects the integration of two sources of information: (1) knowledge of the causal consequences or outcome of an action; and (2) the value of the outcome (Dickinson and Balleine, 1994; Balleine and Dickinson, 1998). The integration of both of these features, causal knowledge and reward value, is essential in producing goal-directed actions. Impairments in such actions can arise through a deficiency in either process, or through an inability to integrate them appropriately to guide decision-making. We will first discuss each of these features in turn and the key neural substrates that current research suggests are involved in these processes. We will then turn to potential deficits in these processes using examples of specific psychopathology, and in particular, how they are related to symptoms common to depression, schizophrenia and ADHD.
Causal learning and action-outcome encoding
Knowledge regarding the causal consequences of specific actions emerges from the experienced contingency. Such contingencies can be positive, promoting performance of an action, or inhibitory; i.e., in some situations actions may prevent a desired outcome and, in these situation, actions should be withheld (Dickinson, 1994). Considerable research using tasks such as the Iowa Gambling Task (IGT; Bechara et al., 1994) and the Wisconsin Card Sorting Task (WCST; Grant and Berg, 1948) suggests that humans and rats are exquisitely sensitive to feedback contingent on their actions, and can flexibly update their choices based on that feedback. However, because specific choice problems are signaled using unique discriminative or localized cues in these tasks, choice performance could reflect knowledge of the action-outcome contingency or associations between the action or the outcome with these task-related cues. This is a non-trivial distinction; as we shall review below, research has shown that different psychological processes and neural circuits exert control when actions are guided by environmental stimuli or by the action-outcome contingency (see Balleine and Ostlund, 2007; Balleine and O’Doherty, 2010 for reviews).
Experimentally, we are able to determine the degree to which choice is guided by the action-outcome contingency using contingency degradation tests. In such tests a specific action-outcome contingency is degraded by introducing an outcome in the absence of its associated action, thereby reducing the causal relationship between them. This treatment decreases the performance of the degraded action in goal-directed agents (Hammond, 1980; Balleine and Dickinson, 1998). For example, Balleine and Dickinson trained rats to perform two actions, lever pressing and chain pulling, with one action earning sucrose and the other, food pellets. They subsequently delivered one of the two outcomes non-contingently, such that the probability of receiving that outcome was the same whether the rat performed its associated action or not. This produced a selective decrease in the performance of the degraded action. Similarly, it has been demonstrated in healthy humans that the degree of contingency degradation is negatively correlated with the rate of performance and with judgments regarding how causal an action is with respect to its outcome (Shanks and Dickinson, 1991; Liljeholm et al., 2011).
A specific corticostriatal circuit mediates the causal effects of actions
Systematic use of contingency degradation tasks in rodent studies has identified specific regions of prefrontal cortex and dorsomedial striatum necessary for encoding the action-outcome contingency (Corbit and Balleine, 2003; Yin et al., 2005; Lex and Hauber, 2010). In humans, there is evidence that homologous regions to those in rodents, i.e., the medial prefrontal cortex (mPFC) and anterior caudate, play a similar role in contingency sensitivity (cf. Balleine and O’Doherty, 2010). Tanaka et al. (2008) and Liljeholm et al. (2011) manipulated experienced action-outcome contingencies, and observed positive modulation of blood oxygenation level dependent (BOLD) activity in the human mPFC, and anterior caudate nucleus (aCN). Furthermore, mPFC activity reflected the local experienced correlation between responding and reward delivery, consistent with a role in the online computation of contingency (Tanaka et al., 2008). Activation of the aCN can also occur even fictively, in cases where a contingency between action and outcome is perceived where one does not actually exist (Tricomi et al., 2004), whereas subjective causality judgments have been shown to correlate with activity in the mPFC, along with the dorsolateral prefrontal cortex (dlPFC), a region implicated in top-down cognitive control (Tanaka et al., 2008). As shown in the green in Figure 1A, these data suggest that signals produced in the mPFC may be relayed to the aCN, where changes in contingency can be assimilated with evaluative information from other cortical regions.
Figure 1
Further evidence for the importance of the caudate in contingency sensitivity and in guiding action selection comes from studies in non-human primates. Samejima et al. (2005) recorded from striatal neurons during a choice task in which monkeys made left or right actions to obtain reward. Importantly, on some trials, action-outcome contingencies were similar whereas on others they differed so that activity related to the action value (in this instance, the strength of the action-outcome contingency) could be dissociated from the motor choice. They found that a large number of striatal neurons encoded action values, which subsequently influenced the probability of selecting a particular action. Lau and Glimcher (2007) also found populations of neurons in the caudate that encoded actions and outcome post-choice. The temporal correlation of neuronal firing rates with behavior suggested that the caudate not only represents the contingency of potential options, but might also update this information once the outcome has been received.
The role of value in goal-directed decision-making
In addition to causal knowledge, determining the current value of available outcomes in the context of current internal states or contexts is also critical for adaptive decision-making. For example, a state of hunger increases the desirability or incentive value of food relative to a satiated state, and increases its motivational impact. Outcome revaluation procedures exploit these variations in value. A common means of changing the value of a specific food is using sensory-specific satiety (Rolls et al., 1981). For example, in studies in which rats were trained to perform two actions for distinct outcomes, giving them an extended opportunity to eat one or other outcome altered the desirability of that outcome without affecting the value of the other uneaten outcome (Balleine and Dickinson, 1998). When given the opportunity to choose between the two actions in the absence of any reward delivery (to prevent learning about the association between the action and the new outcome value during the test) the rats clearly preferred the action that had previously earned the outcome they had not eaten. Selective decreases in the performance of actions associated with a devalued outcome provide clear evidence that, in conjunction with knowledge of the action-outcome contingency, action selection is governed by the current value of the outcome.
An alternate means of revaluing the outcome used in animal research is conditioned taste aversion whereby an outcome is paired with a mild toxin such as lithium chloride that induces gastric malaise. In humans disgust can also be a useful tool for devaluing outcomes. For instance, food desirability ratings can be decreased considerably when an otherwise preferred outcome has been paired with an aversive taste (e.g., Baeyens et al., 1990).
The OFC and vmPFC play a role in encoding value relative to the current motivational state
The OFC and, more broadly, the vmPFC, illustrated in red in Figure 1A, have long been argued to be critical for signaling the current value of an outcome. Single unit recording studies in hungry non-human primates found unit responses in the caudolateral OFC during presentation of a pleasant odor or taste, which decreased to baseline when the monkey were satiated (Rolls et al., 1989). Similarly, when humans were presented with food outcomes, the degree of hunger and pleasantness caused graded OFC/vmPFC BOLD activity (Morris and Dolan, 2001; Kringelbach et al., 2003) that was reduced after satiation with the presented food (O’doherty et al., 2000; Small et al., 2001; Valentin et al., 2007). Interestingly, this reduction in activity was evident even when using instructed devaluation, where participants were simply told via a red X over a predictive stimulus that the outcome was no longer valuable (de Wit et al., 2009) suggesting that revaluation, whether through visceral or cognitive treatments, affects value via a common neural pathway. These data advance the idea that the OFC undertakes simple economic valuation and emphasize its role in determining outcome value in the context of the current motivational state. Jones et al. (2012) have further developed this idea, arguing that the OFC is required when value is inferred from associative structures (i.e., value is computed based on the current state), but not when relying on pre-computed values stored from previous experience.
It is important to note that BOLD activation during evaluation has been reported within both the lateral and medial portions of the OFC. There is, however, evidence for cytoarchitectural and functional heterogeneity within the OFC (Carmichael and Price, 1995; Elliott et al., 2000; Kahnt et al., 2012), suggesting that studies using reward-predictive cues are utilizing alternate or additional learning processes. Though there is still considerable debate on this topic, a converging view is that the mOFC is involved in updating the expected values of different experienced outcomes, whereas the lateral OFC is responsible for the formation and updating of values derived from Pavlovian stimulus-outcome associations (Walton et al., 2010; cf Balleine et al., 2011; Fellows, 2011; Noonan et al., 2011, 2012; Rudebeck and Murray, 2011; Klein-Flügge et al., 2013). Both the predicted value of an outcome based on the presence of a Pavlovian cue, and the experienced value of an instrumental outcome, are incentive processes that play an important role in motivating behavior. Due to the differing circuitry and learning processes (instrumental vs. Pavlovian) however, paradigms that disentangle these processes provide clearer information.
The influence of a limbic cortico-striatal circuit on the value of outcomes and cues that predict outcome delivery
Whereas the mOFC is computing current outcome value, the basolateral amygdala (BLA) plays a more fundamental role, linking value information with the sensory features of the reward or reward-related cues (see Figure 1A). A series of studies by Balleine et al. (2003) found that lesions of the BLA attenuated the sensitivity of rats to outcome devaluation, both when tested in extinction and with the outcome present. Furthermore, BLA lesions have been found to abolish the selective excitatory effects of reward-related cues whilst sparing the general motivational effects that such cues exert over responding (Corbit and Balleine, 2005). In humans, Jenison et al. (2011) acquired single neuron recordings from the BLA whilst subjects made monetary bids on food items that were presented to them as pictorial stimuli. Firing rates were linearly related to the monetary value assigned to food item stimuli, supporting a role for the BLA in assigning value to stimulus events. The strength of association between incentive value (either positive or negative) and both the features of outcomes and predictive cues not only determines their valence but also the magnitude of evaluative judgments, in keeping with a range of human imaging studies that have concluded the amygdala provides an overall magnitude signal for value judgments, or the interaction between intensity and valence (Anderson et al., 2003; Arana et al., 2003; Small et al., 2003; Winston et al., 2005).
Extensive anatomical connectivity exists between the OFC and BLA (see Figure 1A; Stefanacci and Amaral, 2002; Ghashghaei et al., 2007) allowing them to work closely together in encoding and retrieving value information (see Holland and Gallagher, 2004, for a review). Indeed, damage to the BLA can produce similar deficits to those observed from damage to the OFC (Hatfield et al., 1996; Baxter et al., 2000). However, no brain region acts in isolation, something clearly demonstrated when brain structures are left intact and only their anatomical connections with other structures are severed. Using OFC-BLA contralateral disconnection lesions, Zeeb and Winstanley (2013) found that rats were unable to update their choice preference following reward devaluation. This effect occurred both when the reward was delivered during test and also during extinction when rats needed to rely on stored representations of the outcome. The rats with disconnected OFC and BLA, however, did not differ from controls in their press rates or response latencies, suggesting an impairment specific to altering the value of a particular reward rather than a general reduction in motivation. Similar effects have been observed in humans where structural and functional connectivity between the OFC and BLA was found to correlate with rate of acquisition on a reversal learning task (Cohen et al., 2008).
The nucleus accumbens (NAc) also receives excitatory afferents from the OFC and BLA (amongst other regions), and selectively gates information projecting to basal ganglia output nuclei (Figure 1A; Alheid and Heimer, 1988; Groenewegen et al., 1999). It is often described as the limbic-motor interface, mediating the effect of reward value on action selection (Mogenson and Yim, 1991). Lesions of the NAc core impair the ability of rats to selectively reduce responding after outcome devaluation, demonstrating reduced sensitivity of instrumental performance to changes in outcome value (Corbit et al., 2001; Corbit and Balleine, 2011; Laurent et al., 2012) Importantly, lesions of the NAc also cause a reduction in the vigor of performance, indicating that this region may be involved in how the general motivating properties of reward-related stimuli affect performance (Balleine and Killcross, 1994; Corbit et al., 2001). Interestingly, NAc lesions do not impair sensitivity to selective contingency degradation, revealing that this region does not itself encode the action-outcome contingency but, rather, brings changes in reward value to bear on performance (Corbit et al., 2001). These key evaluative circuits are represented by the red connections in Figure 1A.
Action values: the integration of contingency and value
The value of an action is a product of its contingency with a particular outcome and the desirability of that outcome. As a consequence, interest has grown in the analysis of the neural circuits involved in computing these action values. Studies using trial-and-error action-based learning tasks have reported action value-related signals in the supplementary motor area, where actions are presumably planned before execution. In contrast, BOLD activity in the vmPFC was modulated by the expected reward signal of the chosen action, suggesting that this region provides the agent with feedback about the consequences of their actions to guide future choices (Gläscher et al., 2009; Wunderlich et al., 2009; FitzGerald et al., 2012; Hunt et al., 2013). Camille et al. (2011) found that humans with dorsal anterior cingulate cortex (dACC) damage were unable to maintain the correct choice between actions after positive feedback, suggesting that this region is critically involved in updating action values, perhaps passing feedback from the vmPFC to the action planning areas in the supplementary motor areas via the aCN.
Top-down cognitive control exerted by such structures as the dlPFC and dACC may also modulate the integration of value and contingency, and its conversion into performance. Kim and Shadlen (1999) and Wallis and Miller (2003) found dlPFC neurons that encoded both reward value and the forthcoming response, whereas Kim et al. (2008) found neurons that ramped up or down in their firing rate with increasing or decreasing action values until a choice was made. In the ACC, neural signals resembling the difference between action values, or a combination of movement intention and reward expectation, have been reported (Matsumoto et al., 2007; Seo and Lee, 2007; Wunderlich et al., 2009). Furthermore, lesions of this area in non-human primates and humans produces deficits in action-based choice (Kennerley et al., 2006; Camille et al., 2011). Although there is less agreement about the distinctions in function of the dlPFC and ACC, it is clear that disturbances within these regions radically alter goal-directed choice.
We do know however that the anterior caudate, a part of the associative striatum, is a critical node in the goal-directed network, receiving evaluative input from the BLA and OFC, as well as contingency input from the dlPFC and mPFC. This is supported by data showing that the integration of dopamine and glutamate neurotransmission within this region enables learning and action control by shaping synaptic plasticity and cellular excitability (Shiflett and Balleine, 2011a). In particular, the extracellular signal-regulated kinase (ERK) is particularly important for goal-directed action control due to its sensitivity to combined DA and glutamate receptor activation (Shiflett et al., 2010; Shiflett and Balleine, 2011b). Thus, perturbation of ERK activation associated with various forms of psychopathology and/or drug abuse may produce deficits in goal-directed control. Nevertheless, the role of this region in mediating information from limbic and cortical networks has only relatively recently been recognized in other forms of psychopathology such as that involved in schizophrenia (Howes et al., 2009; Kegeles et al., 2010; Simpson et al., 2010).
Summary of neurobiology of goal-directed learning
In summary, the vmPFC is a functionally complex region critically involved in networks that compute and update outcome values based on feedback or changes in state. The BLA assists in this process by associating incentive value with the sensory information that informs the agent of the reward properties of outcomes, whilst the NAc brings this evaluative information to bear on performance. Simultaneously, the associative striatum and mPFC are also involved in the learning of action-outcome associations, providing information on how to obtain desired outcomes. Together, these processes are integrated in the associative striatum to produce goal-directed behavior. For the purpose of brevity, we have focused on what we believe are the key neural regions involved in goal-directed learning. It must be acknowledged, however, that many other regions likely contribute to these processes in ways that are not yet fully understood.
Stimulus-driven effects on instrumental behavior
Multiple learning systems are involved in the production of healthy everyday behavior. So far we have focused on behavior guided by goals rather than cues. Goal-directed processes allow for flexible choices in the face of changing environmental contexts and conditions. Under stable conditions however, the consequences of actions need not be continually assessed. In these instances, habitual actions, established by the formation of stimulus-response associations, allow reflexive, cue-driven responses to occur at higher speeds and with lower cognitive load (see Figure 1B). The associative systems mediating goal-directed actions and habits are thought to coexist and compete for behavioral control in adaptive decision-making (Dickinson and Balleine, 1993). Another major learning process influencing behavior is the formation of Pavlovian stimulus-outcome associations and conditioned responding (see Figure 1C). Cues associated with reward are able to evoke reward anticipation, which may subsequently guide or bias instrumental choices. Both reward-predictive cues and the experienced value of an instrumental outcome are important incentive processes that play an essential role in motivated behavior. Importantly however, although both may be able to induce reward approach behavior, Pavlovian cues exert their effects on actions through stimulus, rather than outcome value, control.
As depicted in Figure 1, these learning systems are situated in functionally organized cortico-basal ganglia loops. The cortical regions of each system send topographically organized inputs to the striatum—motivational or limbic input to the ventral striatum, associative input to the aCN and anterior putamen, and sensorimotor input to the posterior putamen (Nakano, 2000). From the striatum, GABA-ergic medium spiny neurons (MSNs) project to the principle striatal output nuclei, the substantia nigra pars reticulata (SNr) either directly or indirectly via the globus pallidus pars externa (GPe) and subthalamic nucleus (STN). Whereas MSNs in the direct pathway predominantly express dopamine D1 receptors and activate behavioral functions, those in the indirect pathway express dopamine D2 receptors and tend to suppress behavior (Albin et al., 1989). The ascending dopaminergic system, projecting to the striatum from the substantia nigra pars compacta (SNc) and ventral tegmental area (VTA), plays an important role in modulating activity within these pathways due to their differential expression of D1 and D2 receptors. These modulate the activity of the MSNs bidirectionally; whereas dopamine increases the activity in D1 expressing MSNs, it reduces the activity of D2 expressing MSNs (Gerfen and Surmeier, 2011).
The breakdown of goal-directed processes in psychiatric and neurodevelopmental disorders
The nature of the interaction and cooperation between goal-directed and habitual control processes during decision-making has particular implications should problems arise in the cognitively demanding goal-directed system. Under such conditions, behavioral control may become dominated by dysregulated habitual control, resulting in the loss of flexibility of thought, and the increased stereotypy and behavioral disinhibition characteristic of many psychiatric conditions. Deficits in incentive processes may also produce a range of motivational dysfunctions. Having outlined these processes and their interaction in healthy decision-makers, together with the key neural systems involved above, we turn to consider whether deficits in goal-directed decision-making in psychiatric disorders map onto a common framework. Here we review select evidence for patterns of deficits in outcome sensitivity, action-outcome contingency awareness, and in the integration of these features with action selection in three disorders known for their motivational and cognitive deficits: schizophrenia, ADHD and depression.
Schizophrenia
Motivational and associative learning dysfunction have long been noted in schizophrenia, and have been implicated in positive, negative and cognitive symptomology (Gold et al., 2008). It is often noted that individuals with schizophrenia experience difficulties using emotional states, prior rewards and goals to drive goal-directed action (Barch and Dowd, 2010); i.e, the relationship between value representations and action selection appears to be lost (Heerey and Gold, 2007; Gold et al., 2008; Heerey et al., 2008). We propose that this is due to what amount to functional disconnections within the cortico-striatal loops responsible for integrating evaluative and contingency learning for goal-directed action selection.
Reduced sensitivity to changes in reward value
Negative symptoms such as anhedonia (an inability to experience pleasure) and avolition (a reduced motivation to engage in motivated goal-directed behavior) seem to suggest valuation and action selection deficits are primary in this disease. Anhedonia may be produced by a breakdown in the evaluative circuits responsible for the actual consummatory pleasure experienced from the reward (i.e., the red circuit in Figure 1A). Recently however, a number of studies have shown that, on experiencing or consuming rewards, hedonic ratings are often not significantly reduced compared to controls (Burbridge and Barch, 2007; Gard et al., 2007; Heerey and Gold, 2007) and we have found similar effects in the lab. If evaluative learning is intact, then the critical deficit may lie in anticipating hedonic consequences (reward value) or in using experienced reward values to guide action-selection. Numerous behavioral and neuroimaging studies have focused on whether patients can anticipate reward values. For example, patients with severe avolition fail to choose stimuli associated with monetary reward over a stimulus indicating the avoidance of monetary loss (i.e., no reward) (Gold et al., 2012). This deficit in reward anticipation is consistent with neuroimaging evidence that ventral striatal responses to cues predicting reward are dulled in schizophrenia (Juckel et al., 2006a), including amongst unmedicated patients (Juckel et al., 2006b). Patients also have aberrant neural responses to rewards themselves, including predicted and unpredicted rewards (Waltz et al., 2009; Morris et al., 2012). However no study to date has tested whether patients can adjust their actions solely on the basis of experienced reward values. In a recent study, we tested whether patients with schizophrenia could use the anticipated or experienced reward value to select actions. Patients were able to learn action-outcome associations, and subjectively reported reductions in outcome value after an outcome devaluation procedure, however they did not use this updated outcome knowledge to effectively guide their choices, suggesting that the ability of patients to integrate the values of rewards with action selection processes is deficient. Importantly, BOLD activity in the caudate nucleus during the test requiring this integration was also deficient in patients. Moreover, reduced neural responses in the head of the caudate predicted more severe negative symptoms. This is consistent with recent evidence that neuropathology in schizophrenia, including upregulation of striatal D2 receptor density and occupancy, is most prevalent in the associative regions of the striatum (Buchsbaum and Hazlett, 1998; Abi-Dargham et al., 2000; Howes et al., 2009; Kegeles et al., 2010). On the other hand, patients were able to select actions on the basis of the anticipated reward value, when a cue predicting the availability of reward was presented, albeit not to the same extent as healthy adults (Balleine and Morris, 2013). Thus, the integration of reward values with action selection appears to be impaired in schizophrenia. This particularly affects goal-directed actions when cues are not present to indicate the consequences of action.
The caudate is a critical site for goal-directed actions but it does not function in isolation. In addition to aberrant regional activity in schizophrenia, there is also evidence for functional disconnection of the caudate from its cortical afferents, which can also be found during the prodromal state (Buchsbaum et al., 2006; Yan et al., 2012; Fornito et al., 2013; Quan et al., 2013; Quidé et al., 2013; Wadehra et al., 2013). Thus, the caudate-cortical disconnection in schizophrenia is a critical target for understanding the deficit in goal-directed behavior and predicting functional outcomes associated with the disease.
Changes in contingency awareness
Cognitive deficits are the most pervasive and difficult to treat aspects of schizophrenia (Green, 1996). In particular, any deficit in the ability to form and use A-O associations appropriately and learn about the consequences of our everyday choices is likely to have a large impact on social and occupational functioning. Multiple studies have suggested that the initial acquisition of probabilistic contingencies is relatively unimpaired in schizophrenia, with the exception of some reports of slower rates of acquisition (Weickert et al., 2002; Kéri et al., 2005; Waltz and Gold, 2007). When contingencies are reversed many studies have shown schizophrenic patients do show significant impairments (Waltz and Gold, 2007; Murray et al., 2008), suggesting patients are insensitive to changes in action-outcome contingency. However, distinguishing this impairment in reversal learning from slower acquisition more generally has not been convincingly demonstrated. Using cognitive modeling, however, Strauss et al. (2011) found that patients with schizophrenia have a reduced tendency to explore alternative actions in an uncertain environment. This perseverative style of responding during uncertainty is consistent with greater habitual control of actions. A weakened sensitivity to the action-reward correlation and the predominant use of an S-R learning strategy is also consistent with the fact that rapid learning from trial-by-trial feedback is often impaired but more gradual learning remains intact (Kéri et al., 2005; Gold et al., 2008).
At a neural level, the associative striatum plays an integral role in acquiring A-O contingencies, detecting contingency changes and flexibly using this information during the process of action selection. As reviewed above, functional deficits in the associative striatum as well as pathology in cortical afferents appear early in the pathogenesis of schizophrenia and may be a risk factor for the disease. In this case, a deficit in learning action-outcome contingencies, which critically depends on this circuit, may stand in as an important marker of brain function. However, at present the status of contingency learning deficits in schizophrenia is unclear. Reversal learning tasks such as the IGT or the WCST are generally controlled by reward-related stimuli rather than by the relationship between action and outcome, which makes it difficult to discern whether any deficits are due to altered Pavlovian or instrumental learning. In addition, in reversal learning tasks, it is difficult to establish whether changes in outcome value or in contingency are driving choices. Thus, the use of contingency degradation tasks within this cohort will be critical to provide convincing evidence regarding the level of impairment in contingency awareness and the functional status of the related circuits.
Schizophrenia summary
In summary, during goal-directed learning, patients with schizophrenia are only mildly or are unimpaired in their subjective valuation assessments, and in the activation of prefrontal regions that support them. Dysfunction in the associative striatum and its cortical afferents, however, may interfere with the ability to modulate action selection using value information. Evidence also suggests that patients with schizophrenia are able to encode initial A-O associations, but they may be impaired at updating associations for flexible use in action selection. Taken together, these impairments in integrating the key components of goal-directed behavior suggest that patients with schizophrenia may over rely on habit learning and habitual strategies, predicting relatively intact functioning of the circuitry mediating habitual control but not goal-directed performance.
ADHD
Altered sensitivity to reinforcement is acknowledged as an important etiological factor in a number of theoretical frameworks of ADHD (Barkley, 1997; Sergeant et al., 1999; Castellanos and Tannock, 2002; Sagvolden et al., 2005; Frank et al., 2007; Tripp and Wickens, 2008; Sonuga-Barke and Fairchild, 2012). ADHD is characterized by symptoms of inattention, hyperactivity and impulsivity, consistent with dysregulation of top-down control processes modulating goal-directed control. A number of researchers have argued that ADHD is a motivational problem, whereby individuals are unable to use intrinsic motivation to guide choice performance (Douglas, 1989; Sergeant et al., 1999). This is supported by evidence that children with ADHD perform well on continuous reinforcement schedules, whereas their performance deteriorates on partial reinforcement schedules where the consistent extrinsic motivation of reward is not provided (Parry and Douglas, 1983; Luman et al., 2008).
Dopaminergic dysfunction clearly plays a key role in ADHD symptomology. The primary treatment for ADHD, Methylphenidate, preferentially blocks the reuptake of DA in the striatum (Schiffer et al., 2006), and studies have demonstrated its effectiveness in normalizing reinforcement sensitivity in ADHD relative to placebo (Tripp and Alsop, 1999; Frank and Claus, 2006). Furthermore, Volkow et al. (2012) has proposed that disruption of D2/D3 receptors is associated with the motivation deficits observed in ADHD, which may in turn contribute to attention deficits. Attention was found to be negatively correlated with D2/D3 receptor availability in the left NAc and caudate (Volkow et al., 2009), regions key to reward valuation and contingency awareness in goal-directed action. We hypothesize that motivational problems stem primarily from an inability to predict the rewarding consequences of cues or actions. As a consequence actions may be poorly controlled or regulated resulting in inappropriate responses to the situation and undesirable consequences.
The dopamine transfer deficit theory
The Dopamine Transfer Deficit theory of ADHD (Tripp and Wickens, 2008, 2009) proposes that altered phasic dopamine responses to reward-predictive cues results in blunted stimulus-outcome associations, and hence blunted reward anticipation. In this sense, motivational deficiencies may be derived from a lack of stimulus-outcome contingency awareness (i.e., an impairment within the circuitry detailed in Figure 1C). The relatively consistent finding of hypo-activation in the ventral striatum during reward anticipation supports this idea (Scheres et al., 2007; Ströhle et al., 2008; Plichta et al., 2009; Hoogman et al., 2011; Carmona et al., 2012; Edel et al., 2013; Plichta and Scheres, 2013). Wilbertz et al. (2012) found increased OFC activation during outcome delivery consistent with increased excitation to reward; however, as reward-related stimuli were generally less successful at inducing reward anticipation, it may also reflect an aberrant prediction error-like response. Overall, rather than suggesting that reward sensitivity is impaired, the evidence seems to support the notion that an inability to anticipate reward may reduce motivation or impair the ability to select the relevant action.
In comparison to schizophrenia, both patient groups have intact reward sensitivity, however the pathologies can be dissociated by the role of predicted reward-values and experienced reward-values on action-selection. In ADHD, we expect to see impairment in selecting actions on the basis of predicted reward (e.g., a deficit in outcome specific Pavlovian-to-instrumental transfer); whereas in schizophrenia the deficit is related to using experienced reward values to guide action selection (e.g., a deficit in outcome specific devaluation). The amount of overlap between these two groups should, therefore, be predicted to depend on the extent to which both share neuropathology in the ventral striatum, which will disrupt dopamine signaling due to hyper- or hypodopaminergia, regardless.
Incentive learning deficits, response inhibition and impulsivity
Response inhibition and impulsivity are key deficits exhibited in ADHD even when executive function demands are low (Wodka et al., 2007); both children and adult subjects are slower to inhibit responses during the go/no-go or stop-signal reaction time (SSRT) tasks, and make more errors than age-matched controls (Schachar et al., 1995; Purvis and Tannock, 2000; see Solanto, 2002 for a review). Lesions of the BLA and NAc both increase impulsive choice on a delay-discounting task in rats (Winstanley et al., 2004), and measures of impulsivity are generally negatively correlated with white matter integrity in right OFC fiber tracts in adults with ADHD. Thus impulsivity may be induced by dysfunction in key incentive processing regions, or alternatively, these regions may be underutilized due to an over reliance on reflexive actions that are not based on the value of consequences.
Changes in contingency awareness
Tripp and Wickens (2008) postulate that stimulus-outcome associations are disturbed in ADHD due to a lack of transfer of dopamine firing from reward receipt to reward-predictive cues. To date, however, there have not been any comparable studies assessing whether this is also the case for action-outcome learning. We predict that due to dopamine dysregulation within the associative striatum, contingency awareness will be deficient perhaps for both cue and action–based associations with specific outcomes. Firstly, reduced salience or attention allocation due to dysfunction in DA firing may inhibit the formation of action-outcome associations. Furthermore, when a temporal delay occurs between an action and its outcome, DA dysfunction may generate difficulties in “credit assignment”—deciding to which recent action one should attribute the outcome (Johansen et al., 2009). This difficulty could contribute to the delay aversion often documented in ADHD (Sonuga-Barke, 2002), and the easy distraction by extraneous stimuli. For instance, Carlson et al. (2000) found that, relative to controls, ADHD children were more likely to attribute success on an arithmetic task to luck, which seems to support reduced awareness of action-outcome causality. The dopamine transfer deficit theory also predicts that in ADHD, smaller anticipatory dopamine signals relative to the response to actual reinforcers would result in a greater influence of the most recent contingency than longer-term reinforcement history (Tripp and Wickens, 2008). This could result in faster extinction under partial reinforcement, or increases in the performance of occasionally rewarded, but overall suboptimal, actions.
Caudate impairments and action selection in ADHD
Meta-analyses have shown that the most consistent gray matter reductions in ADHD occur in the caudate, a region critical for goal-directed behavior. This morphological deficit was worse in samples with lower levels of stimulant medication, suggesting that dopamine normalization may counteract caudate atrophy (Valera et al., 2007; Nakao et al., 2011). Impairments in the striatum likely affect both contingency awareness and their integration with action selection processes. Reduced structural connectivity may also hinder this integration; indeed, ADHD patients have been shown to have anomalous white matter integrity in fronto-striatal and premotor (PM) regions relative to age matched controls (Ashtari et al., 2005; Silk et al., 2009; Konrad and Eickhoff, 2010).
ADHD summary
In summary, we hypothesize, with others, that motivational impairments in ADHD arise due to an inability to accurately predict the occurrence of rewarding outcomes. This in turn reduces the salience of reward predictive cues and optimal actions potentially contributing to attentional deficits. Dopamine dysfunction within the striatum seems to be a key factor in this contingency awareness impairment. Furthermore, a greater reliance on recent rather than longer-term reinforcement history could explain the rapid extinction of learnt associations, and why patients with ADHD respond better to continuous reinforcement schedules.
Depression
The major diagnostic guidelines state that individuals experiencing depressive episodes often have difficulty making decisions (DSM IV, APA, 2000; ICD-10, WHO, 1992). Traditionally, it has been assumed that this was due to primary motivational impairments, however cognitive deficits associated with the disorder are becomingly increasingly well documented (Lee et al., 2012). We predict that whereas outcome valuation will be strongly affected in those experiencing anhedonia, contingency sensitivity impairments may also be detected in a subset of cognitively-impaired patients. Further, reward learning and cognitive deficits may persist during periods of euthymia, predisposing individuals to future depressive episodes.
Deficits in reward sensitivity
Depression is commonly characterized by blunted reward responsiveness (Henriques and Davidson, 2000; Pizzagalli et al., 2008; McFarland and Klein, 2009) and behavioral neglect of positive stimuli (Clark et al., 2009), which is reflected in the symptoms of anhedonia, social withdrawal and reduced activity level. As experienced rewards are no longer pleasurable, it is easy to envisage how action control could become biased away from goal-directed actions toward habits, which require only the preservation of a sufficient reinforcement signal to form stimulus-response associations.
During both reward and punished responding in depressed subjects, blunted responses are observed in the medial caudate and ventromedial OFC (Elliott et al., 1998). This supports behavioral accounts of blunted reward sensitivity. Interestingly, McCabe et al. (2009) found that, in remitted depressed patients, there were decreased reward responses in the ventral striatum, caudate and anterior cingulate, despite subjective ratings being the same as controls, suggesting that altered reward sensitivity occurs independent of mood symptoms, and may actually be a predisposing factor in the etiology of depressive episodes.
One prominent theory proposes that a defect in the top-down inhibition of the amygdala by the vmPFC may underlie depression symptoms (Myers-Schulz and Koenigs, 2011). For instance, Friedel et al. (2009) reported a negative correlation between depressive symptom severity and connectivity between the mOFC and the amygdala. As discussed earlier, the amygdala and OFC and their connectivity are required for the encoding and use of value-based information. Therefore impairment in either region, or reduced connectivity between them, will likely hamper the updating of value and its integration to mediate goal-directed choice. Due to reduced OFC-BLA connectivity, we predict that individuals with severe anhedonia will be unable to alter their choices appropriately after outcome devaluation.
Significantly reduced ventral striatal activity to positive stimuli has also been observed in depressed patients (Epstein et al., 2006; Robinson et al., 2012; Stoy et al., 2012), which may reflect a deficit in using value information to guide action selection. These studies employed predominantly Pavlovian learning processes and, therefore, the focus was generally on assessing anticipation of reward rather than how value knowledge was used to guide instrumental choices. Nevertheless, Stoy et al. (2012) discovered that treatment with the common antidepressant, escitalopram, normalized anticipatory reward signals in the ventral striatum, highlighting how medications affecting reward circuitry could be effective in improving depressive symptoms. In addition, deep brain stimulation to the bilateral NAc in refractory depression has shown promising results for reduction of the symptoms of anhedonia (Schlaepfer et al., 2008; Malone et al., 2009).
Deficits in contingency awareness
Although it is evident that anhedonia diminishes the impact of reward processes in goal-directed action, there is significantly more debate about how causal awareness is affected in depression. Depressed individuals often experience symptoms of learned helplessness, which may reflect dysfunction in causal knowledge. Learned helplessness is essentially an error in attribution of control (Miller and Seligman, 1975) in the sense that a depressed person may have aberrant beliefs about the causality of their actions in achieving a goal, or the lack thereof, and so not initiate an action. Using Bayesian modeling, Lieder et al. (2013) argued that generalization of action-outcome contingencies is able to account for a range of learned helplessness phenomena. By this account, individuals attribute outcomes to their current situation or state rather than to the chosen action; they generalize across available actions, with the belief that the state will determine the outcome, irrespective of their actions.
Paradoxically however, a large body of research has also supported the idea that dysphoric or depressive individuals often have greater causal sensitivity, an effect referred to as depressive realism (Alloy and Abramson, 1979; Martin et al., 1984; Benassi and Mahler, 1985; Ackermann and DeRubeis, 1991; Allan et al., 2007; Msetfi et al., 2012). Indeed, Alloy and Abramson (1979) found that, during a task incorporating both contingent and non-contingent outcomes non-depressed people were more likely to believe that their actions were causal of the outcome whereas depressed people did not show this illusion of control, and tended to rate their actions in this task as less causal.
These contradictory findings in depressed people might be reconciled by considering the role of competition between actions and cues for causal learning. There are two major predictors of outcomes in our environment: our own instrumental actions and situational stimuli such as Pavlovian cues. These two classes of events will compete as causes for outcomes of interest during causal learning tasks, like those described above. In such tasks, when non-contingent outcomes are provided, situational stimuli can become better predictors of those outcomes than actions. So the illusion of control could reflect a disposition to assign causal status to ones own actions over situational stimuli, even when situational stimuli are better predictors. In contrast, if action-outcome contingency awareness is impaired, then situational stimuli should be predicted to outcompete actions for association with specific outcomes and in their attribution as causes of those outcomes. This should be anticipated to produce more accurate causal judgments of actions, consistent with depressive realism. Furthermore, the deficit in action-outcome contingency awareness will still produce learned helplessness.
An implication of this argument, derived from the distinct neural regions responsible for action-outcome vs. stimulus-outcome contingency awareness, is that pathology in depression should be restricted to those medial prefrontal cortical regions that are critical for A-O learning. Conversely, the lateral PFC regions implicated in S-O learning should be relatively intact on this view. In fact, considerable research has explored the role of mPFC in behavioral control over the effects of chronic stress (Amat et al., 2005; Maier and Watkins, 2010). Resistance to environmental stressors, and as such, resilience against feelings of helplessness, is thought to rely on inhibitory control exerted by the vmPFC over limbic structures. Without this inhibition, it is argued, stressors could cause sensitization of serotonergic neurons in the dorsal raphe, changing how the organism responds to subsequent aversive stimuli (Maier and Watkins, 2005).
Serotonin is a neuromodulator thought to play a key role in the neurochemical basis of depression, with selective serotonin reuptake inhibitors being a first-line treatment of depression. It has also been implicated in the modulation of decision processes. For instance, Doya (2002) proposed that low levels of serotonin may be associated with excessive discounting of future rewards, while others have argued that it is more specifically involved with inhibiting actions and thoughts associated with aversive outcomes (Daw et al., 2002; Dayan and Huys, 2008; Huys et al., 2012; Robinson et al., 2012). This view proposes that serotonin reductions enhance punishment predictions, but do not effect reward predictions. This raises another interesting line of research–whether individuals with depression are perhaps better at learning associations with negative rather than positive consequences (see Eshel and Roiser, 2010, for a review). Numerous studies have demonstrated that depressed individuals exhibit hypersensitivity to negative feedback (Elliott et al., 1997), and hyposensitivity to positive feedback (Pizzagalli et al., 2008), and highlight how aberrance in evaluation, and subsequent allocation of attention, has detrimental effects on contingency learning.
The emerging field of computational psychiatry has provided a promising new avenue for understanding psychiatric illnesses, through applying mathematical models to behavioral and biological problems. Within decision neuroscience, it aims to provide a systematic explanation of the core processes in decision-making in a manner consistent with neurobiologically relevant processes (Dayan and Huys, 2008). A series of studies have recently used this approach in discerning the specific decision-making deficits at play in depression. In this approach, reward sensitivity is related to valuation, while learning rate represents a dimension of contingency awareness. Chase et al. (2010) found a reduced learning rate in depression, however they did note that learning rate was more closely related to severity of anhedonia than diagnosis per se. A recent meta-analysis in un-medicated depression reported that reduced reward sensitivity (reduced prediction errors) had greater affect than learning rate on overall learning performance, and was correlated with anhedonia severity (Huys et al., 2013). This is supported by reduced striatal activation during reward receipt (Pizzagalli et al., 2009; Smoski et al., 2009). Using a medicated sample, however, we found that learning rate was reduced in depression, which may indicate that while overall choice behavior remains impaired, antidepressant medication may change the dynamics of the contributing processes (Griffiths et al., unpublished data).
Structural and resting-state abnormalities in goal-directed circuitry
The difficulties depressed individuals have with learning and performance of goal-directed action correspond with abnormalities in learning and choice related brain regions. Gray matter volumetric studies and postmortem examinations have show neuronal size reductions relative to controls in the OFC (Cotter et al., 2005; Drevets and Price, 2008), left ACC (Drevets et al., 1997; Coryell et al., 2005), dlPFC (Drevets, 2004), caudate and NAc (Baumann et al., 1999). Moreover, symptoms of anhedonia, depression severity and probability of suicide have all been associated with reduced caudate volume (Pizzagalli et al., 2008) and caudate activity (Forbes et al., 2009).
There is a complex relationship between depression severity and the OFC. Some studies report increased OFC activity in treatment responsive depressives, whereas more severely ill patients have relatively normal or decreased OFC metabolism (Drevets et al., 1997; Mayberg, 1997). Drevets et al. (1997) posit that increased OFC activity may reflect a cognitive compensatory effort to attenuate negative emotion, while reduced OFC activity may reflect a primary pathology related to monoamine dysfunction. This is supported by enhanced dextroamphetamine-induced rewarding effects compared to controls (Tremblay et al., 2002, 2005). Functional imaging during a range of tasks involving planning, reward, behavioral choice and feedback have reported abnormal recruitment of the mOFC (Elliott et al., 1998; Taylor Tavares et al., 2008), and lesions of the human OFC have been argued to increase the risk for developing depression (Drevets, 2007), although this is controversial (see e.g., Carson et al., 2000). Nevertheless, reports that this region plays a key role in valuation suggest that any compromised function will likely affect goal-directed action.
In addition to problems with the core circuitry associate with goal-directed action, imaging studies have shown abnormally low dlPFC activity during resting state (Galynker et al., 1998), yet overly activated activation during working memory and cognitive control tasks (Harvey et al., 2005; Wagner et al., 2006), potentially indicating inefficiency in this cognitive control region. This may contribute to the increased indecisiveness experienced in depression.
Depression summary
In summary, depression is characterized by impairments in reinforcement learning, and using affective information to guide behavior. Anhedonia, a common symptom in depression, maps closely onto deficits within outcome valuation circuitry, and is the clearest example of how problems with reward value lead to reductions in goal-directed action. Learned helplessness, or a lack of resistance to environmental stressors, may also occur when S-O associations outcompete A-O associations. This may cause depressed individuals to generalize action-outcome contingencies across different contexts, and become less adaptive to new environments.
Other disorders
It is clear that an associative learning framework can provide testable hypotheses and explanations for a range of deficits in clinical disorders. Though we can only provide a brief discussion of three such disorders here, the potential exists for many others. For instance, Obsessive-Compulsive disorder, where behavior may exhibit an overreliance on habits due to dysfunctional goal-directed circuitry (Gillan et al., 2011), and anorexia nervosa, where there is a tendency to deprive oneself of food, despite, or likely because of, hyperactivity in evaluative neural circuitry during food presentation (Keating et al., 2012), provide interesting examples.
Importantly, assessment of decision-making deficits need not be constrained rigidly by diagnostic classifications. Most psychiatry research uses these classifications with the assumption that it will provide a homogenous subset of participants. However multiple systems may be differentially affected in these patients, and comorbidities and group averaging may contaminate both behavioral and neural results. Further, symptom commonalities also occur across diagnostic boundaries, for instance anhedonia, which can occur in a range of disorders, such as depression, post-traumatic stress disorder and schizophrenia. Thus, behavioral tests that probe specific processes and neural deficits could have great value in guiding research on biologically-based individualized classification.
It is worth mentioning that the wide-ranging use of medications and substance use in psychiatric groups makes testing these populations to clearly delineating the source of their illness very challenging. Most medications affect multiple, predominantly monoamine, neurotransmitter systems, and variance in functional effects occurs over different doses. These neurotransmitter systems are intricately involved in reward and decision processes, thus it can be difficult to distinguish disorder-related findings from those induced by medication, and to untangle the differential effects of medications across tasks. For instance, using SPECT, Paquet et al. (2004) found a correlation between procedural learning ability and D2 receptor occupancy. Patients on second generation antipsychotics (SGA) perform better at procedural learning tasks compared to those on first generation antipsychotics (FGA), which is thought to be due to the comparatively lower affinity for striatal D2 receptors in SGAs (Stevens et al., 2002; Scherer et al., 2004). Conversely, Beninger et al. (2003) found that SGAs adversely affected performance on the IGT, which they surmise may be due to the high affinity of SGAs for serotonin receptors in the PFC.
Conclusions
Though much progress has been made in elucidating the processes and neurobiology of decision-making, a great deal remains to be done. Contradictory findings and interpretations persist, and with contributions from diverse fields such as economics, computer science and psychology, a “common language” has not yet been achieved. Decision-making is an extremely complex process, and as such, the range of tasks used to assess this skill is broad. Great care must be taken when comparing results across tasks, as task-related variables may modulate the underlying circuitry involved.
A key strength of associative learning tasks is the strong theoretical basis, and the broad foundation of animal research that has helped develop our knowledge of the circuitry underlying specific learning processes. By establishing links between well-defined psychological processes (e.g., goal-directed action), neural circuits and even intracellular signaling, we can develop a biologically-based phenotype of psychopathology, grounded in translatable behavioral tests. Nevertheless, important questions remain regarding how we conceptualize the interaction between these learning systems. For instance, a flat architecture assumes that goal-directed and habitual processes exist in parallel, with an arbitrator determining which system is utilized for the following action. A hierarchical structure, however, proposes a global goal-directed system that incorporates habitual action sequences when they can achieve the desired goal. Although beyond the scope of this review, there are a number of neural and computational theories that debate how and where action values are compared and transformed into motor signals, and if in fact, cognitive action selection and motor planning occur as serial or simultaneous processes (Cisek and Kalaska, 2010; Hare et al., 2011; Cisek, 2012; Rushworth et al., 2012; Wunderlich et al., 2012; Dezfouli and Balleine, 2013). These theories are important considerations for determining precisely how fundamental processes such as outcome valuation and contingency learning are transformed into the motor choices producing goal-directed performance.
Decision neuroscience is an exciting field that incorporates translational research from a range of species and scientific techniques. Within this field, associative learning accounts have provided a theoretical basis for the development of a range of biologically relevant behavioral paradigms. This framework endeavors to draws together behavioral and neurological processes, creating impetus for a wide range of testable hypotheses. Through systematic application of biologically relevant paradigms, we could further identify specific problems contributing to maladaptive decision-making across psychiatric disorders. This review has attempted to highlight how a number of deficits across psychiatric disorders may be explained in terms of fundamental reward learning and performance impairments, which could shed some new light on the functional impairment and neurobiological underpinnings of these illnesses.
Statements
Acknowledgments
The preparation of this manuscript was supported by a grant from the Australian Research Council (ARC FL0992409) to Bernard W. Balleine.
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
References
1
Abi-DarghamA.RodenhiserJ.PrintzD.Zea-PonceY.GilR.KegelesL. S.et al. (2000). Increased baseline occupancy of D2 receptors by dopamine in schizophrenia. Proc. Natl. Acad. Sci. U S A97, 8104–8109. 10.1073/pnas.97.14.8104
2
AckermannR.DeRubeisR. J. (1991). Is depressive realism real?Clin. Psychol. Rev.11, 565–584. 10.1016/0272-7358(91)90004-E
3
AlbinR. L.YoungA. B.PenneyJ. B. (1989). The functional anatomy of basal ganglia disorders. Trends Neurosci.12, 366–375. 10.1016/0166-2236(89)90074-x
4
AlheidG. F.HeimerL. (1988). New perspectives in basal forebrain organization of special relevance for neuropsychiatric disorders: the striatopallidal, amygdaloid and corticopetal components of substantia innominata. Neuroscience27, 1–39. 10.1016/0306-4522(88)90217-5
5
AllanL. G.SiegelS.HannahS. (2007). The sad truth about depressive realism. Q. J. Exp. Psychol. (Hove)60, 482–495. 10.1080/17470210601002686
6
AlloyL. B.AbramsonL. Y. (1979). Judgment of contingency in depressed and nondepressed students: sadder but wiser?J. Exp. Psychol. Gen.108, 441–485. 10.1037/0096-3445.108.4.441
7
AmatJ.BarattaM. V.PaulE.BlandS. T.WatkinsL. R.MaierS. F. (2005). Medial prefrontal cortex determines how stressor controllability affects behavior and dorsal raphe nucleus. Nat. Neurosci.8, 365–371. 10.1038/nn1399
8
American Psychiatric Association. (2000). Diagnostic and Statistical Manual of Mental Disorders (DSM-IV TR).Washington, DC: American Psychiatric Association.
9
AndersonA. K.ChristoffK.StappenI.PanitzD.GhahremaniD. G.GloverG.et al. (2003). Dissociated neural representations of intensity and valence in human olfaction. Nat. Neurosci.6, 196–202. 10.1038/nn1001
10
AranaF. S.ParkinsonJ. A.HintonE.HollandA. J.OwenA. M.RobertsA. C. (2003). Dissociable contributions of the human amygdala and orbitofrontal cortex to incentive motivation and goal selection. J. Neurosci.23, 9632–9638.
11
AshtariM.KumraS.BhaskarS. L.ClarkeT.ThadenE.CervellioneK. L.et al. (2005). Attention-deficit/hyperactivity disorder: a preliminary diffusion tensor imaging study. Biol. Psychiatry57, 448–455. 10.1016/j.biopsych.2004.11.047
12
BaeyensF.EelenP.BerghO. V. D. (1990). Contingency awareness in evaluative conditioning: a case for unaware affective-evaluative learning. Cogn. Emot.4, 3–18. 10.1080/02699939008406760
13
BalleineB. W.DickinsonA. (1998). Goal-directed instrumental action: contingency and incentive learning and their cortical substrates. Neuropharmacology37, 407–419. http://www.sciencedirect.com/science/article/pii/S002839089800033110.1016/s0028-3908(98)00033-1
14
BalleineB. W.KillcrossS. (1994). Effects of ibotenic acid lesions of the nucleus accumbens on instrumental action. Behav. Brain Res.65, 181–193. 10.1016/0166-4328(94)90104-x
15
BalleineB. W.KillcrossA. S.DickinsonA. (2003). The effect of lesions of the basolateral amygdala on instrumental conditioning. J. Neurosci.23, 666–675. http://www.jneurosci.org/content/23/2/666.short
16
BalleineB. W.LeungB. K.OstlundS. B. (2011). Orbitofrontal cortex, predicted value, and choice. Ann. N Y Acad. Sci.1239, 43–50. 10.1111/j.1749-6632.2011.06270.x
17
BalleineB. W.MorrisR. (2013). Stimulus- vs reward-guided decision-making in Schizophrenia. Biol. Psychiatry73, 17S–17S.
18
BalleineB. W.O’DohertyJ. P. (2010). Human and rodent homologies in action control: corticostriatal determinants of goal-directed and habitual action. Neuropsychopharmacology34, 48–69. 10.1038/npp.2009.131
19
BalleineB. W.OstlundS. B. (2007). Still at the choice-point: action selection and initiation in instrumental conditioning. Ann. N Y Acad. Sci.1104, 147–171. 10.1196/annals.1390.006
20
BarchD. M.DowdE. (2010). Goal representations and motivational drive in schizophrenia: the role of prefrontal-striatal interactions. Schizophr. Bull.36, 919–934. 10.1093/schbul/sbq068
21
BarkleyR. A. (1997). Behavioral inhibition, sustained attention and executive functions: constructing a unifying theory of ADHD. Psychol. Bull.121, 65–94. 10.1037//0033-2909.121.1.65
22
BaumannB.DanosP.KrellD.DiekmannS.LeschingerA.StauchR.et al. (1999). Reduced volume of limbic system-affiliated basal ganglia in mood disorders: preliminary data from a postmortem study. J. Neuropsychiatry Clin. Neurosci.11, 71–78.
23
BaxterM. G.ParkerA.LindnerC. C.IzquierdoA. D.MurrayE. A. (2000). Control of response selection by reinforcer value requires interaction of amygdala and orbital prefrontal cortex. J. Neurosci.20, 4311–4319.
24
BecharaA.DamasioA. R.DamasioH.AndersonS. W. (1994). Insensitivity to future consequences following damage to human prefrontal cortex. Cognition50, 7–15. 10.1016/0010-0277(94)90018-3
25
BenassiV. A.MahlerH. I. (1985). Contingency judgments by depressed college students: sadder but not always wiser. J. Pers. Soc. Psychol.49, 1323–1329. 10.1037//0022-3514.49.5.1323
26
BeningerR. J.WassermanJ.ZanibbiK.CharbonneauD.MangelsJ.BeningerB. V. (2003). Typical and atypical antipsychotic medications differentially affect two nondeclarative memory tasks in schizophrenic patients: a double dissociation. Schizophr. Res.61, 281–292. 10.1016/s0920-9964(02)00315-8
27
BuchsbaumM. S.HazlettE. A. (1998). Positron emission tomography studies of abnormal glucose metabolism in schizophrenia. Schizophr. Bull.24, 343–364. 10.1093/oxfordjournals.schbul.a033331
28
BuchsbaumM. S.SchoenknechtP.TorosjanY.NewmarkR.ChuK. W.MitelmanS.et al. (2006). Diffusion tensor imaging of frontal lobe white matter tracts in schizophrenia. Ann. Gen. Psychiatry5:19. 10.1186/1744-859X-5-19
29
BurbridgeJ. A.BarchD. M. (2007). Anhedonia and the experience of emotion in individuals with schizophrenia. J. Abnorm. Psychol.116, 30–42. 10.1037/0021-843x.116.1.32.supp
30
CamilleN.TsuchidaA.FellowsL. K. (2011). Double dissociation of stimulus-value and action-value learning in humans with orbitofrontal or anterior cingulate cortex damage. J. Neurosci.31, 15048–15052. 10.1523/jneurosci.3164-11.2011
31
CarlsonC. L.MannM.AlexanderD. K. (2000). Effects of reward and response cost on the performance and motivation of children with ADHD. Cognit. Ther. Res.24, 87–98. 10.1023/A:1005455009154.
32
CarsonA. J.MacHaleS.AllenK.LawrieS. M.DennisM.HouseA.et al. (2000). Depression after stroke and lesion location: a systematic review. Lancet356, 122–126. 10.1016/S0140-6736(00)02448-X
33
CarmichaelS. T.PriceJ. L. (1995). Limbic connections of the orbital and medial prefrontal cortex in macaque monkeys. J. Comp. Neurol.363, 615–641. 10.1002/cne.903630408
34
CarmonaS.HoekzemaE.Ramos-QuirogaJ. A.RicharteV.CanalsC.BoschR.et al. (2012). Response inhibition and reward anticipation in medication-naïve adults with attention-deficit/hyperactivity disorder: a within-subject case-control neuroimaging study. Hum. Brain Mapp.33, 2350–2361. 10.1002/hbm.21368
35
CastellanosF. X.TannockR. (2002). Neuroscience of attention-deficit/hyperactivity disorder: the search for endophenotypes. Nat. Rev. Neurosci.3, 617–628. 10.1016/b978-008045046-9.00378-8
36
ChaseH. W.FrankM. J.MichaelA.BullmoreE. T.SahakianB. J.RobbinsT. W. (2010). Approach and avoidance learning in patients with major depression and healthy controls: relation to anhedonia. Psychol. Med.40, 433–440. 10.1017/s0033291709990468
37
CisekP. (2012). Making decisions through a distributed consensus. Curr. Opin. Neurobiol.22, 927–936. 10.1016/j.conb.2012.05.007
38
CisekP.KalaskaJ. F. (2010). Neural mechanisms for interacting with a world full of action choices. Annu. Rev. Neurosci.33, 269–298. 10.1146/annurev.neuro.051508.135409
39
ClarkL.ChamberlainS. R.SahakianB. J. (2009). Neurocognitive mechanisms in depression: implications for treatment. Annu. Rev. Neurosci.32, 57–74. 10.1146/annurev.neuro.31.060407.125618
40
CohenM. X.ElgerC. E.WeberB. (2008). Amygdala tractography predicts functional connectivity and learning during feedback-guided decision-making. Neuroimage39, 1396–1407. 10.1016/j.neuroimage.2007.10.004
41
CorbitL. H.BalleineB. W. (2003). The role of prelimbic cortex in instrumental conditioning. Behav. Brain Res.146, 145–157. 10.1016/j.bbr.2003.09.023
42
CorbitL. H.BalleineB. W. (2005). Double dissociation of basolateral and central amygdala lesions on the general and outcome-specific forms of pavlovian-instrumental transfer. J. Neurosci.25, 962–970. 10.1523/jneurosci.4507-04.2005
43
CorbitL. H.BalleineB. W. (2011). The general and outcome-specific forms of Pavlovian-instrumental transfer are differentially mediated by the nucleus accumbens core and shell. J. Neurosci.31, 11786–11794. 10.1523/jneurosci.2711-11.2011
44
CorbitL. H.MuirJ. L.BalleineB. W. (2001). The role of the nucleus accumbens in instrumental conditioning: evidence of a functional dissociation between accumbens core and shell. J. Neurosci.21, 3251–3260.
45
CoryellW.NopoulosP.DrevetsW.WilsonT.AndreasenN. C. (2005). Subgenual prefrontal cortex volumes in major depressive disorder and schizophrenia: diagnostic specificity and prognostic implications. Am. J. Psychiatry162, 1706–1712. 10.1176/appi.ajp.162.9.1706
46
CotterD.HudsonL.LandauS. (2005). Evidence for orbitofrontal pathology in bipolar disorder and major depression, but not in schizophrenia. Bipolar Disord.7, 358–369. 10.1111/j.1399-5618.2005.00230.x
47
DawN. D.KakadeS.DayanP. (2002). Opponent interactions between serotonin and dopamine. Neural Netw.15, 603–616. 10.1016/s0893-6080(02)00052-7
48
DayanP.HuysQ. J. (2008). Serotonin, inhibition, and negative mood. PLoS Comput. Biol.4:e4. 10.1371/journal.pcbi.0040004
49
DezfouliA.BalleineB. W. (2013). Actions, action sequences and habits: evidence that goal-directed and habitual action control are hierarchically organized. PLoS Comput. Biol.9:e1003364. 10.1371/journal.pcbi.1003364
50
DickinsonA. (1994). “Instrumental conditioning,” in Animal Learning and Cognition, ed MackintoshJ. N. (San Diego, CA: Academic Press), 45–78.
51
DickinsonA.BalleineB. (1993). “Actions and responses: the dual psychology of behaviour,” in Spatial Representation: Problems in Philosophy and Psychology, eds EilanN.McCarthyR. A.BrewerB. (Malden: Blackwell Publishing), 277–293.
52
DickinsonA.BalleineB. (1994). Motivational control of goal-directed action. Anim. Learn. Behav.22, 1–18. 10.3758/bf03199951
53
DouglasV. I. (1989). Can Skinnerian theory explain attention deficit disorder? A reply to Barkley. Atten. Defic. Disord. Curr. Concepts Emerg. Trends Atten. Behav. Disord. Child.4, 235–254. 10.1016/b978-0-08-036508-4.50018-7
54
DoyaK. (2002). Metalearning and neuromodulation. Neural Netw.15, 495–506. 10.1016/s0893-6080(02)00044-8
55
DrevetsW. C. (2004). Neuroplasticity in mood disorders. Dialogues Clin. Neurosci.6, 199–216.
56
DrevetsW. C. (2007). Orbitofrontal cortex function and structure in depression. Ann. N Y Acad. Sci.1121, 499–527. 10.1196/annals.1401.029
57
DrevetsW. C.PriceJ. L. (2008). “Neuroimaging and neuropathological studies of mood disorders,” in Biology of Depression: From Novel Insights to Therapeutic Strategies, eds LicinioJ.WongM.-L. (Weinheim, Germany: Wiley-VCH Verlag GmbH), 427–465. 10.1002/9783527619672.ch17
58
DrevetsW. C.PriceJ. L.SimpsonJ. R.ToddR. D.ReichT.VannierM.et al. (1997). Subgenual prefrontal cortex abnormalities in mood disorders. Nature386, 824–827. 10.1038/386824a0
59
EdelM. A.EnziB.WitthausH.TegenthoffM.PetersS.JuckelG.et al. (2013). Differential reward processing in subtypes of adult attention deficit hyperactivity disorder. J. Psychiatr. Res.47, 350–356. 10.1016/j.jpsychires.2012.09.026
60
ElliottR.SahakianB. J.HerrodJ. J.RobbinsT. W.PaykelE. S. (1997). Abnormal response to negative feedback in unipolar depression: evidence for a diagnosis specific impairment. J. Neurol. Neurosurg. Psychiatry63, 74–82. 10.1136/jnnp.63.1.74
61
ElliottR.DolanR. J.FrithC. D. (2000). Dissociable functions in the medial and lateral orbitofrontal cortex: evidence from human neuroimaging studies. Cereb. Cortex10, 308–317. 10.1093/cercor/10.3.308
62
ElliottR.SahakianB. J.MichaelA.PaykelE. S.DolanR. J. (1998). Abnormal neural response to feedback on planning and guessing tasks in patients with unipolar depression. Psychol. Med.28, 559–571. 10.1017/s0033291798006709
63
EpsteinJ.PanH.KocsisJ.YangY.ButlerT.ChusidJ.et al. (2006). Lack of ventral striatal response to positive stimuli in depressed versus normal subjects. Am. J. Psychiatry163, 1784–1790. 10.1176/appi.ajp.163.10.1784
64
EshelN.RoiserJ. P. (2010). Reward and punishment processing in depression. Biol. Psychiatry68, 118–124. 10.1016/j.biopsych.2010.01.027
65
FellowsL. K. (2011). Orbitofrontal contributions to value-based decision making: evidence from humans with frontal lobe damage. Ann. N Y Acad. Sci.1239, 51–58. 10.1111/j.1749-6632.2011.06229.x
66
FitzGeraldT. H.FristonK. J.DolanR. J. (2012). Action-specific value signals in reward-related regions of the human brain. J. Neurosci.32, 16417–16423. 10.1523/jneurosci.3254-12.2012
67
ForbesE. E.HaririA. R.MartinS. L.SilkJ. S.MoylesD. L.FisherP. M.et al. (2009). Altered striatal activation predicting real-world positive affect in adolescent major depressive disorder. Am. J. Psychiatry166, 64–73. 10.1176/appi.ajp.2008.07081336
68
FornitoA.HarrisonB. J.GoodbyE.DeanA.OoiC.NathanP. J.et al. (2013). Functional dysconnectivity of corticostriatal circuitry as a risk phenotype for psychosis. JAMA Psychiatry70, 1143–1151. 10.1001/jamapsychiatry.2013.1976
69
FrankM. J.ClausE. D. (2006). Anatomy of a decision: striato-orbitofrontal interactions in reinforcement learning, decision making and reversal. Psychol. Rev.113, 300–326. 10.1037/0033-295x.113.2.300
70
FrankM. J.SantamariaA.O’ReillyR. C.WillcuttE. (2007). Testing computational models of dopamine and noradrenaline dysfunction in attention deficit/hyperactivity disorder. Neuropsychopharmacology32, 1583–1599. 10.1038/sj.npp.1301278
71
FriedelE.SchlagenhaufF.SterzerP.ParkS. Q.BermpohlF.StröhleA.et al. (2009). 5-HTT genotype effect on prefrontal-amygdala coupling differs between major depression and controls. Psychopharmacology (Berl)205, 261–271. 10.1007/s00213-009-1536-1
72
GalynkerI. I.CaiJ.OngsengF.FinestoneH.DuttaE.SerseniD. (1998). Hypofrontality and negative symptoms in major depressive disorder. J. Nucl. Med.39, 608–612. 10.1016/0006-3223(96)84295-8
73
GardD. E.KringA. M.GardM. G.HoranW. P.GreenM. F. (2007). Anhedonia in schizophrenia: distinctions between anticipatory and consummatory pleasure. Schizophr. Res.93, 253–260. 10.1016/j.schres.2007.03.008
74
GerfenC. R.SurmeierD. J. (2011). Modulation of striatal projection systems by dopamine. Annu. Rev. Neurosci.34, 441–466. 10.1146/annurev-neuro-061010-113641
75
GhashghaeiH. T.HilgetagC. C.BarbasH. (2007). Sequence of information processing for emotions based on the anatomic dialogue between prefrontal cortex and amygdala. Neuroimage34, 905–923. 10.1016/j.neuroimage.2006.09.046
76
GillanC. M.PapmeyerM.Morein-ZamirS.SahakianB. J.FinebergN. A.RobbinsT. W.et al. (2011). Disruption in the balance between goal-directed behavior and habit learning in obsessive-compulsive disorder. Am. J. Psychiatry168, 718–726. 10.1176/appi.ajp.2011.10071062
77
GläscherJ.HamptonA. N.O’DohertyJ. P. (2009). Determining a role for ventromedial prefrontal cortex in encoding action-based value signals during reward-related decision making. Cereb. Cortex19, 483–495. 10.1093/cercor/bhn098
78
GoldJ. M.WaltzJ. A.PrenticeK. J.MorrisS. E.HeereyE. A. (2008). Reward processing in schizophrenia: a deficit in the representation of value. Schizophr. Bull.34, 835–847. 10.1093/schbul/sbn068
79
GoldJ. M.WaltzJ. A.MatveevaT. M.KasanovaZ.StraussG. P.HerbenerE. S.et al. (2012). Negative symptoms and the failure to represent the expected reward value of actions: behavioral and computational modeling evidence. Arch. Gen. Psychiatry69, 129–138. 10.1001/archgenpsychiatry.2011.1269
80
GoldsteinJ. M.GoodmanJ. M.SeidmanL. J.KennedyD. N.MakrisN.LeeH.et al. (1999). Cortical abnormalities in schizophrenia identified by structural magnetic resonance imaging. Arch. Gen. Psychiatry56, 537–547. 10.1001/archpsyc.56.6.537
81
GrantD. A.BergE. (1948). A behavioral analysis of degree of reinforcement and ease of shifting to new responses in a Weigl-type card-sorting problem. J. Exp. Psychol.38, 404–411. 10.1037/h0059831
82
GreenM. F. (1996). What are the functional consequences of neurocognitive deficits in schizophrenia?Am. J. Psychiatry153, 321–330.
83
GroenewegenH. J.WrightC. I.BeijerA. V.VoornP. (1999). Convergence and segregation of ventral striatal inputs and outputs. Ann. N Y Acad. Sci.877, 49–63. 10.1111/j.1749-6632.1999.tb09260.x
84
HammondL. J. (1980). The effect of contingency upon the appetitive conditioning of free-operant behavior. J. Exp. Anal. Behav.34, 297–304. 10.1901/jeab.1980.34-297
85
HareT. A.SchultzW.CamererC. F.O’DohertyJ. P.RangelA. (2011). Transformation of stimulus value signals into motor commands during simple choice. Proc. Natl. Acad. Sci. U S A108, 18120–18125. 10.1073/pnas.1109322108
86
HarveyP. O.FossatiP.PochonJ. B.LevyR.LeBastardG.LehéricyS.et al. (2005). Cognitive control and brain resources in major depression: an fMRI study using the n-back task. Neuroimage26, 860–869. 10.1016/j.neuroimage.2005.02.048
87
HatfieldT.HanJ. S.ConleyM.GallagherM.HollandP. (1996). Neurotoxic lesions of basolateral, but not central, amygdala interfere with Pavlovian second-order conditioning and reinforcer devaluation effects. J. Neurosci.16, 5256–5265.
88
HeereyE. A.GoldJ. M. (2007). Patients with schizophrenia demonstrate dissociation between affective experience and motivated behavior. J. Abnorm. Psychol.116, 268–278. 10.1037/0021-843x.116.2.268
89
HeereyE. A.Bell-WarrenK. R.GoldJ. M. (2008). Decision-making impairments in the context of intact reward sensitivity in schizophrenia. Biol. Psychiatry64, 62–69. 10.1016/j.biopsych.2008.02.015
90
HenriquesJ. B.DavidsonR. J. (2000). Decreased responsiveness to reward in depression. Cogn. Emot.14, 711–724. 10.1080/02699930050117684
91
HollandP. C.GallagherM. (2004). Amygdala-frontal interactions and reward expectancy. Curr. Opin. Neurobiol.14, 148–155. 10.1016/j.conb.2004.03.007
92
HoogmanM.AartsE.ZwiersM.Slaats-WillemseD.NaberM.OnninkM.et al. (2011). Nitric oxide synthase genotype modulation of impulsivity and ventral striatal activity in adult ADHD patients and healthy comparison subjects. Am. J. Psychiatry168, 1099–1106. 10.1176/appi.ajp.2011.10101446
93
HowesO. D.MontgomeryA. J.AsselinM. C.MurrayR. M.ValliI.TabrahamP.et al. (2009). Elevated striatal dopamine function linked to prodromal signs of schizophrenia. Arch. Gen. Psychiatry66, 13–20. 10.1001/archgenpsychiatry.2008.514
94
HuntL. T.WoolrichM. W.RushworthM. F.BehrensT. E. (2013). Trial-type dependent frames of reference for value comparison. PLoS Comput. Biol.9:e1003225. 10.1371/journal.pcbi.1003225
95
HuysQ. J.EshelN.O’NionsE.SheridanL.DayanP.RoiserJ. P. (2012). Bonsai trees in your head: how the Pavlovian system sculpts goal-directed choices by pruning decision trees. PLoS Comput. Biol.8:e1002410. 10.1371/journal.pcbi.1002410
96
HuysQ. J.PizzagalliD. A.BogdanR.DayanP. (2013). Mapping anhedonia onto reinforcement learning: a behavioural meta-analysis. Biol. Mood Anxiety Disord.3:12. 10.1186/2045-5380-3-12
97
JenisonR. L.RangelA.OyaH.KawasakiH.HowardM. A. (2011). Value encoding in single neurons in the human amygdala during decision making. J. Neurosci.31, 331–338. 10.1523/jneurosci.4461-10.2011
98
JohansenE. B.KilleenP. R.RussellV. A.TrippG.WickensJ. R.TannockR.et al. (2009). Origins of altered reinforcement effects in ADHD. Behav. Brain Funct.5:7. 10.1186/1744-9081-5-7
99
JonesJ. L.EsberG. R.McDannaldM. A.GruberA. J.HernandezA.MirenziA.et al. (2012). Orbitofrontal cortex supports behavior and learning using inferred but not cached values. Science338, 953–956. 10.1126/science.1227489
100
JuckelG.SchlagenhaufF.KoslowskiM.WüstenbergT.VillringerA.KnutsonB.et al. (2006a). Dysfunction of ventral striatal reward prediction in schizophrenia. Neuroimage29, 409–416. 10.1016/j.neuroimage.2005.07.051
101
JuckelG.SchlagenhaufF.KoslowskiM.FilonovD.WüstenbergT.VillringerA.et al. (2006b). Dysfunction of ventral striatal reward prediction in schizophrenic patients treated with typical, not atypical, neuroleptics. Psychopharmacology187, 222–228. 10.1007/s00213-006-0405-4
102
KahntT.ChangL. J.ParkS. Q.HeinzleJ.HaynesJ. D. (2012). Connectivity-based parcellation of the human orbitofrontal cortex. J. Neurosci.32, 6240–6250. 10.1523/jneurosci.0257-12.2012
103
KeatingC.TilbrookA. J.RossellS. L.EnticottP. G.FitzgeraldP. B. (2012). Reward processing in anorexia nervosa. Neuropsychologia50, 567–575. 10.1016/j.neuropsychologia.2012.01.036
104
KegelesL. S.Abi-DarghamA.FrankleW. G.GilR.CooperT. B.SlifsteinM.et al. (2010). Increased synaptic dopamine function in associative regions of the striatum in schizophrenia. Arch. Gen. Psychiatry67, 231–239. 10.1001/archgenpsychiatry.2010.10
105
KennerleyS. W.WaltonM. E.BehrensT. E.BuckleyM. J.RushworthM. F. (2006). Optimal decision making and the anterior cingulate cortex. Nat. Neurosci.9, 940–947. 10.1038/nn1724
106
KériS.JuhászA.RimanóczyÁ.SzekeresG.KelemenO.CimmerC.et al. (2005). Habit learning and the genetics of the dopamine D3 receptor: evidence from patients with schizophrenia and healthy controls. Behav. Neurosci.119, 687–693. 10.1037/0735-7044.119.3.687
107
KimJ. N.ShadlenM. N. (1999). Neural correlates of a decision in the dorsolateral prefrontal cortex of the macaque. Nat. Neurosci.2, 176–185. 10.1038/5739
108
KimS.HwangJ.LeeD. (2008). Prefrontal coding of temporally discounted values during intertemporal choice. Neuron59, 161–172. 10.1016/j.neuron.2008.05.010
109
Klein-FlüggeM. C.BarronH. C.BrodersenK. H.DolanR. J.BehrensT. E. J. (2013). Segregated encoding of reward-identity and stimulus-reward associations in human orbitofrontal cortex. J. Neurosci.33, 3202–3211. 10.1523/jneurosci.2532-12.2013
110
KonradK.EickhoffS. B. (2010). Is the ADHD brain wired differently? A review on structural and functional connectivity in attention deficit hyperactivity disorder. Hum. Brain Mapp.31, 904–916. 10.1002/hbm.21058
111
KringelbachM. L.O’DohertyJ.RollsE. T.AndrewsC. (2003). Activation of the human orbitofrontal cortex to a liquid food stimulus is correlated with its subjective pleasantness. Cereb. Cortex13, 1064–1071. 10.1093/cercor/13.10.1064
112
LauB.GlimcherP. W. (2007). Action and outcome encoding in the primate caudate nucleus. J. Neurosci.27, 14502–14514. 10.1523/jneurosci.3060-07.2007
113
LaurentV.LeungB.MaidmentN.BalleineB. W. (2012). µ-and δ-opioid-related processes in the accumbens core and shell differentially mediate the influence of reward-guided and stimulus-guided decisions on choice. J. Neurosci.32, 1875–1883. 10.1523/JNEUROSCI.4688-11.2012
114
LeeR. S. C.HermensD. F.PorterM. A.Redoblado-HodgeM. A. (2012). A meta-analysis of cognitive deficits in first-episode major depressive disorder. J. Affect. Disord.140, 113–124. 10.1016/j.jad.2011.10.023
115
LexB.HauberW. (2010). Disconnection of the entorhinal cortex and dorsomedial striatum impairs the sensitivity to instrumental contingency degradation. Neuropsychopharmacology35, 1788–1796. 10.1038/npp.2010.46
116
LiederF.GoodmanN. D.HuysQ. J. (2013). Learned helplessness and generalization. In Cognitive Science Conference. http://www.stanford.edu/~ngoodman/papers/LiederGoodmanHuys2013.pdf
117
LiljeholmM.TricomiE.O’DohertyJ. P.BalleineB. W. (2011). Neural correlates of instrumental contingency learning: differential effects of action-reward conjunction and disjunction. J. Neurosci.31, 2474–2480. 10.1523/jneurosci.3354-10.2011
118
LumanM.OosterlaanJ.SergeantJ. A. (2008). Modulation of response timing in ADHD, effects of reinforcement valence and magnitude. J. Abnorm. Child Psychol.36, 445–456. 10.1007/s10802-007-9190-8
119
MaierS. F.WatkinsL. R. (2005). Stressor controllability and learned helplessness: the roles of the dorsal raphe nucleus, serotonin and corticotropin-releasing factor. Neurosci. Biobehav. Rev.29, 829–841. 10.1016/j.neubiorev.2005.03.021
120
MaierS. F.WatkinsL. R. (2010). Role of the medial prefrontal cortex in coping and resilience. Brain Res.1355, 52–60. 10.1016/j.brainres.2010.08.039
121
MaloneD. A.Jr.DoughertyD. D.RezaiA. R.CarpenterL. L.FriehsG. M.EskandarE. N.et al. (2009). Deep brain stimulation of the ventral capsule/ventral striatum for treatment-resistant depression. Biol. Psychiatry65, 267–275. 10.1016/j.biopsych.2008.08.029
122
MartinD. J.AbramsonL. Y.AlloyL. B. (1984). Illusion of control for self and others in depressed and nondepressed college students. J. Pers. Soc. Psychol.46, 125–136. 10.1037//0022-3514.46.1.125
123
Martin-SoelchC.LinthicumJ.ErnstM. (2007). Appetitive conditioning: neural bases and implications for psychopathology. Neurosci. Biobehav. Rev.31, 426–440. 10.1016/j.neubiorev.2006.11.002
124
MatsumotoM.MatsumotoK.AbeH.TanakaK. (2007). Medial prefrontal cell activity signaling prediction errors of action values. Nat. Neurosci.10, 647–656. 10.1038/nn1890
125
MaybergH. S. (1997). Limbic-cortical dysregulation: a proposed model of depression. J. Neuropsychiatry Clin. Neurosci.9, 471–481.
126
McCabeC.CowenP. J.HarmerC. J. (2009). Neural representation of reward in recovered depressed patients. Psychopharmacology (Berl)205, 667–677. 10.1007/s00213-009-1573-9
127
McFarlandB. R.KleinD. N. (2009). Emotional reactivity in depression: diminished responsiveness to anticipated reward but not to anticipated punishment or to nonreward or avoidance. Depress. Anxiety26, 117–122. 10.1002/da.20513
128
MillerW. R.SeligmanM. E. (1975). Depression and learned helplessness in man. J. Abnorm. Psychol.84, 228–238. 10.1037/h0076720
129
MogensonG. J.YimC. C. (1991). “Neuromodulatory functions of the mesolimbic dopamine system: electrophysiological and behavioral studies,” in The Mesolimbic Dopamine System: From Motivation to Action, eds WillnerP.Scheel-KrugerJ. (New York: Wiley Press), 105–130.
130
MorrisJ. S.DolanR. J. (2001). Involvement of human amygdala and orbitofrontal cortex in hunger-enhanced memory for food stimuli. J. Neurosci.21, 5304–5310.
131
MorrisR. W.VercammenA.LenrootR.MooreL.LangtonJ. M.ShortB.et al. (2012). Disambiguating ventral striatum fMRI-related bold signal during reward prediction in schizophrenia. Mol. Psychiatry17, 280–289. 10.1038/mp.2011.75
132
MsetfiR. M.MurphyR. A.KornbrotD. E. (2012). Dysphoric mood states are related to sensitivity to temporal changes in contingency. Front. Psychol.3:368. 10.3389/fpsyg.2012.00368
133
MurrayG. K.ChengF.ClarkL.BarnettJ. H.BlackwellA. D.FletcherP. C.et al. (2008). Reinforcement and reversal learning in first-episode psychosis. Schizophr. Bull.34, 848–855. 10.1093/schbul/sbn078
134
Myers-SchulzB.KoenigsM. (2011). Functional anatomy of ventromedial prefrontal cortex: implications for mood and anxiety disorders. Mol. Psychiatry17, 132–141. 10.1038/mp.2011.88
135
NakanoK. (2000). Neural circuits and topographic organization of the basal ganglia and related regions. Brain Dev.22, 5–16. 10.1016/s0387-7604(00)00139-x
136
NakaoT.RaduaJ.RubiaK.Mataix-ColsD. (2011). Gray matter volume abnormalities in ADHD: voxel-based meta-analysis exploring the effects of age and stimulant medication. Am. J. Psychiatry168, 1154–1163. 10.1176/appi.ajp.2011.11020281
137
NestlerE. J.CarlezonW. A.Jr. (2006). The mesolimbic dopamine reward circuit in depression. Biol. Psychiatry59, 1151–1159. 10.1016/j.biopsych.2005.09.018
138
NoonanM. P.KollingN.WaltonM. E.RushworthM. F. S. (2012). Re-evaluating the role of the orbitofrontal cortex in reward and reinforcement. Eur. J. Neurosci.35, 997–1010. 10.1111/j.1460-9568.2012.08023.x
139
NoonanM. P.MarsR. B.RushworthM. F. S. (2011). Distinct roles of three frontal cortical areas in reward-guided behavior. J. Neurosci.31, 14399–14412. 10.1523/jneurosci.6456-10.2011
140
O’dohertyJ.RollsE. T.FrancisS.BowtellR.McGloneF.KobalG.et al. (2000). Sensory-specific satiety-related olfactory activation of the human orbitofrontal cortex. Neuroreport11, 893–897. 10.1097/00001756-200002070-00035
141
PaquetF.SoucyJ. P.StipE.LevesqueM.ElieA.BedardM. A. (2004). Comparison between olanzapine and haloperidol on procedural learning and the relationship with striatal D2 receptor occupancy in schizophrenia. J. Neuropsychiatry Clin. Neurosci.16, 47–56. 10.1176/appi.neuropsych.16.1.47
142
ParryP. A.DouglasV. I. (1983). Effects of reinforcement on concept identification in hyperactive children. J. Abnorm. Child Psychol.11, 327–340. 10.1007/bf00912095
143
PizzagalliD. A.HolmesA. J.DillonD. G.GoetzE. L.BirkJ. L.BogdanR.et al. (2009). Reduced caudate and nucleus accumbens response to rewards in unmedicated subjects with major depressive disorder. Am. J. Psychiatry166, 702–710. 10.1176/appi.ajp.2008.08081201
144
PizzagalliD. A.IosifescuD.HallettL. A.RatnerK. G.FavaM. (2008). Reduced hedonic capacity in major depressive disorder: evidence from a probabilistic reward task. J. Psychiatr. Res.43, 76–87. 10.1016/j.jpsychires.2008.03.001
145
PlichtaM. M.ScheresA. (2013). Ventral-striatal responsiveness during reward anticipation in ADHD and its relation to trait impulsivity in the healthy population: a meta-analytic review of the fMRI literature. Neurosci. Biobehav. Rev.38, 125–134. 10.1016/j.neubiorev.2013.07.012
146
PlichtaM. M.VasicN.WolfR. C.LeschK. P.BrummerD.JacobC.et al. (2009). Neural hyporesponsiveness and hyperresponsiveness during immediate and delayed reward processing in adult attention-deficit/hyperactivity disorder. Biol. Psychiatry65, 7–14. 10.1016/j.biopsych.2008.07.008
147
PurvisK. L.TannockR. (2000). Phonological processing, not inhibitory control, differentiates ADHD and reading disability. J. Am. Acad. Child Adolesc. Psychiatry39, 485–494. 10.1097/00004583-200004000-00018
148
QuanM.LeeS. H.KubickiM.KikinisZ.RathiY.SeidmanL. J.et al. (2013). White matter tract abnormalities between rostral middle frontal gyrus, inferior frontal gyrus and striatum in first-episode schizophrenia. Schizophr. Res.145, 1–10. 10.1016/j.schres.2012.11.028
149
QuidéY.MorrisR. W.ShepherdA. M.RowlandJ. E.GreenM. J. (2013). Task-related fronto-striatal functional connectivity during working memory performance in schizophrenia. Schizophr. Res.150, 468–475. 10.1016/j.schres.2013.08.009
150
RobinsonO. J.CoolsR.CarlisiC. O.SahakianB. J.DrevetsW. C. (2012). Ventral striatum response during reward and punishment reversal learning in unmedicated major depressive disorder. Am. J. Psychiatry169, 152–159. 10.1176/appi.ajp.2011.11010137
151
RollsB. J.RollsE. T.RoweE. A.SweeneyK. (1981). Sensory specific satiety in man. Physiol. Behav.27, 137–142. 10.1016/0031-9384(81)90310-3
152
RollsE. T.SienkiewiczZ. J.YaxleyS. (1989). Hunger modulates the responses to gustatory stimuli of single neurons in the caudolateral orbitofrontal cortex of the macaque monkey. Eur. J. Neurosci.1, 53–60. 10.1111/j.1460-9568.1989.tb00774.x
153
RudebeckP. H.MurrayE. A. (2011). Dissociable effects of subtotal lesions within the macaque orbital prefrontal cortex on reward-guided behavior. J. Neurosci.31, 10569–10578. 10.1523/JNEUROSCI.0091-11.2011
154
RushworthM. F.KollingN.SalletJ.MarsR. B. (2012). Valuation and decision-making in frontal cortex: one or many serial or parallel systems?. Curr. Opin. Neurobiol.22, 946–955. 10.1016/j.conb.2012.04.011
155
SagvoldenT.RussellV. A.AaseH.JohansenE. B.FarshbafM. (2005). Rodent models of attention-deficit/hyperactivity disorder. Biol. Psychiatry57, 1239–1247. 10.1016/j.biopsych.2005.02.002
156
SamejimaK.UedaY.DoyaK.KimuraM. (2005). Representation of action-specific reward values in the striatum. Science310, 1337–1340. 10.1126/science.1115270
157
SchacharR.TannockR.MarriottM.LoganG. (1995). Deficient inhibitory control in attention deficit hyperactivity disorder. J. Abnorm. Child Psychol.23, 411–437. 10.1007/bf01447206
158
SchererH.BedardM. A.StipE.PaquetF.RicherF.BériaultM.et al. (2004). Procedural learning in schizophrenia can reflect the pharmacologic properties of the antipsychotic treatments. Cogn. Behav. Neurol.17, 32–40. 10.1097/00146965-200403000-00004
159
ScheresA.MilhamM. P.KnutsonB.CastellanosF. X. (2007). Ventral striatal hyporesponsiveness during reward anticipation in attention-deficit/hyperactivity disorder. Biol. Psychiatry61, 720–724. 10.1016/j.biopsych.2006.04.042
160
SchifferW. K.VolkowN. D.FowlerJ. S.AlexoffD. L.LoganJ.DeweyS. L. (2006). Therapeutic doses of amphetamine or methylphenidate differentially increase synaptic and extracellular dopamine. Synapse59, 243–251. 10.1002/syn.20235
161
SchlaepferT. E.CohenM. X.FrickC.KoselM.BrodesserD.AxmacherN.et al. (2008). Deep brain stimulation to reward circuitry alleviates anhedonia in refractory major depression. Neuropsychopharmacology33, 368–377. 10.1038/sj.npp.1301408
162
SeoH.LeeD. (2007). Temporal filtering of reward signals in the dorsal anterior cingulate cortex during a mixed-strategy game. J. Neurosci.27, 8366–8377. 10.1523/jneurosci.2369-07.2007
163
SergeantJ. A.OosterlaanJ.van der MeereJ. (1999). “Information processing and energetic factors in attention-deficit/hyperactivity disorder,” in Handbook of Disruptive Behavior Disorders, eds QuayH. C.HoganA. E. (Dordrecht, Netherlands: Kluwer Academic Publishers), 75–104.
164
ShanksD. R.DickinsonA. (1991). Instrumental judgment and performance under variations in action-outcome contingency and contiguity. Mem. Cognit.19, 353–360. 10.3758/bf03197139
165
ShiflettM. W.BalleineB. W. (2011a). Molecular substrates of action control in cortico-striatal circuits. Prog. Neurobiol.95, 1–13. 10.1016/j.pneurobio.2011.05.007
166
ShiflettM. W.BalleineB. W. (2011b). Contributions of ERK signaling in the striatum to instrumental learning and performance. Behav. Brain Res.218, 240–247. 10.1016/j.bbr.2010.12.010
167
ShiflettM. W.BrownR. A.BalleineB. W. (2010). Acquisition and performance of goal-directed instrumental actions depends on ERK signaling in distinct regions of dorsal striatum in rats. J. Neurosci.30, 2951–2959. 10.1523/jneurosci.1778-09.2010
168
SilkT. J.VanceA.RinehartN.BradshawJ. L.CunningtonR. (2009). Structural development of the basal ganglia in attention deficit hyperactivity disorder: a diffusion tensor imaging study. Psychiatry Res.172, 220–225. 10.1016/j.pscychresns.2008.07.003
169
SimpsonE. H.KellendonkC.KandelE. (2010). A possible role for the striatum in the pathogenesis of the cognitive symptoms of schizophrenia. Neuron65, 585–596. 10.1016/j.neuron.2010.02.014
170
SmallD. M.GregoryM. D.MakY. E.GitelmanD.MesulamM.ParrishT. (2003). Dissociation of neural representation of intensity and affective valuation in human gustation. Neuron39, 701–711. 10.1016/s0896-6273(03)00467-7
171
SmallD. M.ZatorreR. J.DagherA.EvansA. C.Jones-GotmanM. (2001). Changes in brain activity related to eating chocolate from pleasure to aversion. Brain124, 1720–1733. 10.1093/brain/124.9.1720
172
SmoskiM. J.FelderJ.BizzellJ.GreenS. R.ErnstM.LynchT. R.et al. (2009). fMRI of alterations in reward selection, anticipation and feedback in major depressive disorder. J. Affect. Disord.118, 69–78. 10.1016/j.jad.2009.01.034
173
SolantoM. V. (2002). Dopamine dysfunction in AD/HD: integrating clinical and basic neuroscience research. Behav. Brain Res.130, 65–71. 10.1016/s0166-4328(01)00431-4
174
Sonuga-BarkeE. J. (2002). Psychological heterogeneity in AD/HD—a dual pathway model of behaviour and cognition. Behav. Brain Res.130, 29–36. 10.1016/s0166-4328(01)00432-6
175
Sonuga-BarkeE. J.FairchildG. (2012). Neuroeconomics of attention-deficit/hyperactivity disorder: differential influences of medial, dorsal and ventral prefrontal brain networks on suboptimal decision making?Biol. Psychiatry72, 126–133. 10.1016/j.biopsych.2012.04.004
176
StefanacciL.AmaralD. G. (2002). Some observations on cortical inputs to the macaque monkey amygdala: an anterograde tracing study. J. Comp. Neurol.451, 301–323. 10.1002/cne.10339
177
StevensA.SchwarzJ.SchwarzB.RufI.KolterT.CzekallaJ. (2002). Implicit and explicit learning in schizophrenics treated with olanzapine and with classic neuroleptics. Psychopharmacology (Berl)160, 299–306. 10.1007/s00213-001-0974-1
178
StoyM.SchlagenhaufF.SterzerP.BermpohlF.HägeleC.SuchotzkiK.et al. (2012). Hyporeactivity of ventral striatum towards incentive stimuli in unmedicated depressed patients normalizes after treatment with escitalopram. J. Psychopharmacol.26, 677–688. 10.1177/0269881111416686
179
StraussG. P.FrankM. J.WaltzJ. A.KasanovaZ.HerbenerE. S.GoldJ. M. (2011). Deficits in positive reinforcement learning and uncertainty-driven exploration are associated with distinct aspects of negative symptoms in schizophrenia. Biol. Psychiatry69, 424–431. 10.1016/j.biopsych.2010.10.015
180
StröhleA.StoyM.WraseJ.SchwarzerS.SchlagenhaufF.HussM.et al. (2008). Reward anticipation and outcomes in adult males with attention-deficit/hyperactivity disorder. Neuroimage39, 966–972. 10.1016/j.neuroimage.2007.09.044
181
TanakaS. C.BalleineB. W.O’DohertyJ. P. (2008). Calculating consequences: brain systems that encode the causal effects of actions. J. Neurosci.28, 6750–6755. 10.1523/jneurosci.1808-08.2008
182
Taylor TavaresJ. V.ClarkL.FureyM. L.WilliamsG. B.SahakianB. J.DrevetsW. C. (2008). Neural basis of abnormal response to negative feedback in unmedicated mood disorders. Neuroimage42, 1118–1126. 10.1016/j.neuroimage.2008.05.049
183
TremblayL. K.NaranjoC. A.CardenasL.HerrmannN.BustoU. E. (2002). Probing brain reward system function in major depressive disorder: altered response to dextroamphetamine. Arch. Gen. Psychiatry59, 409–416. 10.1001/archpsyc.59.5.409
184
TremblayL. K.NaranjoC. A.GrahamS. J.HerrmannN.MaybergH. S.HevenorS.et al. (2005). Functional neuroanatomical substrates of altered reward processing in major depressive disorder revealed by a dopaminergic probe. Arch. Gen. Psychiatry62, 1228–1236. 10.1001/archpsyc.62.11.1228
185
TricomiE. M.DelgadoM. R.FiezJ. A. (2004). Modulation of caudate activity by action contingency. Neuron41, 281–292. 10.1016/s0896-6273(03)00848-1
186
TrippG.AlsopB. (1999). Sensitivity to reward frequency in boys with attention deficit hyperactivity disorder. J. Clin. Child Psychol.28, 366–375. 10.1207/s15374424jccp280309
187
TrippG.WickensJ. R. (2008). Research review: dopamine transfer deficit: a neurobiological theory of altered reinforcement mechanisms in ADHD. J. Child Psychol. Psychiatry49, 691–704. 10.1111/j.1469-7610.2007.01851.x
188
TrippG.WickensJ. R. (2009). Neurobiology of ADHD. Neuropharmacology57, 579–589. 10.1016/j.neuropharm.2009.07.026
189
ValentinV. V.DickinsonA.O’DohertyJ. P. (2007). Determining the neural substrates of goal-directed learning in the human brain. J. Neurosci.27, 4019–4026. 10.1523/jneurosci.0564-07.2007
190
ValeraE. M.FaraoneS. V.MurrayK. E.SeidmanL. J. (2007). Meta-analysis of structural imaging findings in attention-deficit/hyperactivity disorder. Biol. Psychiatry61, 1361–1369. 10.1016/j.biopsych.2006.06.011
191
VolkowN. D.WangG. J.TomasiD.KollinsS. H.WigalT. L.NewcornJ. H.et al. (2012). Methylphenidate-elicited dopamine increases in ventral striatum are associated with long-term symptom improvement in adults with attention deficit hyperactivity disorder. J. Neurosci.32, 841–849. 10.1523/jneurosci.4461-11.2012
192
VolkowN. D.WangG. J.KollinsS. H.WigalT. L.NewcornJ. H.TelangF.et al. (2009). Evaluating dopamine reward pathway in adhd clinical implications. JAMA302, 1084–1091. 10.1001/jama.2009.1308
193
WadehraS.PruittP.MurphyE. R.DiwadkarV. A. (2013). Network dysfunction during associative learning in schizophrenia: increased activation, but decreased connectivity: an fMRI study. Schizophr. Res.148, 38–49. 10.1016/j.schres.2013.05.010
194
WagnerG.SinselE.SobanskiT.KöhlerS.MarinouV.MentzelH. J.et al. (2006). Cortical inefficiency in patients with unipolar depression: an event-related FMRI study with the Stroop task. Biol. Psychiatry59, 958–965. 10.1016/j.biopsych.2005.10.025
195
WallisJ. D.MillerE. K. (2003). Neuronal activity in primate dorsolateral and orbital prefrontal cortex during performance of a reward preference task. Eur. J. Neurosci.18, 2069–2081. 10.1046/j.1460-9568.2003.02922.x
196
WaltonM. E.BehrensT. E.BuckleyM. J.RudebeckP. H.RushworthM. F. (2010). Separable learning systems in the macaque brain and the role of orbitofrontal cortex in contingent learning. Neuron65, 927–939. 10.1016/j.neuron.2010.02.027
197
WaltzJ. A.GoldJ. M. (2007). Probabilistic reversal learning impairments in schizophrenia: further evidence of orbitofrontal dysfunction. Schizophr. Res.93, 296–303. 10.1016/j.schres.2007.03.010
198
WaltzJ. A.SchweitzerJ. B.GoldJ. M.KurupP. K.RossT. J.SalmeronB. J.et al. (2009). Patients with schizophrenia have a reduced neural response to both unpredictable and predictable primary reinforcers. Neuropsychopharmacology34, 1567–1577. 10.1038/npp.2008.214
199
WeickertT. W.TerrazasA.BigelowL. B.MalleyJ. D.HydeT.EganM. F.et al. (2002). Habit and skill learning in schizophrenia: evidence of normal striatal processing with abnormal cortical input. Learn. Mem.9, 430–442. 10.1101/lm.49102
200
WilbertzG.Tebartz van ElstL.DelgadoM. R.MaierS.FeigeB.PhilipsenA.et al. (2012). Orbitofrontal reward sensitivity and impulsivity in adult attention deficit hyperactivity disorder. Neuroimage60, 353–361. 10.1016/j.neuroimage.2011.12.011
201
WinstanleyC. A.TheobaldD. E.CardinalR. N.RobbinsT. W. (2004). Contrasting roles of basolateral amygdala and orbitofrontal cortex in impulsive choice. J. Neurosci.24, 4718–4722. 10.1523/jneurosci.5606-03.2004
202
WinstonJ. S.GottfriedJ. A.KilnerJ. M.DolanR. J. (2005). Integrated neural representations of odor intensity and affective valence in human amygdala. J. Neurosci.25, 8903–8907. 10.1523/jneurosci.1569-05.2005
203
de WitS.CorlettP. R.AitkenM. R.DickinsonA.FletcherP. C. (2009). Differential engagement of the ventromedial prefrontal cortex by goal-directed and habitual behavior toward food pictures in humans. J. Neurosci.29, 11330–11338. 10.1523/jneurosci.1639-09.2009
204
WodkaE. L.Mark MahoneE.BlanknerJ. G.Gidley LarsonJ. C.FotedarS.DencklaM. B.et al. (2007). Evidence that response inhibition is a primary deficit in ADHD. J. Clin. Exp. Neuropsychol.29, 345–356. 10.1080/13803390600678046
205
World Health Organization. (1992). The ICD-10 Classification of Mental and Behavioural Disorders: Clinical Descriptions and Diagnostic Guidelines (Vol. 1).Geneva: World Health Organization.
206
WunderlichK.DayanP.DolanR. J. (2012). Mapping value based planning and extensively trained choice in the human brain. Nat. Neurosci.15, 786–791. 10.1038/nn.3068
207
WunderlichK.RangelA.O’DohertyJ. P. (2009). Neural computations underlying action-based decision making in the human brain. Proc. Natl. Acad. Sci. U S A106, 17199–17204. 10.1073/pnas.0901077106
208
YanH.TianL.YanJ.SunW.LiuQ.ZhangY. B.et al. (2012). Functional and anatomical connectivity abnormalities in cognitive division of anterior cingulate cortex in schizophrenia. PLoS One7:e45659. 10.1371/journal.pone.0045659
209
YinH. H.OstlundS. B.KnowltonB. J.BalleineB. W. (2005). The role of the dorsomedial striatum in instrumental conditioning. Eur. J. Neurosci.22, 513–523. 10.1111/j.1460-9568.2005.04218.x
210
ZeebF. D.WinstanleyC. A. (2013). Functional disconnection of the orbitofrontal cortex and basolateral amygdala impairs acquisition of a rat gambling task and disrupts animals’ ability to alter decision-making behavior after reinforcer devaluation. J. Neurosci.33, 6434–6443. 10.1523/jneurosci.3971-12.2013
Summary
Keywords
goal-directed action, basal ganglia, amygdala, schizophrenia, ADHD, depression
Citation
Griffiths KR, Morris RW and Balleine BW (2014) Translational studies of goal-directed action as a framework for classifying deficits across psychiatric disorders. Front. Syst. Neurosci. 8:101. doi: 10.3389/fnsys.2014.00101
Received
18 February 2014
Accepted
09 May 2014
Published
26 May 2014
Volume
8 - 2014
Edited by
Dave J. Hayes, University of Toronto, Canada
Reviewed by
M. Gustavo Murer, Universidad de Buenos Aires, Argentina; Lesley K. Fellows, University of Oxford, UK
Copyright
© 2014 Griffiths, Morris and Balleine.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Bernard W. Balleine, Behavioural Neuroscience Laboratory, Brain and Mind Research Institute, University of Sydney, Level 6, 94 Mallett Street, Camperdown, Sydney, NSW 2050, Australia e-mail: bernard.balleine@sydney.edu.au
This article was submitted to the journal Frontiers in Systems Neuroscience.
Disclaimer
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.