Striatal Dopamine and the Interface between Motivation and Cognition

Brain dopamine has long been known to be implicated in the domains of appetitive motivation and cognition. Recent work indicates that dopamine also plays a role in the interaction between appetitive motivation and cognition. Here we review this work. Animal work has revealed an arrangement of spiraling connections between the midbrain and the striatum that subserves a mechanism by which dopamine can direct information flow from ventromedial to more dorsal regions in the striatum. In line with current knowledge about dopamine's effects on cognition, we hypothesize that these striato-nigro-striatal connections provide the basis for functionally specific effects of appetitive motivation on cognition. One implication of this hypothesis is that appetitive motivation can induce cognitive improvement or impairment depending on task demands.


IntroductIon
The ability to control our behavior requires our actions to be goaldirected, and our goals to be organized hierarchically. Goals can be defined at different levels: motivational goals (e.g., rewards), cognitive goals (e.g., task-sets), and action goals (e.g., stimulusresponse mappings). Thus, goal-directed behavior requires, among other things, the transformation of information about reward into abstract cognitive decisions, which in turn need to be translated into specific actions. The mechanisms underlying this hierarchy of goal-directed control are not well understood.
This paper focuses on the degree to which such goal-directed behavior is controlled by incentive motivation. We have restricted our discussion to the effects of appetitive motivation, while taking note of the wealth of evidence indicating that stimuli that activate the appetitive motivational system have an inhibitory influence on behavior that is controlled by the aversive motivational system (Konorsky, 1967;Dickinson and Balleine, 2002). Unlike aversive motivation, appetitive motivation refers to the state triggered by external stimuli that have rewarding properties and has been argued to have a general potentiating or enhancing effect on behavior and cognition (Dickinson and Balleine, 2002;Robbins and Everitt, 2003;Krawczyk et al., 2007;Pessoa, 2009;Jimura et al., 2010;Pessoa and Engelmann, 2010;. Its effects on behavior and cognition have been associated with changes in neurochemical activity, such as increases in dopamine signaling in the striatum (Lyon and Robbins, 1975;Ikemoto and Panksepp, 1999;Robbins and Everitt, 2003;Berridge, 2007). This observation is generally in keeping with proposals that dopamine plays an important role in reward-related effort (Salamone et al., 2007) and generalized activation/energization of behavior . It is also consistent with data suggesting that dopamine might direct information flow from ventromedial frontostriatal circuits, implicated in reward Our working hypothesis is grounded in (albeit preliminary) empirical evidence indicating opposite effects of both dopaminergic and motivational/affective state manipulations on cognitive flexibility and cognitive focusing, which have been argued to reflect distinct striatal and prefrontal brain regions respectively (Crofts et al., 2001;Bilder et al., 2004;Dreisbach and Goschke, 2004;Dreisbach, 2006;Hazy et al., 2006;Cools et al., 2007;Rowe et al., 2007;van Steenbergen et al., 2009;Cools and D'Esposito, 2011). Indeed current models highlight a role for dopamine, particularly in the striatum, in the flexible updating of current taskrepresentations (Hazy et al., 2006;Maia and Frank, 2011). The finding that appetitive motivation is associated with robust changes in dopamine levels particularly in the striatum, thus concurs with our hypothesis that appetitive motivation potentiates (at least some forms of) cognitive flexibility, perhaps even at the expense of cognitive focusing. Such a bias toward cognitive flexibility should be generally adaptive, given that motivational goals in the real world are not often readily available, thus requiring preparatory behavior that is flexible rather than focused (Baldo and Kelley, 2007).
Together these observations suggest that appetitive motivation acts to enhance cognition in a manner that is functionally specific, varying as a function of task demands, and that these functionally specific effects are mediated by dopamine. Clearly, as in the case of dopamine (Cools and Robbins, 2004;Cools et al., 2009), effects of appetitive motivation will vary not only as a function of task demands, but also as a function of the baseline state of the system. Thus both motivational and neurochemical state changes will have rather different effects in individuals with low and high baseline levels of motivation, consistent with the existence of multiple Yerkes Dodson "inverted U shaped" functions (Yerkes and Dodson, 1908;Cools and Robbins, 2004).
Let us briefly discuss the role of striatal dopamine in the two separate domains of motivation and cognitive control before addressing its role in their interaction.

dopamIne and appetItIve motIvatIon
The ventromedial striatum (VMS, including the nucleus accumbens) is highly innervated by mesolimbic dopaminergic neurons and is well known to be implicated in reward and motivation (Robbins and Everitt, 1992;Berridge and Robinson, 1998;Ikemoto and Panksepp, 1999;Schultz, 2002;Knutson and Cooper, 2005;Baldo and Kelley, 2007). Thus dopamine manipulations in the VMS affect performance on multiple paradigms thought to measure motivated behavior, including conditioned reinforcement, Pavlovian-instrumental transfer paradigms, effort-based decision making tasks, and progressive ratio schedules (Taylor and Robbins, 1984;Dickinson et al., 2000;Berridge, 2000, 2001;Parkinson et al., 2002). These experiments primarily reveal effects of dopamine on so-called preparatory conditioned responses, which are thought to reflect activation of a motivational system (Dickinson and Balleine, 2002), while leaving unaffected, or if anything, having the opposite effect on the more stereotypic patterns of consummatory responding (Robbins and Everitt, 1992;Baldo and Kelley, 2007). Thus administration of the indirect catecholamine enhancer amphetamine in the VMS of hungry rats potentiated locomotor excitement in the presence of food and increased lever pressing in response to, or in anticipation of a reward-predictive cue, while decreasing or leaving unaffected food intake as well as appetitive hedonic responses like taste reactivity (Taylor and Robbins, 1984;Bakshi and Kelley, 1991;Pecina et al., 1997;Berridge, 2000, 2001). Conversely, dopamine receptor blockade or dopamine lesions in the VMS reduced locomotor activity and cue-evoked incentive motivation for reward (Dickinson et al., 2000;Parkinson et al., 2002), while again leaving unaffected or even increasing food intake (Koob et al., 1978). These animal studies emphasize the importance of VMS dopamine in appetitive motivation and suggest that the hedonic or consummatory aspects of reward are likely mediated by a different, possible antagonistic system (Floresco et al., 1996;Robbins andEveritt, 1996, 2003;Berridge and Robinson, 1998;Ikemoto and Panksepp, 1999;Baldo and Kelley, 2007;Berridge, 2007;Phillips et al., 2007;Salamone et al., 2007; for similar suggestions in humans, see Aarts et al., 2010).
At first sight, this well-established observation provides apparently clear grounds for assuming that dopamine contributes to optimal reward-or goal-directed behavior. However, psychologists have also long recognized that there are multiple distinct components to the motivation of behavior (Konorsky, 1967;Dickinson and Balleine, 2002). Thus instrumental behavior is motivated not only by the goals that we set ourselves, but also by generalized drives and/or so-called Pavlovian "wanting," the latter two processes not necessarily always contributing to adaptive, optimized behavior. To clarify this point, it may help to consider the operational definition that psychologists have invoked for distinguishing instrumental behavior that is goal-directed from instrumental behavior that is not goal-directed, i.e., habitual (Dickinson and Balleine, 2002). Following this tradition, behavior is goal-directed only if it accords to two criteria; first, it has to be driven by knowledge about the contingency between the action and the outcome (as measured with contingency degradation tests); second, it has to be sensitive to changes in the value of the goal (as measured with outcome devaluation tests, involving for example selective satiety). Using these operational definitions, Dickinson and Balleine (2002) have established that Pavlovian conditioned stimuli that induce so-called "wanting" can modify instrumental behavior without accessing action-outcome representations, that is, in a manner that is not goal-directed. This is illustrated most clearly by the role of reward-predictive stimuli in compulsive craving for drugs of abuse or other targets of addiction, which of course almost always implicates dopamine dysfunction (Berridge and Robinson, 1998;Everitt and Robbins, 2005;Volkow et al., 2009). In keeping with this observation are suggestions that motivational influences on instrumental behavior by Pavlovian stimulus reinforcer contingencies might reflect modulation of well-established habits rather than of goal-directed behavior (Dickinson and Balleine, 2002). Data showing that dopamine D1/D2 receptor antagonists attenuated Pavlovian-instrumental transfer without affecting instrumental incentive learning (Dickinson et al., 2000) indeed suggested that dopamine might act through Pavlovian processes rather than through modifying action-outcome representations (Dickinson and Balleine, 2002).
In this context, it is perhaps not surprising that the effects of appetitive motivation on cognition that are mediated by dopamine are functionally specific, leading to cognitive improvement paradigm demonstrating that effects of dopamine D1/D2 receptor agonist administration to healthy young volunteers on flexibility (task-switching) and focusing (distractor-resistance) were accompanied by drug effects on the striatum and the PFC respectively (Cools et al., 2007).
In sum, dopamine's effects on cognition are known to be functionally specific rather than global, with opposite effects on cognitive flexibility and cognitive focusing. These opposite effects have been proposed to reflect modulation of distinct brain regions, with dopamine in the striatum playing a prominent role in a form of flexibility that involves shifting to well-established, i.e., "habitized" stimulus-response sets.

dopamIne and the motIvatIon-cognItIon InteractIon
So far we have seen that striatal dopamine's effect on motivated behavior is most prominent in terms of its preparatory component and that such preparatory effects can be maladaptive. This observation that dopamine's effect on motivation might have maladaptive consequences for behavior concurs with observations that effects of dopamine in the cognitive domain depend on task demands and associated neural systems, so that dopaminergic drugs can have detrimental as well as beneficial consequences for cognition. Together these insights have led to the speculation that incentive motivation might act to enhance cognitive performance by potentiating dopamine in the striatum in a manner that is functionally specific, i.e., restricted to a form of cognitive flexibility that involves shifting to well-established habits, and not extending to, or even at the expense of cognitive focusing. Below we review empirical evidence that address the different aspects of this working hypothesis.

evIdence from neuroanatomIcal studIes
Motivation-cognition interactions have long been proposed to reflect dopamine-dependent interfacing between different parallel frontostriatal circuits associated with motivation and cognition (Figure 1). For example, neuroanatomical studies in rats from the 1970s have suggested that activity in the dorsal striatum is modulated by activity in the ventral striatum via the dopaminergic cells in the substantia nigra (Nauta et al., 1978). Tracer experiments in non-human primates have revived this notion by revealing an arrangement of spiraling striato-nigro-striatal (SNS) connections between the dopaminergic cells in the midbrain and striatal regions that were defined on the basis of their frontal cortical input (Haber et al., 2000;Haber, 2003). Similar connections have been found in rodents (Ikemoto, 2007). The SNS connections are thought to direct information flow in a feed-forward manner via stepwise disinhibition of the ascending dopaminergic projections from the VMS (including the nucleus accumbens), via the dorsomedial striatum (DMS, caudate nucleus), to the dorsolateral striatum (DLS, putamen). The resulting information flow from ventromedial to dorsolateral striatal regions provides a hierarchical (or heterarchical, see Haruno and Kawato, 2006) mechanism by which motivational goals can influence cognitive and subsequent motor control processes. Indeed, the VMS has long been hypothesized to provide the basis for the interface between motivation and action on the basis of its major inputs from limbic areas like the amygdala, hippocampus and the anterior cingulate cortex (ACC) and output to the motor areas or cognitive impairment depending on the specific task demands under study. An important implication of this observation is that effects of dopamine on interactions between motivation and cognitive control that appear to be mediated by a modification of motivational influences on cognitively mediated, goal-directed behavior may in fact reflect modification of motivational influences on habitual behavior.

dopamIne and cognItIon
Accumulating evidence in the domain of cognition indicates that manipulations of dopamine can have contrasting effects as a function of task demands. For example, opposite effects have been observed in terms of cognitive flexibility and cognitive focusing (Crofts et al., 2001;Bilder et al., 2004;Cools et al., 2007;Durstewitz and Seamans, 2008;Durstewitz et al., 2010;Cools and D'Esposito, 2011). Mehta et al. (2004) have shown that dopamine D2 receptor blockade after acute administration of the antagonist sulpiride impaired cognitive flexibility (measured in terms of taskswitching), but improved cognitive focusing (measured in terms of delayed response performance with task-irrelevant distractors). Similar contrasting effects on cognitive flexibility and focusing have been reported after dopamine lesions in non-human primates (Roberts et al., 1994;Collins et al., 2000;Crofts et al., 2001), after dopaminergic medication withdrawal in patients with Parkinson's disease (PD; Cools et al., 2001aCools et al., , 2003Cools et al., , 2010a and as a function of genetic variation in human dopamine genes (Bilder et al., 2004;Colzato et al., 2010). Evidence from functional neuroimaging and computational modeling work has suggested that these opposite effects might reflect modulation of distinct brain regions, with the striatum mediating effects on at least some forms of cognitive flexibility, but the prefrontal cortex (PFC) mediating effects on cognitive focusing (Hazy et al., 2006;Cools et al., 2007;Cools and D'Esposito, 2011). This hypothesis likely reflects an oversimplified view of dopamine's complex effects on cognition, with different forms of cognitive flexibility implicating distinct neural and neurochemical systems (Robbins and Arnsten, 2009;Kehagia et al., 2010;Floresco and Jentsch, 2011). In particular, the striatum seems implicated predominantly in a form of cognitive flexibility that involves shifting to well-established ("habitized") stimulus-response sets, that does not require new learning or working memory. For example 6-OHDA lesions in the striatum of marmosets impaired set-shifting to an already established set, but left unaffected set-shifting to a new, to-be-learned set (Collins et al., 2000). This finding paralleled the beneficial effects of dopaminergic medication in PD, which implicates primarily the striatum. These effects were restricted to task-set switching between well-established sets, and did not extend to set shifting to new, to-be-learned sets (Cools et al., 2001b;Lewis et al., 2005;Slabosz et al., 2006). The PFC might well be implicated in higher-order forms of set shifting that do involve new learning and/or working memory (Monchi et al., 2004;Floresco and Magyar, 2006;Cools et al., 2010b;Kehagia et al., 2010). Interestingly, the beneficial effects of dopaminergic medication in PD on this striatal form of well-established, habit-like task-set switching were accompanied by detrimental effects on cognitive focusing, as measured in terms of distractor-resistance during the performance of a delayed response task (Cools et al., 2010a). These findings paralleled pharmacological neuroimaging work with the same delayed response sensitivity of neuronal firing in the DMS as well as midbrain dopamine neurons to appetitive motivation. In this task, one of four directions was randomly assigned as the target location by a cue that also signaled the anticipation of reward. Subsequently, the monkey had to make a saccade to the remembered location. It was found that cues that predicted reward resulted in earlier and faster saccades relative to cues that predicted no reward. Firing patterns in caudate nucleus (DMS) neurons correlated with the change in saccade behavior, changing their preferred direction to the rewarded direction (Kawagoe et al., 1998). In a follow-up study, the authors observed that reward-predictive cues resulted in increased firing of dopaminergic neurons in the midbrain, as well as in neurons of the caudate nucleus (DMS; Kawagoe et al., 2004). Together, these findings demonstrate that effects of reward anticipation on DMS activity and associated motor-planning behavior were accompanied by changes in dopamine activity.
In humans, a role for dopamine in the effects of motivation on cognition has so far been addressed only in the domain of long-term memory associated with the hippocampus (Wittmann et al., 2005;Adcock et al., 2006;Schott et al., 2006; for a review, see Shohamy and Adcock, 2010). This relatively young field suggests that dopamine may well play a role in the long-term plasticity-enhancing effects of motivation. In the next section, we address studies that focus on dopamine-dependent effects of motivation on shorter term plasticity, involving the striatum.

evIdence from human studIes: motIvatIon and cognItIve flexIbIlIty
Data from two recent studies support the hypothesis that dopamine is critical for interactions between motivation and cognition. Specifically, these studies highlight an important role for dopamine in the modification by appetitive motivation of switching between well-established habits. The set-shifting paradigm involved cued task-switching between well-learnt task-sets, minimizing learning and working memory processes (Rogers and Monsell, 1995). Subjects switched between responding according to the direction of the arrow (task A) and responding according to the direction indicated by the word (task B) of a series of arrow-word targets (consisting of the words "left" or "right" in a left or right pointing arrow; Figure 2A). Repetitions or switches of task-set were pseudorandomly preceded by high or low reward cues. In the first study, young healthy adults performed the task in the magnetic resonance scanner and both behavioral and neural responses were assessed as a function of inter-individual variability in dopamine genes (Aarts et al., 2010). In particular, we focused on a common variable number of tandem repeats (VNTR) polymorphism in the dopamine transporter gene (DAT1), expressed predominantly in the striatum. Relative to the 10R homozygotes, the 9R carriers -with presumably increased striatal dopamine levels -exhibited significant reward benefits in terms of overall performance and increased rewardrelated BOLD responses in VMS. However, most critically, they also demonstrated significant reward benefits in terms of task-switching (i.e., reduced switch costs in the high versus low reward condition). This effect was accompanied by a potentiation of switch-related BOLD responses in DMS (caudate nucleus) in the high reward versus the low reward condition (Figures 2B,C). Importantly, the reward-related activity in VMS correlated positively with the effects via the globus pallidus (Mogenson et al., 1980;Groenewegen et al., 1996). However, rather than a direct limbic-motor connection, the SNS connections provide a more physiologically and psychologically plausible mechanism by which motivational goals exert their influence on action (Haber et al., 2000).

evIdence from psychopharmacologIcal studIes In anImals
Rodent research on drug addiction has provided evidence for the functional importance of dopamine-mediated interactions between ventral and dorsal parts of the striatum. For example, Belin and Everitt (2008) have adopted an intrastriatal disconnection procedure in rats to investigate the necessity of the SNS connections in the transition of reward-directed drug-seeking behavior to habitual behavior associated with the DLS. The authors lesioned the VMS selectively on one side of the rat brain and, concomitantly, blocked dopaminergic input from the substantia nigra in the DLS with a receptor antagonist on the contralateral side of the brain. Thus, they functionally disconnected the VMS and DLS on both sides of the brain, while leaving unilateral VMS and DLS on opposite sites intact. This functional disconnection between VMS and DLS greatly reduced the transition of VMS-associated to DLS-associated habitual behavior, whereas the unilateral manipulations were ineffective in isolation (Belin and Everitt, 2008). These data show the functional importance of the spiraling SNS connections in VMS control over dorsal striatal functioning in addiction (Belin et al., 2009).
Functional evidence for a role of dopamine in interactions between motivation and DMS-associated functions has also been established in non-human primates. For example, neurophysiological recordings by Hikosaka and colleagues during the performance of a memory-guided saccadic eye-movement task revealed with the degree of dopamine depletion in different striatal subregions as measured with 123 I-FP-CIT single photon emission computed tomography (SPECT). First, we replicated previous studies by demonstrating a switch deficit in PD relative to healthy controls. Interestingly, this deficit was restricted to certain conditions of the task, revealing a disproportionate difficulty with switching to the best established, most dominant "arrow" task. Additionally, the SPECT measurements showed that this switch deficit in PD was associated with dopamine cell loss in the most affected striatal sub-region (posterior putamen, Figure 2E), thus demonstrating the involvement of striatal dopamine in this particular "habit-like" of reward on subsequent switch-related activity during the targets in DMS, with high dopamine subjects demonstrating high activity in both striatal regions ( Figure 2D; Aarts et al., 2010). These dopamine-mediated motivation-cognition interaction effects were recently replicated in an independent dataset (van Holstein et al., 2011) and strengthened our working hypothesis that striatal dopamine mediates motivational modification of certain cognitive functions in humans.
In a second study, we investigated the effect of appetitive motivation on cognitive flexibility in patients with PD using the same paradigm (Figure 2A). Effects within the PD group were associated type of cognitive flexibility. More critically, our results demonstrated compensatory capacity of reward-predictive signals to facilitate cognitive flexibility in mild PD. Specifically, when anticipating reward, patients were able to reduce the switch cost in the dominant arrow task to such an extent that the switch cost no longer differed from that of controls on high reward trials. Interestingly, the use of reward was also highly correlated with the amount of dopamine depletion in the most affected striatal sub-region (Aarts et al., under review). Patients with greater dopamine cell loss made more use of anticipated reward for reducing the switch cost than did patients with less dopamine cell loss ( Figure 2F). Further exploration of this finding demonstrated that this effect of motivation on task-switching was driven by two opponent processes: first, patients with more dopamine depletion made more errors on repeat trials under high than under low reward. This detrimental effect of reward on repeat trials could reflect a form of impulsivity, where the current task representation is rendered unstable by reward, leading to reduced cognitive "perseverance" or maintenance (see also Hazy et al., 2006). Controls did not show such detrimental impulsive behavior on repeat trials under high reward. Second, patients with more dopamine depletion made fewer errors on switch trials under high than under low reward. Thus, anticipated reward proved beneficial for switching to the other task-set, which profits from reduced cognitive perseverance. This effect of reward on switch trials in patients did not differ from that of controls. The beneficial effects of anticipated reward on task-switching in the young healthy adults mentioned above (Aarts et al., 2010) was driven by a beneficial effect of reward on switch trials only, instead of opposite effects of reward on repeat and switch trials. In sum, PD patients differed from controls in showing detrimental effects of reward on repeat trials, which were greatest in patients with most dopamine cell loss in the striatum (Aarts et al., under review). This result fits with previous findings that a low baseline dopamine state contributes to trait impulsivity and addictive behavior (Cools et al., 2007;Dalley et al., 2007); presumably due to reduced auto-regulatory mechanisms, resulting in increased dopamine release (Buckholtz et al., 2010). Hence, we speculate that reward-induced impulsivity in our PD group was caused by increased reward-related dopamine release in the relatively intact dopamine cells projecting to the ventral striatum ( Figure 2E). In line with this view are the findings of increased dopamine release in ventral striatum in PD patients diagnosed with impulsive-compulsive behavior relative to those without (Evans et al., 2006;Steeves et al., 2009;O'Sullivan et al., 2011). Our PD data are also in accordance with the working hypothesis that striatal dopamine mediates motivational effects on cognition depending on task demands.

evIdence from human studIes: functIonally specIfIc effects of motIvatIon
Motivation has been shown to improve attentional processes in many perceptual and cognitive control domains (for reviews, see Pessoa, 2009;Pessoa and Engelmann, 2010). Data from a number of human imaging studies have suggested that motivation might have non-specific enhancing effects on cognitive processing. For example, in a functional neuroimaging study, motivational incentives increased PFC activity and connectivity during cognitive control tasks, in a manner that seemed to depend on the cognitive effort (i.e., cost-benefit ratio) rather than on the specific qualitative cognitive demand of the tasks (Kouneiher et al., 2009). Based on these data the authors argued that motivation and cognitive control can be regarded as two separate, additive instead of interactive factors of executive functioning (Kouneiher et al., 2009). However, such an additive view of motivation and cognition contrasts with the conclusion drawn by a different set of recent studies which enabled the disentangling of different cognitive control components. These studies have found that effects of appetitive motivation and affect may well depend on the type of cognitive processing at hand (Dreisbach and Goschke, 2004;Dreisbach, 2006;Rowe et al., 2007), consistent with our working hypothesis. Before turning to these studies, we will discuss preliminary data from our own lab.
So far we have seen that appetitive motivation can potentiate certain forms of task-switching to well-established stimulus-response mappings in a dopamine-dependent manner. The observation that these effects were driven by detrimental effects of anticipated reward on repeat trials and beneficial effects on switch trials in the PD group (Aarts et al., submitted) already indicates a level of functional specificity. To test more directly the hypothesis that these beneficial effects of appetitive motivation on some cognitive functions might come at the expense of impairments on other cognitive functions, we designed a Stroop-like conflict task with high and low reward conditions. This task resembled the previously used task-switching paradigm in many ways except that it required cognitive focusing instead of cognitive switching. Seventeen participants performed this Stroop-like task by responding with a left or right button press to the words "left" or "right" in a left or right pointing arrow (Figure 3A). The direction denoted by the word was either congruent or incongruent with the direction indicated by the arrow. Similar to the task-switching paradigm discussed above (Aarts et al., under review), all trials began with a cue predicting high or low reward for correct performance. Critically, following the reward cues, we explicitly informed participants about the (in)congruency of the upcoming Stroop target (see Aarts et al., 2008). In half of the trials, participants were informed about this congruency by informative cues (Figure 3A). In the other half of the trials, the targets were preceded by cues that gave no information about the upcoming congruency. The idea here was that incongruency-predictive cues (relative to non-informative cues) would encourage participants to reduce their attentional focus, whereas the congruency-predictive cues would encourage participants to widen their attentional focus. In other words, cues that signaled upcoming incongruent targets would encourage participants to proactively focus on the task-relevant word, preventing distraction by the task-irrelevant arrow, whereas cues that signal upcoming congruent words encouraged participants to proactively widen attention in order to comprise both the task-relevant word as well as the task-irrelevant arrow (see Aarts and Roelofs, 2010). The combination of reward and information cues enabled us to determine the effects of appetitive motivation on the cognitive focusing of attention.
Consistent with our previous results (Aarts et al., 2008) we showed that (irrespective of reward condition) participants responded faster and made less errors when informative cues preceded the congruent and incongruent targets relative to uninformed changed (Dreisbach, 2006;van Wouwe et al., 2011), but, within the same task, positive affect decreased the ability to maintain the goal when nothing changed (Dreisbach, 2006). Functionally specific effects of positive affect have also been demonstrated in conflict paradigms, like the Eriksen flanker task. Some authors have shown that positive affect increased attention toward the distracting flanker arrows, thus, increasing "the breadth of attentional selection" (Rowe et al., 2007); others have found that positive affect reduced the ability to focus on the target arrow after experienced conflict (van Steenbergen et al., 2010). Our preliminary results from the rewarded Stroop conflict paradigm extend these effects of positive affect in the flanker conflict task, by revealing contrasting effects of appetitive motivation on the widening and focusing of attention within the same task and within the same participants. In sum, both appetitive motivation and positive affect enhance certain forms of cognitive flexibility at the expense of cognitive focusing. According to our working hypothesis, these effects might reflect dopamine-dependent flow of information processing related to Pavlovian incentives from ventromedial parts of the striatum to more dorsal regions in the striatum, associated with habit-like information processing.
It might be noted here again that multiple mechanisms have been proposed to underlie the motivational control of behavior (Dickinson and Balleine, 2002). We have highlighted that some motivational influences can be maladaptive, and these might implicate dopamine. However, there is also evidence for motivational influences on goal-direct behavior, that is, those mediated by instrumental incentive learning and acquisition of actionoutcome representations (Dickinson and Balleine, 2002). These alternate mechanisms might account for findings that at first sight seem incompatible with the current working hypothesis. Specifically, appetitive motivation has been shown to increase spatial orienting to a target location in the face of distractors (Engelmann and Pessoa, 2007;Engelmann et al., 2009), or to reduce conflict by biasing visual selection (Padmala and Pessoa, 2011). Furthermore, in young and old adults as well as in medicated patients with PD, motivation increased anti-saccade performance, encompassing incompatible stimulus-response mappings like in Stroop and flanker paradigms (Harsay et al., 2010). The critical question is whether these effects are also dependent on striatal dopamine, or whether they implicate modulation by different neurochemical systems. Addressing this question requires controlled dopaminergic medication withdrawal and/or pharmacological manipulation approaches.

frontal control of dopamIne-dependent strIatal processIng
The striatum does not act alone and requires interactions with specific frontal regions to operate effectively (Alexander et al., 1986;Passingham, 1993; Figure 1). Recent neuroimaging work in humans and monkeys has revealed that effects of appetitive motivation on cognitive control are accompanied by modulation of responses in the PFC (Ichihara-Takeda and Funahashi, 2008;Kouneiher et al., 2009;Beck et al., 2010;Ichihara-Takeda et al., 2010;Jimura et al., 2010;Wallis and Kennerley, 2010). For example, functional interactions between the medial and the lateral PFC have been shown to accompany effects of appetitive targets (M. van Holstein, E. Aarts, R. Cools, unpublished observations). Importantly, as predicted, appetitive motivation significantly altered the information benefit depending on the congruency of the targets. That is, proactive widening of attention (uninformedinformed congruent targets) benefited from anticipated reward (15 versus 1 cent), whereas proactive focusing of attention (uninformed-informed incongruent targets) was hampered by anticipated reward ( Figure 3B). Intriguingly, these data show that, depending on the task at hand, appetitive motivation can have both beneficial as well as detrimental effects on cognitive function. Similar findings have been obtained when studying the effects of positive affect on cognitive control. Thus, positive affect has been shown to increase cognitive flexibility (i.e., decreasing perseveration), while increasing distractibility (i.e., decreasing cognitive stability) on different types of trials in a set-shifting paradigm (Dreisbach and Goschke, 2004). Opposite effects have also been observed in an AX continuous performance task: positive affect increased cognitive flexibility when a maintained goal unexpectedly  An important implication of this observation is that effects of dopamine on interactions between motivation and cognition that appear to be mediated by a modification of motivational influences on cognitively mediated, goal-directed behavior, like task-switching, may in fact reflect modification of motivation influences on habitual behavior. Findings that the dopamine-dependent effects of motivation on task-switching are strongest when participants are required to switch to well-established stimulus-response mappings are in line with this hypothesis, which requires testing in future work.
A further issue to be addressed in future research is the degree to which the contrasting effects of motivation on habit-like switching and on proactive focusing can be understood in terms of competition between a striatal system controlling habit-like processing and a prefrontal system controlling goal-directed behavior (Dickinson, 1985;Daw et al., 2005). Clearly these questions require a careful integration of traditional psychological approaches, which leverage well-operationalized behavioral definitions of goal-directed and habitual behavior, with pharmacological studies of cognitive control.
Furthermore, given the proposed opponency between appetitive and aversive motivational systems, one might ask what is the effect of punishment-predictive stimuli on cognition? This is particularly interesting in the context of empirical findings that conditioned inhibitors, i.e., stimuli predictive of reward omission do not trigger an increase, but rather if anything a decrease in midbrain dopamine firing (Tobler et al., 2005). Moreover, there is increasing speculation about the involvement of the part-opponent system of serotonin (Daw et al., 2002;Dayan and Huys, 2009;Boureau and Dayan, 2011;, an area that is wide open for empirical work. Finally, progress in the understanding of the motivational control of cognition will depend on the degree to which the balance between transient and sustained, e.g., context effects are taken into account (e.g., Higgins et al., 1997;Maddox and Markman, 2010;. For example, Maddox and Markman (2010) propose that performance does not only depend on local incentives and task demands (as discussed in the current review), but also interacts with global incentives like an overall bonus or punishment at the end of a task. Such advances will no doubt benefit from the recognition that the impact of transient (phasic) changes in neurotransmitter activity depends critically on the tonic neurochemical state of the system. acknowledgments Roshan Cools is supported by a fellowship of the Hersenstichting Nederland for dementia research [F2008 (1)-01], a Vidi Grant from the Innovational Research Incentives Scheme of the Netherlands Organization for Scientific Research and a Human Frontiers Research Program grant. Esther Aarts is supported by a stipend of the Niels Stensen foundation. motivation on the cognitive control processes involved in set shifting (Kouneiher et al., 2009). Another functional neuroimaging study concluded that the lateral PFC incorporates reward value in goal-directed control during working memory processes .
These data concur with the existence of multiple mechanisms for the motivational control of behavior, which may interact in multiple ways, either competitively or synergistically. For example, signals in the PFC might control dopaminergic activity in striatal areas in a top-down manner, thus allowing controlled influences on value assignment to states or actions (Daw et al., 2005;Doll et al., 2009; see Figure 1). Consistent with this hypothesis are observations that stimulation of different parts of the frontal cortex (using transcranial magnetic stimulation) alters focal dopamine release in strongly connected topographically specific parts of the striatum (as measured using [11C]raclopride positron emission tomography; Strafella et al., 2001Strafella et al., , 2003Strafella et al., , 2005Ko et al., 2008). The role of the PFC in integrating motivation, cognition, and action is also highlighted by anatomical tracer studies in non-human primates showing that value-sensitive regions in ventromedial PFC (i.e., ACC/orbitofrontal cortex) project not only to strongly connected regions in VMS, but also diffusely to more dorsal regions in the striatum that receive most projections from the DLPFC (Haber et al., 2006; Figure 1). Electrophysiological work with rodents has revealed that changes in dopamine release and receptor stimulation in the striatum can alter such PFC input to the striatum (Goto and Grace, 2005). More specifically, changes in tonic dopamine release were shown to modulate PFC inputs into the VMS -and to influence set-shifting behavior -through dopamine D2 receptors (Goto and Grace, 2005). These results show that striatal dopamine can modulate motivated behavior not only via altering striatal output but also via altering striatal input from the PFC.

conclusIon and future dIrectIons
There are multiple mechanisms for the control of behavior and cognition by motivation. This paper focuses on the appetitive motivational system, while recognizing that opponent influences on behavior are likely seen of the aversive motivational system. In particular, we have concentrated on those effects of appetitive motivation that implicate dopamine. These dopamine-dependent effects of motivation likely have both detrimental as well as beneficial consequences for cognition, via altering information flow from ventromedial to dorsolateral parts of the striatum. This general observation is in line with the observation that motivational influences on behavior are not necessarily driven by representations of the goals of instrumental behavior, but might well reflect Pavlovian or habit-like anomalies. This is particularly likely in the case of dopamine, which is recognized to play a special role in Pavlovian and habit systems.