The basal ganglia, the ideal machinery for the cost-benefit analysis of action plans

Introduction

The basal ganglia (BG) are subcortical structures implicated in various neurological disorders including Parkinson’s disease, Huntington’s disease, obsessive-compulsive disorder, schizophrenia, and addictions (Steiner and Tseng, 2010). An understanding of BG circuits is essential for developing effective treatments for these disorders and for unraveling the neural basis of motor control, habit-formation, decision-making, and reinforcement learning. The BG contain two parallel circuits, so called direct and indirect pathways. Imbalanced activity of the two pathways has been linked to Parkinson’s disease and Huntington’s disease (Albin et al., 1989; Richfield et al., 1995; Mallet et al., 2006). However, the functional role of the two pathways is still under debate. The recent advent of cell-type specific intervention and recording techniques invigorated efforts to dissect the two pathways in greater detail. Unfortunately, such cutting-edge studies have so far reported confusing discoveries.

For instance, optogenetic activation of the indirect pathway caused bradykinesia (e.g., increased freezing and reduced locomotion), whereas activation of the direct pathway caused the opposite (Kravitz et al., 2010). This and similar causative studies support the prevailing classical model in the field that the direct pathway facilitates movements, whereas the indirect pathway inhibits movements (Durieux et al., 2009; Bateup et al., 2010). In contrast, optical recording of neuronal activity detected concurrent activation in the two pathways during normal movement, challenging the classical model which postulates less activity in the indirect pathway during movement than during rest (Cui et al., 2013). Perhaps even more confounding, repetitive optogenetic activation of the direct pathway following a voluntary movement reinforced that movement, whereas animals avoided movements that were followed by indirect pathway activation, emphasizing dissociative roles of the two pathways in reinforcement learning (Kravitz et al., 2012).

The growing influx of such conflicting and disconnected experimental results demands a new unifying model for the functional role of the two pathways, with the specificity that is necessary to make novel testable predictions and guide future research.

Action Selection

Of the many alternatives to the classical BG model (Mink, 1996; Gillies and Arbuthnott, 2000; Gurney et al., 2004), action selection models are the most prominent. It is well accepted that the anatomical architecture of the BG is fit for the function of selection. The BG receive massive inputs from various cortical and subcortical areas, process these inputs, and return the processed information back to where the inputs originated (Alexander et al., 1986; Alexander and Crutcher, 1990). Action selection models propose that the massive inputs contain competing action plans, the signal processing in the BG determines the re-entrant feedback signal for each action plan, and the action plan with facilitating feedback signal survives, while the others perish. In this view, the BG are a sophisticated action selection device rather than a gross movement generator or brake (Mink, 1996; Redgrave et al., 1999; Hikosaka et al., 2000; Frank, 2011).

Before introducing an elaborated model of action selection, it is necessary to review the principal synaptic connections in the BG. The main input structure of the BG, the striatum, receives glutamatergic, excitatory inputs from the cortex (Figure 1A). BG output nuclei send their GABAergic, inhibitory projections to the thalamic nuclei, which then send glutamatergic projections to primarily the same cortical areas from which the cortico-striatal inputs originated. Because BG output neurons have high spontaneous baseline activity, the thalamic target nuclei are normally inhibited. The excitatory cortical signals entering the BG propagate through direct and indirect pathways. The direct pathway consists of one GABAergic connection from the striatum to the output nuclei. The indirect pathway via two other nuclei consists of two GABAergic and one glutamatergic connections. Thus, direct pathway activation suppressing the activity of BG output neurons disinhibits the thalamic target, whereas indirect pathway activation intensifying output activity suppresses the target.

FIGURE 1

Figure 1. (A) The cortico-basal ganglia-cortico loop (Albin et al., 1989). Most cortical areas send excitatory projections to the striatum. The striatal projection neurons expressing D2 dopamine receptors transmit the cortical signal indirectly to the internal globus pallidus (GPi)/substantia nigra pars reticulate (SNr) via the external globus pallidus (GPe) and subthalamic nucleus (STN). The striatal projection neurons expressing D1 dopamine receptors transmit the cortical signal directly to the GPi/SNr. The GPi/SNr sends inhibitory projections to the thalamic nuclei, which then send excitatory projections back to the cortex. The red lines indicate excitatory connections, and the blue lines indicate inhibitory connections. Note that only principal pathways and not all identified connections are shown. (B) The cost-benefit model. An illustration of a simple scenario in which two action plans, A and B, compete. Two cortico-basal ganglia-cortico loops that are topologically organized, each linking a particular cortical ensemble with a particular set of basal ganglia neurons, operate in parallel. In loop A, (1) action plan A is represented by a set of intra-telencephalic (IT) and pyramidal tract (PT) neurons in the cortex, (2) indirect pathway neurons, preferentially processing PT neuronal input, produce the expected cost of plan A, and direct pathway neurons, preferentially processing IT neuronal input, produce the expected benefit of plan A, and (3) the basal ganglia output neurons combine the two pathways in a subtractive manner to represent the net cost-benefit of plan A. Likewise, in loop B, the basal ganglia output neurons represent the net cost-benefit of plan B. Action plan A producing the better prospective net value sustains through the strong re-entrant feedback signal from the basal ganglia to the cortex, whereas action plan B perishes. Although not depicted, the two pathways receive sensory cortical inputs so that the cost and benefit analysis draws on the sensory state information as well. GPe and STN are also omitted for brevity. The same color scheme as in (A) is used for different brain areas, and the line thickness indicates the signal strength.

According to this synaptic map, action plans that activate the direct pathway would produce facilitating feedback signals and get selected for execution, while action plans activating the indirect pathway would produce suppressing feedback signals and get cancelled. The BG, therefore, can smartly arbitrate competing action plans by channeling a desirable action plan through the direct pathway but competing, unwanted plans through the indirect pathway. Such smart sorting may incorporate reinforcement learning mechanisms within the BG. For example, if the outcome of a particular action is better than expected (i.e., the reward prediction error is positive), cortico-striatal synapses transmitting that action plan to the direct pathway may be strengthened so that the same action will be more likely selected in the future under similar sensory and internal conditions. Indeed, dopaminergic neurons projecting to the striatum appear to encode reward prediction errors to some extent (Schultz et al., 1997), and dopamine dependent plasticity has been observed at cortico-striatal synapses (Shen et al., 2008). As such, action selection models were elegantly inferred from the bottom-up analysis of anatomy and physiology.

Cost-Benefit Analysis

An outstanding question, though, is why a two-pathway structure is necessary given that the two pathways converge at the tonically firing output nuclei (Nambu, 2011) and thus, single pathway schemes seem sufficient to produce the re-entrant feedback in a prospective reward dependent manner. The answer may be sought using a top-down approach by specifying necessary computational elements to explain our action selection behaviors. Imagine you are going out for dinner tonight and have to choose a restaurant among many alternatives. Various aspects will be considered, including food tastes, dining atmospheres, prices, and distances. These are a list of benefits and costs associated with the available options. We and other animals avoid effortful actions if the benefit is held constant across action alternatives requiring different effort levels, yet we are willing to make effortful actions if higher benefits are predicted (Bautista et al., 2001; Stevens et al., 2005; Rudebeck et al., 2006; Gan et al., 2010). Thus, a critical process in action selection is to compute both benefits and costs for action alternatives and to weigh the benefits against costs. Various areas in the brain have been implicated for processing action costs or benefits, respectively, but it has not been clearly shown where and how the expected costs and benefits are measured against each other to affect action choice (Rudebeck et al., 2006; Croxson et al., 2009; Amemori and Graybiel, 2012).

A closer examination of the anatomy of the BG reveals that the two-pathway structure confers the necessary apparatus to perform such cost-benefit analysis for action selection. First, striatal neurons in the two pathways appear to receive different cortical inputs. Direct pathway striatal neurons preferentially receive cortical input from intra-telencephalic neurons that carry associative signals that are important for the computation of benefits, such as the context and abstract level action goals (Turner and DeLong, 2000; Lei et al., 2004). Indirect pathway striatal neurons receive greater inputs from the pyramidal-tract cortical neurons that carry the actual descending motor command signals (Lei et al., 2004). Motor command signals are most relevant for computing cost factors such as the energetic costs (efforts) and control risks of action plans (Todorov, 2000; Lei et al., 2004; Diedrichsen et al., 2010). Therefore, the direct pathway has access to information crucial for benefits, and the indirect pathway has access to information crucial for costs. Furthermore, as described earlier, activation of the two pathways exert opposite effects on the BG output (Figure 1A). In other words, the BG output takes the difference between the signals carried through the two pathways. This unique subtraction circuitry, combined with the distinct cortical input features, makes the two-pathway structure the ideal machinery for cost-benefit analysis.

Therefore this perspective hypothesizes that the sensorimotor BG direct pathway computes the expected benefits for action plans represented in the cortical input, while the indirect pathway computes their expected costs. The BG output combining the two pathways in a subtraction manner represents the net cost-benefit values. Then, due to the re-entrant feedback loops from the BG output to the cortex, the action with the highest prospective net value emerges as the winner (Redgrave et al., 1999). To better understand the model, imagine a situation where two action plans, A and B compete (Figure 1B). The cortical ensembles representing the two action plans form two cortico-BG-cortico loops, A and B, respectively. In loop A, BG direct and indirect pathways compute the expected benefit and cost for action A and BG output represents the prospective net value of action A. The re-entrant feedback facilitates or attenuates the cortical activity representing action A depending on its prospective value. Likewise, the cortical activity representing action B is modulated by its prospective value through loop B. Loop iterations, therefore, unequally modulate the two action representations in the cortex, leading to the selection of the action with the better prospective value.

In the computation of costs through the indirect pathway, various negative consequences of movements can be considered, including the energy expenditures (effort), control risks (e.g., motor errors, the loss of stability due to movements) (Harris and Wolpert, 1998), and punitive outcomes. Factors such as time and uncertainty that modulate the value of reward may or may not be processed through the indirect pathway. Temporal delays between movement and reward discount the value of reward. However, pharmacological activation of the indirect pathway failed to affect delay-dependent choice, whereas effort-dependent action choice was affected (van Gaalen et al., 2006; Salamone et al., 2007). In the case of uncertainty, it is unclear whether uncertainty, per se, has negative valence to be avoided (Platt and Huettel, 2008).

In the cost-benefit model, the BG continuously compute prospective values for instantaneous movement plans represented in the instantaneous cortical input. Theoretical and experimental studies suggest that motor commands are continuously evaluated and optimized moment-by-moment during movement using the latest sensory state information (Todorov, 2000; Shadmehr and Krakauer, 2008; Diedrichsen et al., 2010). The instantaneous action-value evaluation in the BG might underlie this moment-by-moment decision on motor commands. Compatible with this view, BG output activity shows task dependent modulation during movement (Mink, 1996). Furthermore, Huntington’s disease patients produce inappropriately exaggerated in-flight correction, indicating disrupted moment-by-moment decisions (Smith et al., 2000).

Despite its resemblance to old action selection models, the cost-benefit model bears nontrivial differences. Unlike the focused selection model (Mink, 1996) in which desirable actions channel through the direct pathway and undesirable actions through the indirect pathway, in the cost-benefit model every action plan enters both pathways. Moreover, in the cost-benefit model, the role of the indirect pathway is the expected-cost computation for potential motor plans, instead of a blanket inhibition of all motor plans as proposed in earlier action selection models (Gurney et al., 2004). Although the cost-benefit model may not come as a surprise to some in the field, the model for the first time articulates the idea of dissociative roles of the two pathways using computationally tractable parameters.

The Cost-Benefit Dependent Action Selection Model

The cost-benefit model reconciles the seemingly conflicting findings between the causative and correlative studies described in the Introduction. In order to make optimal choices, both benefits and costs must be concurrently computed for potential motor plans. Even when only a single action plan is considered, that action should be better than “not moving at all” in terms of the net cost-benefit, to be executed. Therefore, both pathways, respectively computing the benefit and cost of the single plan, should be activated. The cost-benefit model can also explain the opposing effects of selective intervention between the two pathways. Indirect pathway activation should cause the rise of expected costs for all action plans, impeding the initiation and execution of movements. Likewise, direct pathway activation should cause the rise of expected benefits, justifying otherwise unexecuted costly movements. Similar accounts can be given to the opposing motor symptoms of Parkinson’s and Huntington’s diseases. Indirect pathway underactivity in Huntington’s disease (Reiner et al., 1988; Richfield et al., 1995) can be viewed as abnormal cost deflation for all action plans, resulting in an excess of spontaneous movements. Indirect pathway overactivity in Parkinson’s disease (Mallet et al., 2006) can be viewed as abnormal cost inflation, resulting in slowness, and lack of spontaneous movements. Supporting this view, Parkinson’s patients exhibit an abnormally severe sensitivity to the energy expenditure needed for movements although they are physically capable of making high cost movements (Mazzoni et al., 2007).

The cost-benefit model also provides a functional explanation for the dissociative learning effects between the two pathways (Hikida et al., 2010; Kravitz et al., 2012). Paired stimulations of cortical and striatal neurons can induce long-term potentiation at their glutamatergic synapses (Shen et al., 2008; Bateup et al., 2010). Therefore, repetitive direct pathway stimulation following a particular action over many trials could facilitate the cortical signal transmission representing that action in the direct pathway. The signal facilitation in the direct pathway is equivalent to boosted expected benefit for that particular action, increasing the probability for that action to be selected in the future, i.e., reinforcement learning. Likewise, facilitating action plan transmission in the indirect pathway through paired stimulations would lead to aversive learning of the paired action because of the boosted expected cost.

The cost-benefit model predicts that exogenous stimulations of the two pathways during a choice period or a decision window (i.e., while multiple action plans are competing) should directly and instantly influence the competition and thus the impending choice. Exogenous activations of the direct pathway for a particular action during a choice period should boost the expected benefit of that action and thus the probability for that action to be chosen in the present trial. Activating the indirect pathway should decrease the probability of that action. In fact, transient pre-movement optogenetic activation of striatal neurons induced choice bias between two action alternatives in the predicted way (Tai et al., 2012). Unilateral pre-movement activation of the direct pathway caused bias towards the contralateral choice (e.g., left hemisphere direct pathway activation increased the frequency of rightward turning), whereas indirect pathway activation caused bias away from the contralateral choice. It has been shown that striatal neurons are normally active in movements of the body parts on the contralateral side (Nambu, 2011). Thus, in the framework of the cost-benefit model, the optogenetic experimental results could be explained as follows: unilateral activations of the direct pathway artificially boost the expected benefit for the contralateral movement, whereas unilateral indirect pathway activation boost the expected cost for the contralateral movement. Notably the authors of this optogenetic study proposed that activation of direct pathway striatal neurons mimics an increase of the action value of the contralateral choice and indirect pathway activation mimics a decrease of the action value, which is consistent with the cost-benefit model (Tai et al., 2012).

The cost-benefit model is also compatible with motivational effects of dopamine related drugs. Dopamine exerts opposite effects on striatal neurons in the two pathways through two different dopamine receptors: predominantly D1 receptors in the direct pathway versus predominantly D2 receptors in the indirect pathway (Gerfen et al., 1990). Dopamine binding to D1 receptors enhances dendritic excitability and facilitates glutamatergic signal transmission in direct pathway striatal neurons, whereas dopamine binding to D2 receptors inhibits glutamatergic signal transmission in indirect pathway neurons (Cepeda et al., 1993). Therefore, the cost-benefit model predicts that the overall effect of boosted tonic dopamine, up to a certain level, is to amplify the expected benefit while attenuating the expected cost. Confirming the prediction, rodents under higher tonic levels of dopamine selected energetically costly actions more frequently (Niv, 2007; Floresco et al., 2008). Selective effects of dopamine on the indirect pathway are also consistent with the cost-benefit model. Humans with more D2 receptors tended to choose energetically costly actions more frequently than those with fewer D2 receptors, most likely because dopamine can more effectively suppress indirect pathway activation in the presence of more D2 receptors, discounting cost effects (Treadway et al., 2012). Through the same mechanism but in the opposite direction, rodents injected with a D2-antagonist avoided energetically costly movements more often than before the injection (Salamone et al., 2007).

Future Directions

While the cost-benefit model presented here can account for a wide spectrum of findings, it is nevertheless a working hypothesis to be further tested and elaborated by future experiments. First, double dissociations between the direct and indirect pathways in terms of the benefit versus cost evaluation have yet to be shown. For instance, the model can be tested by directly correlating neural activity in the direct and indirect pathways with the expected benefit versus cost. Contrary to the classical model, the cost-benefit model predicts that vigorous movements would elevate indirect pathway activity because of their associated high cost. Concurrently direct pathway activity would be elevated to the extent to which there is an increased benefit associated with vigorous movements. Second, intervention of two-pathway activity during movement would affect online feedback control behaviors. For instance, amplifying direct pathway activity using D1 agonists would amplify prospective benefits, and the animal might produce more costly in-flight adjustments such as faster corrections. Third, neuromodulator signal pathways that encode cost factors and affect synaptic plasticity in the BG need to be uncovered. Phasic dopamine signals play a critical role in reinforcement learning (Schultz et al., 1997; Shen et al., 2008). However, dopaminergic neurons show heterogeneous, unreliable response to stimuli predicting punishment or effortful actions and to negative outcomes (Bayer and Glimcher, 2005; Matsumoto and Hikosaka, 2009; Gan et al., 2010). Thus, reinforcement learning likely depends on coordinated interactions among heterogeneous dopamine pathways and possibly involves non-dopamine pathways as well (Burke et al., 2012; Lammel et al., 2012). Fourth, the cost-benefit model is likely an oversimplification. A more complete picture of the cost-benefit analysis would emerge with a comprehensive understanding of the interconnections between the two pathways, the hyper-direct pathway from the cortex to the subthalamic nucleus, and structures outside the BG implicated in reward- or cost-dependent action selection such as the anterior cingulate cortex, orbitofrontal cortex, and insula (Rudebeck et al., 2006; Croxson et al., 2009). Finally, the concept of cost-benefit analysis in the sensorimotor BG may be extended to other BG functions, such as limbic and associative functions, given that other functional divisions of the BG follow similar organizational principles (Haber and Knutson, 2010). That is, evaluating positive versus negative outcomes of multiple combinations of cortical inputs via the direct versus indirect pathway to select the cortical process with the best expected outcome may be the fundamental function of the BG.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

Eun Jung Hwang was supported by NIH Career Development Award K99 NS062894. The author wishes to thank Drs. Richard Andersen, Tyson Aflalo, and Bardia Behabadi for their valuable advice and support.

References

Albin, R. L., Young, A. B., and Penney, J. B. (1989). The functional anatomy of basal ganglia disorders. Trends Neurosci. 12, 366–375. doi: 10.1016/0166-2236(89)90074-X

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Alexander, G. E., and Crutcher, M. D. (1990). Functional architecture of basal ganglia circuits: neural substrates of parallel processing. Trends Neurosci. 13, 266–271. doi: 10.1016/0166-2236(90)90107-L

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Alexander, G. E., DeLong, M. R., and Strick, P. L. (1986). Parallel organization of functionally segregated circuits linking basal ganglia and cortex. Annu. Rev. Neurosci. 9, 357–381. doi: 10.1146/annurev.ne.09.030186.002041

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Amemori, K. I., and Graybiel, A. M. (2012). Localized microstimulation of primate pregenual cingulate cortex induces negative decision-making. Nat. Neurosci. 15, 776–785. doi: 10.1038/nn.3088

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Bateup, H. S., Santini, E., Shen, W., Birnbaum, S., Valjent, E., Surmeier, D. J., et al. (2010). Distinct subclasses of medium spiny neurons differentially regulate striatal motor behaviors. Proc. Natl. Acad. Sci. U S A 107, 14845–14850. doi: 10.1073/pnas.1009874107

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Bautista, L. M., Tinbergen, J., and Kacelnik, A. (2001). To walk or to fly? How birds choose among foraging modes. Proc. Natl. Acad. Sci. U S A 98, 1089–1094. doi: 10.1073/pnas.98.3.1089

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Bayer, H. M., and Glimcher, P. W. (2005). Midbrain dopamine neurons encode a quantitative reward prediction error signal. Neuron 47, 129–141. doi: 10.1016/j.neuron.2005.05.020

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Burke, C. J., Huetteroth, W., Owald, D., Perisse, E., Krashes, M. J., Das, G., et al. (2012). Layered reward signalling through octopamine and dopamine in Drosophila. Nature 492, 433–437. doi: 10.1038/nature11614

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Cepeda, C., Buchwald, N. A., and Levine, M. S. (1993). Neuromodulatory actions of dopamine in the neostriatum are dependent upon the excitatory amino acid receptor subtypes activated. Proc. Natl. Acad. Sci. U S A 90, 9576–9580. doi: 10.1073/pnas.90.20.9576

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Croxson, P. L., Walton, M. E., O’Reilly, J. X., Behrens, T. E. J., and Rushworth, M. F. S. (2009). Effort-based cost-benefit valuation and the human brain. J. Neurosci. 29, 4531–4541. doi: 10.1523/JNEUROSCI.4515-08.2009

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Cui, G., Jun, S. B., Jin, X., Pham, M. D., Vogel, S. S., Lovinger, D. M., et al. (2013). Concurrent activation of striatal direct and indirect pathways during action initiation. Nature 494, 238–242. doi: 10.1038/nature11846

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Diedrichsen, J., Shadmehr, R., and Ivry, R. B. (2010). The coordination of movement: optimal feedback control and beyond. Trends. Cogn. Sci. 14, 31–39. doi: 10.1016/j.tics.2009.11.004

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Durieux, P. F., Bearzatto, B., Guiducci, S., Buch, T., Waisman, A., Zoli, M., et al. (2009). D2R striatopallidal neurons inhibit both locomotor and drug reward processes. Nat. Neurosci. 12, 393–395. doi: 10.1038/nn.2286

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Floresco, S. B., St Onge, J. R., Ghods-Sharifi, S., and Winstanley, C. A. (2008). Cortico-limbic-striatal circuits subserving different forms of cost-benefit decision making. Cogn. Affect. Behav. Neurosci. 8, 375–389. doi: 10.3758/CABN.8.4.375

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Frank, M. J. (2011). Computational models of motivated action selection in corticostriatal circuits. Curr. Opin. Neurobiol. 21, 381–386. doi: 10.1016/j.conb.2011.02.013

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Gan, J. O., Walton, M. E., and Phillips, P. E. M. (2010). Dissociable cost and benefit encoding of future rewards by mesolimbic dopamine. Nat. Neurosci. 13, 25–27. doi: 10.1038/nn.2460

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Gerfen, C. R., Engber, T. M., Mahan, L. C., Susel, Z., Chase, T. N., Monsma, F. J., Jr., et al. (1990). D1 and D2 dopamine receptor-regulated gene expression of striatonigral and striatopallidal neurons. Science 250, 1429–1432. doi: 10.1126/science.2147780

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Gillies, A., and Arbuthnott, G. (2000). Computational models of the basal ganglia. Mov. Disord. 15, 762–770. doi: 10.1002/1531-8257(200009)15:5<762::aid-mds1002>3.0.co;2-2

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Gurney, K., Prescott, T. J., Wickens, J. R., and Redgrave, P. (2004). Computational models of the basal ganglia: from robots to membranes. Trends Neurosci. 27, 453–459. doi: 10.1016/j.tins.2004.06.003

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Haber, S. N., and Knutson, B. (2010). The reward circuit: linking primate anatomy and human imaging. Neuropsychopharmacology 35, 4–26. doi: 10.1038/npp.2009.129

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Harris, C. M., and Wolpert, D. M. (1998). Signal-dependent noise determines motor planning. Nature 394, 780–784. doi: 10.1038/29528

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Hikida, T., Kimura, K., Wada, N., Funabiki, K., and Nakanishi, S. (2010). Distinct roles of synaptic transmission in direct and indirect striatal pathways to reward and aversive behavior. Neuron 66, 896–907. doi: 10.1016/j.neuron.2010.05.011

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Hikosaka, O., Takikawa, Y., and Kawagoe, R. (2000). Role of the basal ganglia in the control of purposive saccadic eye movements. Physiol. Rev. 80, 953–978.

Pubmed Abstract | Pubmed Full Text

Kravitz, A. V., Freeze, B. S., Parker, P. R. L., Kay, K., Thwin, M. T., Deisseroth, K., et al. (2010). Regulation of parkinsonian motor behaviours by optogenetic control of basal ganglia circuitry. Nature 466, 622–626. doi: 10.1038/nature09159

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Kravitz, A. V., Tye, L. D., and Kreitzer, A. C. (2012). Distinct roles for direct and indirect pathway striatal neurons in reinforcement. Nat. Neurosci. 15, 816–818. doi: 10.1038/nn.3100

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Lammel, S., Lim, B. K., Ran, C., Huang, K. W., Betley, M. J., Tye, K. M., et al. (2012). Input-specific control of reward and aversion in the ventral tegmental area. Nature 491, 212–217. doi: 10.1038/nature11527

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Lei, W., Jiao, Y., Del Mar, N., and Reiner, A. (2004). Evidence for differential cortical input to direct pathway versus indirect pathway striatal projection neurons in rats. J. Neurosci. 24, 8289–8299. doi: 10.1523/JNEUROSCI.1990-04.2004

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Mallet, N., Ballion, B., Le Moine, C., and Gonon, F. (2006). Cortical inputs and GABA interneurons imbalance projection neurons in the striatum of parkinsonian rats. J. Neurosci. 26, 3875–3884. doi: 10.1523/JNEUROSCI.4439-05.2006

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Matsumoto, M., and Hikosaka, O. (2009). Two types of dopamine neuron distinctly convey positive and negative motivational signals. Nature 459, 837–841. doi: 10.1038/nature08028

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Mazzoni, P., Hristova, A., and Krakauer, J. W. (2007). Why don’t we move faster? Parkinson’s disease, movement vigor, and implicit motivation. J. Neurosci. 27, 7105–7116. doi: 10.1523/jneurosci.0264-07.2007

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Mink, J. W. (1996). The basal ganglia: focused selection and inhibition of competing motor programs. Prog. Neurobiol. 50, 381–425. doi: 10.1016/S0301-0082(96)00042-1

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Nambu, A. (2011). Somatotopic organization of the primate Basal Ganglia. Front. Neuroanat. 5, 26. doi: 10.3389/fnana.2011.00026

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Niv, Y. (2007). Cost, benefit, tonic, phasic: what do response rates tell us about dopamine and motivation? Ann. NY Acad. Sci. 1104, 357–376. doi: 10.1196/annals.1390.018

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Platt, M. L., and Huettel, S. A. (2008). Risky business: the neuroeconomics of decision making under uncertainty. Nat. Neurosci. 11, 398–403. doi: 10.1038/nn2062

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Redgrave, P., Prescott, T. J., and Gurney, K. (1999). The basal ganglia: a vertebrate solution to the selection problem? Neuroscience 89, 1009–1023. doi: 10.1016/s0306-4522(98)00319-4

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Reiner, A., Albin, R. L., Anderson, K. D., D’Amato, C. J., Penney, J. B., and Young, A. B. (1988). Differential loss of striatal projection neurons in Huntington disease. Proc. Natl. Acad. Sci. U S A 85, 5733–5737. doi: 10.1006/nbdi.2002.0554

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Richfield, E. K., Maguire-Zeiss, K. A., Cox, C., Gilmore, J., and Voorn, P. (1995). Reduced expression of preproenkephalin in striatal neurons from Huntington’s disease patients. Ann. Neurol. 37, 335–343. doi: 10.1002/ana.410370309

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Rudebeck, P. H., Walton, M. E., Smyth, A. N., Bannerman, D. M., and Rushworth, M. F. (2006). Separate neural pathways process different decision costs. Nat. Neurosci. 9, 1161–1168. doi: 10.1038/nn1756

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Salamone, J. D., Correa, M., Farrar, A., and Mingote, S. M. (2007). Effort-related functions of nucleus accumbens dopamine and associated forebrain circuits. Psychopharmacology (Berl) 191, 461–482. doi: 10.1007/s00213-006-0668-9

CrossRef Full Text

Schultz, W., Dayan, P., and Montague, P. R. (1997). A neural substrate of prediction and reward. Science 275, 1593–1599. doi: 10.1126/science.275.5306.1593

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Shadmehr, R., and Krakauer, J. W. (2008). A computational neuroanatomy for motor control. Exp. Brain Res. 185, 359–381. doi: 10.1007/s00221-008-1280-5

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Shen, W., Flajolet, M., Greengard, P., and Surmeier, D. J. (2008). Dichotomous dopaminergic control of striatal synaptic plasticity. Science 321, 848–851. doi: 10.1126/science.1160575

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Smith, M. A., Brandt, J., and Shadmehr, R. (2000). Motor disorder in Huntington’s disease begins as a dysfunction in error feedback control. Nature 403, 544–549. doi: 10.1038/35000576

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Steiner, H., and Tseng, K. Y. (2010). Handbook of Basal Ganglia Structure and Function: A Decade of Progress. Oxford: Elsevier Science.

Stevens, J. R., Rosati, A. G., Ross, K. R., and Hauser, M. D. (2005). Will travel for food: spatial discounting in two new world monkeys. Curr. Biol. 15, 1855–1860. doi: 10.1016/j.cub.2005.09.016

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Tai, L. H., Lee, A. M., Benavidez, N., Bonci, A., and Wilbrecht, L. (2012). Transient stimulation of distinct subpopulations of striatal neurons mimics changes in action value. Nat. Neurosci. 15, 1281–1289. doi: 10.1038/nn.3188

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Todorov, E. (2000). Direct cortical control of muscle activation in voluntary arm movements: a model. Nat. Neurosci. 3, 391–398. doi: 10.1038/73964

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Treadway, M. T., Buckholtz, J. W., Cowan, R. L., Woodward, N. D., Li, R., Ansari, M. S., et al. (2012). Dopaminergic mechanisms of individual differences in human effort-based decision-making. J. Neurosci. 32, 6170–6176. doi: 10.1523/JNEUROSCI.6459-11.2012

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Turner, R. S., and DeLong, M. R. (2000). Corticostriatal activity in primary motor cortex of the macaque. J. Neurosci. 20, 7096–7108.

Pubmed Abstract | Pubmed Full Text

van Gaalen, M. M., van Koten, R., Schoffelmeer, A. N. M., and Vanderschuren, L. J. M. J. (2006). Critical involvement of dopaminergic neurotransmission in impulsive decision making. Biol. Psychiatry 60, 66–73. doi: 10.1016/j.biopsych.2005.06.005

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text