The basal ganglia select the expected sensory input used for predictive coding

Prediction in Perception

The importance of predictions in cognition has been extensively reviewed in recent neuroscience and cognitive science literature (Grush, 2004; Hawkins and Blakeslee, 2004; Friston and Stephan, 2007; Pezzulo et al., 2008; Bar, 2009; Bubic et al., 2010; Colder, 2011; Clark, 2013). Sensory perception is thought to result from the neural combination of “top-down” sensory expectations with “bottom-up” information from sensory organs (Bar, 2007; Panichello et al., 2013). In particular, the term “predictive coding” describes the theory that sensory expectations flow down from higher-level association areas to lower-level sensory cortex, and deviations from those expectations (error signals) flow back up to association areas (Rao and Ballard, 1999; Huang and Rao, 2011). Recent empirical evidence supporting predictive coding, (reviewed in Egner and Summerfield, 2013) include brain imaging results demonstrating increased activity in primary visual cortex in response to unexpected stimuli (Alink et al., 2010), and increases in the differentiation in the primary visual cortex responses to houses and faces as stimulus predictability decreases (Egner et al., 2010).

While the studies cited above, and others, support the notion that lower-level interpretation of incoming sensory information is guided by top-down sensory expectations, less is known about the source of the sensory expectations or the mechanisms by which they are spread. The “reafference principle” (von Holst and Mittelstaedt, 1950) states that copies of motor commands (“efference copies” or “corollary discharge”) are transmitted to sensory processing regions so that the sensation resulting from those actions (the “reafference”) can be “subtracted out” from the stream of incoming sensory information. Corollary discharge is used to anticipate and ignore the visual blur that occurs during high-speed saccades (Ross et al., 2001) and to inhibit the cricket’s auditory system response to self-generated noise (Poulet and Hedwig, 2006). Along with alerting the sensory system to very specific information that should be ignored or accounted for, corollary discharge may also provide more general information about potential actions that could be used to guide sensation.

Perception is an ongoing process, requiring a constant flow of top-down expectations to guide interpretation of incoming sensation. Action selection is also an ongoing process, and since action selection must consider predictions for the environment that would result from the action, each representation of a potential action must be tied to a sensory expectation. These action-dependent environmental predictions may serve as the top-down sensory expectations used in predictive coding.

Link Between Potential Action and Expected Sensation

Prediction is a critical aspect of action planning and execution (for a review, see Mehta and Schaal, 2002). For instance, a constant downward force can produce the illusion of an increase in force if visual information leads to a prediction that the sensation of force should decrease (Diedrichsen et al., 2007). Also, trans-cranial magnetic stimulation over the cerebellum leads to reaching errors that suggest the cerebellum holds an estimate of future limb position (Miall et al., 2007). A classic theory of motor cortex states that motor cortex represents images of potential achievement, and continuously monitors progress toward those future goals (Pribram, 1971). The description of motor cortex function by Pribram (1971), based on anatomical and neurophysiological studies of the spinal cord, cerebellum, and motor cortex, emphasizes the similarities between neural representations of action plans and expected sensory states. Similarly, “common coding theory” (Prinz, 1997) explicitly states that perceived events and potential actions are represented in the same manner. Prinz (1997) also introduces the “action-effect” hypothesis, which holds that action planning depends on the expected outcomes of the potential actions. More recently, Friston suggests that motor intentions are tied to sensory predictions, and actions are designed to elicit sensory proprioceptive predictions (Friston, 2003, 2011). Clark (2013) describes this work as “action-oriented predictive processing”.

Further theoretical support for a strong link between planned actions and their expected sensation comes from Gross et al. (1999), who present a theory and model of generative perception based on the integration of action plans and their expected future sensory consequences. In addition, Grush (2004, 2007—skill theory v 2.0) introduces the “emulation framework” to explain how cognition might generate and use predictions of the sensory environment that would result from specific actions. As used here, the term “emulation” describes a single entity made up of the linked representations of a potential action and its expected sensory result. Empirical support for the link between potential actions and predicted outcome comes from a recent study that found anticipatory activity from single cell discharges in rat thalamus, and primary sensory and motor cortex, when rats explored the size of an aperture (Pais-Vieira et al., 2013). This anticipatory activity was disrupted when motor cortex was de-activated with muscimol, indicating that motor cortical activity was important for maintaining predictive representations in thalamus and sensory cortex.

The close ties between potential actions and the predictions for sensation those actions would produce led Colder (2011) to hypothesize that it was the sensory expectations in active emulations that were passed down to primary sensory cortices in the process of predictive coding. While Colder (2011) describes a theoretical source of top-down sensory expectations for perception, insight into how those expectations may be spread to lower-level cortices is still lacking. The link between potential actions and their predicted sensations suggests that the process of action selection may also be involved in spreading sensory expectations. To understand how action selection might spread predicted sensation, first we must consider how emulations are instantiated in neuronal networks.

Parallel, Distributed Networks Instantiate Emulations

Fuster (2006, 2009), proposed that cortical representations of actions and sensory states are instantiated in “cognits”, or widely distributed neuronal networks. Emulations are hypothesized to be implemented in cognits consisting of neurons distributed widely throughout cortex (Colder, 2011). Similar to working memories (Fuster, 2006), emulations are hypothesized to be generated by partial sensory cues that activate neural networks associated with long-term memories. Emulations may begin as more abstract plans and more uncertain sensory expectations (e.g., considering whether to head to the park in the afternoon), in which case active cognits will include neurons from anterior portions of PFC, and higher-level sensory association areas, but few neurons from primary motor or sensory areas. Alternatively, if newly generated emulations include concrete, immediate plans and very specific sensory expectations (e.g., jumping out of the way of an oncoming bus), their initial active cognits will include primary sensory and motor regions.

Figure 1 demonstrates the distributed nature of the active neuronal networks that instantiate emulations. The neural network for the notional abstract emulation depicted in the Figure includes neurons in frontal motor association areas whose activity corresponds to the emulation’s potential action. The emulation’s network also includes neurons in parietal and temporal sensory association areas whose activity corresponds to the expected sensory outcome of the potential action.

FIGURE 1

Figure 1. The figure shows the cortical surface with an overlay of the basal ganglia and thalamus. The blue dots on the cortical surface represent the neuronal activity for a notional emulation. This emulation is abstract, and the active neural networks instantiating the emulation are in higher-level motor and sensory association areas of the brain.

Many emulations may be simultaneously instantiated in parallel, distributed neural networks. This proposal follows the notion that association cortices are comprised of parallel, segregated networks as described by Goldman-Rakic (1988). Recent evidence supporting the idea of physically widespread neural networks as basic units of representation comes from a study of cortical connectivity finding interdigitated, functional networks distributed throughout the cortex (Yeo et al., 2011).

Continuous competition among emulations may be driven by incoming sensory information that defines the expected outcome of actions. In a visual discrimination task requiring saccades, activity in motor cortex reflects the continuous accumulation of sensory information that indicates the choice with best expected outcome (Gold and Shadlen, 2000). More recently, cortico-spinal excitability for action pathways was found to vary with the value of the action (Klein-Flügge and Bestmann, 2012). Incoming sensory information is important not only for updating the expected outcome of actions, but also for specifying which actions are physically possible at a given moment. Cisek (2007) proposes that incoming sensory information drives a continuous competition among potential actions, or “affordances”, by continuously updating the set of viable affordances.

Since emulations are instantiated in cortex as distributed neural networks, neural network operations govern the spread and shrinkage of emulation influence. In particular, spreading sensory expectations to lower-level sensory cortices requires activating lower-level sensory cortical neurons that are connected to the neurons instantiating the higher-level representation of the sensory expectation. Theories of action selection (discussed below) describe how the basal ganglia interact with cortical networks instantiating potential actions to allow the spread of the chosen network and action. The same mechanism can also serve to spread the network instantiating the sensory expectation associated with the action.

Basal Ganglia Gate Actions from Within Cortical-BG-Thalamo-Cortical Information Loops

Tract-tracing studies indicate that information flows from cortex through the basal ganglia and thalamus and back to cortex in mainly segregated loops (Alexander et al., 1986; Middleton and Strick, 2002) although there is some integration of information between loops (Haber and Calzavara, 2009). Although most empirical evidence demonstrates information loops between BG and PFC, retrograde labeling of the substantia nigra pars reticulata (SNr) reveals projections through thalamus to the inferotemporal cortex as well (Middleton and Strick, 1996), suggesting the existence of closed information loops from all parts of cortex through the BG and back to cortex. Figure 2 depicts the cortical-basal ganglia-thalamo-cortical loops for inferotemporal cortex for notional blue and yellow emulations.

FIGURE 2

Figure 2. The yellow dots represent activity in a neural network instantiating another notional emulation that also has activity in higher-level association areas of the brain. The blue and yellow emulations contain different potential actions and expected outcomes. Multiple emulations are hypothesized to compete for realization at any given time.

The neural networks for both emulations have closed-loop connections with the basal ganglia and thalamus. The arrows shown indicate the excitatory input from one sensory association cortical region to the basal ganglia, and the excitatory path back from the thalamus to cortex. Information in these loops travels multiple pathways though the basal ganglia to reach thalamus and then return to cortex. Basal ganglia GO pathways increase excitation in cortical neural networks, while NoGo pathways decrease cortical excitation. Note that the neural networks in other cortical regions (e.g., pre-frontal cortex, and parietal sensory association areas) are also a part of similar cortical-basal ganglia- thalamic loops.

Role of Cortical-BG-Thalamo-Cortical Information Loops in Action Gating

The BG play an important role in determining which of many possible actions might be performed (Mink, 1996; Redgrave et al., 1999). Mink (1996) hypothesized that once a movement is initiated in cortex, pathways from cortex through the BG decrease inhibition for the selected action, and increase inhibition on competing actions, allowing for the smooth execution of the selected action.

Whereas Mink (1996) and Redgrave et al. (1999) suggested that the BG select actions by disinhibiting the cortical portion of a cortical-BG-thalamo-cortical information loop containing an action plan, I propose that the information loops operate on not just action plans, but complete emulations, including the expected resulting sensation associated with action plans. Therefore the same mechanism hypothesized by Mink (1996) and Redgrave et al. (1999) to select actions is also selecting expected sensation.

BG Go and NoGo Pathways

There are multiple pathways information could travel through the BG in a cortical-BG-thalamo-cortical information loop (Schroll and Hamker, 2013). For all pathways, the final connection, from thalamus to cortex, is excitatory. The preceding connection is inhibitory output from the BG [from either the globus pallidus internus (GPi) or SNr] to the thalamus. As a result, the BG provide a varying inhibitory influence on an excitatory connection from thalamus to cortex.

One of these pathways, the “direct” pathway through the BG includes an excitatory signal from the cortex to the striatum, which then sends an inhibitory signal to the BG output nuclei. As a result, activation of the direct pathway results in a decrease of BG output, producing less inhibition of the thalamus and cortex. The direct pathway was termed the “Go” pathway by Hazy et al. (2007). There are also multiple “NoGo” pathways through the BG. These pathways ultimately produce increases in inhibition from the BG to the thalamus, resulting in less cortical excitation (see review in Schroll and Hamker, 2013).

Mink (1996) and Hazy et al. (2007) suggested that actions are selected when Go pathway activation for the cortico-BG-thalamo-cortical loop disinhibits the selected PFC action representation, at the same time as NoGo activity increases for competing actions in different information loops. Both Go and NoGo pathways were recently found to be active during normal movement (Cui et al., 2013), supporting the concept that action selection results from changes in the activation of each pathway relative to the other. Figure 3 shows some of the pathways information travels through the BG in cortical-BG-thalamo-cortical loops. Information about the blue emulation is moving on GO pathways, while the yellow emulation is being deselected by NO GO pathways.

FIGURE 3

Figure 3. In this figure, basal ganglia GO pathways are shown in blue, and they are selecting the “blue” emulation, while NO GO pathways are shown in yellow deselecting the “yellow” emulation. Arrows that are outlined in black denote an excitatory connection, while red-outlined arrows represent an inhibitory connection. Pathway 1 is a NO GO pathway that includes an inhibitory connection from the striatum to the globus pallidus externus (GPe), an inhibitory connection from the GPe to the globus pallidus internus (GPi), and an inhibitory connection to the thalamus. Pathway 2 is a GO pathway that has an inhibitory connection from the striatum to the GPi, and an inhibitory connection from the GPi to the thalamus. Pathway 3 is a NO GO pathway that contains an inhibitory connection from the striatum to the GPe, an inhibitory connection from the GPe to the subthalamic nucleus (STn), an excitatory connection from STn to GPi, and an inhibitory connection from GPi to thalamus. Pathway 4 is a GO pathway that has an inhibitory connection from the striatum to the substantia nigra pars reticulata (SNr), and an inhibitory connection from SNr to thalamus. Instead of starting in the striatum, Pathway 5 is a NO GO pathway that includes an excitatory connection directly from cortex the STn, an excitatory connection from STn to GPi, and an inhibitory connection from GPi to thalamus. Note that the pathways shown are just a selected subset of all the possible pathways information can travel through the basal ganglia in a cortical-basal ganglia-thalamo-cortical loop. See Schroll and Hamker (2013) for a review of basal ganglia pathways.

Proposal for Emulation Realization via Recruitment of “Actualization” Neurons

Following Mink (1996) and Hazy et al. (2007), I propose that a decrease in inhibition caused by excitation of BG Go pathway promotes selected emulations (potential action/expected sensation pairs) by increasing the firing rate in their instantiating neural network, while simultaneously BG NoGo pathways deliver increased cortical inhibition to competing emulations. Although there are many influences on cortical neural networks, this BG-produced relative increase in firing rate of the network for the action in the selected emulation allows the network to recruit more caudal/posterior neurons in PFC. This recruitment advances the activity in the selected action’s neural network toward the motor cortex and eventual execution. Thus, action selection results from BG disinhibition of emulations and their cortical networks that allows recruitment of more caudal/posterior PFC neurons, including motor strip neurons.

But since potential actions and expected sensations are linked, a decrease in inhibition for an emulation also increases the activation of the emulation’s posterior cortical network, allowing the network to recruit neurons that are physically closer to primary sensory cortices. This recruitment is the top-down dissemination of the emulation’s sensory expectations on lower-level sensory areas, and it is the implementation of predictive coding. BG disinhibition of preferred emulations and their distributed cortical networks is thus proposed to allow the spread of top-down expectations that shape ongoing perception. Figure 4 shows that since excitatory input from thalamus has increased for the blue emulation, the active neural networks for that emulation have recruited more neurons. Activity for the blue emulation has spread to primary motor cortex, meaning that the emulation’s action is being performed. Neural networks instantiating the blue emulation have also spread to primary somato-sensory cortex and primary visual cortex, indicating that the emulation’s sensory expectations are being used to interpret incoming sensation.

FIGURE 4

Figure 4. The increased activity in the neural networks instantiating the blue emulation allows those neural networks to recruit neurons in lower-level sensory and motor areas. The spreading of the emulation sends the action plan to the motor strip, which initiates the emulation’s potential action. The spread of the emulation to primary sensory cortices is the top-down propagation of the sensory expectations in the emulation, as described in predictive coding theory.

Assumption that Cortical Networks Compete for Lower-Level Neurons

The theory described above depends on an important assumption about the spread of representations implemented in distributed cortical networks. While output from the basal ganglia is just one of many sources of excitation or inhibition for the neural networks instantiating emulations, the proposed mechanism for moving an emulation toward realization is that the emulation’s neuronal network is able to recruit lower-level neurons by virtue of its being disinhibited by the BG relative to competing emulations/networks. The assumption implicit in this mechanism is that the relative decrease in BG inhibition will increase the discharge rate of neurons in the distributed network, and that this increase in neuronal activity (compared to other networks) enhances the network’s ability to expand. This assumption will hold up in a scenario where lower-level sensory and motor neurons are constantly “in demand” by multiple cortical networks/emulations that are all competing to access lower-level neurons so that the emulations can “become reality”.

Note that action plans will become reality if neurons in primary motor cortex discharge in accord with the plan, causing the body to execute the action. The expected resulting sensory input might not always occur, but imposing expectations tends to bias perception toward the expected sensations (for reviews, see Summerfield and Egner, 2009; Seriès and Seitz, 2013). The importance of expectations in affecting perception was recently affirmed by a psychophysical study in which subjects tended to perceive random dot rotation in the expected direction, even when actual dot motion was completely ambiguous (Sterzer et al., 2008). In another task requiring subjects to press a button when a moving ball was at a specific location, subjects were found to consistently underestimate their own errors relative to errors they believed to be made by others (Wolpe et al., 2014). The authors interpret these findings as indicating that subjects had a prior expectation of successful task completion that biased their perception of their actual performance. These studies support the notion that a cortical network able to recruit lower-level sensory neurons would bias the organism towards experiencing the emulation’s sensory prediction as reality.

The idea that distributed networks based mainly in association cortex compete for neurons in lower-level sensory and motor areas is supported by imaging studies demonstrating increased activation in more anterior PFC as tasks became more complex (Koechlin et al., 2003; Badre and D’Esposito, 2007). Koechlin et al. (2003) explained these results using a “cascade model” of control in the PFC, in which representations at lower (more caudal) levels are controlled by representations at higher (more rostral) levels. Badre and D’Esposito (2007) also noted a general increase in baseline activity in more posterior areas of PFC when more anterior, higher-level areas become engaged. This increase in baseline in lower-level areas is consistent with the notion that abstract action plans implemented in networks that include anterior PFC compete for lower-level neurons in more posterior regions of PFC. Reviews by Fuster (2009) and O’Reilly (2010) conclude that action execution begins with abstract, complex representations in anterior PFC and proceeds to concrete representations and specific movements controlled by posterior PFC. Redgrave et al. (1999) also assumed that multiple action plans competed for access to motor effectors, and proposed the BG as the action plan selector.

Novel Proposal for the Role of BG in Perception

The theory of BG function described above is drawn directly from hypotheses for the role of the BG in action selection (Mink, 1996; Redgrave et al., 1999; Hwang, 2013). Integrating these hypotheses with the notion that potential actions are bound to their expected resulting sensation, and also with theories of the role of expectations in perception suggests the novel role of the BG in also selecting and initiating top-down sensory expectations, and hence influencing perception.

Although BG disinhibition has not been previously suggested as the mechanism for propagating sensory expectations, there is considerable recent evidence suggesting that the BG play an important role in perception (reviewed in Ding and Gold, 2013). Single cell recordings from the caudate revealed the presence of neurons whose discharge rates tended to predict perceptual choices, especially when the sensory evidence for the choice was relatively weaker (Ding and Gold, 2010). In support of the current theory, these neurons appear to act like a source of top-down bias to lower-level sensory areas. Also, micro-stimulation of the caudate nucleus during a perceptual decision task induced a decision preference opposite to that preferred by neighboring caudate neurons (Ding and Gold, 2012). The authors use these results to suggest that the direct and indirect pathways could have opposite effects on perceptual decisions.

Multiple authors have previously modeled BG computations involving perception. Rao (2010) describes a model in which the BG select actions based on previously learned responses to Bayesian belief states represented in cortex. The belief states and learned actions described by Rao (2010) are similar to emulations. Bogacz and Gurney (2007) and Lepora and Gurney (2012) suggest that the BG optimally select actions based on associated perceptual hypotheses. The associations of actions and perceptual hypotheses used in those models are also similar to emulations.

General Discussion

I propose that the internal Go and NoGo pathways of the BG select between multiple alternate futures (emulations) that are implemented in distributed cortical networks and cortical-BG-thalamo-cortical information loops. The result of this selection is the disinhibition of selected cortical networks, allowing them to recruit lower-level sensory and motor neurons, which has the effect of advancing the selected emulation to actualization, including spreading the emulation’s sensory expectation. Although previous authors have made similar proposals to describe how actions are selected, a novel aspect of this proposal is that BG disinhibition initiates the spread of top-down sensory expectations that are known to play an important role in perception. The hypothesis depends on the assumption that the neural networks instantiating emulations compete for access to lower-level neurons that can implement the emulation’s potential action and sensory expectations.

This hypothesis derives from previous research and theory indicating the role of the BG’s Go and NoGo pathways in action selection, combined with another body of literature describing the importance of prediction in cognition, including predictive coding in perception. Extending the BG’s role in action selection to a more general role for selecting between alternate futures follows naturally from the notion that action plan representations are linked to representations of the resulting expected sensation to create complete future scenarios. This assumption agrees with Pribram (1971), who suggested that action plan representations are just the expected sensory outcome, including the state of muscle spindle cells, after the action has been performed. In this view, also shared by Friston (2003, 2011), action planning is based on sensory representations, and a sequence of actions can be conceived as a series of expected sensations that include signals of muscle position. The present hypothesis goes beyond these theories relating potential actions and expected sensation, as well as Prinz (1997)’s common coding theory, by presenting a candidate mechanism, BG disinhibition, for the selection of actions and the propagation of expectations.

The hypothesis that action selection occurs as BG-disinhibited cortical networks recruit neurons closer to primary motor cortex stands in contrast to the proposal from Rizzolatti and Luppino (2001) that prefrontal “control areas” with a predominance of prefrontal connections determine when activity in more parietal-connected, posterior frontal cortex leads to action. Perhaps these hypotheses are in agreement if cortical-BG-cortical networks that originate from prefrontal “control areas” are initially disinhibited, leading to recruitment of more posterior prefrontal networks.

BG Role in Predicting Reward

Evidence that the BG track the outcomes of actions—including reward information—supports the hypothesis that the BG select emulations that include expected sensation. The BG are known to encode reward prediction errors (O’Doherty et al., 2004). Individual anterior caudate neurons encode both reward and task information (Yanike and Ferrera, 2014). Neurons in the caudate, putamen and ventral striatum showed activity related to the magnitude of the expected reward (Cromwell and Schultz, 2003). Also, caudate neurons have been found that are sensitive to the size of the expected reward and their output may produce a bias in the superior colliculus that influences saccadic output (Hikosaka et al., 2006). More recently, multiple, distinct relationships between reward size and baseline neuronal discharge characteristics were found in recordings from neurons throughout the caudate (Nakamura et al., 2012).

From the perspective of the current hypothesis, reward is an important part of the more general expected sensation resulting from an action plan. Ding and Gold (2013) speculate that expectations of reward might be combined with other anticipated sensory information to aid in perceptual decisions. Exact mechanisms of how reward and other aspects of the expected sensory environment influence the BG’s Go and NoGo pathway calculations of inhibition for individual emulations and cortical-BG-thalamo-cortical information loops are yet to be determined.

Potential Changes to Computational Models of BG Function

Previous models focused on the role of the BG in action selection (Hazy et al., 2007), gating of working memory (Frank et al., 2001) and the reinforcement learning needed to guide action selection (Hazy et al., 2006; Rao, 2010; Bolado-Gomez and Gurney, 2013). Other research has included modeling BG and cortex performance on perceptual decisions using models that assume sensory evidence is passively accumulated until a decision is reached (Bogacz and Gurney, 2007; Lepora and Gurney, 2012). The hypothesis that the BG initiate top-down sensory expectations suggests a novel addition to models of BG function, that BG disinhibition should encourage selected perceptual hypotheses before all sensory evidence is obtained, creating a bias that tends to influence perception towards those hypotheses. Thus, instead of passively accumulating evidence, models of the BG’s role in perceptual decisions might include terms that accelerate progress towards a specific decision after evidence supporting that decision has been observed.

Testing the Hypothesis

The proposed hypothesis incorporates established theories of action selection by the BG, while emphasizing the link between representations of action plans and the anticipated sensation resulting from those plans. A large body of empirical evidence already exists describing how BG activity is linked to action selection. Therefore key tests of this hypothesis would shed light on the BG’s role in establishing sensory expectations. Informative experiments could manipulate the Go and NoGo pathways in the BG and measure the resulting effect on sensory expectations. For instance, there are two different NoGo pathways that pass through the subthalamic nucleus (STn; Schroll and Hamker, 2013), the indirect pathway [cortex to striatum to globus pallidus externus (GPe) to STn to GPi], and the hyperdirect pathway (cortex to STn to GPi). Reducing the effectiveness of the STn by electrical, chemical or mechanical means during the presentation of informative stimuli should disrupt these NoGo pathways. According to the proposed hypothesis, this disruption would result in a lack of inhibition for competing actions and expectations, which will decrease the top-down guidance for interpretation of incoming sensory stimuli.

Physiological measurements can provide indirect evidence of sensory expectations. For instance, Summerfield et al. (2006) found fMRI activity that corresponded to specific “perceptual sets” in a visual recognition task with degraded stimuli, and many studies have demonstrated modulation of cortical single neuronal firing patterns by anticipated reward (for example, Rosenkilde et al., 1981; Shuler and Bear, 2006; Burton et al., 2013; Stanişor et al., 2013). If BG disinhibition is responsible for spreading sensory expectations as described above, then inactivating the BG should reduce the neurophysiological changes corresponding to lower-level sensory expectations, including reward expectations. That is, while “reward cells” may still be found in pre-frontal cortex and sensory association cortices, it should not be possible to observe reward cells in primary sensory and motor cortices when the BG are inactivated. Alternatively, if the activity of just the STn of the BG was experimentally decreased, then the effectiveness of the NoGo pathway should be degraded, allowing incorrect expectations to propagate. In this case it may be possible to see reward cell activity even when rewards are not expected.

The degree to which an individual’s sensory expectations match incoming stimuli can also be inferred using behavioral measures such as reaction time or percent correct responses. However, it is difficult to obtain behavioral data on sensory expectations that is not confounded with selected actions—both a misperception and an incorrectly selected action will produce the same result, an incorrect choice. Specific tests of the proposed role of the BG in selecting and advancing potential sensation (as opposed to selecting action plans) must dissociate selection of sensory expectations from selection of an action plan. Perhaps this could be accomplished in a behavioral experiment by requiring the same response for all stimuli and measuring how changes in reaction time vary with manipulations of sensory expectations and BG function.

The experiments described above suggest the use of laboratory methods to manipulate BG function. Further evidence bearing on the hypothesis could come from “natural lesion” studies of patients with BG dysfunction that affects the motor system, such as Parkinson’s or diseases producing chorea. Since the current hypothesis supposes that the BG are involved in propagating sensory expectations, patients with BG dysfunction might be expected to show perceptual problems related to inaccurate sensory predictions. In fact, in an experiment on attentional set shifting, Cools et al. (2010) found that patients with Parkinson’s disease relied more on bottom-up stimulus characteristics—and less on top-down expectations—than did normal controls. Further testing is needed to determine if a reduced emphasis on top-down sensory expectation is a common feature of patients with BG-related disease.

Application of the Emulation Framework

Colder (2011) proposed a theory of prediction in cognition that provided a general emulation-based framework for understanding cognitive neuroscience research. Placing studies of the BG into the emulation framework supports the current re-interpretation of findings that have previously been used to link the BG to action selection, resulting in the more general theory that the BG select between alternate potential futures. This is an example of how the emulation framework can offer a context for cognitive neuroscience research that may provide additional insight into the implications of research results.

Conflict of Interest Statement

The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

The author wishes to acknowledge the professional artwork of Brian Enciso, who created the figures for the manuscript. Jeff Colombe, Hal Greenwald, Lilliam Rosario, and two anonymous reviewers of an earlier version all provided insightful comments that drove the manuscript to its final version.

References

Alexander, G. E., DeLong, M. R., and Strick, P. L. (1986). Parallel organization of functionally segregated circuits linking basal ganglia and cortex. Annu. Rev. Neurosci. 9, 357–381. doi: 10.1146/annurev.neuro.9.1.357

PubMed Abstract | CrossRef Full Text | Google Scholar

Alink, A., Schwiedrzik, C. M., Kohler, A., Singer, W., and Muckli, L. (2010). Stimulus predictability reduces responses in primary visual cortex. J. Neurosci. 30, 2960–2966. doi: 10.1523/jneurosci.3730-10.2010

PubMed Abstract | CrossRef Full Text | Google Scholar

Badre, D., and D’Esposito, M. (2007). Functional magnetic resonance imaging evidence for a hierarchical organization of the prefrontal cortex. J. Cog. Neurosci. 19, 2082–2099. doi: 10.1162/jocn.2007.19.12.2082

PubMed Abstract | CrossRef Full Text | Google Scholar

Bar, M. (2007). The proactive brain: using analogies and associations to generate predictions. Trends Cogn. Sci. 11, 280–289. doi: 10.1016/j.tics.2007.05.005

PubMed Abstract | CrossRef Full Text | Google Scholar

Bar, M. (2009). Predictions: a universal principle in the operation of the human brain. Philos. Trans. R. Soc. Lond. B Biol. Sci. 364, 1181–1182. doi: 10.1098/rstb.2008.0321

PubMed Abstract | CrossRef Full Text | Google Scholar

Bogacz, R., and Gurney, K. (2007). The basal ganglia and cortex implement optimal decision making between alternative actions. Neural Comput. 19, 442–477. doi: 10.1162/neco.2007.19.2.442

PubMed Abstract | CrossRef Full Text | Google Scholar

Bolado-Gomez, R., and Gurney, K. (2013). A biologically plausible embodied model of action discovery. Front. Neurorobot. 7:4. doi: 10.3389/fnbot.2013.00004

PubMed Abstract | CrossRef Full Text | Google Scholar

Burton, A. C., Kashtelyan, V., Bryden, D. W., and Roesch, M. R. (2013). Increased Firing to Cues That Predict Low-Value Reward in the Medial Orbitofrontal Cortex. Cereb. Cortex. 24, 3310–3321. doi: 10.1093/cercor/bht189

PubMed Abstract | CrossRef Full Text | Google Scholar

Bubic, A., von Cramon, D. Y., and Schubotz, R. I. (2010). Prediction, cognition and the brain. Front. Hum. Neurosci. 4:25. doi: 10.3389/fnhum.2010.00025

PubMed Abstract | CrossRef Full Text | Google Scholar

Cisek, P. (2007). Cortical mechanisms of action selection: the affordance competition hypothesis. Philos. Trans. R. Soc. Lond. B Biol. Sci. 362, 1585–1599. doi: 10.1098/rstb.2007.2054

PubMed Abstract | CrossRef Full Text | Google Scholar

Clark, A. (2013). Whatever next? Predictive brains, situated agents and the future of cognitive science. Behav. Brain. Sci. 36, 181–204. doi: 10.1017/s0140525x12000477

PubMed Abstract | CrossRef Full Text | Google Scholar

Colder, B. (2011). Emulation as an integrating principle for cognition. Front. Hum. Neurosci. 5:54. doi: 10.3389/fnhum.2011.00054

PubMed Abstract | CrossRef Full Text | Google Scholar

Cools, R., Rogers, R., Barker, R. A., and Robbins, T. W. (2010). Top-down attentional control in Parkinson’s disease: salient considerations. J. Cogn. Neurosci. 22, 848–859. doi: 10.1162/jocn.2009.21227

PubMed Abstract | CrossRef Full Text | Google Scholar

Cromwell, H. C., and Schultz, W. (2003). Effects of Expectations for Different Reward Magnitudes on Neuronal Activity in Primate Striatum. J. Neurophysiol. 89, 2823–2838. doi: 10.1162/jocn.2009.21227

PubMed Abstract | CrossRef Full Text | Google Scholar

Cui, G., Jun, S. B., Jin, X., Pham, M. D., Vogel, S. S., Lovinger, D. M., et al. (2013). Concurrent activation of striatal direct and indirect pathways during action initiation. Nature. 494, 238–242. doi: 10.1038/nature11846

PubMed Abstract | CrossRef Full Text | Google Scholar

Diedrichsen, J., Verstynen, T., Hon, A., Zhang, Y., and Ivry, R. B. (2007). Illusions of force perception: the role of sensori-motor predictions, visual information and motor errors. J. Neurophysiol. 97, 3305–3313. doi: 10.1152/jn.01076.2006

PubMed Abstract | CrossRef Full Text | Google Scholar

Ding, L., and Gold, J. I. (2012). Separate, causal roles of the caudate in saccadic choice and execution in a perceptual decision task. Neuron. 75, 865–874. doi: 10.1016/j.neuron.2012.07.021

PubMed Abstract | CrossRef Full Text | Google Scholar

Ding, L., and Gold, J. I. (2010). Caudate Encodes Multiple Computations for Perceptual Decisions. J. Neurosci. 30, 15747–15759. doi: 10.1523/jneurosci.2894

PubMed Abstract | CrossRef Full Text | Google Scholar

Ding, L., and Gold, J. (2013). The Basal Ganglia’s Contributions to Perceptual Decision Making. Neuron. 79, 640–649. doi: 10.1016/j.neuron.2013.07.042

PubMed Abstract | CrossRef Full Text | Google Scholar

Egner, T., Monti, J. M., and Summerfield, C. (2010). Expectation and surprise determine neural population responses in the ventral visual stream. J. Neurosci. 30, 16601–16608. doi: 10.1523/JNEUROSCI.2770-10.2010

PubMed Abstract | CrossRef Full Text | Google Scholar

Egner, T., and Summerfield, C. (2013). Grounding predictive coding models in empirical neuroscience research. Behav. Brain Sci. 36, 210–211. doi: 10.1017/S0140525X1200218X

PubMed Abstract | CrossRef Full Text | Google Scholar

Friston, K. (2003). Learning and inference in the brain. Neural Netw. 16, 1325–1352. doi: 10.1016/j.neunet.2003.06.005

PubMed Abstract | CrossRef Full Text | Google Scholar

Friston, K. (2011). What is optimal about motor control? Neuron. 72, 488–498. doi: 10.1016/j.neuron.2011.10.018

PubMed Abstract | CrossRef Full Text | Google Scholar

Friston, K. J., and Stephan, K. E. (2007). Free-energy and the brain. Synthese. 159, 417–458. doi: 10.1007/s11229-007-9237-y

PubMed Abstract | CrossRef Full Text | Google Scholar

Frank, M. J., Loughry, B., and O’Reilly, R. C. (2001). Interactions between the frontal cortex and basal ganglia in working memory: A computational model. Cogn. Affect. Behav. Neurosci. 1, 137–160. doi: 10.3758/cabn.1.2.137

PubMed Abstract | CrossRef Full Text | Google Scholar

Fuster, J. M. (2006). The cognit: a network model of cortical representation. Int. J. Psychophysiol. 60, 125–132. doi: 10.1016/j.ijpsycho.2005.12.015

PubMed Abstract | CrossRef Full Text | Google Scholar

Fuster, J. M. (2009). Cortex and memory: emergence of a new paradigm. J. Cog. Neurosci. 21, 2047–2072. doi: 10.1162/jocn.2009.21280

PubMed Abstract | CrossRef Full Text | Google Scholar

Gold, J. I., and Shadlen, M. N. (2000). Representation of a perceptual decision in developing oculomotor commands. Nature. 404, 390–394. doi: 10.1038/35006062

PubMed Abstract | CrossRef Full Text | Google Scholar

Goldman-Rakic, P. S. (1988). Topography of cognition: parallel distributed networks in primate association cortex. Annu. Rev. Neurosci. 11, 137–156. doi: 10.1146/annurev.neuro.11.1.137

PubMed Abstract | CrossRef Full Text | Google Scholar

Gross, H., Heinze, A., Seiler, T., and Stephan, V. (1999). Generative character of perception: a neural architecture for sensorimotor anticipation. Neural Netw. 12, 1101–1129. doi: 10.1016/s0893-6080(99)00047-7

PubMed Abstract | CrossRef Full Text | Google Scholar

Grush, R. (2004). The emulation theory of representation: motor control, imagery and perception. Behav. Brain Sci. 27, 377–442. doi: 10.1017/s0140525x04000093

PubMed Abstract | CrossRef Full Text | Google Scholar

Grush, R. (2007). Skill theory v2. 0: dispositions, emulation and spatial perception. Synthese. 159, 389–416. doi: 10.1007/s11229-007-9236-z

CrossRef Full Text | Google Scholar

Haber, S. N., and Calzavara, R. (2009). The cortico-basal ganglia integrative network: the role of the thalamus. Brain Res. Bull. 78, 69–74. doi: 10.1016/j.brainresbull.2008.09.013

PubMed Abstract | CrossRef Full Text | Google Scholar

Hawkins, J., and Blakeslee, S. (2004). On Intelligence. New York: Times Books.

Google Scholar

Hazy, T. E., Frank, M. J., and O’Reilly, R. C. (2006). Banishing the Homunculus: Making Working Memory Work.Neuroscience. 139, 105–118. doi: 10.1016/j.neuroscience.2005.04.067

PubMed Abstract | CrossRef Full Text | Google Scholar

Hazy, T. E., Frank, M. J., and O’Reilly, R. C. (2007). Towards an executive without a homunculus: computational models of the prefrontal cortex/basal ganglia system. Philos. Trans. R. Soc. Lond. B. Biol. Sci. 362, 1601–1613. doi: 10.1098/rstb.2007.2055

PubMed Abstract | CrossRef Full Text | Google Scholar

Hikosaka, O., Nakamura, K., and Nakahara, H. (2006). Basal ganglia orient eyes to reward. J. Neurophysiol. 95, 567–584. doi: 10.1152/jn.00458.2005

PubMed Abstract | CrossRef Full Text | Google Scholar

von Holst, E., and Mittelstaedt, H. (1950). Das reafferenz princip: (Wedlselwirkungen zwischen Zentrainervensystem und Peripherie). Die Naturwissenschaften. 37, 464–476. doi: 10.1007/bf00622503

CrossRef Full Text | Google Scholar

Huang, Y., and Rao, R. P. (2011). Predictive coding. Wiley Interdiscip. Rev. Cogn. Sci., 2, 580–593. doi: 10.1002/wcs.142

PubMed Abstract | CrossRef Full Text | Google Scholar

Hwang, E. J. (2013). The basal ganglia, the ideal machinery for the cost-benefit analysis of action plans. Front. Neural Circuits. 7:121. doi: 10.3389/fncir.2013.00121

PubMed Abstract | CrossRef Full Text | Google Scholar

Klein-Flügge, M. C., and Bestmann, S. (2012). Time-dependent changes in human corticospinal excitability reveal value-based competition for action during decision processing. J. Neurosci. 32, 8373–8382. doi: 10.1523/jneurosci.0270-12.2012

PubMed Abstract | CrossRef Full Text | Google Scholar

Koechlin, E., Ody, C., and Kouneiher, F. (2003). The architecture of cognitive control in the human prefrontal cortex. Science. 302, 1181–1185. doi: 10.1126/science.1088545

PubMed Abstract | CrossRef Full Text | Google Scholar

Lepora, N. F., and Gurney, K. N. (2012). The basal ganglia optimize decision making over general perceptual hypotheses. Neural Comput. 24, 2924–2945. doi: 10.1162/neco_a_00360

PubMed Abstract | CrossRef Full Text | Google Scholar

Mehta, B., and Schaal, S. (2002). Forward models in visuomotor control. J. Neurophysiol. 88, 942–953.

PubMed Abstract | Google Scholar

Miall, R. C., Christensen, L. O., Cain, O., and Stanley, J. (2007). Disruption of state estimation in the human lateral cerebellum. PLoS Biol. 5:e316. doi: 10.1371/journal.pbio.0050316

PubMed Abstract | CrossRef Full Text | Google Scholar

Middleton, F. A., and Strick, P. L. (2002). Basal-ganglia ‘projections’ to the prefrontal cortex of the primate. Cereb. Cortex. 12, 926–935. doi: 10.1093/cercor/12.9.926

PubMed Abstract | CrossRef Full Text | Google Scholar

Middleton, F. A., and Strick, P. L. (1996). The temporal lobe is a target of output from the basal ganglia. Proc. Natl. Acad. Sci. U S A 93, 8683–8687. doi: 10.1073/pnas.93.16.8683

PubMed Abstract | CrossRef Full Text | Google Scholar

Mink, J. W. (1996). The basal ganglia: focused selection and inhibition of competing motor programs. Prog. Neurobiol. 50, 381–425. doi: 10.1016/s0301-0082(96)00042-1

PubMed Abstract | CrossRef Full Text | Google Scholar

Nakamura, K., Santos, G. S., Matsuzaki, R., and Nakahara, H. (2012). Differential Reward Coding in the Subdivisions of the Primate Caudate during an Oculomotor Task. J. Neurosci. 32, 15963–15982. doi: 10.1523/jneurosci.1518-12.2012

PubMed Abstract | CrossRef Full Text | Google Scholar

O’Reilly, R. C. (2010). The What and How of prefrontal cortical organization. Trends Neurosci. 33, 355–361. doi: 10.1016/j.tins.2010.05.002

PubMed Abstract | CrossRef Full Text | Google Scholar

O’Doherty, J., Dayan, P., Schultz, J., Deichmann, R., Friston, K., and Dolan, R. J. (2004). Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science. 304, 452–454. doi: 10.1126/science.1094285

PubMed Abstract | CrossRef Full Text | Google Scholar

Pais-Vieira, M., Lebedev, M. A., Wiest, M. C., and Nicolelis, M. A. (2013). Simultaneous top-down modulation of the primary somatosensory cortex and thalamic nuclei during active tactile discrimination. J. Neurosci. 33, 4076–4093. doi: 10.1523/jneurosci.1659-12.2013

PubMed Abstract | CrossRef Full Text | Google Scholar

Panichello, M. F., Cheung, O. S., and Bar, M. (2013). Predictive feedback and conscious visual experience. Front. Psychol. 3:620. doi: 10.3389/fpsyg.2012.00620

PubMed Abstract | CrossRef Full Text | Google Scholar

Pezzulo, G., Butz, M., Castelfranchi, C., and Falcone, R. (2008). “Anticipation in natural and artificial cognition,” in The Challenge of Anticipation: A Unifying Framework for the Analysis and Design of Artificial Cognitive Systems, eds G. Pezzulo, M. Butz, C. Castelfranchi, and R. Falcone (New York: Springer), 3–22. doi: 10.1007/978-3-540-87702-8_1

CrossRef Full Text | Google Scholar

Poulet, J. F. A., and Hedwig, B. (2006). The cellular basis of a corollary discharge. Science. 311, 518–522. doi: 10.1126/science.1120847

PubMed Abstract | CrossRef Full Text | Google Scholar

Pribram, K. (1971). Languages of the Brain: Experimental Paradoxes and Principles of Neuropsychology. (Englewood Cliffs, NJ: Prentice-Hall, Inc).

Google Scholar

Prinz, W. (1997). Perception and action planning. Eur. J. Cogn. Psychol. 9, 129–154. doi: 10.1080/713752551

CrossRef Full Text | Google Scholar

Rao, R. P., and Ballard, D. H. (1999). Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. Nat. Neurosci. 2, 79–87. doi: 10.1038/4580

PubMed Abstract | CrossRef Full Text | Google Scholar

Rao, R. P. N. (2010). Decision making under uncertainty: a neural model based on partially observable Markov decision processes. Front. Comput. Neurosci. 4, 146. doi: 10.3389/fncom.2010.00146

PubMed Abstract | CrossRef Full Text | Google Scholar

Redgrave, P., Prescott, T. J., and Gurney, K. (1999). The basal ganglia: a vertebrate solution to the selection problem? Neuroscience. 89, 1009–1023. doi: 10.1016/s0306-4522(98)00319-4

PubMed Abstract | CrossRef Full Text | Google Scholar

Rizzolatti, G., and Luppino, G. (2001). The cortical motor system. Neuron. 31, 889–901. doi: 10.1016/S0896-6273(01)00423-8

PubMed Abstract | CrossRef Full Text | Google Scholar

Rosenkilde, C. E., Bauer, R. H., and Fuster, J. M. (1981). Single cell activity in ventral prefrontal cortex of behaving monkeys. Brain Res. 209, 375–394. doi: 10.1016/0006-8993(81)90160-8

PubMed Abstract | CrossRef Full Text | Google Scholar

Ross, J., Morrone, M. C., Goldberg, M. E., and Burr, D. C. (2001). Changes in visual perception at the time of saccades. Trends Neurosci. 24, 113–121. doi: 10.1016/s0166-2236(00)01685-4

PubMed Abstract | CrossRef Full Text | Google Scholar

Schroll, H., and Hamker, F. H. (2013). Computational models of basal-ganglia pathway functions: focus on functional neuroanatomy. Front. Syst. Neurosci. 7:122. doi: 10.3389/fnsys.2013.00122

PubMed Abstract | CrossRef Full Text | Google Scholar

Seriès, P., and Seitz, A. R. (2013). Learning what to expect (in visual perception). Front. Hum. Neurosci. 7:668. doi: 10.3389/fnhum.2013.00668

PubMed Abstract | CrossRef Full Text | Google Scholar

Shuler, M. G., and Bear, M. F. (2006). Reward timing in the primary visual cortex. Science. 311, 1606–1609. doi: 10.1126/science.1123513

PubMed Abstract | CrossRef Full Text | Google Scholar

Stanişor, L., van der Togt, C., Pennartz, C. M. A., and Roelfsema, P. R. (2013). A unified selection signal for attention and reward in primary visual cortex. Proc. Natl. Acad. Sci. U S A 110, 9136–9141. doi: 10.1073/pnas.1300117110

PubMed Abstract | CrossRef Full Text | Google Scholar

Sterzer, P., Frith, C., and Petrovic, P. (2008). Believing is seeing: expectations alter visual awareness. Curr. Biol. 18, 697–698. doi: 10.1016/j.cub.2008.06.021

PubMed Abstract | CrossRef Full Text | Google Scholar

Summerfield, C., and Egner, T. (2009). Expectation (and attention) in visual cognition. Trends Cogn. Sci. 13, 403–409. doi: 10.1016/j.tics.2009.06.003

PubMed Abstract | CrossRef Full Text | Google Scholar

Summerfield, C., Egner, T., Greene, M., Koechlin, E., Mangels, J., and Hirsch, J. (2006). Predictive codes for forthcoming perception in the frontal cortex. Science. 314, 1311–1314. doi: 10.1126/science.1132028

PubMed Abstract | CrossRef Full Text | Google Scholar

Wolpe, N., Wolpert, D. M., and Rowe, J. B. (2014). Seeing what you want to see: priors for one’s own actions represent exaggerated expectations of success. Front. Behav. Neurosci. 8:232. doi: 10.3389/fnbeh.2014.00232

PubMed Abstract | CrossRef Full Text | Google Scholar

Yanike, M., and Ferrera, V. P. (2014). Representation of Outcome Risk and Action in the Anterior Caudate Nucleus. J. Neurosci. 34, 3279–3290. doi: 10.1523/jneurosci.3818-13.2014

PubMed Abstract | CrossRef Full Text | Google Scholar

Yeo, B. T., Krienen, F. M., Sepulcre, J., Sabuncu, M. R., Lashkari, D., Hollinshead, M., et al. (2011). The organization of the human cerebral cortex estimated by intrinsic functional connectivity. J. Neurophysiol. 106, 1125–1126. doi: 10.1152/jn.00338.2011

PubMed Abstract | CrossRef Full Text | Google Scholar