Timing is not Everything: Neuromodulation Opens the STDP Gate

Abstract

Spike timing dependent plasticity (STDP) is a temporally specific extension of Hebbian associative plasticity that has tied together the timing of presynaptic inputs relative to the postsynaptic single spike. However, it is difficult to translate this mechanism to in vivo conditions where there is an abundance of presynaptic activity constantly impinging upon the dendritic tree as well as ongoing postsynaptic spiking activity that backpropagates along the dendrite. Theoretical studies have proposed that, in addition to this pre- and postsynaptic activity, a “third factor” would enable the association of specific inputs to specific outputs. Experimentally, the picture that is beginning to emerge, is that in addition to the precise timing of pre- and postsynaptic spikes, this third factor involves neuromodulators that have a distinctive influence on STDP rules. Specifically, neuromodulatory systems can influence STDP rules by acting via dopaminergic, noradrenergic, muscarinic, and nicotinic receptors. Neuromodulator actions can enable STDP induction or – by increasing or decreasing the threshold – can change the conditions for plasticity induction. Because some of the neuromodulators are also involved in reward, a link between STDP and reward-mediated learning is emerging. However, many outstanding questions concerning the relationship between neuromodulatory systems and STDP rules remain, that once solved, will help make the crucial link from timing-based synaptic plasticity rules to behaviorally based learning.

Introduction

The first groundbreaking in vitro STDP studies seemed to paint a very clear picture: The near-coincidence of presynaptic input and postsynaptic spiking enables neurons to enhance or decrease their synaptic weights depending on the exact timing of these two events (Magee and Johnston, 1997; Markram et al., 1997; Bi and Poo, 1998; Debanne et al., 1998). This finding was a giant step forward in our view of synaptic plasticity rules: It tied together the idea that both single spikes and their precise timing matter; with the implication that neurons have a means to associate arriving inputs with the outgoing spikes and adapt the synaptic weights accordingly. It was therefore very intuitive to postulate STDP as a more temporally specific extension of Hebbian associative learning and experience driven plasticity (for definition, see below; and for modeling approaches see: Gerstner et al., 1993, 1996; Abbott and Blum, 1996; Blum and Abbott, 1996; Mehta et al., 2000).

However, for this idea to be relevant for behavioral learning as first formally proposed by Hebb (1949), it must hold true in vivo. This is where the STDP concept faces two conundrums, the first based on ongoing spiking activity, and the second based on the timing of spikes in relation to behavioral outcome.

• (1)

  Unlike in vitro, in vivo there are continuous barrages of ongoing presynaptic activity impinging on the dendritic tree that can generate postsynaptic spikes (Figures 1A,B) (Wilson and Groves, 1981; Cowan et al., 1994; Wilson and Kawaguchi, 1996; Stern et al., 1997). These spikes backpropagate throughout the neuron (Waters et al., 2003; Waters and Helmchen, 2004), potentially interacting with the vast number of dendritically located synapses as these synapses continue to receive barrages of excitatory inputs (Figure 1C₂). Under these conditions, the implications of STDP rules on individual synapses would be that a synapse active just prior to a spike event will increase in efficacy, whereas a synapse that is active just after the spike, will decrease its efficacy. The question arises, whether the mere association of presynaptic input and postsynaptic spiking activity would be enough to alter synaptic efficacy, and whether individual synapses in turn continuously scale up and down as inputs and backpropagating spikes constantly interact? Moreover, do spontaneously occurring spikes (Figure 1C₂) and stimulus-evoked spikes (Figure 1C₃) equally change synaptic weights as they interact with presynaptic input (Figure 1D)? When a spike is fired, whether it is spontaneous or evoked, how are active synaptic inputs that are driven by a stimulus separated from those that are due to the ongoing activity? One possible solution to these selectivity problems was originally proposed in relation to reward mediated learning (Miller, 1981; Wickens, 1990). These theoretical studies proposed that, in addition to the associated pre- and postsynaptic activity, a “third factor” was available to the network that enabled both the temporal and the spatial selection of specific inputs. To apply this to the in vivo situation, raises several further questions: could a neuromodulator represent such a third factor for selecting specific active inputs to a neuron that is embedded in a continuously active network? Given that many repetitions of timed pre–post pairings are typically necessary for STDP induction, could a third factor modify the number of repetitions needed for plasticity induction? In addition to the third factor requirement, other possible solutions for plasticity induction in vivo have been proposed that are not covered in the present review (Gerstner et al., 1996; Kempter et al., 1999; Beggs, 2001; Seung, 2003; Xie and Seung, 2004).

• (2)

  If spike timing dependent synaptic plasticity rules are the basis for the modification of behavior, and neuromodulatory systems are critical for this process, then a second conundrum of temporal credit assignment is faced. Both the behavioral signals and the behavioral outcome must be taken into account temporally. This likely also includes the activation of subcortical modulatory nuclei that can mediate for example alerting or rewarding signals to target structures (Schultz, 2000). How does such behavioral activation of a modulatory center influence the interaction of near-coincident pre- and postsynaptic activity with spatial and temporal specificity? Near-coincident pre- and postsynaptic spiking activity and neuromodulators most likely act on different timescales, ranging from tens of milliseconds for pre- and postsynaptic spikes to seconds or longer for some neuromodulators (for review see: Schultz, 2007). How will these different temporal activation schemes work together during behavioral learning? This temporal credit assignment problem is not new in neuroscience, as for example in reward mediated learning the “distal-reward problem” has been recognized years ago: How can the reward relate to specific events that happened earlier in time than the reward (Hull, 1943; Blum and Abbott, 1996; Schultz, 1998, 2006; Sutton and Barto, 1998; Izhikevich, 2007; Vasilaki et al., 2009)? Specific subcortical “reward systems” have been implicated in such learning with the neuromodulator dopamine being the most characterized (for review see: Schultz, 2000, 2002).

Figure 1

Although the rules associated with STDP have started to be addressed in vivo (Meliza and Dan, 2006; Jacob et al., 2007), to date, all the data about the involvement of neuromodulators in STDP have come from in vitro studies. In vitro, the dopaminergic system, amongst a number of other neuromodulatory systems, has been found to influence timing-dependent plasticity (Bissiere et al., 2003; Couey et al., 2007; Seol et al., 2007; Lin et al., 2008; Pawlak and Kerr, 2008; Shen et al., 2008; Zhang et al., 2009). The following sections will present the existing in vitro experimental evidence concerning how neuromodulators are involved in timing-based plasticity. This review will be restricted to neuromodulator-actions on timing-dependent plasticity in the mammalian central nervous system. Furthermore, we concentrate on long-range neuromodulatory systems (that are thought to become activated by distinct behavioral states in vivo), although locally acting systems and retrograde messengers undoubtedly play an important role in STDP. Such locally acting systems important for STDP are for example endocannabinoids (Sjostrom et al., 2003; Bender et al., 2006; Nevian and Sakmann, 2006; Tzounopoulos et al., 2007), metabotropic glutamate receptors (mGluRs; Egger et al., 1999), and brain-derived neurotrophic factor (Mu and Poo, 2006; Sakata et al., 2009; Sivakumaran et al., 2009).

Experimental Evidence for Involvement of Neuromodulators in STDP

Neuromodulators are involved in most forms of synaptic plasticity ranging from short-term plasticity (ms) (for review see: Lovinger, 2010) to long-term plasticity (hours) (Neuman and Harley, 1983; Frey et al., 1990; Huerta and Lisman, 1993; Thomas et al., 1996), to experience-dependent plasticity (Bear and Singer, 1986; Kilgard and Merzenich, 1998) as well as structural plasticity (Ingham et al., 1998; Day et al., 2006; Gerfen, 2006) (for definitions, see below). Although over the past few decades the role of neuromodulation in certain forms of synaptic plasticity that mainly used high frequency stimulation induction protocols has been well established, it is not clear how these results relate to STDP (for reviews see: Jay, 2003; Hasselmo, 2006; Sara, 2009; Wickens, 2009).

To identify common neuromodulatory rules across the existing STDP studies is potentially difficult as not all studies have used the same induction protocols. The induction protocols used range from pairing of single spikes with a single synaptic input (Lin et al., 2003; Couey et al., 2007; Pawlak and Kerr, 2008; Zhang et al., 2009), and spike bursts with a single synaptic input (Seol et al., 2007; Shen et al., 2008) to “theta” burst paradigms, in which multiple excitatory postsynaptic potentials (EPSPs) are interleaved with multiple spikes (Bissiere et al., 2003; Shen et al., 2008) (see also Table 1). The use of multiple spikes with multiple EPSPs during theta burst protocols, usually evoked at around 30–50 Hz, does not allow true timing to be investigated as the preceding spike is always temporally close to the following evoked EPSP (for more details see: Froemke et al., 2010) but this may, in some cell types, be closer to what occurs during behavior. Studies into such complex pairing protocols indicate that the complex EPSP–spike interactions affect downstream signaling cascades differently to seemingly more simple EPSP–single spike interactions (Wang et al., 2005; Froemke et al., 2006). Also, other alterations to STDP recording conditions, for example the presence of GABAergic transmission or pre–post repetition rate may change the STDP window (for further reading see: Wickens, 2009). This opens the possibility that neuromodulators can activate different second messenger pathways depending upon the STDP induction protocol and recording conditions that were used. The identification of common neuromodulatory rules is further complicated by the use of different tools to manipulate neuromodulatory systems amongst studies, for example application of receptor agonists, receptor antagonists, or of the neuromodulator itself, often with different application times.

Therefore, while we attempt to summarize neuromodulatory actions during timing-based plasticity in the following paragraph, one should be aware that differences in STDP induction protocols as well as method of neuromodulatory manipulation might impede finding common principles of neuromodulatory actions across studies.

Permission to change: neuromodulators and STDP

Studies investigating the effect of neuromodulators on STDP have either used a fixed relative pre–post timing to induce timing-dependent long-term potentiation and depression (t-LTP or t-LTD) (Bissiere et al., 2003; Couey et al., 2007; Pawlak and Kerr, 2008; Shen et al., 2008), or used a whole range of pre–post stimulation timings to investigate the effects of neuromodulation on the STDP timing window shape (Lin et al., 2003; Seol et al., 2007; Zhang et al., 2009). As will be discussed in detail in the following section, the studies using a fixed pre–post timing have often identified neuromodulatory signaling as a requirement for STDP to occur.

Of the neuromodulators investigated, dopamine is the most widely studied and has been shown to influence timing-dependent plasticity across several brain regions. In amygdala, t-LTP was only induced by a protocol consisting of short bursts of afferent stimulation timed to action potential (AP) bursts, when either dopamine was applied or GABAergic inhibition was blocked (Bissiere et al., 2003). Here, dopamine acted by activating dopamine D2 receptors, thereby suppressing feedforward inhibition from local interneurons, which permitted t-LTP induction by burst-pairing. The effect of dopamine depended on intact GABAergic transmission, since no potentiation occurred when dopamine was applied during pairing when GABAergic transmission was blocked, suggesting that the pairing protocol triggers different processes depending on the absence or presence of synaptic inhibition (Bissiere et al., 2003). In dorsal striatum under GABA_A block, timing-dependent LTP was induced when a single AP closely followed an EPSP, while timing-dependent LTD was induced when the order was reversed (Pawlak and Kerr, 2008). Here, blocking dopamine D1/D5 receptors prevented t-LTP as well as t-LTD (Figures 2A,B), while blocking dopamine D2 receptors altered the onset, but not the final peak change in plasticity.

Figure 2

Hence, the activation of dopamine D1/D5 receptors allowed for two events, the presynaptic input and the postsynaptic spike that occurred on a timescale of a few tens of milliseconds, to induce a lasting change in synaptic efficiency. Because dopamine receptors were blocked throughout the experiment with specific antagonists, the issue still remains whether dopamine alone acts in the same way when applied during the induction period (see “Changes to the Shape of the STDP Window”). In the subpopulation of striatal principal neurons that do not express dopamine D1 receptors, other neuromodulatory receptor systems were required for STDP. Here, adenosine A2 receptors, which are coupled to the similar second messenger cascades as D1 receptors (Premont et al., 1977; Schwarzschild et al., 2006) had to be activated for t-LTP induction. Both, endocannabinoid CB1 receptor as well as dopamine D2 receptor activation was required for t-LTD induction (Shen et al., 2008). In contrast to the amygdala, in the striatum the effect of dopamine on STDP seems to operate independently of GABAergic transmission (Pawlak and Kerr, 2008; Shen et al., 2008).

Nicotine was shown to be involved in STDP by acting on nicotinic acetylcholine receptors (nAChRs) in prefrontal cortex (Couey et al., 2007). Here, nicotine application caused normally t-LTP-inducing pre–post pairings, consisting of EPSP and single APs to induce a small amount of t-LTD in layer 5 pyramidal neurons. As an underlying mechanism, nicotine was found to strongly increase inhibition of layer 5 pyramidal neurons. Accordingly, the blocking effect of nicotine on t-LTP was partly overcome when inhibition was also blocked or a stronger t-LTP-inducing stimulus, consisting of pairing EPSPs with AP bursts, was applied. Since calcium is thought to be a crucial second messenger in synaptic plasticity induced by using spike-timing or other plasticity inducing protocols (for reviews see: Artola and Singer, 1993; Malenka and Bear, 2004), and since pre–post timing protocols produce characteristic spatiotemporal calcium signals (Koester and Sakmann, 1998), Couey et al. (2007) also investigated dendritic calcium dynamics during AP–EPSP pairing. Under the influence of nicotine, calcium changes were reduced during a pairing protocol that normally induced t-LTP in control condition (single-AP pairings). In contrast, during a stronger t-LTP-inducing protocol (AP-burst pairings), changes in dendritic calcium were comparable between control groups and nicotine groups. Both, pre- and postsynaptic nAChRs, distributed across several classes of interneurons, were suggested as potential targets of nicotine when reversing prefrontal t-LTP into t-LTD (Couey et al., 2007).

Not only manipulations of nicotinergic signaling, but also manipulations of the balance between dopaminergic and adenosinergic signaling are capable of reversing the sign of plasticity (i.e., converting t-LTP into t-LTD or vice versa); when dopaminergic signaling via D2 receptors was blocked and adenosine signaling was “boosted” by application of adenosine A2 receptor agonists, t-LTP was reversed into t-LTD upon a pre–post timing protocol (Shen et al., 2008).

In summary, dopamine receptor activation is often a prerequisite for timing-dependent plasticity to occur, and nicotine's action is to increase the threshold for t-LTP induction. While the investigation of neuromodulatory influences using a few selective EPSP–AP timing protocols has led to important insights into specifically dopamine's and nicotine's actions during STDP, a more complete picture emerges when neuromodulatory influences are studied across the entire STDP window.

Changes to the shape of the STDP window

In hippocampus, unlike in striatum and amygdala, dopamine receptor activation was not a critical requirement for STDP induction. Here, t-LTP was induced by a pre–post protocol in the presence of dopamine receptor blockers (Zhang et al., 2009). However, dopamine application resulted in a modest, albeit not significant, increase in the amount of t-LTP observed when single postsynaptic APs closely followed the presynaptic activation with a delay of 10 ms. The much more dramatic effect observed with dopamine application was a change in the shape of the STDP window, allowing for longer pre–post timing delays to increase synaptic efficiency (Zhang et al., 2009). This widening effect was attributed to dopamine D1/D5 receptor activation, and was estimated to expand the t-LTP window by at least 25 ms. Surprisingly, t-LTD, as normally induced by a post–pre protocol, was converted into t-LTP by dopamine (Figure 3A). A similar broadening effect on the t-LTP window during the pairing of single APs with presynaptic activation was reported for the neuromodulator noradrenaline in hippocampal CA1 neurons (Lin et al., 2003). While the activation of β-adrenergic receptors widened the t-LTP window by about 15 ms, the overall amount of plasticity induced by the close pre–post pairings was not affected. Post–pre pairings were not tested. An unexpected similarity between both, β-adrenergic and dopaminergic actions on the “widening” of the t-LTP window is that the effect was expressed slowly, meaning that synaptic efficiency was unchanged directly after the pairing protocol and gradually started to increase from around 15 min post pairing protocol. The implications of window widening are that the activation of dopamine or noradrenaline receptors reduces the threshold for t-LTP induction by allowing for a wider range of pre–post timings to induce synaptic potentiation. The next question is whether the number of spikes needed to induce plasticity is altered with neuromodulation?

Figure 3

Changes to the number of trials required to achieve plasticity

In addition to having an effect on the shape of the STDP window, dopamine also affected the number of pre–post pairing episodes required to induce plasticity in hippocampal neurons (Zhang et al., 2009). Specifically, when dopamine D1/D5 receptors were activated, successful t-LTP induction required a strongly reduced number of timed pre–post pairings, namely instead of the typically required 60 pairing trials, less than 10 pairings were required (Figure 3B). Thus by decreasing the required number of spike pairings, dopamine decreases the threshold for t-LTP induction.

Cooperation between neuromodulators

As suggested by the anatomy of converging neuromodulatory fibers as well as direct physiological evidence, one neuromodulator often does not act in isolation, but several neuromodulatory systems interact (for example, see Bear and Singer, 1986; Zhou et al., 2001). In visual cortex layer 2/3 pyramidal neurons, pairing stimulation of layer 4 afferents with AP bursts did not result in plasticity, neither for pre–post protocols, nor for post–pre protocols (Seol et al., 2007; but compare Feldman, 2000; Froemke et al., 2005). For plasticity to occur, neuromodulatory receptors had to be activated during the pre–post timing protocols. Specifically under stimulation with β-adrenergic agonists, pre–post pairings, with timings between −50 and +50 ms, always induced t-LTP. Conversely, activation of M1 muscarinic receptors always resulted in t-LTD within the same range of timings. Finally, the “normal” standard STDP window displaying bidirectional plasticity, with causal pre–post timings leading to t-LTP and anticausal post–pre timings leading to t-LTD, was achieved with the combined application of β-adrenergic and M1 muscarinic agonists (Figure 4).

Figure 4

The activation of β-adrenergic and M1 muscarinic receptors resulted in temporary phosphorylation of distinct sites at AMPA receptors that have been suggested to be crucial for t-LTP and t-LTD, respectively. This led the authors to conclude that neuromodulators supply AMPA receptors with distinct “tags” that allow during certain pre–post spiking timings the induction of t-LTP and t-LTD, respectively. In summary, the activation of noradrenergic and muscarinic acetylcholine receptors (mAChRs) is required for STDP, and more specifically, only the concurrent activation of the two neuromodulatory systems is required to achieve a “standard” STDP window with a t-LTP and a t-LTD side.

Conclusion of this section

The observed neuromodulatory actions so far can be divided into two categories (see also Table 1): In the first category, neuromodulator receptor activation is necessary for plasticity (Bissiere et al., 2003; Seol et al., 2007; Pawlak and Kerr, 2008; Shen et al., 2008), thereby representing in addition to the precise timing of pre- and postsynaptic activity, a third factor essentially required for plasticity induction. Notably, in one study, two neuromodulators acted in concert to enable bidirectional STDP (Seol et al., 2007).

In the second category, the neuromodulator changes the conditions for plasticity by either increasing (Couey et al., 2007) or decreasing the threshold for plasticity induction (Lin et al., 2003; Zhang et al., 2009; but see also Bienenstock et al., 1982). An effect observed in studies from both categories is that specific manipulations of one or several neuromodulator systems, result in sign reversal of plasticity, meaning that a normally t-LTP-inducing stimulus induced t-LTD, or vice versa (Bissiere et al., 2003; Couey et al., 2007; Seol et al., 2007; Shen et al., 2008; Zhang et al., 2009). Although the mechanisms underlying such sign reversal are not clear, activity patterns that “boost” backpropagating APs in remote dendrites have been shown to “switch” t-LTD to t-LTP (Sjostrom and Hausser, 2006). Since some neuromodulators can exert a short-term effect on dendritic excitability and backpropagation (for review see: Waters et al., 2005), neuromodulators could also modulate backpropagating APs during STDP protocols, although this might not occur with all neuromodulators (Gulledge and Stuart, 2003).

The effect of neuromodulators on dendritic excitability is not restricted to short-term effects, since for example, the combination of mACh receptor activation with weak dendritic spikes in a distinct dendritic compartment resulted in a long-lasting excitability increase restricted to the involved dendritic compartment (Losonczy et al., 2008). This excitability increase transformed the weak dendritic spikes into strong dendritic spikes. Strong dendritic spikes have been implicated in drastic trial reduction to induce plasticity (Remy and Spruston, 2007). Such dendritic spikes are usually evoked with rather focused massive or convergent activity. Alternatively, a neuromodulator could reduce the trial number required for plasticity induction by making a dendritic compartment receptive for strong dendritic spike initiation (Losonczy et al., 2008).

From a temporal point of view, neuromodulators have been found to influence STDP on at least three timescales: on the scale of tens of milliseconds, neuromodulators influenced the interaction of pre- and postsynaptic spikes to induce plasticity; on the scale of seconds, neuromodulators influenced the number of repetitions of pre–post activity needed to evoke plasticity; and on the order of minutes, neuromodulators influenced the time course of plasticity.

Does the Third Factor Have a Timing Issue, Too?

The studies listed in Section “Experimental Evidence for Involvement of Neuromodulators in STDP” have either constantly manipulated neuromodulator receptors during the entire experimental period (Lin et al., 2003; Pawlak and Kerr, 2008; Shen et al., 2008) or only during the induction period (Bissiere et al., 2003; Couey et al., 2007; Seol et al., 2007; Zhang et al., 2009). An important question that is very difficult to address experimentally, is how the outcome, in terms of plasticity, depends on the exact point in time of activation of neuromodulator-receptors, relative to pre- and postsynaptic spiking?

Since neuromodulator release sites and the receptors for the neuromodulator are not necessarily located close together on either side of the synaptic cleft, as in the classical concept of a synapse, the time required for diffusion of the released molecules has to be taken into account (see also Figure 2C). The modulator molecules have first to be released, then “travel” and bind to the respective receptors and initiate some G-protein coupled signaling cascade, which is a very different scenario from fast glutamate transmission. The time course of neuromodulator action was traditionally thought to be slow (on the scale of minutes), but recent evidence suggests that the time course is on the order of a few seconds (for review see: Sarter et al., 2009). Despite this recent change in thinking the question still arises how do these different timescales of spikes (1–2 ms) and neuromodulation (seconds) fit together in a working mechanism?

Three possible scenarios can be devised of how such a mechanism could work. The first two are at the single neuron level involving an “eligibility trace”, and the third is at the network level and relies on reverberating activity.

In the first scenario, the coordinated pre–post activity occurs before the neuromodulator release, as would be the case during unexpected reward. Here, spike and synaptic activation could leave a time decaying eligibility trace (Wang et al., 2000; Sarkisov and Wang, 2008) that subsequent neuromodulator receptor activation may then interact with to modulate plasticity.

In the second scenario, the neuromodulatory receptors are activated before the coordinated pre–post activity occurs, as would be expected during the learning of attention-based tasks. Here, the signaling mechanisms activated by neuromodulatory receptors themselves may create a slowly decaying eligibility trace, with which the coordinated pre–post activity can then interact with to modulate plasticity.

Experimental evidence for either scenario or the underlying molecular mechanisms is mostly lacking. However, the two presented scenarios resemble problems faced in the field of metaplasticity (Abraham, 2008) in which the concept of an eligibility trace has also been proposed. During metaplasticity, synapses will more easily undergo plasticity after a “priming” stimulus has changed the state of specific molecular signaling cascades; this change may for example “kick” plasticity-relevant enzymes into a more receptive state or it may result in enhanced phosphorylation of intracellular or extrasynaptic AMPARs that allows them to be inserted into postsynaptic membranes when an appropriate stimulus arrives (Sun et al., 2005; Abraham, 2008). For STDP, evidence for a similar gating mechanism in accordance with scenario two (neuromodulator receptor activation occurs first, followed by near-coincident pre–post spiking) was found by Seol and colleagues (see Sections “Experimental Evidence for Involvement of Neuromodulators in STDP and “Cellular and Molecular Targets of Neuromodulators During Timing-Dependent Plasticity”). When M1 muscarinic and β-adrenergic agonists were applied and washed out, a subsequent episode of timed pre–post pairings still initiated t-LTP or t-LTD, respectively. In addition, because neuromodulators have been shown to have a direct effect on glutamatergic receptor (AMPA and NMDA) location within the synapse and activated current efficacy (Seol et al., 2007; for review see: Cepeda and Levine, 2006), this is a possible mechanism that could create an eligibility trace to interact with subsequent pre–post pairing.

A third scenario how the three factors may interact in vivo, may be that the respective pre–post-activity patterns “reverberate” in the local circuit for some time (Hebb, 1949), and that such a memory trace can be transduced into a lasting modification if a third-factor success signal is present during the reverberation (Histed et al., 2009). A problem with such a mechanism is that the reverberating activity should not produce overt action, however if different circuits are involved, it is difficult to connect the success signal to the activity that did cause overt action.

How Can Neuromodulators Influence the Interaction of Pre- and Postsynaptic Spikes: Anatomical and Physiological Considerations

Increasing evidence for the critical involvement of neuromodulator systems in STDP raises the question of how the physical location of neuromodulator release sites relates to the pre- and postsynaptic complex, which is thought to be the locus of STDP induction. Typically, neuromodulatory centers are located quite distally from the brain regions they influence (see more in Section “Activation of Neuromodulatory Systems In Vivo”). In their distal target areas, generally only a subgroup of neuromodulatory fibers makes direct contact with dendritic spines that receive excitatory inputs (Freund et al., 1984; Groves et al., 1994; Smith et al., 1994), whereas other neuromodulatory fibers target dendritic shafts and somata or form varicosities that lack synaptic specializations (Seguela et al., 1989, 1990). Therefore the question arises if only this subset of directly targeted synapses is influenced by neuromodulators when pre- and postsynaptic spikes collide? This is unlikely, because the receptors for the respective neurotransmitters are widely distributed across pre- and postsynaptic sites of principal neurons and interneurons (Gerfen et al., 1990, 1995; Sesack et al., 1994; Bergson et al., 1995; Caille et al., 1996) (Figure 2C), and receptor density and location can change (Paspalas et al., 2006). A certain degree of spatial specificity of neuromodulator-actions is probably achieved during behavior, when phasic release events occur, which temporally increases the concentration of neuromodulator, locally (reviewed in: Arbuthnott and Wickens, 2007; Sarter et al., 2009, see below for definition of phasic release and also Section “Activation of Neuromodulatory Systems In Vivo”). Finally, both AMPA and NMDA receptors are located presynaptically on neuromodulatory release terminals, and the activation of these receptors by overspill from neighboring active glutamatergic synapses is thought to convey further spatial specificity to the neuromodulator signal (Roberts and Sharif, 1978; Desce et al., 1992, 1994; Jin and Fredholm, 1994). Because the actions of neuromodulators are through receptors, the specific receptor subtype involved in STDP has important implications for the interpretation of neuromodulatory actions at the single neuron level. Indeed, during STDP, neuromodulators acted through specific receptor subtypes (Bissiere et al., 2003; Lin et al., 2003; Seol et al., 2007; Pawlak and Kerr, 2008; Shen et al., 2008; Zhang et al., 2009). This is a complex issue as, for example for the neuromodulator dopamine, low concentrations are thought to activate dopamine D2-like receptors in their high-affinity state, whereas high concentrations are thought to activate dopamine D1-like receptors (Richfield et al., 1989). These two receptor-subgroups are differentially expressed across neuronal populations, and can activate opposing downstream target enzymes (Girault and Greengard, 2004). Hence, during phasic release, the heterogeneous structural arrangement of release sites, different receptors subtypes, and regulated degrading/reuptake mechanisms in combination with diffusional processes are likely to generate further spatiotemporal heterogeneity of the neuromodulator signal. In addition, there is strong evidence that local spines within a dendritic region are topographically organized functionally (Jia et al., 2010) and that activity-initiated signaling cascades within the postsynaptic spines and dendrites interact locally with other spines (Harvey and Svoboda, 2007). This implies that in addition to the spatial specificity of neuromodulator release, there is a postsynaptic organization that can potentially provide very spatially defined neuromodulator action without the need for individual fibers to innervate each and every postsynaptic spine. The implication for timing-dependent plasticity in vivo would be that the timing of neuromodulator release in relation to correlated pre- and postsynaptic activity can enable the spatiotemporal selection of specific synapses for plasticity.

Cellular and Molecular Targets of Neuromodulators During Timing-Dependent Plasticity

Only few studies so far have addressed the issue of the exact cellular and molecular targets of neuromodulators when they “gate” STDP. In general, their receptors are (often) coupled to G-proteins and hereby influence intracellular second messenger cascades; (for example dopamine D1 receptors are coupled directly to adenylyl cyclase (AC) and indirectly to protein kinase A (PKA) and protein phosphatase 1 (Hemmings et al., 1984),M1 muscarinic ACh receptors are coupled to phospholipase C (PLC), β-adrenergic receptors are coupled to PKA (exception: nicotinic ACh receptors are ligand-gated channels). As a result, many different voltage-gated and calcium-dependent ion channels are influenced, which can affect membrane potential, neuronal spiking and excitatory transmission as well as inhibitory transmission (for review, see: Hasselmo, 1995; Nicola et al., 2000; Magee and Johnston, 2005; Sara, 2009). By their ability to affect dendritic ion channels, neuromodulators are certainly empowered to influence how the backpropagating AP will interact with incoming synaptic input during spike-timing paradigms (Hoffman and Johnston, 1999; Zhou et al., 2005; Sjostrom and Hausser, 2006; for review see: Tsubokawa, 2000), although specific studies investigating neuromodulatory influences on such interactions are required (but see: Couey et al., 2007).

Particularly in older animals, a preventing effect of inhibition on STDP has been described (Meredith et al., 2003). The influence that some neuromodulators have on inhibitory tone is certainly a means to affect STDP rules (D2, A2, mGluR5; Bissiere et al., 2003; Schwarzschild et al., 2006; Couey et al., 2007). However, several studies describe an effect of neuromodulators on STDP while inhibition is blocked, indicating at least one alternative mode of action of neuromodulators (Pawlak and Kerr, 2008; Shen et al., 2008).

Such an alternative mode of action is putatively an influence on the postsynaptic anchoring of glutamate receptors. For example, dopamine D1/D5 receptors and β-adrenergic receptors can increase surface expression of AMPA receptors (Chao et al., 2002; Sun et al., 2005; Oh et al., 2006) promoting synaptic insertion (for reviews see: Derkach et al., 2007; Lee and Huganir, 2008). The trafficking of AMPA receptors in and out of the synapse depends on phosphorylation of AMPA receptors at distinct sites (Lee et al., 2000, 2003; Boehm et al., 2006; He et al., 2009). In agreement with this, acetylcholine (coupled to PLC via M1 muscarinic receptors) and noradrenaline (coupled to AC via β-adrenergic receptors) gate phosphorylation at AMPA receptor sites implicated in t-LTP and t-LTD (Seol et al., 2007). In addition, β-adrenergic receptors were recently found to be anchored postsynaptically, forming a signaling complex with PKA and AMPA receptors (Joiner et al., 2010). Also, for dopamine, a complex interaction between D1 receptors and NMDA receptor channels has been reported (Cepeda et al., 1992; O'Donnell and Grace, 1994; Levine et al., 1996; Gao et al., 2001; Cepeda and Levine, 2006). Given that both dopamine and NMDA receptor activation were required for STDP (Pawlak and Kerr, 2008; Shen et al., 2008; Zhang et al., 2009), an interaction of these two receptor systems during the correlated pre- and postsynaptic spiking is possible.

Finally, it is worth considering the possibility that neuromodulators might also alter the dynamic balance of the phosphatases and kinases that control the induction of t-LTP and t-LTD (for review see: Lisman and McIntyre, 2001). For example, it is well established that PKA can reduce the activation of the phosphatases subserving LTD (Blitzer et al., 1998). This could be a plausible mechanism to account for the observation made in some studies that receptors coupled to AC, like D1 dopaminergic and β-adrenergic receptors, not only promote t-LTP but prevent t-LTD (Seol et al., 2007; Lin et al., 2008; Zhang et al., 2009).

In summary, the neuromodulatory systems can potentially affect STDP through a variety of mechanisms like changing the recruitment of inhibition at the network level, or changing excitability, kinase/phosphatase balance and priming AMPA receptor trafficking at the subcellular level. An important question that remains largely open is how those multiple actions are integrated in the different behavioral states defined by the neuromodulatory systems.

Activation of Neuromodulatory Systems In Vivo

If one attempts a synthesis of STDP and neuromodulation, the question arises at which points during behavior neuromodulatory nuclei become activated? Due to tonic background activity of these nuclei, their innervated areas experience a constant low tone of release resulting in neuromodulator concentrations in the low nanomolar range. Salient behavioral events serve to drastically increase and decrease the activity of the respective nuclei (see below). The exact spatiotemporal profile of neuromodulator concentrations achieved during behavior is mostly unknown. Perhaps the best studied neuromodulator in this respect is dopamine, and some information about dopamine's in vivo concentration is available (see below), whereas for noradrenaline and acetylcholine the concentration reached during behavior is not well studied.

Dopamine

Dopaminergic fibers arise from the ventral tegmental area and the substantia nigra. Dopaminergic neurons are activated by primary-rewarding stimuli: Unexpected rewards, but also the attentional and rewarding aspects of novel stimuli cause midbrain dopaminergic neurons to increase their firing rate (Ljungberg et al., 1992; Mirenowicz and Schultz, 1996). As a certain task is being learned, dopamine neurons shift their firing temporally toward the stimulus that indicates reward is to follow (Schultz et al., 1993). Hereby, the success-predicting stimulus has become rewarding. After a certain task has been learned, the primary-rewarding stimulus does not activate dopaminergic signals anymore; a dopaminergic signal is only initiated when a reward is unexpected or better than predicted. If a predicted reward is omitted, dopaminergic cells respond by decreasing their firing (Hollerman and Schultz, 1998). Together, this is consistent with theories of reinforcement learning stating that reinforcers only contribute to learning when they are not entirely predictable (Sutton and Barto, 1981). Recently, it has been found that a subpopulation of dopamine neurons also fire in response to aversive stimuli or associated cues (Joshua et al., 2008; Matsumoto and Hikosaka, 2009) suggesting that dopamine can code for multiple external events (Redgrave and Gurney, 2006).

The timescale of the phasic increase in firing rate of dopaminergic neurons is 50–110 ms (latency) and <200 ms (duration) with dopamine concentrations at target structures remaining elevated (150–400 nM) for up to 400 ms (Chergui et al., 1994; Dugast et al., 1994; Schultz, 2002). It is less clear how pauses in dopamine cell firing would affect local concentration levels, since the time course of clearance is relatively slow. However subtle changes in the degree of synchrony of firing have significant effects (Joshua et al., 2009).

Noradrenaline

Noradrenaline neurons located in locus coeruleus seem to play a role in vigilance, since these neurons show low firing rates during drowsiness and slow-wave sleep, regular firing at quiet wakefulness, and burst-firing in response to arousing stimuli (Aston-Jones and Bloom, 1981). A large variety of arousing and attention-demanding stimuli cause a response in these noradrenergic neurons, this also includes primary-rewarding stimuli and aversive stimuli (Foote et al., 1980; Rasmussen et al., 1986; Sara and Segal, 1991; Aston-Jones et al., 1994). In detail, this response consists of a very brief increase in AP firing (15–70 ms latency, 2–3 APs) followed by a longer suppression of AP firing (300–700 ms duration) (Berridge and Waterhouse, 2003). The noradrenergic response disappears with repeated stimulus presentations, but reappears when the stimulus is followed by reinforcement (Sara and Segal, 1991). In general, the noradrenaline signal is thought to be involved in sensory processing, decision-making, working memory, and memory formation (Cahill and McGaugh, 1996; Robbins and Roberts, 2007).

Acetylcholine

Cholinergic neurons in the basal forebrain nuclei are activated during arousal and attention (Paxinos, 2004; Sarter et al., 2005), they respond to unfamiliar stimuli (Wilson and Rolls, 1990), but also to unpredicted and predicted rewards (Richardson and DeLong, 1986, 1990). Also in striatum, tonically active striatal interneurons (TANs), which are cholinergic (Wilson et al., 1990; Zhou et al., 2002) respond to primary rewards and reward-predicting stimuli with a pause and sometimes a subsequent increase in firing (Aosaki et al., 1995; Apicella et al., 1997; Sardo et al., 2000; Zhou et al., 2002). The firing of TANs mainly encodes outcome delivery and omission at termination of the behavioral trial episode (Joshua et al., 2008). Within the cortex, acetylcholine has been suggested to enhance the response to sensory stimuli, and on more broad terms, to be important for attention and working memory (Hasselmo and Giocomo, 2006).

Conclusion of this section

This section shows that release of neuromodulators occurs in a wide range of behavioral situations. Hence, the combination of theoretical and experimental work suggests that neuromodulatory influence on STDP might be linked to an equally wide range of behavioral learning processes, namely fear-conditioning (Bissiere et al., 2003), rapid learning (Zhang et al., 2009), reward-based learning (Pawlak and Kerr, 2008; Zhang et al., 2009), cognitive performance (Couey et al., 2007), but also pathological states (Shen et al., 2008) (for modeling approaches see: Baras and Meir, 2007; Florian, 2007; Legenstein et al., 2008; Vasilaki et al., 2009; Fremaux et al., 2010; Potjans et al., 2010).

Future Directions

Our knowledge about STDP and its regulation by neuromodulators has substantially increased during the last years, although the overall number of published studies concerning this topic remains low. Current experimental evidence suggests that neuromodulators shape the interaction between presynaptic and postsynaptic spike activity across many brain areas, and the predominating effect of neuromodulators is to allow plasticity or to make plasticity induction easier. Although there is amassing data from many different brain regions, it needs to be clarified how universal this additional modulatory factor is in regulating STDP. In addition, many brain areas are targeted and influenced by not only one, but by several neuromodulators (Bear and Singer, 1986; Zhou et al., 2001; Wang et al., 2006; Sara, 2009), and accordingly, there is experimental evidence that several neuromodulatory systems can interact to influence STDP rules (Seol et al., 2007). How the interaction of multiple neuromodulator systems exactly occurs during STDP, and if this interaction is a universal principle across many brain areas, will be interesting targets for future studies. Perhaps the most important outstanding question regarding STDP and neuromodulation concerns the exact time, when neuromodulatory receptors need to be activated to exert an influence on synaptic efficacy during causal and anticausal pre- and postsynaptic spiking. Along these lines, it will be important to directly test whether neuromodulators are capable of spatial or temporal selection of specific synapses for plasticity (see Sections “How Can Neuromodulators Influence the Interaction of Pre- and Postsynaptic Spikes: Anatomical and Physiological Considerations” and “Conclusion”).

Finally, not only will both experimentalists and theorists need to translate the effect of neuromodulators on STDP rules from in vitro to in vivo conditions (see: Meliza and Dan, 2006; Jacob et al., 2007), but also to the behaving animal. In addition, since it is almost certain that specific memories are stored across neuronal populations (Penfield, 1958, 1959), it will be important to see how STDP rules relate to neuronal populations in the behaving animal (Sawinski et al., 2009).

Conclusion

Spike timing dependent plasticity rules have been developed mainly on the basis of in vitro experimental data and have provided a temporally specific extension to the activity-based synaptic plasticity rules first proposed by Hebb (1949). However, when one tries to apply these rules to in vivo conditions and to the behaving animal, two conundrums arise: First, in vivo, a large amount of pre- and postsynaptic activity constantly arrives at the individual synapses. This raises the possibility that in vivo, synapses are constantly adapting their synaptic efficacy as pre- and postsynaptic spikes collide, which would be energetically inefficient for the involved neurons. An alternative possibility is that a “third factor” using a neuromodulator signal may represent a selection criteria that potentially allows presynaptic activity and postsynaptic spiking to be associated, both spatially and temporally. Thus, neuromodulators might enable the neuronal networks to select certain inputs and to make them eligible for changes in efficacy. To this end a large amount of indirect in vitro experimental evidence from many brain regions as well as theoretical evidence is being amassed that this may be the case, but a direct measurement of the third factor rule in vivo has yet to be achieved (for experimental evidence, see Section “Experimental Evidence for Involvement of Neuromodulators in STDP”; for modeling approaches see: Baras and Meir, 2007; Florian, 2007; Izhikevich, 2007; Legenstein et al., 2008; Vasilaki et al., 2009; Fremaux et al., 2010; Potjans et al., 2010).

Second, if one attempts to transfer the concepts of STDP to in vivo conditions, the obvious next questions are (a) if STDP rules are actually used for behaviorally based learning, and (b) how neuromodulation might be involved in this process. Neuromodulation alone is certainly an important factor involved in behavioral learning, as demonstrated by decades of research. If neuromodulation was instrumental in shaping STDP rules during behaviorally based learning, it would require fast time scale events like pre- and postsynaptic spikes and the putatively “slowly acting” neuromodulators to interact. The existing in vitro studies are only starting to provide insights how this temporal interaction might work. And for the in vivo situation, this picture will be much more complex, as a variety of behavioral states will release different combinations of neuromodulators at different timings and at different concentrations, activating different target receptor subtypes.

Dopamine, to date, is the most investigated neuromodulator and represents an interesting case for neuromodulator-regulation of STDP, as it has both the effect of “broadening” the t-LTP window and of changing what would normally be t-LTD into t-LTP. With regards to reward-mediated learning, the implication of these experimental findings is that any spike occurring within a certain window either before or after the synaptic input will increase the synaptic efficacy, which implies that many different external events that occurred temporally around the rewarding event could be associated with the reward.

It is an open question how an animal succeeds in linking the specific neuronal activity involved in a behavior to the behavioral outcome. How neuromodulators released during different states such as attention, arousal and reward influence this linking process, is also unknown. To achieve a full understanding of the principles of how neuromodulation shapes STDP rules might represent a first step toward solving these important questions.

References

• 1

  AbbottL. F.BlumK. I. (1996). Functional significance of long-term potentiation for sequence learning and prediction. Cereb. Cortex6, 406–416.10.1093/cercor/6.3.406

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 2

  AbrahamW. C. (2008). Metaplasticity: tuning synapses and networks for plasticity. Nat. Rev. Neurosci.9, 387.10.1038/nrn2356

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 3

  AosakiT.KimuraM.GraybielA. M. (1995). Temporal and spatial characteristics of tonically active neurons of the primate's striatum. J. Neurophysiol.73, 1234–1252.

  • Pubmed Abstract
  • Google Scholar

• 4

  ApicellaP.LegalletE.TroucheE. (1997). Responses of tonically discharging neurons in the monkey striatum to primary rewards delivered during different behavioral states. Exp. Brain Res.116, 456–466.10.1007/PL00005773

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 5

  ArbuthnottG. W.WickensJ. (2007). Space, time and dopamine. Trends Neurosci.30, 62–69.10.1016/j.tins.2006.12.003

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 6

  ArtolaA.SingerW. (1993). Long-term depression of excitatory synaptic transmission and its relationship to long-term potentiation. Trends Neurosci.16, 480–487.10.1016/0166-2236(93)90081-V

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 7

  Aston-JonesG.BloomF. E. (1981). Activity of norepinephrine-containing locus coeruleus neurons in behaving rats anticipates fluctuations in the sleep-waking cycle. J. Neurosci.1, 876–886.

  • Pubmed Abstract
  • Google Scholar

• 8

  Aston-JonesG.RajkowskiJ.KubiakP.AlexinskyT. (1994). Locus coeruleus neurons in monkey are selectively activated by attended cues in a vigilance task. J. Neurosci.14, 4467–4480.

  • Pubmed Abstract
  • Google Scholar

• 9

  BarasD.MeirR. (2007). Reinforcement learning, spike-time-dependent plasticity, and the BCM rule. Neural. Comput.19, 2245–2279.10.1162/neco.2007.19.8.2245

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 10

  BearM. F.SingerW. (1986). Modulation of visual cortical plasticity by acetylcholine and noradrenaline. Nature320, 172–176.10.1038/320172a0

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 11

  BeggsJ. M. (2001). A statistical theory of long-term potentiation and depression. Neural. Comput.13, 87–111.10.1162/089976601300014646

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 12

  BenderV. A.BenderK. J.BrasierD. J.FeldmanD. E. (2006). Two coincidence detectors for spike timing-dependent plasticity in somatosensory cortex. J. Neurosci.26, 4166–4177.10.1523/JNEUROSCI.0176-06.2006

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 13

  BergsonC.MrzljakL.SmileyJ. F.PappyM.LevensonR.Goldman-RakicP. S. (1995). Regional, cellular, and subcellular variations in the distribution of D1 and D5 dopamine receptors in primate brain. J. Neurosci.15, 7821–7836.

  • Pubmed Abstract
  • Google Scholar

• 14

  BerridgeC. W.WaterhouseB. D. (2003). The locus coeruleus-noradrenergic system: modulation of behavioral state and state-dependent cognitive processes. Brain Res. Brain Res. Rev.42, 33–84.10.1016/S0165-0173(03)00143-7

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 15

  BiG. Q.PooM. M. (1998). Synaptic modifications in cultured hippocampal neurons: dependence on spike timing, synaptic strength, and postsynaptic cell type. J. Neurosci.18, 10464–10472.

  • Pubmed Abstract
  • Google Scholar

• 16

  BienenstockE. L.CooperL. N.MunroP. W. (1982). Theory for the development of neuron selectivity: orientation specificity and binocular interaction in visual cortex. J. Neurosci.2, 32–48.

  • Pubmed Abstract
  • Google Scholar

• 17

  BissiereS.HumeauY.LuthiA. (2003). Dopamine gates LTP induction in lateral amygdala by suppressing feedforward inhibition. Nat. Neurosci.6, 587–592.10.1038/nn1058

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 18

  BlissT. V.CollingridgeG. L. (1993). A synaptic model of memory: long-term potentiation in the hippocampus. Nature361, 31–39.10.1038/361031a0

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 19

  BlitzerR. D.ConnorJ. H.BrownG. P.WongT.ShenolikarS.IyengarR.LandauE. M. (1998). Gating of CaMKII by cAMP-regulated protein phosphatase activity during LTP. Science280, 1940–1942.10.1126/science.280.5371.1940

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 20

  BlumK. I.AbbottL. F. (1996). A model of spatial map formation in the hippocampus of the rat. Neural. Comput.8, 85–93.10.1162/neco.1996.8.1.85

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 21

  BoehmJ.KangM. G.JohnsonR. C.EstebanJ.HuganirR. L.MalinowR. (2006). Synaptic incorporation of AMPA receptors during LTP is controlled by a PKC phosphorylation site on GluR1. Neuron51, 213–225.10.1016/j.neuron.2006.06.013

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 22

  CahillL.McGaughJ. L. (1996). Modulation of memory storage. Curr. Opin. Neurobiol.6, 237–242.10.1016/S0959-4388(96)80078-X

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 23

  CailleI.DumartinB.BlochB. (1996). Ultrastructural localization of D1 dopamine receptor immunoreactivity in rat striatonigral neurons and its relation with dopaminergic innervation. Brain Res.730, 17–31.

  • Pubmed Abstract
  • Google Scholar

• 24

  CepedaC.LevineM. S. (2006). Where do you think you are going? The NMDA-D1 receptor trap. Sci. STKE2006, pe20.10.1126/stke.3332006pe20

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 25

  CepedaC.RadisavljevicZ.PeacockW.LevineM. S.BuchwaldN. A. (1992). Differential modulation by dopamine of responses evoked by excitatory amino acids in human cortex. Synapse11, 330–341.10.1002/syn.890110408

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 26

  ChaoS. Z.ArianoM. A.PetersonD. A.WolfM. E. (2002). D1 dopamine receptor stimulation increases GluR1 surface expression in nucleus accumbens neurons. J. Neurochem.83, 704–712.10.1046/j.1471-4159.2002.01164.x

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 27

  CherguiK.Suaud-ChagnyM. F.GononF. (1994). Nonlinear relationship between impulse flow, dopamine release and dopamine elimination in the rat brain in vivo. Neuroscience62, 641–645.10.1016/0306-4522(94)90465-0

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 28

  CoueyJ. J.MeredithR. M.SpijkerS.PoorthuisR. B.SmitA. B.BrussaardA. B.MansvelderH. D. (2007). Distributed network actions by nicotine increase the threshold for spike-timing-dependent plasticity in prefrontal cortex. Neuron54, 73–87.10.1016/j.neuron.2007.03.006

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 29

  CowanR. L.SesackS. R.Van BockstaeleE. J.BranchereauP.ChainJ.PickelV. M. (1994). Analysis of synaptic inputs and targets of physiologically characterized neurons in rat frontal cortex: combined in vivo intracellular recording and immunolabeling. Synapse17, 101–114.10.1002/syn.890170206

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 30

  DayM.WangZ.DingJ.AnX.InghamC. A.SheringA. F.WokosinD.IlijicE.SunZ.SampsonA. R.MugnainiE.DeutchA. Y.SesackS. R.ArbuthnottG. W.SurmeierD. J. (2006). Selective elimination of glutamatergic synapses on striatopallidal neurons in Parkinson disease models. Nat. Neurosci.9, 251–259.10.1038/nn1632

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 31

  DebanneD.GahwilerB. H.ThompsonS. M. (1998). Long-term synaptic plasticity between pairs of individual CA3 pyramidal cells in rat hippocampal slice cultures. J. Physiol.507(Pt 1), 237–247.10.1111/j.1469-7793.1998.237bu.x

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 32

  DerkachV. A.OhM. C.GuireE. S.SoderlingT. R. (2007). Regulatory mechanisms of AMPA receptors in synaptic plasticity. Nat. Rev. Neurosci.8, 101–113.10.1038/nrn2055

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 33

  DesceJ. M.GodeheuG.GalliT.ArtaudF.CheramyA.GlowinskiJ. (1992). L-glutamate-evoked release of dopamine from synaptosomes of the rat striatum: involvement of AMPA and N-methyl-D-aspartate receptors. Neuroscience47, 333–339.10.1016/0306-4522(92)90249-2

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 34

  DesceJ. M.GodeheuG.GalliT.GlowinskiJ.CheramyA. (1994). Opposite presynaptic regulations by glutamate through NMDA receptors of dopamine synthesis and release in rat striatal synaptosomes. Brain Res.640, 205–214.10.1016/0006-8993(94)91874-0

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 35

  DugastC.Suaud-ChagnyM. F.GononF. (1994). Continuous in vivo monitoring of evoked dopamine release in the rat nucleus accumbens by amperometry. Neuroscience62, 647–654.10.1016/0306-4522(94)90466-9

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 36

  EggerV.FeldmeyerD.SakmannB. (1999). Coincidence detection and changes of synaptic efficacy in spiny stellate neurons in rat barrel cortex. Nat. Neurosci.2, 1098–1105.10.1038/16026

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 37

  FeldmanD. E. (2000). Timing-based LTP and LTD at vertical inputs to layer II/III pyramidal cells in rat barrel cortex. Neuron27, 45–56.10.1016/S0896-6273(00)00008-8

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 38

  FlorianR. V. (2007). Reinforcement learning through modulation of spike-timing-dependent synaptic plasticity. Neural. Comput.19, 1468–1502.10.1162/neco.2007.19.6.1468

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 39

  FooteS. L.Aston-JonesG.BloomF. E. (1980). Impulse activity of locus coeruleus neurons in awake rats and monkeys is a function of sensory stimulation and arousal. Proc. Natl. Acad. Sci. U.S.A.77, 3033–3037.10.1073/pnas.77.5.3033

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 40

  FoxK. (2009). Experience-dependent plasticity mechanisms for neural rehabilitation in somatosensory cortex. Philos. Trans. R. Soc. Lond., B, Biol. Sci.364, 369–381.10.1098/rstb.2008.0252

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 41

  FremauxN.SprekelerH.GerstnerW. (2010). Functional requirements for reward-modulated spike-timing dependent plasticity. J. Neurosci.30, 13326–13337.10.1523/JNEUROSCI.6249-09.2010

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 42

  FreundT. F.PowellJ. F.SmithA. D. (1984). Tyrosine hydroxylase-immunoreactive boutons in synaptic contact with identified striatonigral neurons, with particular reference to dendritic spines. Neuroscience13, 1189–1215.10.1016/0306-4522(84)90294-X

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 43

  FreyU.SchroederH.MatthiesH. (1990). Dopaminergic antagonists prevent long-term maintenance of posttetanic LTP in the CA1 region of rat hippocampal slices. Brain Res.522, 69–75.10.1016/0006-8993(90)91578-5

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 44

  FroemkeR. C.DebanneD.BiG.-Q. (2010). Temporal modulation of spike-timing-dependent plasticity. Front. Syn. Neurosci.2:12. 10.3389/fnsyn.2010.00019

  • CrossRef
  • Google Scholar

• 45

  FroemkeR. C.PooM. M.DanY. (2005). Spike-timing-dependent synaptic plasticity depends on dendritic location. Nature434, 221–225.10.1038/nature03366

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 46

  FroemkeR. C.TsayI. A.RaadM.LongJ. D.DanY. (2006). Contribution of individual spikes in burst-induced long-term synaptic modification. J. Neurophysiol.95, 1620–1629.10.1152/jn.00910.2005

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 47

  GaoW. J.KrimerL. S.Goldman-RakicP. S. (2001). Presynaptic regulation of recurrent excitation by D1 receptors in prefrontal circuits. Proc. Natl. Acad. Sci. U.S.A.98, 295–300.10.1073/pnas.011524298

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 48

  GerfenC. R. (2006). Indirect-pathway neurons lose their spines in Parkinson disease. Nat. Neurosci.9, 157–158.10.1038/nn0206-157

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 49

  GerfenC. R.EngberT. M.MahanL. C.SuselZ.ChaseT. N.MonsmaF. J.JrSibleyD. R. (1990). D1 and D2 dopamine receptor-regulated gene expression of striatonigral and striatopallidal neurons. Science250, 1429–1432.10.1126/science.2147780

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 50

  GerfenC. R.KeefeK. A.GaudaE. B. (1995). D1 and D2 dopamine receptor function in the striatum: coactivation of D1- and D2-dopamine receptors on separate populations of neurons results in potentiated immediate early gene response in D1-containing neurons. J. Neurosci.15, 8167–8176.

  • Pubmed Abstract
  • Google Scholar

• 51

  GerstnerW.KempterR.van HemmenJ. L.WagnerH. (1996). A neuronal learning rule for sub-millisecond temporal coding. Nature383, 76–81.10.1038/383076a0

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 52

  GerstnerW.RitzR.van HemmenJ. L. (1993). Why spikes? Hebbian learning and retrieval of time-resolved excitation patterns. Biol. Cybern.69, 503–515.

  • Pubmed Abstract
  • Google Scholar

• 53

  GiraultJ. A.GreengardP. (2004). The neurobiology of dopamine signaling. Arch. Neurol.61, 641–644.10.1001/archneur.61.5.641

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 54

  GrovesP. M.LinderJ. C.YoungS. J. (1994). 5-hydroxydopamine-labeled dopaminergic axons: three-dimensional reconstructions of axons, synapses and postsynaptic targets in rat neostriatum. Neuroscience58, 593–604.10.1016/0306-4522(94)90084-1

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 55

  GulledgeA. T.StuartG. J. (2003). Action potential initiation and propagation in layer 5 pyramidal neurons of the rat prefrontal cortex: absence of dopamine modulation. J. Neurosci.23, 11363–11372.

  • Pubmed Abstract
  • Google Scholar

• 56

  HarveyC. D.SvobodaK. (2007). Locally dynamic synaptic learning rules in pyramidal neuron dendrites. Nature450, 1195–1200.10.1038/nature06416

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 57

  HasselmoM. E. (1995). Neuromodulation and cortical function: modeling the physiological basis of behavior. Behav. Brain Res.67, 1–27.10.1016/0166-4328(94)00113-T

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 58

  HasselmoM. E. (2006). The role of acetylcholine in learning and memory. Curr. Opin. Neurobiol.16, 710–715.10.1016/j.conb.2006.09.002

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 59

  HasselmoM. E.GiocomoL. M. (2006). Cholinergic modulation of cortical function. J. Mol. Neurosci.30, 133–135.10.1385/JMN:30:1:133

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 60

  HeK.SongL.CummingsL. W.GoldmanJ.HuganirR. L.LeeH. K. (2009). Stabilization of Ca²⁺-permeable AMPA receptors at perisynaptic sites by GluR1-S845 phosphorylation. Proc. Natl. Acad. Sci. U.S.A.106, 20033–20038.

  • Pubmed Abstract
  • Google Scholar

• 61

  HebbD. O. (1949). The Organization of Behavior. A Neuropsychological Theory. New York: John Wiley & Sons, Inc.

  • Google Scholar

• 62

  HemmingsH. C.JrGreengardP.TungH. Y.CohenP. (1984). DARPP-32, a dopamine-regulated neuronal phosphoprotein, is a potent inhibitor of protein phosphatase-1. Nature310, 503–505.10.1038/310503a0

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 63

  HistedM. H.PasupathyA.MillerE. K. (2009). Learning substrates in the primate prefrontal cortex and striatum: sustained activity related to successful actions. Neuron63, 244–253.10.1016/j.neuron.2009.06.019

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 64

  HoffmanD. A.JohnstonD. (1999). Neuromodulation of dendritic action potentials. J. Neurophysiol.81, 408–411.

  • Pubmed Abstract
  • Google Scholar

• 65

  HollermanJ. R.SchultzW. (1998). Dopamine neurons report an error in the temporal prediction of reward during learning. Nat. Neurosci.1, 304–309.10.1038/1124

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 66

  HoltmaatA.SvobodaK. (2009). Experience-dependent structural synaptic plasticity in the mammalian brain. Nat. Rev. Neurosci.10, 647–658.10.1038/nrn2699

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 67

  HooksB. M.ChenC. (2007). Critical periods in the visual system: changing views for a model of experience-dependent plasticity. Neuron56, 312–326.10.1016/j.neuron.2007.10.003

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 68

  HuertaP. T.LismanJ. E. (1993). Heightened synaptic plasticity of hippocampal CA1 neurons during a cholinergically induced rhythmic state. Nature364, 723–725.10.1038/364723a0

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 69

  HullC. L. (1943). Principles of Behavior. New York: Appleton-Century.

  • Google Scholar

• 70

  InghamC. A.HoodS. H.TaggartP.ArbuthnottG. W. (1998). Plasticity of synapses in the rat neostriatum after unilateral lesion of the nigrostriatal dopaminergic pathway. J. Neurosci.18, 4732–4743.

  • Pubmed Abstract
  • Google Scholar

• 71

  IzhikevichE. M. (2007). Solving the distal reward problem through linkage of STDP and dopamine signaling. Cereb. Cortex17, 2443–2452.10.1093/cercor/bhl152

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 72

  JacobV.BrasierD. J.ErchovaI.FeldmanD.ShulzD. E. (2007). Spike timing-dependent synaptic depression in the in vivo barrel cortex of the rat. J. Neurosci.27, 1271–1284.10.1523/JNEUROSCI.4264-06.2007

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 73

  JayT. M. (2003). Dopamine: a potential substrate for synaptic plasticity and memory mechanisms. Prog. Neurobiol.69, 375–390.10.1016/S0301-0082(03)00085-6

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 74

  JiaH.RochefortN. L.ChenX.KonnerthA. (2010). Dendritic organization of sensory input to cortical neurons in vivo. Nature464, 1307–1312.10.1038/nature08947

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 75

  JinS.FredholmB. B. (1994). Role of NMDA, AMPA and kainate receptors in mediating glutamate- and 4-AP-induced dopamine and acetylcholine release from rat striatal slices. Neuropharmacology33, 1039–1048.10.1016/0028-3908(94)90141-4

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 76

  JoinerM. L.LiseM. F.YuenE. Y.KamA. Y.ZhangM.HallD. D.MalikZ. A.QianH.ChenY.UlrichJ. D.BuretteA. C.WeinbergR. J.LawP. Y.El-HusseiniA.YanZ.HellJ. W. (2010). Assembly of a beta2-adrenergic receptor – GluR1 signalling complex for localized cAMP signalling. EMBO J.29, 482–495.10.1038/emboj.2009.344

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 77

  JoshuaM.AdlerA.MitelmanR.VaadiaE.BergmanH. (2008). Midbrain dopaminergic neurons and striatal cholinergic interneurons encode the difference between reward and aversive events at different epochs of probabilistic classical conditioning trials. J. Neurosci.28, 11673–11684.10.1523/JNEUROSCI.3839-08.2008

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 78

  JoshuaM.AdlerA.PrutY.VaadiaE.WickensJ. R.BergmanH. (2009). Synchronization of midbrain dopaminergic neurons is enhanced by rewarding events. Neuron62, 695–704.10.1016/j.neuron.2009.04.026

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 79

  KempterR.GerstnerW.van HemmenJ. L. (1999). Hebbian learning and spiking neurons. Phys. Rev. E59, 4498.10.1103/PhysRevE.59.4498

  • CrossRef
  • Google Scholar

• 80

  KilgardM. P.MerzenichM. M. (1998). Cortical map reorganization enabled by nucleus basalis activity. Science279, 1714–1718.10.1126/science.279.5357.1714

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 81

  KoesterH. J.SakmannB. (1998). Calcium dynamics in single spines during coincident pre- and postsynaptic activity depend on relative timing of back-propagating action potentials and subthreshold excitatory postsynaptic potentials. Proc. Natl. Acad. Sci. U.S.A.95, 9596–9601.10.1073/pnas.95.16.9596

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 82

  LamprechtR.LeDouxJ. (2004). Structural plasticity and memory. Nat. Rev. Neurosci.5, 45–54.10.1038/nrn1301

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 83

  LeeH. K.BarbarosieM.KameyamaK.BearM. F.HuganirR. L. (2000). Regulation of distinct AMPA receptor phosphorylation sites during bidirectional synaptic plasticity. Nature405, 955–959.10.1038/35016089

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 84

  LeeH. K.HuganirR. L. (2008). AMPA Receptor Regulation and the Reversal of Synaptic Plasticity – LTP, LTD, Depotentiation, and Dedepression, Vol. 4. San Diego: Elsevier Press.

  • Google Scholar

• 85

  LeeH. K.TakamiyaK.HanJ. S.ManH.KimC. H.RumbaughG.YuS.DingL.HeC.PetraliaR. S.WentholdR. J.GallagherM.HuganirR. L. (2003). Phosphorylation of the AMPA receptor GluR1 subunit is required for synaptic plasticity and retention of spatial memory. Cell112, 631–643.

  • Pubmed Abstract
  • Google Scholar

• 86

  LegensteinR.PecevskiD.MaassW. (2008). A learning theory for reward-modulated spike-timing-dependent plasticity with application to biofeedback. PLoS Comput. Biol.4: e1000180.10.1371/journal.pcbi.1000180

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 87

  LevineM. S.LiZ.CepedaC.CromwellH. C.AltemusK. L. (1996). Neuromodulatory actions of dopamine on synaptically-evoked neostriatal responses in slices. Synapse24, 65–78.10.1002/syn.890240102

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 88

  LinY. W.MinM. Y.ChiuT. H.YangH. W. (2003). Enhancement of associative long-term potentiation by activation of beta-adrenergic receptors at CA1 synapses in rat hippocampal slices. J. Neurosci.23, 4173–4181.

  • Pubmed Abstract
  • Google Scholar

• 89

  LinY. W.YangH. W.MinM. Y.ChiuT. H. (2008). Inhibition of associative long-term depression by activation of beta-adrenergic receptors in rat hippocampal CA1 synapses. J. Biomed. Sci.15, 123–131.10.1007/s11373-007-9205-z

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 90

  LismanJ. E.McIntyreC. C. (2001). Synaptic plasticity: a molecular memory switch. Curr. Biol.11, R788–R791.10.1016/S0960-9822(01)00472-9

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 91

  LjungbergT.ApicellaP.SchultzW. (1992). Responses of monkey dopamine neurons during learning of behavioral reactions. J. Neurophysiol.67, 145–163.

  • Pubmed Abstract
  • Google Scholar

• 92

  LosonczyA.MakaraJ. K.MageeJ. C. (2008). Compartmentalized dendritic plasticity and input feature storage in neurons. Nature452, 436–441.10.1038/nature06725

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 93

  LovingerD. M. (2010). Neurotransmitter roles in synaptic modulation, plasticity and learning in the dorsal striatum. Neuropharmacology58, 951–961.10.1016/j.neuropharm.2010.01.008

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 94

  LuscherC.NicollR. A.MalenkaR. C.MullerD. (2000). Synaptic plasticity and dynamic modulation of the postsynaptic membrane. Nat. Neurosci.3, 545–550.10.1038/75714

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 95

  MageeJ. C.JohnstonD. (1997). A synaptically controlled, associative signal for Hebbian plasticity in hippocampal neurons. Science275, 209–213.10.1126/science.275.5297.209

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 96

  MageeJ. C.JohnstonD. (2005). Plasticity of dendritic function. Curr. Opin. Neurobiol.15, 334–342.10.1016/j.conb.2005.05.013

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 97

  MalenkaR. C.BearM. F. (2004). LTP and LTD: an embarrassment of riches. Neuron44, 5–21.10.1016/j.neuron.2004.09.012

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 98

  MarkramH.LubkeJ.FrotscherM.SakmannB. (1997). Regulation of synaptic efficacy by coincidence of postsynaptic APs and EPSPs. Science275, 213–215.10.1126/science.275.5297.213

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 99

  MatsumotoM.HikosakaO. (2009). Two types of dopamine neuron distinctly convey positive and negative motivational signals. Nature459, 837–841.10.1038/nature08028

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 100

  MehtaM. R.QuirkM. C.WilsonM. A. (2000). Experience-dependent asymmetric shape of hippocampal receptive fields. Neuron25, 707–715.10.1016/S0896-6273(00)81072-7

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 101

  MelizaC. D.DanY. (2006). Receptive-field modification in rat visual cortex induced by paired visual stimulation and single-cell spiking. Neuron49, 183–189.10.1016/j.neuron.2005.12.009

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 102

  MeredithR. M.Floyer-LeaA. M.PaulsenO. (2003). Maturation of long-term potentiation induction rules in rodent hippocampus: role of GABAergic inhibition. J. Neurosci.23, 11142–11146.

  • Pubmed Abstract
  • Google Scholar

• 103

  MillerR. (1981). Meaning and Purpose in the Intact Brain. Oxford: Oxford University Press.

  • Google Scholar

• 104

  MirenowiczJ.SchultzW. (1996). Preferential activation of midbrain dopamine neurons by appetitive rather than aversive stimuli. Nature379, 449–451.10.1038/379449a0

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 105

  MuY.PooM. M. (2006). Spike timing-dependent LTP/LTD mediates visual experience-dependent plasticity in a developing retinotectal system. Neuron50, 115–125.10.1016/j.neuron.2006.03.009

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 106

  NeumanR. S.HarleyC. W. (1983). Long-lasting potentiation of the dentate gyrus population spike by norepinephrine. Brain Res.273, 162–165.10.1016/0006-8993(83)91106-X

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 107

  NevianT.SakmannB. (2006). Spine Ca²⁺ signaling in spike-timing-dependent plasticity. J. Neurosci.26, 11001–11013.10.1523/JNEUROSCI.1749-06.2006

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 108

  NicolaS. M.SurmeierJ.MalenkaR. C. (2000). Dopaminergic modulation of neuronal excitability in the striatum and nucleus accumbens. Annu. Rev. Neurosci.23, 185–215.10.1146/annurev.neuro.23.1.185

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 109

  O'DonnellP.GraceA. A. (1994). Tonic D2-mediated attenuation of cortical excitation in nucleus accumbens neurons recorded in vitro. Brain Res.634, 105–112.10.1016/0006-8993(94)90263-1

  • CrossRef
  • Google Scholar

• 110

  OhM. C.DerkachV. A.GuireE. S.SoderlingT. R. (2006). Extrasynaptic membrane trafficking regulated by GluR1 serine 845 phosphorylation primes AMPA receptors for long-term potentiation. J. Biol. Chem.281, 752–758.10.1074/jbc.M509677200

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 111

  PaspalasC. D.RakicP.Goldman-RakicP. S. (2006). Internalization of D2 dopamine receptors is clathrin-dependent and select to dendro-axonic appositions in primate prefrontal cortex. Eur. J. Neurosci.24, 1395–1403.10.1111/j.1460-9568.2006.05023.x

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 112

  PawlakV.KerrJ. N. (2008). Dopamine receptor activation is required for corticostriatal spike-timing-dependent plasticity. J. Neurosci.28, 2435–2446.10.1523/JNEUROSCI.4402-07.2008

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 113

  PaxinosG. (2004). The Rat Nervous System, 3rd Edn.London: Academic Press.

  • Google Scholar

• 114

  PenfieldW. (1958). Some mechanisms of consciousness discovered during electrical stimulation of the brain. Proc. Natl. Acad. Sci. U.S.A.44, 51–66.10.1073/pnas.44.2.51

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 115

  PenfieldW. (1959). The interpretive cortex; the stream of consciousness in the human brain can be electrically reactivated. Science129, 1719–1725.10.1126/science.129.3365.1719

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 116

  PotjansW.MorrisonA.DiesmannM. (2010). Enabling functional neural circuit simulations with distributed computing of neuromodulated plasticity. Front. Comput. Neurosci.4:141.10.3389/fncom.2010.00141

  • CrossRef
  • Google Scholar

• 117

  PremontJ.PerezM.BockaertJ. (1977). Adenosine-sensitive adenylate cyclase in rat striatal homogenates and its relationship to dopamine- and Ca²⁺-sensitive adenylate cyclases. Mol. Pharmacol.13, 662–670.

  • Pubmed Abstract
  • Google Scholar

• 118

  RasmussenK.MorilakD. A.JacobsB. L. (1986). Single unit activity of locus coeruleus neurons in the freely moving cat. I. During naturalistic behaviors and in response to simple and complex stimuli. Brain Res. 371, 324–334.10.1016/0006-8993(86)90370-7

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 119

  RedgraveP.GurneyK. (2006). The short-latency dopamine signal: a role in discovering novel actions?Nat. Rev. Neurosci.7, 967–975.10.1038/nrn2022

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 120

  RemyS.SprustonN. (2007). Dendritic spikes induce single-burst long-term potentiation. Proc. Natl. Acad. Sci. U.S.A.104, 17192–17197.10.1073/pnas.0707919104

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 121

  RichardsonR. T.DeLongM. R. (1986). Nucleus basalis of Meynert neuronal activity during a delayed response task in monkey. Brain Res.399, 364–368.10.1016/0006-8993(86)91529-5

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 122

  RichardsonR. T.DeLongM. R. (1990). Context-dependent responses of primate nucleus basalis neurons in a go/no-go task. J. Neurosci.10, 2528–2540.

  • Pubmed Abstract
  • Google Scholar

• 123

  RichfieldE. K.PenneyJ. B.YoungA. B. (1989). Anatomical and affinity state comparisons between dopamine D1 and D2 receptors in the rat central nervous system. Neuroscience30, 767–777.10.1016/0306-4522(89)90168-1

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 124

  RobbinsT. W.RobertsA. C. (2007). Differential regulation of fronto-executive function by the monoamines and acetylcholine. Cereb. Cortex17(Suppl. 1), i151–i160.10.1093/cercor/bhm066

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 125

  RobertsP. J.SharifN. A. (1978). Effects of l-glutamate and related amino acids upon the release of [3H]dopamine from rat striatal slices. Brain Res.157, 391–395.10.1016/0006-8993(78)90048-3

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 126

  SakataK.WooN. H.MartinowichK.GreeneJ. S.SchloesserR. J.ShenL.LuB. (2009). Critical role of promoter IV-driven BDNF transcription in GABAergic transmission and synaptic plasticity in the prefrontal cortex. Proc. Natl. Acad. Sci. U.S.A.106, 5942–5947.10.1073/pnas.0811431106

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 127

  SaraS. J. (2009). The locus coeruleus and noradrenergic modulation of cognition. Nat. Rev. Neurosci.10, 211–223.10.1038/nrn2573

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 128

  SaraS. J.SegalM. (1991). Plasticity of sensory responses of locus coeruleus neurons in the behaving rat: implications for cognition. Prog. Brain Res.88, 571–585.10.1016/S0079-6123(08)63835-2

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 129

  SardoP.RavelS.LegalletE.ApicellaP. (2000). Influence of the predicted time of stimuli eliciting movements on responses of tonically active neurons in the monkey striatum. Eur. J. Neurosci.12, 1801–1816.10.1046/j.1460-9568.2000.00068.x

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 130

  SarkisovD. V.WangS. S. (2008). Order-dependent coincidence detection in cerebellar Purkinje neurons at the inositol trisphosphate receptor. J. Neurosci.28, 133–142.10.1523/JNEUROSCI.1729-07.2008

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 131

  SarterM.HasselmoM. E.BrunoJ. P.GivensB. (2005). Unraveling the attentional functions of cortical cholinergic inputs: interactions between signal-driven and cognitive modulation of signal detection. Brain Res. Brain Res. Rev.48, 98–111.10.1016/j.brainresrev.2004.08.006

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 132

  SarterM.ParikhV.HoweW. M. (2009). Phasic acetylcholine release and the volume transmission hypothesis: time to move on. Nat. Rev. Neurosci.10, 383–390.10.1038/nrn2635

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 133

  SawinskiJ.WallaceD. J.GreenbergD. S.GrossmannS.DenkW.KerrJ. N. (2009). Visually evoked activity in cortical cells imaged in freely moving animals. Proc. Natl. Acad. Sci. U.S.A.106, 19557–19562.10.1073/pnas.0903680106

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 134

  SchultzW. (1998). Predictive reward signal of dopamine neurons. J. Neurophysiol.80, 1–27.

  • Pubmed Abstract
  • Google Scholar

• 135

  SchultzW. (2000). Multiple reward signals in the brain. Nat. Rev. Neurosci.1, 199–207.10.1038/35044563

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 136

  SchultzW. (2002). Getting formal with dopamine and reward. Neuron36, 241–263.10.1016/S0896-6273(02)00967-4

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 137

  SchultzW. (2006). Behavioral theories and the neurophysiology of reward. Annu. Rev. Psychol.57, 87–115.10.1146/annurev.psych.56.091103.070229

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 138

  SchultzW. (2007). Multiple dopamine functions at different time courses. Annu. Rev. Neurosci.30, 259–288.10.1146/annurev.neuro.28.061604.135722

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 139

  SchultzW.ApicellaP.LjungbergT. (1993). Responses of monkey dopamine neurons to reward and conditioned stimuli during successive steps of learning a delayed response task. J. Neurosci.13, 900–913.

  • Pubmed Abstract
  • Google Scholar

• 140

  SchwarzschildM. A.AgnatiL.FuxeK.ChenJ. F.MorelliM. (2006). Targeting adenosine A2A receptors in Parkinson's disease. Trends Neurosci.29, 647–654.10.1016/j.tins.2006.09.004

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 141

  SeguelaP.WatkinsK. C.DescarriesL. (1989). Ultrastructural relationships of serotonin axon terminals in the cerebral cortex of the adult rat. J. Comp. Neurol.289, 129–142.10.1002/cne.902890111

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 142

  SeguelaP.WatkinsK. C.GeffardM.DescarriesL. (1990). Noradrenaline axon terminals in adult rat neocortex: an immunocytochemical analysis in serial thin sections. Neuroscience35, 249–264.10.1016/0306-4522(90)90079-J

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 143

  SeolG. H.ZiburkusJ.HuangS.SongL.KimI. T.TakamiyaK.HuganirR. L.LeeH. K.KirkwoodA. (2007). Neuromodulators control the polarity of spike-timing-dependent synaptic plasticity. Neuron55, 919–929.10.1016/j.neuron.2007.08.013

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 144

  SesackS. R.AokiC.PickelV. M. (1994). Ultrastructural localization of D2 receptor-like immunoreactivity in midbrain dopamine neurons and their striatal targets. J. Neurosci.14, 88–106.

  • Pubmed Abstract
  • Google Scholar

• 145

  SeungH. S. (2003). Learning in spiking neural networks by reinforcement of stochastic synaptic transmission. Neuron40, 1063–1073.10.1016/S0896-6273(03)00761-X

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 146

  ShenW.FlajoletM.GreengardP.SurmeierD. J. (2008). Dichotomous dopaminergic control of striatal synaptic plasticity. Science321, 848–851.10.1126/science.1160575

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 147

  SivakumaranS.MohajeraniM. H.CherubiniE. (2009). At immature mossy-fiber-CA3 synapses, correlated presynaptic and postsynaptic activity persistently enhances GABA release and network excitability via BDNF and cAMP-dependent PKA. J. Neurosci.29, 2637–2647.10.1523/JNEUROSCI.5019-08.2009

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 148

  SjostromP. J.HausserM. (2006). A cooperative switch determines the sign of synaptic plasticity in distal dendrites of neocortical pyramidal neurons. Neuron51, 227–238.10.1016/j.neuron.2006.06.017

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 149

  SjostromP. J.TurrigianoG. G.NelsonS. B. (2003). Neocortical LTD via coincident activation of presynaptic NMDA and cannabinoid receptors. Neuron39, 641–654.10.1016/S0896-6273(03)00476-8

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 150

  SmithY.BennettB. D.BolamJ. P.ParentA.SadikotA. F. (1994). Synaptic relationships between dopaminergic afferents and cortical or thalamic input in the sensorimotor territory of the striatum in monkey. J. Comp. Neurol.344, 1–19.10.1002/cne.903440102

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 151

  SternE. A.KincaidA. E.WilsonC. J. (1997). Spontaneous subthreshold membrane potential fluctuations and action potential variability of rat corticostriatal and striatal neurons in vivo. J. Neurophysiol.77, 1697–1715.

  • Pubmed Abstract
  • Google Scholar

• 152

  SunX.ZhaoY.WolfM. E. (2005). Dopamine receptor stimulation modulates AMPA receptor synaptic insertion in prefrontal cortex neurons. J. Neurosci.25, 7342–7351.10.1523/JNEUROSCI.4603-04.2005

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 153

  SuttonR. S.BartoA. G. (1981). Toward a modern theory of adaptive networks: expectation and prediction. Psychol. Rev.88, 135–170.10.1037/0033-295X.88.2.135

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 154

  SuttonR. S.BartoA. G. (1998). Reinforcement Learning: An Introduction. Cambridge, MA: MIT Press.

  • Google Scholar

• 155

  ThomasM. J.MoodyT. D.MakhinsonM.O'DellT. J. (1996). Activity-dependent beta-adrenergic modulation of low frequency stimulation induced LTP in the hippocampal CA1 region. Neuron17, 475–482.10.1016/S0896-6273(00)80179-8

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 156

  TsubokawaH. (2000). Control of Na+ spike backpropagation by intracellular signaling in the pyramidal neuron dendrites. Mol. Neurobiol.22, 129–141.10.1385/MN:22:1-3:129

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 157

  TzounopoulosT.RubioM. E.KeenJ. E.TrussellL. O. (2007). Coactivation of pre- and postsynaptic signaling mechanisms determines cell-specific spike-timing-dependent plasticity. Neuron54, 291–301.10.1016/j.neuron.2007.03.026

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 158

  VasilakiE.FremauxN.UrbanczikR.SennW.GerstnerW. (2009). Spike-based reinforcement learning in continuous state and action space: when policy gradient methods fail. PLoS Comput. Biol.5, e1000586.10.1371/journal.pcbi.1000586

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 159

  WangH. X.GerkinR. C.NauenD. W.BiG. Q. (2005). Coactivation and timing-dependent integration of synaptic potentiation and depression. Nat. Neurosci.8, 187–193.10.1038/nn1387

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 160

  WangS. S.DenkW.HausserM. (2000). Coincidence detection in single dendritic spines mediated by calcium release. Nat. Neurosci.3, 1266–1273.10.1038/81792

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 161

  WangZ.KaiL.DayM.RonesiJ.YinH. H.DingJ.TkatchT.LovingerD. M.SurmeierD. J. (2006). Dopaminergic control of corticostriatal long-term synaptic depression in medium spiny neurons is mediated by cholinergic interneurons. Neuron50, 443–452.10.1016/j.neuron.2006.04.010

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 162

  WatersJ.HelmchenF. (2004). Boosting of action potential backpropagation by neocortical network activity in vivo. J. Neurosci.24, 11127–11136.10.1523/JNEUROSCI.2933-04.2004

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 163

  WatersJ.LarkumM.SakmannB.HelmchenF. (2003). Supralinear Ca²⁺ influx into dendritic tufts of layer 2/3 neocortical pyramidal neurons in vitro and in vivo. J. Neurosci.23, 8558–8567.

  • Pubmed Abstract
  • Google Scholar

• 164

  WatersJ.SchaeferA.SakmannB. (2005). Backpropagating action potentials in neurones: measurement, mechanisms and potential functions. Prog. Biophys. Mol. Biol.87, 145–170.10.1016/j.pbiomolbio.2004.06.009

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 165

  WickensJ. (1990). Striatal dopamine in motor activation and reward-mediated learning: steps towards a unifying model. J. Neural Transm. Gen. Sect.80, 9–31.10.1007/BF01245020

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 166

  WickensJ. R. (2009). Synaptic plasticity in the basal ganglia. Behav. Brain Res.199, 119–128.10.1016/j.bbr.2008.10.030

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 167

  WilsonC. J.ChangH. T.KitaiS. T. (1990). Firing patterns and synaptic potentials of identified giant aspiny interneurons in the rat neostriatum. J. Neurosci.10, 508–519.

  • Pubmed Abstract
  • Google Scholar

• 168

  WilsonC. J.GrovesP. M. (1981). Spontaneous firing patterns of identified spiny neurons in the rat neostriatum. Brain Res.220, 67–80.10.1016/0006-8993(81)90211-0

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 169

  WilsonC. J.KawaguchiY. (1996). The origins of two-state spontaneous membrane potential fluctuations of neostriatal spiny neurons. J. Neurosci.16, 2397–2410.

  • Pubmed Abstract
  • Google Scholar

• 170

  WilsonF. A.RollsE. T. (1990). Neuronal responses related to the novelty and familarity of visual stimuli in the substantia innominata, diagonal band of Broca and periventricular region of the primate basal forebrain. Exp. Brain Res.80, 104–120.10.1007/BF00228852

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 171

  XieX.SeungH. S. (2004). Learning in neural networks by reinforcement of irregular spiking. Phys. Rev. E. Stat. Nonlin. Soft Matter Phys.69, 041909.10.1103/PhysRevE.69.041909

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 172

  ZhangJ. C.LauP. M.BiG. Q. (2009). Gain in sensitivity and loss in temporal contrast of STDP by dopaminergic modulation at hippocampal synapses. Proc. Natl. Acad. Sci. U.S.A.106, 13028–13033.10.1073/pnas.0900546106

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 173

  ZhouF. M.LiangY.DaniJ. A. (2001). Endogenous nicotinic cholinergic activity regulates dopamine release in the striatum. Nat. Neurosci.4, 1224–1229.10.1038/nn769

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 174

  ZhouF. M.WilsonC. J.DaniJ. A. (2002). Cholinergic interneuron characteristics and nicotinic properties in the striatum. J. Neurobiol.53, 590–605.10.1002/neu.10150

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 175

  ZhouY. D.AckerC. D.NetoffT. I.SenK.WhiteJ. A. (2005). Increasing Ca2+ transients by broadening postsynaptic action potentials enhances timing-dependent synaptic depression. Proc. Natl. Acad. Sci. U.S.A. 102, 19121–19125.10.1073/pnas.0509856103

  • CrossRef
  • Google Scholar