A model-based prediction of the calcium responses in the striatal synaptic spines depending on the timing of cortical and dopaminergic inputs and post-synaptic spikes

The dopamine-dependent plasticity of the cortico-striatal synapses is considered as the cellular mechanism crucial for reinforcement learning. The dopaminergic inputs and the calcium responses affect the synaptic plasticity by way of the signaling cascades within the synaptic spines. The calcium concentration within synaptic spines, however, is dependent on multiple factors including the calcium influx through ionotropic glutamate receptors, the intracellular calcium release by activation of metabotropic glutamate receptors, and the opening of calcium channels by EPSPs and back-propagating action potentials. Furthermore, dopamine is known to modulate the efficacies of NMDA receptors, some of the calcium channels, and sodium and potassium channels that affect the back propagation of action potentials. Here we construct an electric compartment model of the striatal medium spiny neuron with a realistic morphology and predict the calcium responses in the synaptic spines with variable timings of the glutamatergic and dopaminergic inputs and the postsynaptic action potentials. The model was validated by reproducing the responses to current inputs and could predict the electric and calcium responses to glutamatergic inputs and back-propagating action potential in the proximal and distal synaptic spines during up- and down-states. We investigated the calcium responses by systematically varying the timings of the glutamatergic and dopaminergic inputs relative to the action potential and found that the calcium response and the subsequent synaptic potentiation is maximal when the dopamine input precedes glutamate input and action potential. The prediction is not consistent with the hypothesis that the dopamine input provides the reward prediction error for reinforcement learning. The finding suggests that there is an unknown learning mechanisms at the network level or an unknown cellular mechanism for calcium dynamics and signaling cascades.


INTRODUCTION
The striatum is the input region of the basal ganglia and receives glutamate input from the cortex and dopaminergic input from the midbrain (Smith and Bolam, 1990). The plasticity of the the cortico-striatal synapses is affected by the timing between the presynaptic dopamine inputs, presynaptic (cortical) glutamatergic inputs and the postsynaptic (striatal) action potentials, which is known as the spike-timing-dependent plasticity (STDP) (Fino et al., 2005;Pawlak and Kerr, 2008;Shen et al., 2008). Among these, the dopamine input is regarded as the reward prediction error and plays a critical role in the reinforcement learning theory in the basal ganglia (Reynolds et al., 2001;Doya, 2002). While the learning theory predicts that the dopamine input following the spike output induces plasticity, the effect of timing of dopamine input on STDP is still unknown. The dopaminedependent plasticity can be mediated directly by activation of the intracellular signaling cascades within the synaptic spines and indirectly by the modulation of the synaptic receptors and the membrane currents that affect the calcium influx to synaptic spines. We previously proposed a computational model of the intracellular signaling cascades and showed that the direction and the strength of the plasticity are determined by the amplitude and relative timing between the dopamine input and calcium concentration within the synaptic spines (Nakano et al., 2010).
The calcium concentration has intrinsically complex dynamics and is dependent on multiple factors including the calcium influx through ionotropic glutamate receptors, the intracellular calcium release by activation of metabotropic glutamate receptors, and the opening of calcium channels by EPSPs and back propagating action potentials. Furthermore, dopamine is known to modulate the efficacies of NMDA receptors, some types of the calcium channels, and sodium and potassium channels that affect the back propagation of action potentials. Accordingly, the striatal synaptic plasticity is known to be modulated by the spontaneous oscillations of the postsynaptic membrane potential between the up-state and down-state (Charpier and Deniau, 1997) and by the location of the synapse on the dendrite (Kampa et al., 2007). Here we have constructed a morphologically realistic electric compartment model of the striatal medium spiny neuron and we predict the calcium responses in the synaptic spines with variable timings of the glutamatergic and dopaminergic inputs and the postsynaptic action potentials. The model is based upon our own imaging and 3D reconstruction of medium spiny neurons and upon previous models of ionic and synaptic currents, calcium dynamics, and their dopaminergic modulation. We investigated the calcium responses in the proximal and distal synaptic spines during upand down-states under various timing parameters for glutamate, dopamine, and postsynaptic action potentials using numerical simulations.
This paper is organized as follows: In Section 2, we present the specification of the model constructed for this study. In Section 3, we demonstrate that the model well reproduces electrophysiological properties reported in the literature, to support the validity of the model. Subsequently, we predict the intracellular calcium responses to various timing of the triplet inputs: presynaptic glutamate and dopamine inputs, and a postsynaptic spike. Herein, we show the distribution of the calcium sources to clarify which factors predominantly contribute to calcium responses. Finally, we summarize the main findings of this study and discuss their relationship with the corticostriatal synaptic plasticity in Section 4.

MORPHOLOGICAL MODELING
An electric compartment model was constructed using realistic morphology based on measurements from actual medium spiny neurons. This allowed us to precisely evaluate the effects of back-propagating action potentials. To obtain morphological images, acute corticostriatal slices (300 μm thickness) were prepared from p21-25 Drd1a eGFP Swiss Webster mice (Gong et al., 2003). A neuron was filled with biocytin through a patch pipette and tagged with Alexa 488. A 3D morphological image (Figure 1) was obtained using the Neurolucida neuronal tracing system with a DSU confocal microscope. Cell morphology was manually traced using Neurolucida. The traced data included information regarding the lengths and diameters of the dendrites (Hines and Carnevale, 2001), and were converted to NEURON hoc files using NLMorphologyViewer and NLMorphologyConverter software 1 . In this process, all spines were ignored.
To measure calcium transients in different spines, two spines of diameter 1 μm and length 1.273 μm (i.e., volume of 1 μm 3 ) were attached to the proximal and distal dendrites (see arrows in Figure 1), which were located on 25 μm and 100 μm away from the soma, respectively. FIGURE 1 | Morphology of a medium spiny neuron expressing D1-type dopamine receptors. Top: A medium spiny neuron filled with the fluorescent dye Alexa 488 and observed using a DSU confocal microscope. Bottom: Morphological data imported into the NEURON simulator. The arrow heads indicate the proximal and distal dendrites described in Section 2.1.

MULTI-COMPARTMENT MODEL
The electric properties of the neuron was approximated by splitting the dendrites into small compartments. The membrane potential of the jth compartment, denoted by V j , was represented as: is a leak current through passive channels of the jth compartment, g leak, j is passive transmembrane conductance, and E leak is the passive reversal potential. I chan, j and I syn, j are ionic currents through active channels and synaptic channels in the jth compartment, respectively (See the details in Section 2.3). I stim, j is a current induced by external stimuli (e.g., current injection in whole-cell patch clamp experiments in Section 3.1). The last two terms in Equation (1) are membrane currents from neighboring compartments connected by coupling conductance g j − 1, j and g j, j + 1 , respectively. The coupling conductance g j, j + 1 is calculated as 1/g j, j + 1 = 2l j /(g a πd 2 j ) + 2l j + 1 /(g a πd 2 j + 1 ), where d j and l j are the diameter and length of j th compartment. Spatial discretization of the model neurons was completed automatically according to the d_lambda rule implemented in the NEURON simulator (Hines and Carnevale, 2001). The neuron is decided to 156 compartments.
The effects of the spines ignored in the morphological modeling were mimicked by locally adjusting the conductance per area g leak, j and the capacitance per area C j based on the area ratio of the membrane surfaces between the dendrites and the spines (Holmes, 1989;Koch, 1998). More specifically, we let F j be defined as F j = 1 + A spine, j /A dend, j , where A dend, j and A spine, j are the membrane surface area of the dendrites and the spines in the jth compartment, respectively. We set g leak, j = g leak F j and C j = CF j , where g leak and C is the passive transmembrane conductance and the membrance capacitance averaged over the whole neuron, respectively 2 . The parameters used for the multi-compartment model are shown in Table 1.

IONIC AND SYNAPTIC CURRENTS
The ionic and synaptic currents were modeled based on (Wolf et al., 2005;Moyer et al., 2007). In this section, we briefly review the original model and clarify our modifications. The model included the following ionic channels: fast (NaF) and persistent (NaP) sodium channels; inwardly rectifying (KIR), slow A-type (KAs), fast A-type (KAf), 4-AP resistant persistent (KRP), small conductance calcium-dependent (SK) and largeconductance calcium-dependent (BK) potassium channels; N-(CaN), Q-(CaQ), R-(CaR), L-type Cav 1.2 (Cav1.2), L-type Cav 1.3 (Cav1.3), and T-(CaT) calcium channels (Catterall, 2000). For each compartment, the total current through the ionic channels was given by I chan = I NaChan + I KChan + I CaChan (2a) I NaChan = μ NaF I NaF + μ NaP I NaP (2b) where the index of the compartment, j, is omitted for notational simplicity 3 . I NaChan , I KChan , and I CaChan are the currents summed over sodium, potassium, and calcium channels, respectively. The coefficient μ z for each channel-type z is the dopamine modulation factor, and the details are described in Section 2.5. For the moment, we assume that μ z = 1 is fixed for each channel-type z so that the dopamine modulation can be ignored for simplicity. The current through each type z of sodium and potassium channel was given by whereḡ z is the maximum conductance of the channel, and E rev is the reversal potential that was set to E rev = 50 mV for every sodium channel and E rev = −90 mV for every potassium channel. x z (t, V) is the time-and voltage-dependent variable that summarizes the activation and inactivation states of the channel z given by the Hodgkin-Huxley formulation [See the detail in Supplementary Material of (Moyer et al., 2007)]. The current through each type z of calcium channel, was given by the Goldman-Hodgkin-Katz (GHK) current formulation: where the gas constant, Faraday constant, and the temperature were set to R = 8.31 J/mol/K, F = 96489 C/mol, and wherep z is the maximum permeability and x z (t, V) has the same meaning as Equation (3). The model also included AMPA-type glutamate receptors (AMPARs) and NMDA-type glutamate receptors (NMDARs). For each compartment, the total postsynaptic current through the receptors, denoted by I syn , was given by where the current though each receptor z, I z , was given by the same form as Equation (3), and the reversal potential was set to E rev = 0 mV for both receptors. In our study, the variable x z (t, V) for every z had the same dynamics as (Moyer et al., 2007). The only difference was the setting of the maximum conductanceḡ z and the maximum permeabilityp z , and we adjusted these parameters to fit our experimental data using Neurofitter (Geit et al., 2007). The resulting parameters are listed in Table 2.

CALCIUM DYNAMICS
The model proposed in Wolf et al. (2005); Moyer et al. (2007) does not consider calcium release from intracellular calcium stores [i.e., endoplasmic reticulum (ER)] through ryanodine and inositol-1,4,5-triphosphate (IP 3 ) channels, which is suggested to significantly affect the activity of various neurons (Falcke et al., 2000;Varona et al., 2001b,a). To precisely evaluate the calcium responses in two spines attached to the proximal and distal dendrites, we refined the calcium dynamics model based on De Schutter and Smolen (1998). Figure 2 shows all the processes contributing to calcium dynamics in our model.
More specifically, the transient change in intracellular calcium [Ca 2+ ] i were given by Frontiers in Computational Neuroscience www.frontiersin.org September 2013 | Volume 7 | Article 119 | 3 FIGURE 2 | Calcium dynamics. Calcium sources are divided into two groups. One group consists of extracellular sources, and includes calcium influx from voltage-dependent calcium channels (VDCCs), and from calcium permeable ionotropic glutamate receptors (iGluRs). The iGluRs in our model consists of AMPARs and NMDARs. Calcium is also pumped out of cells into the extracellular matrix by calcium pumps. The other group consists of intracellular calcium stores in the endoplasmic reticulum (ER), which are accessed by IP 3 receptors (IP 3 Rs) and ryanodine receptors (RyRs). IP 3 is produced by metabotropic glutamate receptors (mGluRs). Calcium uptake is mediated by the Sarco/ER Ca 2+ -ATPase (SERCA), and leaks occur through the ER.
Here, J CICR is the flux caused by calcium-induced calcium release (CICR) from intracellular stores. This process is mediated by ryanodine receptors, and was modeled as where V CICR = 3 × 10 −12 s −1 is the maximum rate of CICR and K CICR = 0.2 μM. [Ca 2+ ] ER = 0.20 mM is the calcium concentration in the ER. J IP 3 is the flux caused by IP 3 -induced calcium release from intracellular stores. It is known that the process has a bell-shaped steady state curve that depends on [Ca 2+ ] i with a sharp peak around 0.2 μM; thus, it was modeled as where V IP 3 = 1 × 10 −9 s −1 is the maximum rate of the IP 3induced release. m and h are an activation gate and an inactivation gate, respectively. They were given by wheret Glu is the time elapsed since the last glutamate release from the presynaptic neuron, and τ IP 3 = 220 ms is the time to peak. γ IP 3 = 5 × 10 −6 mM/ms 2 determines the maximum rate of IP 3 production, β IP 3 = 0.2 ms −1 is the removal rate, and [IP 3 ] min = 0.24 μM is the minimum level of [IP 3 ] i . J uptake is the calcium uptake to the ER, which was modeled as where V uptake = 0.75 × 10 −9 μM/s is the maximum rate of uptake and K uptake = 0.2 μM is the dissociation constant. J leak is the calcium leak from the ER, which was modeled as where V leak = 6.15 × 10 −14 s −1 is the maximum rate of leak. J cur is the calcium current through calcium channels and calcium permeable glutamate receptors, and was given by where F = 96489 C/mol is Faraday's constant, v = 1 μm 3 is the volume of the compartment. c AMPA and c NMDA are the effectiveness of calcium ions in the synaptic currents through AMPARs and NMDAs, respectively. They were set to c AMPA = 0.0005 μ AMPA and c AMPA = 0.01 μ AMPA . J pump is pumping activity to the outside of the cell, where V pump = 0.1 μM/ms is the time constant of the pump and K pump = 0.1 μM is the equilibrium calcium value. The last term in Equation (7) is a simple diffusion or buffering process, in which the parameters were set to the same as (Wolf et al., 2005), that is, τ = 43 ms and [Ca 2+ ] ∞ = 0.01 μM.

DOPAMINE MODULATION
While the effects of dopamine on channel conductance vary in different cell types and brain regions (Surmeier et al., 1995;Yang and Seamans, 1996;Hernández-López et al., 1997;Cepeda et al., 1998;Nicola et al., 2000;Johnson et al., 2003;Surmeier et al., 2007;Steephen, 2011;Zhou and Antic, 2012), the dopamine enhances KIR, Cav1.2 and NMDAR conductances and reduces NaF, CaN and CaQ conductances in the striatum, as summarized in Moyer et al. (2007). To reflect the findings, the dopamine modulation factor μ z for each channel/receptor type z was variable in our model. Based on Gruber et al. (2003), the transient was modeled as wheret DA is the time elapsed since the last dopamine input arrived. μ peak, z is the peak level of μ z and t peak is the time to reach the peak. τ inc and τ dec are the time constants in the increasing and decreasing phases of μ z , respectively. The peak level μ peak, z for each channel/receptor type is listed in Table 3. The other parameters were set to t peak = 60 ms, τ inc = 30 ms, and τ dec = 100 ms. Figure 3 shows the typical behaviors of μ z for some types of channels or receptors. These values were based on Moyer et al. (2007).
In addition to channel conductance, dopamine affects the voltage dependence of the activation gating of Cav1.3 ionic channels (Moyer et al., 2007). We modeled it in the same manner as the conductance modulation 4 .

VOLTAGE AND CALCIUM RESPONSES: MODEL VALIDATION
The model parameters were calibrated to fit the electrophysiological properties of medium spiny neurons expressing D1-type dopamine receptors, which were examined by wholecell patch clamp experiments in vitro. Figure 4 compares the membrane potential responses of the model to those of a real neuron. The model replicated the characteristic properties of the medium spiny neurons, with a resting membrane potential around −85 mV, small voltage responses to hyperpolarizing currents, and shallow after-hyperpolarization 4 Specifically, the parameter V half for the activation gate variable m of Cav1.3 channels in Moyer et al. (2007) was varied so that V half = −33 mV in the resting state and V half = −43 mV at the peak. The time course followed the same curve as Figure 3.  (AHP) following spike firing. In addition, the model reproduced calcium spikes observed in the experimental condition during a step current input with the application of 4-AP (potassium channel blocker) and TTX (O'Donnell and Grace, 1993) ( Figure 5). The voltage and calcium responses to a back-propagating postsynaptic action potential (bAP) or a single glutamate input were investigated in the spines on proximal and distal dendrites of the model neuron. The membrane potential of medium spiny neurons with intact cortical input fluctuates between the "down-state" of approximately −85 mV and the "up-state" of approximately −65 mV (Wilson and Kawaguchi, 1996). The upstate was simulated by a steady current input of 0.2 nA to the soma and the down-state was simulated by no current input. Figure 6 shows the voltage and calcium responses in the dendritic spines to bAPs, where each postsynaptic action potential was evoked by a 2 ms step current application to the soma. The bAPs arrived in the spine without delay, but their amplitudes were attenuated with distance from the soma ( Figure 6A). Corresponding to the attenuation, the amplitudes of the calcium responses were also attenuated with distance from the soma ( Figure 6B). Calcium responses in the up-state were bigger than in the down-state because of greater calcium influx through L-type calcium (Cav1.2 and Cav1.3) channels in the upstate (Figures 6C,D). However, interestingly, when a wide step current (30 ms duration) was applied to evoke an action potential, the calcium response was smaller in the up-state than in the down-state (Figure 6E), which is consistent with the experimental result (Carter and Sabatini, 2004). In contrast, when T-type calcium channels were blocked, the calcium transients evoked by the wide current pulse were larger in the up-state than in the down-state. Although calcium currents predominantly moved through T-type calcium channels at the moment of the current input, T-type calcium channels in the up-state were inactivated so rapidly that this inversion phenomenon was engendered.  shows the voltage responses (i.e., excitatory postsynaptic potentials; EPSPs) and calcium responses in the dendritic spines to a single glutamate input. Here, the inputs were applied to the same spines in which the responses were measured. There were no significant difference in EPSPs or calcium responses between the proximal and distal spines, and the EPSPs and calcium responses were greater in the up-state (Figures 7A,B). The increased calcium influx in the up-state was mediated by voltage-gated calcium channels (CaT and Cav1.3 channels), NMDARs, and calcium-dependent calcium release from the ER (Figures 7C,D).
In (Carter and Sabatini, 2004), the following experimental findings were reported: (1) The AMPAR-mediated calcium response was larger in the down-state than in the up-state during the application of CPP (NMDAR antagonist), TTX (sodium channel blocker), and VDCC blockers; and (2) the NMDARmediated calcium response was larger in the up-state than in the down-state, and it was smaller in the up-state than in the downstate in magnesium-free conditions in the presence of NBQX (AMPAR antagonist), TTX, and VDCC blockers. We simulated the experiments in the following manners: For the Experiment 1), we fixed the sodium, calcium, and AMPAR currents in our model at zero and measured the calcium responses to glutamate input. Similarly, for Experiment 2), we fixed the sodium, calcium, and NMDAR currents at zero. Figures 7E,F show the simulation results, which accurately reproduced the properties reported in (Carter and Sabatini, 2004). When calcium responses to glutamate input and to bAP are compared, glutamate input made a larger response than bAP in the proximal spine and smaller responses in the distal spine. This amplitude relationship was also observed in the  In (A,B,E,F), the blue lines indicate the down-state and the red lines indicate the up-state simulated by a steady current input. experimental study Carter and Sabatini (2004), even though a conflicting finding, which glutamate input made a smaller response than bAP, has also been reported recently (Shindou et al., 2011).

TIMING-DEPENDENT CALCIUM RESPONSES TO PAIRED INPUTS
Using the model, we predicted how the calcium responses depend on the relative timing of paired inputs: a presynaptic input (either glutamate (Glu) or dopamine (DA)) and a postsynaptic spike (Post). Hereafter, the time difference between Glu and Post is denoted by t Glu where t Glu > 0 if Glu precedes Post. For more intuitive notations, we also use "Glu-Post" and "Post-Glu" for t Glu > 0 and t Glu < 0, respectively. The notational rules are applied to the time difference between DA and Post, where the above-mentioned "Glu" are all replaced by "DA." Figure 8A shows the calcium responses to Glu preceding or following Post by 20 ms in the down-state. Figures 8B,C shows timing-dependent peak calcium responses as a function of t Glu in the down-and up-states, respectively. In both the up-and down-states, Glu-Post engendered higher calcium responses than Post-Glu. As for the depencence on the location of spines, the amplitude of calcium responses was larger in the proximal dendrite than in the distal dendrite, but a tendency toward timing dependence was preserved in both spines. The effects of DA timing relative to Post on calcium responses were similar to those of Glu timing: Namely, DA-Post engendered a larger calcium response than Post-DA (Figures 8D-F).
The simulation shows sources of calcium influx (Figure 9). The increased calcium response to a Glu-Post condition was caused by enhanced calcium influx through NMDARs (Figure 9 upper panels). This is consistent with the calcium imaging experiment investigating the calcium response to the time difference between the glutamate input and bAP (Shindou et al., 2011). The simulation results showed that the calcium release from ER helps to sustain the long-lasting calcium increase by glutamate, and emphasizes the difference in the calcium responses between Glu-Post and Post-Glu conditions. The increased calcium response to DA-Post was caused by enhanced calcium influx through L-type calcium (Cav1.2 and Cav1.3) channels (Figure 9 lower panels).

TRIPLET INTERACTION
We then investigated the dependence of calcium responses on the timing of triplet inputs: Glu, DA and Post. Figure 10A shows the mapping of the temporal order of the triplet inputs onto the ( t DA , t Glu )-space. Figures 10B-E shows the peak calcium concentration in the proximal and distal spines as a function of ( t DA , t Glu ). In the both down-and upstates, the effects of dopamine inputs were the most prominent when DA preceded Post by approximately 60 ms, and amplified more when Glu preceded Post. While this tendency was also observed in the both proximal and distal spines, the peak amplitude was lower in the distal spines than in the proximal spines. We then predicted the calcium responses in the proximal spines when different synaptic channels were blocked. When Ltype calcium (Cav1.2 and Cav1.3) channels were blocked, the peak amplitude approached that in the distal spines ( Figure 11A).
This suggests that the weaker amplitudes in the distal spines were caused by insufficiently large bAPs to activate the L-type calcium channels (Figure 6). The timing dependence of Glu and Post was weakened by blocking NMDARs (Figure 11B). The timing For all panels, blue and black lines indicate the measurements at the proximal and distal dendritic spines, respectively. Post was evoked by a 2 ms postsynaptic step current. dependence of DA and Post was eliminated when both NMDARs and L-type calcium channels were blocked ( Figure 11C). It is noteworthy that the results showed that calcium responses to the triplet inputs cannot be explained by just a linear sum of DA-and Glu-derived effects. Rather, calcium levels were drastically elevated only when both Glu and DA adequately preceded Post. The optimal timing to induce the largest calcium response was ( t Glu , t DA ) = (+50 ms, +60 ms) in the up-state, but it shifted to ( t Glu , t DA ) = (+20 ms, +60 ms) in the down-state because of the non-linear interaction between DA-and Gluderived effects.

DOPAMINE-MEDIATED MODULATION OF NMDARs
An uncaging calcium study in the cerebellum reported that the amplitude of long-term depression was well predicted from a leaky integration of the intracellular calcium concentration (Tanaka et al., 2007). The leaky integration x of calcium concentration is defined as where τ is the time constant of integration, and a is a scaling factor that converts the calcium concentration into the transduction signal regulating the synaptic efficacy. We set to τ = 600 ms and a = 20 and evaluated the peak amplitude of x as a function of ( t Glu , t DA ). Interestingly, the dependence of the leaky integration x on the glutamate and dopamine input timings was different from that of the peak calcium concentration (Figure 12A). The timing between dopamine and glutamate is more important than the timing between glutamate and bAP. The DA timing modulated the leaky integrator response to Glu-Post bidirectionally: Glu-Post enhanced x in the DA-Post condition but Post-Glu enhanced x in the Post-DA condition.
To investigate the mechanisms of the DA-timing modulation, we performed additional simulations where all DA effects on NMDAR were blocked ( Figure 12B). In this blockade condition, Glu-Post did not enhance x even in the DA-Post timing at t DA = +100 ms. Also, the Glu-timing dependence on the leaky integrator almost vanished regardless of the DA-timing.

CONCLUSION AND DISCUSSION
We constructed a multi-compartment model of a medium spiny neuron of the striatum based on real morphological data. The model could reproduce the major electrophysiological properties the neuron and allowed us to predict the calcium responses to timed presynaptic inputs (glutamate and dopamine) and a postsynaptic spike under various conditions, including the up-and down-states. We measured calcium dynamics in both proximal and distal spines and evaluated the peak and leaky integration of calcium.
Our major findings are summarized as follows: 1. Glutamate input preceding a postsynaptic spike induced higher calcium responses than glutamate input following a postsynaptic spike, which was mediated by NMDARs, L-type calcium channels (i.e., Cav1.2 and Cav1.3) and the intracellular calcium store (Figures 8, 9). 2. Dopamine input preceding a postsynaptic spike also induced higher calcium responses than dopamine input following a postsynaptic spike, which was mediated by L-type calcium channels (Figures 8, 9). 3. Although their amplitudes differed, there was no difference in the timing dependence on calcium responses between the up-and down-states. At distal spines, the back-propagating action potential was attenuated, so that the dopamine timing effect was small. Nevertheless, the timing dependence on calcium responses between distal and proximal spines is preserved. 4. The timing dependence of the leaky integration of the calcium was different from that of the peak calcium concentration. The timing of dopamine input could modulate the timing dependency of the glutamate input and postsynaptic spike, which was mediated by dopaminergic modulation of NMDARs (Figure 12).
Calcium regulation in synaptic spines plays a key role in the synaptic plasticity (Artola and Singer, 1993). In the corticostriatal synapses, dopamine also plays a critical role in inducing long-term potentiation by increasing cAMP and activating PKA in the cells expressing D1 receptors (Nakano et al., 2010). Our model shows that dopamine regulates intracellular calcium changes, and suggests the possibility that dopamine regulates synaptic plasticity through calcium in parallel with the PKArelated cascade. In particular, dopamine preceding a postsynaptic spike could increase the leaky integration of calcium induced by glutamate and a postsynaptic spike, and make it easier to potentiate the synaptic efficacy. In addition, our results show that application of dopamine before glutamate induces the largest calcium responses. From the perspective of reinforcement learning theory, dopamine is hypothesized to be the reward prediction error signal that reinforces the association between the sensory information encoded in the glutamatergic input from the cortex and the action realized by the striatal spike output. Based on this hypothesis, the reinforcement signal should follow the sensory input and action output, but our results are inconsistent with this prediction. An alternative hypothesis suggests that dopamine codes salience rather than reward prediction error (Redgrave and Gurney, 2006;Berridge, 2007): The dopamine is released even by unexpected sensory events that have no obvious appetitive reinforcement consequence (Redgrave and Gurney, 2006), or by novel stimulations that trigger a motivational state, i.e., "wanting" for the reward (Berridge, 2007). According to this scenario, our simulation results might suggest that the dopamine signal amplifies the striatal response (i.e., attention) to the coincident events (including action selection) coded by presynaptic glutamate input, and marks the input as biologically significant events. However, the effective temporal order of the dopamine-glutamate input in our simulation is still inconsistent with this alternative theory, as well as with the reinforcement learning theory. Irrespective of which hypothesis is correct , this inconsistency suggests that there is an unknown network mechanism to overcome these timing constraints, or else there is a missing link in the dopaminergic signaling cascades. A multilevel study integrating calcium dynamics and intracellular signal transduction may be critical to elucidating these mechanisms.
Our contribution to the model is the refinement of the previous studies (Wolf et al., 2005;Moyer et al., 2007) by incorporating (1) real morphological data; (2) a kinetic model of calcium release from intracellular calcium stores; and (3) the time variation of the dopamine modulation. It turned out that the difference between the presence and the absence of a kinetic model of calcium release was significant in the timing effect of glutamate and bAP (Figure 9). On the other hand, in this study, real morphological data do not makes a significant difference in calcium dynamics between these two models with the sophisticated and simplified morphology, while morphological variability leads to variability of neural activity in other neurons (Mainen and Sejnowski, 1996). The reason would be involved in our assumption that the membrane characteristic of compartments is homogeneous within each of three dendritic regions (proximal, middle and distal). Such validation is out of scope in this study but will be an important issue in the future.
Another important issue to discuss is that there is the other type of the medium spiny neurons in the striatum expressing D2-type dopamine receptors, and showing different projection and morphology (Gertler et al., 2008). Since it is unclear whether the morphological difference is involved in the difference of neural activity, our model will serve as a basic model for D1-type medium spiny neurons to solve this question. Also, our model excludes a direct mechanism of calcium buffers, which is known to affect the electrical properties Frontiers in Computational Neuroscience www.frontiersin.org September 2013 | Volume 7 | Article 119 | 10 of neurons and other excitable cells as has been reported previously Torres et al. (2004); Harks et al. (2003). Another important question is whether such a mechanism affects the neural activity in the striatum, though none of the electrophysiological experiments compared with our simulation results employed the calcium buffer.