Homeostatic Plasticity Achieved by Incorporation of Random Fluctuations and Soft-Bounded Hebbian Plasticity in Excitatory Synapses

1. Introduction

Synaptic modifications are the fundamental feature for brain development and cognitive processes such as attention, perception, and memory (Bliss and Lømo, 1973; Martin et al., 2000; Whitlock et al., 2006; Nabavi et al., 2014). Synaptic efficacy is adjusted mainly by the process termed as Hebbian plasticity, which depends on the correlation between the activities of the pre- and post-synaptic neurons (Löwel and Singer, 1992; Katz and Shatz, 1996). Traditional Hebbian plasticity and its temporally asymmetric form, spike-timing dependent plasticity (STDP) (Markram et al., 1997; Bi and Poo, 1998; Sjöström et al., 2001), are positive feedback processes that are inherently prone to destabilize the neuronal activity (Rochester et al., 1956; von der Malsburg, 1973; Oja, 1982; Miller and MacKay, 1994; Miller, 1996). Synaptic potentiation due to a correlation between activities of pre- and post-synaptic neurons further increases the correlation, leading to further potentiation. Once depressed synapse no longer leads to the correlated activities of neurons, leading to further depression and becoming silent. To prevent these instability issues, various constraints on synaptic efficacy have been introduced; limit in the total sum of synaptic efficacy projecting to one neuron (von der Malsburg, 1973; Effenberger et al., 2015), decay in synaptic efficacy (Oja, 1982; Miller and MacKay, 1994), and negative feedback process (Bienenstock et al., 1982; Bear et al., 1987; Miller, 1996). Moreover, so-called “soft-bounded” STDP models have been proposed (van Rossum et al., 2000; Gütig et al., 2003; Morrison et al., 2007; Gilson and Fukai, 2011), in which depression at a strong synapse is much larger than that at a weak synapse. Soft-bounded STDP models provide an equilibrium at which potentiation and depression are balanced and prevent runaway potentiation or depression. Such dependence on efficacy is supported by evidences from biological studies (Bi and Poo, 1998). However, stability of synaptic efficacy does not always indicate stability of neuronal activity. The ranges of parameter values with which soft-bounded STDP models provide stability of neuronal activity are limited (Morrison et al., 2007; Gilson and Fukai, 2011; Yger and Gilson, 2015). Another negative feedback process, homeostatic plasticity, has also been proposed to maintain neuronal activity within a functional range (van Rossum et al., 2000; Turrigiano, 2008; Watt and Desai, 2010; Zenke et al., 2013; Toyoizumi et al., 2014). It is typically modeled as a proportional or proportional-integral controller that monitors post-synaptic neuronal activity on a time scale of hours, scales synaptic efficacy up or down, and stabilizes post-synaptic neuronal activity. Several neurobiological studies have confirmed the existence of homeostatic plasticity in mammalian cortex and hippocampus (Rutherford et al., 1998; Turrigiano et al., 1998; Turrigiano, 1999, 2008; Burrone et al., 2002; Ibata et al., 2008; Keck et al., 2013; Vitureira and Goda, 2013; Félix-Oliveira et al., 2014). After neuronal activity is decreased or silenced by drugs or lesions in sensory organs, synapses are potentiated on average, despite a decrease in correlated activity. Conversely, after neuronal activity is increased by drugs, synapses are depressed on average. Moreover, the normalized distribution of excitatory synaptic efficacy is almost similar before and after the change in neuronal activity. Therefore, this type of homeostatic plasticity is called synaptic scaling or activity-dependent scaling¹. Synaptic scaling is considered to consist of two distinct underlying mechanisms; scaling up (potentiation against low neuronal activity.) and scaling down (depression against high neuronal activity). Neuronal activity converges to the rate at which scaling up and scaling down are balanced. However, the mechanism underlying synaptic scaling still remains unclear.

Other studies have identified activity-independent (non-Hebbian) excitatory synaptic modifications called intrinsic fluctuations (Yasumatsu et al., 2008; Kasai et al., 2010; Loewenstein et al., 2011; Statman et al., 2014); the size of a post-synaptic element, dendritic spine, fluctuates even after neuronal activity is silenced. Because the efficacy of an excitatory synapse in the mammalian central nervous system is dependent on the corresponding dendritic spine size (Matsuzaki et al., 2001, 2004; Kasai et al., 2003; Kopec et al., 2006; Wang et al., 2007; Yuste, 2010; Keck et al., 2013), it is possible that these intrinsic fluctuations serve homeostatic plasticity mechanism. However, the intrinsic fluctuations are thought to induce change with zero expected value as opposed to synaptic scaling.

This paper proposes an excitatory synaptic plasticity model incorporating intrinsic fluctuations (Yasumatsu et al., 2008) and soft-bounded STDP model (van Rossum et al., 2000), and demonstrates a possible role for these intrinsic fluctuations in scaling of synaptic efficacy to prevent destabilization of neuronal activity. The proposed model demonstrated the robust stability of post-synaptic firing rate in response to change in pre-synaptic firing rate. The distribution of synaptic efficacy was scaled multiplicatively with varying pre-synaptic firing rate. After neuronal activity was silenced, synapses were scaled up. Intrinsic fluctuations also contributed to synaptic competition (Miller, 1996; Song et al., 2000; van Rossum et al., 2000). These numerical experiments and corresponding theoretical analyses indicate that intrinsic fluctuations achieve scaling up of synapses and soft-bounded STDP model achieves scaling down of synapses. As a result, synaptic plasticity model incorporating intrinsic fluctuations and soft-bounded STDP model acts as a form of homeostatic plasticity without monitoring neuronal activity.

2. Models and Methods

2.1. Intrinsic Fluctuations

Recent studies have reported non-Hebbian excitatory synaptic modifications called intrinsic fluctuations (Yasumatsu et al., 2008; Kasai et al., 2010; Loewenstein et al., 2011; Statman et al., 2014). Dendritic spines change size even after neuronal activity is silenced by drugs. Because spine size is strongly related to synaptic efficacy (Matsuzaki et al., 2001, 2004; Kasai et al., 2003; Kopec et al., 2006; Wang et al., 2007; Yuste, 2010; Keck et al., 2013), these intrinsic fluctuations are thought to influence synaptic efficacy independently from pre-synaptic firing and post-synaptic firing. According to the original studies, intrinsic fluctuations follow the stochastic differential equation as follows:

$\begin{array}{cc}d{W}_t=(S{W}_t+s)d{B}_t,& \left(1\right)\end{array}$

where W_t denotes the excitatory synaptic weight W at time t, and B_t is the standard Wiener process (i.e., the continuous-time stochastic process with zero mean and unit variance per unit time). Equation (1) implies that the expected change E[W_t − W₀] in synaptic weight W is zero. However, in this study, the synaptic weight was bounded at zero to prevent a negative weight; thus, the expected change E[W_t − W₀] at near-zero synaptic weight was positive. Yasumatsu et al. (2008) estimated the parameter values S = 0.2 and s = 0.01 μm³ per day. This study empirically employed parameter values of S = 0.2 and s = 7000 pS per day.

2.2. Spike-Timing Dependent Plasticity

Spike-timing dependent plasticity (STDP) is a temporally asymmetric form of Hebbian plasticity that allows both depression and potentiation depending on the temporal relationship between pre-synaptic firing and post-synaptic firing (Markram et al., 1997; Bi and Poo, 1998; Sjöström et al., 2001). When a pre-synaptic firing precedes a post-synaptic firing, the synapse is potentiated, whereas when a pre-synaptic firing follows a post-synaptic firing, the synapse is depressed. The STDP model is commonly expressed as follows:

$\begin{array}{cc}\Delta W(W,\Delta t)=\{\begin{array}{cc}{A}_{+}(W)\exp \left(-\frac{\left|\Delta t\right|}{{\tau }_{+}}\right)& \text{if }\Delta t<0\\ A_{-}(W)\exp \left(-\frac{\left|\Delta t\right|}{{\tau }_{-}}\right)& \text{if }\Delta t>0,\end{array}& \left(2\right)\end{array}$

where W is the synaptic weight and ΔW is the magnitude of the change in the synaptic weight W. The synaptic weight W is updated to W + ΔW immediately after whichever occurs later, the pre-synaptic firing (at t_pre in depression) or the post-synaptic firing (at t_post in potentiation). This is expressed as the temporal difference Δt = t_post − t_pre. The functions A₊(W) and A₋(W) determine the magnitudes ΔW of potentiation and depression, and the parameters τ₊ and τ₋ are the time constants of the magnitude ΔW decay. This study employed the multiplicative form of the STDP proposed by van Rossum et al. (2000), which is expressed as

$\begin{array}{cc}\begin{array}{c}{A}_{+}(W)=c_{+}+{\nu }_{p}W,A_{-}(W)=-c_{-}W+{\nu }_{p}W,\end{array}& \left(3\right)\end{array}$

where ν_p follows a normal distribution with zero mean and variance ${\sigma }_p^2$, i.e., ${\nu }_p~{N}(0,{\sigma }_p^2)$. A strong synapse is easily depressed and further potentiation of the strong synapse is prevented because the amount A₋(W) of depression increases with synaptic weight W. Thus, the multiplicative STDP model is also referred to as soft-bounded STDP model (van Rossum et al., 2012). In this study, STDP was assumed to occur not between all possible pairs of pre- and post-synaptic firings but between the pair of a newly generated post-synaptic (pre-synaptic) firing and the latest pre-synaptic (post-synaptic) firing (Morrison et al., 2008; Watt and Desai, 2010). The parameter values were the same as those estimated in the original study by van Rossum et al. (2000): τ₊ = τ₋ = 20 ms, c₊ = 1 pS, c₋ = 0.003, and σ_p = 0.015.

2.3. Activity-Dependent Scaling

For comparison, a phenomenological model of activity-dependent scaling is introduced (van Rossum et al., 2000). This model monitors the activity of a post-synaptic neuron with a slow-varying sensor a as follows:

$\begin{array}{cc}{\tau }_a\frac{da}{dt}=-a+{\displaystyle \sum _k\delta }(t-t_k),& \left(4\right)\end{array}$

where δ(·) is the delta function, and t_k is the time of the k-th post-synaptic firing. At the state a, the weight W of a synapse connected to the neuron is regulated as follows:

$\begin{array}{cc}\frac{dW}{dt}=\beta W(a_g-a(t))+\gamma W{\displaystyle {\int }_0^t(a_g-a(}t\prime ))dt\prime ,& \left(5\right)\end{array}$

where a_g (Hz) is the target post-synaptic firing rate. This formulation follows the classical proportional-integral controller. This study employed the same parameter values as those used in the original study, i.e., τ_a = 100 s, β = 4 × 10⁻⁵, and γ = 10⁻⁷ s⁻¹, except that the target post-synaptic firing rate a_g was set to 5 Hz, which is more biologically realistic than the original value of 20 Hz (Mizuseki and Buzsáki, 2013; Buzsáki and Mizuseki, 2014). The features of these models are compared in Table 1.

TABLE 1

Table 1. Comparison between properties among models.

2.4. Dynamics of Neuron and Synapses

The dynamics of the neuron were described by a leaky integrate-and-fire model (van Rossum et al., 2000; Izhikevich, 2004):

$\begin{array}{cc}{\tau }_m\frac{dv}{dt}=(v_L-v)+g_E(v_E-v)R+g_I(v_I-v)R,& \left(6\right)\end{array}$

where v is the membrane potential, τ_m = 20 ms is its time constant, and v_L = −60 mV, v_E = 0 mV, and v_I = −70 mV are the reversal potentials of leak, excitatory, and inhibitory currents, respectively. Hereafter, the subscripts E and I represent excitatory and inhibitory states, respectively; thus, g_E and g_I are the conductances of excitatory and inhibitory synaptic currents. R = 100 MΩ is the input resistance. When the membrane potential v reaches the threshold voltage v_th = −50 mV, it is reset to v_r = −60 mV immediately, and the neuron fires (i.e., generates an action potential). The number of post-synaptic firings per unit time is the post-synaptic firing rate f_post. The dynamics of excitatory and inhibitory synaptic conductances are both described as follows:

$\begin{array}{cc}\frac{d{g}_X}{dt}=-\frac{g_X}{{\tau }_X}+{\displaystyle \sum _i{W}_i}{\displaystyle \sum _k\delta }\left(t-s_i^{(k)}\right)\text{ for }X\in \{E,I\},& \left(7\right)\end{array}$

where τ_X = 5 ms is the time constant of the conductance. The subscript i is the label of the synapse, W_i is the synaptic weight, and $s_i^{(k)}$ is the time of the k-th event at synapse i. The weights W of the excitatory synapses were plastic and followed the synaptic plasticity models described above. They are bounded at 0 to prevent a negative weight. The weights W of all inhibitory synapses were fixed at W_I = 4000 pS. All the parameter values were the same as those in the original study (van Rossum et al., 2000).

For numerical simulations, all the equations were solved by the forward Euler method with the time step h = 0.1 ms. A single post-synaptic neuron model was constructed that receives inputs from N_f = 100 excitatory and N_f ∕ 4 = 25 inhibitory synapses. The pre-synaptic firing rate was denoted by f_pre (Hz). The pre-synaptic firing rate per time step was p = f_preh. Excitatory pre-synaptic firings were assumed to be mutually correlated (van Rossum et al., 2000; Bair et al., 2001). This study generated a correlated input as a set of simultaneous excitatory pre-synaptic firings. A correlated group contained N_c excitatory synapses. An event within the correlated group followed a Poisson process of rate p_c per time step. For each event within the correlated group, m excitatory synapses were randomly chosen from the correlated group and were simultaneously activated. The rate p_c was set to pN_c ∕ m so that the pre-synaptic firing rate per time step was p. If the random variable X_i ∈ {0, 1} represents the occurrence of an event at the excitatory synapse i, the Pearson correlation coefficient between two excitatory synapses i and j in the correlated group is

$\begin{array}{cc}c=\frac{\text{cov}(X_i,X_j)}{{\sigma }_{X_i}{\sigma }_{X_j}}=\frac{\frac{p{N}_c}{m}\frac{{}_{N_c-2}{\text{C}}_{m-2}}{{}_{N_c}{\text{C}}_m}-p^2}{p(1-p)}=\frac{\frac{m-1}{N_c-1}-p}{1-p}\simeq \frac{m-1}{N_c-1}& \left(8\right)\end{array}$

as long as p ≪ 1. This study divided the excitatory synapses into four groups, i.e., N_c = 25. Thus,

$\begin{array}{cc}c\simeq 0.04\times (m-1).& \left(9\right)\end{array}$

In the case of uncorrelated inputs (i.e., c = 0), the excitatory synapses were activated by homogeneous Poisson processes at f_pre (Hz). In both cases, the inhibitory synapses followed homogeneous Poisson processes at f_pre (Hz). All the results were obtained from ten trials.

3. Results

3.1. Numerical Simulations

3.1.1. Stability of Neuronal Activity

This section examines the capacity of different STDP models (see Table 1) to maintain neuronal activity in response to a change in synaptic input (f_pre). The excitatory synaptic weights W were initialized to a sufficiently large value, the synapses were activated with a pre-synaptic firing rate f_pre = 5 Hz and a correlation c = 0.04, and the post-synaptic neuron accepted the pre-synaptic firings. After a sufficiently long duration, the post-synaptic firing rate f_post in response to the pre-synaptic firings reached a steady state. The changes in post-synaptic firing rate f_post in response to a sudden decrease in the pre-synaptic firing rate f_pre are shown for each model in the left panel of Figure 1. The black line denotes the result obtained from the STDP model with no additional factors, which is referred to as the +NO model. The initial steady-state (baseline) post-synaptic firing rate f_post of this model in response to the pre-synaptic firing rate f_pre = 5 Hz was very low (< 0.1 Hz). At time t = 0, the pre-synaptic firing rate f_pre was decreased to 3 Hz to simulate a condition such as sensory deprivation (Keck et al., 2013). After this decrease, the post-synaptic firing rate f_post decreased to approximately zero and never returned to its previous value. To prevent this excessively low post-synaptic firing rate f_post, the amplitude c₊ of potentiation was increased to 1.5 pS in a model referred to hereafter as +LP model, (blue lines in Figure 1). A larger potentiation c₊ = 1.5 pS induced larger synaptic weights W and a higher baseline post-synaptic firing rate f_post in response to the pre-synaptic firing rate f_pre = 5 Hz. After decreasing the pre-synaptic firing rate f_pre, the post-synaptic firing rate f_post temporally fell to zero and quickly started to increase. However, it did not return to baseline. Both models cannot prevent excessively low neuronal activity. In contrast, the model incorporating intrinsic fluctuations (red line in Figure 1), hereafter denoted as the +IF model, prevented this drastic decrease in the post-synaptic firing rate f_post and restored it to baseline, thereby indicating homeostatic plasticity. For comparison, the synaptic plasticity model incorporating activity-dependent scaling and STDP (the +ADS model, purple line in Figure 1) also showed a rapid return to the baseline post-synaptic firing rate f_post after an overshoot. Ten synapses were randomly chosen and their trajectories are shown in the lower part of Figure 1. In the case of the +NO model, the synapses were almost unchanged owing to very low post-synaptic firing rate f_post. In the other cases, some synapses became strong but others became weak; synapses were continuously shuffled even after the post-synaptic firing rate f_post converged to a steady state.

FIGURE 1

Figure 1. Changes in post-synaptic firing rate f_post following a change in the pre-synaptic firing rate f_pre. Homeostatic plasticity is expected to restore the post-synaptic firing rate f_post to its previous (baseline) value. (Left panel) The pre-synaptic firing rate f_pre is decreased from 5 to 3 Hz. The different colored lines denote results of STDP models with no additional factors (+NO model, black), with a larger potentiation c+ = 1.5 pS (+LP model, blue), with intrinsic fluctuations (+IF model, red), and with activity-dependent scaling model (+ADS model, purple). The post-synaptic firing rates f_post of the +NO and +LP models decrease to approximately zero and never return to the previous baseline value. On the contrary, an excessively low post-synaptic firing rate f_post is prevented in the +IF and +ADS models. The trajectories of randomly chosen ten synapses are shown in the lower part. In the case of the +NO model, the synapses are almost unchanged owing to very low post-synaptic firing rate f_post. In the other cases, some synapses become strong but others become weak; synapses are continuously shuffled even after the post-synaptic firing rate f_post converged to a steady state. (Right panel) The pre-synaptic firing rate f_pre is increased from 5 to 7 Hz. As is the case with the decreased pre-synaptic firing rate f_pre, the +IF and +ADS models restore the post-synaptic firing rate f_post to the previous baseline value, whereas the +NO and +LP models do not.

The pre-synaptic firing rate f_pre was then increased from 5 to 7 Hz to simulate increased neuronal activity (right panel of Figure 1). In all four models, the post-synaptic firing rate f_post decreased after an initial increase. However, similar to the case of the simulation of decreased pre-synaptic firing rate f_pre, the +IF and +ADS models showed rapid restoration of post-synaptic firing rate f_post to the baseline value, whereas the +NO and +LP models did not. Thus, the +IF and +ADS models prevented excessive changes in neuronal activity and thereby demonstrated homeostatic plasticity. The trajectories of synapses are also shown in the lower part. In cases with the +IF and +ADS models, synapses were depressed on average. More detailed analyses are provided in the following sections.

In subsequent examinations, the pre-synaptic firing rate f_pre and the correlation c of the pre-synaptic firings were widely varied. Figure 2 shows the relationship between pre-synaptic firing rate f_pre and resulting post-synaptic firing rates f_post at steady state. In the +NO model, the post-synaptic firing rate f_post reached a value of ≥1 Hz over a range of pre-synaptic firing rate f_pre = 0.01–20 Hz only with a high correlation c≥0.08. With a low correlation c < 0.08, the post-synaptic firing rate f_post decreased to approximately zero (f_post < 0.1 Hz), regardless of the pre-synaptic firing rate f_pre. Thus, the soft-bounded STDP model prevented excessively strong synaptic weights W but simultaneously leaded to a post-synaptic firing rate f_post of approximately zero because of positive feedback. The +LP model was expected to prevent this excessively low post-synaptic firing rate f_post because of a larger potentiation (c₊ = 1.5 pS). However, again the post-synaptic firing rate f_post fell to approximately zero at a pre-synaptic firing rate f_pre of < 1 Hz. Moreover, the post-synaptic firing rate f_post rose to >100 Hz at a high pre-synaptic firing rate f_pre > 20 Hz or a high correlation c > 0.04. This result indicates that increasing the amplitude c₊ of potentiation is not enough for preventing almost silent activity and leads to a runaway excitatory as a side effect. As mentioned in the previous studies, the ranges of parameter values with which soft-bounded STDP models provide stability of neuronal activity are strictly limited (Morrison et al., 2007; Gilson and Fukai, 2011; Yger and Gilson, 2015). On the other hand, the +IF model maintained a post-synaptic firing rate f_post within 3–10 Hz over a broad range of pre-synaptic firing rate f_pre (0.01–30 Hz) and correlation c (0–0.12) values because the intrinsic fluctuations potentiated synapses with near zero weights (as mentioned in Section 2.1). The post-synaptic firing rate f_post varied little with pre-synaptic firing rate f_pre but nonetheless increases with correlation c (Figure 2, third panel, red lines). The +IF model has doubly noise terms; intrinsic fluctuations and the noise term ${\nu }_p~{N}(0,{\sigma }_p^2)$ in potentiation and depression of soft-bounded STDP model. The +IF model without the noise term ν_p (σ_p = 0) was also examined (Figure 2, third panel, gray lines). The results were almost the same as those obtained from the +IF model with the noise term ν_p. For comparison, the +ADS model that incorporates both the activity-dependent scaling model and the STDP model, was examined. In this case, regardless of pre-synaptic firing rate f_pre and correlation c, the resultant post-synaptic firing rate f_post was almost similar to the target post-synaptic firing rate a_g = 5 Hz.

FIGURE 2

Figure 2. The post-synaptic firing rates f_post in response to varying the pre-synaptic firing rate f_pre and the correlation c. The axis of abscissas represents to the pre-synaptic firing rate f_pre and the line type refers to the correlation c. The +NO, +LP, +IF, and +ADS models are all STDP models but with no additional factors, a larger potentiation c+ = 1.5 pS, intrinsic fluctuations, or activity-dependent scaling model. At a high pre-synaptic firing rate f_pre and a low correlation c, the +NO and +LP models cannot prevent a near-zero post-synaptic firing rate f_post. The +LP model also cannot prevent an excessively high post-synaptic firing rate f_post. On the other hand, the +IF and +ADS models maintain the post-synaptic firing rate f_post within a functional range, indicating that intrinsic fluctuations and activity-dependent scaling model achieve homeostatic plasticity. The gray lines in the third panel denote the results obtained from the +IF model without the noise term ν_p, which were almost the same as those obtained from the +IF model with the noise term ν_p denoted by the red lines.

This section concludes that intrinsic fluctuations maintain neuronal activity within limits in response to changing conditions (pre-synaptic firing rate and correlation) and can mediate homeostatic plasticity (see Table 1).

3.1.2. Scaling of Synaptic Weights

According to the hypothesis of synaptic scaling (Turrigiano et al., 1998; Ibata et al., 2008; Keck et al., 2013), neuronal activity is adjusted by scaling the excitatory synaptic weights multiplicatively. To confirm this hypothesis, Figure 3 summarizes the cumulative distributions of synaptic weights W at steady states for the four models. First, the effects of changing pre-synaptic firing rate f_pre are shown in Figure 3A. The +NO, +IF, and +ADS models all yielded lower synaptic weights W with progressive increases in pre-synaptic firing rate f_pre; these results are consistent with the synaptic scaling observed in previous biological experiments (Turrigiano et al., 1998; Ibata et al., 2008; Keck et al., 2013). Contrary to the previous theoretical studies (van Rossum et al., 2000; Gilson and Fukai, 2011), the +NO model (the STDP model with no additional factors) also achieved synaptic scaling. A pre-synaptic firing always has a chance to induce post-synaptic firing and potentiation. With decreasing pre-synaptic firing rate f_pre, the chance that a post-synaptic firing immediately precedes a pre-synaptic firing decreases (as they become sparse); thus, the probability of depression decreases and the synapses are potentiated more easily (see Section 3.2 for details). However, this potentiation is not sufficient for the post-synaptic neuron to elicit action potentials above a certain rate, as shown in Section 3.1.1. In the +LP model with pre-synaptic firing rate f_pre = 20 Hz, the synaptic weights W become exceptionally high. According to Figure 2, the post-synaptic firing rate f_post diverged from baseline, representing runaway excitation. Therefore, all of these models can adjust synaptic weight, but only the +IF and the +ADS models can adjust it sufficiently to prevent runaway excitation and silence (see Table 1). The +IF model without the noise term ν_p (σ_p = 0) was also examined (Figure 3A, third panel, gray lines). The results were also almost the same as those obtained from the +IF model with the noise term ν_p.

FIGURE 3

Figure 3. The cumulative distributions of the excitatory synaptic weights W at steady states with varying pre-synaptic firing rate f_pre or correlation c. (A) The pre-synaptic firing rate f_pre is varied. (B) The pre-synaptic firing rate f_pre is varied and the synaptic weights W are scaled appropriately. (C) The correlation c is varied. Colors as in Figures 1, 2.

The synaptic weights W were scaled so that their first and third quartiles were constant with varying pre-synaptic firing rate f_pre. The results are summarized in Figure 3B. The shapes of the cumulative distributions of the synaptic weights W were similar in most cases; however, the +NO model with pre-synaptic firing rate f_pre = 1 Hz yielded a different shape. This indicates that STDP, which is incorporated into all models examined, achieves synaptic scaling by itself.

Next, the correlation c is varied. It is to be noted that the original biological experiments (Turrigiano et al., 1998; Ibata et al., 2008; Keck et al., 2013) did not refer to synaptic scaling induced by varying the correlation c because varying correlation c is difficult in biological systems (but not in silico). The results are summarized in Figure 3C. The +NO and +LP models show increased synaptic weights W with higher correlation c, indicating that these models do not provide negative feedback against increasing correlation c. With increase in the correlation c, the +IF model exhibits a decreased number of strong synapses but a greater number of weak synapses. The +ADS model shows depression of all synapses with increasing correlation c. Because synaptic scaling induced by varying the correlation c has not been examined in vivo, this examination cannot draw biological conclusions.

3.1.3. Stability of Strong Synapses

Memory is considered to be stored as synaptic connections and thus strong synapses should stay strong (Fusi et al., 2005; Fusi and Abbott, 2007; Morrison et al., 2007; Gilson and Fukai, 2011). Intrinsic fluctuations however have a risk disturbing retention of memory by depressing strong synapses since they are random fluctuations. However, not only intrinsic fluctuations but also the soft-bounded STDP model shuffles synapses (see the lower part of Figure 1). Effects of intrinsic fluctuations on stability of strong synapses were examined. The pre-synaptic firing rate f_pre and the correlation c were set to 5 Hz and c = 0.08, and the post-synaptic firing rates f_post reached a steady state. Synapses whose weights are stronger than the 90th percentile of all the synaptic weights of each respective trial at t = 0 were treated as strong. The survival rates, i.e., the rates of the synapses remaining strong from t = 0, are depicted as the solid lines in Figure 4A. The following half-lives were estimated by maximum likelihood; 7.5, 1.9, 4.0, and 4.4 min for the +NO, +LP, +IF, and +ADS models, respectively (see the dotted lines in Figure 4A). In this case, the post-synaptic firing rates f_post were 2.02, 16.37, 5.23, and 4.97 Hz for the +NO, +LP, +IF, and +ADS models, respectively (see Figure 2). The estimated half-lives with varying pre-synaptic firing rate f_pre are shown in the left panel of Figure 4B; the half-lives obtained from the +IF model were almost equal to those from the +ADS model, shorter than those from the +NO model, and longer than those from the +LP model. The increasing order of the half-lives was similar to the decreasing order of post-synaptic firing rates f_post (see also Figure 2). Only with a small pre-synaptic firing rate f_pre, the +IF model resulted in a shorter half-life despite the lower post-synaptic firing rate f_post when compared with the +ADS model. The middle panel of Figure 4B shows the same results except that the axis of abscissas represents the post-synaptic firing rate f_post instead of the pre-synaptic firing rate f_pre. Owing to homeostatic function, the post-synaptic firing rates f_post were varied a little with the +IF and +ADS models. No obvious relationship between the half-life and the post-synaptic firing rate f_post was found. The right panel has the product f_pre × f_post of the pre- and post-synaptic firing rates on the axis of abscissas. The dotted gray line represents 130∕(f_pre × f_post) (min · Hz⁻²). The half-life was roughly inversely proportional to the product f_pre × f_post of the pre- and post-synaptic firing rates; the product f_pre × f_post is also roughly proportional to the occurrence probability of STDP (van Rossum et al., 2000; Gilson and Fukai, 2011) (see also Section 3.2.1). These results suggest that the half-life (i.e., the stability) of the strong synapse is mainly related to the neuronal activity and the resulting occurrence probability of STDP. Intrinsic fluctuations have a slightly negative influence on the stability of strong synapses but it is almost negligible as long as the neuronal activity (i.e., the occurrence probability of STDP) is not too small.

FIGURE 4

Figure 4. Stability of strong excitatory synapses. Synapses whose weights are stronger than the 90th percentile of all the synaptic weights of each respective trial at t = 0 are treated as strong. (A) The solid lines denote the survival rate of the strong synapses in response to the pre-synaptic firing with the rate of f_pre = 5 Hz and the correlation of c = 0.08. The dotted lines denote the half-lives estimated by maximum likelihood; 7.5, 1.9, 4.0, and 4.4 min for +NO, +LP, +IF and +ADS models respectively. (B) The estimated half-lives with varying pre-synaptic firing rate f_pre. The axes of abscissas represent the pre-synaptic firing rate f_pre (left), the post-synaptic firing rate f_post (middle), and the product f_pre f_post of them (right). The numerals in the middle panel indicate the corresponding pre-synaptic firing rates f_pre (Hz).

3.1.4. Scaling after Silenced Neuronal Activity

Homeostatic plasticity has been also found during prolonged bath application of the voltage-gated sodium channel antagonist tetrodotoxin (TTX), which silences neuronal activity (Turrigiano et al., 1998; Ibata et al., 2008). This section highlights the examination of the cumulative distribution of excitatory synaptic weights W in transient states after the pre-synaptic firing rate f_pre is reduced to zero. The correlation c was set to 0, and the pre-synaptic firing rate f_pre was decreased from 5 to 0 Hz at t = 0. The results are shown in Figure 5. Figure 5A shows the trajectories of randomly chosen ten synapses. The +NO and +LP models did not regulate the synaptic weights after the pre-synaptic firing rate f_pre was silenced; the synaptic weights were unchanged. The +IF model shuffled synaptic weights continuously and slowly potentiated synaptic weights on average. The +ADS model potentiated synaptic weights drastically; the trajectories obtained from the +ADS model were clipped for visibility. Figure 5B shows the survival rates of the synapses remaining stronger than the 90th percentile. The +ADS model kept the order of weight since it potentiated synapses multiplicatively. The survival rates obtained from the +NO, +LP, and +ADS models are 1.0. Since intrinsic fluctuations induce change with zero expected value, the +IF model had probabilities for potentiating the weak synapses and depressing the strong synapses; the half-life is 9.7 min. As mentioned in Section 3.1.3, their negative influence on the stability of the strong synapses is no longer negligible. Furthermore, the half-life was obtained in a transient state differently from the half-life in Figure 4, which was obtained in steady state. The synaptic weights are considered to be less stable than those in steady state. Figure 5C shows the cumulative distributions of the excitatory synaptic weights W. The +IF model potentiated the synaptic weights W over several days. The +ADS model potentiated the synaptic weights W, but the post-synaptic firing rate f_post never returned to the target post-synaptic firing rate a_g (Hz) because the pre-synaptic firing rate f_pre was zero. Consequently, the synaptic weights W became infinite. This result is not consistent with that of the previous biological experiments (Turrigiano et al., 1998; Turrigiano, 1999, 2008; Burrone et al., 2002; Ibata et al., 2008; Keck et al., 2013; Félix-Oliveira et al., 2014). Hence, based on these results, it can be concluded that the +IF and +ADS models achieve synaptic scaling even after neuronal activity is silenced, but the +ADS model exhibits strange behavior (yields saturated synaptic weights W) under this condition (see Table 1 for comparison). A theoretical analysis is provided in Figure 7A.

FIGURE 5

Figure 5. Excitatory synaptic weights W in transient states after neuronal activity is silenced. The correlation c is 0, and the pre-synaptic firing rate f_pre is decreased from 5 to 0 Hz at t = 0. (A) The trajectories of randomly chosen ten synapses. The +NO and +LP models cannot regulate the synaptic weights after the pre-synaptic firing rate f_pre is silenced; the synaptic weights are unchanged. The +IF model shuffles synaptic weights continuously and slowly potentiates synaptic weights on average. The +ADS model potentiates synaptic weights drastically; the trajectories are clipped for visibility. (B) The survival rates of the synapses remaining stronger than the 90th percentile. The +ADS model keeps the order of weight since it potentiates synapses multiplicatively. The survival rates obtained from the +NO, +LP, and +ADS models are 1.0. The +IF model has a probability for depressing the strong synapses; the half-life is 9.7 min. Note that the half-life is obtained in a transient state differently from the half-life in Figure 4, which is obtained in steady state. (C) The cumulative distributions of the excitatory synaptic weights W. The +IF model potentiates the synaptic weights W over several days. The +ADS model potentiates the synaptic weights W, but the post-synaptic firing rate f_post never returns to the target post-synaptic firing rate ag (Hz) because the pre-synaptic firing rate f_pre is zero. Consequently, the synaptic weights W become infinite.

3.1.5. Competition

Co-occurring excitatory pre-synaptic firings have a greater chance to induce post-synaptic firing because of summation, thereby potentiating synapses. In contrast to this type of cooperation, synaptic plasticity is also competitive (Miller, 1996; Song et al., 2000; van Rossum et al., 2000; Gütig et al., 2003; Morrison et al., 2007; Gilson and Fukai, 2011); when a group of excitatory synapses has an effect, another group is depressed. Competition between synapses contributes to selectivity of neurons for different input patterns. The traditional STDP models exhibit strong competition often with destabilization of synaptic weights, the soft-bounded STDP models exhibit weak competition, and the activity-dependent scaling model promotes competition. Similar to the activity-dependent scaling model, intrinsic fluctuations are expected to contribute to competition. To examine the competition achieved by synaptic plasticity, two groups A and B of excitatory synapses were chosen from the four groups. Groups A and B had the same pre-synaptic firing rate f_pre = 5.0 Hz and had the same or different correlations c_A and c_B. First, the correlations were set to c_A = c_B = 0.04. After a sufficiently long duration, the correlation c_B was increased to c_B = 0.16 at t = 0, as shown in Figure 6A. The resulting changes in the post-synaptic firing rate f_post and synaptic weights W are summarized in Figure 6B. In all cases, the post-synaptic firing rate f_post increased to a larger value at time t = 0 and the average weight ${\bar{W}}_A$ of group A synapses started increasing. The response of the average weight ${\bar{W}}_B$ of group B synapses depended on the plasticity model. In the +NO model, the average weight ${\bar{W}}_B$ of group B was depressed. In other words, the model resulted in competition between synapses. The +IF and +ADS models also exhibited competition. On the other hand, the average weight ${\bar{W}}_B$ was almost unchanged in the +LP model. Thus, whether soft-bounded STDP model exhibits competition depends on parameter values (e.g., the amplitude c₊ of the potentiation). A theoretical analysis is shown in Section 3.2.3.

FIGURE 6

Figure 6. The competition between the correlated groups (A,B) of excitatory synapses. The time courses of the correlations cA and cB. (A) The time courses of the post-synaptic firing rates f_post. (B) The time courses of the average synaptic weights $\bar{W}$ of the groups A and B. The +NO, +IF, and +ADS models all exhibit the competition, while the +LP model do not.

3.2. Theoretical Analyses

3.2.1. Under Uncorrelated Activity

In this section, a procedure to obtain a synaptic weight distribution P(W) theoretically is explained, and the mechanisms of homeostatic plasticity and competition are analyzed. The synaptic modifications ΔW yielded by different synaptic plasticity models can be treated as stochastic differential equations as long as ΔW are sufficiently small. A synaptic weight distribution P(W) is obtained using the Fokker-Planck equation (Øksendal, 2003) with the moments ${M}$_n of the synaptic modifications ΔW, where the moments ${M}$_n are derived as

$\begin{array}{cc}{{M}}_n(W)={\displaystyle \int P}(\Delta W)\text{E}[\Delta W{(W)}^n]d\Delta W.& \left(10\right)\end{array}$

First, for simplicity, post-synaptic firing is assumed to be unrelated to pre-synaptic firings and to follow a single Poisson process of f_post (Hz), which is obtained from the numerical simulations in Section 3.1. This assumption is reasonable when a post-synaptic neuron accepts inputs from numerous synapses or its membrane potential is noisy. In addition, STDP is assumed to occur between all possible pairs of pre- and post-synaptic firings (all-to-all STDP model Morrison et al., 2008; Watt and Desai, 2010) also for simplicity. This approximation is reasonable as long as pre- and post-synaptic neuron firing rates are not excessively high (Gilson and Fukai, 2011). The n-th moments ${{M}}_n^{STDP}(W)$ of the synaptic modifications yielded by the STDP model are derived as follows (van Rossum et al., 2000; Yasumatsu et al., 2008; Kasai et al., 2010; Gilson and Fukai, 2011; Statman et al., 2014):

$\begin{array}{cc}\begin{array}{l}{{M}}_n^{\text{STDP}}(W)={\displaystyle {\int }_{-\text{}\infty }^{\infty }P(\Delta W)\text{E}[\Delta W{(W)}^n]d\Delta W}\\ =f_{\text{post}}{\displaystyle {\int }_{-\text{}\infty }^0\text{E}}[\Delta W{(W,\Delta t)}^n]P(\Delta t)d\Delta t\\ +f_{\text{pre}}{\displaystyle {\int }_0^{\infty }\text{E}}[\Delta W{(W,\Delta t)}^n]P(\Delta t)d\Delta t\\ =f_{\text{pre}}{f}_{\text{post}}{\displaystyle {\int }_{-\text{}\infty }^0\text{E}[A_{+}\text{}{(W)}^n]\exp \left(-n\frac{\left|\Delta t\right|}{{\tau }_{+}}\right)d\Delta t}\\ +f_{\text{pre}}{f}_{\text{post}}{\displaystyle {\int }_0^{\infty }\text{E}[A_{-}\text{}{(W)}^n]\exp \left(-n\frac{\left|\Delta t\right|}{{\tau }_{-}}\right)d\Delta t}\\ =\frac{1}{n}{f}_{\text{pre}}{f}_{\text{post}}\left({\tau }_{+}\text{E}[A_{+}{(W)}^n]+{\tau }_{-}\text{E}[A_{-}{(W)}^n]\right),\end{array}& \left(11\right)\end{array}$

where P(ΔW) and P(Δt) are the occurrence probabilities of the potentiation (or depression) with the amount of ΔW and with the temporal difference Δt. P(Δt) is equal to the pre-synaptic firing rate f_pre when Δt > 0 and to the post-synaptic firing rate f_post when Δt < 0 because of the all-to-all STDP model. Because this study employs the multiplicative STDP model, described in Equation (3), the moments ${M}$_n are as follows:

$\begin{array}{cc}\begin{array}{l}{{M}}_1^{\text{STDP}}(W)=f_{\text{pre}}{f}_{\text{post}}\left({\tau }_{+}{c}_{+}-{\tau }_{-}{c}_{-}W\right),\\ {{M}}_2^{\text{STDP}}(W)=\frac{1}{2}{f}_{\text{pre}}{f}_{\text{post}}({\tau }_{+}(c_{+}^2+{\sigma }_p^2{W}^2)\\ +{\tau }_{-}(c_{-}^2{W}^2+{\sigma }_p^2{W}^2)).\end{array}& \left(12\right)\end{array}$

In addition to STDP, the +IF and +ADS models take into account synaptic modifications yielded by intrinsic fluctuations and the activity-dependent scaling model, respectively. The moments ${{M}}_n^H$ of their synaptic modifications are called as the homeostatic terms in this study and expressed as follows (van Rossum et al., 2000; Yasumatsu et al., 2008; Kasai et al., 2010):

$\begin{array}{cc}\begin{array}{l}{{M}}_1^{\text{H}}(W)=\{\begin{array}{ll}\beta W(a_g-a(t))\hfill & \text{for}\text{the}+\text{ADS}\text{}\text{model},\hfill \\ +\gamma W{\displaystyle {\int }_0^t(a_g-a}(t\prime ))dt\prime \hfill & \hfill \\ 0\hfill & \text{for other models}.\hfill \end{array}\hfill \\ {{M}}_2^{\text{H}}(W)=\{\begin{array}{ll}{(SW+s)}^2\hfill & \text{for}\text{the}+\text{IF}\text{model},\hfill \\ 0\hfill & \text{for other models}.\hfill \end{array}\hfill \end{array}\hfill & \left(13\right)\end{array}$

Because of the assumption that the post-synaptic firing follows a single Poisson process of f_post, a(t) is clamped to a_g and the activity-dependent scaling model does not work. Thus, the moments ${M}$_n of the synaptic modifications are as follows:

$\begin{array}{cc}\begin{array}{l}{{M}}_1(W)={{M}}_1^{\text{STDP}}(W)+{{M}}_1^{\text{H}}(W),\\ {{M}}_2(W)={{M}}_2^{\text{STDP}}(W)+{{M}}_2^{\text{H}}(W).\end{array}& \left(14\right)\end{array}$

The Fokker-Planck equation describes the changes in the synaptic weight distribution P(W, t) as follows:

$\begin{array}{cc}\frac{\partial }{\partial t}P(W,t)=-\frac{\partial }{\partial W}{{M}}_1(W)P(W,t)+\frac{1}{2}\frac{{\partial }^2}{\partial W^2}{{M}}_2(W)P(W,t).& \left(15\right)\end{array}$

After a sufficiently long duration, the synaptic weight distribution P(W, t) converges to a steady state, which is denoted by P(W). Because the left side is equal to zero, the steady state synaptic weight distribution P(W) is

$\begin{array}{cc}P(W)=\frac{C}{{{M}}_2(W)}\exp \left({\displaystyle {\int }_0^W\frac{2{{M}}_1(W\prime )}{{{M}}_2(W\prime )}}dW\prime \right)& \left(16\right)\end{array}$

where C is a normalization factor. Figure 7A shows the theoretically-obtained synaptic weight distributions P(W). The colored lines denote the theoretical results and the gray lines denote the corresponding numerical results shown in Figure 3A. They are almost different except for the case of the +IF model. In the absence of intrinsic fluctuations, both the first moment ${M}$₁(W) and the second moment ${M}$₂(W) are directly proportional to the product of the pre-synaptic firing rate f_pre and the post-synaptic firing rate f_post. They cancel out in Equation (16), and the synaptic weight distribution P(W) is independent of neuronal activity. In the presence of intrinsic fluctuations, the second moment ${M}$₂(W) is the sum of the activity-dependent term ${{M}}_2^{STDP}(W)$ and the activity-independent term ${{M}}_2^H(W)$; thus, the synaptic weight distribution P(W) depends on the product of the pre-synaptic firing rate f_pre and the post-synaptic firing rate f_post. With increasing pre-synaptic firing rate f_pre, the activity-independent term ${{M}}_2^H(W)$ is relatively negligible and the synaptic weight distribution P(W) resembles that in the absence of intrinsic fluctuations. With decreasing pre-synaptic firing rate f_pre, the activity-independent term ${{M}}_2^H(W)$ is dominant and the synaptic weights are widely distributed because of the relatively large diffusion term ${M}$₂(W). This is the mechanism by which the +IF model regulates synaptic weights W against a change in neuronal activity.

FIGURE 7

Figure 7. The theoretically-obtained cumulative distributions of excitatory synaptic weights W. The black and colored lines denote the theoretical results and the gray lines denote the corresponding numerical results shown in Figure 3A. (A) Post-synaptic firing is assumed to be unrelated to pre-synaptic firings and to follow a single Poisson process of f_post (Hz) (see Equation 14). Synaptic scaling is observed only in the case of the +IF model. The theoretical results and the numerical results are almost different except for the case of the +IF model. (B) The dependency of post-synaptic firing on pre-synaptic firings is taken into account (see Equation 17). All the models achieve synaptic scaling. By taking into account the dependency terms ${{M}}_n^{Dep}(W)$, the theoretical results closely resemble the numerical results except for the +LP model with a high pre-synaptic firing rate f_pre = 20.0. (C) The cumulative distributions of excitatory synaptic weights W under the competitive situation. The solid lines denote the synaptic weight distribution P(W) when correlated excitatory synapse groups A and B had the same correlation c_A = c_B = 0.04. The dashed lines denote the synaptic weight distribution P_A(W) of correlated group A after its correlation c_A is increased to 0.16. The synapses are potentiated. The dotted lines denote the synaptic weight distribution P_B(W) of correlated group B after c_A is increased to 0.16. When the synapses in the correlated group B are depressed, synaptic scaling is achieved.

In the case of silenced neuronal activity (f_pre = 0), the activity-dependent terms ${{M}}_n^{STDP}(W)$ are zero. Thus, the synaptic weight distributions P(W) obtained from the +NO and +LP models are unchanged. Owing to the activity-independent term ${{M}}_2^H(W)$, the +IF and +ADS models mediate synaptic weight. The steady-state synaptic weight distribution P(W) obtained from the +IF model is a power-law distribution $\frac{Ss}{{(SW+s)}^2}$ with the exponent of 2; it has no well-defined mean weight. The steady-state synaptic weight distribution P(W) obtained from the +ADS model cannot be well-defined; activity-dependent scaling model keeps on potentiating synaptic weights unboundedly. Both the +IF and +ADS models lose stability of synaptic weight distribution in the case of silenced neuronal activity, but the +IF model needs much longer duration than the +ADS model according to the numerical simulations (see Figure 5).

3.2.2. Under Correlated Activity

The pre-synaptic firings have a strong impact on post-synaptic firing. Events at strong synapses or correlated synapses have greater chances to induce post-synaptic firing and potentiation. Next, the dependency of post-synaptic firing on pre-synaptic firings is taken into account. The excitatory synapses are divided into the correlated groups labeled l, each containing N_{c, l} excitatory synapses. A set of m_l excitatory synapses in a correlated group l is simultaneously stimulated. A pre-synaptic firing with a weight W in a correlated group l has a chance $P_l^{fire}(W)$ to induce post-synaptic firing. Each group l shares the post-synaptic firing rate f_post, but has its own synaptic weight distribution P_l(W). Equation (14) can be rewritten as

$\begin{array}{cc}\begin{array}{l}{{M}}_{1,l}(W)={{M}}_{1,l}^{\text{STDP}}(W)+{{M}}_{1,l}^{\text{H}}(W)+{{M}}_{1,l}^{\text{Dep}}(W),\\ {{M}}_{2,l}(W)={{M}}_{2,l}^{\text{STDP}}(W)+{{M}}_{2,l}^{\text{H}}(W)+{{M}}_{2,l}^{\text{Dep}}(W),\end{array}& \left(17\right)\end{array}$

where the terms ${{M}}_{n,l}^{Dep}(W)$ are the dependency terms. Under the simplified condition that the post-synaptic neuron fires immediately after the pre-synaptic firing, i.e., Δt < 0 and |Δt| ≪ 1, the dependency terms ${{M}}_{n,l}^{Dep}(W)$ are expressed as

$\begin{array}{cc}\begin{array}{l}{{M}}_{1,l}^{\text{Dep}}(W)=f_{\text{pre}}{P}_l^{fire}(W)\text{E}[A_{+}(W)]=f_{\text{pre}}{P}_l^{fire}(W)c_{+},\\ {{M}}_{2,l}^{\text{Dep}}(W)=f_{\text{pre}}{P}_l^{fire}(W)\text{E}[A_{+}{(W)}^2]=f_{\text{pre}}{P}_l^{fire}(W)\\ (c_{+}^2+{\sigma }_p^2{W}^2).\end{array}& \left(18\right)\end{array}$

The firing chance $P_l^{fire}(W)$ depends on the probability distribution P(v) over time of the membrane potential v. A detailed derivation of the firing chance $P_l^{fire}(W)$ is shown in Appendix. Unlike the STDP terms ${{M}}_{n,l}^{STDP}(W)$, the dependency terms ${{M}}_{n,l}^{Dep}(W)$ are proportional to the pre-synaptic firing rate f_pre, but not to the post-synaptic firing rate f_post. Thus, the ratio between the first moment ${M}$₁(W) and the second moment ${M}$₂(W) depends on the post-synaptic firing rate f_post. The post-synaptic firing rate f_post is assumed to be

$\begin{array}{cc}{f}_{\text{post}}=f_{\text{pre}}{\displaystyle \sum _l\frac{N_{c,l}}{m_l}}\text{E}[P_l^{fire}(W)].& \left(19\right)\end{array}$

This value is used for the STDP terms ${{M}}_{n,l}^{STDP}(W)$. Figure 7B shows the theoretical synaptic weight distributions P(W) taking into account the dependency terms ${{M}}_n^{Dep}(W)$. The colored lines denote the theoretical results and the gray lines denote the corresponding numerical results shown in Figure 3A. By taking into account the dependency terms ${{M}}_n^{Dep}(W)$, the theoretical results closely resemble the numerical results.

In the cases of the +NO and +LP models, the synapses are potentiated with a low pre-synaptic firing rate f_pre and conversely the synapses are depressed with a high pre-synaptic firing rate f_pre. The relationships between the STDP terms ${{M}}_{n,l}^{STDP}(W)$ and the dependency terms ${{M}}_{n,l}^{Dep}(W)$ of the correlated group l are

$\begin{array}{cc}\begin{array}{l}\frac{{{M}}_{n,l}^{\text{STDP}}(W)}{{{M}}_{n,l}^{\text{Dep}}(W)}=\frac{\frac{1}{n}{f}_{\text{pre}}{f}_{\text{post}}\left({\tau }_{+}\text{E}[A_{+}{(W)}^n]+{\tau }_{-}\text{E}[A_{-}{(W)}^n]\right)}{f_{\text{pre}}{P}_l^{fire}(W)\text{E}[A_{+}{(W)}^n]}\hfill \\ =\frac{1}{n}\left({\tau }_{+}-{\tau }_{-}\frac{\text{E}[A_{-}{(W)}^n]}{\text{E}[A_{+}{(W)}^n]}\right)\frac{f_{\text{post}}}{P_l^{fire}(W)}\hfill \\ =\frac{1}{n}\left({\tau }_{+}-{\tau }_{-}\frac{\text{E}[A_{-}{(W)}^n]}{\text{E}[A_{+}{(W)}^n]}\right)\hfill \\ \frac{f_{\text{pre}}{\displaystyle {\sum }_k\frac{N_{c,k}}{m_k}}\text{E}[P_k^{fire}(W)]}{P_l^{fire}(W)}.\hfill \end{array}& \left(20\right)\end{array}$

At a low pre-synaptic firing rate f_pre, the dependency terms ${{M}}_{n,l}^{Dep}(W)$ gain influence and potentiate the synapses. At a high pre-synaptic firing rate f_pre, the dependency terms ${{M}}_{n,l}^{Dep}(W)$ are relatively negligible and the synaptic weight distribution P(W) resembles that when the dependency terms ${{M}}_n^{Dep}(W)$ are ignored. In other words, a frequent pre-synaptic firing is more likely to induce a depression of other synapses. This is the mechanism by which the soft-bounded STDP model achieves the synaptic scaling. However, synapses potentiated by the dependency terms ${{M}}_n^{Dep}(W)$ lead to increase in post-synaptic firing rate f_post, thereby reducing the influence of the dependency terms ${{M}}_n^{Dep}(W)$; thus, the soft-bounded STDP model cannot increase post-synaptic firing rate f_post sufficiently. This is one reason why the +NO and +LP models cannot prevent excessively low post-synaptic firing rates f_post as confirmed in Section 2.1. In contrast, intrinsic fluctuations regulate the synaptic weights W depending not on the dependency terms ${{M}}_n^{Dep}(W)$ but on the homeostatic terms ${{M}}_n^H(W)$. The homeostatic terms ${{M}}_n^H(W)$ with large values of parameters S and s can keep their influence after the post-synaptic firing rate f_post becomes larger; thus, the +IF model prevents the excessively low post-synaptic firing rate f_post. In the +ADS model, when the post-synaptic firing rate f_post departs from the target a_g, the homeostatic terms ${{M}}_n^H(W)$ gain influence and regulate the synaptic weights W to move f_post closer to a_g.

The result theoretically-obtained from the +LP model with a high pre-synaptic firing rate f_pre = 20.0 is not consistent with the numerical result shown in Section 3.1.2 because of a factor not taken into account in the theoretical analysis. When an excitatory pre-synaptic firing with a strong weight is transmitted to the post-synaptic neuron, the membrane potential v gradually approaches the threshold voltage v_th. If another excitatory pre-synaptic firing, which is not correlated to the former, is transmitted before the membrane potential v reaches the threshold voltage v_th, it hastens the approach. Consequently, the synapses are more likely to be potentiated. This type of correlation between excitatory pre-synaptic firings gains influence when many synapses become strong and have high firing chances P^fire(W); however, it is ignored in this theoretical analysis. For this reason, the +LP model leads to excessively high synaptic weights W. A more detailed analysis is outside the scope of this study.

3.2.3. Competition

The synaptic weight distributions P_l(W) of correlated groups l with different correlations c_l are also obtained from Equation (17). As is the case with Section 3.1.5, two groups are chosen and labeled A and B. Figure 7C shows the theoretical synaptic weight distributions P_A(W) and P_B(W) of the correlated groups A and B. According to Equation (20), the dependency terms ${{M}}_{n,l}^{Dep}(W)$ of the correlated group l lose their influence and the synapses are depressed when another correlated group k increases its firing chance $E\left[P_k^{fire}(W)\right]$. When the correlations are c_A = c_B = 0.04, the theoretical synaptic weight distributions P_A(W) and P_B(W) are the same. After the correlation for the correlated group A is changed to c_A = 0.16, the firing chance $P_A^{fire}(W)$ is increased, and the synapses in the correlated group A are potentiated. The responses of the correlated group B depend on the model. In the cases of the +NO and +LP models, the STDP terms ${{M}}_{n,B}^{STDP}(W)$ become dominant because of the increased post-synaptic firing rate f_post; the dependency terms ${{M}}_{n,B}^{Dep}(W)$ lose their influence. The correlated group B loses a chance to induce a post-synaptic firing and to be potentiated. In the case of the +IF model, the STDP terms ${{M}}_{n,B}^{STDP}(W)$ become dominant, and the homeostatic terms ${{M}}_{n,B}^H(W)$ lose their influence. In addition to loss of a chance to be potentiated, the correlated group B is scaled down by soft-bounded STDP model because of increase in post-synaptic firing rate f_post. In the case of the +ADS model, the activity-dependent scaling model reacts against the increasing post-synaptic firing rate f_post and the homeostatic terms ${{M}}_{n,B}^H(W)$ gain influence. Consequently, the synapses in the correlated group B are depressed. The three models exhibit competition, but their mechanisms differ. If a pre-synaptic firing rate f_pre is high, the dependency terms ${{M}}_{n,B}^{Dep}(W)$ have lost their influence, and the synaptic weight distribution P_B(W) never changes in response to an increase in post-synaptic firing rate f_post. This is why the +NO model with the pre-synaptic firing rate f_pre = 20 Hz did not exhibit competition in the previous studies (van Rossum et al., 2000; Gilson and Fukai, 2011). However, the +LP model does not exhibit competition in the numerical simulation as shown in Section 3.1.5 for the same reason as that for the excessive potentiation induced of the +LP model shown in the previous section.

4. Discussion

4.1. Stability of Neuronal Activity

Homeostatic plasticity called synaptic scaling is expected to maintain neuronal activity within a functional range by scaling up and down of synapses (Bienenstock et al., 1982; Turrigiano et al., 1998; Turrigiano, 1999, 2008; Burrone et al., 2002; Ibata et al., 2008; Keck et al., 2013; Vitureira and Goda, 2013; Zenke et al., 2013; Félix-Oliveira et al., 2014; Toyoizumi et al., 2014). Without homeostatic plasticity, the soft-bounded STDP model leaded to a post-synaptic firing rate of excessively low or high (see the left two panels of Figure 2). The soft-bounded STDP model prevents excessive change in synaptic weight; strong synapse is prone to be depressed and weak synapse is prone to be potentiated. Synaptic weights depart from upper and lower bounds and get closer to the equilibrium at which potentiation and depression are balanced with each other. The equilibrium depends on neuronal activity and the soft-bounded STDP model apparently achieves synaptic scaling by itself (see Figure 3A). However, with very low or very high pre-synaptic firing rate, synaptic weights at around the equilibrium lead to excessively high or low post-synaptic neuronal activity. The soft-bounded STDP model achieves at least stability of synaptic weight distribution but does not achieve stability of neuronal activity.

The soft-bounded STDP model incorporating intrinsic fluctuations (+IF model) maintained a post-synaptic firing rate within a certain range over a broad range of pre-synaptic firing rate. According to theoretical analyses, the contribution of intrinsic fluctuations is preventing excessively low neuronal activity by scaling up of synapses. Since the occurrence probability of STDP is roughly proportional to the product of the pre-synaptic and post-synaptic firing rates (van Rossum et al., 2000; Gilson and Fukai, 2011), with decreasing neuronal activity, the soft-bounded STDP model loses its influence and intrinsic fluctuations are relatively dominant; strong synapses are not depressed by the soft-bounded STDP model but rather diffused randomly by intrinsic fluctuations. Therefore, strong synapses stay strong and excessively low neuronal activity is prevented. This is the mechanism for scaling up achieved by intrinsic fluctuations. With increasing neuronal activity, the soft-bounded STDP model gains its influence and intrinsic fluctuations become negligible; strong synapses are depressed and thereby excessively high neuronal activity is prevented. This is the mechanism for scaling down. The soft-bounded STDP model also has the noise term ${\nu }_p~{N}(0,{\sigma }_p^2)$. It is almost negligible when compared with intrinsic fluctuations (see Figures 2, 3A) and also loses its influence with decreasing neuronal activity; the noise term ν_p does not contribute to stability of neuronal activity. Therefore, incorporation of intrinsic fluctuations and soft-bounded STDP model achieves scaling up and down of synapses and neuronal activity converges to the equilibrium at which activity-independent scaling up and activity-dependent scaling down are balanced with each other.

Intrinsic fluctuations contribute to competition between synapses but its mechanism differs from the others. When a group of excitatory synapses has an effect, another group reduces correlated activity and loses a chance to being potentiated. This is the mechanism of soft-bounded STDP model for synaptic competition (Song et al., 2000; van Rossum et al., 2000; Morrison et al., 2007). The activity-dependent scaling model reacts against increase in neuronal activity and depresses synapses as is the case with classical negative feedback processes (Bienenstock et al., 1982; Bear et al., 1987; Miller, 1996). On the other hand, intrinsic fluctuations do not react against increase in neuronal activity. Nonetheless increase in neuronal activity promotes soft-bounded STDP model, leading to scaling down of strong synapses which have been potentiated by intrinsic fluctuations. As a result, incorporation of intrinsic fluctuations and soft-bounded STDP model contribute to competition between synapses.

In the case of silenced neuronal activity, the soft-bounded STDP model cannot regulate synapses. Intrinsic fluctuations potentiate synapses slowly because soft-bounded Hebbian plasticity, which normally depresses strong synapses, is blocked. This result is consistent with homeostatic plasticity observed in biological experiments (Turrigiano et al., 1998; Ibata et al., 2008). After sufficiently long duration, the synaptic weight distribution converges to a power-law distribution with no well-defined mean. Nonetheless synaptic weights are kept within plausible values for several days. The activity-dependent scaling model potentiates synapses drastically, leading implausibly strong synapses within an hour. This is because neuronal activity falls below the target and never returns to the target, no matter how much activity-dependent scaling model potentiates synapses. Although both intrinsic fluctuations and the activity-dependent scaling model lose stability of synaptic weight distribution in the case of silenced neuronal activity, intrinsic fluctuations lead to more biologically plausible results. Intrinsic fluctuations derive a power-law distribution with an exponent equal to the degree in synaptic weight of their expression. When expression of intrinsic fluctuations has a term having degree greater than 2 in synaptic weight derives a power-law distribution with a well-defined mean. Examining such intrinsic fluctuations is a future work.

4.2. Stability of Synapses

Even when the synaptic weight distribution is stable, it does not imply that the individual synapse is stable. Memory is considered to be stored as synapses connections, stability of synapse contributes to retention of memory. Intrinsic fluctuations however have a risk for disturbing memory retention since they are random fluctuations. This study has shown that intrinsic fluctuations have a slightly negative influence on the stability of strong synapses but it is almost negligible as long as the neuronal activity is not too small. Synapses are shuffled randomly not only by intrinsic fluctuations but also by STDP (see the lower part of Figure 1). A pre-synaptic firing has a chance to induce a post-synaptic firing leading to potentiation of the synapse and depression of another synapse. Frequent pre- and post-synaptic firings induce frequent occurrence of STDP, leading to shuffling of synapses. The influence of intrinsic fluctuations on stability of synapses is much smaller than that of STDP. A more detailed comparison is a future work.

Even after neuronal activity is silenced, intrinsic fluctuations shuffle synapses in contrast to the other models. Their negative influence on the stability of the strong synapses is no longer negligible when the neuronal activity is silenced. This result however does not imply instability of memory; intrinsic fluctuations adapt synapses to the environment that has changed.

4.3. Mechanism Underlying Intrinsic Fluctuations

Several molecules and proteins have been considered to be associated with scaling up and down (Turrigiano, 2008). If intrinsic fluctuations are the mechanism underlying scaling up of synapses, they can be mediated by these factors. Another possible hypothesis is that these factors are associated with STDP but not with intrinsic fluctuations, and intrinsic fluctuations are just a noise in post-synaptic structure. Scaling up is not necessarily achieved by promotion of intrinsic fluctuations; it can also be achieved by disturbance of soft-bounded STDP (i.e., scaling down) as shown in Figure 5. In either case, examining effects of these factors on intrinsic fluctuations is a possible future work.

4.4. Conclusion

In summary, incorporation of intrinsic fluctuations and soft-bounded STDP model achieves scaling up and down of synapses and it is one plausible candidate mechanism for homeostatic plasticity called the synaptic scaling.

Author Contributions

TM designed research; TM and KU performed research and analyzed data; and TM wrote the paper.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

This work was partially funded by the KAKENHI (16K12487 and 26280040) and Kayamori Foundation of Information Science Advancement.

Footnotes

1. ^Another type of homeostatic plasticity, namely intrinsic plasticity, is also confirmed but is outside the scope of this paper (Desai et al., 1999; Turrigiano and Nelson, 2000; Abraham, 2008; Watt and Desai, 2010).

References

Abraham, W. C. (2008). Metaplasticity: tuning synapses and networks for plasticity. Nat. Rev. Neurosci. 9, 387. doi: 10.1038/nrn2356

PubMed Abstract | CrossRef Full Text | Google Scholar

Bair, W., Zohary, E., and Newsome, W. T. (2001). Correlated firing in macaque visual area MT: time scales and relationship to behavior. J. Neurosci. 21, 1676–1697.

PubMed Abstract | Google Scholar

Bear, M. F., Cooper, L. N., and Ebner, F. F. (1987). A physiological basis for a theory of synapse modification. Science 237, 42–48. doi: 10.1126/science.3037696

PubMed Abstract | CrossRef Full Text | Google Scholar

Bi, G.-Q., and Poo, M.-M. (1998). Synaptic modifications in cultured hippocampal neurons: dependence on spike timing, synaptic strength, and postsynaptic cell type. J. Neurosci. 18, 10464–10472.

PubMed Abstract | Google Scholar

Bienenstock, E. L., Cooper, L. N., and Munro, P. W. (1982). Theory for the development of neuron selectivity: orientation specificity and binocular interaction in visual cortex. J. Neurosci. 2, 32–48.

PubMed Abstract | Google Scholar

Bliss, T. V. P., and Lømo, T. (1973). Long-lasting potentiation of synaptic transmission in the dentate area of the anaesthetized rabbit following stimulation of the perforant path. J. Physiol. 232, 331–356. doi: 10.1113/jphysiol.1973.sp010273

PubMed Abstract | CrossRef Full Text | Google Scholar

Burrone, J., O'Byrne, M., and Murthy, V. N. (2002). Multiple forms of synaptic plasticity triggered by selective suppression of activity in individual neurons. Nature 420, 414–418. doi: 10.1038/nature01242

PubMed Abstract | CrossRef Full Text | Google Scholar

Buzsáki, G., and Mizuseki, K. (2014). The log-dynamic brain: how skewed distributions affect network operations. Nat. Rev. Neurosci. 15, 264–278. doi: 10.1038/nrn3687

PubMed Abstract | CrossRef Full Text | Google Scholar

Desai, N. S., Rutherford, L. C., and Turrigiano, G. G. (1999). Plasticity in the intrinsic excitability of cortical pyramidal neurons. Nat. Neurosci. 2, 515–520. doi: 10.1038/9165

PubMed Abstract | CrossRef Full Text | Google Scholar

Effenberger, F., Jost, J., and Levina, A. (2015). Self-organization in balanced state networks by STDP and homeostatic plasticity. PLoS Comput. Biol. 11:e1004420. doi: 10.1371/journal.pcbi.1004420

PubMed Abstract | CrossRef Full Text | Google Scholar

Félix-Oliveira, A., Dias, R. B., Colino-Oliveira, M., Rombo, D. M., and Sebastião, A. M. (2014). Homeostatic plasticity induced by brief activity deprivation enhances long-term potentiation in the mature rat hippocampus. J. Neurophysiol. 112, 3012–3022. doi: 10.1152/jn.00058.2014

PubMed Abstract | CrossRef Full Text | Google Scholar

Fusi, S., and Abbott, L. F. (2007). Limits on the memory storage capacity of bounded synapses. Nat. Neurosci. 10, 485–493. doi: 10.1038/nn1859

PubMed Abstract | CrossRef Full Text | Google Scholar

Fusi, S., Drew, P. J., and Abbott, L. F. (2005). Cascade models of synaptically stored memories. Neuron 45, 599–611. doi: 10.1016/j.neuron.2005.02.001

PubMed Abstract | CrossRef Full Text | Google Scholar

Gilson, M., and Fukai, T. (2011). Stability versus neuronal specialization for STDP: long-tail weight distributions solve the dilemma. PLoS ONE 6:e25339. doi: 10.1371/journal.pone.0025339

PubMed Abstract | CrossRef Full Text | Google Scholar

Gütig, R., Aharonov, R., Rotter, S., and Sompolinsky, H. (2003). Learning input correlations through nonlinear temporally asymmetric Hebbian plasticity. J. Neurosci. 23, 3697–3714.

PubMed Abstract | Google Scholar

Ibata, K., Sun, Q., and Turrigiano, G. G. (2008). Rapid synaptic scaling induced by changes in postsynaptic firing. Neuron 57, 819–826. doi: 10.1016/j.neuron.2008.02.031

PubMed Abstract | CrossRef Full Text | Google Scholar

Izhikevich, E. M. (2004). Which model to use for cortical spiking neurons? IEEE Trans. Neural Netw. 15, 1063–1070. doi: 10.1109/TNN.2004.832719

PubMed Abstract | CrossRef Full Text | Google Scholar

Kasai, H., Fukuda, M., Watanabe, S., Hayashi-Takagi, A., and Noguchi, J. (2010). Structural dynamics of dendritic spines in memory and cognition. Trends Neurosci. 33, 121–129. doi: 10.1016/j.tins.2010.01.001

PubMed Abstract | CrossRef Full Text | Google Scholar

Kasai, H., Matsuzaki, M., Noguchi, J., Yasumatsu, N., and Nakahara, H. (2003). Structure-stability-function relationships of dendritic spines. Trends Neurosci. 26, 360–368. doi: 10.1016/S0166-2236(03)00162-0

PubMed Abstract | CrossRef Full Text | Google Scholar

Katz, L. C., and Shatz, C. J. (1996). Synaptic activity and the construction of cortical circuits. Science 274, 1133–1138. doi: 10.1126/science.274.5290.1133

PubMed Abstract | CrossRef Full Text | Google Scholar

Keck, T., Keller, G. B., Jacobsen, R. I., Eysel, U. T., Bonhoeffer, T., and Hübener, M. (2013). Synaptic scaling and homeostatic plasticity in the mouse visual cortex in vivo. Neuron 80, 327–334. doi: 10.1016/j.neuron.2013.08.018

PubMed Abstract | CrossRef Full Text | Google Scholar

Kopec, C. D., Li, B., Wei, W., Boehm, J., and Malinow, R. (2006). Glutamate receptor exocytosis and spine enlargement during chemically induced long-term potentiation. J. Neurosci. 26, 2000–2009. doi: 10.1523/JNEUROSCI.3918-05.2006

PubMed Abstract | CrossRef Full Text | Google Scholar

Loewenstein, Y., Kuras, A., and Rumpel, S. (2011). Multiplicative dynamics underlie the emergence of the log-normal distribution of spine sizes in the neocortex in vivo. J. Neurosci. 31, 9481–9488. doi: 10.1523/JNEUROSCI.6130-10.2011

PubMed Abstract | CrossRef Full Text | Google Scholar

Löwel, S., and Singer, W. (1992). Selection of intrinsic horizontal connections in the visual cortex by correlated neuronal activity. Science 255, 209–212. doi: 10.1126/science.1372754

PubMed Abstract | CrossRef Full Text | Google Scholar

Markram, H., Lübke, J., Frotscher, M., and Sakmann, B. (1997). Regulation of synaptic efficacy by coincidence of postsynaptic APs and EPSPs. Science 275, 213–215. doi: 10.1126/science.275.5297.213

PubMed Abstract | CrossRef Full Text | Google Scholar

Martin, S. J., Grimwood, P. D., and Morris, R. G. M. (2000). Synaptic plasticity and memory: an evaluation of the hypothesis. Ann. Rev. Neurosci. 23, 649–711. doi: 10.1146/annurev.neuro.23.1.649

PubMed Abstract | CrossRef Full Text | Google Scholar

Matsuzaki, M., Ellis-Davies, G. C. R., Nemoto, T., Miyashita, Y., Iino, M., and Kasai, H. (2001). Dendritic spine geometry is critical for AMPA receptor expression in hippocampal CA1 pyramidal neurons. Nat. Neurosci. 4, 1086–1092. doi: 10.1038/nn736

CrossRef Full Text | Google Scholar

Matsuzaki, M., Honkura, N., Ellis-Davies, G. C. R., and Kasai, H. (2004). Structural basis of long-term potentiation in single dendritic spines. Nature 429, 761–766. doi: 10.1038/nature02617

PubMed Abstract | CrossRef Full Text | Google Scholar

Miller, K. D. (1996). Synaptic economics: competition and cooperation in synaptic plasticity. Neuron 17, 371–374. doi: 10.1016/S0896-6273(00)80169-5

PubMed Abstract | CrossRef Full Text | Google Scholar

Miller, K. D., and MacKay, D. J. C. (1994). The role of constraints in Hebbian learning. Neural Comput. 6, 100–126. doi: 10.1162/neco.1994.6.1.100

CrossRef Full Text | Google Scholar

Mizuseki, K., and Buzsáki, G. (2013). Preconfigured, skewed distribution of firing rates in the hippocampus and entorhinal cortex. Cell Rep. 4, 1010–1021. doi: 10.1016/j.celrep.2013.07.039

PubMed Abstract | CrossRef Full Text | Google Scholar

Morrison, A., Aertsen, A., and Diesmann, M. (2007). Spike-timing-dependent plasticity in balanced random networks. Neural Comput. 19, 1437–1467. doi: 10.1162/neco.2007.19.6.1437

CrossRef Full Text | Google Scholar

Morrison, A., Diesmann, M., and Gerstner, W. (2008). Phenomenological models of synaptic plasticity based on spike timing. Biol. Cybern. 98, 459–478. doi: 10.1007/s00422-008-0233-1

PubMed Abstract | CrossRef Full Text | Google Scholar

Nabavi, S., Fox, R., Proulx, C. D., Lin, J. Y., Tsien, R. Y., and Malinow, R. (2014). Engineering a memory with LTD and LTP. Nature 511, 348–352. doi: 10.1038/nature13294

CrossRef Full Text | Google Scholar

Oja, E. (1982). A simplified neuron model as a principal component analyzer. J. Math. Biol. 15, 267–273. doi: 10.1007/BF00275687

CrossRef Full Text | Google Scholar

Øksendal, B. (2003). Stochastic Differential Equations: An Introduction with Applications. Berlin; Heidelberg: Springer-Verlag Berlin Heidelberg. doi: 10.1007/978-3-642-14394-6

CrossRef Full Text | Google Scholar

Rochester, N., Holland, J., Haibt, L., and Duda, W. (1956). Tests on a cell assembly theory of the action of the brain, using a large digital computer. IRE Trans. Inform. Theory 2, 80–93. doi: 10.1109/TIT.1956.1056810

CrossRef Full Text | Google Scholar

Rutherford, L. C., Nelson, S. B., and Turrigiano, G. G. (1998). BDNF has opposite effects on the quantal amplitude of pyramidal neuron and interneuron excitatory synapses. Neuron 21, 521–530. doi: 10.1016/S0896-6273(00)80563-2

PubMed Abstract | CrossRef Full Text | Google Scholar

Sjöström, P. J., Turrigiano, G. G., and Nelson, S. B. (2001). Rate, timing, and cooperativity jointly determine cortical synaptic plasticity. Neuron 32, 1149–1164. doi: 10.1016/S0896-6273(01)00542-6

PubMed Abstract | CrossRef Full Text | Google Scholar

Song, S., Miller, K. D., and Abbott, L. F. (2000). Competitive Hebbian learning through spike-timing-dependent synaptic plasticity. Nat. Neurosci. 3, 919–926. doi: 10.1038/78829

PubMed Abstract | CrossRef Full Text | Google Scholar

Statman, A., Kaufman, M., Minerbi, A., Ziv, N. E., and Brenner, N. (2014). Synaptic size dynamics as an effectively stochastic process. PLoS Comput. Biol. 10:e1003846. doi: 10.1371/journal.pcbi.1003846

PubMed Abstract | CrossRef Full Text | Google Scholar

Toyoizumi, T., Kaneko, M., Stryker, M. P., and Miller, K. D. (2014). Modeling the dynamic interaction of Hebbian and homeostatic plasticity. Neuron 84, 497–510. doi: 10.1016/j.neuron.2014.09.036

PubMed Abstract | CrossRef Full Text | Google Scholar

Turrigiano, G. G. (1999). Homeostatic plasticity in neuronal networks: the more things change, the more they stay the same. Trends Neurosci. 22, 221–227. doi: 10.1016/S0166-2236(98)01341-1

CrossRef Full Text | Google Scholar

Turrigiano, G. G. (2008). The self-tuning neuron: synaptic scaling of excitatory synapses. Cell 135, 422–435. doi: 10.1016/j.cell.2008.10.008

PubMed Abstract | CrossRef Full Text | Google Scholar

Turrigiano, G. G., Leslie, K. R., Desai, N. S., Rutherford, L. C., and Nelson, S. B. (1998). Activity-dependent scaling of quantal amplitude in neocortical neurons. Nature 391, 892–896. doi: 10.1038/36103

PubMed Abstract | CrossRef Full Text | Google Scholar

Turrigiano, G. G., and Nelson, S. B. (2000). Hebb and homeostasis in neuronal plasticity. Curr. Opin. Neurobiol. 10, 358–364. doi: 10.1016/S0959-4388(00)00091-X

PubMed Abstract | CrossRef Full Text | Google Scholar

van Rossum, M. C. W., Bi, G.-Q., and Turrigiano, G. G. (2000). Stable Hebbian learning from spike timing-dependent plasticity. J. Neurosci. 20, 8812–8821.

PubMed Abstract | Google Scholar

van Rossum, M. C. W., Shippi, M., and Barrett, A. B. (2012). Soft-bound synaptic plasticity increases storage capacity. PLoS Comput. Biol. 8:e1002836. doi: 10.1371/journal.pcbi.1002836

PubMed Abstract | CrossRef Full Text | Google Scholar

Vitureira, N., and Goda, Y. (2013). The interplay between hebbian and homeostatic synaptic plasticity. J. Cell Biol. 203, 175–186. doi: 10.1083/jcb.201306030

PubMed Abstract | CrossRef Full Text | Google Scholar

von der Malsburg, C. (1973). Self-organization of orientation selective cells in the striate cortex. Kybernetik 14, 85–100. doi: 10.1007/BF00288907

CrossRef Full Text | Google Scholar

Wang, X.-B., Yang, Y., and Zhou, Q. (2007). Independent expression of synaptic and morphological plasticity associated with long-term depression. J. Neurosci. 27, 12419–12429. doi: 10.1523/JNEUROSCI.2015-07.2007

PubMed Abstract | CrossRef Full Text | Google Scholar

Watt, A. J., and Desai, N. S. (2010). Homeostatic plasticity and STDP: keeping a neuron's cool in a fluctuating world. Front. Synaptic Neurosci. 2:5. doi: 10.3389/fnsyn.2010.00005

PubMed Abstract | CrossRef Full Text | Google Scholar

Whitlock, J. R., Heynen, A. J., Shuler, M. G., and Bear, M. F. (2006). Learning induces long-term potentiation in the hippocampus. Science 313, 1093–1097. doi: 10.1126/science.1128134

PubMed Abstract | CrossRef Full Text | Google Scholar

Yasumatsu, N., Matsuzaki, M., Miyazaki, T., Noguchi, J., and Kasai, H. (2008). Principles of long-term dynamics of dendritic spines. J. Neurosci. 28, 13592–13608. doi: 10.1523/JNEUROSCI.0603-08.2008

CrossRef Full Text | Google Scholar

Yger, P., and Gilson, M. (2015). Models of metaplasticity: a review of concepts. Front. Comput. Neurosci. 9:138. doi: 10.3389/fncom.2015.00138

PubMed Abstract | CrossRef Full Text | Google Scholar

Yuste, R. (2010). Dendritic Spines. Cambridge, MA; London: The MIT Press. doi: 10.7551/mitpress/9780262013505.001.0001

CrossRef Full Text | Google Scholar

Zenke, F., Hennequin, G., and Gerstner, W. (2013). Synaptic plasticity in neural networks needs homeostasis with a fast rate detector. PLoS Comput. Biol. 9:e1003330. doi: 10.1371/journal.pcbi.1003330

PubMed Abstract | CrossRef Full Text | Google Scholar

Appendix

In this section, the synaptic weight distribution P(W) is derived taking into account the dependency of post-synaptic firing on pre-synaptic firings. The time courses of the membrane potential v and the conductance g_E are as follows:

$\begin{array}{cc}\frac{d}{dt}\left(\begin{array}{c}v(t)\\ g_E(t)\end{array}\right)=\left(\begin{array}{cc}-1/{\tau }_m& -(v(t)-v_E)R/{\tau }_m\\ 0& -1/{\tau }_g\end{array}\right)\left(\begin{array}{c}v(t)-v_L\\ g_E(t)\end{array}\right).& \left(\text{A}1\right)\end{array}$

A pre-synaptic firing with weight W at t = 0 immediately increases the conductance g_E by W. In this section, simplified condition is introduced in which the rise time and the rise amount of the membrane potential v in response to a pre-synaptic firing are assumed to be sufficiently small; thus, the decay of the membrane potential v from the initial value v(0) is ignored. Under this condition, the terms v(t) − v_E and v(t) − v_L are approximated by v(0) − v_E and v(t) − v(0), and Equation (A1) is solved as

$\begin{array}{cc}v(t)-v_0=\frac{g_0(v_E-v_0)R(e^{-t/{\tau }_m}-e^{-t/{\tau }_g})}{{\tau }_m/{\tau }_g-1},& \left(\text{A}2\right)\end{array}$

where v₀ = v(0) and g₀ = g(0) = W. The membrane potential v reaches the maximum value at time

$\begin{array}{cc}{t}_{\text{max}}=\frac{\log (\frac{{\tau }_g}{{\tau }_m})}{{\tau }_m^{-1}-{\tau }_g^{-1}}.& \left(\text{A}3\right)\end{array}$

The excitatory post-synaptic potential (EPSP) Δv_E, which is the change in the membrane potential v, is

$\begin{array}{cc}\Delta v_E(W,v_0):=v(t_{\text{max}})-v_0.& \left(\text{A}4\right)\end{array}$

The inhibitory post-synaptic potential (IPSP) Δv_I, which is the change in the membrane potential v induced by an inhibitory pre-synaptic firing, is obtained in the same way. Immediately after the membrane potential v exceeds the threshold voltage v_th, the post-synaptic neuron fires and potentiation is induced. The temporal difference Δt between the pre-synaptic firing t_pre and the post-synaptic firing t_post is assumed to be approximately zero because of the simplified condition of the sufficiently small rise time. According to Equation (A2), a set of m_l synapses with weights W = {W₀, …, W_ml−1} stimulated at one time induces an EPSP of

$\begin{array}{cc}\Delta v_E(W,v)={\displaystyle \sum _{i=0}^{m_l-1}\Delta }{v}_E(W_i,v),& \left(\text{A}5\right)\end{array}$

because of the linear approximation. The chance $P_l^{fire}(\text{W})$ that the correlated pre-synaptic firing induces a post-synaptic firing is

$\begin{array}{cc}{P}_l^{fire}(W)={\displaystyle {\int }_{v_{min}(W)}^{v_{th}}P}(v)dv,& \left(\text{A}6\right)\end{array}$

where v_min(W) is the minimal value of the membrane potential at which a post-synaptic neuron fires in response to the correlated pre-synaptic firing, i.e., v_min = min{v|v + Δv_E(W, v) ≥ v_th}; P(v) is the probability distribution over time of the membrane potential v. The chance $P_l^{fire}(W)$ that an event at a synapse with weight W induces a post-synaptic firing is as follows:

$\begin{array}{cc}\begin{array}{l}{P}_l^{fire}(W)=\text{}\text{E}[P_l^{fire}(W)|W_0=W]\hfill \\ ={\displaystyle \int \cdots }{\displaystyle \int P}(W_1)\cdots P(W_{m-1})P_l^{fire}(W)d{W}_1\cdots d{W}_{m-1}\hfill \end{array}& \left(\text{A}7\right)\end{array}$

The probability distribution P(v) over time of the membrane potential is also derived by the Fokker-Planck equation. However, the membrane potential v is reset from v_th to v_r. Taking into account this discontinuous phenomenon, the Fokker-Planck equation is as follows:

$\begin{array}{cc}\frac{\partial }{\partial t}P(v,t)=-\frac{\partial }{\partial v}{{M}}_1(v)P(v,t)+\frac{1}{2}\frac{{\partial }^2}{\partial v^2}{{M}}_2(v)P(v,t)+I(v),& \left(\text{A}8\right)\end{array}$

under the constraints that

$\begin{array}{cc}P(v,t)=0\text{ for }v\ge v_{th},{\displaystyle {\int }_{-\infty }^{v_{th}}P}(v,t)dv=1,I(v)=\delta (v-v_r)f_{\text{post}}.& \left(\text{A}9\right)\end{array}$

The n-th moment ${M}$_n(v) of the changes in the membrane potential v is

$\begin{array}{cc}\begin{array}{l}{{M}}_n(v)={\displaystyle \int P}(\Delta v)\Delta v^{n}d\Delta v\\ ={\displaystyle \sum _l\frac{f_{pre}{N}_{c,l}}{m_l}}{\displaystyle \int P}(W)\Delta v_E{(W,v)}^{n}dW\\ +f_{pre}{N}_I\Delta v_I{(W_I,v)}^n\\ ={\displaystyle \sum _l\frac{f_{pre}{N}_{c,l}}{m_l}}{\displaystyle \int \cdots }{\displaystyle \int P}(W_0)\cdots P(W_{m-1})\\ \Delta v_E{(\{W_0,\dots ,W_{m-1}\},v)}^{n}d{W}_0\cdots d{W}_{m-1}\\ +f_{pre}{N}_I\Delta v_I{(W_I,v)}^n.\end{array}& \left(\text{A}10\right)\end{array}$

Since the synaptic weight distribution P(W) and the probability distribution P(v) over time of the membrane potential v depend on each other, the Fokker-Planck equations in Equations (15) and (A8) are solved simultaneously.