ORIGINAL RESEARCH article
Modeling Working Memory in a Spiking Neuron Network Accompanied by Astrocytes
- 1Scientific and Educational Mathematical Center “Mathematics of Future Technology,” Lobachevsky State University of Nizhny Novgorod, Nizhny Novgorod, Russia
- 2Neuroscience and Cognitive Technology Laboratory, Center for Technologies in Robotics and Mechatronics Components, Innopolis University, Innopolis, Russia
- 3Center for Analysis of Complex Systems, Sechenov First Moscow State Medical University, Sechenov University, Moscow, Russia
- 4Institute for Women's Health and Department of Mathematics, University College London, London, United Kingdom
- 5Neuroscience Research Institute, Samara State Medical University, Samara, Russia
- 6Department of Mathematics, University of Leicester, Leicester, United Kingdom
We propose a novel biologically plausible computational model of working memory (WM) implemented by a spiking neuron network (SNN) interacting with a network of astrocytes. The SNN is modeled by synaptically coupled Izhikevich neurons with a non-specific architecture connection topology. Astrocytes generating calcium signals are connected by local gap junction diffusive couplings and interact with neurons via chemicals diffused in the extracellular space. Calcium elevations occur in response to the increased concentration of the neurotransmitter released by spiking neurons when a group of them fire coherently. In turn, gliotransmitters are released by activated astrocytes modulating the strength of the synaptic connections in the corresponding neuronal group. Input information is encoded as two-dimensional patterns of short applied current pulses stimulating neurons. The output is taken from frequencies of transient discharges of corresponding neurons. We show how a set of information patterns with quite significant overlapping areas can be uploaded into the neuron-astrocyte network and stored for several seconds. Information retrieval is organized by the application of a cue pattern representing one from the memory set distorted by noise. We found that successful retrieval with the level of the correlation between the recalled pattern and ideal pattern exceeding 90% is possible for the multi-item WM task. Having analyzed the dynamical mechanism of WM formation, we discovered that astrocytes operating at a time scale of a dozen of seconds can successfully store traces of neuronal activations corresponding to information patterns. In the retrieval stage, the astrocytic network selectively modulates synaptic connections in the SNN leading to successful recall. Information and dynamical characteristics of the proposed WM model agrees with classical concepts and other WM models.
In neuroscience, the understanding of the functional role of astrocytes in the central nervous system (CNS) is still open to debate (Savtchouk and Volterra, 2018), but now there is accumulating evidence demonstrating the involvement of astrocytes in local synaptic plasticity and the coordination of network activity (Durkee and Araque, 2019), and as a result in information processing and memory encoding (Santello et al., 2019). Astrocytes sense synaptic activity and respond to it with the transient elevation of intracellular Ca2+ concentration (lasting from hundreds of a millisecond to dozens of seconds). Such Ca2+ signals in astrocytes have been observed in different brain regions and also in the cortex, appearing there in response to mechanic sensory stimulation (Wang X. et al., 2006; Takata et al., 2011; Stobart et al., 2018) and visual sensory stimulation (Schummers et al., 2008; Chen et al., 2012; Perea et al., 2014). Ca2+ activation can trigger the release of gliotransmitters from astrocytes, which in turn affect the dynamics of presynaptic and postsynaptic terminals resulting in modulations of synaptic transmission (Araque et al., 2014). The gliotransmitter-mediated synaptic modulation lasts from a dozen seconds (Jourdain et al., 2007; Perea et al., 2014) to a dozen minutes (Stellwagen and Malenka, 2006; Perea and Araque, 2007; Navarrete et al., 2012) contributing to both short- and long-term synaptic plasticity.
Obviously, there is a qualitative coincidence between the time scales of astrocyte-mediated synaptic modulation and the working memory (WM) timings during decision making. Based on this and the other following facts of astrocytes participation in neuronal signaling, we hypothesized that astrocytes may be involved in WM formation. In particular, recent in vivo studies have shown the participation of astrocytes in the synchronization of certain cortical network activities (Takata et al., 2011; Chen et al., 2012; Paukert et al., 2014; Perea et al., 2014), cognitive functions, and behaviors (Poskanzer and Yuste, 2016; Sardinha et al., 2017). Experimental evidence shows that astrocyte pathology in the medium prefrontal cortex (PFC) impairs WM and learning functions (Lima et al., 2014), increasing astrocyte density enhances short-term memory performance (Luca et al., 2020), and recognition memory performance and disruption of WM depend on gliotransmitter release from astrocytes in the hippocampus (Han et al., 2012; Robin et al., 2018). Despite these numerous experimental insights of the contribution of astrocytes to synaptic modulations in neuronal signaling, the possible role of astrocytes in information processing and learning is still a subject of discussion (Kanakov et al., 2019; Kastanenka et al., 2019).
Considering the significance of WM processes and the challenge of finding alternative mechanisms and experimental evidence of the astrocytic role in information processing in CNS, it is interesting to study astrocyte-induced modulation of synaptic transmission in WM organization. Specifically, we assume that the potentiation of excitatory synapses induced by Ca2+ elevation-mediated glutamate release from astrocytes (Fellin et al., 2004; Perea and Araque, 2007; Navarrete and Araque, 2008, 2010; Chen et al., 2012) plays an essential role in WM. To test this hypothesis, we developed a novel neuron-astrocyte network model for visual WM to reflect experimental data on the structure, connectivity, and neurophysiology of the neuron-astrocytic interaction in underlying cortical tissue. We focused on the implementation of a multi-item WM task in a delayed matching to sample (DMS) framework representing a classical neuropsychological paradigm (Miller et al., 1996). During the experiment, a test animal was given a sample stimulus, which it had to remember for several seconds until it could begin executing a certain task. For example, a stimulus was followed by a sequence of several test stimuli and the animal was rewarded for indicating when one of the test stimuli. matched the original sample. The DMS paradigm was previously studied using a recurrent neural network (Brunel and Wang, 2001; Amit, 2003; Amit et al., 2013; Fiebig and Lansner, 2016). The novelty of our model is that we associate memory with item-specific patterns of astrocyte-induced enhancement of excitatory synaptic transmission. We present a new case of how the biologically relevant neuron-astrocyte network model implements loading, storage, and cued retrieval of multiple items with significant overlapping. The memory items are encoded in neuronal populations in the form of discrete high-frequency bursts rather than persistent spiking.
In this paper, we review some related works (section 2), describe the proposed model and methods in detail (section 3), present the results (section 4), and finally, we conclude this work in section 5.
2. Related Work
The concept of WM implies the ability to temporarily store and process information in goal-directed behavior. WM is crucial in the generation of higher cognitive functions for both humans and other animals (Baddeley, 1986, 2012; Conway et al., 2003). In primates, visual WM has been studied in delay tasks, such as DMS, which require a memory to be held during a brief delay period lasting for several seconds (Miller et al., 1996). Recordings in the monkeys' PFCs during the delay task showed that some neurons displayed persistent and stimulus-specific delay-period activity (Fuster and Alexander, 1971; Funahashi et al., 1989; Shafi et al., 2007; Barak et al., 2010; Funahashi, 2017). Delay persistent activity is considered the neural correlate of WM (Goldman-Rakic, 1995; Constantinidis et al., 2018).
The classical theoretical memory models suggest that an information item can be stored with sustained neural activity which emerges via activation of stable activity patterns in the network (e.g., attractors) (Hopfield, 1982; Amit, 1995; Wang, 2001; Wimmer et al., 2014) recently reviewed by Zylberberg and Strowbridge (2017) and Chaudhuri and Fiete (2016). These WM models propose that the generation of persistent activity can be the result of an intrinsic property of the neurons [including the generation of the bistability mediated by the voltage-gated inward currents (Kass and Mintz, 2005) and Ca2+-triggered long-term changes in neuronal excitability (Fransén et al., 2006)] and can be induced by the connectivity within the neural circuit with feed-forward (Ganguli and Latham, 2009; Goldman, 2009) or recurrent architecture (Koulakov et al., 2002; Kilpatrick et al., 2013). In such models, memory recall is impossible from a silent inactive state. For many WM models of persistent activity based on recurrent connectivity, small deviations in the network structure destroy the persistence. Moreover, a spiking form of information storage is energetically unfavorable because of the high metabolic value of action potentials (Attwell and Laughlin, 2001).
In theoretical studies, a concept of oscillatory sub-cycles storing 7 ± 2 in oscillatory neuronal networks was proposed by Lisman and Idiart (1995). Other models employ oscillatory activity of spiking neuron networks after depolarization to memorize a set of information patterns at different phases of rhythmic oscillations (Klinshov and Nekorkin, 2008; Borisyuk et al., 2013).
Recently, the persistent activity hypothesis has been undergoing critical reviews (Lundqvist et al., 2018) based on the experimental findings in rodents and primates showing that the robust persistent activity does not last for the entire delay period, but rather sequential neuronal firing is observed during the delay period suggesting that the PFC neural network may support WM based on dynamically changing neuronal activity (Fujisawa et al., 2008; Lundqvist et al., 2016; Runyan et al., 2017; Park et al., 2019; Ozdemir et al., 2020). Despite the considerable progress that has been made in identifying the neurophysiological mechanisms contributing to WM in mammals (D'Esposito and Postle, 2015; Zylberberg and Strowbridge, 2017), the ongoing debate focuses on the generation mechanisms of the delay period activity that appears to underlie WM (Constantinidis et al., 2018; Sreenivasan and D'Esposito, 2019).
Currently, one of the recognized experimentally based hypotheses of the WM mechanism underlining the delay activity (not necessarily persistent) is the synaptic plasticity in the PFC (Tsodyks and Markram, 1997; Hempel et al., 2000; Wang Y. et al., 2006; Erickson et al., 2010). Synaptic plasticity implies a rapid regulation of the strengths of individual synapses in response to specific patterns of correlated synaptic activity and contributes to the activity-dependent refinement of neural circuitry. Following these findings, alternative synaptic-based WM models have been proposed (Mongillo et al., 2008; Barak and Tsodyks, 2014; Koutsikou et al., 2018; Manohar et al., 2019). In these models, memory items are stored by stimulus-specific patterns of synaptic facilitation in a neuronal circuit. Synaptic plasticity does not require neurons to show a persistent activity for the entire period of the memory task, which results in a robust and more metabolically efficient mechanism. Some synaptic WM models based on short-term non-associative synaptic facilitation (Mongillo et al., 2008; Lundqvist et al., 2011; Mi et al., 2017) allow for reading out and refreshing existing representations maintained in the synaptic structure. Others have proposed fast Hebbian activity-dependent synaptic plasticity (Sandberg et al., 2003; Fiebig and Lansner, 2016) for encoding and maintenance of novel associations.
Despite the numerous models describing the astrocytic impact on signaling in neuronal networks (see Oschmann et al., 2018 for a recent review) (Makovkin et al., 2020), there are few attempts to theoretically investigate the role of astrocyte-induced modulation of synaptic transmission in memory formation. Shen and Wilde (2007) demonstrate one of the first results of simulating the coupling of a Hopfield neural network, astrocytes, and cerebrovascular activity. Although this is not yet a true biophysical model, the results suggest that a modification of the synapse strengths allows the neuronal firing and the cerebrovascular flow to be compatible on a meso-scale; with astrocyte signaling added, limit cycles exist in the coupled networks. Tewari and Parpura (2013) and Wade et al. (2011) study how bidirectional coupling between astrocytes and neurons in small neuron-astrocyte ensembles mediates learning and dynamic coordination in the brain. A recent interesting theoretical study proposes a self-repairing spiking astrocyte-neural network combined with a novel learning rule based on the spike-timing-dependent plasticity and Bienenstock, Cooper, and Munro learning rule (Liu et al., 2019).
3. Materials and Methods
3.1. Neuron-Astrocyte Network Model
Even though the balance of inhibition and excitation was shown to play an important role in WM stabilization and can influence WM capacity (Barak and Tsodyks, 2014), we focus on the properties of astrocyte-induced modulation of excitatory synaptic transmission in the PFC. We take spiking neuronal network with dimension W × H consisting of synaptically coupled excitatory neurons formed by the Izhikevich model (Izhikevich, 2003). Neurons in the network are connected randomly with the connection length determined by the exponential distribution.
It has been experimentally estimated that there is some overlap in the spatial territories occupied by individual astrocytes in the cortex (Halassa et al., 2007). An individual cortical astrocyte contacts on average 4-8 neuronal somata and 300–600 neuronal dendrites (Halassa et al., 2007). A cortical astrocyte has a “bushy” ramified structure in fine perisynaptic processes, which cover most of the neuronal membranes within their reach (Allen and Eroglu, 2017). This allows the astrocyte to integrate and coordinate a unique volume of synaptic activity. Following the experimental data, the astrocytic network compartment of our model is organized into a two-dimensional square lattice with only nearest-neighbor connectivity. Each astrocyte interacts with the neuronal ensemble of Na neurons with some overlapping. We consider bidirectional communication between neuronal and astrocytic networks. The scheme of the network topology is shown in Figure 1.
Figure 1. A neuron-astrocyte network topology. The neuron-astrocyte network consists of two interacting layers: a neural network layer and an astrocytic layer. The neuronal network with dimension W × H (79 × 79) consists of synaptically coupled excitatory neurons modeled by the Izhikevich neuron. Neurons in the network are connected randomly. The astrocytic network consists of diffusely connected astrocytes with dimension M × N (26 × 26). Blue lines show connections between elements in each layer. We consider the bidirectional interaction between the neuronal and astrocytic layers. Each astrocyte is interconnected with a neuronal ensemble of Na = 16 neurons with dimensions 4 × 4 (red lines) with overlapping in one row (violet lines). The input signal is fed to the neural network.
Model equations are integrated using the Runge-Kutta fourth-order method with a fixed time step, Δt = 0.1 ms. A detailed list of model parameters and values can be found in Table 1 (neural network model), Table 2 (astrocytic network parameters), Table 3 (neuron-astrocytic interaction parameters), and Table 4 (stimulation and recall testing). The code is available at: https://github.com/altergot/neuro-astro-network.
Table 2. Astrocytic network parameters Ullah et al. (2006).
Table 3. Neuron-astrocytic interaction parameters Gordleeva et al. (2012).
3.2. Neuronal Network
There are many available biophysical models of neuronal membrane potential dynamics (Morris and Lecar, 1981; Hodgkin and Huxley, 1990; Xu et al., 2020a,b; Zhang et al., 2020). For our purpose we chose the Izhikevich model as it is quite functional and computationally effective for network simulations (Izhikevich, 2003):
with the auxiliary after-spike resetting
where the superscripts (i = 1, …, 79, j = 1, …, 79) correspond to a neuronal index, the transmembrane potential V is given in mV and time t in ms. The applied currents simulate the input signal. Neurons receive a number of synaptic currents from other presynaptic neurons in the network, , which are summed at the membrane according to the following equation (Kazantsev and Asatryan, 2011; Esir et al., 2018):
Parameter describes the synaptic weight, . The astrocyte (m, n) modulates the synaptic currents of the neuron (i, j). The variable νCa introduces the astrocyte-induced modulation of synaptic strength and will be discussed below. In this study, we concentrate exclusively on the role of astrocyte-induced modulation of synaptic transmission and did not include mechanisms of synaptic plasticity in the model. The synaptic reversal potential for excitatory synapses is taken with Esyn = 0. Vpre denotes the membrane potential of the presynaptic neuron. For simplicity, we neglect the axonal and synaptic delays.
The architecture of synaptic connections between neurons is non-specific (random) with the following parameters. The number of output connections per each neuron is fixed at Nout = 40. Each neuron innervates Nout local postsynaptic targets, which are randomly chosen in polar coordinates. The distances between neurons r are determined according to the exponential distribution fR(r) and the angles ϕ are chosen from the uniform distribution in the range [0; 2π]:
In this way, the coordinates of postsynaptic neurons are computed as follows:
where xpre, ypre denote the coordinates of the presynaptic neuron, xpost, ypost are coordinates of the postsynaptic neurons. Coordinates are picked repeatedly in case of a duplicated connection.
3.3. Astrocytic Network
In the model, we try to implement the biologically plausible organization of the astrocytic network and neuron-astrocyte interaction. The astrocytic network is configured in the form of a two-dimensional square lattice with dimension M × N. Cortical astrocytes are coupled via Cx43 gap junctions mostly permeable to inositol 1,4,5-trisphosphate (IP3) (Yamamoto et al., 1990; Nagy and Rash, 2000). Hence, in the model, we consider local diffusive coupling. Besides, each astrocyte is interconnected with a neuronal ensemble of Na neurons. It was experimentally shown that the sensory stimulation evokes fast intracellular Ca2+ signals in fine processes of cortical astrocytes in response to local synaptic activity in the neuronal circuit (Wang X. et al., 2006; Takata et al., 2011; Stobart et al., 2018). Multiple rapid spatially restricted Ca2+ events in the astrocytic process are induced by intense neuronal firing. Local events are spatially and temporally integrated by the astrocytic cell, which results in a global long lasting Ca2+ event. In turn, this event induces the release of gliotransmitters affecting synaptic transmission in the local territory of individual astrocytes (Bekar et al., 2008; Henneberger et al., 2010; Araque et al., 2014). For simplicity, we did not model the detailed process of spatial-temporal integration of the rapid Ca2+ signals in the morphological structure of astrocytes modeled earlier by Gordleeva et al. (2018, 2019) and Wu et al. (2018). Here we employ a mean-field approach to describe the emergence of a global Ca2+ signal and its impact on the synchronization of neuronal ensemble controlled by a certain astrocyte.
As pyramidal neurons generate the spike, glutamate is released from the presynaptic terminal into the synaptic cleft (Figures 3B,D). The amount of glutamate, G, that was diffused from the synaptic cleft and reached the astrocytic process can be described by the following equation (Gordleeva et al., 2012; Pankratova et al., 2019):
here αglu is the glutamate clearance constant, kglu is the efficacy of the release, Θ denotes the Heaviside step function, and V(i, j) is the membrane potential of the corresponding presynaptic neuron (i, j). Binding of glutamate to metabotropic glutamate receptors (mGluR) on the astrocytic membrane, which is located close to the synapse, triggers the production of IP3 in the astrocytes (Figure 3E). We use the approaches from earlier studies to describe the dynamics of the intracellular concentration of IP3 in astrocytes (Nadkarni and Jung, 2003; Ullah et al., 2006):
with m = 1, …, 26, n = 1, …, 26. Parameter denotes the steady state concentration of the IP3 and JPLCδ describes the IP3 production by phospholipase Cδ (PLCδ) (Ullah et al., 2006):
The variable Jglu describes the glutamate-induced production of the IP3 in response to neuronal activity and is modeled as a rectangular-shaped pulse with amplitude Aglu μM and duration tglu ms:
here t0 denotes the periods when the total level of glutamate in all synapses associated with this astrocyte reaches a threshold:
here we use the parameter Gthr = 0.7. [x] denotes the Iverson bracket. Fact is the fraction of synchronously spiking neurons of the neuronal ensemble corresponding to the astrocyte. For the emergence of the calcium, elevation Fact = 0.5 is required. In other words, according to the experimental data (Bindocci et al., 2017), activation of the production term, Jglu, which results in the generation of a calcium signal in the astrocyte, can be induced only when correlated activity in the neuronal ensemble interacting with the astrocyte reaches a sufficient level of coherence.
Increase of IP3 concentration in the astrocytes induces the release of Ca2+ from internal stores, mostly from the endoplasmic reticulum (ER), to cytosol. For a simplified description of the biophysical mechanism underlying the calcium dynamics in astrocytes, we use the Ullah model (Ullah et al., 2006). Changes of the intracellular Ca2+ concentration, [Ca2+], are described by the following equations:
where h is the fraction of the activated IP3 receptors (IP3Rs) on the ER surface. Flux JER is Ca2+ flux from the ER to the cytosol through IP3Rs, Jpump is the Ca2+ flux pumped back into ER via the sarco/ER Ca2+-ATPase (SERCA), and Jleak is the leakage flux from the ER to the cytosol. Fluxes Jin and Jout describe the calcium exchange with extracellular space. The fluxes are expressed as follows:
Cortical astrocytes are coupled by Cx43 gap junctions (Yamamoto et al., 1990; Nagy and Rash, 2000; Nimmerjahn et al., 2004). Thus, the diffusion of active chemicals becomes possible between the neighboring astrocytes. Currents diffCa and diffIP3 describe the diffusion of Ca2+ ions and IP3 molecules via gap junctions between the astrocytes in the network and can be expressed as follows:
where parameters dCa and dIP3 describe the Ca2+ and IP3 diffusion rates, respectively. Following experimental data, we assume that Cx43 is less permeable to Ca2+ than to IP3. We consider that each astrocyte is diffusively coupled with only four nearest-neighbors. (Δ[Ca2+])(m, n) and are the discrete Laplace operators:
Equations (7)–(9), (11)–(13) predict that the synchronized activity in the neuronal ensemble trigger astrocytic Ca2+ signals, and in the absence of neuronal stimulus in the astrocytic network, steady state Ca2+ concentration is maintained.
Next, we account for the effect of the enhancement of excitatory synaptic transmission through the action of the glutamate released from astrocytes. We consider that the astrocytic glutamate-induced potentiation of the synapse consists in NMDAR-dependent postsynaptic slow inward currents (SICs) generation (Fellin et al., 2004; Chen et al., 2012) and mGluR-dependent heterosynaptic facilitation of presynaptic glutamate release (Perea and Araque, 2007; Navarrete and Araque, 2008, 2010). In the model, we propose that global events of Ca2+ elevation in astrocytes result in glutamate release, which can modulate the synaptic strength of all synapses corresponding to the morphological territory of a given astrocyte. For simplicity, the relationship between the astrocyte Ca2+ concentration and synaptic weight of the affected synapses gsyn, is described as follows:
where the parameter denotes the strength of the astrocyte-induced modulation of the synaptic weight and Θ(x) is the Heaviside step-function. The feedback from the astrocytes to the neurons is activated when the astrocytic Ca2+ concentration is larger than and the fraction of synchronously spiking neurons of the neuronal ensemble corresponding to the astrocyte Fastro during the time period of τsyn = 10 ms. According to the experimental data on the kinetics of NMDAr-dependent SICs that is evoked by glutamate released from astrocytes (Fellin et al., 2004), the duration of the astrocyte-induced facilitation of synaptic transmission is fixed and is equal to τastro = 250 ms.
3.4. Stimulation Protocol
The term Iapp in Equation (1) represents specific and non-specific external inputs. A non-specific noisy input simulates input signals from networks of other brain areas and is applied continuously to all neurons in the form of independent Poisson pulse trains of a certain rate, fbg, with amplitudes randomly and uniformly distributed in the interval [−10, 10] μA. This input evokes a background network state with low-rate spontaneous spiking.
Specific input contains training samples in the form of binary spatial patterns. The patterns represent different spatial distributions relative to background state with non-specific input only. The average size of a sample is 1,078 neurons (18% of the network) stimulated by the specific input, with an average 35.2% overlapping in the population. For a visual representation of samples, we take binary images of numerals (0,1,2,3,4,..) with size W × H pixels, where each pixel corresponds to a neuron in the neuronal layer. Neurons corresponding to the shape of the numerals receive a rectangular excitatory pulse with length tstim and amplitude Astim. The shape of each sample was spatially distorted by 5% random noise such as “salt and pepper noise.” Then transient inputs were applied to simulate the nonmatching test items and the cue (length ttest, and amplitude Atest). In the cued recall for simulating the loss in saliency, we applied a shorter input with lower amplitude and a higher level (20%) of random noise. Figure 2 illustrates the time course of a training and test protocol used in the WM paradigm.
Figure 2. Time course of a training and test protocol in the multi-item WM paradigm used. During training the samples were loaded sequentially by applying external inputs of 200 ms durations with 100 ms inter-item intervals. After a 700 ms training stimulus was applied, we tested the maintenance of the memory by applying matching and non-matching items of 150 ms durations and 250 ms inter-item intervals.
3.5. Memory Performance Metrics
To measure the memory performance of the system we calculate the correlation of a recalled pattern with the ideal item in the following way:
here w = 10 frames = 1 ms, P—a set of pixels belonging ideal pattern, W, H—network dimensions, thr—spike threshold, I—indicator function, and TP—a set of frames in the tracking range of pattern P. In a sense, this correlation metric can be associated with 1 − d averaged between the pattern and background, where d is the Hamming distance.
Let us show how the neuron-astrocyte network model exhibits memory formation. First, we will consider a simple single-item memory task illustrating information loading, storage, and retrieval. Next, we will demonstrate how the network can be successfully trained to memorize and recall several patterns with significant overlaps. Finally, we will analyze model performance metrics, capacity, and characteristics of pattern remembering on different parameters.
4.1. Single-Item WM
First, we will test the neuron-astrocyte network in the most common experimental paradigm of WM studies—the DMS task. This task requires a single item to be held in memory during a brief delay period. Before specific stimulation, the neural network demonstrates irregular, low-rate background activity (see activity beginning in Figure 4). At the 500 ms mark, we load an item by applying transient external input to the corresponding neuronal population for 200 ms (Figures 3A, 4). During training, each astrocyte tracks the activity of the neuronal subnetwork associated with it. As soon as the extracellular concentration of glutamate (Figures 3B,D) and correlated firing in neurons achieve a certain level, which satisfies the condition Equation (10), Ca2+ concentration in matching astrocytes elevates (Figure 3E). In accordance with the experimental data (Bindocci et al., 2017), we tuned the model parameters in such a way that the onset of calcium elevation in the astrocytes induced by synchronous neuronal discharge had a delay of ≤ 2 s. Following the increased firing in the stimulus-specific part of the neuronal network upon reaching a threshold of 0.15 μM, the astrocytes release gliotransmitters modulating the synaptic strengths in corresponding locations (Figure 3). The calcium pulse in astrocytes lasts for several seconds. Its duration determines the length of the delay interval in the DMS task, during which the item is maintained in the memory. After the training stimulus ends, we test maintenance of the single-item memory by applying two non-matching items and cue item with ttest durations and 250 ms inter-item intervals (Figures 3, 4). Because the astrocytic feedback also depends on the activity of the neuronal subnetwork, the model responds differently to the applied items. A short presentation of the cue to the neural network evokes the astrocytic-induced increase in the synaptic strength between stimulus-specific neurons and results in a local spatial synchronization in the whole stimulus-specific neuronal population (see Figure 3A in comparison to Figure 3C). Similar to experimental data (Miller et al., 1996), delay activity in our model is sample-selective. We observe that the pattern-specific firing rate in the neuronal network increases and is equal to 270 Hz in comparison with the response to a non-specific stimulus (80 Hz) (Figure 4B). Such a high frequency is determined by the choice of a fast-spiking neuron model (Izhikevich, 2003). The firing rates in simulations with a regular spiking neuron model (Izhikevich, 2003) are almost 10 times lower: 30 Hz for stimulus-specific and 4.5 Hz for non-specific stimulus. The elevation of the frequency in the stimulus-specific neuronal population can continue after the end of the cue, which is determined by the duration of the astrocyte-induced enhancement of the synaptic weight.
Figure 3. Model of neuron-astrocyte interaction. (A) Spike train and (B) concentration of the neurotransmitter, G(t), of the stimulus-specific neuron. (C,D) Same as in (A,B) but for an unspecific neuron. (E) Intracellular concentration of Ca2+ and IP3 in a stimulus-specific astrocyte. Black bars at the top indicate periods when each of the stimuli (training stimulus—sample, nonmatching test items—nonmatch, test cue—match) were presented. In response to the presynaptic spike train (A,C), the neurotransmitter, glutamate, G releases (B,D) into extracellular space and the concentration of IP3 increases in the astrocyte (E, blue line) inducing the elevation of intracellular Ca2+ (E, red line).
Figure 4. Delayed matching to sample WM task. (A) One trial of the task simulated in the network. Spike raster of the neuronal network showing sample-selective delay activity. Neurons belonging to the stimulus-specific population are indicated by blue color. Black bars at the top indicate periods when each of the stimuli (training stimulus—sample, nonmatching test items—non-match, test cue—match) were presented. (B,C) The averaged firing rate of the stimulus-specific and unspecific neurons over time, respectively (20 ms bins).
For a visual representation of memory formation, we follow the space-time distribution of sample-selective delay activity. Figure 5 illustrates the spatial distribution of activity in neuronal and astrocytic layers at the different moments of training and cued recall for the same single-item memory task as presented in Figure 4. Training induces the emergence of synchronized calcium activity of spatially clustered astrocytes (Figure 5C). Note that locally synchronized astrocytes have been found in the neocortex and hippocampus in situ and in vivo (Takata and Hirase, 2008; Sasaki et al., 2011). Such calcium activity correlated in time and space can lead to spatial-temporal synchronization in the neuronal network (Araque et al., 2014). This mechanism of neuron-astrocyte network interaction underlies the sample selectivity and pattern retrieval in the model. Note that 20 % of noisy cue items (Figure 5D) can be identified and cleared from noise by the neuronal network due to the astrocyte-induced feedback (Figure 5E). In other words, the spiking neuronal network accompanied by astrocytes can filter a cue pattern distorted by noise. Video of the single-item memory encoding and cued recall in the neuron-astrocyte network can be found in the Supplementary Material.
Figure 5. Snapshots of the training (A–C) and testing (D,E) of the neuron-astrocyte network in the single-item working memory task. (A) Training sample. (B) Response of the neuronal network to the sample. The values of the membrane potentials are shown. (C) Intracellular Ca2+ concentrations in the astrocytic layer. (D) Testing item with 20% salt and pepper noise. (E) Cued recall in the neuronal network. The averaged firing rate on the test time interval for each neuron is shown.
4.2. Multi-Item WM
Next, we consider the multi-item WM formation. In this case, we loaded four items, images of numerals 0, 1, 2, 3 in the following way. The images were loaded sequentially by applying external inputs of tstim durations with 100 ms inter-item intervals (see Figures 2, 6, 7A, 8). Due to the coincidence of different stimulus-specific neuronal populations in space, the spatial calcium patterns in astrocytic layers for different items overlap significantly (Figure 8H). After a 700 ms training stimulus was applied, we tested the maintenance of the memory by applying matching and non-matching items of ttest durations and 250 ms inter-item intervals (Figures 2, 6, 7A, 8). The images were distorted by 20% noise. The astrocyte-mediated feedback modulating coherent neuronal activity provided the selectivity of the model response. The system remembered the correct image. Thus, we observed that all items were successfully filtered only in the cued recall. Needless to say that firing rate increases significantly in the cued recall due to the selective increase of synaptic strengths (Figures 7B,C). To evaluate the performance of the neuron-astrocyte WM, we used the correlation between recalled item and the ideal item during the multi-item WM task as a metric (see section 3.5; Figure 7D). It is important to note, that during multi-item remembering, spurious correlations never dominate in that sense as accuracy of our system is always equal to 100%. There was an increase in correlation with the target image and no attraction to the wrong image or chimeras. Maximum correlation reached 95% in training and 93% in testing sets on average for four samples. Also note that the model is quite robust to the type of binary images input (images of numerals, letters, etc.). Video of the multi-item WM in the neuron-astrocyte network can be found in Supplementary Material 2.
Figure 6. Neuron-astrocyte network simulation with four loaded memory items. (A,C,E) Spike train and (B,D,F) concentration of the neurotransmitter, G(t), of three neurons belonging to different stimulus-specific populations. (A,B) Stimulus-specific neuron to sample 0. (C,D) Stimulus-specific neuron to sample 2. (E,F) Neuron unspecific to all samples. Black bars at the top indicate periods when each of the stimuli (training stimulus—sample, non-matching test items—nm, match cue—m) were presented.
Figure 7. Multi-item WM in the neuron-astrocyte network. (A) Spike raster of the neuronal network with four training patterns. Neurons are colored according to their pattern selectivity. Pattern overlapping in neuronal populations is 35.2% on average for four patterns. Black bars indicate periods when each of the stimuli were presented. (B,C) The averaged firing rate of the stimulus-specific and unspecific neurons over time, respectively (20 ms bins). (D) Correlation of filtered items. The different colors correspond to the correlations with different ideal samples. The dotted line shows the correlation of the testing item (for 20% noise level in test). Black bars at the top indicate periods when each of the stimuli (training stimulus—sample, non-matching test items—nm, match cue—m) were presented.
Figure 8. Snapshots of the training (A,B,C,F,G,H) and testing (D,E,I,J) of the neuron-astrocyte network in the multi-item working memory task. We used the following training set consisting of four samples: 0,1,2,3. (A,F) The example of the first and last training samples, respectively. (B,G) The response of the neuronal network to samples. The values of the membrane potentials are shown. (C,H) The intracellular Ca2+ concentrations in the astrocytic layer. (D,I) The testing items with 20% salt and pepper noise. (E,J) The cued recalls in the neuronal network. The firing rate averaged on the test time interval for each neuron is shown.
To characterize the quality of memory formation in the model, we examined the dependencies of correlation of retrieval pattern in cued recall on variable parameters of the input patterns and astrocytic, synaptic, and network structure (Figures 9, 10). First, we investigated its dependence on noise parameters. The dependence of correlation of recalled pattern on the noise level in training and test experiments is shown in Figure 9A. Specifically, the correlation difference between the recalled pattern and noisy input is presented. In other words, the model can improve test images depending on noise in training and testing. Training the network with samples with a low noise level (up to 25%) provides a high correlation. The elevation of the noise level in the training sample induces a random activity pattern in the astrocytic network, which in turn leads to noisy recall.
Figure 9. Correlation between recalled pattern (see section 3.5) and ideal item in a multi-item WM task performed by the neuron-astrocyte network. The correlation averaged over four patterns is shown. (A) Noise-resistance of the model. Dependence of correlation on the noise level in training and testing. The correlation difference between cued recall pattern and noisy input is shown. (B,C) The influence of the neuron-astrocytic interaction structure. (B) Dependence of correlation on the number of spiking neurons required for the calcium elevation in the astrocyte, Fact, and on the number of spiking neurons required for the emergence of astrocyte-induced enhancement synaptic transmission, Fastro. (C) Dependence of correlation on noise level in training samples and on the parameter, Fact. Fastro = 0.5. For (B,C) noise level in cue is 20%.
Figure 10. Influence of synaptic connectivities architecture in the neural network on the correlation of recalled pattern in the multi-item WM task performed by the neuron-astrocyte network. The correlation averaged over four patterns is shown. (A) Dependence of correlation on the number of output synaptic connections for each neuron, Nout, and synaptic weight, η. λ = 5. (B) Dependence of correlation on the number of output synaptic connections for each neuron, Nout, and synaptic connection distribution parameter, λ Equation (4). η = 0.025.
The morpho-functional structure of connections between neurons and astrocytes can affect pattern retrieval in the model (Figures 9B,C). The key parameters determining this structure are the fraction of synchronously spiking neurons of the neuronal ensemble corresponding to the astrocyte required for the emergence of calcium elevation in the astrocyte, Fact, and the fraction of synchronously spiking neurons of the neuronal ensemble corresponding to the astrocyte required for the emergence of astrocyte-induced enhancement of synaptic transmission, Fastro. Here we do not account for various spatiotemporal properties of gliotransmitter release and astrocyte Ca2+ signals evoked by different levels of neuronal activity (Araque et al., 2014). We assume that simultaneous activation of synapses induces multiple Ca2+ events at different processes of the astrocyte, which are spatially and temporally integrated and result in the generation of a global long-lasting Ca2+ elevation (Bindocci et al., 2017) that can affect synaptic transmission in the territory of individual astrocytes. For this purpose, parameters Fact and Fastro estimate the correlation level of synapse activity in the model. The optimal range of Fastro for correlation of recalled pattern is [0.4–0.6] (Figure 9B). Smaller values of the parameter, Fastro, lead to the effect of astrocyte-induced synchronization initiated even by non-stimulus specific uncorrelated noise activity in a small ensemble of neurons. On the contrary, the use of larger Fastro values implies that a highly correlated activity of almost all neurons located in the territory of a given astrocyte is required for the existence of astrocytic modulation of synapses. Hence, the neuron-astrocytic network can not perform a correct recall of noisy cue. Another point is that Figure 9B was obtained for a training set with a low 5% noise level and did not reveal the dependence of the correlation of recalled pattern on the parameter Fact. We studied the influence of the parameter Fact on the correlation in the simulations with different noise in training samples (Figure 9C). For lower noise level in training samples, the network memorizes items regardless of the value of the parameter, Fact. Increasing the level of noise in the training samples for small values of the parameter, Fact, leads to Ca2+ elevation in randomly distributed astrocytes first, and then in the whole astrocytic layer. Nevertheless, such non-stimulus-specific astrocytic activation can result in a high correlation of recalled pattern because of the moderate noise level in the cue and optimally chosen value of the parameter Fastro. On the contrary, for a larger value of the Fact > 0.85 Ca2+ signal in astrocytes can only be evoked by a relatively "clean" sample with a small percentage of noise < 5%, therefore increasing the level of noise in training samples results in poor correlation (Figure 9C). We found that the range of Fact = 0.8 − 0.85 was optimal for performing the WM tasks by the neuron-astrocyte network. In this range, astrocyte activations were stimulus-specific and the astrocyte layer could memorize training samples with a low noise level. The obtained high value of optimal Fact denotes that correct functioning of WM in the neuron-astrocyte network dependent on the generation of global Ca2+ signals in astrocytes required highly correlated activity in corresponding neuronal ensembles as confirmed by recent experimental work (Bindocci et al., 2017).
Next, we studied the influence of synaptic connectivity architecture in the neural network, specifically the number, weight, and distribution of synaptic connections, on the correlation in the multi-item WM task (Figure 10). The minimal number of synaptic connections, Nout, required for the existence of cued recall is 20 (Figure 10A). A smaller number of connections is not enough to activate all the neurons from the stimulus-specific population. Simultaneous increase of weights and number of connections induces the generation of large synaptic currents resulting in self-sustained overactivation of the neuron-astrocyte network. Therefore there exists an optimal range of synaptic weight values to ensure high correlation. We found that for our model this range is η ∈ [0.005 − 0.05]. Figure 10B illustrates the dependence of correlation on the number of output synaptic connections and their distribution. The smaller the parameter λ from Equation (4), the lower the probability of long-distance connections. The highest correlation was observed for local connections, λ < 7, due to the fact that short-range connections do not lead to blurring of the pattern retrieval boundaries in the neural network. Figure 10B also determined the optimal range for the number of synaptic connections: Nout ∈ [25, 55].
The key parameters, which determine WM capacity in the proposed neuron-astrocyte network, are the duration of calcium signals in astrocytes and duration of the astrocyte-induced modulation of synaptic transmission. Duration of astrocytic calcium elevations is determined by the intrinsic mechanisms of the IP3-evoked Ca2+-induced Ca2+ release from the astrocyte endoplasmic reticulum stores, which is described by the biophysical model (Li and Rinzel, 1994) used in this study. Fragmentary experimental data on duration of the gliotransmitter-induced modulation of synaptic transmission shows that the short-lived version of this modulation lasts from fractions of a second to a few minutes, while long-term plasticity can last for tens of minutes (for review see Pittà et al., 2016). In this study we have chosen the duration of astrocytic effect on synaptic transmission in accordance with the kinetics of SICs triggered by astrocytic glutamate (Fellin et al., 2004).
To characterize memory capacity, we subjected it to longer trains of samples. Samples were applied to neuronal ensembles with an average 35.2% overlapping in population. To check the memorization, we presented cue items in reverse order compared to the learning mode (e.g., learning: 0, 1, 2, ..., 7, 8; test: 8, 7, ..., 2, 1, 0). The number of items with a correlation of recalled pattern higher than 90% indicated the capacity of the system. Figure 11 shows the capacity as a function of the sample number in the training sequence. For a chosen set of parameters, the capacity of WM ranges from five to six. It is interesting that such a limited capacity coincides with psychological studies indicating that human ability to keep information in readily accessible WM is limited, ranging between three and five items for the majority of healthy people (Cowan, 2010). However, it is obvious that the organization of the WM in the human brain involves the coordinated work of a much larger number of cells and even several CNS regions. We investigated the influence of the duration of astrocyte-induced modulation of synaptic transmission, τastro, on the capacity of WM. The number of items stored in the system memory is maximum for parameter values in the range: τastro ∈ [60, 660] ms. For small values of τastro, excitation does not have time to spread to the stimulus-specific neural population; for long astrocytic modulation the different items in cued recalls interfered with each other.
Figure 11. Capacity of the multi-item WM in the neuron-astrocyte network. Capacity as a function of the sample number in training. The number of images with a correlation of recalled pattern higher than 90% is shown.
The capacity of WM in the proposed neuron-astrocyte network is primarily limited by the duration of the astrocytic Ca2+ event τCa and does not depend on the number of neurons or astrocytes in the network. In the protocol of our testing session, the duration of the Ca2+ signal is sufficient to keep six samples only, and after that it starts forgetting the images received at the beginning. So, the maximum capacity of the model is six. In case more than 12 items are applied, the network starts dumping them and chimeras appear (Figure 11). So, the system begins to make mistakes and the capacity of the network shrinks (here the capacity is understood as the number of the retrieval items with a correlation level with the sample of more than 90%). This is valid for the case considered in the paper when a stimulus is applied to the whole network and different stimuli overlap strongly in neuronal subnetworks. For this case, the capacity does not depend on the number of neurons and astrocytes.
If we consider the case of applying items to different unique non-overlapping neuronal subpopulations, an increase in the size of the network will result in increased capacity. The WM capacity of each subnetwork in this case is unequivocally determined by the duration of the calcium signal and can be obtained analytically. We estimated the Ca2+ signal duration to be τCa = 3.8 s. We consider the case of training on K samples in a fixed order: 1, 2, ..., K. During the test, we look at a permutation p consisting of K patterns. For the permutation, we estimate the number of correctly recalled patterns, Kp. The pattern is considered correctly recalled if no more than τastro has passed since its presentation. The average capacity, C, for this case is defined as the average number of correctly recalled patterns over all possible permutations p. According to this description, the average capacity can be calculated by the following equation:
where τ11—train sample duration, τ12—duration between train samples, τ21—test sample duration, τ22—duration between test samples, τshift—delay between train and test, τCa—calcium event duration, —time of i-th train sample finishes, . Figure 12 shows the capacity as a function of the sample number, K. After it reaches a maximum of 6.6 for 8 samples, the capacity begins to decrease monotonically, while the number of samples increases.
Figure 12. Capacity of the multi-item WM in the neuron-astrocyte network in the case of non-overlapping neuronal populations obtained analytically. Capacity as a function of the sample number in the training sequence.
We proposed a new biologically motivated spiking neuron network model accompanied by astrocytes that demonstrates working memory formation. The model acts at multiple timescales: at a millisecond scale of firing neurons and the second scale of calcium dynamics in astrocytes. The neuronal network consists of randomly sparsely connected excitatory spiking neurons with non-plastic synapses. Astrocyte-induced activity-dependent short-term synaptic plasticity results in local spatial synchronization in neuronal ensembles. The WM realized by such astrocytic modulation is characterized by one-shot learning and is maintained for seconds. The astrocyte influence on the synaptic connections during the elevation of calcium concentration implements Hebbian-like synaptic plasticity differentiating between specific and non-specific activations. Note that the proposed model is crucially different from the attractor-based network memory models (Hopfield, 1982; Amit, 1995; Wang, 2001; Wimmer et al., 2014) and works similarly to WM models based on synaptic plasticity (Mongillo et al., 2008; Lundqvist et al., 2011; Mi et al., 2017; Manohar et al., 2019). In particular, in its functionality, the model is quite close to short-term associative (Hebbian) synaptic facilitation (Sandberg et al., 2003; Fiebig and Lansner, 2016).
The concept of our WM model operation is schematically summarized in Figure 13. Composed of two building blocks, e.g., fast-spiking neurons and slow astrocytes, the proposed memory architecture eventually demonstrated synergetic functionality in loading information and its readout by the neuronal block and storage implemented by the astrocytes. In contrast with solely neuronal circuit models where memory is encoded in synaptic connections and their plasticity, which inevitably leads to the problem of overlapping, our model splits functionality using astrocytes as a pool for stored patterns. Even with significant overlaps, they can be successfully retrieved due to coherent synaptic modulations by the astrocytes and synchronous neuron firing, which provide the selectivity. When the memory is maintained at the time scale of calcium elevation in astrocytes, the synapses are not specifically modulated and theoretically can be employed for other tasks. Note also that the memory is transient with no long-lasting changes in structure and parameters. Thus, new information can be successfully uploaded and stored without interference of traces of previously memorized information. Complete memory overwrite interval is estimated at several seconds and is also defined by the duration of astrocyte activations. The circuit in Figure 13 can be also treated as a building block for functional spiking neuron networks (SNN) which are now intensively discussed in the IT community as an alternative to artificial neural networks (ANNs) used in machine learning and artificial intelligence. Theoretically, SNNs as more accurate models of brain circuits are believed to be much more powerful than traditional formal networks in processing efficiency. However, learning and memory algorithms working well in ANNs can not be directly translated to spiking neurons representing, in fact, analogous dynamical systems. We hypothesized that astrocytes may be a missing block in the implementation of learning and memory in SNNs as systems to solve information processing tasks.
The proposed model clearly confirms the theoretical hypothesis that astrocytic modulation of synaptic transmission can be involved in the formation of functional cortical WM. We show that the potentiation of excitatory synapses induced by the glutamate released from astrocytes could serve as a possible molecular mechanism for WM. Stages of multi-item WM, which include loading, storage, and cued recall, manifest in brief oscillatory bursts, which are functionally similar to WM activity in nonhuman primate PFC (Lundqvist et al., 2016) rather than sustained neuron spiking. Interestingly, the activity of the neuron-astrocyte network corresponding to a memorized pattern exhibits a sufficient degree of stability, which ensures memory retention despite the presence of significant overlaps in the stimulus-specific subnetworks.
Needless to say the astrocyte-induced modulation of synaptic transmission proposed in this study as a mechanism for WM organization does not exclude but rather complements other synaptic and neural plasticity mechanisms (fast Hebbian synaptic plasticity/short-term synaptic plasticity, facilitation, augmentation, dendritic voltage bistability, etc.) and may well act in parallel to them.
On the one hand, there has been much experimental evidence that astrocytes contribute to synaptic plasticity, coordination of neural network oscillatory activity, and memory function (Santello et al., 2019). It was shown recently that astrocytic impact is circuit-specific (Martin et al., 2015) and stimulus-specific (Mariotti et al., 2018). Improved Ca2+ imaging approaches have identified a spatiotemporal diversity of astrocytic signals that may underlie the capacity of astrocytes to encode and process different patterns of activation (Bindocci et al., 2017; Stobart et al., 2018). Besides, the temporal scale of the astrocytic calcium dynamics and dynamics of the neuron-astrocyte bidirectional communication including the effects of astrocytic influence on synaptic plasticity fit very well in the timing required in WM processes.
On the other hand, the ongoing intense debate about principles of WM organization challenges the canonical theory of persistent delay activity in network attractors with recurrent excitation (Bouchacourt and Buschman, 2019) and offers alternative models incorporating different biophysical network mechanisms of WM (Barak and Tsodyks, 2014; Lundqvist et al., 2018). The principal reasons for such a debate are the reexamination of experimental data, which show a large heterogeneity in delay neuronal activity during WM tasks (Stokes et al., 2013). A non-classical WM model includes short-term synaptic plasticity (Mongillo et al., 2008; Hansel and Mato, 2013), the balance of inhibition and excitation (Boerlin et al., 2013), NMDA currents affecting on the neuronal excitability (Durstewitz, 2009), and other parameters. These models, however, have a number of shortcomings: inability to describe encoding of novel associations in synaptic facilitation-based models; unclear mechanisms for achieving precise tuning of recurrent excitation and inhibition; and the time constant of the NMDA receptor is appropriate to maintain memories for 1–5 s, but not for longer. The investigation of the synaptic mechanisms underlying WM is an ongoing process. Therefore, incorporation of astrocytes as spatiotemporal integrators and modulators of synaptic transmission in neural network models may help advance the theoretical framework of WM encoding and maintenance mechanisms.
Our simulations in a biologically relevant but still quite general model have eventually illustrated the hypothesis of astrocytes participation in WM functioning. This hypothesis emerged from several experimental facts on the astrocyte contributions to the cognitive functions and their impairments (Santello et al., 2019; Gordleeva et al., 2020; Whitwell et al., 2020). Indeed, in many of these cases, the understanding of the precise mechanisms of the astrocytic involvement is quite fragmentary and more work on the specific role of astrocytic action in the memory processes is needed. We believe that our model demonstrating WM functionality based on a quite clear mechanism of inter-coordination between fast spiking neuronal circuits and an astrocyte reservoir will be helpful in further experimental research including cellular and in vivo studies. In particular, depressing or facilitating the neuron-astrocyte interaction by specific drug injections can be used to monitor behavior impairments in animal studies.
To conclude, the recent controversies in the field of WM and the constant growth of experimental evidence about the participation of astrocytes in information processing, cognitive function, and dysfunction open up questions about mechanisms of astrocyte involvement in memory formation. Using a computational model, we demonstrated that the astrocyte-induced facilitation of excitatory transmission in PFC is a plausible mechanism for WM organization. The proposed model accounts for the astrocytic modulation of synaptic transmission in a spiking neural network. The biologically relevant neuron-astrocytic network implements loading, storage, and cued retrieval of multiple items presented to the neuronal populations with significant overlapping. The mechanism of WM functioning in the model is based on coherent neuronal firing in response to sensory inputs which are coordinated by astrocytes in time, serving as a “memory reservoir” and in space serving as a “signal bridge” providing a certain level of synchrony.
Future research in the framework of the proposed WM model in the neuron-astrocyte network will be focused on the interplay of excitation and inhibition that can stabilize WM (Boerlin et al., 2013); the effects of synaptic plasticity (Mongillo et al., 2008; Hansel and Mato, 2013) namely associative short-term potentiation (a fast-expressing form of Hebbian synaptic plasticity) that can provide an encoding of novel associations (Fiebig and Lansner, 2016); the subcellular calcium dynamics in astrocytes (Bindocci et al., 2017; Gordleeva et al., 2019); and on the structure of the cortical microcircuit reflecting the columnar organization of the neocortex.
Data Availability Statement
The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.
All authors jointly worked on all aspects of the study and the preparation of the manuscript. The literature analysis, coding, and data gathering were mainly done by YT, MK, SG. The manuscript was mainly written by SG and VK. AZ, MI and AG supervised the work.
This research was supported by the Ministry of Science and Higher Education of the Russian Federation (project no. 075-15-2020-808).
Conflict of Interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
AZ acknowledged support from MRC Grant no. MR/R02524X/1.
The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fncel.2021.631485/full#supplementary-material
Bindocci, E., Savtchouk, I., Liaudet, N., Becker, D., Carriero, G., and Volterra, A. (2017). Three-dimensional ca2+imaging advances understanding of astrocyte biology. Science 356:eaai8185. doi: 10.1126/science.aai8185
Borisyuk, R., Chik, D., Kazanovich, Y., and da Silva Gomes, J. (2013). Spiking neural network model for memorizing sequences with forward and backward recall. Biosystems 112, 214–223. doi: 10.1016/j.biosystems.2013.03.018
Brunel, N., and Wang, X.-J. (2001). Effects of neuromodulation in a cortical network model of object working memory dominated by recurrent inhibition. J. Comput. Neurosci. 11, 63–85. doi: 10.1023/a:1011204814320
Chen, N., Sugihara, H., Sharma, J., Perea, G., Petravicz, J., Le, C., et al. (2012). Nucleus basalis-enabled stimulus-specific plasticity in the visual cortex is mediated by astrocytes. Proc. Natl. Acad. Sci. U.S.A. 109, E2832–E2841. doi: 10.1073/pnas.1206557109
Constantinidis, C., Funahashi, S., Lee, D., Murray, J. D., Qi, X.-L., Wang, M., et al. (2018). Persistent spiking activity underlies working memory. J. Neurosci. 38, 7020–7028. doi: 10.1523/jneurosci.2486-17.2018
Erickson, M. A., Maramara, L. A., and Lisman, J. (2010). A single brief burst induces (glur)1-dependent associative short-term potentiation: a potential mechanism for short-term memory. J. Cogn. Neurosci. 22, 2530–2540. doi: 10.1162/jocn.2009.21375
Esir, P. M., Gordleeva, S. Y., Simonov, A. Y., Pisarchik, A. N., and Kazantsev, V. B. (2018). Conduction delays can enhance formation of up and down states in spiking neuronal networks. Phys. Rev. E 98:052401. doi: 10.1103/physreve.98.052401
Fellin, Pascual, Gobbo, Pozzan, Haydon, Fellin, T., Pascual, O., Gobbo, S., Pozzan, T., Haydon, P. G., and Carmignoto, G. (2004). Neuronal synchrony mediated by astrocytic glutamate through activation of extrasynaptic NMDA receptors. Neuron 43, 729–743. doi: 10.1016/j.neuron.2004.08.011
Fransén, E., Tahvildari, B., Egorov, A. V., Hasselmo, M. E., and Alonso, A. A. (2006). Mechanism of graded persistent cellular activity of entorhinal cortex layer v neurons. Neuron 49, 735–746. doi: 10.1016/j.neuron.2006.01.036
Fujisawa, S., Amarasingham, A., Harrison, M. T., and Buzsáki, G. (2008). Behavior-dependent short-term assembly dynamics in the medial prefrontal cortex. Nat. Neurosci. 11, 823–833. doi: 10.1038/nn.2134
Funahashi, S., Bruce, C. J., and Goldman-Rakic, P. S. (1989). Mnemonic coding of visual space in the monkey's dorsolateral prefrontal cortex. J. Neurophysiol. 61, 331–349. doi: 10.1152/jn.1918.104.22.1681
Gordleeva, S. Y., Ermolaeva, A. V., Kastalskiy, I. A., and Kazantsev, V. B. (2019). Astrocyte as spatiotemporal integrating detector of neuronal activity. Front. Physiol. 10:294. doi: 10.3389/fphys.2019.00294
Gordleeva, S. Y., Lebedev, S. A., Rumyantseva, M. A., and Kazantsev, V. B. (2018). Astrocyte as a detector of synchronous events of a neural network. JETP Lett. 107, 440–445. doi: 10.1134/s0021364018070032
Gordleeva, S. Y., Stasenko, S. V., Semyanov, A. V., Dityatev, A. E., and Kazantsev, V. B. (2012). Bi-directional astrocytic regulation of neuronal activity within a network. Front. Comput. Neurosci. 6:92. doi: 10.3389/fncom.2012.00092
Halassa, M. M., Fellin, T., Takano, H., Dong, J.-H., and Haydon, P. G. (2007). Synaptic islands defined by the territory of a single astrocyte. J. Neurosci. 27, 6473–6477. doi: 10.1523/jneurosci.1419-07.2007
Han, J., Kesner, P., Metna-Laurent, M., Duan, T., Xu, L., Georges, F., et al. (2012). Acute cannabinoids impair working memory through astroglial CB1 receptor modulation of hippocampal LTD. Cell 148, 1039–1050. doi: 10.1016/j.cell.2012.01.037
Hempel, C. M., Hartman, K. H., Wang, X.-J., Turrigiano, G. G., and Nelson, S. B. (2000). Multiple forms of short-term plasticity at excitatory synapses in rat medial prefrontal cortex. J. Neurophysiol. 83, 3031–3041. doi: 10.1152/jn.2000.83.5.3031
Hodgkin, A., and Huxley, A. (1990). A quantitative description of membrane current and its application to conduction and excitation in nerve. Bull. Math. Biol. 52, 25–71. doi: 10.1016/s0092-8240(05)80004-7
Jourdain, P., Bergersen, L. H., Bhaukaurally, K., Bezzi, P., Santello, M., Domercq, M., et al. (2007). Glutamate exocytosis from astrocytes controls synaptic strength. Nat. Neurosci. 10, 331–339. doi: 10.1038/nn1849
Kanakov, O., Gordleeva, S., Ermolaeva, A., Jalan, S., and Zaikin, A. (2019). Astrocyte-induced positive integrated information in neuron-astrocyte ensembles. Phys. Rev. E 99:012418. doi: 10.1103/physreve.99.012418
Kass, J. I., and Mintz, I. M. (2005). Silent plateau potentials, rhythmic bursts, and pacemaker firing: three patterns of activity that coexist in quadristable subthalamic neurons. Proc. Natl. Acad. Sci. U.S.A. 103, 183–188. doi: 10.1073/pnas.0506781102
Kastanenka, K. V., Moreno-Bote, R., Pittà, M. D., Perea, G., Eraso-Pichot, A., Masgrau, R., et al. (2019). A roadmap to integrate astrocytes into systems neuroscience. Glia 68, 5–26. doi: 10.1002/glia.23632
Kilpatrick, Ermentrout, Kilpatrick, Z. P., Ermentrout, B., and Doiron, B. (2013). Optimizing working memory with heterogeneity of recurrent cortical excitation. J. Neurosci. 33, 18999–19011. doi: 10.1523/jneurosci.1641-13.2013
Koutsikou, S., Merrison-Hort, R., Buhl, E., Ferrario, A., Li, W.-C., Borisyuk, R., et al. (2018). A simple decision to move in response to touch reveals basic sensory memory and mechanisms for variable response times. J. Physiol. 596, 6219–6233. doi: 10.1113/jp276356
Li, Y.-X., and Rinzel, J. (1994). Equations for InsP3 receptor-mediated [ca2+]i oscillations derived from a detailed kinetic model: a Hodgkin-Huxley like formalism. J. Theor. Biol. 166, 461–473. doi: 10.1006/jtbi.1994.1041
Lima, A., Sardinha, V. M., Oliveira, A. F., Reis, M., Mota, C., Silva, M. A., et al. (2014). Astrocyte pathology in the prefrontal cortex impairs the cognitive function of rats. Mol. Psychiatry 19, 834–841. doi: 10.1038/mp.2013.182
Liu, J., McDaid, L. J., Harkin, J., Karim, S., Johnson, A. P., Millard, A. G., et al. (2019). Exploring self-repair in a coupled spiking astrocyte neural network. IEEE Trans. Neural Netw. Learn. Syst. 30, 865–875. doi: 10.1109/tnnls.2018.2854291
Luca, S. N. D., Soch, A., Sominsky, L., Nguyen, T.-X., Bosakhar, A., and Spencer, S. J. (2020). Glial remodeling enhances short-term memory performance in Wistar rats. J. Neuroinflamm. 17:52. doi: 10.1186/s12974-020-1729-4
Lundqvist, M., Herman, P., and Lansner, A. (2011). Theta and gamma power increases and alpha/beta power decreases with memory load in an attractor network model. J. Cogn. Neurosci. 23, 3008–3020. doi: 10.1162/jocn_a_00029
Makovkin, S. Y., Shkerin, I. V., Gordleeva, S. Y., and Ivanchenko, M. V. (2020). Astrocyte-induced intermittent synchronization of neurons in a minimal network. Chaos Solitons Fract. 138:109951. doi: 10.1016/j.chaos.2020.109951
Manohar, S. G., Zokaei, N., Fallon, S. J., Vogels, T. P., and Husain, M. (2019). Neural mechanisms of attending to items in working memory. Neurosci. Biobehav. Rev. 101, 1–12. doi: 10.1016/j.neubiorev.2019.03.017
Mariotti, Losi, Lia, Melone, Chiavegato, Mariotti, L., Losi, G., Lia, A., Melone, M., Chiavegato, A., Gómez-Gonzalo, M., et al. (2018). Interneuron-specific signaling evokes distinctive somatostatin-mediated responses in adult cortical astrocytes. Nat. Commun. 9:82. doi: 10.1038/s41467-017-02642-6
Martin, R., Bajo-Graneras, R., Moratalla, R., Perea, G., and Araque, A. (2015). Circuit-specific signaling in astrocyte-neuron networks in basal ganglia pathways. Science 349, 730–734. doi: 10.1126/science.aaa7945
Miller, E. K., Erickson, C. A., and Desimone, R. (1996). Neural mechanisms of visual working memory in prefrontal cortex of the macaque. J. Neurosci. 16, 5154–5167. doi: 10.1523/jneurosci.16-16-05154.1996
Navarrete, M., Perea, G., de Sevilla, D. F., Gómez-Gonzalo, M., Núñnez, A., Martín, E. D., et al. (2012). Astrocytes mediate in vivo cholinergic-induced synaptic plasticity. PLoS Biol. 10:e1001259. doi: 10.1371/journal.pbio.1001259
Oschmann, F., Berry, H., Obermayer, K., and Lenk, K. (2018). From in silico astrocyte cell models to neuron-astrocyte network models: a review. Brain Res. Bull. 136, 76–84. doi: 10.1016/j.brainresbull.2017.01.027
Ozdemir, A. T., Lagler, M., Lagoun, S., Malagon-Vina, H., Lasztóczi, B., and Klausberger, T. (2020). Unexpected rule-changes in a working memory task shape the firing of histologically identified delay-tuned neurons in the prefrontal cortex. Cell Rep. 30, 1613–1626.e4. doi: 10.1016/j.celrep.2019.12.102
Pankratova, E. V., Kalyakulina, A. I., Stasenko, S. V., Gordleeva, S. Y., Lazarevich, I. A., and Kazantsev, V. B. (2019). Neuronal synchronization enhanced by neuron–astrocyte interaction. Nonlin. Dyn. 97, 647–662. doi: 10.1007/s11071-019-05004-7
Park, J. C., Bae, J. W., Kim, J., and Jung, M. W. (2019). Dynamically changing neuronal activity supporting working memory for predictable and unpredictable durations. Sci. Rep. 9:15512. doi: 10.1038/s41598-019-52017-8
Paukert, M., Agarwal, A., Cha, J., Doze, V. A., Kang, J. U., and Bergles, D. E. (2014). Norepinephrine controls astroglial responsiveness to local circuit activity. Neuron 82, 1263–1270. doi: 10.1016/j.neuron.2014.04.038
Robin, L. M., da Cruz, J. F. O., Langlais, V. C., Martin-Fernandez, M., Metna-Laurent, M., Busquets-Garcia, A., et al. (2018). Astroglial CB1 receptors determine synaptic d-serine availability to enable recognition memory. Neuron 98, 935–944.e5. doi: 10.1016/j.neuron.2018.04.034
Sardinha, V. M., Guerra-Gomes, S., Caetano, I., Tavares, G., Martins, M., Reis, J. S., et al. (2017). Astrocytic signaling supports hippocampal-prefrontal theta synchronization and cognitive function. Glia 65, 1944–1960. doi: 10.1002/glia.23205
Shafi, M., Zhou, Y., Quintana, J., Chow, C., Fuster, J., and Bodner, M. (2007). Variability in neuronal activity in primate cortex during working memory tasks. Neuroscience 146, 1082–1108. doi: 10.1016/j.neuroscience.2006.12.072
Stobart, J. L., Ferrari, K. D., Barrett, M. J., Glück, C., Stobart, M. J., Zuend, M., et al. (2018). Cortical circuit activity evokes rapid astrocyte calcium signals on a similar timescale to neurons. Neuron 98, 726–735.e4. doi: 10.1016/j.neuron.2018.03.050
Takata, N., Mishima, T., Hisatsune, C., Nagai, T., Ebisui, E., Mikoshiba, K., et al. (2011). Astrocyte calcium signaling transforms cholinergic modulation to cortical plasticity in vivo. J. Neurosci. 31, 18155–18165. doi: 10.1523/jneurosci.5289-11.2011
Tewari, S., and Parpura, V. (2013). A possible role of astrocytes in contextual memory retrieval: an analysis obtained using a quantitative framework. Front. Comput. Neurosci. 7:145. doi: 10.3389/fncom.2013.00145
Tsodyks, M. V., and Markram, H. (1997). The neural code between neocortical pyramidal neurons depends on neurotransmitter release probability. Proc. Natl. Acad. Sci. U.S.A. 94, 719–723. doi: 10.1073/pnas.94.2.719
Ullah, G., Jung, P., and Cornell-Bell, A. (2006). Anti-phase calcium oscillations in astrocytes via inositol (1, 4, 5)-trisphosphate regeneration. Cell Calcium 39, 197–208. doi: 10.1016/j.ceca.2005.10.009
Wade, J. J., McDaid, L. J., Harkin, J., Crunelli, V., and Kelso, J. A. S. (2011). Bidirectional coupling between astrocytes and neurons mediates learning and dynamic coordination in the brain: a multiple modeling approach. PLoS ONE 6:e29445. doi: 10.1371/journal.pone.0029445
Wang, Y., Markram, H., Goodman, P. H., Berger, T. K., Ma, J., and Goldman-Rakic, P. S. (2006). Heterogeneity in the pyramidal network of the medial prefrontal cortex. Nat. Neurosci. 9, 534–542. doi: 10.1038/nn1670
Whitwell, H. J., Bacalini, M. G., Blyuss, O., Chen, S., Garagnani, P., Gordleeva, S. Y., et al. (2020). The human body as a super network: Digital methods to analyze the propagation of aging. Front. Aging Neurosci. 12:136. doi: 10.3389/fnagi.2020.00136
Wimmer, K., Nykamp, D. Q., Constantinidis, C., and Compte, A. (2014). Bump attractor dynamics in prefrontal cortex explains behavioral precision in spatial working memory. Nat. Neurosci. 17, 431–439. doi: 10.1038/nn.3645
Wu, Y.-W., Gordleeva, S., Tang, X., Shih, P.-Y., Dembitskaya, Y., and Semyanov, A. (2018). Morphological profile determines the frequency of spontaneous calcium events in astrocytic processes. Glia 67, 246–262. doi: 10.1002/glia.23537
Yamamoto, T., Ochalski, A., Hertzberg, E. L., and Nagy, J. I. (1990). On the organization of astrocytic gap junctions in rat brain as suggested by LM and EM immunohistochemistry of connexin43 expression. J. Compar. Neurol. 302, 853–883. doi: 10.1002/cne.903020414
Keywords: spiking neural network, astrocyte, neuron-astrocyte interaction, working memory, delayed activity
Citation: Gordleeva SY, Tsybina YA, Krivonosov MI, Ivanchenko MV, Zaikin AA, Kazantsev VB and Gorban AN (2021) Modeling Working Memory in a Spiking Neuron Network Accompanied by Astrocytes. Front. Cell. Neurosci. 15:631485. doi: 10.3389/fncel.2021.631485
Received: 20 November 2020; Accepted: 04 March 2021;
Published: 31 March 2021.
Edited by:Yu-Wei Wu, Academia Sinica, Taiwan
Reviewed by:Randy Franklin Stout, New York Institute of Technology, United States
Mahmood Amiri, Kermanshah University of Medical Sciences, Iran
Jun Ma, Lanzhou University of Technology, China
Copyright © 2021 Gordleeva, Tsybina, Krivonosov, Ivanchenko, Zaikin, Kazantsev and Gorban. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Susanna Yu. Gordleeva, firstname.lastname@example.org