Computational Models for Calcium-Mediated Astrocyte Functions

The computational neuroscience field has heavily concentrated on the modeling of neuronal functions, largely ignoring other brain cells, including one type of glial cell, the astrocytes. Despite the short history of modeling astrocytic functions, we were delighted about the hundreds of models developed so far to study the role of astrocytes, most often in calcium dynamics, synchronization, information transfer, and plasticity in vitro, but also in vascular events, hyperexcitability, and homeostasis. Our goal here is to present the state-of-the-art in computational modeling of astrocytes in order to facilitate better understanding of the functions and dynamics of astrocytes in the brain. Due to the large number of models, we concentrated on a hundred models that include biophysical descriptions for calcium signaling and dynamics in astrocytes. We categorized the models into four groups: single astrocyte models, astrocyte network models, neuron-astrocyte synapse models, and neuron-astrocyte network models to ease their use in future modeling projects. We characterized the models based on which earlier models were used for building the models and which type of biological entities were described in the astrocyte models. Features of the models were compared and contrasted so that similarities and differences were more readily apparent. We discovered that most of the models were basically generated from a small set of previously published models with small variations. However, neither citations to all the previous models with similar core structure nor explanations of what was built on top of the previous models were provided, which made it possible, in some cases, to have the same models published several times without an explicit intention to make new predictions about the roles of astrocytes in brain functions. Furthermore, only a few of the models are available online which makes it difficult to reproduce the simulation results and further develop the models. Thus, we would like to emphasize that only via reproducible research are we able to build better computational models for astrocytes, which truly advance science. Our study is the first to characterize in detail the biophysical and biochemical mechanisms that have been modeled for astrocytes.


INTRODUCTION
Astrocytes have traditionally been regarded as glial cells responsible for the homeostasis and metabolic support for neurons (Carmignoto and Gómez-Gonzalo, 2010). Current evidence indicates that astrocytes support neuronal functions also in many other ways. Astrocytes are in a close proximity to a large number of brain synapses (Bushong et al., 2002;Fellin et al., 2006;Halassa et al., 2007;Oberheim et al., 2009;Freeman, 2010;Volterra et al., 2014;Olude et al., 2015;Cali et al., 2016), and the astrocytes together with pre-and postsynaptic neurons have been proposed to form functional tripartite synapses (Araque et al., 1999). Astrocytes can thus broadly sense and regulate synaptic activity. A growing body of evidence also suggests that astrocytes modulate neural circuits and are involved in many brain functions, perhaps also in cognitive functions and behavior (Perea et al., 2009;Wang et al., 2009;Halassa and Haydon, 2010;Henneberger et al., 2010;Suzuki et al., 2011;Araque et al., 2014;Sibille et al., 2014;Khakh and Sofroniew, 2015;Poskanzer and Yuste, 2016). Although more and more evidence is accumulating on the astrocytic modulation of neurotransmission and synaptic plasticity, in vivo evidence to support that astrocytes are directly activated by neurotransmission and signal back to neurons to modulate neurons' output remains unclear. Particularly, how this modulation occurs in time and space is unresolved.
Since the 1990s a variety of experimental techniques and organisms have been used to study astrocytes. Until 2010 most of the studies were performed using in vitro cell cultures and slice preparations. Recently, studies addressing astrocytes' roles in brain functions in vivo have accumulated. In short, one could identify three waves of astrocyte research over the past three decades, as proposed by Bazargani and Attwell (2016). The first wave of evidence revealed that neurotransmitter glutamate increases the astrocytic calcium (Ca 2+ ) concentration in vitro and this yields to Ca 2+ wave propagation between astrocytes (Cornell-Bell et al., 1990;Charles et al., 1991;Dani et al., 1992;Newman and Zahs, 1997), which could lead to Ca 2+ increase in the nearby neurons (Nedergaard, 1994;Parpura et al., 1994). The second wave of evidence showed that pharmacological tools used to separate astrocytic and neuronal components are not selective (Parri et al., 2001;Agulhon et al., 2010;Hamilton and Attwell, 2010). Furthermore, it was speculated that astrocytic processes close to synapses do not have endoplasmic reticulum (ER) present and that blocking the inositol trisphosphate (IP 3 ) receptors (IP 3 Rs) in the astrocytes has an effect on the astrocytic Ca 2+ but not on the synaptic events (Fiacco et al., 2007;Petravicz et al., 2008;Agulhon et al., 2010;Patrushev et al., 2013). The third wave of evidence (Bazargani and Attwell, 2016) led to the conclusion that the Ca 2+ transients in the astrocytic processes near vascular capillaries (Otsu et al., 2015) and neuronal synapses  and not in the soma are the key that needs to be addressed in more detail. In summary, the challenges in astrocyte research have been the lack of selective pharmacological tools and the partially contradictory results obtained in in vivo in contrast to various in vitro preparations.
Although there is partial controversy, which hinders attempts to explain all findings on astrocytes' roles in the central nervous system in an unambiguous way, the majority of data collected over the past decades strongly suggests that fluctuations in Ca 2+ concentrations in both soma and processes are important measures of astrocytic activities. Then astrocytic Ca 2+ activity is utilized, in one way or another, by neurons to sense ongoing neural activity in closeby or more distant networks. The dynamic, far-reaching fluctuations, or transients, in astrocytic Ca 2+ concentration were also recently recorded in awake behaving mice in vivo by several independent studies (Ding et al., 2013;Paukert et al., 2014;Srinivasan et al., 2015). Moreover, astrocytes, similarly to any other cell in the mammalian body, are known to express an overwhelming complexity of molecular and celllevel signaling. The full complexity of the signaling pathways which control Ca 2+ transients and exert their effects in astrocytes is poorly understood, and the question about their relevance in awake behaving animals remains unanswered. It is essential that the research community seeks to systematically characterize the key signaling mechanisms in astrocytes to understand the interactions between different systems, including neuronal, glial, and vascular, in brain circuitry. Astrocytic signaling may provide a potential, widespread mechanism for regulating brain functions and states (Yang et al., 2014;Haim and Rowitch, 2017).
Several factors might be important in orchestrating how astrocytes exert their functional consequences in the brain. These include (a) different receptors or other mechanisms that trigger an increase in Ca 2+ concentration in astrocytes, (b) Ca 2+ -dependent signaling pathways or other mechanisms that govern the production and release of different mediators from astrocytes, and (c) released substances that target other glial cells, the vascular system, and the neuronal system. The listed three factors (a-c) operate at different temporal and spatial scales and depend on the developmental stage of an animal and on the location of astrocytes. Namely, a substantial amount of data on a diverse array of receptors to detect neuromodulatory substances in astrocytes in vitro has been gathered (Backus et al., 1989;Kimelberg, 1995;Jalonen et al., 1997), and accumulating evidence is becoming available for in vivo organisms as well (Beltrán-Castillo et al., 2017). Neuromodulators have previously been expected to act directly on neurons to alter neural activity and animal behavior. It is, however, possible that at least part of the neuromodulation is directed through astrocytes, thus contributing to the global effects of neurotransmitters (see e.g., Ma et al., 2016). Experimental manipulation of astrocytic Ca 2+ concentration is not a straightforward practice and can produce different results depending on the approach and context (for more detailed discussion, see e.g., Agulhon et al., 2010;Fujita et al., 2014;Sloan and Barres, 2014). Additional tools, both experimental and computational, are required to understand the vast complexity of astrocytic Ca 2+ signaling and how it is decoded to advance functional consequences in the brain.
Several reviews of theoretical and computational models have already been presented (for a review, see e.g., Jolivet et al., 2010;Mangia et al., 2011;De Pittà et al., 2012;Fellin et al., 2012;Min et al., 2012;Volman et al., 2012;Wade et al., 2013;Linne and Jalonen, 2014;Tewari and Parpura, 2014;Manninen et al., 2018). We found out in our previous study (Manninen et al., 2018) that most astrocyte models are based on the models by De Young and Keizer (1992), Li and Rinzel (1994), and Höfer et al. (2002), of which the model by Höfer et al. (2002) is the only one built specifically to describe astrocytic functions and data obtained from astrocytes. Some of the other computational astrocyte models that steered the field are the models by Nadkarni and Jung (2003), Bennett et al. (2005), Volman et al. (2007), De Pittà et al. (2009a), Postnov et al. (2009), and Lallouette et al. (2014). However, irreproducible science, as we have reported in our other studies, is a considerable problem also among the developers of the astrocyte models (Manninen et al., 2017(Manninen et al., , 2018Rougier et al., 2017). Several other review, opinion, and commentary articles have addressed the same issue as well (see e.g., Cannon et al., 2007;De Schutter, 2008;Nordlie et al., 2009;Crook et al., 2013;Topalidou et al., 2015;McDougal et al., 2016). We believe that only via reproducible science are we able to build better computational models for astrocytes and truly advance science.
This study presents an overview of computational models for astrocytic functions. We only cover the models that describe astrocytic Ca 2+ signaling by biophysical means. We first categorize the existing models based on whether they are describing astrocytes or neuron-astrocyte interactions. We have previously described some aspects of the astrocyte and neuronastrocyte models in our related study (Manninen et al., 2018), where we listed the details of both the astrocyte and neuron models in a simplistic, educational manner. In this review, we characterize in detail the existing models based on what kind of astrocytic mechanisms have been taken into account. Our study is expected to help guide future computational studies addressing the cross-talk between astrocytes and other systems in the brain and help researchers select suitable models for their research questions.

MATERIALS AND METHODS
In this section, we first outline the basics of astrocyte biology. The mechanisms presented here are not typically included in computational neuroscience models and one of our aims is to carefully assess which of the mechanisms are presently modeled and how realistically. We then list the computational astrocyte models, developed by the end of 2017, for our detailed evaluation. At the end of the section, we give the details how we characterized the models.

Astrocyte Biophysics and Biochemistry for Modeling of Astroglial Functions
Astrocytes are the most diverse glial cells in the central nervous system. Astrocytes from different brain regions differ in morphology, physiology, and expression of genes encoding the most fundamental proteins responsible for astroglial function. In general, astrocytes can have a soma, perisynaptic processes which engulf neuronal synapses and also enclose some extracellular space, called perisynaptic or extrasynaptic (or sometimes periastrocytic) space within, and perivascular processes which connect the astrocyte with blood vessels and enclose some extracellular space called perivascular space. Below we present a generic view of some of the most important biophysical and cellular mechanisms that are shown to underlie important astrocytic functions (for more information, see also Kettenmann and Ransom, 2013;Verkhratsky and Butt, 2013).

Ion Distribution and Ion Channels for Basic Membrane Excitability
Astroglial cells express all major ion channel types, including potassium (K + ), sodium (Na + ), and Ca 2+ channels, and also various types of anion and chloride (Cl − ) channels, water channels (aquaporins), transient receptor potential (TRP) channels, and non-selective channels. The ion distribution is also somewhat different from neurons: intracellular concentrations of K + and Ca 2+ are similar to neurons, but the concentrations of Na + and especially Cl − are higher compared to neurons. Astrocytes have a rather negative resting membrane potential (around −80 to −90 mV) because of the predominance of K + conductance. Electrical depolarization of astroglia does not produce regenerative action potentials as in neurons.
Ca 2+ -mediated signals have been proposed to be the main mediator of communication between astrocytes and other cellular elements in the brain (Nimmerjahn, 2009;Volterra et al., 2014;Bazargani and Attwell, 2016). Transient Ca 2+ increases restricted to single cells are called Ca 2+ oscillations. In isolated astrocytes, intracellular Ca 2+ oscillations have been shown to depend mainly on the Ca 2+ -induced Ca 2+ release (CICR) via IP 3 Rs from the ER to the cytosol (see e.g., Agulhon et al., 2008). CICR has been shown to depend on Ca 2+ with or without the influence of IP 3 . Ca 2+ influx via voltage-gated Ca 2+ channels (VGCCs) and other types of Ca 2+ influxes from the extracellular space have been linked with Ca 2+ oscillations as well (Aguado et al., 2002).

Membrane Transporters for Uptake and Homeostatic Control of Ions, Neurotransmitters, and Other Substances
The membrane transporters are particularly important for astroglia because they control movements of various substances, including ions, neurotransmitters, and metabolic substrates. Astroglial transporters include adenosine and adenosine triphosphate (ATP)-dependent transporters such as the Na + /K + -ATPase (NKA, also called Na + /K + pump) and Ca 2+ -ATPase [also called Ca 2+ pump or plasma membrane Ca 2+ -ATPase (PMCA)] on the plasma membrane, in addition to sarco/ER Ca 2+ -ATPase (SERCA) located on the ER membrane. They also contain so-called secondary transporters, such as glutamate transporters [excitatory amino acid transporters (EAATs)], gamma-aminobutyric acid (GABA) transporters, glycine transporters, Na + /Ca 2+ exchangers (NCXs), Na + /hydrogen (H + ) exchangers, Na + /bicarbonate (HCO − 3 ) cotransporters, Na + /K + /Cl − cotransporters (NKCC1), and some others. Although, for example, glutamate transporters are expressed by all cell types in the brain, astrocytes are the main cell type responsible for glutamate uptake. Astrocytes have enzymes that convert both glutamate and GABA into glutamine. Glutamine is then released into the extracellular space and taken up by the presynaptic terminal, and can be converted to glutamate or GABA.
The NKCC1 cotransporter specifically contributes to the regulation of extracellular K + homeostasis in the central nervous system. During excessive neuronal firing, the local extracellular K + concentration can increase markedly and lead to neural dysfunction. Uptaking extracellular K + , transporting it intracellularly and releasing to areas with lower concentration are some of the most important functions of astrocytes. This process of intracellularly transporting K + from extracellular areas of high concentration to areas of low concentration, via active uptake and release, is called spatial K + buffering. K + can also diffuse in extracellular space which also helps in lowering the local high concentration. Furthermore, astrocytes also buffer other ions than K + .

Ionotropic and Metabotropic Receptors for Sensing the Environment
Astrocytes are able to sense with the help of their receptors local and more distant environments, including the neural activity and the synaptically released neurotransmitters, both in in vitro cell cultures and brain slices as well as in in vivo (Glaum et al., 1990;Dani et al., 1992;Porter and McCarthy, 1996;Hirase et al., 2004). Astroglial cells can express the same variety of receptors as neurons, both ionotropic and metabotropic. These include glutamate, GABA, glycine, acetylcholine, adrenergic, serotonin, histamine, cannabinoid, and neuropeptide receptors, and purinoceptors for adenosine and ATP. For example, extracellular ATP can bind to astroglial purinergic G-protein receptors, resulting in IP 3 -mediated CICR, and, eventually, release of ATP to the extracellular space.

Release of Transmitters and Modulators
As described above, astroglia are capable of receiving signals from neurons via membrane receptors and converting the received information into Ca 2+ excitability. The fact that astrocytes could release extracellular signaling molecules, regulated by this Ca 2+ excitability, not only to the vascular system but also to the neuronal one implies a very active role for astroglia in the brain. The concept of regulated transmitter/modulator release from astrocytes to neurons is generally known as gliotransmission (Parpura et al., 1994;Bezzi and Volterra, 2001), and the released substances are referred to as gliotransmitters. Most common gliotransmitters are glutamate, D-serine, and ATP (Parpura et al., 1994;Parri et al., 2001). Astrocytes are, in general, thought to release transmitters, and probably also modulators, by several different mechanisms, which can be broadly divided into exocytotic release, diffusional release through membrane pores, and transporter mediated release. It is, however, not known which of these mechanisms are used in certain astroglial functions.
These terms "gliotransmission" and "gliotransmitter" may be somewhat misleading as the very same transmitters are also released by neurons. Moreover, it is assumed that a mechanism similar to vesicular release from neuronal synaptic terminals exists also in astrocytes. Some studies have indeed detected vesicle-type structures in astrocytes in vitro, and thus it has been proposed that gliotransmitter release might be purely vesicular. It is, however, important to bear in mind that the existence of molecular machinery for packing gliotransmitters into vesicles and for arranging the vesicles into a readily releasable pool has not so far been supported by experimental evidence in vivo (see e.g., Fujita et al., 2014;Sloan and Barres, 2014). More evidence on the release mechanism, using improved experimental model systems and techniques that allow studies at deeper resolution in physiological conditions, is required (Li et al., 2013;Bazargani and Attwell, 2016;Fiacco and McCarthy, 2018;Savtchouk and Volterra, 2018).
In our evaluation of models, we use the term "gliotransmission" for all biophysical and phenomenological mechanisms that were modeled to take into account the release of substances from astrocytes and targeting neurons. The reason for this is that the term "gliotransmission" is often used in the original modeling publications.
In addition, glutamate released from astrocytes can activate extrasynaptic N-methyl-D-aspartate receptor (NMDAR)dependent currents, often called NMDAR-dependent slow inward current (SIC). In modeling studies, SIC is many times modeled similarly to, for example, the modulating current (I astro ) presented by Nadkarni and Jung (2003).

Connexin-Based Gap Junction Hemichannels
It is not just neurons that form networks but also astrocytes. A fundamental difference between neuronal and astroglial networks is that astrocytes are connected, through gap junctions composed mainly of connexin 43 hemichannels, to form a functional cellular syncytium in the central nervous system. In their open state, these channels are permeable to large hydrophilic solutes with molecular mass of several hundred Daltons, and are permeable to small solutes in their closed state (see e.g., Bao et al., 2007). The gap junction connectivity is instrumental for astrocytes' functions, including generation of Ca 2+ waves, water transport, K + buffering, and control of vascular system, and is one of the most important mechanisms to be modeled in astrocyte networks.
Three different pathways have been discovered so far to induce Ca 2+ waves in astroglial networks. The first route depends on the transfer of IP 3 via gap junctions (Giaume and Venance, 1998). Transported IP 3 via gap junctions triggers CICR in the coupled astrocytes and induces Ca 2+ wave propagation in astroglial syncytium. The second route to induce Ca 2+ waves depends on the extracellular diffusion of ATP (see e.g., Newman and Zahs, 1997;Guthrie et al., 1999 and section 2.1.4). The third route has been shown to depend on extracellularly applied potassium chloride, causing, among others, a pathophysiological phenomenon called cortical spreading depression (Peters et al., 2003). The regulation of gap junction communication within the astroglial syncytium is a complex process and is intensively studied.
Most of the above described biophysical and biochemical mechanisms have been modeled in some detail in astrocytes. Below we address altogether 106 models developed until the end of 2017 and describe their capacity to represent the dynamics of astrocyte biophysics and biochemistry.

Selection of Models
The number of computational astrocyte models has increased over the past couple of years. This increase in the number of models does not, however, solely reflect the vast amount of experimental data that is currently presented for astrocytes and for their roles in brain functions and the regulation of the neuronal system. Several focused reviews of computational astrocyte and neuron-astrocyte models have appeared during the last few years (see e.g., Jolivet et al., 2010;Mangia et al., 2011;De Pittà et al., 2012;Fellin et al., 2012;Min et al., 2012;Volman et al., 2012;Wade et al., 2013;Linne and Jalonen, 2014;Tewari and Parpura, 2014;Manninen et al., 2018); of which our study (Manninen et al., 2018) is the most comprehensive evaluation of more than 60 models published by the end of 2014. In the present study, we characterize in more detail the biophysical and biochemical components of astrocytes that were taken into account in the astrocyte and neuron-astrocyte interaction models published by the end of 2017. Table 1 presents altogether 106 astrocyte models. As in our other study (Manninen et al., 2018), we here limited our evaluation of models to astrocytic signal transduction pathways that were defined using several characteristics. First, models were able to include pre-and postsynaptic neuron models as part of the whole models. Second, part of intracellular signaling in the astrocytes was explicitly modeled, thus models were required to include (biophysical) mechanisms for astrocytic Ca 2+ dynamics. We considered in the present study only models where astrocytic Ca 2+ signaling was described by a differential equation that was a function of time and at least one of the other astrocytic variables, for example IP 3 . Third, astrocytic Ca 2+ affected some signaling variables or other intracellular signals in the astrocytes. Models which described Ca 2+ dynamics but were not explicitly made for astrocytes were excluded from the present study. Moreover, models that mainly concentrated on describing ionic homeostasis, such as regulation of extracellular K + ions, were also excluded from the evaluation unless they incorporated astrocytic Ca 2+ signaling. These strict criteria were needed because of the large number of models.
Next, we characterized the models based on what type of signaling events were modeled in astrocytes. We listed for each model the number of astrocytes modeled, input used for the astrocytic module, astrocytic variables described by differential equations, and astrocytic Ca 2+ fluxes related to cytosolic Ca 2+ . It is important to note that neurotransmitters were not named here as astrocytic variables but in some models they were used as inputs to the astrocyte models [e.g., synaptic glutamate ([Glu] syn ) or neurotransmitter (NT)]. By contrast, we listed gliotransmitters as astrocytic variables if they were modeled with a differential equation. As the exact mechanisms of astrocytic signaling that regulate exocytosis and release of molecular substances from astrocytes are still largely unknown, most of the models used phenomenological descriptions of gliotransmitters or other substances released. Proposed gliotransmitters included, for example, glutamate ([Glu] ext ) and ATP ([ATP] ext ). Note the difference between [Glu] syn and [Glu] ext , the former meaning the neurotransmitter glutamate released from the presynaptic neuron and the latter meaning the gliotransmitter glutamate released from the astrocyte. Other examples of strategies for modeling gliotransmission or similar exocytotic or modulating signaling mechanisms from astrocytes to neurons included modeling currents depending on astrocytic Ca 2+ (I astro , Nadkarni and Jung, 2003) and several other types of currents depending, for example, on astrocytic mediator (G m , I ast , Postnov et al., 2007Postnov et al., , 2009, or modeling phenomenological gating variable (f , Volman et al., 2007). In addition, we listed diffusion of astrocytic variables either in the cytosol, ER, or extracellular space. In the present study, we did not catalog the transfer of molecules or ions between intracellular space and extracellular space under the attribute "diffusion, " but we listed, for example, different Ca 2+ related fluxes over the plasma membrane under "Ca 2+ fluxes." We chose to list gap junction signaling between astrocytes which is an important, fundamental form of intercellular signaling directly from the intracellular space of one cell to the intracellular space of another cell. We did not incorporate gap junctions under the attribute "diffusion, " but only included it in the attribute "gap junction." IP 3 and Ca 2+ are examples of second messengers that are able to pass through gap junctions. The last items we listed were the output of the astrocytic module and experimentally shown event that the model was finetuned to capture [Ca 2+ dynamics (Ca 2+ ), homeostasis (Home.), vascular (Vasc.), synchronization (Synch.), information transfer (Inf.), plasticity (Plast.), and hyperexcitability (Hyper.)] for each model. To get a general idea of how these different mechanisms can be modeled, see, for example, books by Keener and Sneyd (2009) and Dupont et al. (2016).
Sometimes the authors of the publications used incorrect terminology when explaining their model components. As an example, Silchenko and Tass (2008) and Ding et al. (2018) presented one Ca 2+ flux as a leak from cytosol into extracellular space in their study. We named it here as Ca 2+ efflux because Ca 2+ leak flux via astrocytic cell membrane is normally used for a flux from extracellular space into cytosol, thus from the larger Ca 2+ concentration in the extracellular space toward the smaller Ca 2+ concentration in the cytosol. Goto et al. (2004) named the direction of fluxes incorrectly for CICR, ER leak flux, and ER pump (that we assumed to be SERCA) in their text, but in their equations the direction of fluxes were marked correctly for CICR and leak and again incorrectly for ER pump. Yao et al. (2018) also marked the signs incorrectly for some of the fluxes. Directions of the CICR and ER leak flux should be from the ER to the cytosol, and SERCA pump from the cytosol to the ER. Often the authors of the modeling publications did not name their Ca 2+ pump on the membrane of ER as SERCA pump, but since the very same equation was used for SERCA pump by other authors, we marked it as SERCA pump for all the authors, except for the model by Guo et al. (2017) because they pointed out that their pump was ATP-independent. Naming of the variable h, taken from the model by Li and Rinzel (1994), was also presented in a contradictory manner by several authors. The very same variable h was given alternative explanations, some of them having completely opposite meanings, or no explanation at all. We decided to name h in this work as active fraction of IP 3 Rs. Authors did not always state clearly how many astrocytes were modeled. In these cases we marked it as n/a. Some authors named gliotransmitters in their work as neurotransmitters (see e.g., Stamatakis and Mantzaris, 2006). We tried our best to give as correct a view of the models as possible.
Most of the astrocyte models presented here are based on ordinary differential equations (ODEs), modeling well-stirred astrocytes (see e.g., Nadkarni and Jung, 2003;Ullah et al., 2006;Volman et al., 2007;De Pittà et al., 2009a;Goldberg et al., 2010;Dupont et al., 2011;Valenza et al., 2011;Amiri et al., 2012a;Wade et al., 2012;Hadfield et al., 2013;Tang et al., 2013;Lallouette et al., 2014;Wallach et al., 2014;Li et al., 2016a;Chan et al., 2017;Guo et al., 2017;Handy et al., 2017;Taheri et al., 2017). These models make the assumption that chemical species have the same concentrations throughout the entire volume of the astrocyte, which can also be called as average concentrations. However, it has been shown that the concentration of certain chemical species, such as Ca 2+ , can vary drastically in different parts of the cell (see e.g., Thul, 2014;Dupont et al., 2016), and several models took into account the spatiality by modeling diffusion of at least some of the chemical species in astrocytes using partial differential equations (PDEs, see e.g., Roth et al., 1995;Höfer et al., 2002;Bennett et al., 2008a;Gibson et al., 2008;Allegrini et al., 2009;Kang and Othmer, 2009;Edwards and Gibson, 2010;Wei and Shuai, 2011;Li et al., 2012;López-Caamal et al., 2014;Mesiti et al., 2015a;Yao et al., 2018). Some models took a simpler approach to spatiality by modeling ODEs combined with fluxes between different astrocytic compartments, such as cytosol and ER (see e.g., Larter and Craig, 2005;Di Garbo et al., 2007;Postnov et al., 2007;Lavrentovich and Hemkin, 2008;Di Garbo, 2009;Zeng et al., 2009;Amiri et al., 2011a;DiNuzzo et al., 2011;Farr and David, 2011;Oschmann et al., 2017;Kenny et al., 2018). In addition of modeling Ca 2+ fluxes between ER and cytosol, Silchenko and Tass (2008) modeled free diffusion of extracellular glutamate as a flux. It seems that most of the authors implemented their ODE and PDE models using some programming language, but a few times, for example, XPPAUT (Ermentrout, 2002) was named as the simulation tool used. Because of the stochastic nature of cellular processes (see e.g., Rao et al., 2002;Raser and O'Shea, 2005;Ribrault et al., 2011) and oscillations (see e.g., Perc et al., 2008;Skupin et al., 2008), different stochastic methods have been developed for both reaction and reactiondiffusion systems. These stochastic methods can be divided into discrete and continuous-state stochastic methods. Some discretestate reaction-diffusion simulation tools can track each molecule individually in a certain volume with Brownian dynamics combined with a Monte Carlo procedure for reaction events (see e.g., Stiles and Bartol, 2001;Kerr et al., 2008;Andrews et al., 2010). On the other hand, the discrete-state Gillespie stochastic simulation algorithm (Gillespie, 1976(Gillespie, , 1977 and τleap method (Gillespie, 2001) can be used to model both reaction and reaction-diffusion systems. A few simulation tools already exist for reaction-diffusion systems using these methods (see e.g., Wils and De Schutter, 2009;Oliveira et al., 2010;Hepburn et al., 2012). In addition, continuous-state chemical Langevin equation (Gillespie, 2000) and several other stochastic differential equations (SDEs, see e.g., Shuai and Jung, 2002;Manninen et al., 2006a,b) have been developed for reactions to ease the computational burden of discrete-state stochastic methods. A few simulation tools providing hybrid approaches also exist and they combine either deterministic and stochastic methods or different stochastic methods (see e.g., Salis et al., 2006;Lecca et al., 2017). Of the above-named methods, most realistic simulations are provided by the discrete-state stochastic reactiondiffusion methods, but none of the covered astrocyte models used these stochastic methods or available simulation tools for both reactions and diffusion for the same variable. However, four models combined stochastic reactions with deterministic diffusion in the astrocytes. Skupin et al. (2010) and Komin et al. (2015) modeled with the Gillespie algorithm the detailed IP 3 R model by De Young and Keizer (1992), had PDEs for Ca 2+ and mobile buffers, and ODEs for immobile buffers. Postnov et al. (2009) modeled diffusion of extracellular glutamate and ATP as fluxes, had an SDE for astrocytic Ca 2+ with fluxes between ER and cytosol, and ODEs for the rest. MacDonald and Silva (2013) had a PDE for extracellular ATP, an SDE for astrocytic IP 3 , and ODEs for the rest. In addition, a few studies modeling just reactions and not diffusion used stochastic methods (SDEs or Gillespie algorithm) at least for some of the variables (see e.g., Nadkarni et al., 2008;Postnov et al., 2009;Sotero and Martínez-Cancino, 2010;Riera et al., 2011a,b;Toivari et al., 2011;Tewari and Majumdar, 2012a,b;Liu and Li, 2013a;Tang et al., 2016;Ding et al., 2018).

RESULTS
Previous studies in experimental and computational cell biology fields have guided the development of computational models for astrocytes and their interactions with neurons. Most of the firstly developed astrocyte models were relatively simplistic but they were gradually expanded to cover astrocytic regulation of a variety of phenomena and cells in the nervous system. Next, we will present the computational models for astrocytes in section 3.1 and the computational models that include bidirectional signaling between neurons and astrocytes in section 3.2.

Computational Astrocyte Models
The early phase of model development concentrated more on single astrocytes and astrocyte-astrocyte communication. We will go through the single astrocyte models in section 3.1.1 and the astrocyte network models in section 3.1.2.
The first models developed in this category were the models by Roth et al. (1995) and Larter and Craig (2005). Roth et al. (1995) extended the model by Li and Rinzel (1994) by adding Ca 2+ diffusion in the cytosol and ER. They also divided their cell to alternating active and passive compartments. In passive compartments, Ca 2+ signal propagated by diffusion, whereas in active compartments there were also additional Ca 2+ fluxes propagating the signal. Roth et al. (1995) showed with their model that different parts of the astrocyte, such as the ER membrane, produced different frequencies of Ca 2+ oscillations if the diffusion constant had a value close to the physiological range. If the diffusion constant had unphysiologically high value, then the different parts of the astrocyte all oscillated with the same frequency. Larter and Craig (2005) presented the only model in this category that included a differential equation for extracellular glutamate ([Glu] ext ). Their equation for extracellular glutamate included the rate of glutamate vesicle release from the astrocyte, the input glutamate rate arriving from nearly synapses, and two terms describing the rates of glutamate uptake by neurons and astrocytes. Larter and Craig (2005) tested their model by setting the extracellular glutamate to zero during the simulations (meaning both glutamate release from the astrocyte and glutamate arriving from the nearby synapses were zero) and showed that the astrocytic Ca 2+ was not oscillating in this case. They further tested their model when the astrocyte was not releasing glutamate, but they still had the input glutamate rate arriving from nearly synapses. In this case, the astrocytic Ca 2+ was oscillating. They also tested how the number of glutamate in the astrocytic vesicles affected the model behavior. They showed that when increasing the number of glutamate in the astrocytic vesicles, the Ca 2+ oscillations transformed into Ca 2+ bursting oscillations. Thus, their message was that the presence of glutamate, but not the positive astrocytic glutamate feedback, was required for astrocytic Ca 2+ oscillations. In addition, the positive glutamate feedback process made it possible for the astrocyte to have bursting oscillations when the input glutamate rate arriving from nearby synapses had low values.
Only six of the single astrocyte models had K + concentration modeled in the perisynaptic space, perivascular space [in Table 2 called as extracellular space (ext)], or intracellular space of the astrocyte. These are marked in Table 2  This table shows the following: model, number (No.) of astrocytes modeled, input for the astrocytes, astrocytic variables described by differential equations, astrocytic Ca 2+ fluxes related to cytosolic Ca 2+ , diffusion of astrocytic variables either in the cytosol, ER, or extracellular space, not between these spaces, gap junction signaling (GJ) between astrocytes, output of the astrocytes, and experimentally shown event that the model was finetuned to capture [Ca 2+ dynamics ( (Hodgkin and Huxley, 1952). In addition, Ding et al. (2018) simulated two models, with and without detailed IP [K + ], respectively. The model by Diekman et al. (2013) had the most simple K + model, they modeled only astrocytic K + that depended on NKA. Oschmann et al. (2017) modeled both the astrocytic and extracellular K + concentrations. The astrocytic K + concentration depended on three transmembrane transporters; glutamate transporter, NKA, and NCX. The extracellular K + concentration was a function of astrocytic K + concentration. Farr and David (2011) and Witthoft and Karniadakis (2012) modeled the K + concentration both in the perisynaptic space [named as synaptic cleft and synaptic space by Farr and David (2011) and Witthoft and Karniadakis (2012)] and in the perivascular space. They both used the same simple model for the K + concentration in the perisynaptic space. K + was released from the neurons into the perisynaptic space and removed from the perisynaptic space via the astrocytic NKA (Farr and David, 2011). However, Witthoft and Karniadakis (2012) named the removal so that it included both astrocytic NKA and astrocytic inwardly rectifying K + -and voltage-gated K + (KIR) channel. Farr and David (2011) and Witthoft and Karniadakis (2012) also used the same simple model for the K + concentration in the perivascular space. The K + concentration in the perivascular space depended on astrocytic voltage-and Ca 2+ -operated K + (BK) channels and smooth muscle cell's KIR channels. They also had a K + -dependent decay term in the perivascular space. Witthoft et al. (2013) modeled in more detail the K + concentration in the perisynaptic space (named as synaptic space by Witthoft et al., 2013), intracellular space of the astrocyte, and perivascular space. In the model by Witthoft et al. (2013), the K + concentration in the perisynaptic space depended on K + released from the neurons and removed via the astrocytic NKCC1 and NKA, in addition to (bidirectional) astrocytic KIR channels and a decay term. The astrocytic K + concentration depended on K + entering from the perisynaptic space via NKA and NKCC1, in addition to astrocytic KIR channels on the perisynaptic side as well as astrocytic BK and KIR channels on the perivascular side of the astrocyte. The equation also had a decay term. The K + concentration in the perivascular space depended on astrocytic BK and KIR channels as well as arteriolar KIR channels and a decay term. Kenny et al. (2018) modeled the K + concentration in the perisynaptic space (named as synaptic cleft by Kenny et al., 2018), intracellular space of the astrocyte, perivascular space, intracellular space of the smooth muscle cell, and extracellular space. In the model by Kenny et al. (2018), the K + concentration in the perisynaptic space depended on K + released from the neuron and removed via the astrocytic K + /Cl − cotransporter (KCC1), NKCC1, and NKA, in addition to K + diffusion between extracellular space and perisynaptic space as well as astrocytic K + channels. The astrocytic K + concentration depended on K + entering from the perisynaptic space via KCC1, NKCC1, and NKA, in addition to K + channels on the perisynaptic side and BK channels on the perivascular side of the astrocyte. The K + concentration in the perivascular space depended on astrocytic BK channels and smooth muscle cell's KIR channels. In conclusion, only the model by Witthoft et al. (2013) took into account spatial K + buffering. Some of the most recent models developed in this category were the models by Komin et al. (2015), Handy et al. (2017), and Taheri et al. (2017). Komin et al. (2015) presented two models, a reaction-diffusion model and a reaction model. With both models they tested if the temperature-dependent SERCA activity was the reason for the differences in Ca 2+ activity. They showed that their reaction-diffusion model behaved similarly to the experimental data, thus increased SERCA activity (higher temperature) led to decreased Ca 2+ activity. On the other hand, their reaction model showed the opposite. Thus, they claimed that spatiality was needed to be taken into account to get biologically correct results. However, since the core models were different in the reaction-diffusion and reaction models, it would be interesting to see how the results would look like if the same core model was tested with and without diffusion. Handy et al. (2017) and Taheri et al. (2017) used the same model but explored somewhat different parameter spaces. They studied the role of SOC channels as well as the PMCA and SERCA pumps in Ca 2+ activity. They specifically tested which form the Ca 2+ response had with different parameter values of the channel and pumps (single peak, multiple peaks, plateau, or long-lasting response). They found out that SOC channels were necessary for plateau and long-lasting responses as well as for stable oscillations with multiple peaks. Stable oscillations disappeared when the SERCA pump was partially blocked, but plateau and long-lasting responses were still present. The likelihood of having multiple peaks increased when the PMCA pump was blocked. Taheri et al. (2017) also did Ca 2+ imaging on cortical astrocytes in mice. They applied ATP on acute brain slices and recorded the Ca 2+ responses from different subcompartments of the astrocytes, from soma as well as from large and short processes, and categorized the results into four different types of responses named above. Their conclusion was that the variability mainly stemmed from differences in IP 3 dynamics and Ca 2+ fluxes via SOC channels. To take into account the experimental variability between the different subcompartments, Taheri et al. (2017) ran simulations with different parameter values of the SOC channel and the PMCA and SERCA pumps together with the input IP 3 kinetics. Next, they chose the parameter subspaces that matched their experimental data the best. Their simulations suggested that the soma had higher PMCA and lower SERCA flux rates as well as shorter rise duration for the IP 3 transient than the small and large processes.

Astrocyte Network Models
Half of the astrocyte network models were so-called generic. Others, however, were specified to model astrocytes in the cerebrum (Iacobas et al., 2006;Ghosh et al., 2010), cortex (Goldberg et al., 2010;Wallach et al., 2014), neocortex (Li et al., 2012), visual cortex and somatosensory cortex (Bennett et al., 2008a), hippocampus (Goto et al., 2004;Ullah et al., 2006), retina (Edwards and Gibson, 2010), spinal cord (Bennett et al., 2006;Gibson et al., 2008), as well as the striatum (Höfer et al., 2002). One fourth of the astrocyte network models took into account neurotransmitters in a simplistic way just as a stimulus, having either the glutamate as a constant, step function, or something similar (see e.g., Goto et al., 2004;Ullah et al., 2006;Bennett et al., 2008a;Kang and Othmer, 2009;MacDonald and Silva, 2013). Only Wallach et al. (2014) actually modeled the amount of neurotransmitter glutamate with a differential equation. The stimulus to the astrocyte model by Wallach et al. (2014) was taken from the model by Tsodyks and Markram (1997). We included this model under astrocyte models because this model was not bidirectional between astrocytes and neurons. The characteristics of astrocyte network models can be found in Table 3.
All the astrocyte network models studied Ca 2+ waves and few models specifically addressed spontaneous Ca 2+ waves and vascular events (see Table 3). All the models except the model by Iacobas et al. (2006) had the components for all three; CICR, leak from the ER into the cytosol, and the SERCA pump. About one fourth of the models took into account Ca 2+ buffering. About one third of the models had either influx of Ca 2+ from outside of the astrocyte or efflux of Ca 2+ to outside of the astrocyte, or both. About half of the models took into account astrocytic release of signaling molecules. Thus, the models had equations mostly for extracellular ATP, but one considered equations for extracellular glutamate (Bellinger, 2005). However, none of the models presented a detailed mechanistic description of how the release occurs. More than half of the models took into account diffusion, and, especially, almost half of the models studied the ATP diffusion in the extracellular space. Three quarters of the astrocyte network models had gap junctions for IP 3 but some models had them also for Ca 2+ . Thus, these models had similar core structure with small variations. As an example, Li et al. (2012) were the only ones that modeled K + concentration, both in astrocytic and extracellular spaces, and VGCCs. Goto et al. (2004) were the only ones to use the detailed IP 3 R model by The first model developed in this category was the model by Höfer et al. (2002). Höfer et al. (2002) showed with their two-dimensional (19 × 19) astrocyte network model that IP 3 permeability in gap junctions was a more important factor in intercellular Ca 2+ waves than Ca 2+ permeability. When blocking the IP 3 permeability, intercellular Ca 2+ wave propagation was prevented. However, intercellular Ca 2+ wave propagation was not prevented when the Ca 2+ permeability was blocked. In the model by Höfer et al. (2002), the IP 3 concentration depended on two distinct production terms via phospholipase C (PLC), one corresponding to PLC isotype β (PLCβ) and the other to PLC isotype δ (PLCδ). Höfer et al. (2002) showed that PLCδ was needed to be modeled to get the downstream cells to respond to the stimulus with a Ca 2+ increase.
Two of the newest models developed in this category were the models by Lallouette et al. (2014) and Wallach et al. (2014). Lallouette et al. (2014) simplified the astrocyte network model by Goldberg et al. (2010) to be able to simulate the function of a three-dimensional (11 × 11 × 11) astrocyte network. With this network of more than a thousand astrocytes, Lallouette et al. (2014) studied how the variability in the topology of gapjunction coupled astrocytes affected the intercellular Ca 2+ wave propagation. They tested five different coupling rules and found out that these different coupling rules can be used to reproduce the variation in the experimental data. They showed that dense connectivity or having long-distance gap-junction coupled astrocytes reduced the intercellular Ca 2+ wave propagation. Wallach et al. (2014) continued the previous study by stimulating astrocyte network by Tsodyks and Markram (1997) model. In the present study, the model by Wallach et al. (2014) is listed in the category of astrocyte network models because the astrocytes did not have an effect on the neuron. Wallach et al. (2014) demonstrated through experimental and simulation studies that there was a threshold stimulation frequency when astrocytes started to respond with Ca 2+ oscillations. However, this threshold frequency was different for different astrocytes and it increased with the number of astrocytes coupled.

Computational Neuron-Astrocyte Models
In recent years, bidirectional neuron-astrocyte communication has been the focus of much research in the field of neuroscience. Most of the existing neuron-astrocyte models concentrated on astrocytic Ca 2+ dynamics affected by glutamate in the synaptic cleft, and reciprocal neuron-astrocyte signaling. Many of the models were presented without a specific biological or diseaserelated question, and the focus was on combining existing models into a new construction, or adding the authors' own model components to previously published models. Next, we will go through neuron-astrocyte synapse models in section 3.2.1 and neuron-astrocyte network models in section 3.2.2.
2007; Amiri et al., 2011bAmiri et al., , 2013b, and Suffczynski et al. (2004) neuronal population model (Amiri et al., 2011a). The released neurotransmitter was modeled explicitly by Jung (2005, 2007), Volman et al. (2007), Nadkarni et al. (2008), Silchenko and Tass (2008), Amiri et al. (2011bAmiri et al. ( , 2013b, Wade et al. (2011), Tewari and Majumdar (2012a,b), Tang et al. (2013Tang et al. ( , 2016, Tewari and Parpura (2013), Nazari et al. (2015a,b,c), De Pittà and, and Li et al. (2017), however not necessarily with a differential equation. Other models utilized, for example, phenomenological transfer functions between neuronal events and astrocytic IP 3 concentration. Several models had increased astrocytic IP 3 concentration after the neuronal membrane potential increased over some threshold (marked in Table 4 as V m,N → [IP 3 ]). The details of the neuron-astrocyte synapse models can be found in Table 4. The neuron-astrocyte synapse models were developed to explain many different biological events as can be seen in Table 4. Examples of the studied phenomena included Ca 2+ dynamics, plasticity, hyperexcitability, information transfer, synchronization, and vascular events. All the other models except the models by Oku et al. (2016) and Guo et al. (2017) had all astrocytic components for CICR, leak from the ER into the cytosol, and the SERCA pump. About one third of the models had influx of Ca 2+ from outside of the astrocyte and efflux of Ca 2+ to outside of the astrocyte. The models had neither gap junction signaling because these models had only one astrocyte nor buffers. Thus, these models had similar core structure with small variations. As an example, two modeled CCE (Di Garbo et al., 2007;Di Garbo, 2009) and one modeled efflux via the PMCA pump and NCX  as well as diffusion (Silchenko and Tass, 2008).
The first model developed in this category was the model by Nadkarni and Jung (2003). Nadkarni and Jung (2003) studied Ca 2+ oscillations with a model consisting of a single astrocyte and single Hodgkin-Huxley type neuron. The astrocyte model was based on the model by Li and Rinzel (1994) but they added a differential equation for IP 3 . When the membrane potential of the neuron was higher than certain threshold, IP 3 concentration was increased in the astrocyte. The model also included a degradation of IP 3 . The membrane potential of the neuron depended on additional inward current (I astro ) that depended on astrocytic Ca 2+ . Nadkarni and Jung (2003) showed that with higher rate of IP 3 production, the astrocytic Ca 2+ concentration was higher. In addition, with large enough rate of IP 3 production, the neuron oscillated spontaneously without any external stimulus. Their hypothesis was that a higher expression of metabotropic glutamate receptors (mGluRs) and the resulting spontaneous oscillations caused epilepsy.
As the exact mechanisms of signaling from astrocytes to neurons are still largely unknown or controversial, most of the models had phenomenological descriptions of gliotransmitter release. About one third of the models took into account gliotransmitter release with equations for extracellular glutamate (see Table 4). Other models used phenomenological transfer functions to relay the effect of gliotransmission to the target synaptic terminal (I astro , part of I ast , f , and G m ). Only a few of the studied models had detailed astrocytic vesicle release model, This table shows the following: model, number (No.) of astrocytes modeled, input for the astrocytes, astrocytic variables described by differential equations, astrocytic Ca 2+ fluxes related to cytosolic Ca 2+ , diffusion of astrocytic variables either in the cytosol, ER, or extracellular space, not between these spaces, gap junction signaling (GJ) between astrocytes, output of the astrocytes, and experimentally shown event that the model was finetuned to capture [Ca 2+ dynamics ( Cortical spiking neural population model, which is based on the LIF model, was used by Amiri et al. (2016). Sotero and Martínez-Cancino (2010) used the dynamical mean field approximation to the system of 500,000 synapses with 13 equations per synapse. They reduced the number of differential equations to 13 (13 + 2) = 195. Nazari et al. (2015a) presented two different astrocyte models, with or without Ca 2+ ER and fluxes via ER membrane. Amiri et al. (2016) and Nazari et al. (2017) presented three different models, one was the original and the other two were reductions of the original model. However, the details of the two reduction models by Amiri et al. (2016) were given in such a way that they did not fulfill our criteria given in section 2.2. Li et al. (2017) simulated two single astrocytes models, and also combined the more complex astrocyte model with a neuronal seizure model. Di Garbo et al. (2007) and Di Garbo (2009)  ] − 196.69(nM) (Nadkarni and Jung, 2003).
namely the models by Silchenko and Tass (2008), Tewari and Majumdar (2012a,b), and Tewari and Parpura (2013). Silchenko and Tass (2008) modeled five vesicle states which represented different stages in the transmitter vesicle cycle. The rate of prefusion complex formation depended on astrocytic Ca 2+ concentration which was modeled by two steps. In the first step, they simplified the equation where Ca 2+ activated Ca 2+ -binding soluble N-ethylmaleimide-sensitive factor attachment protein receptor (SNARE) proteins by assuming that the concentration of activated SNARE-proteins was considered stationary. In the second step, they simplified the equation for the fusion of vesicles leading to an irreversible exocytosis of glutamate. However, Silchenko and Tass (2008) did not provide all the details of the model which makes the reuse of the model difficult.
The models by Tewari and Majumdar (2012a,b) and Tewari and Parpura (2013) assumed, based on experimental data on cultured hippocampal astrocytes, that the binding of three Ca 2+ ions was required for gliotransmitter release. The fusion and recycling process of the synaptic-like micro-vesicle was modeled using two differential equations that both depended on the probability that the synaptic-like micro-vesicle was ready to be released. In addition to these more detailed vesicle release models, De Pittà and Brunel (2016) modeled astrocytic glutamate exocytosis in a phenomenological way with just a few equations. They assumed that a fraction of gliotransmitter resources was available for release at any time. Then, every time astrocytic Ca 2+ increased beyond a certain threshold, the fraction of readily releasable astrocytic glutamate resources was released into the periastrocytic space. Two of the newest models were provided by Li et al. (2016aLi et al. ( , 2017. However, these studies contained, to the best of our understanding, fundamental errors in the biological terminology. Basically, the model by Li et al. (2016a) was the same as presented by Nadkarni and Jung (2004), but the neuronal membrane potential depended on astrocytic glutamate, as presented by Postnov et al. (2009), instead of astrocytic Ca 2+ , as presented by Nadkarni and Jung (2004). Li et al. (2017) developed a GABAactivated astrocyte model (which they, misleadingly, termed GABAergic). The model by Li et al. (2017) is similar to the model by Li et al. (2016a), but Li et al. (2017) added a more complex differential equation for IP 3 by taking into account both the GABA released by the interneuron and glutamate released by the astrocyte, somewhat similarly to Ullah et al. (2006), Nadkarni and Jung (2005), Volman et al. (2007), and others. The differential equations for the extracellular glutamate released by the astrocyte had similar forms as the IP 3 equations and were somewhat similar to the equation by Wade et al. (2012). Li et al. (2016a) showed how a higher equilibrium concentration of extracellular glutamate or glutamate degradation time constant predicted a higher neuronal firing frequency and existence of epileptic seizures. Li et al. (2017), on the other hand, presented using their GABA-activated astrocyte model (misleadingly termed GABAergic) that after a 0.5 s long GABA stimulation, astrocytic Ca 2+ oscillations were long-lasting. After combining the GABAactivated astrocyte model (misleadingly termed GABAergic) and a neuronal seizure model, they concluded that in this model, the astrocyte, through stimulating pyramidal neurons and thus increasing excitatory activity, prevented the transition from seizure activity into a normal firing activity state, which GABA alone was capable of inducing by inhibiting pyramidal neuron activity.
The neuron-astrocyte network models were developed to explain many different biological events as can be seen in Table 5. Examples included Ca 2+ dynamics, synchronization, information transfer, plasticity, and hyperexcitability. All the other models except the model by Allegrini et al. (2009) had components for all three; CICR, leak from the ER into the cytosol, and the SERCA pump. More than half of the models had influx of Ca 2+ from outside of the astrocyte and efflux of Ca 2+ to outside of the astrocyte. About one third of the models took into account gliotransmitter release by modeling extracellular glutamate, and few were also modeling extracellular ATP. Other models used phenomenological transfer functions to relay the effect of gliotransmission to the target synaptic terminal (I astro , I syn , part of I ast , and G m ). None of the studied models had a detailed astrocytic vesicle release model. Most of the models had gap junction signaling for IP 3 , and some also for Ca 2+ . Thus, these models had a similar core structure with small variations. As an example, only Yao et al. (2018) modeled buffering as well as astrocytic and extracellular K + . Diffusion was taken into account in the models by Allegrini et al. (2009), Postnov et al. (2009), Mesiti et al. (2015a, Yang and Yeo (2015), Li et al. (2016c), and Yao et al. (2018). Yao et al. (2018) presented one of the available models for cortical spreading depression.  Amiri et al. (2012b,c). The presentation of the model by Mesiti et al. (2015a) was confusing.
They seemed to present several models but the details were not given clearly. They seemed to have variables that were not used in the equations. Thus, it was difficult to know the actual model components. They simulated their model both with and without diffusion. Amiri et al. (2013a) simulated two models, the one was similar to their earlier neuron-astrocyte synapse model (Amiri et al., 2011b), and thus the details are not given here. Soleimani et al. (2015) and Haghiri et al. (2016Haghiri et al. ( , 2017 presented two different models, the other ones were reductions of the main ones. However, the simplified models by Soleimani et al. (2015) and Haghiri et al. (2017) were not detailed enough based on our criteria in section 2.2. Hayati et al. (2016) presented three different models, of which two models were detailed enough. A few models did not detail the mechanisms by which astrocytes communicated with each other (Haghiri et al., , 2017Hayati et al., 2016;Soleimani et al., 2015), thus it is possible that in some of these models each astrocyte is only connected to neurons (see e.g., Haghiri et al., 2017;Soleimani et al., 2015). Iastro =
One of the first models developed in this category was the two-dimensional model by Postnov et al. (2009). They studied how different lengths of stimulus affected astrocytic Ca 2+ and showed how short stimulus of less than 100 s did not induce Ca 2+ wave propagation. However, a longer stimulus of 320 s showed Ca 2+ wave propagation for a short distance and a stimulus of about 2,000 s showed Ca 2+ wave propagation along the astrocyte network. They also tested how Ca 2+ wave propagation was affected by different noise levels added to the model. They found out that the stronger the noise, the more accelerated was the Ca 2+ wave propagation. With the largest noise level they tested, they found out that the spatially synchronized behavior was destroyed, and the model started to behave irregularly.
A few publications presented simplification of model complexity. Simplification is, in general, used to reduce the model order to allow cost-effective computation yet preserving the major, key dynamical behavior of the model. Soleimani et al. (2015), Haghiri et al. (2016Haghiri et al. ( , 2017, and Hayati et al. (2016) presented the original and simplified versions of the earlier published models by Postnov et al. (2007Postnov et al. ( , 2009). However, most of the reduced astrocyte models were not detailed enough based on our criteria in section 2.2. In the future, it is important to put more emphasis on the model order reduction of the complex neuron-astrocyte interaction models to be able to simulate the behavior of large networks biologically more accurately (see e.g., Lehtimäki et al., 2017).
One of the newest models was the model by Chan et al. (2017). Their neuron-astrocyte network model was able to generate ultra-slow neuronal oscillatory activity patterns with frequencies less than 0.01 Hz, recorded by high-density microelectrode arrays and examined using methods reflecting conventional EEG analysis rather than conventional microelectrode array analysis. Their model showed that the frequency of these neuronal oscillations depended on the astrocytic Ca 2+ oscillations. These astrocytic Ca 2+ oscillations depended on the properties of IP 3 Rs and lasted from a few seconds to minutes. The results of Chan et al. (2017) also suggested that astrocytes preferred asynchronous neuronal firing to synchronous neuronal firing. Figure 1 shows how the computational astrocyte and neuronastrocyte models have evolved from each other (see also, Manninen et al., 2018). We realized that several similar or exactly the same models have been published without clearly explaining the similarity and without including citations to the other, earlier published similar models. This is both negligent and unethical. The very rules of science should be that every publication includes a description of what has been done before and what has been done now on top of the previously published publications. We have tried our best to give here as complete a picture as possible. The starting point of an arrow represents the model which was used as a reference by the latter model indicated as the arrowhead. We minimized the number of arrows so that the minimum number of arrows are pointing to the arrowheads. This means basically that all the previous models in the same chain of arrows might have been used to built the model in the arrowheads, but of course not all of them probably FIGURE 1 | Evolution of astrocyte and neuron-astrocyte models published from 1995 to 2017. The starting point of an arrow represents the model which was used as a reference by the latter model indicated as the arrowhead. The number of arrows was minimized so that the minimum number of arrows are pointing to the arrowheads. This means basically that all the previous models in the same chain of arrows might have been used to built the model in the arrowhead, but of course not all of them probably were necessary. With blue, we presented De Young and Keizer (1992) and Li and Rinzel (1994) -type models. With pink, we presented Höfer et al. (2002) -type models. With purple, we presented De Young and Keizer (1992), Li and Rinzel (1994), and Höfer et al. (2002) -type models. All the other types of models appear green.

Model Evolution
were necessary. Models are excluded from Figure 1 if there was no clear evidence that the authors had used any other models presented in this study as a reference and their model was not used as a reference by any other models presented in this study. The models by De Young and Keizer (1992), Li and Rinzel (1994), and Höfer et al. (2002) were most often used as a starting point when developing new models (Manninen et al., 2018). In addition, the other models that have helped to steer the field are the models by Nadkarni and Jung (2003)

DISCUSSION AND CONCLUSIONS
In this review, we present the state-of-the-art in the computational modeling of astrocytes and the interactions astrocytes have with vascular and neuronal systems. We were delighted to see the number of researchers presently interested in modeling astrocytic functions. Due to the large number of models, we concentrated on about a hundred models that included biophysical descriptions for Ca 2+ signaling and dynamics in astrocytes. The models were categorized into four groups: single astrocyte models, astrocyte network models, neuron-astrocyte synapse models, and neuron-astrocyte network models. We characterized the components of the models in detail, contrasted and compared the models with one another, and evaluated their usability in future work. Selecting the best models to be utilized in one's own research can be very tedious and time-consuming due to the large number of available models. We therefore wish to provide practical help and guidelines in choosing the proper astrocyte models for future modeling projects. Our work also promotes the transparency of scientific work and recommends actions to establish good practices in computational modeling and the presentation of results.
One of the fundamental questions in neuroscience is how different mechanisms of astrocytes and neuron-astrocyte interactions are linked with cognitive functions and behavior in mammals. A variety of evidence is accumulating on the roles of astrocytes in neuronal excitability, synaptic transmission, plasticity, and in higher cognitive functions, including the initiation, maintenance and consolidation of memories Magistretti and Allaman, 2015;Bazargani and Attwell, 2016). Most of the evidence stems from in vitro experimental studies, but also from in vivo studies. Experimental wet-lab work on astrocytes has given rise to a variety of studies to computationally address astrocytes' Ca 2+ excitability and its putative role in neural functions in a variety of conditions (Manninen et al., 2018). Although there is partial controversy and ongoing debate on the existence of gliotransmission in vitro and in vivo (see Fiacco and McCarthy, 2018;Savtchouk and Volterra, 2018, and section 2.1.4), recent attempts to model the astrocytes' roles in synaptic and network dynamics are a welcome and useful additional tool to help testing various hypotheses. This is indeed a good sign: astrocytes, the important but mostly neglected glial cells, are gradually being taken into account in efforts to understand the roles of astrocytes in neural network dynamics. Based on our evaluation, we found out that in silico models have been presented for many of the above-mentioned, experimentally observed neural phenomena. However, it was not always clear which of the models were based on cell culture and which ones on slice studies. Very few of the models were constructed using in vivo data. According to our evaluation of 106 models, 53 models were constructed to study Ca 2+ dynamics and 15 models were constructed to study neural synchronization. In addition, 12 models were used to study information transfer, 11 models were used to study vascular events, 10 models were used to study plasticity, four models were used to study hyperexcitability, and one model was used to study homeostasis. However, the models often described a limited set of molecular mechanisms which sometimes led us to doubt if the models were detailed enough to answer the questions asked.
Even though some promising studies on the modeling of glial functions exist (see e.g., Höfer et al., 2002;Nadkarni et al., 2008;De Pittà et al., 2009a;Lallouette et al., 2014;Taheri et al., 2017), the actual value of the models can only be assessed with time and the reimplementation and resimulation of published models. Astrocytic mechanisms, such as astrocytic Ca 2+ fluxes related to cytosolic Ca 2+ , diffusion of astrocytic variables either in the cytosol, ER, or extracellular space, and gap junction signaling between astrocytes, were characterized in more detail here than in our other, educational study (Manninen et al., 2018) to facilitate the viewing of the similarities and differences of the models, as well as to support the utilization of the models in future work. In the present study, a careful review of the details of the published models revealed that several publications gave inaccurate descriptions of the models. Examples include misleading or completely missing graphical illustrations of the models, incorrect mathematical equations, biologically incorrectly or sometimes misleadingly named variables, unclear or non-existent statements of the number of cells modeled, and non-existent description of the applicability of the selected model components (see also, Manninen et al., 2018). Moreover, our detailed evaluation revealed that most models were generated making slight variations to a small set of older models that did not originally represent data obtained for astrocytes. However, neither citations to previous models with similar core structure nor explanations about what exactly was added to the previous models were provided. This made it possible, in some cases, to publish the same or a very similar model several times. Very few models provided a detailed sensitivity analysis, that is, an analysis of the robustness of the model against changes in parameter values. We therefore conclude that most of the models published thus far do not serve the scientific community in their best potential and the simulation results of the models are very difficult to reproduce. A proper validation of the simulation results against experimental findings and a careful review process of manuscripts are needed to promote the transparency and utility of in silico models. Large-scale neuroscience projects, such as presented by Markram et al. (2015), Amunts et al. (2016), and Grillner et al. (2016), are seeking to solve these challenges by providing sophisticated informatics tools for the construction, estimation and validation of models.
Our study highlights the need for reproducible research, which is an enormous challenge in all areas of science (Baker, 2016;Munafò et al., 2017;Rougier et al., 2017). In our other studies, we have shown how tedious and difficult it is to reproduce and replicate the simulation results of published astrocyte models (Manninen et al., 2017(Manninen et al., , 2018. We have shown that it is often impossible to reproduce the results without first carefully assessing and verifying all equations or contacting the authors for more details of the published model. In our previous studies, we have reimplemented altogether seven astrocyte models and were able to reproduce the simulation results of only two of the publications completely, based on the information in the original publications and corrigenda (Manninen et al., 2017(Manninen et al., , 2018. After fixing the observed errors in the original equations, we were able to reproduce the original results of one more model completely (Manninen et al., 2017). One of the goals of the present study is to show how many similar models have already been developed and how emphasis should be put on making the developed models usable for other researchers by publishing the model codes online. Furthermore, reviewers should be able to verify that the implementation and equations presented in the manuscript match. One solution would be to submit all the details of the model, such as equations, parameter values, initial values, and stimuli, in table format with the manuscript, similarly to what was presented in our previous studies (see e.g., Manninen et al., 2017). It would also be helpful to present the outline of the model in a table (see e.g., Tables 2-5 and Manninen et al., 2010Manninen et al., , 2018. Reproducible models can truly advance the computational fields of neuroscience and glioscience. It seems increasingly evident that astrocytes modulate a variety of events in the brain. Astrocytes are fully integrated into the brain cellular circuitry and are an essential part of the neural networks. Future mathematical and computational models of brain functions should better take into account the hidden capacity of astrocytes and other glial cells, as also pointed out by Sompolinsky (2014), and available methodologies (see e.g., Stiles and Bartol, 2001;Kerr et al., 2008;Skupin et al., 2008;Wils and De Schutter, 2009;Andrews et al., 2010;Oliveira et al., 2010;Hepburn et al., 2012;Tilūnaitė et al., 2017). Such comprehensive models will help us detail the complex interactions between different types of glial cells and other systems in the brain, including the vascular and neuronal systems, and may lead to a better understanding of the roles astrocytes have in the brain. Most urgently, in silico modeling in neuroscience should strive to incorporate state-of-the-art wet-lab data on glial cells to advance the construction and validation of models. In this process, more emphasis should be put on the selection of the animal species, the developmental stage of an organism, and the type of wet-lab preparation. Future work should also include the development of theory, based on constraints of physics and chemistry, to provide a linkage between the phenomena measured at different levels and scales in the brain. In conclusion, the description of the state-of-the-art in neuron-glia modeling and the constructive criticism presented in this work will be useful in setting new goals and guidelines for future modeling in neuroscience.

AUTHOR CONTRIBUTIONS
TM designed the study, acquired the in silico models, characterized and evaluated the models in detail, tabulated the details of the models, drafted the first version of the manuscript, and coordinated the production of the final version. RH contributed to the design of the study, interpreted biological terminology used in in silico models, evaluated some of the models, as well as contributed to the drafting and critical revision of the manuscript. M-LL conceived and designed the study, acquired funding, interpreted concepts in in silico models, contributed to the drafting and critical revision of the manuscript, and supervised the study. All approved the final version of the manuscript.