# MULTISCALE MODELING OF RHYTHM, PATTERN AND INFORMATION GENERATION: FROM GENOME TO PHYSIOME

EDITED BY : Shangbin Chen and Alexey Zaikin PUBLISHED IN : Frontiers in Physiology

#### Frontiers eBook Copyright Statement

The copyright in the text of individual articles in this eBook is the property of their respective authors or their respective institutions or funders. The copyright in graphics and images within each article may be subject to copyright of other parties. In both cases this is subject to a license granted to Frontiers. The compilation of articles constituting this eBook is the property of Frontiers.

Each article within this eBook, and the eBook itself, are published under the most recent version of the Creative Commons CC-BY licence. The version current at the date of publication of this eBook is CC-BY 4.0. If the CC-BY licence is updated, the licence granted by Frontiers is automatically updated to the new version.

When exercising any right under the CC-BY licence, Frontiers must be attributed as the original publisher of the article or eBook, as applicable.

Authors have the responsibility of ensuring that any graphics or other materials which are the property of others may be included in the CC-BY licence, but this should be checked before relying on the CC-BY licence to reproduce those materials. Any copyright notices relating to those materials must be complied with.

Copyright and source acknowledgement notices may not be removed and must be displayed in any copy, derivative work or partial copy which includes the elements in question.

All copyright, and all rights therein, are protected by national and international copyright laws. The above represents a summary only. For further information please read Frontiers' Conditions for Website Use and Copyright Statement, and the applicable CC-BY licence.

ISSN 1664-8714 ISBN 978-2-88963-746-1 DOI 10.3389/978-2-88963-746-1

#### About Frontiers

Frontiers is more than just an open-access publisher of scholarly articles: it is a pioneering approach to the world of academia, radically improving the way scholarly research is managed. The grand vision of Frontiers is a world where all people have an equal opportunity to seek, share and generate knowledge. Frontiers provides immediate and permanent online open access to all its publications, but this alone is not enough to realize our grand goals.

#### Frontiers Journal Series

The Frontiers Journal Series is a multi-tier and interdisciplinary set of open-access, online journals, promising a paradigm shift from the current review, selection and dissemination processes in academic publishing. All Frontiers journals are driven by researchers for researchers; therefore, they constitute a service to the scholarly community. At the same time, the Frontiers Journal Series operates on a revolutionary invention, the tiered publishing system, initially addressing specific communities of scholars, and gradually climbing up to broader public understanding, thus serving the interests of the lay society, too.

#### Dedication to Quality

Each Frontiers article is a landmark of the highest quality, thanks to genuinely collaborative interactions between authors and review editors, who include some of the world's best academicians. Research must be certified by peers before entering a stream of knowledge that may eventually reach the public - and shape society; therefore, Frontiers only applies the most rigorous and unbiased reviews.

Frontiers revolutionizes research publishing by freely delivering the most outstanding research, evaluated with no bias from both the academic and social point of view. By applying the most advanced information technologies, Frontiers is catapulting scholarly publishing into a new generation.

#### What are Frontiers Research Topics?

Frontiers Research Topics are very popular trademarks of the Frontiers Journals Series: they are collections of at least ten articles, all centered on a particular subject. With their unique mix of varied contributions from Original Research to Review Articles, Frontiers Research Topics unify the most influential researchers, the latest key findings and historical advances in a hot research area! Find out more on how to host your own Frontiers Research Topic or contribute to one as an author by contacting the Frontiers Editorial Office: researchtopics@frontiersin.org

# MULTISCALE MODELING OF RHYTHM, PATTERN AND INFORMATION GENERATION: FROM GENOME TO PHYSIOME

Topic Editors:

Shangbin Chen, Huazhong University of Science and Technology, China Alexey Zaikin, University College London, United Kingdom

Citation: Chen, S., Zaikin, A., eds. (2020). Multiscale Modeling of Rhythm, Pattern and Information Generation: from Genome to Physiome. Lausanne: Frontiers Media SA. doi: 10.3389/978-2-88963-746-1

# Table of Contents


Thanos Manos, Magteld Zeitler and Peter A. Tass

*86 Computationally Developed Sham Stimulation Protocol for Multichannel Desynchronizing Stimulation*

Magteld Zeitler and Peter A. Tass


Péter Makra, Ákos Menyhárt, Ferenc Bari and Eszter Farkas

*133 Dendritic and Axonal Propagation Delays May Shape Neuronal Networks With Plastic Synapses*

Mojtaba Madadi Asl, Alireza Valizadeh and Peter A. Tass

*141 Features of Neural Network Formation and Their Functions in Primary Hippocampal Cultures in the Context of Chronic TrkB Receptor System Influence*

Tatiana A. Mishchenko, Elena V. Mitroshina, Alexandra V. Usenko, Natalia V. Voronova, Tatiana A. Astrakhanova, Olesya M. Shirokova, Innokentiy A. Kastalskiy and Maria V. Vedunova

*158 TREC and KREC Levels as a Predictors of Lymphocyte Subpopulations Measured by Flow Cytometry*

Ilya Korsunskiy, Oleg Blyuss, Maria Gordukova, Nataliia Davydova, Susanna Gordleeva, Robert Molchanov, Alan Asmanov, Dmitrii Peshko, Nataliia Zinovieva, Sergey Zimin, Vladimir Lazarev, Aminat Salpagarova, Maxim Filipenko, Ivan Kozlov, Andrey Prodeus, Anatoliy Korsunskiy, Peter Hsu and Daniel Munblit


# Editorial: Multiscale Modeling of Rhythm, Pattern and Information Generation: from Genome to Physiome

Shangbin Chen<sup>1</sup> \* and Alexey Zaikin2,3,4

*<sup>1</sup> Britton Chance Center for Biomedical Photonics, Wuhan National Laboratory for Optoelectronics-Huazhong University of Science and Technology, Wuhan, China, <sup>2</sup> Department of Mathematics and Institute for Women's Health, University College London, London, United Kingdom, <sup>3</sup> Lobachevsky State University of Nizhny Novgorod, Nizhny Novgorod, Russia, <sup>4</sup> Centre of Complex Systems Analysis, Sechenov University, Moscow, Russia*

Keywords: quantitative physiology, multiscale model, computational model, physiome, systems biology, complexity

**Editorial on the Research Topic**

#### **Multiscale Modeling of Rhythm, Pattern and Information Generation: from Genome to Physiome**

The rapid advances of technology have revolutionized the research of physiology in the era of big data. More and more anatomical, physiological, and clinical data are collected to support precision medicine. Effective progress in research can be obtained only by merging data mining with modeling and analysis in the frame of Quantitative Physiology. Quantitative Physiology is the quantitative description, modeling and computational study of physiology, which is an increasingly important branch of systems biology. It will take the power of physics, mathematics, information technology, etc., to implement quantitative, testable, and predictable research to boost understanding the function in living systems. It is quite similar to mathematical physiology and computational physiology that address the theoretical or computational nature of physiology, but it is somehow different to stress data acquisition and quantitative description in the era of big data.

In 2003, the completion of the Human Genome Project (HGP) was announced effectively with the publication of the first complete human genome sequence. Due to the tight correlation between genes and functions, physiological genomics and functional genomics are hot topics in modern physiology. Among the long list of "ome" and "omics," we should pay more attention to Physiome and Physiomics. Physiome comes from "physio-" (nature) and "-ome" (as a whole). The Physiome is the quantitative and integrated description of the physiological dynamics and functional behavior of the physiological (normal) and pathophysiological states of an individual or species (Bassingthwaighte, 2000). The Physiome describes the physiological dynamics of the normal intact organism and is built upon information and structure (genome, proteome, and morphome). Obviously, Quantitative Physiology matches very well with the definition of Physiome.

Stephen Hawking said that the next twenty-first century would be the century of complexity and indeed now Systems Biology or Medicine means dealing with complexity. This reality is that a huge amount of biological or physiological data, emerging from different-omics sources on very different scales, from genome to Physiome, exceeds our abilities to analyze these data. To link molecular and cellular events with physiological function and behavior is associated with wide ranges of space and time scales (Hunter and Borg, 2003). This Research Topic aims to provide state-of-the-art review of multi-scale modeling and data analysis to investigate the function in living systems, organisms, organ systems, organs, cells, and biomolecules carrying out the chemical or physical functions that

Edited and reviewed by:

*Raimond L. Winslow, Johns Hopkins University, United States*

\*Correspondence: *Shangbin Chen sbchen@mail.hust.edu.cn*

#### Specialty section:

*This article was submitted to Computational Physiology and Medicine, a section of the journal Frontiers in Physiology*

Received: *27 November 2019* Accepted: *12 March 2020* Published: *07 April 2020*

#### Citation:

*Chen S and Zaikin A (2020) Editorial: Multiscale Modeling of Rhythm, Pattern and Information Generation: from Genome to Physiome. Front. Physiol. 11:281. doi: 10.3389/fphys.2020.00281* exist in a living system. So far, this Research Topic has collected 14 articles (12 original researches and 2 mini reviews) that represent a cross sectional sample of Quantitative Physiology.

There are 4 typical modeling studies. The multi-scale cardiac computational modeling (Bai et al.) investigated the potential effects of a R858H mutation on the intracellular calcium handling, action potential profiles, action potential duration restitution curves, dispersion of repolarization, QT interval and spiral wave dynamics. It revealed that the L-type calcium current altered by mutation increases arrhythmia risk due to after depolarizations and increased tissue vulnerability to unidirectional conduction block. It provided a causal link between a R858H mutation and ventricular fibrillation. A neuron-astrocyte network model composed of 100 excitatory neurons and two astrocytes was developed (Gordleeva et al.). It showed that spatiotemporal properties of Ca2<sup>+</sup> dynamics in spatially extended astrocyte can coordinate (e.g., synchronize) networks of neurons and synapses. In addition, A stochastic model of immune response (Fatehi et al.) and a simple spin glasslike model for the collective sensing of β-cells were proposed (Korosak and Rupnik).

Particularly, there are 4 papers of computational studies on deep brain stimulation (DBS). Tass' group reviewed that the dendritic and axonal propagation delays may lead to the emergence of neuronal activity and synaptic connectivity patterns, which cannot be captured by classic spike-timingdependent plasticity models (Asl et al.). A short-term dosage regimen of coordinated reset stimulation could induce longlasting desynchronization of the networks (Manos et al.). A novel stimulation method of pulsatile multisite linear delayed feedback was employed to modulate the pulse amplitude of high-frequency DBS (Popovych and Tass). And, a sham stimulation protocol for multichannel desynchronizing stimulation was proposed to provide controls (Zeitler and Tass). All these findings and implementations are helpful to develop therapeutic DBS.

The other 6 papers are all related with data analysis. Both short-time Fourier transform and multifractal detrended fluctuation analysis were used to seek the age-related signature of local field potential recordings (Makra et al.). Another work implemented the intelligent assessment and classification of ECG (Zhao and Zhang). Cross-correlation analysis were used in study of primary immunodeficiency diseases (Korsunskiy et al.) and development of brain neural networks (Mishchenko et al.). Both differential expression and network analysis were introduced in brain transcriptome (Wang and Wang) and functional genomics (Zhu et al.).

The work in this Research Topic involved the different levels of biomolecules, cells, organs, and different events of genetic expression, transcriptional regulation, calcium signaling, cell signaling, heart beating, brain integration. Here, a central concept is generation and processing of information. This Research Topic is devoted to set a paradigm for Quantitative Physiology by integrating biology, mathematics, physics or informatics. Both Editors of this Research Topic have joined the project "Digital Personalized Medicine of Healthy Aging (DPM-AGING): network analysis of Big Multi-omics data to search for new diagnostic, prognostic, and therapeutic targets." This Research Topic is also a good example of Quantitative Physiology.

It is not easy to perform multi-scale modeling and data analysis to investigate the functions in living systems. Just as Schrödinger's question and answer: "How can the events in space and time which take place within the spatial boundary of a living organism be accounted for by physics and chemistry? The obvious inability of present-day physics and chemistry to account for such events is no reason at all for doubting that they can be accounted for by those sciences."

#### AUTHOR CONTRIBUTIONS

All authors listed have made a substantial, direct and intellectual contribution to the work, and approved it for publication.

### FUNDING

This work was supported by the National Natural Science Foundation of China (Grant No. 61371014), and by the grant of the Ministry of Education and Science of the Russian Federation Agreement No. 075-15-2019-871. AZ thanks MRC grant MR/R02524X/1.

### ACKNOWLEDGMENTS

The authors would like to thank Profs. Ling Fu and Jürgen Kurths for their help and support for our teaching of Quantitative Physiology.

### REFERENCES


**Conflict of Interest:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2020 Chen and Zaikin. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Computational Cardiac Modeling Reveals Mechanisms of Ventricular Arrhythmogenesis in Long QT Syndrome Type 8: CACNA1C R858H Mutation Linked to Ventricular Fibrillation

#### Edited by:

*Shangbin Chen, Huazhong University of Science and Technology, China*

#### Reviewed by:

*Alok Ranjan Nayak, International Institute of Information Technology, India Chris Patrick Bradley, University of Auckland, New Zealand*

#### \*Correspondence:

*Kuanquan Wang wangkq@hit.edu.cn; wangkq@ieee.org Henggui Zhang henggui.zhang@manchester.ac.uk*

#### Specialty section:

*This article was submitted to Computational Physiology and Medicine, a section of the journal Frontiers in Physiology*

Received: *05 April 2017* Accepted: *21 September 2017* Published: *04 October 2017*

#### Citation:

*Bai J, Wang K, Liu Y, Li Y, Liang C, Luo G, Dong S, Yuan Y and Zhang H (2017) Computational Cardiac Modeling Reveals Mechanisms of Ventricular Arrhythmogenesis in Long QT Syndrome Type 8: CACNA1C R858H Mutation Linked to Ventricular Fibrillation. Front. Physiol. 8:771. doi: 10.3389/fphys.2017.00771* Jieyun Bai <sup>1</sup> , Kuanquan Wang<sup>1</sup> \*, Yashu Liu<sup>1</sup> , Yacong Li <sup>1</sup> , Cuiping Liang<sup>1</sup> , Gongning Luo<sup>1</sup> , Suyu Dong<sup>1</sup> , Yongfeng Yuan<sup>1</sup> and Henggui Zhang1, 2, 3 \*

*<sup>1</sup> School of Computer Science and Technology, Harbin Institute of Technology, Harbin, China, <sup>2</sup> Biological Physics Group, School of Physics and Astronomy, University of Manchester, Manchester, United Kingdom, <sup>3</sup> Space Institute of Southern China, Shenzhen, China*

Functional analysis of the L-type calcium channel has shown that the *CACNA1C* R858H mutation associated with severe QT interval prolongation may lead to ventricular fibrillation (VF). This study investigated multiple potential mechanisms by which the *CACNA1C* R858H mutation facilitates and perpetuates VF. The Ten Tusscher-Panfilov (TP06) human ventricular cell models incorporating the experimental data on the kinetic properties of L-type calcium channels were integrated into one-dimensional (1D) fiber, 2D sheet, and 3D ventricular models to investigate the pro-arrhythmic effects of *CACNA1C* mutations by quantifying changes in intracellular calcium handling, action potential profiles, action potential duration restitution (APDR) curves, dispersion of repolarization (DOR), QT interval and spiral wave dynamics. R858H "mutant" L-type calcium current (*ICaL*) augmented sarcoplasmic reticulum calcium content, leading to the development of afterdepolarizations at the single cell level and focal activities at the tissue level. It also produced inhomogeneous APD prolongation, causing QT prolongation and repolarization dispersion amplification, rendering R858H "mutant" tissue more vulnerable to the induction of reentry compared with other conditions. In conclusion, altered *ICaL* due to the *CACNA1C* R858H mutation increases arrhythmia risk due to afterdepolarizations and increased tissue vulnerability to unidirectional conduction block. However, the observed reentry is not due to afterdepolarizations (not present in our model), but rather to a novel blocking mechanism.

Keywords: CACNA1C mutations, L-type calcium channel, Long QT syndrome, Timothy syndrome, ventricular fibrillation, dispersion of repolarization, computational cardiac modeling

### INTRODUCTION

Congenital long QT syndrome (LQTS) is characterized by an abnormally prolonged QT and high risk of ventricular arrhythmias in susceptible families (Goldenberg et al., 2008). LQTS type 8 (LQT8, Timothy syndrome, TS), a specific subtype of LQTS, is a dysfunction syndrome involving multiple organs, which can manifest as severe QT interval prolongation, T wave alternans, 2:1 atrioventricular block, syndactyly, facial dysmorphism, autistic spectrum disorders, immunodeficiency, severe hypoglycemia, etc. (Splawski et al., 2004, 2005; Etheridge et al., 2011; Gillis et al., 2012). The CACNA1C gene encodes CaV1.2 that is a subunit of L-type voltage-dependent calcium channel and gain-of-function mutations in CACNA1C have been suggested to be responsible for LQT8 manifestations, but the interplay between CACNA1C genotypes and malignant clinical phenotypes is likely complex (Giudicessi and Ackerman, 2013). For instance, the G406R mutation was believed to be the possible cause of TS associated with many extra-cardiac phenotypes, such as syndactyly, cognitive delay, and craniofacial abnormalities (Splawski et al., 2004), while two de novo mutations (G406R and G402S) also induced QT prolongation but without syndactyly (Splawski et al., 2005; Frohler et al., 2014; Hiippala et al., 2015). Recently, a handful of other CACNA1C mutations were identified in patients exhibiting only modest QT prolongation (Gillis et al., 2012; Boczek et al., 2013, 2015a,b; Fukuyama et al., 2013, 2014; Hennessey et al., 2014; Wemhöner et al., 2015; Landstrom et al., 2016; Sutphin et al., 2016). In particular, Fukuyama et al. identified five novel CACNA1C mutations: G1783C, P381S, M456I, A582D, and R858H (Fukuyama et al., 2013, 2014). Patients with the R858H mutation displayed excessive QT prolongation and episodes of ventricular fibrillation (VF). Although functional analysis of R858H mutant channels reveals a significant increase in the L-type calcium current (ICaL), relatively little is known about the pathogenic mechanisms underlying VF in the setting of the "mutant" ICaL.

ICaL plays a major role in regulating cardiovascular functions because it regulates excitation-contraction coupling (ECC) by triggering the calcium release from the sarcoplasmic reticulum (SR), modulates cellular excitability, and action potential (AP) shape by participating in AP repolarization and is thereby involved in the heart rhythm and contractility (Benitah et al., 2010). Abnormalities in ICaL due to CACNA1C mutations have been suggested as factors contributing to ventricular arrhythmogenesis (Venetucci et al., 2012). In previous simulation studies, it has been shown that changes in ICaL due to gainof-function mutations in CACNA1C prolongs action potential duration (APD; Faber et al., 2007; Zhu and Clancy, 2007; Yarotskyy et al., 2009; Morotti et al., 2012; Wemhöner et al., 2015; Bai et al., 2016c) linked to early afterdepolarizations (EADs; Sung et al., 2010; Boczek et al., 2015a), and increases SR calcium content which is then responsible for spontaneous calcium release and delayed afterdepolarizations (DADs; Splawski et al., 2005; Thiel et al., 2008; Sung et al., 2010; Yazawa et al., 2011). Although these studies may provide a potential mechanistic link between CACNA1C mutations and ventricular arrhythmias, altered AP in single cells cannot be extrapolated directly to reentrant arrhythmias in the human heart, where electrotonic coupling between cardiomyocytes may smooth out electrical heterogeneity between cells. Integrative computational models have been widely used to build a bridge between CACNA1C mutations and pro-arrhythmic phenotypes, and these simulated results have shed valuable light on the mechanisms of arrhythmogenesis (Roberts et al., 2012; Bai et al., 2016a). Indeed, our previous models have indicated that changes in ICaL due to the reduced voltage-dependent inactivation caused by the G1911R mutation in LQT8 extremely prolongs APD and augments dispersion of repolarization (DOR), increasing susceptibility to reentrant arrhythmias (Bai et al., 2016c). By contrast, changes in ICaL due to the augmented current density caused by the R858H mutation has been suggested to increase the likelihood of VF (Fukuyama et al., 2014), but this link remains to be demonstrated directly. In this study, we focus on the mechanisms by which altered ICaL caused by a R858H CACNA1C mutation promotes and perpetuates ventricular arrhythmia using mathematical modeling.

For this purpose, we modified the Ten Tusscher-Panfilov (TP06) human ventricular cell model (Ten Tusscher et al., 2004, 2006) to incorporate experimental data on the kinetic properties of ICaL (Fukuyama et al., 2014). We used this model to investigate the electrophysiological consequences of CACNA1C mutations in single cells, wave propagation in one-dimensional (1D) ventricular cables, and the onset of spiral waves in twodimensional (2D) and three-dimensional (3D) models.

In particular, the modified model can reproduce currentvoltage (I-V) relationships of CACNA1C mutations and prolongation of the QT interval (Fukuyama et al., 2014). We found that ICaL arising from the CACNA1C R858H mutation augments SR calcium content, leading to spontaneous calcium release and afterdepolarizations in single cells, and thereby focal activity in the 1D fiber. The R858H mutation produced electrical

**Abbreviations:** VF, Ventricular fibrillation; PVC, premature ventricular complex; LQTS, long QT syndrome; LQT8, LQTS type 8; TS, timothy syndrome; WT, wild-type; ECC, excitation-contraction coupling; 1D, 2D, and 3D, one-, two- and three- dimensional; ICaL, L-type calcium current; INCX, sodium-calcium exchanger current; Irel, the calcium-induced-calcium release flux; Irel(m) , Irel amplitude; Ito, the transient outward potassium channel current; IKs, slow delayed rectifier potassium channel current; Cm, the capacitance; D, the effective diffusion constant; Iion, the total transmembrane current; Vm, the trans-membrane potential; [Ca2+]<sup>i</sup> , cytoplasmic calcium concentration; [Ca2+]i(m) , [Ca2+]<sup>i</sup> amplitude; SR, sarcoplasmic reticulum; [Ca2+]SR, SR calcium concentration; [Ca2+]SR(m) , SR calcium content; AP, action potential; APD, AP duration; APD90, APD at 90% repolarization; APDR, APD restitution; ENDO, endocardial-cell; MCELL, midmyocardial-cell; EPI, epicardial-cell; ENDO-EPI, between endocardial- and epicardial- cells; ENDO-M, between endocardial- and midmyocardial- cells; EPI-M, between epicardial- and midmyocardial- cells; EAD, early afterdepolarization; DAD, delayed afterdepolarization; TP06 model, Ten Tusscher-Panfilov human ventricular cell model; ORd model, O'Hara-Rudy dynamic human ventricular cell model; I-V relationships, current-voltage relationships; CSF, the scaling factor of the ICaL conductance; Va,0.5, the midpoint voltage of the voltage-activation curve; Sa, the slope of the voltage-activation curve; Vina,0.5, the midpoint voltage of the voltage-inactivation curve; Sina, the slope of the voltage-inactivation curve; TCSF, the scaling factor of voltage-inactivation time constant; PCL, pacing cycle length; MPCL, the maximum PCL that produced 2:1 block; DOR, dispersion of repolarization; RT, repolarization time, SG, spatial gradient of APD; MSG, maximal SG; δ, the membrane potential gradient; CV, conduction velocity; DI, diastolic interval.

Bai et al. Arrhythmogenesis in LQT8

heterogeneity within the ventricular wall, amplified the intrinsic spatial DOR, and increased tissue vulnerability to generate unidirectional conduction block facilitating the development of reentry in the transmural ventricular sheet. These simulation data imply that patients with the R858H mutation are at high risk of ventricular arrhythmias.

### MATERIALS AND METHODS

### Model of ICaL

The equations for ICaL in the TP06 model (Ten Tusscher and Panfilov, 2006a) were modified to incorporate experimental data on CACNA1C mutation-induced changes. First, we determined the modifications to the original ICaL model to reproduce the behavior of the mutant ICaL (**Figure 1A**) during the same voltage-clamp employed in experiments (Fukuyama et al., 2014). Theoretical steady-state activation and inactivation curves which were used to simulate G1783C, wild-type (WT), P381S, M456I, A582D, and R858H ICaL, are shown in **Figure 1C**. Second, based on experimental I-V relationships (**Figure 1B**), mathematical models of ICaL were constructed (formulations are listed in Supplementary Material). This was achieved by simulating the experimental voltage-clamp protocol (Fukuyama et al., 2014) and scaling relative current proportions for G1783C, WT, P381S, M456I, A582D, and R858H conditions. Peak inward G1783C, P381S, M456I, A582D and R858H ICaL density was, respectively, ∼0.86-, ∼1.04-, ∼1.08-, ∼1.19-, ∼1.54-fold that for WT ICaL (**Figure 1B**). The simulated I-V curves (**Figure 1D**) matched closely with the experimental observation (**Figure 1B**).

Upon biophysical analysis of the experimental data on CACNA1C mutations (Fukuyama et al., 2014), four major changes to ICaL were considered. These changes included: ICaL current densities, steady-state activation curves, voltagedependent inactivation curves as well as the time constant for the voltage-dependent inactivation. Parameters for ICaL, including the scaling factor of the ICaL conductance (CSF), the midpoint voltage of the voltage-activation curve (Va,0.5), the slope of the voltage-activation curve (Sa), the midpoint voltage of the voltage-inactivation curve (Vina,0.5), the slope of the voltage-inactivation curve (Sina) and the scaling factor of voltage-inactivation time constant (TCSF), were modified to reproduce the experimental I–V relationships (Comparison between simulation and experimental results for ICaL can be found in Table S1).

#### Single Cell Simulations

The TP06 model, based on human experimental data, was developed to reproduce transmural heterogeneity of electrical properties (Ten Tusscher et al., 2004; Ten Tusscher and Panfilov, 2006a) by changing maximum conductivities of the transient outward potassium channel current (Ito) and slow delayed rectifier potassium channel current (IKs). The ICaL formulations were the same for endocardial (ENDO), midmyocardial (MCELL) and epicardial (EPI) cell models. The model (Ten Tusscher et al., 2006) was suggested to be suitable for simulating wave dynamics at the tissue and organ levels (Ten Tusscher and Panfilov, 2006b; Ten Tusscher et al., 2009). In 2013, the calcium-induced-calcium release flux (Irel) was modeled as the combination of both SR calcium release and SR calcium leak by Lascano et al. (2013). The same modifications were employed in our previous studies (Bai et al., 2016b, 2017; Liu et al., 2016; formulations are listed in Supplementary Material).

In single cell simulations, APs were elicited by pre-pacing the models for 100 cycles to reach a stable steady state. APD was computed as AP duration at 90% repolarization (APD90). Changes in ICaL, sodium-calcium exchanger current (INCX), Irel, cytoplasmic calcium concentration ([Ca2+]i) and SR calcium concentration ([Ca2+]SR) were used to analyze CACNA1C mutations-induced calcium handling. Differences of electrical properties between endocardial- and epicardial- cells (ENDO-EPI), between endocardial- and midmyocardial- cells (ENDO-M) and between epicardial- and midmyocardial- cells (EPI-M) may contribute to transmural electrical heterogeneities in tissues (Zhang et al., 2008). Transmural electrical heterogeneity caused by CACNA1C mutations was assessed by quantifying [Ca2+]<sup>i</sup> amplitude ([Ca2+]i(m) ), SR calcium content ([Ca2+]SR(m) ), Irel amplitude (Irel(m)), and APD. The rate dependence of APD was also investigated by using a pacing cycle length (PCL) of 2000, 1000, and 500 ms, respectively.

The maximum slope of the APD restitution (APDR) curve is a main marker for determining whether alternans and spiral breakup will occur (Ten Tusscher and Panfilov, 2006a). We used both the standard S1-S2 and dynamic protocols to determine APDR. For the S1-S2 protocol, 30 S1 stimuli were applied at a PCL of 1,000 ms, and the S2 stimulus was applied at varying diastolic intervals (DI) after the AP evoked by the last S1 stimulus. For the dynamic protocol, a series of 30 stimuli were applied at a PCL of 1,000 ms, after which the PCL was decreased. APDR curves were obtained by plotting APD against DI.

### Multicellular 1D, 2D, and 3D Models

**Figure 2** shows multicellular tissue models which consist of a 1D transmural ventricular fiber (**Figure 2A**), a 1D MCELL homogeneous cable (**Figure 2B**), a 2D transmural ventricular sheet (**Figure 2C**), and a 2D MCELL tissue (**Figure 2D**).

The 1D transmural ventricular fiber of length 24.75 mm (**Figure 2A**), which contained a 9 mm long ENDO region, a 6.75 mm long MCELL region and a 9 mm long EPI region, was constructed to compute a pseudo-ECG and investigate the tissue vulnerability to unidirectional conduction block. The fiber model consisted of 60 ENDO nodes, 45 MCELL nodes and 60 EPI nodes (O'Hara et al., 2011). Following the application of a sequence of 10 conditioning S1 pulses (with amplitude of −40 µA/cm<sup>2</sup> for 3 ms) applied to a 0.45 mm long ENDO segment at a PCL of 1,000 ms, repolarization characteristics were evaluated by computing repolarization time (RT), DOR, and maximum spatial gradient of APD (MSG). RT was computed as the largest APD in the transmural cable. DOR was measured as the difference between the largest and smallest APD of cells in the fiber. Spatial gradient of APD (SG) along the transmural cable was calculated as changes of APD per millimeter. In addition, the pseudo-ECG was obtained by using the method of Gima and Rudy (Gima and Rudy, 2002; formulations are listed in Supplementary Material). QT

interval, T-wave width and T-wave amplitude were quantified to examine the changes in the ECG due to CACNA1C mutations. QT interval was estimated as the time interval between the Qwave onset and the T-wave end, T-wave width was computed as the peak and the end of the T-wave, and T-wave amplitude was defined as the peak voltage of the T-wave. The T-wave end was determined by the intersection point of the baseline (y = 0 mV) and the T wave. The inducibility of unidirectional conduction block for each mutant CACNA1C was also quantified by computing the maximum PCL (MPCL) that produced 2:1 block.

A 1D MCELL cable of length 24.75 mm (**Figure 2B**) was used to measure conduction velocity (CV) by calculating the time 1t for the wavefront to propagate from x − 1x to x + 1x, defining CV = 2 1x/1t. To examine whether focal activity in the 1D cable was induced at a PCL of 500 ms, solitary planar waves were initiated by applying S1 pulses (with the same size, strength and duration as the one used for the 1D transmural simulation).

A transmural ventricular sheet (**Figure 2C**), which was constructed by expanding the 1D transmural fiber into a sheet with a length of 24.75 mm and a width of 150 mm, was developed to examine tissue vulnerability to the initiation of reentrant waves. A S1-S1 stimulation (with the same strength and duration as the one used for the 1D simulations) was applied to a region of 0.45 × 75 mm<sup>2</sup> , located at the center of the left ENDO boundary.

A 2D MCELL tissue model of 375 × 375 mm<sup>2</sup> (**Figure 2D**) was developed to investigate effects of CACNA1C mutations on the spatiotemporal behavior of spiral waves. Spiral waves were induced by a standard S1–S2 stimulation. A plane wave was initiated by applying the S1 stimulus to the left side of the domain (0.75 × 375 mm<sup>2</sup> ). Once the plane wave had passed over the first half of the domain, the S2 stimulus was applied to the first

quarter of the domain so that a spiral wave was produced. AP was recorded from the representative point (x = 187.5 mm, y = 187.5 mm) and the fundamental frequency was obtained from the power spectra of the AP.

For the 3D model, simulations were performed using an anatomical human ventricular geometry developed in our previous studies (Bai et al., 2015, 2016c). It has a spatial resolution of 0.5 mm with ∼6.38 million cells in total. For both left and right ventricles, the tissue was segmented into distinctive ENDO, MCELL, and EPI layers with similar contiguous figurations in the transmural wall as in the 1D transmural ventricular fiber model. The S1-S1 stimulation was applied to a 2.5 mm wide region of the ENDO layer. For the purposes of the present study, the 3D anatomical model was assumed to be electrically homogeneous.

### Numerical Methods

The cell models were incorporated into a parabolic partial differential equation (PDE) to construct mono-domain models of cardiac electrophysiology to describe the reaction-diffusion system in simulating cardiac dynamics (Clayton and Panfilov, 2008). The governing equation is

$$\mathcal{C}\_{\mathfrak{m}} \frac{\partial V\_{\mathfrak{m}}}{\partial t} \; \; \;= \mathcal{D} \nabla^2 V\_{\mathfrak{m}} - I\_{\text{ion}} \tag{1}$$

where **C**<sup>m</sup> = 1 µF/cm<sup>2</sup> is the capacitance, **D** is the effective diffusion constants, and **I**ion is the total transmembrane current.

We used a forward-Euler method for marching, with a time step (∆t) of 0.02 ms and a space step (∆x = ∆y = ∆z), to solve the PDEs (Equation 1). The value of D is set to be 0.0385 mm<sup>2</sup> /ms for simulating excitation waves in the 1D transmural ventricular fiber, the 1D homogeneous cable, the 2D transmural ventricular sheet and the 3D ventricular model. The spatial resolution in 1D, 2D and 3D models are as follows: 1D ventricular model: ∆x = 0.15 mm; 2D transmural ventricular model: ∆x = ∆y = 0.15 mm; 3D ventricular model: ∆x = ∆y = ∆z = 0.5 mm. For the 2D MCELL ventricular model, D is set to be 0.154 mm<sup>2</sup> /ms and the spatial resolution is chosen to be 0.25 mm to investigate the role of APDR in the occurrence of electrical instability (Ten Tusscher et al., 2004; Vandersickel et al., 2014; Nayak and Pandit, 2015; Zimik et al., 2015; Nayak et al., 2017). We also used Neumann (i.e., no-flux) boundary conditions (Clayton and Panfilov, 2008). Simulations were carried out on a 64-bit Intel core i7-3930K CPU system with 64 GB memory. Efficient parallelization was implemented using GPU acceleration (Bai et al., 2015). Although different time, space and diffusion coefficient were used, solutions to the mono-domain model with isotropic diffusion fulfilled the stability criterion: (i.e., D∆t/ ∆x <sup>2</sup> < 1/2; Clayton and Panfilov, 2008).

sarcoplasmic reticulum (SR) calcium concentration (*[Ca2*+*]SR*, <sup>F</sup>) in the G1783C (Magenta), wild-type (WT, Black), P381S (Red), M456I (Green), A582D (Blue), and R858H (Cyan) conditions are shown. The pacing cycle length (PCL) used in these simulations is 1,000 ms. Action potential duration, *INCX* , *I rel*, *[Ca2*+*] i ,* and *[Ca2*+*]SR* augment with an increase in *ICaL*. Changes in action potential duration are indicated by an enlargement of the (A).

### RESULTS

### Effects of CACNA1C Mutations on Intracellular Calcium Handling and Action Potential

ICaL arising from the CaV1.2 mutations altered intracellular calcium handling and prolonged AP as shown in **Figure 3A**. For ENDO cells, the ICaL amplitude was increased progressively from 4.12 pA/pF in the G1783C condition (Magenta) to 4.28 pA/pF (WT, Black), 4.35 pA/pF (P381S, Red), 4.44 pA/pF (M456I, Green), 4.71 pA/pF (A582D, Blue), and 5.0 pA/pF (R858H, Cyan), respectively (**Figure 3B**). Altered intracellular calcium handling resulted from increased ICaL was shown by the time courses of INCX(**Figure 3C**), Irel (**Figure 3D**), [Ca2+]<sup>i</sup> (**Figure 3E**) and [Ca2+]SR(**Figure 3F**). Changes in [Ca2+]i(m) , [Ca2+]SR(m) , and Irel(m) were related to an increase in ICaL (changes are summarized in Table S2). Increased ICaL during the AP plateau triggered a large Irel, leading to cytoplasmic calcium overload and APD prolongation. The measured APD was 264.4 ms (G1783C), 267.8 ms (WT), 271.8 ms (P381S), 277 ms (M456I), 282.6 ms (A582D), and 294 ms (R858H), respectively.

Simulations of transmural electrical heterogeneity were performed for each mutation, producing APs of ENDO, MCELL, and EPI cells. [Ca2+]i(m) , [Ca2+]SR(m) , Irel(m) , and APD were used to quantify electrical properties of the different cell types (listed in Table S2). The electrical differences associated with each mutant CACNA1C for ENDO-EPI, ENDO-M as well as EPI-M are summarized in Table S3. There were small electrical differences for ENDO-EPI, whilst large electrical heterogeneities were found for EPI-M and ENDO-M. For one "mutant" ICaL tested (R858H), [Ca2+]SR(m) of ENDO-EPI (∼0.1 mM) was smaller than that of EPI-M (∼0.64 mM) as well as ENDO-M (∼0.74 mM). Among in these mutations, the CaV1.2 R858H mutation with the largest effect on electrophysiological heterogeneity was present. For the [Ca2+]SR(m) , the ENDO-M difference for the R858H condition was larger (∼0.74 mM) than the one (∼0.64 mM) produced by the presence of the WT ICaL, while other mutations caused changes ranging from ∼0.59 mM to ∼0.69 mM. Other characteristics of transmural heterogeneity are listed in Table S3, which demonstrate that the largest electrophysiological heterogeneity is with the R858H condition.

### Effects of CACNA1C Mutations on Induction of Afterdepolarizations

The effects of ICaL associated with CACNA1C mutations on AP shape are shown in **Figure 4**. Cell simulations were conducted by increasing the pacing frequency from 0.5 to 2 Hz [corresponding to PCL of 2,000 ms (Green), 1,000 ms (Red) as well as 500 ms (Black)] and the measured APDs for G1783C, WT, P381S, M456I, A582D, and R858H cells are listed in Table S4. For the ENDO (**Figure 4**, left column), MCELL (**Figure 4**, middle column) and EPI (**Figure 4**, right column) cells, APD<sup>90</sup> was

abbreviated with PCL and no afterdepolarizations were triggered under the G1783C, WT, P381S, M456I, and A582D conditions. However, under the R858H condition, a DAD in the MCELL cells was induced, but no ectopic beats in the EPI and ENDO cells were

afterdepolarizations (DAD, marked with a red circle) were triggered in MCELL cells.

observed, when the PCL was 500 ms. To illustrate afterdepolarizations generating events, the time courses of AP, ICaL, INCX, Irel, [Ca2+]<sup>i</sup> , and [Ca2+]SR for the ENDO (**Figure 5**, left column), MCELL (**Figure 5**, middle column) and EPI (**Figure 5**, right column) cells at PCL of 500 ms (Red) and 1,000 ms (Black) are shown. As can be seen, ICaL arising from the R858H mutation contributed to an increase in [Ca2+]SR, which, consequently, enhanced spontaneous Irel, accompanied by an inward INCX that depolarized the cell. Subsequently, EADs (marked with a blue circle) and DADs (marked with red circles) were triggered in R858H-MCELL cells. Compared with R858H-ENDO and R858H-EPI cells, spontaneous Irel frequency was higher in R858H-MCELL cells. Additionally, spontaneous Irel occurred during repolarization and therefore led to the EADs, however, DADs were induced by spontaneous Irel after repolarization. In addition, the APD (≈421.2 ms) of the DAD-induced AP was larger than that (348.2 ms) of the R858H-MCELL cells at a PCL of 500 ms, indicating QT interval prolongation at rapid heart rates.

## Effects of CACNA1C Mutations on APD Restitution

The effects of the mutant ICaL on ventricular APDR are shown in **Figure 6**. The APD reduction was rate-dependent for ENDO (**Figures 6A,D**), EPI (**Figures 6B,E**) and MCELL (**Figures 6C,F**) cells. Across the range of DIs tested, the measured APD was larger for the R858H (Cyan) condition than for the G1783C (Magenta), WT (Black), P381S (Red), M456I (Green), and A582D (Blue) conditions. For the R858H settings, the maximum restitution slope of 2 for MCELL cells (**Figure 6E**) was larger than those for ENDO (**Figure 6D**, 1.35) as well as EPI cells (**Figure 6F**, 1.5). For MCELL cells, the computed APDR slopes of MCELL cells (**Figure 6E**) were 2 (G1783C), 1.8 (WT), 1.7 (P381S), 1.7 (M456I), 1.7 (A582D), and 2 (R858H), respectively. Details of maximum APDR slopes for three cell types can be found in Table S5.

middle column) and epicardial (EPI, right column) cells, time courses of membrane potential (Em), underlying L-type calcium current (*ICaL*), the sodium-calcium exchanger current (*INCX* ), calcium induced calcium release flux (*I rel*), cytoplasmic calcium concentration (*[Ca2*+*] i* ) as well as sarcoplasmic reticulum (SR) calcium concentration (*[Ca2*+*]SR*) at the pacing cycle length (PCL) of 500 (Red) and 1,000 ms (Black), respectively. Among different cell types, in MCELL cells early afterdepolarizations (EAD, marked with a blue circle) and delayed afterdepolarizations (DAD, marked with a red circle) occurred at the PCL of 500 ms. These afterdepolarizations were triggered by *I rel* via *INCX* .

### Effects of CACNA1C Mutations on ECG

To examine the manifestation of the temporal and spatial dispersion of AP in the pseudo-ECG, AP propagation (**Figure 7A**), spatial distribution of APD (**Figure 7B**), spatial gradient of APD (**Figure 7C**) and the computed pseudo-ECG (**Figure 7D**) of 1D transmural strand at a PCL of 1,000 ms for each "mutant" ICaL are shown. Repolarization time (**Figure 7E**), DOR (**Figure 7F**), the maximum spatial gradient of APD at the EPI-M junction (**Figure 7G**), QT interval (**Figure 7H**), and T wave width (**Figure 7I**) were computed for the G1783C, WT, P381S, M456I, A582D, and R858H conditions. As can be seen in **Figure 7D**, the QT interval was prolonged from 397.1 ms (G1783C) to 398.5 ms (WT), 401.2 ms (P381S), 408.2 ms (M456I), 415.9 ms (A582D), and 425.7 ms (R858H), respectively (**Figure 7H**). The QT interval prolongation underlying increased APD in single cells has been linked to repolarization time (RT; Gima and Rudy, 2002). Therefore, the effects of the mutant ICaL on RT were examined. The computed RT was 360.48 ms (G1783C), 363.6 ms (WT), 367.34 ms (P381S), 374.84 ms (M456I), 381.14 ms (A582D), and 393.32 ms (R858H), respectively (**Figure 7E**). The QT prolongation was consistent with the prediction of RT. Moreover, the APD of MCELL cells with the longest repolarization time was increased from 359 ms (G1783C) to 364.2 ms (WT), 370.4 ms (P381S), 379.6 ms (M456I), 389.4 ms (A582D), and 410.8 ms (R858H), respectively (listed in Table S6). The QT interval prolongation can be attributed to ICaL due to CACNA1C mutations that influence APD in single cells and ventricular RT in ventricular tissues.

In addition, changes in the T wave were examined. As is shown in **Figure 7H**, the T-wave width was 60 ms (G1783C), 59.9 ms (WT), 59.9 ms (P381S), 61.7 ms (M456I), 63.9 ms (A582D), and 64.5 ms (R858H), respectively (**Figure 7I**). The augmented T-wave width has been attributed to an increase in DOR (Antzelevitch et al., 1998), therefore the effects of

and for the R858H mutation, respectively. Among in these mutations, the R858H mutation obviously shifts APDR curves upwards.

the mutant ICaL on APD dispersion in the transmural strand were also investigated. The computed DOR was enhanced from 60.94 ms (G1783C) to 61.3 ms (WT), 62.2 ms (P381S), 64.12 ms (M456I), 65.64 ms (A582D), and 67.96 ms (R858H; **Figure 7F**). In addition, it has been suggested that DOR is a marker of electrical heterogeneity in APD (Antzelevitch, 2001). Comparatively, the maximal APD difference for EPI-M was 93 ms (G1783C), 94.8 ms (WT), 97 ms (P381S), 101 ms (M456I), 104.8 ms (A582D), and 115.2 ms (R858H), respectively (summarized in Table S3). The APD heterogeneity induced by CACNA1C mutations may contribute to DOR and thereby to T-wave width.

Similarly, T-wave amplitudes were calculated. The T-wave amplitude was 1.051 mV (G1783C), 1.053 mV (WT), 1.056 mV (P381S), 1.075mV (M456I), 1.087 mV (A582D), and 1.104 mV (R858H), respectively (listed in Table S6). Compared with the WT condition, there was no evident difference in T-wave amplitude. Changes of T-wave amplitude can be attributed to altered temporal and spatial gradients in membrane potential (Gima and Rudy, 2002; Zhang et al., 2008). Therefore, the APD gradient (**Figure 7C**) and the membrane potential gradient (δ, **Figure 8**) were computed. **Figure 8A** shows simulated ENDO (Black), EPI (Red), MCELL (Green) APs for each mutant condition whilst **Figure 8B** shows corresponding time-course plots of δ for EPI-ENDO (Light Gray), EPI-M (Gray), and ENDO-M (Dark Gray). The maximal δs for EPI-M (**Figure 8D**) and ENDO-M (**Figure 8E**) were greater than that for EPI-ENDO (**Figure 8C**). This was consistent with the prediction of electrical heterogeneities of AP in single cells (summarized in Table S3). There was no significant change in the maximum EPI-ENDO δ between these mutations (**Figure 8F**). However, the maximum EPI-M δ was 73.8 mV (G1783C), 74.2 mV (WT), 75.2 mV (P381S), 75.6 mV (M456I), 77.1 mV (A582D), and 79.4 mV (R858H), respectively (**Figure 8G**), and the maximum ENDO-M δ was 91.59 mV (G1783C), 92.06 mV (WT), 93.06 mV (P381S), 93.14 mV (M456I), 94.69 mV (A582D), and 96.27 mV (R858H), respectively (**Figure 8H**). Also, the maximum spatial gradient (MSG) of APD at the EPI-M junction was 18 ms/mm (G1783C), 18.1 ms/mm (WT), 18.5 ms/mm (P381S), 18.9 ms/mm (M456I), 19.2 ms/mm (A582D), and 19.5 ms/mm (R858H), respectively (**Figure 7G**). Changes in MSG at the EPI-M junction and maximal EPI-M δ were consistent with the altered T-wave amplitude.

Taken together, the ECG phenotypes were dependent on changes of AP caused by altered ICaL due to CACNA1C mutations in single cells and the longest QT interval was obtained under the R858H condition (listed in Table S6).

### Effects of CACNA1C Mutations on Action Potential Propagation

To investigate the cellular level conditions required for afterdepolarizations to trigger a premature ventricular complex

FIGURE 7 | Space-time plot of action potential propagation and the computed pseudo-ECG. (A) Color mapping of membrane potential of cells along the one-dimensional (1D) strand from blue (−86 mV) to red (+42 mV). Space runs from the endocardial (ENDO) region to the midmyocardial (MCELL) and epicardial (EPI) regions. Spatial distribution of action potential duration (APD, B), spatial gradient of APD (C), and pseudo-ECGs (D) corresponding to the G1783C (Magenta), wild-type (WT, Black), P381S (Red), M456I (Green), A582D (Blue), and R858H (Cyan) conditions, respectively. Computed repolarization time (RT, E), dispersion of repolarization (DOR, F), maximum spatial gradient of APD (MSGA) at the epicardial- and midmyocardial- (EPI-M) junction (G), QT interval (H) and T wave width (I) are shown. From bottom to top, curves for the spatial distribution of APD are for the G1783C mutation, for the WT condition, for the P381S mutation, for the M456I mutation, for the A582D mutation and for the R858H mutation, respectively. Changes in the spatial gradient of APD at the EPI-M junction (marked with a red circle) are indicated by an enlargement of the (C). Changes in the T wave (marked with a red circle) are indicated by an enlargement of the (D). RT, DOR, MSGA at the EPI-M junction and QT interval augment with an increase in APD. T wave widths show no apparent differences between G1783C, WT and P381S.

(PVC) at the multicellular tissue level, a 24.75-mm long MCELL strand model was constructed and space-time plots of AP propagation in the 1D homogeneous cable at the PCL of 500 ms are shown in **Figure 9**. For the G1783C, WT, P381S, M456I and A582D conditions, no focal activity was observed whilst a R858H-mediated focal activity (marked with a white rectangle) was triggered at Time = 35,000 ms. Although a DAD mediated-AP was induced in MCELL cells (**Figure 5, middle column**), such simulations may not necessarily reflect the situation for intact tissue, in which electrical coupling occurs between cells and may smooth out these electrical differences. Thus, the propagation of the DAD mediated-AP was not observed (Figure S1) and thereby no PVC occurred under the R858H condition.

To identify the electrophysiological substrates that promote arrhythmogenesis, tissue vulnerability to unidirectional conduction block necessary to the genesis of reentry was investigated, and space-time plots of AP propagation in the 1D transmural cable at different PCLs are shown in **Figure 10**. As can be seen, the tissue susceptibility to unidirectional conduction block showed a RT dependency and increased with a decrease in PCL. For instance, the MPCL (listed in Table S6) increased from 350 ms (G1783C) to 352 ms (WT), 354 ms (P381S), 356 ms (M456I), 365 ms (A582D), and 370 ms (R858H), respectively, with RT prolongation from 360.48 ms (G1783C) to 363.6 ms (WT), 367.34 ms (P381S), 374.84 ms (M456I), 381.14 ms (A582D), and 393.32 ms (R858H), respectively. As for R858H conditions, no unidirectional conduction block was observed at PCL = 1,000 ms, while unidirectional conduction block occurred at PCL = 370 ms. Bidirectional conduction was obtained in G1783C, WT, P381S, M456I, and A582D conditions at PCL = 370 ms. Therefore, R858H tissue was more vulnerable to unidirectional conduction block than other type tissues.

### Effects of CACNA1C Mutations on Dynamic Behavior of Reentrant Excitation Waves in 2D and 3D Models

To examine if mutations-induced changes in APDR curves promote the breakup of a spiral wave, an idealized homogeneous tissue model was constructed and spiral waves were induced by S1-S2 stimulation. **Figure 11** shows snapshots of spiral waves (Time = 5,000 ms, left column), time series of action potentials (middle column) from the point indicated by an asterisk (left column) and power spectra (right column) for the G1783C, WT, P381S, M456I, A582D, and R858H conditions. Reentrant waves were stable and persistent. The fundamental frequencies obtained from these power spectra of action potentials were

region (marked with a white rectangle) close to the pacing site exhibiting a subthreshold delayed afterdepolarization (DAD) with no premature ventricular complex (PVC) under the R858H condition. Changes in membrane potential of the local region are indicated by an enlargement of the marked zone. Color mapping of membrane potential from blue (−85 mV) to red (−73 mV).

2.8 Hz. These ordered plots of action potential provided evidence for the rotating spiral state.

To determine if R858H-induced changes were necessary to induce ventricular arrhythmias, an idealized transmural tissue model was developed to investigate the initiation of reentry with a S1-S1 stimulation. As is shown in **Figure 12**, when the PCL was 370 ms, the first S1 stimulus (t = 0 ms) produced a wave to propagate from the ENDO layer to the EPI layer. The second S1 stimulus (t = 370 ms) produced a conditioning wave and no unidirectional conduction block occurred under G1783C, WT, P381S, M456I, and A582D conditions (t = 750 ms). A unidirectional conduction block was initiated under the R858H condition, leading to the genesis of spiral waves (t = 750 ms) and fibrillation-like activity. The reentry wave for the R858H conditions was unstable and promoted self-termination when it collided with its own prolonged repolarization tail or tissue borders (t = 1,000 ms). These results support the previous notion that the MPCL of 2:1 block for the R858H mutation is 370 ms (shown in **Figure 10**) and unidirectional conduction block is responsible for the genesis of reentry. These 2D results concur with the 1D simulations data, further illustrating the pro-arrhythmic effects of the R858H mutation. In addition, reentry can be induced under other conditions (Figure S2) if the S1-S1 interval was decreased. For instance, the S1-S1 interval for initiating spiral waves was 365 ms (R858H), 340 ms (A582D), 335 ms (M456I), 320 ms (P381S), 315 ms (WT), and 310 ms (G1783C), respectively.

Further simulations were performed in a 2D slice model. **Figure 13** shows that the R858H ICaL induced reentrant spiral waves, which lead to sustained multiple reentrant wavelets in a 2D ventricular slice. Snapshots of excitation waves at different time points (t = 10, 350, 440, 520, and 990 ms) are shown for the G1783C, WT, P381S, M456I, A582D, and R858H conditions. As can be seen, a S1 stimulus was applied to the four pacing sites on the ENDO layer (marked with white asterisks, t = 10 ms) and excitation waves were initiated. At t = 350 ms, another S1 stimulus was used to induce spiral waves. At t = 440 ms, excitation waves propagated to the whole sheet under the G1783C and WT conditions, local conduction block was observed under the P381S, M456I, and A582D conditions, and unidirectional conduction block occurred under the R858H condition. Several spiral waves were produced at t = 520 ms and sustained reentry

was observed at t = 990 ms. It was found that spiral waves can be induced under other conditions by decreasing S1-S1 interval. The S1-S1 interval for initiating spiral waves was 310 ms (G1783C), 315 ms (WT), 320 ms (P381S), 325 ms (M456I), 330 ms (A582D), and 345 ms (R858H), respectively (Figure S3). Detailed movies of spiral wave initiation can be found in Videos S1–S6.

To examine if R858H-induced changes promote ventricular arrhythmias in human ventricles, further simulations were performed using a 3D human heart geometry. As is shown in **Figure 14**, the first S1 stimulus (t = 10 ms) produced a wave to propagate from the ENDO layer to the EPI layer. Ventricular repolarization (marked with a black circle or a blue circle) under the WT condition occurred much earlier than that for the R858H settings (t = 340 ms). At t = 350 ms, another excitation wave initiated by the second S1 stimulus conducted bidirectionally (marked with a bidirectional arrow) under the WT condition whilst the wave was locally blocked (marked with unidirectional arrows) by unrecovered tissues (marked with a red circle). Therefore, spiral waves (marked with unidirectional arrows) were induced under the R858H condition (t = 700 ms and t = 800 ms), and reentrant waves were persistent throughout the simulation (t = 990 ms). Moreover, when the S1-S1 interval was gradually decreased, spiral waves can also be induced (Figure S4). In details, the S1-S1 interval for initiating spiral waves was 250 ms (G1783C), 300 ms (WT), 309 ms (P381S), 340 ms (M456I), 345 ms (A582D), and 348 ms (R858H), respectively. Detailed movies of the spiral wave initiation can be found in Videos S7–S12.

Taken together, these data demonstrated that the R858H mutation facilitates initiation of reentrant excitation waves and suggested that the R858H tissue was more vulnerable to the initiation of reentrant excitation waves than other type tissues. A summary of the effects of CACNA1C mutated ICaL on simulated human ventricular electrical activity is listed in **Table 1**.

## DISCUSSION

#### Summary of Major Findings

To our knowledge, this is the first study to investigate mechanisms underlying the genesis of VF in patients with the CACNA1C R858H mutation. The study found five major findings, (i) due to the R858H mutation ICaL caused intracellular calcium overload, resulting in afterdepolarizations in single cells and focal activity in 1D fiber tissues; (ii) The R858H mutation induced an increase in ICaL prolonged APD and augmented RT, leading to QT interval prolongation; (iii) The R858H mutationinduced electrical differences between cells augmented electrical heterogeneity, causing repolarization dispersion and thereby increasing T-wave width; (iv) Although, the R858H mutation steepened the APDR relationships in single cells, a stable spiral wave remained in homogeneous tissues; (v) Changes in cellular electrophysiology modulated wave conduction at tissue level facilitating unidirectional conduction block and thereby

increasing tissue susceptibility to VF genesis in transmural ventricular tissues.

These simulation data in this study constitute novel evidence that the pro-arrhythmic effects of ICaL associated with CaV1.2 R858H mutation involve both increased cell susceptibility to afterdepolarizations and tissue vulnerability to the reentry. The effects of the R858H mutation were investigated at cellular, 1D strand and 2D tissue and 3D organ levels, showing not only alterations in calcium handling and QT prolongation, but also repolarization dispersion and reentry.

## Computer Modeling of CACNA1C Mutations

In this study, the electrophysiological consequences of CACNA1C mutations were investigated by modeling the ICaL, as in previous studies (Faber et al., 2007; Zhu and Clancy, 2007; Sung et al., 2010; Morotti et al., 2012; Boczek et al., 2015a; Bai et al., 2016c). Based on experimental data on ICaL current densities, steady-state activation curves, voltage-dependent inactivation curves as well as the time constant for the voltage-dependent inactivation (Fukuyama et al., 2014), the ICaL model was

developed by changing the ICaL conductance, the half-activation voltage, the slope of the activation curve, the half-inactivation voltage, the slope of the inactivation curve and the scaling factor of inactivation time constant. The "mutant" ICaL models successfully reproduced the I-V relationships obtained from experimental studies (Fukuyama et al., 2014). These ICaL models were incorporated into a human ventricular cell model, allowing us to relate changes in ICaL to AP at the cellular level, in agreement with previous studies (Splawski et al., 2005; Sung et al., 2010; Wemhöner et al., 2015; Bai et al., 2016c). According to transmural ventricular wedge preparation models (Gima and Rudy, 2002; O'Hara et al., 2011), the 1D transmural strand model, integrating human ENDO-, MCELL- and EPI- cells, was constructed to compute unipolar pseudo-ECGs. In the cable, the proportion of each region composed of each distinct cell type is consistent with that used in other studies (O'Hara et al., 2011). In agreement with clinical findings, R858H-induced ICaL led to a QT interval of 425.7 ms, which is within the QT range (420–476 ms) of R858H-porbands (Fukuyama et al., 2014). These models could be considered as a first step toward the validation of electrophysiological model. Thus, cardiac models provide a powerful tool for the study of mechanisms underlying ventricular arrhythmias caused by CACNA1C mutation effects.

Previous studies have demonstrated that distinct mutations can have variable effects on current morphology and lead to varying degrees of electrophysiological consequences, depending on kinetic changes induced by the CACNA1C mutation (Sung et al., 2010; Wemhöner et al., 2015; Sutphin et al., 2016). The "mutant" ICaL represents state-specific kinetic properties of ion channels, allowing us to relate functional changes in ICaL to AP at the cellular level, the electrocardiogram characteristics at the fiber tissue level and the spatiotemporal behavior of excitation waves at the sheet tissue level. In the present study, although G1783C, P381S, M456I, A582D, and R858H mutations changed CaV1.2 in activation and inactivation, VF was only observed in R858H patients (Fukuyama et al., 2013, 2014). Therefore, we have focused on the R858H mutation marked by an increase in the ICaL and have investigated mechanisms underlying the genesis of reentry.

waves at *t* = 990 ms under the R858H condition.

## Ionic Mechanism of Afterdepolarizations in R858H Cells

These analyses of R858H revealed an unexpected, distinct electrophysiological phenotype from the classical TS mutations (G406R/G402S; Splawski et al., 2004, 2005). Electrophysiological studies of G406R in exon 8A and G406R/G402S in exon8 showed almost complete loss of inactivation of CaV1.2 (Splawski et al., 2004, 2005). In contrast, R858H showed a significant gain of current density, ∼2 mV negative shift of activation and ∼2 mV positive shift of inactivation, resulting in an increased window current (Fukuyama et al., 2014). Due to the different electrophysiological phenotypes between the canonical G406R/G402S mutations and R858H, we performed modeling studies to better understand how the increased window current may affect the ventricular AP. Like the modeling studies of G406R/G402S (Splawski et al., 2004; Zhu and Clancy, 2007), our results showed prolongation of APD in TP06 and O'Hara-Rudy dynamic (ORd) human ventricular cell models (Figure S5). At fast pacing rates, R858H-induced SR calcium loading, which may lead to spontaneous afterdepolarizations, was also observed (Figure S6). The result of R858H-induced afterdepolarizations is consistent with other reports that pro-arrhythmic actions of CACNA1C mutations are due to SR calcium leak, thereby enabling an INCX that triggers afterdepolarizations (Splawski et al., 2005). In agreement with experimental findings (Priori and Corr, 1990), the R858H induced EADs and DADs share the same mechanism, where NCX-mediated inward current is involved.

These afterdepolarizations may mainly result from an increase ICaL arising from the R858H mutation, because the VF in the R858H patients was completely prevented by atenolol, a betablocker (Fukuyama et al., 2014). Thus, our results support the concept that the R858H-induced defect is sufficient to cause significant action potential prolongation and afterdepolarizations in ventricular myocytes. Our data highlight how small changes in cellular calcium entry through CaV1.2 can lead to far-reaching changes in action potentials and intracellular calcium handling.

Computational modeling has shed valuable light on the cellular mechanisms underlying ventricular arrhythmia. Splawski

et al. showed that a G406R mutation prolonged action potentials, altered intracellular calcium handling, and triggered DADs (Splawski et al., 2005). Sung et al. showed that G406R-induced EADs and DADs shared the same mechanism, with NCXmediated inward current (Sung et al., 2010). However, how the G406R mutation is linked to ventricular arrhythmias is not yet fully understood. Thiel et al. showed that a G406R mutation altered intracellular calcium handling and calcium signaling, which could in turn contribute to DADs due to calcium signalinginduced SR calcium overload (Thiel et al., 2008). Recently, Dick et al. suggested that TS mutations (G406R/ G402S) disrupted calcium-dependent inactivation, which played an important role in APD prolongation and development of EADs (Dick et al., 2016). Different from the mechanism underlying G406R-induced arrhythmias, an I1166T mutation led to an EAD due to ICaL reactivation (Boczek et al., 2015a). Other CACNA1C mutations also caused QT prolongation and cardiac arrhythmias (Gillis

et al., 2012; Boczek et al., 2013, 2015b; Fukuyama et al., 2013, 2014; Hennessey et al., 2014; Wemhöner et al., 2015; Landstrom et al., 2016; Sutphin et al., 2016), but their cellular mechanisms underlying cardiac arrhythmias remain relatively unexplored, and the causal link between the CACNA1C mutations and ventricular arrhythmias is not addressed directly. In this study, the cellular mechanism that ICaL due to the R858H mutation prolongs APD and increases SR calcium load, resulting in spontaneous Irel and afterdepolarizations, is proposed. These simulation data of the present study add to the growing weight of evidence implicating cellular mechanisms of CACNA1C mutations.

#### Reentry Initiation Mechanism

The LQT8 is associated with malignant ventricular arrhythmias and a patient with a R858H mutation experienced an episode of VF (Fukuyama et al., 2013, 2014). Our data showed that

red circle and the S1-S1 interval was 348 ms.



R858H-induced electrical heterogeneity increased the transmural DOR and favored unidirectional conduction block necessary to the development of VF (Zhu and Clancy, 2007; Sung et al., 2010). The simulations may provide evidence for the proarrhythmic effects of the R858H mutation in facilitating reentrant excitation waves. They support the notion that DOR is a marker of tissue electrical heterogeneity and a substrate for reentrant arrhythmias (Antzelevitch and Fish, 2001; Chauhan et al., 2006). In agreement with previous studies (Zhang et al., 2008), cell simulation results indicate that membrane potential differences and inhomogeneous APDs of ventricular cardiomyocytes contribute to the tissue electrical heterogeneity. As for membrane potential differences, afterdepolarizations became inducible under the R858H condition when the SR calcium load was progressively increased in the MCELL cells. The maximal membrane potential differences both ENDO-M and EPI-M APs were significantly increased. As for inhomogeneous APD, the R858H mutation produced inhomogeneous APD prolongation in EPI, MCELL and ENDO cells, leading to the lager spatial gradient of APD along the transmural ventricular wall. Therefore, these electrical differences due to the R858H mutation at the cellular level contributed to an augmented transmural DOR, increasing tissue vulnerability to unidirectional conduction block.

Linking APD prolongation to unidirectional conduction block can provide insights into the risk of cardiac arrhythmias. The APD corresponding to the R858H model was longer than that of other models at S1-S1 = 1000 ms. The RT prolongation was consistent with the prediction of APD. At S1-S1 = 370 ms, the AP maps showed 2:1 block under the R858H condition. A similar 2:1 block was also observed under rapid conditions in the other heart models. Moreover, in the R858H heart, a local conduction block corresponded to unrecovered tissues and spiral waves were induced (**Figure 14**, the R858H heart model at S1-S1 = 348 ms). As the S1-S1 interval was further decreased (at PCL = 300 ms), we observed unidirectional conduction block and reentrant waves in the WT ventricles (Figure S4). This suggested that the prolonged APD tissues formed excitable obstacles which can anchor spiral waves.

Our simulations indicate that the pro-arrhythmic effect of the R858H mutation is reflected by the increased MPCL of 2:1 block. This is compatible with observations from previous studies (Zhu and Clancy, 2007), in which a long PCL led to 2:1 block under the G406R/G402S mutation condition. Compared with other mutations, unidirectional conduction block under the R858H condition occurred at a longer S1-S1 interval, implying higher risk of cardiac arrhythmias in R858H patients at relatively slow heart rates. Furthermore, the 2D and 3D simulated results support the notion that the R858H mutation increases the proarrhythmic risk. Although the spiral waves can be induced under all conditions, a relatively long S1-S1 interval is required for the initiation of reentrant waves under the R858H condition.

#### Mechanisms for VF

VF is the most common cause of sudden cardiac death. Fibrillation results when an electrical wave break induces reentry and triggers a cascade of new wave breaks (Weiss et al., 2005). In our simulations, three potential mechanisms underlying VF caused by the R858H mutation were considered, (a) the restitution hypothesis; (b) afterdepolarizations-mediated fibrillation hypothesis; and (c) dispersion of refractoriness hypothesis.

According to the restitution hypothesis, the chaotic excitation dynamics during VF are the result of dynamical instabilities in APD, the occurrence of which requires that the slope of the APD restitution curve exceeds 1 (Fenton et al., 2002). Although the APDR relationships were steepened by the R858H mutation, stable spiral wave dynamics occurred in homogeneous ventricular tissues. The measured maximum slope of APDR for the EPI cell was 1.5 which didn't exceed the minimum APDR slope (1.5) for the occurrence of spiral breakup in the TP06 model (Ten Tusscher and Panfilov, 2006a). Also, a spiral wave remained in the MCELL tissue with the APDR slope of 2 which was within the range of experimentally measured slopes (1.28–3.78; Pak et al., 2004). In addition, the slopes measured in the ORd model were within 1 under the R858H condition (Figure S7). Thus, our study confirmed that the effects of R858H-induced ICaL are model independent. These results indicated that steep APDR is not the mechanism of VF caused by the R858H mutation.

According to afterdepolarizations-mediated fibrillation hypothesis, abnormal electrical excitations caused by afterdepolarizations can disrupt the normal propagation of electrical waves and cause life-threatening arrhythmias like VF. The mechanisms of afterdepolarizations generation include the reactivation of ICaL and the enhancement of INCX arising from spontaneous calcium release. Simulation studies showed that three types of spiral fibrillation emerged when EAD was mainly induced by reactivation of ICaL (Vandersickel et al., 2014; Zimik et al., 2015). For the R858H condition, afterdepolarizations were caused by the enhancement of the INCX arising from spontaneous calcium release. Although, focal activity occurred in the 1D fiber tissue, electrical coupling between cells suppressed the formation of PVC under the R858H condition. Experimental and computational studies were done to investigate how clumps of cells, eliciting afterdepolarizations in synchrony, give rise to triggered activities, which can disturb any prevailing course of wave propagation and induce electrical-wave turbulence (Sato et al., 2009; de Lange et al., 2012; Myles et al., 2012). Local synchronization of afterdepolarizations cells was suggested the mechanism underlying the formation of PVCs. Of course, our previous study has demonstrated that afterdepolarizations induced by SR calcium overload are synchronized to overcome the source-sink mismatch and produce PVCs in cardiac tissues (Bai et al., 2017). These studies support the notion that R858Hinduced afterdepolarizations in MCELL cells are implicated in the induction of electrical-wave turbulence.

According to the dispersion of refractoriness hypothesis, wave break is produced by electrophysiological and anatomic heterogeneities in the tissue (Xie et al., 2001). For the R858H condition, intrinsic electrophysiological heterogeneity was sufficiently increased by the altered ICaL due to a R858H mutation. Electrophysiological heterogeneity produced DOR responsible for 2:1 conduction block necessary for the fibrillation-like activity. Simulation studies also demonstrated that this breakup mechanism did not require steep APDR (Xie et al., 2001; Fenton et al., 2002). Thus, these data indicated that DOR was a breakup mechanism for the R858H condition.

Taken together, steep APDR is not the mechanism responsible for VF in the current work. Our results suggest that R858H induced afterdepolarizations and transmural APD dispersion are responsible for the potential mechanisms underlying the formation of VF. For afterdepolarizations-mediated fibrillation, it must be synchronized across many cells (Xie et al., 2010; Myles et al., 2012). And DOR caused by the R858H mutation increased the susceptibility to dynamic instabilities.

#### Clinical Implications

The presented results showed a QT interval prolongation from 397.1 ms (WT) to 425.7 ms (R858H), which was within the range of the QTc interval (420–476 ms) in R858H-probands (Fukuyama et al., 2014). Thus, a bridge between CACNA1C mutations and pro-arrhythmic phenotypes was built by the cardiac model. Also, the model can be used to investigate the effect of therapies targeting selected ion channels. More importantly, defining a classification based on a reliable VF risk factor would be very useful in guiding the selection of LQT8 patients who are in need of a ventricular defibrillator. In the current work, we have shown that, DAD-mediated AP at fast pacing rates isn't shortened but further prolonged, indicating QT interval prolongation at fast pacing (Viskin et al., 2010; Adler et al., 2012; Kaye et al., 2013). The QT abnormality may become a clinical marker to determine the VF risk in LQT8 patients.

#### Limitations of the Study

The TP06 model was used to investigate the pro-arrhythmic effects of CACNA1C mutations and its limitations were discussed elsewhere (Zhang et al., 2008; Bai et al., 2016c). Here, we address several limitations in this study. (i) Due to the lack of a precise model of complex calcium cycling, this study does not consider the effect of a calcium spark on the genesis of afterdepolarizations and further refinement of the model is required. (ii) Our simulations show that R858H-induced defects can lead to the cellular arrhythmogenesis in the absence of neural influences. However, clinically observed arrhythmias typically occur in the setting of increased sympathetic tone (Best and Kamp, 2010), such as crying, and so future efforts will expand the study to investigate the effects of the autonomic nervous system on the development of cardiac arrhythmias. (iii) To maintain structures of the TP06 (Ten Tusscher and Panfilov, 2006a) and ORd (O'Hara et al., 2011) models, the original ICaL conductance was used for the WT condition and the maximum conductance of each "mutant" ICaL was determined by scaling relative current proportions. In agreement with the TP06 model (Ten Tusscher et al., 2004; Ten Tusscher and Panfilov, 2006a), the ICaL was same for ENDO, MCELL and EPI cells in the ORd model (O'Hara et al., 2011) and electrophysiological heterogeneity caused by different ICaL densities for three cell types was not considered in this study. Thus, special attention must be paid to explain these simulated data. (iv) In the multicellular tissue model, due to a lack of detailed experimental data on cell properties and well-delineated regions, the proportion of each region composed of each distinct cell type was chosen to produce a positive Twave in the 1D transmural cable, similar to those used in other studies (Gima and Rudy, 2002; O'Hara et al., 2011). However, simulated ECG, which might be influenced by torso effects and the dimensions of the regions, wasn't completely consistent with clinical findings (Fukuyama et al., 2014). (v) Fibrillation at the tissue level resulted from afterdepolarizations at a singlecell level was not observed, but modeling and experimental studies have suggested that spatial-temporal synchronization of SR calcium overload and release can overcome the sourcesink mismatch and produce focal arrhythmia in the heart (Xie et al., 2010; Myles et al., 2012), and so future efforts will expand the study to investigate that how the heart produces spatial-temporal synchronization of afterdepolarizations and the R858H mutation triggers VF in the heart. (vi) In the 1D and 2D tissue models, we assumed isotropic cell-to-cell electrical coupling. On the one hand, possible anisotropic intercellular electrical coupling may play important roles in the initiation and perpetuation of reentry. On the other hand, omitting anisotropy can be useful, in that changes to tissue behavior observed in the present study can be attributed with confidence to the implemented modifications to ICaL. (vii) The space step for the 2D homogeneous ventricular model is different from that used for 2D transmural ventricular sheet. The space step is chosen to produce a stable spiral wave close to those in other studies (Ten Tusscher and Panfilov, 2006a; Vandersickel et al., 2014; Nayak and Pandit, 2015; Zimik et al., 2015; Nayak et al., 2017), in that rotating spiral can be attributed with confidence to the APDR slope which is below the minimum value for the occurrence of electrical instability. (viii) Although the 2D homogeneous tissue was designed to examine if mutations-induced changes in APDR curves promote the breakup of a spiral wave and we expected the simulated results to hold in tissues of reasonable size, a reasonable domain cannot avoid the frequent termination of spiral waves by collisions with the boundaries of the simulation domain. Therefore, the size of the 2D homogeneous tissue chosen is similar to that used in other studies (Ten Tusscher and Panfilov, 2006a; Vandersickel et al., 2014; Nayak and Pandit, 2015; Zimik et al., 2015; Nayak et al., 2017), but it is larger than that of a typical human heart. Special attention must be paid to explain these simulated data. (ix) Due to a lack of fiber orientation data,

#### REFERENCES


fiber orientation was not included in the 3D model and its effect on reentry was not studied. Nevertheless, whilst it is important to make explicit the potential limitations of the approaches adopted in the present study, these potential limitations are not expected to influence fundamental conclusions that can be drawn from our data on the mechanisms by which increased ICaL due to the R858H mutation can increase risk of ventricular arrhythmias.

#### CONCLUSIONS

Computational modeling indicated that AP shape at the cellular level, ECG and unidirectional conduction block at the fiber tissue level, and spiral waves at the sheet tissue level as well as at the organ level under the R858H condition, provided a causal link between a R858H mutation and VF. Based on our findings, we propose that R858H induced transmural APD dispersion promotes the genesis of VF.

#### AUTHOR CONTRIBUTIONS

JB, KW, and HZ conceived and designed the experiments; JB performed the simulations, prepared figures and analyzed the results; JB, KW, and HZ drafted and edited the manuscript. YLiu, GL, SD, YLi, CL, and YY gave valuable suggestions. All authors reviewed the final version of the manuscript.

#### FUNDING

This work was supported by project grants from the National Key R&D Program of China (No. 2017YFC0113000), the National Natural Science Foundation of China (No: 61571165 and 61572152) and Science Technology and Innovation Commission of Shenzhen Municipality (No: JCYJ20151029173639477 and JSGG20160229125049615).

#### SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: http://journal.frontiersin.org/article/10.3389/fphys. 2017.00771/full#supplementary-material


multisystem disorder including arrhythmia and autism. Cell 119, 19–31. doi: 10.1016/j.cell.2004.09.011


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2017 Bai, Wang, Liu, Li, Liang, Luo, Dong, Yuan and Zhang. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Collective Sensing of β-Cells Generates the Metabolic Code

#### Dean Korošak 1,2,3 \* and Marjan Slak Rupnik 1,4,5 \*

1 Institute for Physiology, Faculty of Medicine, University of Maribor, Maribor, Slovenia, <sup>2</sup> Faculty of Civil Engineering, Transportation Engineering and Architecture, University of Maribor, Maribor, Slovenia, <sup>3</sup> Percipio Ltd., Maribor, Slovenia, <sup>4</sup> Center for Physiology and Pharmacology, Institute for Physiology, Medical University of Vienna, Vienna, Austria, <sup>5</sup> Alma Mater Europaea - European Center Maribor, Maribor, Slovenia

Major part of a pancreatic islet is composed of β-cells that secrete insulin, a key hormone regulating influx of nutrients into all cells in a vertebrate organism to support nutrition, housekeeping or energy storage. β-cells constantly communicate with each other using both direct, short-range interactions through gap junctions, and paracrine long-range signaling. However, how these cell interactions shape collective sensing and cell behavior in islets that leads to insulin release is unknown. When stimulated by specific ligands, primarily glucose, β-cells collectively respond with expression of a series of transient Ca2<sup>+</sup> changes on several temporal scales. Here we reanalyze a set of Ca2<sup>+</sup> spike trains recorded in acute rodent pancreatic tissue slice under physiological conditions. We found strongly correlated states of co-spiking cells coexisting with mostly weak pairwise correlations widespread across the islet. Furthermore, the collective Ca2<sup>+</sup> spiking activity in islet shows on-off intermittency with scaling of spiking amplitudes, and stimulus dependent autoassociative memory features. We use a simple spin glass-like model for the functional network of a β-cell collective to describe these findings and argue that Ca2<sup>+</sup> spike trains produced by collective sensing of β-cells constitute part of the islet metabolic code that regulates insulin release and limits the islet size.

Keywords: collective sensing, pancreatic islets, spin glass models, metabolic code, Ca2<sup>+</sup> imaging, Ca2<sup>+</sup> signaling, correlations, intercellular communication

### 1. INTRODUCTION

Endocrine cells in vertebrates act both as coders and decoders of metabolic code (Tomkins, 1975) that carries information from primary endocrine sensors to target tissues. In endocrine pancreas, energy-rich ligands provide a continuous input to a variety of specific receptor proteins on and in individual β-cells and initiate signaling events in and between these cells (Henquin, 2009). In an oversimplified medical physiology textbook interpretation, glucose is transported into a β-cell through facilitated diffusion, is phosphorylated and converted within a metabolic black box to ATP, leading to closure of KATP channels, cell membrane depolarization and activation of voltage-activated calcium channels (VACCs), followed by a rise in cytosolic Ca2<sup>+</sup> to a micromolar range and triggering of SNARE-dependent insulin release (Ashcroft and Rorsman, 1989). However, glucose may influence β-cells signaling through several additional routes. There may be alternative glucose entry routes, like for example active Na-glucose cotransport (Tomita, 1976; Trautmann and Wollheim, 1987), alternative calcium release sites, like ryanodine (Islam, 2002) and IP<sup>3</sup> receptors (Lang, 1999) or glucose may directly activate the sweet taste receptor and initiate signaling (Henquin, 2012), to name a few. Activation of a β-cell on a single cell level therefore

#### Edited by:

Shangbin Chen, Huazhong University of Science and Technology, China

#### Reviewed by:

Bryan C. Daniels, Arizona State University, United States Silvina Ponce Dawson, Universidad de Buenos Aires, Argentina

#### \*Correspondence:

Dean Korošak dean.korosak@um.si Marjan Slak Rupnik marjan.slakrupnik@muv.ac.at

#### Specialty section:

This article was submitted to Computational Physiology and Medicine, a section of the journal Frontiers in Physiology

Received: 19 October 2017 Accepted: 09 January 2018 Published: 24 January 2018

#### Citation:

Korošak D and Slak Rupnik M (2018) Collective Sensing of β-Cells Generates the Metabolic Code. Front. Physiol. 9:31. doi: 10.3389/fphys.2018.00031 likely involves triggering of a variety of elementary Ca2<sup>+</sup> events (Berridge et al., 2000), which interfere in space and time into a unitary β-cell Ca2<sup>+</sup> response to support Ca2+ dependent insulin release. This Ca2+-dependent insulin release can be further modulated by activation of different protein phosphorylation/dephosphorylation patterns (PKA, PKC, Cdk5, etc.) (Mandic et al., 2011; Skelin and Rupnik, 2011) or other protein modifications (Paulmann et al., 2009) to either reduce or increase the insulin output.

One of the important features of the sensory collectives is the optimization of the spatial relations between its elements to maximize the precision of sensing (Fancher and Mugler, 2017; Saakian, 2017). In islets of Langerhans, β-cells dwell as morphologically well defined cellulo-social collectives. These ovoid microorgans are typically not longer than 150µm. The relatively small and constant pancreatic islet size is an intriguing feature in vertebrate biology. The size distribution of islets is comparable in humans, rodents and wider within different vertebrate species, irrespective of evident differences in overall body and pancreas size as well as total β-cell mass (Kim et al., 2009; Dolenšek et al., 2015). In mice, islet sizes range between 50 and 600 µm, with a median values below 150 µm (Lamprianou et al., 2011). To accommodate differences in the body size, there is nearly a linear relationship between the total number of similarly sized islets and body mass across different vertebrate species (Montanya et al., 2000; Bouwens and Rooman, 2005). However, why are islets so conserved in size is unknown.

All β-cells within an islet collective represent a single functional unit, electrically and chemically coupled network, with gap junction proteins, Connexins 36 (Cx36) (Bavamian et al., 2007), for short-range interactions and with paracrine signaling (Caicedo, 2013) for long-range interactions between cells. The unitary cell response in one β-cell influences the formation of similar responses in neighboring β-cells and contributes to coordination of a large number of βcells (Cigliola et al., 2013; Stožer et al., 2013a). Explorations of these functional β-cell networks, constructed from thresholded pairwise correlations of Ca2<sup>+</sup> imaging signals (Stožer et al., 2013b; Markovicˇ et al., 2015; Johnston et al., 2016; Gosak et al., 2017a), showed that strongly correlated subsets of βcell collective organize into modular, broad-scale networks with preferentially local correlations reaching up to 40 µm (Markovicˇ et al., 2015), but understanding of mechanisms that lead to these strongly correlated networks states in β-cell populations is still lacking. We argue that β-cells sense, compute and respond to information as a collective, organized in a network similar to sensory neuron populations (Schneidman et al., 2006; Tkacik ˇ and Bialek, 2016), and not as a set of independent cells strongly coupled only when stimulation is high enough.

Here we reanalyze pairwise correlations of Ca2<sup>+</sup> spike trains (unitary β-cell responses on the shortest temporal scale) in βcell collective recorded in fresh pancreatic tissues slice under changing glucose stimulation conditions (6 mM subthreshold– 8 mM stimulatory) using methodological approaches previously described (Stožer et al., 2013b; Markovic et al., ˇ 2015; Gosak et al., 2017a,b). We specifically look at weak correlations between βcells which we found to be widely spread across the islet (Azhar and Bialek, 2010). Guided by the use of statistical physics models in describing populations of neurons (Schneidman et al., 2006; Tkacik et al., 2009), we use a simple spin glass model for Ca2<sup>+</sup> βcells activity and show that it well captures the features observed in the measured data. In a way, we recognize this efficiency of simple models in both neuronal and endocrine cell collectives as one manifestation of the "beauty in function" (Rasmussen, 1970).

#### 2. SPIN MODEL OF A β-CELL COLLECTIVE

Spin models have been borrowed from statistical physics to describe the functional behavior of large, highly interconnected systems like sensory neurons (Schneidman et al., 2006; Tkacik et al., 2009; Tkacik et al., 2014 ˇ ), immune system (Parisi, 1990), protein interactions (Bryngelson and Wolynes, 1987), financial markets (Bornholdt, 2001; Krawiecki et al., 2002), and social interactions between mammals (Daniels et al., 2016, 2017).

The model of the islet consist of N cells; at time t each of the cells can be in one of two states, spiking or silent, represented by a spin variable Si(t) = ±1, (i = 1, ..., N). The discrete time steps in model computations correspond to 2 s binning size of the Ca2<sup>+</sup> data. The effective field E<sup>i</sup> of the i-th cell has two contributions: one from the cell interacting with all other cells with interaction strength Jij, and one from external field h<sup>i</sup> . We assume that interactions extend over the whole system.

$$E\_i(t) = h\_i(t) + \sum\_{j=1}^{N} J\_{lj} \mathbf{S}\_j(t) \tag{1}$$

At the next moment (t + 1) each cell updates its state Si(t) with the probability p to Si(t + 1) = +1 and with the probability 1 − p to Si(t +1) = −1. The probability p depends on the effective field E<sup>i</sup> that the i-th cell senses:

$$p = \frac{1}{1 + \exp(-2E\_i)}.\tag{2}$$

The interaction strength Jij is a fluctuating quantity with contributions from amplitude J common to all links and from the pairwise contributions with amplitude I (Krawiecki et al., 2002): Jij = Jλ(t) + Iνij(t). Here are the fluctuations λ(t) and νij(t) random variables uniformly distributed in the interval [−1, 1]. The external field hi(t) = η(t) is also a random variable, uniformly distributed in the interval η(t) = [hmin, hmax]. In the mean-field approximation the average state of the system m(t) = 1 N P j Sj , evolves with time according to Krawiecki et al. (2002):

$$m(t+1) = \tanh(I\lambda(t)m(t) + h\_{mf}(t)),\tag{3}$$

where we hmf = η(t)/N. In the Results section below we demonstrate that the model describes the important features observed in the data well. In all computations we used a model with N = 200 spins, and we set, following the original model (Krawiecki et al., 2002), the pairwise interaction amplitude to I = 2J. The values of the remaining three free parameters of the model, J, hmax and hmin, were chosen to fit the model computations to the qualitative features of the Ca2<sup>+</sup> data as described in the next section.

## 3. RESULTS

The functional multicellular imaging (fMCI) records a full temporal activity trace for every cell in an optical plane of an islet from which meaningful quantitative statements about the dynamics of unitary Ca2<sup>+</sup> responses and information flow in the β-cell collective are possible (Dolenšek et al., 2013; Stožer et al., 2013a). Briefly, after the stimulation with increased glucose level, first asynchronous Ca2<sup>+</sup> transients appear, followed by a sustained plateau phase with oscillations on different temporal scales, from slow oscillations (100–200 s) to trains of fastest Ca2<sup>+</sup> spikes (1–2 s). As the relation between the rate of insulin release and cytosolic Ca2<sup>+</sup> activity shows saturation kinetics with high cooperativity (Skelin and Rupnik, 2011), the insulin release probability is significantly increased during these Ca2<sup>+</sup> spikes.

Initially, fMCI has been done at the glucose concentrations much higher than those at which β-cells usually operate. The main reason for this was to ensure comparability of the results with the mainstream research in the field using mostly biochemical approaches. At 16 mM glucose, a collective of β-cells responds in a fast, synchronized, and step-like manner. Therefore the first interpretation has been that gap junction coupling between neighboring β-cells presents a major driving force for the β-cell activation and inhibitory dynamics (Hraha et al., 2014; Markovic et al., ˇ 2015). Accordingly, the removal of Cx36 proteins does cause hyperinsulinemia at resting glucose levels and blunted responses to stimulatory glucose concentration (Speier et al., 2007). Since β-cells in fresh pancreatic tissue slices are sensitive to physiological concentration of glucose (6–9 mM) (Speier and Rupnik, 2003), we here focused on this less explored concentration range. We looked at the spiking part of the Ca2<sup>+</sup> imaging signals for which it has been previously shown to contain enough information to allow reconstruction of functional cell networks (Stetter et al., 2012).

For the present analysis we reused a dataset of individual Ca2+-dependent events from N = 188 ROIs with known positions from the central part of the fresh rodent pancreatic oval shaped islet (370 um in length and 200 um wide), representing β-cells, recorded with fMCI technique at 10 Hz over period of 40 min (for methodological details see Stožer et al., 2013b; Markovic et al., 2015; Gosak et al., 2017a,b ˇ ). During the recording the glucose concentration in the solution filling the recording chamber has been increased from 6–8 mM, reaching equilibration at around 200 s after the start of the experiment, and then decreased to initial concentration near the end of experiment at around 2,000 s (dashed red lines in **Figures 4**, **5** represent points where glucose levels were completely equilibrated in the recording chamber). We applied ensemble empirical mode decomposition (Luukko et al., 2016) on recorded traces to isolate the Ca2<sup>+</sup> spiking component of the signal. Finally, based on previous experiments in our laboratory, we binarized the signals using 2 s wide bins (**Figure 1**, left panel) and obtained binary spike trains Sj(t) ± 1, (j = 1...N), of βcells' Ca2<sup>+</sup> activity, each cell represented as a spin. As can be seen from the **Figure 1** the chosen bin width adequately describe the unitary events seen in the calcium traces. An example of spiking dynamics of 30 randomly chosen spins is shown as a raster plot in the right panel of **Figure 1**.

Statistical methods based on mostly pairwise correlations between neurons populations have been successfully used in predicting spiking patterns in cell populations (Schneidman et al., 2006; Tkacik et al., 2009; Tkacik et al., 2014; Ferrari ˇ et al., 2017). It may seem surprising that models with first and second-order correlation structure work not only when the cell activity is very sparse so the correlations could be described by perturbation theory (Roudi et al., 2009), but can reproduce the statistics of multiple co-spiking activity (Barton and Cocco, 2013; Merchan and Nemenman, 2016; Ferrari et al., 2017). We computed truncated correlations

$$
\mathcal{L}(i, j) = \langle \mathcal{S}\_i \mathcal{S}\_j \rangle - \langle \mathcal{S}\_i \rangle \langle \mathcal{S}\_j \rangle \tag{4}
$$

for all pairs of cells. The pairwise correlations found are mostly weak with the distribution shown in **Figure 2** (left panel), but they extend widely over the distances up to 170 µm across the islet, which is larger than an average vertebrate islet size (**Figure 2**, right panel). At distances larger than 170 µm the correlations decrease sharply toward zero. Such weak and long-ranging pairwise correlations could be the root of criticality and of strongly correlated network states in biological systems (Schneidman et al., 2006; Azhar and Bialek, 2010; Mora and Bialek, 2011; Tkacik et al., 2015 ˇ ).

To check for the existence of strongly correlated states in weakly correlated β-cell collective we computed probability distributions PN(K) of K simultaneously spiking cells in groups of N = 10, 20, 30 cells. Here, we used the entire dataset, the low and the high glucose concentration parts, from which we sampled cells signals. While the PN(K) of randomly reshuffled spike trains expectedly follows Poisson distribution (left panel in **Figure 3**, black crosses and dashed line for N = 10 spins), the observed co-spiking probabilities are orders of magnitude higher (diamonds in left panel of **Figure 3** for N = 10 spins) than corresponding probabilities in groups of independent spins. The statistics of these co-spiking events were described by an exponential distribution (Schneidman et al., 2006), by finding the effective potential (Tkacik et al., 2013, 2014 ˇ ) matching the observed PN(K) and adding it to the hamiltonian of the

model, or by using beta-binomial distribution (Nonnenmacher et al., 2017) PN(K) = C(N, K)B(α + K, β + N − K)/B(α, β) where C(N, K) is binomial coefficient and B(α, β) is the beta function.

We next run the spin model of 200 β-cells and then sampled the computed spike trains to obtain PN(K) from the model for N = 10, 20, 30. Despite its simple structure, the model matches order of magnitude of the observed PN(K) well when we set the interaction strength at J = 2.0, as shown in the left panel of **Figure 3** (red pluses and red dashed line), particularly for larger

FIGURE 2 | (Left) Distribution of pairwise correlations of β-cell collective computed from Ca2<sup>+</sup> imaging spiking signals. (Right) Pair correlations distribution over distance. Weak correlations extend over the whole system up to 170 µm. Black line shows the average values of correlations at particular cell-cell distances.

simultaneously. Randomly shuffled spike trains (black crosses, N = 10) with dashed line - Poisson distribution; N = 10 (diamonds), N = 20 (squares), N = 30 (open dots); model (red pluses + red dashed line with J = 2.0 used for the entire dataset, Nspins = 200 spins, hmin = −2.65, hmax = −1.65), beta-binomial model (Nonnenmacher et al., 2017) (black dashed line, α = 0.38, β = 11.0); (Right) Scaling of mean field return: open dots - data, red pluses - mean field approximation from the spin model of β-cells computed with <sup>J</sup> <sup>=</sup> 2.0, <sup>h</sup>mf <sup>=</sup> <sup>η</sup>(t)/N. Dashed line <sup>P</sup>(G) <sup>∼</sup> <sup>G</sup>−2.0

K values. In the model here we did not treat the low and the high glucose concentration part separately, we used J = 2.0 for the entire dataset. For comparison, we also show how the betabinomial model fits to the observed data using the parameters α = 0.38, β = 11.0 in all N = 10, 20, 30 cases. These values are also close to the best-fitting parameters (α = 0.38, β = 12.35) to the simulated and observer correlated neural population activity data as reported in Nonnenmacher et al. (2017).

The microscopic model of interacting spins with interactions randomly varying in time (Krawiecki et al., 2002), adopted here to describe interacting β-cell collective, exhibits scaling of price fluctuations (Bornholdt, 2001) observed in financial markets (Gopikrishnan et al., 1999) and on-off intermittency with attractor bubbling dynamics of average price (Krawiecki et al., 2002). Following this idea, we looked at the logarithmic return of average state of β-cell collective at time t (Bornholdt, 2001): G(t) = log(m(t))−log(m(t−1)). As presented in the right panel of **Figure 3**, the distribution P(G) (of positive G values) can indeed be approximated with a scaling law: P(G) ∼ G <sup>−</sup><sup>γ</sup> with γ = 2.0. There is an analytical relationship (Krawiecki et al., 2002) between J and exponent γ of the distribution of amplitudes of the return of the mean field, J = γ <sup>1</sup>/(<sup>γ</sup> <sup>−</sup>1), which gives J = 2.0 for γ = 2.0. We used this as a consistency check between the model computations and mean field approximation. Computing the average state with the Equation (3) of the model, we can reproduce the observed distribution by setting on the interaction strength to J = 2.0 at ton = 400 s and off to J = 0 at toff = 2,200 s. The amplitude of the interaction J is consistent with the computation of the co-spiking probability.

In **Figure 4** we show the plots of both, observed and computed, returns of average state of interacting β-cells for comparison. The glucose concentration was changed during the experiment in a stepwise manner: from 6 to 8 mM at the beginning and back to 6 mM near the end of recording period. The effect of both changes is nicely visible in the

FIGURE 4 | (Upper) Observed logarithmic return of the average state of of β-cell collective G(t), (Lower) logarithmic return of the average state computed from the model with J = 2.0 for ton < t < toff , denoted with vertical dashed lines in figures. Dashed red lines represent points where glucose levels were completely equilibrated in the recording chamber.

G(t) plot (upper panel, **Figure 4**) where the on-off intermittent dynamics of the average state starts around ton = 400 s and lasts until around toff = 2, 200 s in the experiment. Both observed events are delayed with respect to the times of glucose concentration change due to the asynchronous Ca2<sup>+</sup> transients (Stožer et al., 2013a). We expect that the response of β-cell collective to the stimulus increase must be visible in the variance of average state Var(m) which is in Ising-like model we are using here equal to susceptibility of the system χ = Var(m) = hm<sup>2</sup> i − hmi 2 . We used the low glucose concentration part (6 mM) of the data to estimate the boundaries of the external field interval [hmax, hmin] to describe the first part of susceptibility. Using the maximal and minimal spiking rates of cells (mmax, mmin) in 6 mM glucose from the data and the mean-field approximation with J = 0 corresponding to the non-stimulatory glucose regime we have [hmax, hmin] = [tanh−<sup>1</sup> (mmax), tanh−<sup>1</sup> (mmin)] = [−1.65, −2.65]. In upper panel of **Figure 5** (open black dots) we show the plot of susceptibility as a function of recording time, focusing around the transition to increased glucose concentration during the experiment. There is a sharp increase of susceptibility at around ton, the same time the on-off intermittency starts to appear in G(t). Using mean field approximation of the spin model Equation (3) for computation of susceptibility (averaged over many runs) and setting J = 0 for t < ton and J = 2.0 for t > ton we can well describe the observed evolution of susceptibility and capture the rapid onset of increased sensibility of the islet (red line in upper part of **Figure 5**).

FIGURE 5 | (Upper) Susceptibility of β-cell collective around transition to stimulatory glucose level. Open dots are the experimental data, red line shows the result of the mean field computations with J = 2.0 onset at t = ton(blackdashedline). Dashed red line represent the point where glucose level completely equilibrated during the 6–8 mM transition in the recording chamber. (Lower) Normalized conditional entropy. Open dots are experimental data at 8 mM glc, open squares at 6 mM glc. Red pluses show the results of the spin model computations with Nspins = 200 spins, and the parameters: hmin = −2.65, hmax = −1.65, J = 2.0 for the upper, and J = 0 for lower the lower part.

Pairwise correlation structure enables error-correction features of population coding in neural systems (Schneidman et al., 2006). To check for memory-like or error-correcting properties in islets, we use the conditional entropy H(S<sup>i</sup> |S), the measure for the information we need to determine the state of N-th cell (i.e., spiking or not) if we know the states of N − 1 cells (S = S<sup>j</sup> 6= i) in a group of N cells. If the state of the N-th cell is completely determined by other N − 1 cells, the conditional entropy is zero H(S<sup>i</sup> |S) = 0 and the error correction is perfect. When S<sup>j</sup> are independent random states, the conditional entropy equals the entropy of the N-th cell H(Si).

We computed the quantity 1 − H(S<sup>i</sup> |S)/H(Si) (normalized mutual information) as a function of number of cells (for small groups of cells) and extrapolate the trend toward the limit H(S<sup>i</sup> |S) = 0 that determines the critical number of cells, N<sup>c</sup> , needed to predict the state of another cell. As seen in the lower panel of **Figure 5**, the predictability is a glucose-dependent parameter. With non-stimulatory glucose concentration, the complete set of data is required for predictions, whereas at 8 mM glucose we find that order of magnitude smaller number of measured cells are needed to predict the states of other cells.

#### 4. DISCUSSION

Pancreatic β-cell continuously intercepts a variety of energyrich or signaling ligands using the whole spectrum of specific receptors on the cell membrane, as well as in metabolic and signaling pathways within the cell. The cell converts these signals into a binary cellular code, for example a train of Ca2<sup>+</sup> spikes, which drive insulin release that fits current physiological needs of the body. This allow already a single cell to sense its chemical environment with extraordinary, often diffusion limited precision (Bialek and Setayeshgar, 2005), however, judging by their heterogeneous secretory behavior in cell culture, the precision of sensing among the individual β-cells is quite diverse (Hiriart and Ramirez-Medeles, 1991). Recent experimental evidence and modeling have shown that cell collectives sense better compared to an individual cell. The precise mechanism of this collective sensing improvement depends on cell-cell communication type, which can be short-range with direct cell contacts or longrange with paracrine signaling (Fancher and Mugler, 2017; Saakian, 2017). Furthermore, also long-range interaction have its finite reach which can poise a limit to the cell collective size and therefore determines its optimal as well as maximal size. As mentioned in the Introduction here, it is intriguing how well conserved the pancreatic islet size is in vertebrates of dramatically different body dimensions (Montanya et al., 2000). In a single vertebrate organism the size of the islets can be bigger that 150 um, but functional studies revealed that the islets bigger than 200 um secrete 50% less insulin after glucose stimulation (Fujita et al., 2011). These functional differences between small and large islets have been partially attributed to diffusion barriers for oxygenation and nutrition, limiting the survival of core β-cells in bigger islets after isolation. However, reducing these diffusion barriers had no influence on insulin secretory capacity (Williams et al., 2010) suggesting that other factors, like diffusion of paracrine signaling molecules (Caicedo, 2013), could limit the collective β-cell function in bigger islets. This dominance of a long-range information flow, likely limited to some physical constraints, has indicated the use of the mathematical equivalency with spin glass-like systems (Tkacik ˇ and Bialek, 2016).

We strongly believe that advanced complex network analysis based on strong short-range correlations can continue to provide valuable information regarding the β-cell network topologies, network on network interactions and describe the functional heterogeneity of individual β-cells (Gosak et al., 2015, 2017a; Markovic et al., ˇ 2015; Johnston et al., 2016). However, the main goal of the present study was to determine the influence of weak long-range correlations between pairs of β-cells on the probability of activation of single β-cells. Recently has been shown that it suffice to use pairwise correlations to quantitatively describe the collective behavior of cell collectives (Merchan and Nemenman, 2016). The typically small values of pairwise correlation coefficients with the median values below 0.02, would intuitively be ignored and β-cells described as if they act independently, however in larger populations of cells this assumption completely fails (Schneidman et al., 2006). In fact, at physiological stimulatory glucose levels between 6 and 9 mM, β-cell collectives are entirely dominated by weak average pairwise correlations (**Figure 2**). Nevertheless, this is the glucose concentration range, where β-cells are most responsive to the nutrient to, as a collective, compute their activity state and pulsatile insulin release, and to meet the organismal needs between the environmental and behavioral extremes of food shortage and excess (Schmitz et al., 2008)?

Based on the range of the calculated weak pairwise correlations of up to 170 um (**Figure 2**), we predict that βcells collective falls into a category of sparse packed tissues with dominant paracrine interactions and that cell-cell distances contribute to optimal sensing and functional response in creating the metabolic code governing the release of insulin. It remains unclear whether and how the position of β-cells within an islet is controllable. As many other cells, β-cells are polarized and possess a primary cilium (Gan et al., 2017), which should have a primary role in sensory function, i.e., insulin sensing in paracrine signaling (Doganer et al., 2016 ˘ ), and not in cell motility. It is quite interesting though, that the ciliopathies are highly associated with reduced β-cell function and increased susceptibility to diabetes mellitus (Gerdes et al., 2014). Future experiments are required to test for the possible motility of β-cells within the islet to adopt an optimal separation of key sensitive β-cells. To further extrapolate the collective sensing idea, it is also possible that the diffuse arrangement of a collective of islets within different parts of pancreas, which are exposed to different vascular inputs (Dolenšek et al., 2015), serves to optimize nutrient sensing experience, yet on a higher organizational level, providing a topological information regarding the nutrient levels in different parts of the gastrointestinal tract. The nature and level of interactions between individual islets in the pancreas are currently also unknown.

As in retinal neuron networks, β-cells encode information about the presence of energy-rich nutrients into sequences of intermittent Ca2<sup>+</sup> spikes. In a natural setting of sensory neural networks with stimuli derived from a space with very high dimensionality the coding seems challenging and interpretations require some strong assumptions (Tkacˇik et al., 2014). We currently do not understand the input dimensionality of a typical ligand mixture around the β-cells, we simply assume it is not high. As in retinal networks (Schneidman et al., 2006; Tkacik ˇ et al., 2014), the predictability regarding the functional state of individual β-cells is defined by the network and not the chemical environment. This suggests that the sensory information at physiological glucose levels is substantially redundant. It is likely that the nutrient mixture presents a noisy challenge for the information transfer which is typical for biological system. But why study the insulin release pattern or the metabolic code? The β-cell network possess associative or error-correcting properties (**Figure 5**), so this idea from the sensory neuron networks can be generalized also to populations of endocrine cells (Schneidman et al., 2006), which may again influence the optimal islet size and suggest the presence of functional subunits within the islet that could adapt, for example, to changing environment in a dynamic fashion. Furthermore, error-correction properties are glucose dependent and can be physiologically modulated (**Figure 5**). The trains of Ca2<sup>+</sup> spikes at constant glucose stimulation (8 mM) are inhomogeneous, display on-off intermittency (**Figure 4**) and scaling of log returns of average state (**Figure 3**) analog to models of financial time series (Krawiecki et al., 2002). For the spin glass approach we also postulate that the sources of stochasticity in an islet collective are various. On one hand, the β-cells make decisions on activation under the influence of the external environment and other β-cells. Second, also the time-dependent interaction strength among β-cells is random, which could reflect their socio-cellular communication network and indicate that the external environment can be sensed differently between different β-cells in an islet (Gosak et al., 2017b).

Biological systems seem to poise themselves at criticality, with a major advantage of enhanced reactivity to external perturbations (Mora and Bialek, 2011). Often a limited number of individual functional entities, cell or groups of cells as found in pancreatic islets, appeared to be limiting to address criticality. However, it has been recently demonstrated that even in biological systems with small number of interacting entities one can operationally define criticality and observe changes in robustness and sensitivity of adaptive collective behavior (Daniels et al., 2017). Our results suggest that β-cells collective within the islet sits near its critical point and we could determine the susceptibility in the islet. Stimulatory glucose concentration (8 mM) has been decreasing distance to criticality by increasing sensitivity (**Figure 5**). Smaller distance to criticality at unphysiologically high glucose levels has its possible adverse consequences in a phenomenon called critical slowing down as the system takes more and more time to relax as it comes nearer to the critical point (Mora and Bialek, 2011). Our preliminary results show that at very strong stimulation (i.e., glucose levels above 12 mM) the whole system freezes into a certain state where short-term interaction take over enabling global phenomena within the islets, e.g., Ca2<sup>+</sup> waves (Stožer et al., 2013b) requiring progressively longer periods to relax to baseline with increasing glucose concentrations.

Further work will be needed to exploit at what circumstances deviations in islet size can contribute to islet malfunction and pathogeneses of different forms of diabetes mellitus. Until recently it has been thought that insulin release is no longer functional in type 1 diabetes mellitus. We now know that even in type 1 diabetic patients small and functional collectives of βcells persist in the pancreata of these patients even decades after the diagnosis (Faustman, 2014). On the other hand, the β-cells mass in an islet can be increased in type 2 diabetic patients in the initial phases after the diagnosis (Rahier et al., 2008) or in animal models (Daraio et al., 2017) and can only be reduced in the later phases (Rahier et al., 2008). The detailed relations

#### REFERENCES


between the reduced or increased insulin release, changed islet size and therefore changed circumstances for paracrine signaling in disturbed collective nutrient sensing and during the aforementioned pathogeneses of diabetes mellitus remain to be established.

#### AUTHOR CONTRIBUTIONS

All authors listed, have made substantial, direct and intellectual contribution to the work, and approved it for publication.

#### ACKNOWLEDGMENTS

The authors acknowledge the financial support from the Slovenian Research Agency (research core funding, No. P3- 0396), as well as research project, No. N3-0048).


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Korošak and Slak Rupnik. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Multisite Delayed Feedback for Electrical Brain Stimulation

Oleksandr V. Popovych<sup>1</sup> \* and Peter A. Tass <sup>2</sup>

1 Institute of Neuroscience and Medicine, Brain and Behaviour (INM-7), Research Centre Jülich, Jülich, Germany, <sup>2</sup> Department of Neurosurgery, Stanford University, Stanford, CA, United States

Demand-controlled deep brain stimulation (DBS) appears to be a promising approach for the treatment of Parkinson's disease (PD) as revealed by computational, pre-clinical and clinical studies. Stimulation delivery is adapted to brain activity, for example, to the amount of neuronal activity considered to be abnormal. Such a closed-loop stimulation setup might help to reduce the amount of stimulation current, thereby maintaining therapeutic efficacy. In the context of the development of stimulation techniques that aim to restore desynchronized neuronal activity on a long-term basis, specific closed-loop stimulation protocols were designed computationally. These feedback techniques, e.g., pulsatile linear delayed feedback (LDF) or pulsatile nonlinear delayed feedback (NDF), were computationally developed to counteract abnormal neuronal synchronization characteristic for PD and other neurological disorders. By design, these techniques are intrinsically demand-controlled methods, where the amplitude of the stimulation signal is reduced when the desired desynchronized regime is reached. We here introduce a novel demand-controlled stimulation method, pulsatile multisite linear delayed feedback (MLDF), by employing MLDF to modulate the pulse amplitude of high-frequency (HF) DBS, in this way aiming at a specific, MLDF-related desynchronizing impact, while maintaining safety requirements with the charge-balanced HF DBS. Previously, MLDF was computationally developed for the control of spatio-temporal synchronized patterns and cluster states in neuronal populations. Here, in a physiologically motivated model network comprising neurons from subthalamic nucleus (STN) and external globus pallidus (GPe), we compare pulsatile MLDF to pulsatile LDF for the case where the smooth feedback signals are used to modulate the amplitude of charge-balanced HF DBS and suggest a modification of pulsatile MLDF which enables a pronounced desynchronizing impact. Our results may contribute to further clinical development of closed-loop DBS techniques.

Keywords: neuronal synchronization, delayed feedback, deep brain stimulation, desynchronization, electrical pulse stimulation, closed-loop stimulation

### 1. INTRODUCTION

High-frequency (HF) deep brain stimulation (DBS) is the standard therapy for medically refractory Parkinson's disease (PD), where electrical pulse trains are permanently delivered via depth electrodes at high frequencies (> 100 Hz) (Benabid et al., 1991, 2009; Kuncel and Grill, 2004; Johnson et al., 2008). The mechanism of action of HF DBS is still debated (Johnson et al., 2008; Gradinaru et al., 2009; Deniau et al., 2010). HF DBS may cause side effects by stimulation of the target as well as surrounding structures (Ferraye et al., 2008; Moreau et al., 2008; van Nuenen et al., 2008; Xie et al., 2012). It is, hence, desirable to reduce the integral current delivered.

#### Edited by:

Alexey Zaikin, University College London, United Kingdom

#### Reviewed by:

Anuj Agarwal, Signal Solutions LLC, United States Mikhail Ivanchenko, N. I. Lobachevsky State University of Nizhny Novgorod, Russia

> \*Correspondence: Oleksandr V. Popovych o.popovych@fz-juelich.de

#### Specialty section:

This article was submitted to Computational Physiology and Medicine, a section of the journal Frontiers in Physiology

Received: 19 October 2017 Accepted: 15 January 2018 Published: 01 February 2018

#### Citation:

Popovych OV and Tass PA (2018) Multisite Delayed Feedback for Electrical Brain Stimulation. Front. Physiol. 9:46. doi: 10.3389/fphys.2018.00046

**37**

Accordingly, different types of closed-loop, demand-controlled and adaptive DBS (aDBS) have been developed in computational and engineering studies (Tass, 2001, 2003; Rosenblum and Pikovsky, 2004a,b; Hauptmann et al., 2005a,b; Popovych et al., 2005, 2006; Kiss et al., 2007; Pyragas et al., 2007; Tukhlina et al., 2007; Luo et al., 2009; Popovych and Tass, 2010; Montaseri et al., 2013). Closed-loop aDBS approach received recent development in pre-clinical and clinical studies (Graupe et al., 2010; Rosin et al., 2011; Carron et al., 2013; Little et al., 2013; Priori et al., 2013; Yamamoto et al., 2013; Grahn et al., 2014; Hosain et al., 2014; Rosa et al., 2015).

Closed-loop aDBS was successfully tested in parkinsonian monkeys under acute conditions (Rosin et al., 2011). In the considered setup the globus pallidus internal (GPi) was stimulated by a short pulse train delayed by 80 ms following an action potential recorded from the primary motor cortex. Under such conditions aDBS was shown to be more effective in reducing pallidal discharge rate and pathological oscillatory neuronal activity as well as in alleviation of akinesia than the conventional continuous HF DBS (cDBS). In a proof of principle study in PD patients (Little et al., 2013), aDBS was switched on and off depending on whether the amplitude of the subthalamic nucleus (STN) local field potential (LFP) in the beta band increased above or decreased below a certain threshold. During aDBS the clinical motor scores strongly improved by about 30% better than during cDBS, whereas aDBS was switched on during 44% of the time (reduced by 56%) as compared to 100% of cDBS (Little et al., 2013). Demand-controlled aDBS was applied for suppression of essential tremor(Graupe et al., 2010). The onset of the tremor was predicted from the measured electromyographic (EMG) signal, which was used to initiate aDBS stimulation epochs. In patients with intention tremor aDBS was switched on and off based on the threshold crossing by EMG power (Yamamoto et al., 2013), where the accurately triggered switching of aDBS resulted in a complete control of the tremor.

Instead of the on-off strategy of the papers (Graupe et al., 2010; Rosin et al., 2011; Little et al., 2013; Yamamoto et al., 2013) mentioned above, the stimulation intensity can also be adapted in real time to the amplitude of the synchronized neuronal activity. Such an approach was used in a clinical study (Rosa et al., 2015), where the voltage of the stimulation was adapted to the beta-band power of the LFP each second (Rosa et al., 2015). The latter approach resembles closed-loop methods that have been developed in the past for the specifically desynchronizing control of abnormal neuronal synchronization that is characteristic for several neurological disorders including PD (Nini et al., 1995; Hammond et al., 2007), essential tremor (Schnitzler et al., 2009), epilepsy (Wong et al., 1986), and tinnitus (Llinas et al., 1999; Weisz et al., 2005; Eggermont and Tass, 2015). These techniques are feedback approaches utilizing the mean field of the synchronized population, which is measured and processed (e.g., filtered, delayed, amplified, etc.), and then fed back as stimulation signal to desynchronize neuronal populations (Rosenblum and Pikovsky, 2004a,b; Hauptmann et al., 2005a,b; Popovych et al., 2005, 2006; Kiss et al., 2007; Pyragas et al., 2007; Tukhlina et al., 2007; Luo et al., 2009; Popovych and Tass, 2010; Montaseri et al., 2013). Direct electrical stimulation of the neuronal tissue with smooth and slowly oscillating feedback signal may however cause an irreversible charge deposit in the neuronal tissue that can exceed safety limits (Harnack et al., 2004; Kuncel and Grill, 2004; Merrill et al., 2005). Two desynchronizing delayed feedback methods, linear delayed feedback (LDF) and nonlinear delayed feedback (NDF) were recently adapted and computationally tested for electrical closed-loop DBS (Popovych et al., 2017a,b). In both cases, the amplitude of charge-balanced short pulses composing the stimulation signal of the standard HF DBS was modulated by the slow feedback signal. The feedback method with such a pulsatile stimulation signal is referred to as a pulsatile feedback stimulation that can be used for electrical DBS.

In principle, abnormal neuronal synchronization can be counteracted in different ways. For instance, LDF (Rosenblum and Pikovsky, 2004a,b) and NDF (Popovych et al., 2005, 2006; Popovych and Tass, 2010) aim at restoring incoherent neuronal activity. In contrast, in this study we consider a multisite linear delayed feedback (MLDF) which has been designed for the control of excessive neuronal synchronization (Hauptmann et al., 2005a,b, 2007a,b; Popovych et al., 2006; Omel'chenko et al., 2008). In previous modeling studies it was observed that MLDF stimulation can counteract the synchronized dynamics by inducing clustering states, which may lead to a variety of spatio-temporal patterns of neuronal activity. Such patterns of the neuronal activity are important, for example, in the context of central pattern generators (CPG) (Marder and Calabrese, 1996; Yuste et al., 2005), where synchronized neuronal discharges have to be well-coordinated both in space and time. The MLDF stimulation approach was suggested for inducing and control of such a patterned activity, for example, in the case when physiological CPG dynamics needs to be restored (Hauptmann et al., 2007a,b; Omel'chenko et al., 2008).

We introduce a pulsatile MLDF for electrical brain stimulation and test it on a physiologically motivated model of interacting neuronal populations of STN and external globus pallidus (GPe) neurons (Terman et al., 2002; Rubin and Terman, 2004). We show that for four-site stimulation setup of MLDF with smooth stimulation signals, a weak clustering, mostly twocluster states can be observed in a limited parameter range. For pulsatile MLDF stimulation the stimulation-induced clustering becomes even less pronounced such that the main impact of the pulsatile MLDF stimulation consists in a desynchronization, i.e., a suppression of in-phase synchronization in the stimulated population. The pulsatile MLDF stimulation is however less effective in inducing desynchronization than the pulsatile LDF. For effective desynchronization, we here introduce differential pulsatile MLDF stimulation and show this stimulation method can effectively and robustly desynchronize the model STN neurons.

# 2. METHODS

### 2.1. Model

We consider a model network of STN and GPe neuronal populations suggested by Terman et al. (2002), where the dynamics of individual neurons is described by the following system:

$$C\_m \nu' = \underset{\text{L}}{-I\_{\text{L}}} -I\_{\text{K}} -I\_{\text{Na}} -I\_{\text{T}} -I\_{\text{Ca}} -I\_{\text{AHP}}$$

$$-I\_{\rm syn} + I\_{\rm app} + I\_{\rm stim},\tag{1}$$

$$\text{[Ca]}' = \varepsilon \left( -I\_{\text{Ca}} - I\_{\text{T}} - k\_{\text{Ca}} \text{ [Ca]} \right), \tag{2}$$

$$X' = \phi\_X \left( X\_\infty(\nu) - X \right) / \pi\_X(\nu). \tag{3}$$

In Equations (1–3), v is a membrane potential of the neuron, the currents IL, IK, INa, IT, ICa, IAHP, Isyn, and Iapp are the corresponding leak, potassium, sodium, low threshold calcium, high threshold calcium, afterhyperpolarization potassium, synaptic, and external current, respectively. [Ca] is the intracellular concentration of Ca2<sup>+</sup> ions, and X = n, h, r are the gating variables.

The following currents are given by the same expressions for STN and GPe neurons:

$$\begin{array}{lll} I\_{\mathrm{L}} = \mathrm{g\_{\mathrm{L}}}(\nu - \nu\_{\mathrm{L}}), & I\_{\mathrm{K}} = \mathrm{g\_{\mathrm{K}}}n^{4}(\nu - \nu\_{\mathrm{K}}), \\\ I\_{\mathrm{Na}} = \mathrm{g\_{\mathrm{Na}}}m\_{\infty}^{3}(\nu)h(\nu - \nu\_{\mathrm{Na}}), & I\_{\mathrm{Ca}} = \mathrm{g\_{\mathrm{Ca}}}s\_{\infty}^{2}(\nu)(\nu - \nu\_{\mathrm{Ca}}), \\\ I\_{\mathrm{A}\mathrm{HP}} = \mathrm{g\_{\mathrm{A}}\mathrm{HP}}(\nu - \nu\_{\mathrm{K}})(\mathrm{[Ca]} // \mathrm{[Ca]} + \mathrm{k\_{1}})). \end{array}$$

On the other hand, current I<sup>T</sup> is different for the excitatory STN neurons for the inhibitory GPe neurons:

$$\text{STN: } I\_{\Gamma} = \text{g}\_{\Gamma} a\_{\infty}^{\text{3}}(\nu) b\_{\infty}^{2}(r) (\nu - \nu\_{\text{Ca}}), \text{ GPe: } I\_{\Gamma} = \text{g}\_{\Gamma} a\_{\infty}^{\text{3}}(\nu) r (\nu - \nu\_{\text{Ca}}),$$

where b∞(r) = 1/(1 + exp[(r − θ<sup>b</sup> )/σ<sup>b</sup> ]) − 1/(1 + exp[−θb/σ<sup>b</sup> ]). The functions X∞(v) and τX(v) used in the above equations read

$$\begin{array}{ll} X\_{\infty}(\nu) &= 1/\left(1 + \exp[- (\nu - \theta\_X)/\sigma\_X]\right), & X = n, \, h, \, r, \, m, \, s, \, a;\\ \tau\_X(\nu) &= \tau\_X^0 + \tau\_X^1/\left(1 + \exp[- (\nu - \theta\_X^\tau)/\sigma\_X^\tau]\right), & X = n, \, h, \, r. \end{array}$$

For GPe neurons τr(v) = τ<sup>r</sup> is a constant parameter.

The STN and GPe neuronal populations contain N = 200 neurons each and arranged on 1Dim lattices with periodic boundary conditions. Each GPe neuron receives an exitatory input from a single neighboring STN neuron and inhibits three neighboring STN neurons. The considered model was introduced to study pathological neuronal dynamics in PD and was investigated in several papers (Terman et al., 2002; Rubin and Terman, 2004; Park et al., 2011; Popovych et al., 2017a,b). The neurons are interacting via chemical synapses, where the synaptic currents are

$$\begin{aligned} \text{STN}: I\_{\text{syn}} &= \text{g}\_{\text{G}\to\text{S}} (\nu - \nu\_{\text{G}\to\text{S}}) \sum s\_{\text{j}}, \\ \text{GPe}: I\_{\text{syn}} &= \text{g}\_{\text{S}\to\text{G}} (\nu - \nu\_{\text{S}\to\text{G}}) \sum s\_{\text{j}}. \end{aligned}$$

Summation is taken over all corresponding presynaptic neurons, where j is the index of neurons. The coupling strength between neurons is defined by parameters of synaptic weights gS→<sup>G</sup> = 0.4 nS/µm<sup>2</sup> (from STN to GPe) and gG→<sup>S</sup> (from GPe to STN, will be specified below). The reversal potential for the excitatory coupling from STN to GPe vS→<sup>G</sup> = 0 mV, and vG→<sup>S</sup> = −100 mV for the inhibitory coupling from GPe to STN. Synaptic variables s<sup>j</sup> are governed by

$$s\_{\circ}^{\prime} = \alpha H\_{\infty} (\nu\_{\circ} - \theta\_{\mathcal{G}}) (1 - s\_{\circ}) - \beta s\_{\circ},$$

$$H\_{\infty}(\mathbf{x}) = 1/(1 + \exp\left[- (\mathbf{x} - \theta\_{\mathcal{G}}^{H}) / \sigma\_{\mathcal{G}}^{H}\right]).\tag{4}$$

The neurons are nonidentical such that the applied currents Iapp = Iapp, <sup>j</sup> for STN neurons are Gaussian distributed around the mean 10 pA/µm<sup>2</sup> and with the standard deviation 0.015 pA/µm<sup>2</sup> . For GPe neurons the parameter ε = ε<sup>j</sup> are also Gaussian distributed around the mean 0.0055 ms−<sup>1</sup> and with the standard deviation 2 · 10−<sup>5</sup> ms−<sup>1</sup> . The other parameters for the STN and GPe neurons and their values are listed in Table S1.

#### 2.2. Synchronized Dynamics of STN Neurons

In this study we investigate how the synchronized dynamics of the STN-GPe network can be controlled by an external stimulation. We estimate the extent of synchronization by the order parameters (Haken, 1983; Kuramoto, 1984; Tass, 1999).

$$R\_k(t) = \left| N^{-1} \sum\_{j=1}^{N} \exp(ik\psi\_j(t)) \right|, \ k = 1, 2, \dots, \tag{5}$$

where ψj(t) is the phase of neuron j, which attains the values ψj(tn) = 2πn, n = 0, 1, ... at the burst onset time moments t<sup>n</sup> of the jth neuron. The phase linearly increases between two consecutive bursts ψj(t) = 2π(t − tn)/(tn+<sup>1</sup> − tn) + 2πn for t ∈ (tn, tn+1), n = 0, 1, ... (Pikovsky et al., 2001). The order parameters R<sup>k</sup> (t) range from 0 to 1, where the values of the first order parameter R = R<sup>1</sup> correspond to the extent of in-phase synchronization in the population. Large values of the k-th order parameter R<sup>k</sup> together with small values of the order parameters R<sup>n</sup> of smaller degree n < k are characteristic for a k-cluster state, where the oscillators are in-phase synchronized within the clusters, but the clusters are time (and phase) shifted with respect to each other equidistantly in the oscillation period.

Examples of the collective dynamics of STN neurons without stimulation (Istim = 0 in Equation 1) are illustrated in **Figure 1**. Depending on the coupling strength as given by the values of parameter gG→S, STN neurons can exhibit synchronization of different extents and forms. For weak coupling, e.g., gG→<sup>S</sup> = 1.28 nS/µm<sup>2</sup> , the neurons are weakly and intermittently synchronized, and the order parameter fluctuates around small values hRi ≈ 0.2, see **Figure 1A** (black curve). STN neurons exhibit desynchronized bursting dynamics (**Figure 1B**, black dots), where the individual bursting frequencies (number of bursts per second) of STN neurons are relatively broadly distributed in the range 9.91 ± 0.017 Hz (mean ± standard deviation), see Figure S1. The firing patters exhibit strong variation as time evolves (Figure S1). The inter-bursts intervals (IBI, time intervals between the first spikes of two consecutive bursts) vary irregularly from one burst to the next, which is illustrated in **Figure 1E** (black circles), where the next IBIn+<sup>1</sup> are plotted vs. the previous IBI<sup>n</sup> as a scatter plot of the first return map.

An increase of the coupling leads to synchronized dynamics of bursting STN neurons as illustrated by the spike raster plot in **Figure 1C** (black dots) for gG→<sup>S</sup> = 1.38 nS/µm<sup>2</sup> . The order parameter fluctuates around a larger value hRi ≈ 0.8 (**Figure 1A**, blue curve). The individual bursting frequencies become much narrowly distributed in the range 9.75 ± 0.003 Hz

indicating that most neurons become frequency synchronized, see Figure S1. Although the firing patters still vary in time, they clearly demonstrate an in-phase synchronized dynamics of STN neurons (**Figure 1C** and Figure S1). The behavior of the IBIs remains however irregular (**Figure 1E**, blue diamonds).

Only stronger coupling can regularize the dynamics of IBIs. For example, for gG→<sup>S</sup> = 1.7 nS/µm<sup>2</sup> the IBIs can attain only two values as illustrated in **Figure 1E** (red triangles) such that IBIs periodically alternate between them in a period-2 manner. STN neurons synchronize at the same frequency in the very narrow range 9.06 ± 2 · 10−<sup>6</sup> Hz, and the firing pattern demonstrates periodic dynamics, see Figure S1. In such a way the STN neurons become periodically synchronized for strong coupling (**Figure 1D**, black dots), and the order parameter is nearly constant with hRi ≈ 0.7 (**Figure 1A**, red curve).

The extent of synchronization is also reflected by the amplitude dynamics of the local field potential (LFP). The latter can be modeled as an ensemble-averaged synaptic activity of the neuronal population LFP(t) = N <sup>−</sup><sup>1</sup> P<sup>N</sup> j=1 s<sup>j</sup> (Buzsaki, 2004), where sj(t) are the synaptic variables (Equation 4). For a more sophisticated approach see the papers (Lindén et al., 2011; Parasuram et al., 2016) . The measured raw LFP(t) can be on-line filtered by means of a linear damped oscillator

$$
\ddot{u} + \alpha\_d \dot{u} + \omega^2 u = k\_\text{f} LFP(t), \tag{6}
$$

where ω approximates the mean frequency of the LFP oscillations, ω = 2π/T, and T is the LFP mean period. Variable x(t) = ˙u of Equation (6) has a zero phase shift with respect to the input raw LFP signal (Tukhlina et al., 2007), and we consider it as the filtered LFP. The other parameters of Equation (6) were chosen α<sup>d</sup> = k<sup>f</sup> = 0.008, which approximately preserves the amplitude of the input LFP signal (Popovych et al., 2017a,b).

The dynamics of raw and filtered LFP is illustrated in the raster plots in **Figures 1B–D** (blue and red curves) for the above three considered values of the coupling parameter, where large- and small-amplitude oscillations of LFP are in correspondence with strong and weak neuronal synchronization, respectively.

#### 2.3. Delayed Feedback Stimulation

We stimulate the considered STN-GPe model neuronal network by multisite linear delayed feedback (MLDF), where the stimulation is administered to the STN neurons only. This stimulation techniques has been suggested and investigated in the papers (Hauptmann et al., 2005a,b, 2007a,b; Popovych et al., 2006; Omel'chenko et al., 2008). We assume that, for example, 4 stimulation sites are implanted in the STN population of N = 200 neurons at the equidistant lattice coordinates (index of neurons) j = 25, 75, 125, and 175 as schematically illustrated in **Figure 2A** (upper plot). The feedback stimulation signal Si(t) administered via the i-th stimulation site is calculated as (Hauptmann et al., 2005a,b, 2007a,b; Popovych et al., 2006; Omel'chenko et al., 2008)

$$S\_i(t) = K \cdot x(t - \tau\_i), \quad \tau\_i = \frac{11 - 2(i - 1)}{8}\tau, \quad i = 1, 2, 3, 4. \tag{7}$$

The delays τ<sup>i</sup> are considered in such a form in order to achieve a time shift by T/4 between the feedback signals for τ = T. Indeed, if the measured signal x(t) is periodically oscillating with period T, the corresponding oscillating feedback signals (Equation 7) with neighboring indices will be time shifted with respect to each other by T/4 (e.g., τ<sup>1</sup> − τ<sup>2</sup> = T/4) including S<sup>1</sup> with respect to S4, where the latter signal is considered over the next oscillation period, i.e., T + τ<sup>4</sup> − τ<sup>1</sup> = T/4.

To calculate the feedback signals (Equation 7), the LFP of synchronized STN neurons is measured, filtered by the linear oscillator (Equation 6), i.e., the signal x(t) = ˙u from Equation (6) is delayed with delays τ<sup>i</sup> from Equation (7), and amplified by the factor K that is a dimensionless feedback gain and will be referred to as parameter of the stimulation intensity.

Instead of four delays, we use only two of them, e.g., τ<sup>1</sup> and τ2, and calculate the feedback signals S<sup>1</sup> and S<sup>2</sup> according to Equation (7). The other two feedback signals are defined by reversing polarity S<sup>3</sup> = −S<sup>1</sup> and S<sup>4</sup> = −S<sup>2</sup> (Hauptmann et al., 2005a,b, 2007a,b; Popovych et al., 2006; Omel'chenko et al.,

2008). We thus have two stimulation parameters that can be varied: The stimulation intensity (feedback gain) K and the stimulation delay τ . We also assume that neurons within the same sub-population assigned to the corresponding stimulation site receive approximately the same signal administered via that stimulation site, which is called a segmental stimulation, see Hauptmann et al. (2005a,b), Popovych et al. (2006), Hauptmann et al. (2007a,b), and Omel'chenko et al. (2008). More precisely, the neurons j = 1, 2, . .. , 50 are stimulated with the same signal S1, neurons j = 51, 52, ... , 100 are stimulated with the same signal S2, and so on.

The feedback signals S<sup>i</sup> are illustrated in **Figure 2A** (lower plot) together with filtered LFP (dashed red curve). For the considered delay τ ≈ T, where T ≈ 102 ms is the mean period of LFP oscillations, the neighboring feedback signals are time shifted by approximately T/4 with respect to each other if the filtered LFP signal is periodic or close to that. Delayed feedback stimulation with such smooth signals may be referred to as smooth feedback stimulation. Direct electrical stimulation of the brain with such signals might violate safety requirements and cause an irreversible charge deposit to the neuronal tissue and could lead to its damage (Harnack et al., 2004; Kuncel and Grill, 2004; Merrill et al., 2005). This problem was studied recently for single-site delayed feedback stimulation (Popovych et al., 2017a,b). By a similar token, we here use a high-frequency stimulation pulse train of the standard HF DBS consisting of biphasic charge-balanced pulses (Volkmann et al., 2002; Kuncel and Grill, 2004; Butson and McIntyre, 2007), whose amplitude is modulated by the slowly oscillating feedback signals Si(t) as schematically illustrated in **Figure 2B**, where an example of the pulsatile stimulation current Istim in Equation (1) is shown. The cathodic and anodic phases of the pulses administer the same charge of opposite polarity, and a charge-balanced stimulation is realized in this way. The resulting zero net charge injection after each short biphasic pulse can prevent from damaging nervous tissue (Lilly et al., 1955; Harnack et al., 2004; Kuncel and Grill, 2004; Merrill et al., 2005). Each pulse can contain an interphase time gap between its cathodic and anodic phases (**Figure 2B**, insert). We refer to the stimulation with such pulsatile signal whose amplitude is modulated by the smooth MLDF signals Si(t) as pulsatile MLDF stimulation.

## 3. RESULTS

### 3.1. Smooth MLDF

The impact of the smooth MLDF is illustrated in **Figure 3**, where synchronized STN neurons are directly stimulated by the smooth feedback signals (Equation 7), i.e., the stimulation currents Istim = S<sup>i</sup> in Equation (1) for STN neurons. Depending on the parameter of the stimulation delay τ , the stimulation can induce several qualitatively different dynamical regimes as reflected by the values of the order parameters R1, R2, and R<sup>4</sup> (**Figure 3A**). The stimulation can desynchronize the STN neurons, where all order parameters are small as illustrated in **Figure 3B** for τ = 72 ms. Another stimulation-induced regime is a two-cluster state characterized by small values of R<sup>1</sup> and large values of R2, see **Figure 3A** (red circles and green squares). One of such regimes is illustrated in **Figure 3C** for τ = 10 ms, where the stimulation splits the stimulated neuronal population into two groups of nearly in-phase synchronized neurons which are shifted by approximately 48 ms with respect to each other. These two clusters are thus nearly in anti-phase to each other (LFP oscillation period T ≈ 103), which results in a small first order parameter R<sup>1</sup> and large second order parameter R<sup>2</sup> (**Figure 3C**). Further regimes mimic a four-cluster state, where the fourth order parameter R<sup>4</sup> attains greater values as compared to R<sup>1</sup> and R2, see **Figures 3A,D** for τ = 114 ms. Although for such parameters the stimulation clearly splits the stimulated population into four distinct groups phase shifted with respect to each other (with the borders at the lattice coordinates i = 1, 50, 100, and 150, see **Figure 3D**, right panel), the neurons within these clusters can be far from in-phase synchronization. This leads to relatively small values of R<sup>4</sup> and little pronounced four-cluster states.

### 3.2. Pulsatile MLDF

As mentioned above, the safety requirements for the electrical stimulation of neuronal tissue may be violated for direct electrical stimulation with smooth and slowly oscillating feedback signals.

parameters (left plots) and spike raster plots of STN neurons (right plots) for fixed delay (B) τ = 72 ms, (C) τ = 10 ms, and (D) τ = 114 ms. The stimulation starts at t = 20 s. In plot (B) the time course of the order parameter R1 of the stimulation-free STN population is also shown (black curve). Coupling <sup>g</sup>G→<sup>S</sup> <sup>=</sup> 1.38 nS/µm<sup>2</sup> .

We thus utilize a pulsatile stimulation protocol for MLDF, see section Methods and **Figure 2B**. Such a pulsatile MLDF stimulation is administered to synchronized STN neurons (**Figure 1**) for different values of the stimulation parameters τ and K. The time-averaged first order parameter of the obtained stimulation-induced regimes is depicted in **Figure 4** in color vs. parameters (τ , K) for width of the interphase gap GW = 0 ms (**Figure 4A**) and 5 ms (**Figure 4B**). As compared to the case of smooth MLDF stimulation (**Figure 3**) the first order parameter R<sup>1</sup> exhibits much more pronounced alterations when the delay parameter τ is varied such that several desynchronization regions emerge in the parameter space characterized by small values of R1.

pulsatile MLDF. The values of the time-averaged order parameter R1 are encoded in color vs. the stimulation intensity K and the stimulation delay τ for the width of the interphase gap (A) GW = 0 ms and (B) 5 ms. Coupling <sup>g</sup>G→<sup>S</sup> <sup>=</sup> 1.38 nS/µm<sup>2</sup> .

When stimulation pulses include an interphase gap (**Figure 2B**) the desynchronizing effect of the pulsatile MLDF stimulation can significantly be improved as illustrated in **Figure 4B** for the interphase gap GW = 5 ms. In fact, for vanishing interphase gap the desynchronizing effect is only moderate (**Figure 4A**). The structure of the parameter space with desynchronization regions is preserved, but the extent of the stimulation-induced desynchronization is enhanced, as reflected by the values of the first order parameter, i.e., the values of R<sup>1</sup> get smaller, compare **Figure 4A** and **Figure 4B**. This indicates a favorable effect of the interphase gap on the desynchronization outcome of the pulsatile MLDF.

A detailed consideration by calculating all order parameters R1, R2, and R<sup>4</sup> reveals that pulsatile MLDF stimulation with zero gap does not induce any kind of clustering. This is implied by relatively large values of the first order parameter R<sup>1</sup> and small values of the other order parameters, see **Figure 5A**. As mentioned above (**Figure 4**), an increase of the interphase gap results in a decrease of the first order parameter, which can also be observed in **Figures 5A–C** (red circles). Simultaneously the other order parameters increase. For example, for the stimulation intensity K = 20, delay τ = 30 ms, and interphase gaps GW = 0 ms, 2 ms, and 5 ms the order parameters R<sup>1</sup> ≈ 0.54, 0.4, 0.3,

R<sup>2</sup> ≈ 0.23, 0.41, 0.5, and R<sup>4</sup> ≈ 0.14, 0.16, 0.26, respectively. A relatively large R<sup>2</sup> and small R<sup>1</sup> are indicative for a two-cluster state. The pattern of the neuronal firing of STN neurons for such parameters is illustrated in **Figure 5D**, where two groups of neurons (clusters) with different patterns of activity can be distinguished for neuron indices i < 100 and i > 100. Another example of a two-cluster state is illustrated in **Figure 5E** for the gap width GW = 5 ms and delay τ = 212 ms from another parameter region of a two-cluster regime of large R<sup>2</sup> and small R1, see **Figure 5C**.

We compare the desynchronizing impact of the pulsatile MLDF to that of pulsatile linear delayed feedback (LDF). The smooth and pulsatile LDF administered to synchronized STN neurons has been investigated in Popovych et al. (2017a,b) together with smooth and pulsatile nonlinear delayed feedback (NDF). The feedback signal S(t) of the differential LDF can be obtained as (Rosenblum and Pikovsky, 2004a,b):

$$S(t) = K(\mathbf{x}(t - \tau) - \mathbf{x}(t)),\tag{8}$$

where, as before, the variable x(t) is a filtered LFP and calculated by means of Equation (6), i.e., x(t) = ˙u, and K and τ are the parameters of the stimulation intensity and delay, respectively. The smooth feedback signal S(t) of LDF is then used to modulate the amplitude of the stimulation pulses as discussed above, where we assume that all STN neurons receive the same stimulation current Istim depicted in **Figure 2B**.

STN neurons can be desynchronized by the pulsatile LDF stimulation as illustrated in **Figure 6**. The parameter space of

and (D) pulsatile LDF for different width of the interphase gap as indicated in the legends. Stimulation delay (A,C) τ = 90 ms for MLDF and (B,D) τ = 60 ms for LDF. Coupling <sup>g</sup>G→<sup>S</sup> <sup>=</sup> 1.38 nS/µm<sup>2</sup> .

the pulsatile LDF contains large desynchronization regions, where the first order parameter R<sup>1</sup> exhibits pronounced minima (**Figures 6A,B**, red circles). An interphase gap of a finite width can enhance the desynchronizing effect of pulsatile LDF, where the desynchronization regions are enlarged and deepened such that the stimulation induces stronger desynchronization as reflected by smaller values of R<sup>1</sup> for larger interphase gap, compare **Figure 6A** and **Figure 6B** (red circles). The other order parameters of higher degree are however not significantly affected by the values of the interphase gap. For example, the values of the second order parameter R<sup>2</sup> are nearly preserved within the desynchronization regions when the width of the interphase gap increases from GW = 0 to 5 ms, compare **Figure 6A** and **Figure 6B** (green squares). Albeit the suppressed second order parameter R<sup>2</sup> can still be somewhat larger than the first order parameter R<sup>1</sup> (**Figure 6B**), the stimulated neurons do not exhibit any consistent clustering as illustrated in **Figure 6C**. This indicates that the pulsatile LDF stimulation does not induce clustering among stimulated STN neurons for any widths of the interphase gap, which is different to the impact of the pulsatile MLDF, see **Figure 5**.

The pulsatile LDF is more efficient in inducing desynchronization than the pulsatile MLDF. For the same stimulation intensity K and width of the interphase gap, the pulsatile LDF can induce much stronger desynchronization than the pulsatile MLDF as given by the values of the first order parameter R<sup>1</sup> within desynchronization regions, see **Figures 6A,B** and compare black solid curves (R<sup>1</sup> for MLDF) to red circles (R<sup>1</sup> for LDF). For a more detailed comparison, we fix optimal stimulation delay τ = 90 ms for pulsatile MLDF and τ = 60 ms for pulsatile LDF, where the stimulation induces strongest desynchronization, see **Figures 4**–**6**, and increase the stimulation intensity K. We find that both pulsatile MLDF and LDF stimulations with larger intensity can induce stronger desynchronization, and the first order parameter R<sup>1</sup> decreases when K increases, see **Figure 7A** for MLDF and **Figure 7B** for LDF (empty symbols). We also observe that R<sup>1</sup> decreases much faster with increasing K and can reach much smaller values for pulsatile LDF than for MLDF for the same range on the stimulation intensity.

An important characteristics of the stimulation of the neuronal tissue is the amount of the administered stimulation. We thus estimate it for the considered feedback stimulations by the time-averaged absolute value h|S|i of the smooth feedback signals (Equation 7) for MLDF and (Equation 8) and LDF. The amount of the administered stimulation is depicted in **Figure 7A** for MLDF and **Figure 7B** for LDF (filled symbols) vs. parameter K of the stimulation intensity. When K increases, the amount of the administered stimulation also increases such that stronger desynchronization can be obtained at stronger stimulation. Since the amplitude of the feedback signals depends on the amplitude of the LFP, it also inversely relates to the extent of the stimulationinduced desynchronization. For a given value of the stimulation intensity K, the amount of the administered stimulation h|S|i will be smaller if the stimulation-induced desynchronization is stronger, i.e., when the order parameter R<sup>1</sup> and amplitude of the LFP are smaller. Therefore, for the considered range of parameter K, h|S|i increases more slowly for pulsatile LDF than for pulsatile MLDF with increasing stimulation intensity, see **Figures 7A,B** (filled symbols). For both stimulation methods larger interphase gap leads to better desynchronization and smaller amount of the administered stimulation.

The latter claim is also supported by **Figures 7C,D**, where the amount of the administered stimulation h|S|i is depicted vs. the extent of the stimulation-induced desynchronization as reflected by the first order parameter R1. As follows, the same extent of desynchronization can be obtained at smaller amount of the administered stimulation for larger interphase gap. In such a way we can compare the efficacy in inducing desynchronization of MLDF and LDF methods by comparing the amount of the administered stimulation necessary to achieve the same extent of the stimulation-induced desynchronization. Comparing the depicted data for pulsatile MLDF (**Figure 7C**) and pulsatile LDF (**Figure 7D**) we conclude that the pulsatile LDF is more effective in inducing desynchronization than pulsatile MLDF, where desynchronization can be obtained for much smaller amount of the administered stimulation.

The above results obtained for the synchronized regime of STN-GPe neurons with irregular interburst intervals (**Figure 1** for coupling gG→<sup>S</sup> = 1.7 nS/µm<sup>2</sup> ) are preserved for other parameters and synchronized regimes in the considered populations of the STN-GPe neurons. We consider, for instance, stronger coupling gG→<sup>S</sup> = 1.7 nS/µm<sup>2</sup> , where the STN-GPe neurons are periodically synchronized, see **Figure 1A** (red curve) and **Figure 1D**. The desynchronizing effect of the pulsatile MLDF and LDF on the periodically synchronized STN-GPe neurons is illustrated in **Figure 8A**. The structure of the parameter space is preserved for both stimulation methods except for that the pulsatile MLDF induces a somewhat weaker desynchronization as compared to the above case of a weaker coupling. The clustering induced by the pulsatile MLDF for the considered stronger coupling also becomes less pronounced. The efficacy of the pulsatile MLDF and LDF in inducing desynchronization is compared in **Figure 8B**, where the amount of the administered stimulation h|S|i is depicted vs. the time-averaged first order parameter hR1i. As for the case of the irregular synchronization for weaker coupling (**Figure 7**), the interphase gap has the same favorable impact on the stimulation outcome, and the pulsatile LDF is apparently superior to MLDF and can induce stronger desynchronization for smaller amount of administered stimulation.

#### 3.3. Differential MLDF

To overcome the limitations of pulsatile MLDF revealed above, we suggest to use a differential MLDF. The feedback signals are constructed by analogy with the differential LDF (Equation 8) and read.

S the reached extent of the stimulation-induced desynchronization as given by values of R1 for pulsatile MLDF (empty symbols) with τ = 50 ms and LDF (filled symbols) with τ = 70 ms. The width of the interphase gap is indicated in the legend. Coupling <sup>g</sup>G→<sup>S</sup> <sup>=</sup> 1.7 nS/µm<sup>2</sup> .

$$S\_i(t) = K \cdot \left(\mathbf{x}(t - \tau\_i) - \mathbf{x}(t)\right), \ i = 1, 2, 3, 4,\tag{9}$$

where the signal x(t) is the filtered LFP from Equation (6), and the delays are as in Equation (7). As before, we reverse the polarity of the two feedback signals, such that S<sup>3</sup> = K · −x(t − τ1) − x(t) and S<sup>4</sup> = K · −x(t − τ2) − x(t) . An example of the feedback signals S<sup>i</sup> is illustrated in **Figure 9A**. As for the case of direct MLDF (Equation 7), the feedback signals of the differential MLDF are time shifted with respect to each other, but, in contrast to the direct MLDF, they may however have very different amplitude.

Stimulation by differential smooth MLDF can perturb the neuronal synchronization of the stimulated STN neurons as illustrated in **Figures 9B–D**, where the time-averaged order parameters hR1i, hR2i, and hR4i are plotted vs. the stimulation delay τ . Based on the values of the first order parameter

<sup>g</sup>G→<sup>S</sup> <sup>=</sup> 1.38 nS/µm<sup>2</sup> .

R1, smooth differential MLDF is less effective in inducing desynchronization than smooth direct MLDF, compare values of R<sup>1</sup> in **Figure 9B** of differential MLDF (red circles) to those of direct MLDF (black curve). For differential MLDF large values of the other order parameters are accompanied by large values of R1. This indicates that clusters of a possible clustered state are not equidistantly spaced over the oscillation period. Indeed, this is illustrated in **Figure 9C** for delay τ = 10 ms, where the fourth order parameter R<sup>4</sup> attains maximal values, see **Figure 9B**. In this example the stimulation divides the neurons into two groups, where the burst onsets of the two clusters are time shifted with respect to each other by 1T ≈ 26 ms (**Figure 9C**). Such firing patterns result in large values of the order parameters R<sup>1</sup> and R4, but in small values of R<sup>2</sup> as depicted in **Figure 9B** for the mentioned value of the stimulation delay. The best desynchronization can be achieved, for example, at τ = 90 ms as reflected by small values of R<sup>1</sup> (**Figure 9B**). The corresponding firing pattern of STN neurons is illustrated in **Figure 9D**.

In contrast to the stimulation with smooth signals, the differential pulsatile MLDF can induce stronger desynchronization as compared to the direct pulsatile MLDF. Two-parameter diagrams for differential pulsatile MLDF, where the order parameters are depicted in color in the (τ , K)-parameter plane, are shown in **Figure 10**. For the same range of the stimulation parameters the first order parameter R<sup>1</sup> (**Figures 10A,B**) exhibits smaller values as compared to those depicted in **Figure 4** for direct pulsatile MLDF. Interphase gap enhances the extent of the stimulation-induced desynchronization, compare **Figure 10A** and **Figure 10B**. For large interphase gap the (τ , K)-parameter plane contains a narrow region around τ ≈ 15 ms, where the second order parameter R<sup>2</sup> is relatively large (**Figure 10C**). For such parameters of the best two-cluster regime, the pattern of the neuronal firing looks similar to that in **Figure 5D**, i.e., there are no pronounced two- and four-cluster states.

To evaluate the stimulation outcome of the differential pulsatile MLDF we calculate the amount of the administered stimulation h|S|i that is approximated by the average of the absolute values of the MLDF feedback signals (Equation 9). In **Figure 11A** h|S|i is depicted together with the extent of the stimulation-induced desynchronization as given by the values of the first order parameter hR1i vs. parameter of the stimulation intensity K. For the differential pulsatile MLDF, the order parameter hR1i decreases much faster than for the direct pulsatile MLDF, and, as a results, h|S|i increases more slowly, compare **Figure 11A** to **Figure 7A**. The efficacy of the differential pulsatile MLDF in inducing desynchronization is comparable with that of the pulsatile LDF, see **Figure 7B**. Indeed, this conclusion apparently follows from **Figure 11B**, where a given extent of the stimulation-induced desynchronization as given by values of R<sup>1</sup> can be obtained at approximately the same amount of the administered stimulation for differential pulsatile MLDF (**Figure 11B**, empty symbols) and for pulsatile LDF (**Figure 11B**, filled symbols).

We also verified that differential pulsatile MLDF stimulation is robust with respect to the extent of the initial synchronization in the stimulated neuronal population. If the stimulation is administered to weakly synchronized neurons as, for example, for the coupling parameter gG→<sup>S</sup> = 1.28 nS/µm<sup>2</sup> , see **Figure 1**, the synchronization can further be suppressed by differential pulsatile MLDF practically irrespective of the values of the delay parameter τ , see **Figure 12**. Such a reduction of an already weak and intermittent neuronal synchronization is comparable with or even slightly better than that of pulsatile LDF, see also Popovych et al. (2017a,b).

#### 4. DISCUSSION

Multisite linear delayed feedback (MLDF) has been suggested for control of neuronal synchronization patterns (Hauptmann et al., 2005a,b, 2007a,b; Popovych et al., 2006; Omel'chenko et al., 2008). As shown computationally, stimulation by MLDF can

suppress synchronization in a stimulated neuronal population and was, hence, suggested for counteracting abnormal neuronal synchronization characteristic for several neurological disorders (Hauptmann et al., 2005a,b, 2007b; Popovych et al., 2006). The desynchronization induced by MLDF stimulation was found to be accompanied by the emergence of several interacting clusters of neurons equidistantly distributed over the oscillation period and space. Depending on the stimulation setup and parameters the stimulation-induced spatio-temporal patters can consist of two or four clusters, for example, for four-site MLDF stimulation (Hauptmann et al., 2005a,b, 2007a,b; Popovych et al., 2006; Omel'chenko et al., 2008). These properties of MLDF made the method appropriate for the control of spatio-temporal patterns of neuronal activity, for example, for regulating activity of central pattern generator (CPG) in case of its malfunction as suggested in Hauptmann et al. (2007a,b) and Omel'chenko et al. (2008).

In this study we adapted the MLDF technique for electrical stimulation of the neuronal tissue. As mentioned above, direct electrical stimulation with smooth feedback signals may violate safety aspects like charge density limits (Harnack et al., 2004; Kuncel and Grill, 2004; Merrill et al., 2005). The feedback signals are slow, so that an irreversible charge can be deposited into the neuronal tissue during the comparably long feedback stimulation periods, which can exceed safety limits. We resolved this problem and showed that the demand-controlled character and desynchronizing impact of the MLDF feedback technique can be preserved together with gaining the advantages of the pulsatile HF DBS signal with the charge-balanced property. For this, the slow feedback signal is used to modulate the amplitude of the HF train of charge-balanced pulses of HF DBS. We computationally illustrated the desynchronizing properties of smooth and pulsatile MLDF in a network of STN-GPe neurons suggested to model parkinsonian neuronal dynamics (Terman et al., 2002; Rubin and Terman, 2004).

We showed that both smooth and pulsatile MLDF stimulation can suppress neuronal synchronization in the stimulated STN neurons. While smooth MLDF could induce relatively well pronounced two-cluster states in some parameter ranges, the expected four-cluster states were only weakly expressed and could be observed in a limited parameter range. For pulsatile MLDF we found that interphase gap in the stimulation pulses could significantly enhance the desynchronizing impact of the stimulation. The clustering state was observed for large interphase gap only, where some two-cluster states could be induced by the stimulation for some selected parameter values. We thus conclude that the pulsatile MLDF is mostly a desynchronizing stimulation rather than inducing coordinated spatio-temporal clustering patterns. We however showed that the efficacy of the pulsatile MLDF in inducing desynchronization was much lower than that of the pulsatile LDF. Therefore, in the standard realization the pulsatile MLDF can be suggested as effective method neither for desynchronization nor for the control of spatio-temporal clustering patterns.

We therefore proposed to use a differential pulsatile MLDF. Such a modified pulsatile MLDF turned out to hardly induce clustering for any width of the interphase gap. Nevertheless, we showed that differential pulsatile MLDF can effectively and robustly desynchronize the stimulated neurons. We verified that the efficacy of the differential pulsatile MLDF in inducing desynchronization is comparable with that of the pulsatile LDF, and we suggest this technique for closed-loop desynchronizing DBS together with pulsatile LDF and NDF investigated in recent papers (Popovych et al., 2017a,b). The differential pulsatile

MLDF is robust with respect to variations of the stimulation parameters, in particular, if initially weakly synchronized neuronal populations need to be further desynchronized. However, MLDF requires several stimulation sites to be placed in the neuronal target population. In the case of small targets this might be difficult, so that single-site stimulation techniques such as pulsatile LDF and NDF may be more appropriate in those cases. It would be interesting to investigate how the effectiveness and efficacy of pulsatile NDF stimulation are affected if a multisite stimulation protocol is adapted for this method. It would also be interesting to use a realistic 3-Dim reconstruction of the target neuronal structures, e.g., STN and GPe, as well as localization of the stimulation sites within these structures to explore the spatiotemporal patterns induced by a multisite stimulation (Ebert et al., 2014), which however essentially requires the usage of a supercomputer.

However, as a word of caution, it should be noted that the approach presented in this study relies on the assumption that abnormal neuronal synchrony is recordable and represents the patient's individual symptoms in a sufficient manner, like a biomarker (Beudel and Brown, 2016; Kühn and Volkmann, 2017). For instance, it is doubtful that beta band oscillations might be a biomarker-like feedback signal (Özkurt et al., 2011; Johnson et al., 2016; Kühn and Volkmann, 2017). Beta band oscillations are no stand-alone oscillations, but interact with brain oscillations in other frequency bands under both physiological (Yanagisawa et al., 2012) and pathological (Yang et al., 2014; Beudel and Brown, 2016) conditions. Different PD phenotypes might require different biomarkers, since the amplitude of beta band oscillations may decrease during tremor epochs in tremor dominant PD patients (Quinn et al., 2015). Changes of the amplitude of abnormal brain oscillations in the course of physiological processes have to be taken into account, too. For instance, in an MPTP monkey study HF DBS and closed-loop DBS (CL-DBS) reduced rigidity to a comparable extent, where CL-DBS reduced the DBS ON time by approx. 50% (Johnson et al., 2016). However, only HF DBS improved bradykinesia during a cued reaching task, likely because the amplitude of beta band oscillations was reduced related to the reaching process, in this way reducing the extent of the presumed biomarker (Johnson et al., 2016). Also, beta band oscillations need not be entirely pathological. Rather activity in the beta frequency range might be key for compensatory purposes, as demonstrated in an MPTP monkey study with sensorimotor rhythm neurofeedback (Philippens et al., 2017).

FIGURE 12 | Impact of the differential pulsatile MLDF on collective dynamics

(as indicated in the legend) vs. parameter of the feedback delay τ for fixed

stimulated by pulsatile LDF is also shown by the black solid curve for comparison. The horizontal dashed line indicates the value of

Ri 

> R1

, i = 1, 2, 4, are plotted

R1 without

of the STN neurons

of weakly coupled and weakly synchronized STN-GPe neurons

(Equations 1–4). Time-averaged order parameters

stimulation intensity K = 10. The first order parameter

stimulation (K = 0), see Figure 1A (black curve). Coupling <sup>g</sup>G→<sup>S</sup> <sup>=</sup> 1.28 nS/µm<sup>2</sup> and interphase gap GW <sup>=</sup> 5 ms.

Apart from merely reducing the stimulation current, differential pulsatile MLDF may be beneficial because of its specifically desynchronizing effect. As shown computationally in the context of Coordinated Reset (CR) stimulation (Tass, 2003), desynchronizing stimulation may cause an anti-kindling, where abnormal synaptic connectivity and neuronal synchrony can be unlearned, ultimately leading to sustained desynchronizing effects (Tass and Majtanik, 2006; Popovych and Tass, 2012). In accordance with these theoretical predictions, long-lasting therapeutic effects were demonstrated in pre-clinical studies

in MPTP monkeys with CR-DBS (Tass et al., 2012b; Wang et al., 2016) as well as in a clinical proof of concept study with CR-DBS in Parkinson's patients (Adamchic et al., 2014). Analogously, long-lasting therapeutic effects were observed in a proof of concept study with acoustic CR stimulation in tinnitus patients (Tass et al., 2012a) as well as in a first in man study with vibrotactile CR stimulation in patients with Parkinson's disease (Syrkin-Nikolau et al., 2018).

The requirements for CR-DBS and MLDF are quite different. CR stimulation can be administered in open loop as well as closed loop, e.g., demand-controlled manner (Tass, 2003). In particular, CR stimulation does not require a feedback signal. So far, pre-clinical (Tass et al., 2012b; Wang et al., 2016) and clinical (Adamchic et al., 2014) proof of concept of CR-DBS were obtained with open loop CR-DBS. In contrast, MLDF requires a reliably measurable clean biomarker signal sufficiently representing the amount of abnormal synchronization. Despite first positive results (Little et al., 2013; Rosa et al., 2015), several findings indicate that beta-band STN LFP does not provide a reliable biomarker (Özkurt et al., 2011; Quinn et al., 2015; Johnson et al., 2016; Kühn and Volkmann, 2017; Philippens et al., 2017), see above. In addition, MLDF requires a challenging registration-stimulation setup: An LFP signal, representative for the entire neuronal target population, has to be measured at one site, while stimuli have to be delivered to different sites of the

#### REFERENCES


target population. Because of stimulation artifacts this might be difficult. However, to overcome this limitation, alternatively, one might try to measure a representative LFP and stimulate different parts of fibers projecting on the target population. One could also separate stimulation and recording in time (Ratas and Pyragas, 2014).

### AUTHOR CONTRIBUTIONS

PT conceived HFS amplitude modulation by feedback. OP performed the experiments, analyzed the data, and prepared the initial draft of the manuscript. PT contributed to the numerical analysis and extended the manuscript. All authors reviewed the manuscript.

#### FUNDING

The study was funded by the Helmholtz Society and by the John A. Blume Foundation.

#### SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fphys. 2018.00046/full#supplementary-material


neuromodulation. Rest. Neurol. Neurosci. 30, 137–159. doi: 10.3233/RNN-2012-110218


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Popovych and Tass. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Stochastic Effects in Autoimmune Dynamics

Farzad Fatehi <sup>1</sup> , Sergey N. Kyrychko<sup>2</sup> , Aleksandra Ross <sup>1</sup> , Yuliya N. Kyrychko<sup>1</sup> and Konstantin B. Blyuss <sup>1</sup> \*

<sup>1</sup> Department of Mathematics, University of Sussex, Brighton, United Kingdom, <sup>2</sup> Institute of Geotechnical Mechanics, Dnipro, Ukraine

Among various possible causes of autoimmune disease, an important role is played by infections that can result in a breakdown of immune tolerance, primarily through the mechanism of "molecular mimicry". In this paper we propose and analyse a stochastic model of immune response to a viral infection and subsequent autoimmunity, with account for the populations of T cells with different activation thresholds, regulatory T cells, and cytokines. We show analytically and numerically how stochasticity can result in sustained oscillations around deterministically stable steady states, and we also investigate stochastic dynamics in the regime of bi-stability. These results provide a possible explanation for experimentally observed variations in the progression of autoimmune disease. Computations of the variance of stochastic fluctuations provide practically important insights into how the size of these fluctuations depends on various biological parameters, and this also gives a headway for comparison with experimental data on variation in the observed numbers of T cells and organ cells affected by infection.

#### Edited by:

Alexey Zaikin, University College London, United Kingdom

#### Reviewed by:

Anna Zakharova, Technische Universität Berlin, Germany Silvina Ponce Dawson, Universidad de Buenos Aires, Argentina

#### \*Correspondence:

Konstantin B. Blyuss k.blyuss@sussex.ac.uk

#### Specialty section:

This article was submitted to Computational Physiology and Medicine, a section of the journal Frontiers in Physiology

Received: 19 October 2017 Accepted: 15 January 2018 Published: 02 February 2018

#### Citation:

Fatehi F, Kyrychko SN, Ross A, Kyrychko YN and Blyuss KB (2018) Stochastic Effects in Autoimmune Dynamics. Front. Physiol. 9:45. doi: 10.3389/fphys.2018.00045 Keywords: pathogen-induced autoimmunity, immune response, mathematical model, bi-stability, stochastic effects

### 1. INTRODUCTION

Breakdown of immune tolerance and the resulting autoimmune disease occur when the immune system fails to distinguish the host's own healthy cells from the cells affected by the infection, thus triggering an immune response that also targets healthy cells. Autoimmune disease is usually focused in a specific organ or part of the body, such as retina in the case of uveitis, central nervous system in multiple sclerosis, or pancreatic β-cells in insulin-dependent diabetes mellitus type-1 (Prat and Martin, 2002; Kerr et al., 2008; Santamaria, 2010). Whilst it is close to impossible to pinpoint precise causes of autoimmunity in each individual case, it can usually be attributed to a number of factors, which can include the genetic predisposition, age, previous immune challenges, exposure to pathogens etc. A number of distinct mechanisms have been identified for how an infection of the host with a pathogen can result in the subsequent onset of autoimmune disease, and these include bystander activation (Fujinami, 2011) and molecular mimicry (von Herrath and Oldstone, 1996; Ercolini and Miller, 2008), which is particularly important in the context of autoimmunity caused by viral infections.

Over the years, a number of mathematical models have investigated various origins and aspects of immune response, with an emphasis on the onset and the development of autoimmune disease. Some of the earlier models studied interactions between regulatory and effector T cells without looking at causes of autoimmunity but instead focusing on T cell vaccination (Segel et al., 1995). Borghans et al. (Borghans and De Boer, 1995; Borghans et al., 1998) looked into this process in more

**52**

detail and showed the onset of autoimmunity, which was defined as oscillations in the number of autoreactive cells that exceeded a certain threshold. León et al. (2000, 2003, 2004) and Carneiro et al. (2005) have analyzed interactions between different T cells and their effect on regulation of immune response and control of autoimmunity. More recently, Iwami et al. (2007, 2009) considered a model of immune response to a viral infection, in which they explicitly included the dynamics of a virus population. Although this model is able to demonstrate an emergence of autoimmunity, it fails to produce a regime of normal viral clearance. Alexander and Wahl (2011) have focused on the role of professional antigen-presenting cells (APCs) and their interactions with regulatory and effector effector cells for the purposes of controlling autoimmune response. Burroughs et al. (2011a,b) have analyzed the autoimmunity through the mechanism of cytokine-mediated bystander activation. A special issue on "Theories and modeling of autoimmunity" provides an excellent overview of recent research in the area of mathematical modeling of various aspects of onset and development of autoimmune disease (Root-Bernstein, 2015).

These are several different frameworks for modeling the role of T cells in controlling autoimmune response. Alexander and Wahl (2011) and Burroughs et al. (2011a,b) have explicitly included a separate compartment for regulatory T cells that are activated by autoantigens and suppress the activity of autoreactive T cells. Another modeling approach is to consider the possibility of the same T cells performing different immune functions through having different or tunable activation thresholds, which allows T cells to adjust their response to T cell antigen receptor stimulation by autoantigens. This methodology was originally proposed theoretically to study peripheral and central T cell activation (Grossman and Paul, 1992, 2000; Grossman and Singer, 1996), and has been subsequently used to analyse differences in activation/response thresholds that are dependent on the activation state of the T cell (Altan-Bonnet and Germain, 2005). van den Berg and Rand (2004) and Scherer et al. (2004) have studied stochastic tuning of activation thresholds. Interestingly, the need for T cells to have tunable activation can be shown to emerge from the fundamental principles of of signal detection theory (Noest, 2000). A number of murine and human experiments have confirmed that activation of T cells can indeed dynamically change during their circulation (Nicholson et al., 2000; Bitmansour et al., 2002; Stefanova et al., 2002; Römer et al., 2011), thus supporting the theory developed in earlier papers.

Since immune response is known to be a complex multi-factor process (Perelson and Weisbuch, 1997), a number of studies have looked into various stochastic aspects of immune dynamics, such as T cell selection and proliferation. Deenick et al. (2003) have analyzed stochastic effects of interleukin-2 (IL-2) on T cell proliferation from precursors. Blattman et al. (2000) have shown that repertoires of the CTL (cytotoxic T cell lymphocyte) populations during primary response to a viral infection and in the memory pool are similar, thus providing further support to the theory of stochastic selection for the memory pool. Detours and Perelson (2000) have explored the distribution of possible outcomes during T cell selection with account for variable affinity between T cell receptors and MHC-peptide complexes. Chao et al. (2004) analyzed a detailed stochastic model of T cell recruitment during immune response to a viral infection. Stirk et al. (2010a,b) have developed a stochastic model for T cell repertoire and investigated the role of competitive exclusion between different clonotypes. Using the methodology of continuous-time Markov processes, the authors computed extinction times, a limited multivariate probability distribution, as well as the size of fluctuations around the deterministic steady states. Reynolds et al. (2012) have used a similar methodology to investigate an important question of asymmetric cell division and its impact on the extinction of different T cell populations and the expected lifetimes of naïve T cell clones. With regards to modeling autoimmune dynamics, Alexander and Wahl (2011) have studied the stochastic model of immune response with an emphasis on professional APCs to show that the probability of developing a chronic autoimmune response increases with the initial exposure to self-antigen or autoreactive effector T cells. An important aspect of stochastic dynamics that has to be accounted for in the models is the so-called stochastic amplification (Alonso et al., 2007; Kuske et al., 2007), which denotes a situation where periodic solutions with decaying amplitudes in the deterministic model can result in sustained stochastic periodic oscillations in individual realizations of the same model. This suggests that whilst on average the behavior may show decaying-amplitude oscillations, individual realizations represented by stochastic oscillations can explain relapses/remissions in clinical manifestations of the disease as caused by endogenous stochasticity of the immune processes.

Blyuss and Nicholson (2012, 2015) have proposed and analyzed a mathematical model of immune response to a viral infection that explicitly takes into account the populations of two types of T cells with different activation thresholds and also allows for infection and autoimmune response to occur in different organs. This model supports the regimes of normal viral clearance, a chronic infection, and an autoimmune state represented by exogenous oscillations in cell populations, associated with episodes of high viral production followed by long periods of quiescence. Such behavior, that in the clinical observation could be associated with relapses and remissions, has been observed in a number of autoimmune diseases, such as MS, autoimmune thyroid disease and uveitis (Ben Ezra and Forrester, 1995; Davies et al., 1997; Nylander and Hafler, 2012). Despite its successes, this model has a limitation that the periodic oscillations are only possible when the amount of free virus and the number of infected cells are also exhibiting oscillations, while in laboratory and clinical situations, one rather observes a situation where the initial infection is completely cleared, and this is then followed by the onset of autoimmune reaction. To overcome this limitation, Fatehi et al. (unpublished) have recently extended the model of Blyuss and Nicholson to also include the population of regulatory T cells and the cytokine mediating T cell activity.

In this paper we analyse the effects of stochasticity on the dynamics of immune response in a model with the populations of T cells with different activation thresholds, regulatory cells and cytokines, as presented in Methods. Starting with a system of ordinary differential equations, we apply the methodology of continuous-time Markov chains (CTMC) to derive a Kolmogorov, or chemical master equation, describing the dynamics of a probability distribution of finding the system in a particular state. To make further analytical and numerical progress, we derive an Itô stochastic differential equation, whose solutions provide similar stochastic paths to those of the CTMC models. This then allows us to numerically study the stationary multivariate probability distributions for the states in the model, explore stochastic amplification, determine how the magnitude of stochastic fluctuations around deterministic steady states depends on various parameters, and investigate the effects of initial conditions on the outcome in the case of bi-stability between different dynamical states. These results suggest that the experimentally observed variation in the progression of autoimmune disease can be attributed to stochastic amplification, and they also provide insights into how the variance of fluctuations depends on parameters, which can guide new laboratory experiments.

#### 2. METHODS

### 2.1. Continuous-Time Markov Chain Model of Immune Dynamics

In a recent paper we introduced and analyzed a deterministic model for autoimmune dynamics with account for the populations of T cells with different activation thresholds and cytokines (Fatehi et al. unpublished). The analysis showed that depending on parameters and initial conditions, the model can support the regimes of normal disease clearance, where an initial infection is cleared without further consequences for immune dynamics, chronic infection characterized by a persistent presence of infected cells in the body, and the state of autoimmune behavior where after clearance of initial infection, the immune system supports stable endogenous oscillations in the number of autoreactive T cells, which can be interpreted in the clinical practice of autoimmune disease as periods of relapses and remissions. This work extended earlier results on modeling the effects of tunable activation thresholds (Blyuss and Nicholson, 2012, 2015) by including regulatory T cells, as well as the cytokine mediating proliferation and activity of different types of T cells.

A deterministic model for immune response to a viral infection, as illustrated in a diagram shown in **Figure 1**, has the form

$$\begin{aligned} \frac{dS}{dt} &= rS\left(1 - \frac{S}{N}\right) - \beta SF - \mu\_a T\_{\text{aut}} S, \\ \frac{dF}{dt} &= \beta SF - d\_F F - \mu\_F T\_{\text{nor}} F - \mu\_a T\_{\text{aut}} F, \\ \frac{dT\_{in}}{dt} &= \lambda\_{in} - d\_{in} T\_{in} - \alpha T\_{in} F, \\ \frac{dT\_{\text{reg}}}{dt} &= \lambda\_r - d\_r T\_{\text{reg}} + p\_1 \alpha T\_{in} F + \rho\_1 T\_{\text{reg}}, \\ \frac{dT\_{\text{nor}}}{dt} &= p\_2 \alpha T\_{\text{in}} F - d\_n T\_{\text{nor}} + \rho\_2 T\_{\text{nor}}, \\ \frac{dT\_{\text{aut}}}{dt} &= (1 - p\_1 - p\_2) \alpha T\_{\text{in}} F - d\_a T\_{\text{aut}} - \delta T\_{\text{reg}} T\_{\text{aut}} + \rho\_3 T\_{\text{aut}}, \\ \frac{dT}{dt} &= \sigma\_1 T\_{\text{nor}} + \sigma\_2 T\_{\text{aut}} - d\_i I, \end{aligned}$$

where S(t) is the number of susceptible organ cells, F(t) is the number of infected cells, Tin(t) is the number of naïve T cells, Treg (t) is the number of regulatory T cells, Tnor(t) is the number of activated T cells which recognize foreign antigen and destroy infected cells, Taut(t) is the number of autoreactive T cells which destroy cells presenting both foreign and self-antigen, and I(t) is the amount of interleukin 2 (IL-2) cytokine. In this model, it is assumed that in the absence of infection, organ cells in the host reproduce logistically with a linear growth rate r and carrying capacity N, and they can become infected at rate β by already infected cells that are producing new virus particles. Unlike earlier models (Blyuss and Nicholson, 2012, 2015; Fatehi et al. unpublished), we consider the situation where the process of producing virions by infected cells is quite fast, hence, we do not explicitly incorporate a separate compartment for free virus. Regarding immune response, we assume that naïve T cells remain in homeostasis, and upon activation at rate α by a signal from infected cells, a proportion p<sup>1</sup> of them will develop into regulatory T cells, a proportion p<sup>2</sup> will become normal activated T cells able to destroy infected cells at rate µF, and the remaining T cells will become autoreactive, in which case their threshold for activation by susceptible cells is reduced, and hence, they will be destroying both infected and susceptible host cells at rate µa. The effect of regulatory T cells is in reducing the number of autoreactive T cells at rate δ, and regulatory T cells are themselves assumed to be in a state of homeostasis. Finally, normal and autoreactive T cells produce IL-2 at rates σ<sup>1</sup> and σ2, and IL-2 in turn facilitates proliferation of regular, normal and autoreactive T cells at rates ρ1, ρ2, and ρ3, respectively. One should note that in light of experimental evidence suggesting the possibility of autoimmunity in the absence of B cells (Wolf et al., 1996) and the fact that the development of antibodies can itself depend on

Prob(1**X** = **i**|**X** = **n**) = q11t + o(1t), **i** = (1, 0, 0, 0, 0, 0, 0), q21t + o(1t), **i** = (−1, 0, 0, 0, 0, 0, 0), q31t + o(1t), **i** = (−1, 1, 0, 0, 0, 0, 0), q41t + o(1t), **i** = (0, 0, 1, 0, 0, 0, 0), q51t + o(1t), **i** = (0, 0, −1, 0, 0, 0, 0), q61t + o(1t), **i** = (0, 0, −1, 0, 1, 0, 0), q71t + o(1t), **i** = (0, 0, −1, 0, 1, 0, 0), q81t + o(1t), **i** = (0, 0, −1, 0, 0, 1, 0), q91t + o(1t), **i** = (0, −1, 0, 0, 0, 0, 0), q101t + o(1t), **i** = (0, 0, 0, 1, 0, 0, 0), q111t + o(1t), **i** = (0, 0, 0, −1, 0, 0, 0), q121t + o(1t), **i** = (0, 0, 0, 0, 1, 0, 0), q131t + o(1t), **i** = (0, 0, 0, 0, −1, 0, 0), q141t + o(1t), **i** = (0, 0, 0, 0, 0, 1, 0), q151t + o(1t), **i** = (0, 0, 0, 0, 0, −1, 0), q161t + o(1t), **i** = (0, 0, 0, 0, 0, 0, 1), q171t + o(1t), **i** = (0, 0, 0, 0, 0, 0, −1), 1 − P 17 i=1 qi1t + o(1t), **i** = (0, 0, 0, 0, 0, 0, 0), o(1t), otherwise,

prior T cell activation with bacteria (Wu et al., 2010), the above model does not take into account antibody response, but rather focuses on T cell dynamics.

As a first step in the analysis of stochastic effects in immune dynamics, we construct a CTMC model based on the ODE model (1) using the methodology developed earlier in the context of modeling stochastic effects in epidemic and immunological models (Brauer et al., 2008; Stirk et al., 2010a; Allen, 2011). To this end, we introduce variables X1(t), . . . , X7(t) ∈ {0, 1, 2, . . .} as discrete random variables representing the number of uninfected cells, infected cells, naïve T cells, regulatory T cells, normal activated T cells, autoreactive T cells, and interleukin-2 at time t, respectively. Let the initial condition be fixed as

$$\mathbf{X}\_0 = (X\_1(0), \dots, X\_7(0)) = (n\_{10}, n\_{20}, n\_{30}, n\_{40}, n\_{50}, n\_{60}, n\_{70}).$$

The probability of finding the system in the state **n** = (n1, n2, n3, n4, n5, n6, n7) with n<sup>i</sup> ∈ {0, 1, 2, ...} at time t can be defined as

$$P(\mathbf{n}, t) = \text{Prob}\{\mathbf{X}(t) = \mathbf{n} | \mathbf{X}(0) = \mathbf{X}\_0\}.$$

Let 1t be sufficiently small such that 1Xi(t) = Xi(t + 1t) − Xi(t) ∈ {−1, 0, 1} for 1 ≤ i ≤ 7. The CTMC can then be formulated as a birth and death process in each of the variables (Allen, 2011). The infinitesimal transition probabilities corresponding to **Figure 1** are as follows,

(2)

where

$$\begin{aligned} q\_1 &= b\_1 n\_1 + b\_2 n\_1^2, \quad q\_2 = d\_1 n\_1 + d\_2 n\_1^2 + \mu\_a n\_1 n\_6, \quad q\_3 = \beta n\_1 n\_2, \quad q\_4 = \lambda\_{\text{in}}, \\ q\_5 &= d\_{\text{in}} n\_5, \quad q\_6 = p\_1 \alpha n\_2 n\_3, \quad q\_7 = p\_2 \alpha n\_2 n\_3, \quad q\_8 = (1 - p\_1 - p\_2) \alpha n\_2 n\_3, \\ q\_9 &= (d\_F + \mu\_F n\_5 + \mu\_a n\_6) n\_2, \quad q\_{10} = \lambda\_r + \rho\_1 n\_4 n\_7, \quad q\_{11} = d\_r n\_4, \quad q\_{12} = \rho\_2 n\_5 n\_7, \\ q\_{13} &= d\_n n\_5, \quad q\_{14} = \rho\_3 n\_6 n\_7, \quad q\_{15} = (d\_a + \delta n\_4) n\_6, \quad q\_{16} = \sigma\_1 n\_5 + \sigma\_2 n\_6, \quad q\_{17} = d\_i n\_7. \end{aligned}$$

Here, b1n<sup>1</sup> + b2n 2 1 and d1n<sup>1</sup> + d2n 2 1 are natural birth and death rates for uninfected cells with b<sup>1</sup> − d<sup>1</sup> = r and d<sup>2</sup> − b<sup>2</sup> = r/N (Allen, 2011).

The probabilities P(**n**, t) satisfy the following master equation (forward Kolmogorov equation)

transitions in the CTMC model (2). Using this table of possible state changes, one can compute the expectation vector and covariance matrix of 1**Y** for sufficiently small 1t (Allen et al., 2008; Mandal et al., 2014). The expectation vector to order 1t is given by

$$\begin{split} \frac{dP(\mathbf{n},t)}{dt} &= \mathbb{I}(\varepsilon\_1^- - 1)q\_1 + (\varepsilon\_1^+ - 1)q\_2 + (\varepsilon\_1^+ \varepsilon\_2^- - 1)q\_3 + (\varepsilon\_3^- - 1)q\_4 + (\varepsilon\_3^+ - 1)q\_5 \\ &+ (\varepsilon\_3^+ \varepsilon\_4^- - 1)q\_6 + (\varepsilon\_3^+ \varepsilon\_5^- - 1)q\_7 + (\varepsilon\_3^+ \varepsilon\_6^- - 1)q\_8 + (\varepsilon\_2^+ - 1)q\_9 + (\varepsilon\_4^- - 1)q\_{10} \\ &+ (\varepsilon\_4^+ - 1)q\_{11} + (\varepsilon\_5^- - 1)q\_{12} + (\varepsilon\_5^+ - 1)q\_{13} + (\varepsilon\_6^- - 1)q\_{14} + (\varepsilon\_6^+ - 1)q\_{15} \\ &+ (\varepsilon\_7^- - 1)q\_{16} + (\varepsilon\_7^+ - 1)q\_{17})P(\mathbf{n},t). \end{split} \tag{3}$$

where the operators ε ± i are defined as follows,

$$
\varepsilon\_i^{\pm} f(n\_1, n\_2, n\_3, n\_4, n\_5, n\_6, n\_7, t) \;= f(n\_1, \ldots, n\_i \pm 1, \ldots, n\_7, t),
$$

for each 1≤ i ≤ 7, and if n<sup>i</sup> < 0 for any 1 ≤ i ≤ 7, then P(**n**, t) = 0.

By solving this master equation, one can find the probability density function for this model. However, since this is a highdimensional difference-differential equation, solving it is a very challenging task. Normally, the number of events occurring in a small time step in the CTMC model is extremely large, hence using the CTMC model for plotting stochastic trajectories is very computationally intensive (Mandal et al., 2014). A much more computationally efficient approach is to use chemical Langevin equations (Gillespie, 2000, 2002), also known as Itô stochastic differential equation (SDE) models, which provide very similar sample paths to those of the CTMC models (Mandal et al., 2014). While both Itô and Stratonovich interpretations of stochastic calculus can be applied (Øksendal, 2000), in biological applications Itô formulation is more frequently used due to its non-anticipatory nature and a closer connection to numerical implementation (Allen, 2007, 2011; Braumann, 2007).

#### 2.2. Stochastic Differential Equation Model

To derive Itô SDE model, let **Y**(t) = (Y1(t), Y2(t), Y3(t), Y4(t), Y5(t), Y6(t), Y7(t)) be a continuous random vector for the sizes of various cell compartments at time t. Similar to the CTMC model, we assume that 1t is small enough so that during this time interval at most one change can occur in state variables. These changes together with their probabilities are listed in **Table 1**, which is again based on **Figure 1** and where

$$\mu = \begin{pmatrix} P\_1 - P\_2 - P\_3 \\ P\_3 - P\_9 \\ P\_4 - P\_5 - P\_6 - P\_7 - P\_8 \\ P\_6 + P\_{10} - P\_{11} \\ P\_7 + P\_{12} - P\_{13} \\ P\_8 + P\_{14} - P\_{15} \\ P\_{16} - P\_{17} \end{pmatrix}$$

Pi(1**Y**)i1t = µ1t,

E(1**Y**) ≈

X 17

i=1



is the drift vector, which can be easily seen to be identical to the right-hand side of the deterministic model Equation (1). The covariance matrix is obtained by keeping terms of order 1t only, i.e.,

$$\begin{aligned} \text{cov}(\Delta \mathbf{Y}) &= \mathbb{E}\left[ (\Delta \mathbf{Y})(\Delta \mathbf{Y})^{T} \right] \\ &- \mathbb{E}\left[ \Delta \mathbf{Y} \right] (\mathbb{E}\left[ \Delta \mathbf{Y} \right])^{T} \approx \mathbb{E}\left[ (\Delta \mathbf{Y})(\Delta \mathbf{Y})^{T} \right] \\ &= \sum\_{i=1}^{17} P\_{i} (\Delta \mathbf{Y})\_{i} (\Delta \mathbf{Y}\_{i})^{T} \Delta t = \Sigma \Delta t, \end{aligned}$$

where

and **W**(t) = [W1(t), W2(t), ..., W11(t)]<sup>T</sup> is a vector of 11 independent Wiener processes (Allen, 2007).

In order to make further analytical progress, we find an approximate probability density function for the model (4) as given by an approximate solution of the master equation (van Kampen, 1981; Allen, 2007). Let P(**Y**, t) be the probability density function of the model (4). Then P(**Y**, t) satisfies the following Fokker-Planck equation (Gardiner, 2004; Allen, 2007) which is an approximation of the master equation

$$
\Sigma = \begin{pmatrix}
 P\_1 + P\_2 + P\_3 & -P\_3 & 0 & 0 & 0 & 0 & 0 \\
 -P\_3 & P\_3 + P\_9 & 0 & 0 & 0 & 0 & 0 \\
 0 & 0 & P\_4 + P\_5 + P\_6 + P\_7 + P\_8 & -P\_6 & -P\_7 & -P\_8 & 0 \\
 0 & 0 & -P\_6 & P\_6 + P\_{10} + P\_{11} & 0 & 0 & 0 \\
 0 & 0 & -P\_7 & 0 & P\_7 + P\_{12} + P\_{13} & 0 & 0 \\
 0 & 0 & -P\_8 & 0 & 0 & P\_8 + P\_{14} + P\_{15} & 0 \\
 0 & 0 & 0 & 0 & 0 & 0 & P\_{16} + P\_{17}
\end{pmatrix}
$$

is a 7 × 7 covariance matrix. To derive Itô SDE model, we need to find a diffusion matrix H defined according to HH<sup>T</sup> = 6. Although this matrix is not unique, different forms of this matrix give equivalent systems (Allen, 2007; Allen et al., 2008).

If one rewrites the covariance matrix 6 in the form

$$
\Sigma = \begin{pmatrix} U & \mathbf{0} & \mathbf{0} \\ \mathbf{0} & W & \mathbf{0} \\ \mathbf{0} & \mathbf{0} & Z \end{pmatrix},
$$

with

$$U = \begin{pmatrix} P\_1 + P\_2 + P\_3 & -P\_3 \\ -P\_3 & P\_3 + P\_9 \end{pmatrix}, \qquad Z = P\_{16} + P\_{17},$$

and

W = P<sup>4</sup> + P<sup>5</sup> + P<sup>6</sup> + P<sup>7</sup> + P<sup>8</sup> −P<sup>6</sup> −P<sup>7</sup> −P<sup>8</sup> −P<sup>6</sup> P<sup>6</sup> + P<sup>10</sup> + P<sup>11</sup> 0 0 −P<sup>7</sup> 0 P<sup>7</sup> + P<sup>12</sup> + P<sup>13</sup> 0 −P<sup>8</sup> 0 0 P<sup>8</sup> + P<sup>14</sup> + P<sup>15</sup> ,

we can define three matrices H1, H<sup>2</sup> and H<sup>3</sup> as follows,

$$\begin{aligned} H\_1 &= \begin{pmatrix} \sqrt{P\_1 + P\_2} & -\sqrt{P\_3} & 0 \\ 0 & \sqrt{P\_3} & \sqrt{P\_9} \end{pmatrix}, \qquad H\_3 = \sqrt{P\_{16} + P\_{17}}, \\\\ H\_2 &= \begin{pmatrix} \sqrt{P\_4 + P\_5} & -\sqrt{P\_6} & -\sqrt{P\_7} & -\sqrt{P\_8} & 0 & 0 & 0 \\ 0 & \sqrt{P\_6} & 0 & 0 & \sqrt{P\_{10} + P\_{11}} & 0 & 0 \\ 0 & 0 & \sqrt{P\_7} & 0 & 0 & \sqrt{P\_{12} + P\_{13}} & 0 \\ 0 & 0 & 0 & \sqrt{P\_8} & 0 & 0 & \sqrt{P\_{14} + P\_{15}} \end{pmatrix}. \end{aligned}$$

Now if we consider

$$H = \begin{pmatrix} H\_1 & \mathbf{0} & \mathbf{0} \\ \mathbf{0} & H\_2 & \mathbf{0} \\ \mathbf{0} & \mathbf{0} & H\_3 \end{pmatrix},$$

then HH<sup>T</sup> = 6, where H is a 7 × 11 matrix. The Itô SDE model now has the form

$$\begin{cases} d\mathbf{Y}(t) = \mu dt + H d\mathbf{W}(t), \\ \mathbf{Y}(0) = \left( A(0), F(0), T\_{\text{in}}(0), T\_{\text{reg}}(0), T\_{\text{nor}}(0), T\_{\text{aut}}(0), I(0) \right)^T, \end{cases} \tag{4}$$

$$\begin{cases} \frac{\partial P(\mathbf{Y},t)}{\partial t} = -\sum\_{i=1}^{\mathcal{T}} \frac{\partial}{\partial \mathbf{y}\_i} \left[ \mu\_i P(\mathbf{Y},t) \right] + \frac{1}{2} \sum\_{i=1}^{\mathcal{T}} \sum\_{j=1}^{\mathcal{T}} \frac{\partial^2}{\partial \mathbf{y}\_i \partial \mathbf{y}\_j} \left[ \Sigma\_{ij} P(\mathbf{Y},t) \right], \\\\ P(\mathbf{Y},0) = \delta \boldsymbol{\tau} (\mathbf{Y} - \mathbf{Y}\_0). \end{cases}$$

By solving this PDE, one can find the probability density function of our model, but since this equation is high-dimensional and nonlinear, solving it analytically is impossible. Hence, we use another approach, a so-called system size expansion or van Kampen's -expansion (van Kampen, 1981), which is a method for constructing a continuous approximation to a discrete stochastic model (Stirk et al., 2010a,b), which allows one to study

stochastic fluctuations around deterministic attractors (Black et al., 2009).

#### 2.3. System Size Expansion

In order to apply the van Kampen's approach, we consider fluctuations within a systematic expansion of the master equation for a large system size . Specifically, we write each ni(t) as a deterministic part of order plus a fluctuation of order 1/2 as follows,

,

$$n\_i(t) = \mathfrak{Q}\chi\_i(t) + \mathfrak{Q}^{\dagger \natural}\zeta\_i(t), \qquad i = 1, \ldots, 7,\tag{5}$$

where xi(t) and ζi(t) are two continuous variables, and xi(t) = E[ni(t)]. The probability density P(**n**, t) satisfying the master Equation (3) is now represented by the probability density 5(ζ , t), i.e., 5(ζ , t) = P(**n**, t) = P **x** + 1/2 ζ , t , which implies

$$\frac{dP(\mathbf{n},t)}{dt} = \frac{\partial \Pi}{\partial t} - \sum\_{i=1}^{7} \Omega^{\mathbb{I}\_{\tilde{1}}} \frac{d\mathbf{x}\_i}{dt} \frac{\partial \Pi}{\partial \boldsymbol{\xi}\_i}.\tag{6}$$

To expand the master equation (3) in a power series in <sup>−</sup> <sup>1</sup>/<sup>2</sup> , we use the following expansion for the step operators

$$
\kappa\_i^{\pm} = 1 \pm \Omega^{-1/\varepsilon} \frac{\partial}{\partial \xi\_i} + \frac{1}{2} \Omega^{-1} \frac{\partial^2}{\partial \xi\_i^2} \pm \dotsb \,. \tag{7}
$$

Substituting expressions (6, 7) into the master equation (see Supplementary Material for details) and collecting terms of order 1/2 yields the following deterministic model for macroscopic behavior

$$\begin{aligned} \frac{dx\_1}{dt} &= b\_1 \mathbf{x}\_1 + \widetilde{b}\_2 \mathbf{x}\_1^2 - d\_1 \mathbf{x}\_1 - \widetilde{d}\_2 \mathbf{x}\_1^2 - \widetilde{\beta} \mathbf{x}\_1 \mathbf{x}\_2 - \widetilde{\mu}\_a \mathbf{x}\_1 \mathbf{x}\_6, \\ \frac{dx\_2}{dt} &= \widetilde{\beta} \mathbf{x}\_1 \mathbf{x}\_2 - d\_F \mathbf{x}\_2 - \widetilde{\mu}\_F \mathbf{x}\_2 \mathbf{x}\_5 - \widetilde{\mu}\_a \mathbf{x}\_2 \mathbf{x}\_6, \\ \frac{dx\_3}{dt} &= \widetilde{\lambda}\_{in} - d\_{in} \mathbf{x}\_3 - \widetilde{\alpha} \mathbf{x}\_2 \mathbf{x}\_3, \\ \frac{dx\_4}{dt} &= \widetilde{\lambda}\_r - d\_r \mathbf{x}\_4 + p\_1 \widetilde{\alpha} \mathbf{x}\_2 \mathbf{x}\_3 + \widetilde{\rho}\_1 \mathbf{x}\_4 \mathbf{x}\_7, \\ \frac{dx\_5}{dt} &= p\_2 \widetilde{\alpha} \mathbf{x}\_2 \mathbf{x}\_3 - d\_n \mathbf{x}\_5 + \widetilde{\rho}\_2 \mathbf{x}\_5 \mathbf{x}\_7, \\ \frac{d\mathbf{x}\_6}{dt} &= (1 - p\_1 - p\_2) \widetilde{\alpha} \mathbf{x}\_2 \mathbf{x}\_3 - d\_a \mathbf{x}\_6 - \widetilde{\delta} \mathbf{x}\_4 \mathbf{x}\_6 + \widetilde{\rho}\_3 \mathbf{x}\_6 \mathbf{x}\_7, \\ \frac{d\mathbf{x}\_7}{dt} &= \sigma\_1 \mathbf{x}\_5 + \sigma\_2 \mathbf{x}\_6 - d\_t \mathbf{x}\_7, \end{aligned} \tag{8}$$

where

$$\begin{aligned} b\_{2} &= \frac{\widetilde{b}\_{2}}{\Omega}, \quad d\_{2} = \frac{\widetilde{d}\_{2}}{\Omega}, \quad \beta = \frac{\widetilde{\beta}}{\Omega}, \quad \mu\_{a} = \frac{\widetilde{\mu}\_{a}}{\Omega}, \\\ \mu\_{F} &= \frac{\widetilde{\mu}\_{F}}{\Omega}, \quad \alpha = \frac{\widetilde{\alpha}}{\Omega}, \quad \delta = \frac{\widetilde{\delta}}{\Omega}, \\\ \rho\_{i} &= \frac{\widetilde{\rho}\_{i}}{\Omega}, \quad i = 1, 2, 3, \quad \lambda\_{in} = \widetilde{\lambda}\_{in}\Omega, \quad \lambda\_{r} = \widetilde{\lambda}\_{r}\Omega. \end{aligned}$$

Model (8) has been analyzed in Fatehi et al. (unpublished), and it can have at most four biologically feasible steady states. The first one, a disease-free steady state, is given by

$$S\_1^\* = \left(\frac{b\_1 - d\_1}{\widetilde{d\_2} - \widetilde{b\_2}}, 0, \frac{\widetilde{\lambda}\_{in}}{d\_{in}}, \frac{\widetilde{\lambda}\_r}{d\_r}, 0, 0, 0\right),$$

and it is stable if <sup>d</sup><sup>F</sup> <sup>&</sup>gt; <sup>β</sup>e. The second and third steady states can be found as

$$S\_2^\* = \left(0, 0, \frac{\widetilde{\lambda}\_{in}}{d\_{in}}, \varkappa\_4^\*, 0, \frac{d\_i\left(d\_a + \widetilde{\delta}\varkappa\_4^\*\right)}{\widetilde{\rho}\_3\sigma\_2}, \frac{d\_a + \widetilde{\delta}\varkappa\_4^\*}{\widetilde{\rho}\_3}\right), 0$$

and

$$S\_3^\* = \left(\frac{\widetilde{\rho}\_3 \sigma\_2 (b\_1 - d\_1) - \widetilde{\mu}\_a d\_i \left(d\_a + \widetilde{\delta} \chi\_4^\*\right)}{\widetilde{\rho}\_3 \sigma\_2 \left(\widetilde{d}\_2 - \widetilde{b}\_2\right)},$$

$$0, \frac{\widetilde{\lambda}\_{\dot{m}}}{d\_{\dot{m}}}, \chi\_4^\*, 0, \frac{d\_i \left(d\_a + \widetilde{\delta} \chi\_4^\*\right)}{\widetilde{\rho}\_3 \sigma\_2}, \frac{d\_a + \widetilde{\delta} \chi\_4^\*}{\widetilde{\rho}\_3}\right).$$

where x ∗ 4 satisfies the following quadratic equation

$$\left(\widetilde{\rho}\_1 \widetilde{\delta}\left(\mathbf{x}\_4^\*\right)^2 + \left(\widetilde{\rho}\_1 d\_a - \widetilde{\rho}\_3 d\_r\right) \mathbf{x}\_4^\* + \widetilde{\rho}\_3 \widetilde{\lambda}\_r = \mathbf{0}.\tag{9}$$

These steady states are stable, provided

$$\begin{aligned} \frac{\sigma\_2}{\widetilde{\mu}\_a d\_i} K &< \frac{d\_a + \widetilde{\delta} \boldsymbol{\chi}\_4^\*}{\widetilde{\rho}\_3} < \frac{d\_n}{\widetilde{\rho}\_2}, \quad \widetilde{\delta} \widetilde{\rho}\_1 (\boldsymbol{\chi}\_4^\*)^2 > \widetilde{\lambda}\_r \widetilde{\rho}\_3, \\\\ \widetilde{\rho}\_3 \widetilde{\lambda}\_r^2 &+ \widetilde{\rho}\_3 d\_i \widetilde{\lambda}\_r \boldsymbol{\chi}\_4^\* - \widetilde{\rho}\_3 d\_i d\_a (\boldsymbol{\chi}\_4^\*)^2 - \widetilde{\delta} (\widetilde{\rho}\_1 d\_a + \widetilde{\rho}\_3 d\_i) (\boldsymbol{\chi}\_4^\*)^3 \\\\ - \widetilde{\rho}\_1 \widetilde{\delta}^2 (\boldsymbol{\chi}\_4^\*)^4 &> 0, \end{aligned}$$

where K = 1 for S ∗ 2 , and K = <sup>β</sup>e<sup>−</sup> <sup>d</sup><sup>F</sup> / <sup>1</sup> <sup>+</sup> <sup>β</sup><sup>e</sup> for S ∗ 3 . Biologically, the steady state S ∗ 2 represents the death of organ cells, while S ∗ 3 corresponds to an autoimmune regime.

The last steady state S ∗ 4 has all of its components positive and corresponds to the state of chronic infection.

At the next order, stochastic fluctuations are determined by linear stochastic processes, hence, this is known as a linear noise approximation (van Kampen, 1981; Wallace et al., 2012). The dynamics of these fluctuations is described by the following linear Fokker-Planck equation

$$\frac{\partial \Pi(\boldsymbol{\xi},t)}{\partial t} = -\sum\_{i,j} A\_{ij} \frac{\partial}{\partial \boldsymbol{\xi}\_i} (\boldsymbol{\xi}\_j \boldsymbol{\Pi}) + \frac{1}{2} \sum\_{i,j} B\_{ij} \frac{\partial^2 \boldsymbol{\Pi}}{\partial \boldsymbol{\xi}\_i \partial \boldsymbol{\xi}\_j},\tag{10}$$

where A is the Jacobian matrix of system (8)

A = <sup>b</sup><sup>1</sup> <sup>+</sup> <sup>2</sup>eb2x<sup>1</sup> <sup>−</sup> <sup>d</sup><sup>1</sup> <sup>−</sup> <sup>2</sup>ed2x<sup>1</sup> <sup>−</sup> <sup>e</sup>µax<sup>6</sup> <sup>−</sup> <sup>β</sup>ex<sup>2</sup> <sup>−</sup>βex<sup>1</sup> <sup>0</sup> 0 0 <sup>−</sup>eµax<sup>1</sup> <sup>0</sup> <sup>β</sup>ex<sup>2</sup> <sup>β</sup>ex<sup>1</sup> <sup>−</sup> <sup>d</sup><sup>F</sup> <sup>−</sup> <sup>e</sup>µFx<sup>5</sup> <sup>−</sup> <sup>e</sup>µax<sup>6</sup> <sup>0</sup> <sup>0</sup> <sup>−</sup>eµFx<sup>2</sup> <sup>−</sup>eµax<sup>2</sup> <sup>0</sup> <sup>0</sup> <sup>−</sup>eαx<sup>3</sup> <sup>−</sup>din <sup>−</sup> <sup>e</sup>αx<sup>2</sup> 0 0 0 0 <sup>0</sup> <sup>p</sup>1eαx<sup>3</sup> <sup>p</sup>1eαx<sup>2</sup> <sup>ρ</sup>e1x<sup>7</sup> <sup>−</sup> <sup>d</sup><sup>r</sup> <sup>0</sup> <sup>0</sup> <sup>ρ</sup>e1x<sup>4</sup> <sup>0</sup> <sup>p</sup>2eαx<sup>3</sup> <sup>p</sup>2eαx<sup>2</sup> <sup>0</sup> <sup>ρ</sup>e2x<sup>7</sup> <sup>−</sup> <sup>d</sup><sup>n</sup> <sup>0</sup> <sup>ρ</sup>e2x<sup>5</sup> <sup>0</sup> (1 <sup>−</sup> <sup>p</sup><sup>1</sup> <sup>−</sup> <sup>p</sup>2)eαx<sup>3</sup> (1 <sup>−</sup> <sup>p</sup><sup>1</sup> <sup>−</sup> <sup>p</sup>2)eαx<sup>2</sup> <sup>−</sup>eδx<sup>6</sup> <sup>0</sup> <sup>ρ</sup>e3x<sup>7</sup> <sup>−</sup> <sup>d</sup><sup>a</sup> <sup>−</sup>eδx<sup>4</sup> <sup>ρ</sup>e3x<sup>6</sup> 0 0 0 0 σ<sup>1</sup> σ<sup>2</sup> −d<sup>i</sup> ,

and B is a 7 × 7 symmetric matrix given by

Bij = <sup>b</sup>1x<sup>1</sup> <sup>+</sup>eb2<sup>x</sup> 2 <sup>1</sup> <sup>+</sup> <sup>d</sup>1x<sup>1</sup> <sup>+</sup>ed2<sup>x</sup> 2 <sup>1</sup> <sup>+</sup> <sup>β</sup>ex1x<sup>2</sup> <sup>+</sup> <sup>e</sup>µax1x6, if (i, <sup>j</sup>) <sup>=</sup> (1, 1), <sup>β</sup>ex1x<sup>2</sup> <sup>+</sup> <sup>d</sup>Fx<sup>2</sup> <sup>+</sup> <sup>e</sup>µFx2x<sup>5</sup> <sup>+</sup> <sup>e</sup>µax2x6, if (i, <sup>j</sup>) <sup>=</sup> (2, 2), <sup>e</sup>λin <sup>+</sup> <sup>d</sup>inx<sup>3</sup> <sup>+</sup> <sup>e</sup>αx2x3, if (i, <sup>j</sup>) <sup>=</sup> (3, 3), <sup>e</sup>λ<sup>r</sup> <sup>+</sup> <sup>d</sup>rx<sup>4</sup> <sup>+</sup> <sup>p</sup>1eαx2x<sup>3</sup> <sup>+</sup> <sup>ρ</sup>e1x4x7, if (i, <sup>j</sup>) <sup>=</sup> (4, 4), <sup>p</sup>2eαx2x<sup>3</sup> <sup>+</sup> <sup>d</sup>nx<sup>5</sup> <sup>+</sup> <sup>ρ</sup>e2x5x7, if (i, <sup>j</sup>) <sup>=</sup> (5, 5), (1 <sup>−</sup> <sup>p</sup><sup>1</sup> <sup>−</sup> <sup>p</sup>2)eαx2x<sup>3</sup> <sup>+</sup> <sup>d</sup>ax<sup>6</sup> <sup>+</sup>eδx4x<sup>6</sup> <sup>+</sup> <sup>ρ</sup>e3x6x7, if (i, <sup>j</sup>) <sup>=</sup> (6, 6), σ1x<sup>5</sup> + σ2x<sup>6</sup> + dix7, if (i, j) = (7, 7), <sup>−</sup>βex1x2, if (i, <sup>j</sup>) <sup>=</sup> (1, 2) or (2, 1), <sup>−</sup>p1eαx2x3, if (i, <sup>j</sup>) <sup>=</sup> (3, 4) or (4, 3), <sup>−</sup>p2eαx2x3, if (i, <sup>j</sup>) <sup>=</sup> (3, 5) or (5, 3), <sup>−</sup>(1 <sup>−</sup> <sup>p</sup><sup>1</sup> <sup>−</sup> <sup>p</sup>2)eαx2x3, if (i, <sup>j</sup>) <sup>=</sup> (3, 6) or (6, 3), 0, otherwise.

Since the Fokker-Planck Equation (10) is linear, the probability density 5(ζ , t) is Gaussian, and hence, just the first two moments are enough to characterize it (Hayot and Jayaprakash, 2004; Pahle et al., 2012). Due to the way the system size expansion was introduced in (Equation 5), the mean values of fluctuations for all variables are zero, i.e., hζi(t)i = 0 for all 1 ≤ i ≤ 7, while the covariance matrix 4 with 4ij = hζi(t)ζj(t)i − hζi(t)ihζj(t)i = hζi(t)ζj(t)i satisfies the following equation (van Kampen, 1981; Pahle et al., 2012)

$$
\partial\_t \Xi = A \Xi + \Xi A^T + B,\tag{11}
$$

where A T is the transpose of A.

We are mainly interested in the dynamics of fluctuations when the oscillations of the deterministic model have died out, and the system is in a stationary state, i.e., the fluctuations take place around the steady states (Black et al., 2009). If the model (8) tends to a steady state as t → ∞, then in the equation (10) one can substitute the values of x<sup>i</sup> 's with the corresponding constant components of that steady state to study the fluctuations around it, as described by the linear Fokker-Planck equation. At any steady state, the covariance matrix 4 is independent of time, and the fluctuations are described by a Gaussian distribution with the zero mean and the stationary covariance satisfying the equation

$$A\Xi + \Xi A^T + B = 0.$$

In order to be able to relate the results of this analysis to simulations, it is convenient to express the covariance matrix in terms of actual numbers of cells in each compartment, rather than deviations from stationary values. To this end, we instead use the covariance matrix C defined as Cij = h(n<sup>i</sup> − hnii)(n<sup>j</sup> − hnji)i, which, in light of the relation Cij = 4ij, satisfies the following Lyapunov equation (Pahle et al., 2012)

$$\text{AC} + \text{CA}^T + \text{\textdegree} \Omega\\B = \text{0.} \tag{12}$$

This equation can be solved numerically for each of the stable steady states to determine the variance of fluctuations around that steady state depending on system parameters.

#### 3. RESULTS

To simulate the dynamics of the model, we solve the system Equation (4) numerically using the Euler-Maruyama method with parameter values given in **Table 2**, and = 1000. The initial condition is chosen to be of the form

$$(\chi\_1(0), \chi\_2(0), \chi\_3(0), \chi\_4(0), \chi\_5(0), \chi\_6(0), \chi\_7(0))$$

$$= (18, 2, 7.2, 6.3, 0, 0, 0), \tag{13}$$

which corresponds to a small number of host cells being initially infected.

**Figure 2** shows the results of 20,000 simulations with the initial condition (13) and σ<sup>2</sup> = 1. In the deterministic model (8), for σ<sup>2</sup> = 1 both steady states S ∗ 1 (disease-free) and S ∗ 3 (autoimmune state) are stable, but with the initial condition (13) the system is in the basin of attraction of S ∗ 3 . In the stochastic model, the majority of trajectories also enter the attraction region of S ∗ 3 , but a small proportion of them went into the basin of attraction of S ∗ 1 . This figure illustrates a single stochastic path around S ∗ 1 , and a single stochastic path around S ∗ 3 , together with the deterministic trajectory. These individual solutions indicate that whilst deterministically, the system exhibits decaying oscillations around S ∗ 3 , the same behavior is observed in the stochastic simulations only upon taking an average of a very large number of simulations. At the same time, individual realizations exhibit sustained stochastic oscillations in a manner similar to that observed in models of stochastic amplification in epidemics (Alonso et al., 2007; Kuske et al., 2007). **Figure 2** also illustrates the size of areas of one standard deviation from the mean for trajectories in the basins of attraction S ∗ 1 and S ∗ 3 , in which individual stochastic trajectories may exhibit stochastic oscillations (Conway and Coombs, 2011; Reynolds et al., 2012).

**Figures 3A,B** show temporal evolution of the probability distribution in the case of bi-stability between the steady states S ∗ 1 and S ∗ 3 , as illustrated in **Figure 2**. They indicate that after some initial transient, the system reaches a stationary bimodal normal distribution. The width of the probability distribution around each stable steady state, as described by its variance or standard deviation, gives the size of fluctuations around

TABLE 2 | Table of parameters.


deterministic trajectory from (1), and the shaded areas indicate the regions of one standard deviation from the mean.

this steady state observed in individual stochastic realizations, as is shown in **Figure 2**. Similar behavior has been observed in stochastic realizations of other deterministic models with bi-stability (Earnest et al., 2013; Bruna et al., 2014; Hufton et al., 2016). For the parameter values given in **Table 2**, the deterministic system exhibits a bi-stability between S ∗ 1 and S ∗ 2 , and with the initial condition

$$(\varkappa\_1(0), \varkappa\_2(0), \varkappa\_3(0), \varkappa\_4(0), \varkappa\_5(0), \varkappa\_6(0), \varkappa\_7(0))$$

$$= (18, 9, 7.2, 6.3, 0, 0, 0), \tag{14}$$

it is in the basin of attraction of S ∗ 2 . Due to stochasticity, the stationary probability distribution in this case is also bimodal, with the majority of solutions being distributed around S ∗ 2 , and a very small number being centered around S ∗ 1 , as can be seen in **Figures 3C,D**. Increasing the system size is known to result in the bimodal distribution becoming unimodal due to the size of fluctuations scaling as −1/<sup>2</sup> , which results in a reduced variability in trajectories (Black and McKane, 2012; Hufton et al., 2016), and the same conclusion holds for the system (4).

To gain better insights into the role of initial conditions, in **Figure 4** we fix all parameter values, and vary initial numbers of infected cells and regulatory T cells. For the parameter combination illustrated in **Figure 4A**, the deterministic model exhibits a bi-stability between a stable disease-free steady state S ∗ 1 and a periodic oscillation around the state S ∗ 3 , which biologically corresponds to an autoimmune regime. In the deterministic case, the black boundary provides a clear separation of the basins of attraction of these two dynamical states, in a manner similar to that investigated recently in the context of within-cell dynamics of RNA interference (Neofytou et al., 2017). For stochastic simulations, the color indicates the probability of the solution going to a disease-free state S ∗ 1 , and it shows that even in the case where deterministically the system is in the basin of attraction of one of the states, there is a non-zero probability that it will actually end up at another state, with this probability varying smoothly across the deterministic basin boundary. This figure suggests that if the initial number of infected cells is sufficiently small, or if the number of regulatory T cells is sufficiently large, the system tends to clear the infection and approach the disease-free state. On the contrary, for higher numbers of infected cells and lower numbers of regulatory cells, autoimmune regime appears to be a more likely outcome. Qualitatively similar behavior is observed for another combination of parameters illustrated in **Figure 4B**, in which case the deterministic system has a bi-stability between a disease-free steady state S ∗ 1 , and a state S ∗ <sup>2</sup> which represents the death of host cells.

In order to understand how biological parameters affect the size of fluctuations around steady states, in **Figure 5** we explore several parameter planes by first identifying parameter regions where the deterministic system has a stable steady state S ∗ 3 , and then for each combination of parameters inside these regions, we use the Bartels-Stewart method (Bartels and Stewart, 1972; Hammarling, 1982) to numerically solve the Lyapunov equation (12) and compute the variance in the number of regulatory T cell when the deterministic model is at the steady state S ∗ 3 . The value of variance gives the square of the magnitude of oscillations observed in individual stochastic realizations. One should note that getting closer to the deterministic boundary of stability of S ∗ 3 increases the stochastic variance of fluctuations around this steady state. The reason for this is that closer parameters are to the deterministic stability boundary, the less stable is the steady state, hence the larger is the amplitude of stochastic oscillations around it. Moreover, the variance increases with the rate of production of IL-2 by autoreactive T cells and the rate at which regulatory T cells suppress autoreactive T cells; it decreases with the higher rate of production of regulatory T cells, and it appears to not depend on the rate at which autoreactive T cells destroy infected cells, or on the infection rate.

#### 4. DISCUSSION

In this paper we have analyzed stochastic aspects of immune response against a viral infection with account for the

stationary joint probability histograms.

populations of T cells with different activation thresholds, as well as cytokines mediating T cell activity. The CTMC model has provided an exact master equation, for which we applied a van Kampen's expansions to derive a linear Fokker-Planck equation that characterizes fluctuations around the deterministic solutions. We have also explored actual stochastic trajectories of the system by deriving an SDE model and solving it numerically.

One biologically important aspect we have looked at is the influence of stochasticity on the dynamics of the system in the case where deterministically it exhibits a bi-stability between either two steady states, or a steady state and a periodic solution. In such a situation, bi-stability in the deterministic version of the model translates in the stochastic case into a stationary bimodal distribution for the probability density. To obtain further insights into details of how stochasticity affects bi-stability, we have investigated how for the fixed parameter values time evolution of the system changes depending on the initial numbers of the regulatory T cells and infected cells.

Our analysis reinforces the need to distinguish mean dynamics from individuals realizations: where in the deterministic case the system can approach a stable steady state (which represents mean behavior of a very large number of simulations), individual realizations can exhibit sustained stochastic oscillations around that steady state, as we have seen in numerical simulations. Since in the clinical or laboratory setting one is usually dealing with single measurements of some specific biological quantities rather than their averaged values, the stochastic oscillations exhibited by our model may quite well explain observed variability in the measured levels of infection or T cell populations. To better understand the magnitude of stochastic fluctuations around the deterministic steady states, we have solved the Lyapunov equation, which has provided us with a quantitative information on the dependence of variance of fluctuations on system parameters.

There are several directions in which the work presented in this paper can be extended. In terms of fundamental

FIGURE 4 | Probability of solution entering and staying in the basin of attraction of the disease-free steady state S \* 1 in the bi-stability regime with A(0) = 18, 000 and <sup>T</sup>in(0) <sup>=</sup> 7, 200. Black curves are the boundaries between different basins of attraction in the deterministic model. (A) With parameter values from Table 2, <sup>e</sup>λ<sup>r</sup> <sup>=</sup> <sup>45</sup> and <sup>e</sup>µ<sup>a</sup> <sup>=</sup> <sup>10</sup>/9, in the region below the black curve, the deterministic model exhibits a periodic solution around <sup>S</sup> \* 3 , and above this curve is the deterministic basin of attraction of S \* 1 . (B) With parameter values from Table 2, area below the black curve is the basin of attraction of S \* 2 , and above it is again the basin of attraction of S \* 1 .

immunology, the model can be made more realistic by including additional effects, such as the control of IL-2 secretion by regulatory T cells (Burroughs et al., 2006), or the memory T cells (Antia et al., 2005; Skapenko et al., 2005). Whilst we have used numerical simulations to compute the probability of attraction to a given steady state in the case of bi-stability, one could approach the same problem theoretically from the perspective of computing extinction probability within the framework of the CTMC model (Yuan and Allen, 2011; Mandal et al., 2014). The van Kampen's system size expansion could yield an expression for the power spectrum, which allows one to compute the peak frequency and amplification (McKane and Newman, 2005; Alonso et al., 2007; Black et al., 2009; Black and McKane, 2010). From a practical perspective, future work could focus on validating theoretical results presented in this paper using experimental measurements of the progress of autoimmune disease in animal hosts, with experimental autoimmune uveoretinitis (EAU), an autoimmune inflammation in the eyes, being one interesting possibility. In one such recent experiment, all animals were genetically identical C57BL/6 mice, but once the EAU was induced in them through inoculation, the autoimmune disease then progressed at slightly different rates (Boldison et al., 2015; Boldison and Nicholson, unpublished) and the measured variability in the numbers of infected cells and T cell responses could be compared to theoretical estimates of the variance as

### REFERENCES


predicted by our model. From a clinical perspective, comparison of variance in the measured populations of different cells with the model conclusions will facilitate an efficient parameter identification and provide a set of prognostic criteria for the progress of autoimmunity, which can be used for risk stratification and assessment of patients with autoimmune disease.

#### AUTHOR CONTRIBUTIONS

YK and KB designed the model; FF, SK, and AR performed the analysis and simulations, and produced the figures. All authors drafted and edited the manuscript.

### FUNDING

FF has been sponsored by the Chancellor's Award from Sussex University. AR acknowledges funding for her PhD studies from the Engineering and Physical Sciences Research Council (EPSRC) through iCASE Award EP/N509784/1.

#### SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fphys. 2018.00045/full#supplementary-material


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Fatehi, Kyrychko, Ross, Kyrychko and Blyuss. This is an openaccess article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Short-Term Dosage Regimen for Stimulation-Induced Long-Lasting Desynchronization

#### Thanos Manos 1,2 \*, Magteld Zeitler <sup>1</sup> and Peter A. Tass <sup>3</sup>

1 Institute of Neuroscience and Medicine (INM-7), Research Centre Jülich, Jülich, Germany, <sup>2</sup> Institute of Systems Neuroscience, Medical Faculty, Heinrich Heine University Düsseldorf, Düsseldorf, Germany, <sup>3</sup> Department of Neurosurgery, Stanford University, Stanford, CA, United States

In this paper, we computationally generate hypotheses for dose-finding studies in the context of desynchronizing neuromodulation techniques. Abnormally strong neuronal synchronization is a hallmark of several brain disorders. Coordinated Reset (CR) stimulation is a spatio-temporally patterned stimulation technique that specifically aims at disrupting abnormal neuronal synchrony. In networks with spike-timing-dependent plasticity CR stimulation may ultimately cause an anti-kindling, i.e., an unlearning of abnormal synaptic connectivity and neuronal synchrony. This long-lasting desynchronization was theoretically predicted and verified in several pre-clinical and clinical studies. We have shown that CR stimulation with rapidly varying sequences (RVS) robustly induces an anti-kindling at low intensities e.g., if the CR stimulation frequency (i.e., stimulus pattern repetition rate) is in the range of the frequency of the neuronal oscillation. In contrast, CR stimulation with slowly varying sequences (SVS) turned out to induce an anti-kindling more strongly, but less robustly with respect to variations of the CR stimulation frequency. Motivated by clinical constraints and inspired by the spacing principle of learning theory, in this computational study we propose a short-term dosage regimen that enables a robust anti-kindling effect of both RVS and SVS CR stimulation, also for those parameter values where RVS and SVS CR stimulation previously turned out to be ineffective. Intriguingly, for the vast majority of parameter values tested, spaced multishot CR stimulation with demand-controlled variation of stimulation frequency and intensity caused a robust and pronounced anti-kindling. In contrast, spaced CR stimulation with fixed stimulation parameters as well as singleshot CR stimulation of equal integral duration failed to improve the stimulation outcome. In the model network under consideration, our short-term dosage regimen enables to robustly induce long-term desynchronization at comparably short stimulation duration and low integral stimulation duration. Currently, clinical proof of concept is available for deep brain CR stimulation for Parkinson's therapy and acoustic CR stimulation for tinnitus therapy. Promising first in human data is available for vibrotactile CR stimulation for Parkinson's treatment. For the clinical development of these treatments it is mandatory to perform dose-finding studies to reveal optimal stimulation parameters and dosage regimens. Our findings can straightforwardly be tested in human dose-finding studies.

Keywords: coordinated reset, desynchronization, spike time-dependent plasticity, anti-kindling, stimulation patterns, dosing

#### Edited by:

Alexey Zaikin, University College London, United Kingdom

#### Reviewed by:

Oleg Blyuss, Queen Mary University of London, United Kingdom Grigory Osipov, N. I. Lobachevsky State University of Nizhny Novgorod, Russia

> \*Correspondence: Thanos Manos t.manos@fz-juelich.de

#### Specialty section:

This article was submitted to Computational Physiology and Medicine, a section of the journal Frontiers in Physiology

Received: 16 October 2017 Accepted: 27 March 2018 Published: 12 April 2018

#### Citation:

Manos T, Zeitler M and Tass PA (2018) Short-Term Dosage Regimen for Stimulation-Induced Long-Lasting Desynchronization. Front. Physiol. 9:376. doi: 10.3389/fphys.2018.00376

## INTRODUCTION

To establish a pharmacological treatment for clinical use, in humans typically a 4-phase sequence of clinical trials is performed (Friedman et al., 2010). In pre-clinical studies pharmacokinetic, toxicity and efficacy are studied in non-human subjects. In first in human-studies (phase I) safety and tolerability of a drug are studied in healthy volunteers. Proof of concept studies (phase IIA) determine whether a drug can have any efficacy, whereas dose-finding studies (phase IIB) are performed to reveal optimum dose at which a drug has biological activity with minimal side-effects. Effectiveness and the clinical value of a new intervention are studied in a randomized controlled trial (phase III), compared with state of the art treatment, if available. Finally, post-marketing surveillance trials (phase IV) are performed to detect rare or long-term adverse effects within a much larger patient population and over longer time periods. There might also be combinations of different phases.

In principle, this 4-phase pattern is also valid for medical technology, e.g., neuromodulation technologies. However, if neuromodulation technologies aim at the control of complex dynamics of e.g., neural networks, different parameters and dosage regimens may have complex, non-linear and even counterintuitive effects, (see e.g., Gao et al., 2014; Popovych et al., 2015; Gates and Rocha, 2016; Zañudo et al., 2017; Zhang et al., 2017). This computational paper illustrates how computational modeling can be used to generate hypotheses for dose-finding studies. In general, performing dose-finding studies simply by trial and error may be impossible because of the substantial parameter space to be tested, with trial durations and related costs getting out of hands.

The development of proper dosage strategies and regimens enables favorable compromises between therapeutic efficacy and detrimental factors such as side-effects or treatment duration. This is relevant, e.g., for the development of pharmaceutical (Williams, 1992; Bertau et al., 2008; Peters, 2012; Dash et al., 2014) or radiation therapy (Symonds et al., 2012). Deep brain stimulation (DBS) is the standard treatment of medically refractory movement disorders (Benabid et al., 1991; Krack et al., 2003; Deuschl et al., 2006). The clinical (Temperli et al., 2003) and electrophysiological (Kühn et al., 2008; Bronte-Stewart et al., 2009) effects of standard high-frequency (HF) DBS occur only during stimulation and cease after stimulation offset.

Coordinated reset (CR) stimulation (Tass, 2003a,b) was computationally developed to specifically counteract abnormal neuronal synchrony by desynchronization. CR stimulation uses sequences of stimuli delivered to neuronal sub-populations engaged in abnormal neuronal synchronization (Tass, 2003a,b). As shown computationally, in neuronal populations with spiketiming-dependent plasticity (STDP) (Gerstner et al., 1996; Markram et al., 1997; Bi and Poo, 1998) CR stimulation may have long-lasting, sustained effects (Tass and Majtanik, 2006; Hauptmann and Tass, 2007; Popovych and Tass, 2012). This is because in the presence of STDP, CR stimulation reduces the rate of coincidences. Accordingly, the network may be shifted from an attractor with abnormal synaptic connectivity and abnormal neuronal synchrony to an attractor with weak connectivity and synchrony (Tass and Majtanik, 2006; Hauptmann and Tass, 2007; Popovych and Tass, 2012). This process was termed anti-kindling (Tass and Majtanik, 2006).

Abnormal neuronal synchronization has been shown to be associated with a number of brain diseases, for example, Parkinson's disease (PD) (Lenz et al., 1994; Nini et al., 1995; Hammond et al., 2007), tinnitus (Ochi and Eggermont, 1997; Llinás et al., 1999; Weisz et al., 2005; Eggermont and Tass, 2015), migraine (Angelini et al., 2004; Bjørk and Sand, 2008). In parkinsonian non-human primates it was shown that electrical CR stimulation of the subthalamic nucleus (STN) has sustained, long-lasting after-effects on motor function (Tass et al., 2012b; Wang et al., 2016). In contrast, long-lasting after-effects were not observed with standard HF DBS (Tass et al., 2012b; Wang et al., 2016). For instance, unilateral CR stimulation of the STN of parkinsonian MPTP monkeys, delivered for only 2 h per day during 5 consecutive days led to significant and sustained bilateral therapeutic after-effects for at least 30 days, whereas standard HF DBS had no after-effects (Tass et al., 2012b). By the same token, cumulative and lasting after-effects of electrical CR stimulation of the STN were also observed in PD patients (Adamchic et al., 2014).

HF DBS may not only cause side effects by electrical current spreading outside of the target region, but also by chronic stimulation of the target itself or by functional disconnection of the stimulated structure (Ferraye et al., 2008; Moreau et al., 2008; van Nuenen et al., 2008). Accordingly, it is key to reduce the integral stimulation current. Electrical CR stimulation of the STN employs significantly less current compared to HF DBS (Tass et al., 2012b; Adamchic et al., 2014; Wang et al., 2016). However, to further improve the CR approach, in a previous computational study the spacing principle (Ebbinghaus et al., 1913) was used to achieve an anti-kindling at subcritical intensities, i.e., particularly weak intensities rendering permanently delivered CR stimulation ineffective (Popovych et al., 2015). According to the spacing principle (Ebbinghaus et al., 1913), learning effects can be improved by repeated stimuli spaced by pauses as opposed to delivering a massed stimulus in a single long stimulation session. The spacing principle was investigated on different levels, ranging from behavioral and cognitive (Cepeda et al., 2006, 2009; Pavlik and Anderson, 2008; Xue et al., 2011; Kelley and Whatson, 2013) to neuroscientific and molecular (Itoh et al., 1995; Frey and Morris, 1997; Menzel et al., 2001; Scharf et al., 2002; Naqib et al., 2012). Computationally it was demonstrated that the spacing principle can also be applied to unlearn unwanted, upregulated synaptic connectivity at subcritical stimulation intensities (Popovych et al., 2015). In principle, the results were intriguing, but required rather long pauses and total stimulation durations (Popovych et al., 2015). Spaced CR stimulation at subcritical intensities might possibly be applied to CR DBS. However, for clinical applications, in particular, for non-invasive applications of CR stimulation (Popovych and Tass, 2012), such as acoustic CR stimulation for tinnitus (Tass et al., 2012a) or vibrotactile stimulation for PD (Tass, 2017; Syrkin-Nikolau et al., 2018), it is crucial to achieve therapeutic effects within a reasonable amount of time. Applications of non-invasive medtech devices typically rely on the patients' compliance and should favorably require short stimulation durations. Accordingly, we here set out to apply the spacing principle to CR stimulation at supercritical intensities, i.e., intensities that enable an anti-kindling for moderate stimulation duration and properly selected stimulation frequencies. The overall goal of this study is to design short-term dosage regimen that improve CR stimulation efficacy, while keeping the integral amount of stimulation as well as the overall duration of the protocols at comparably low levels.

In Manos et al. (in review) we studied the influence of the CR stimulation frequency and the intensity on the outcome of CR stimulation with Rapidly Varying Sequences (RVS) and Slowly Varying Sequences SVS (Zeitler and Tass, 2015). CR stimulation consists of sequences of stimuli delivered to each sub-population (Tass, 2003a,b). For RVS CR stimulation, the CR sequence is randomly varied from one CR stimulation period to another (Tass and Majtanik, 2006). Conversely, SVS CR stimulation is characterized by repeating a sequence for a number of times before randomly switching to the next sequence (Zeitler and Tass, 2015). In Manos et al. (in review) we demonstrated that the efficacy of singleshot CR stimulation with moderate stimulation duration depends on the stimulation parameters, in particular, on the intensity as well as the relationship between CR stimulation frequency and intrinsic firing rates. RVS CR stimulation turned out to induce pronounced long-lasting desynchronization, e.g., at weak intensities and CR stimulation frequencies in a certain range around the neurons' intrinsic firing frequencies. In contrast, SVS CR stimulation enabled even more pronounced anti-kindling, however, at the cost of a significantly stronger dependence of the stimulation outcome on the CR stimulation frequency.

Dosage regimen design is an integral part of pharmacokinetic methodology, aiming at an optimization of drug delivery and effects (Williams, 1992). By a similar token, we hypothesize that appropriate dosage regimens might further enhance the efficacy of RVS and SVS CR stimulation. To probe different dosage regimens, we here consider different stimulation singleshot and multishot CR stimulation protocols. Protocols A and B have identical integral stimulation duration, whereas Protocols C and D may require less stimulation.

### Protocol A: Spaced Multishot CR Stimulation With Fixed Stimulation Parameters

Instead of one singleshot CR stimulation we deliver the identical CR shot five times, where the duration of each single pause equals the duration of each identical singleshot. Intersecting singleshot stimuli by pauses to increase stimulation efficacy, resembles the so-called spacing principle, a learning-related mechanism that is well-established in psychology (Ebbinghaus et al., 1913), education (Kelley and Whatson, 2013), and neuroscience (Naqib et al., 2012). According to the spacing principle, learning effects can be enhanced by delivering a stimulus in a spaced manner, as opposed to administering a massed stimulus in a single long stimulation session. Computationally, it was shown that subcritical CR stimulation at subcritical (ineffective) intensities may become effective if intersected by rather long pauses and delivered sufficiently often, e.g., eight times (Popovych et al., 2015). However, shorter pauses were not sufficient (Popovych et al., 2015). As yet, spaced CR stimulation at supercritical intensities was not studied. Here, we focus on comparably short stimulation protocols. Accordingly, we use CR stimulation of sufficient intensity and deliver five single CR shots intersected by pauses. We consider a symmetric dosage regimen, with identical duration of single shots and pauses.

## Protocol B: Long Singleshot CR Stimulation With Fixed Stimulation Parameters

To assess the impact of the spacing principle, as a control condition we simply stimulate five times longer, without any pause and with stimulation parameters kept constant. Protocol B is shorter, but employs the same integral stimulation duration as Protocol A.

### Protocol C: Spaced Multishot CR Stimulation With Demand-Controlled Variation of the CR Stimulation Frequency and Intensity

As in Protocol A, we deliver spaced CR stimulation comprising five identical CR shots, intersected by pauses, where all shots and pauses are of equal duration. However, at the end of each CR shot we monitor the stimulation effect and perform a three-stage control: (i) If no pronounced desynchronization is achieved, the CR stimulation frequency of the subsequent CR shot is mildly varied by no more than ±3%. (ii) If an intermediate desynchronization is observed, the CR stimulation frequency remains unchanged and CR stimulation is continued during the subsequent shot. (iii) If a pronounced desynchronization is achieved, no CR stimulation is delivered during the subsequent shot. Note, for stage (i) we do not adapt the CR stimulation frequency to a measured quantity. We consider two different variation types employed for stage (i): with regular and with random variation of the CR stimulation frequency. Regular variation means to increase or decrease the CR stimulation frequency in little unit steps. In contrast, random variation stands for randomly picking the CR stimulation frequency from a restricted interval.

### Protocol D: Long Singleshot CR Stimulation With Demand-Controlled Variation of the Stimulation Frequency

To assess the specific pausing-related impact of the evolutionary spacing principle, as a direct control condition we perform Protocol C without pauses. To this end, we string five CR shots together, without pauses, and evaluate the stimulation effect at the end of each CR shot. If no pronounced desynchronization is achieved, the CR stimulation frequency is slightly varied by no more than ±3% for the subsequent CR shot. During each single CR shot stimulation parameters are kept constant. Only from one CR shot to the next the CR stimulation frequency can be varied. Overall, Protocol D is shorter than Protocol C, but uses the same integral stimulation duration as in Protocols A-C.

CR stimulation and, especially, SVS CR stimulation has pronounced periodic characteristics. Accordingly, the CR stimulation frequency turned out to be a sensitive parameter, in particular, for SVS CR stimulation (see Manos et al., in review). For this reason, for stage (i) of Protocol C and D we perform a demand-controlled variation of the CR stimulation frequency to prevent from, e.g., unfavorable resonances or phase locking dynamics. Note these demand-controlled changes of the CR stimulation frequency are mild and hardly change the networks' firing rates.

In this study, we test the performance of the different Protocols A-D by selecting unfavorable stimulation parameters, which render CR stimulation ineffective according to Manos et al. (in review). By design, Protocols C and D work well for all parameter pairs (K, Ts) related to effective singleshot CR stimulation. In that case, CR stimulation actually ceases due to lack of demand. Note, in all four stimulation protocols we keep the stimulation intensity fixed. Only Protocols C and D require feedback of the stimulation outcome.

This paper is organized as follows: in the Materials and Methods section we briefly describe the computational model, the neural network (and its initialization), the synaptic plasticity rule, the CR stimulation, the analysis methods used throughout the paper as well as the summary of the CR frequency and intensity global trends which were thoroughly studied in Manos et al. (in review). In the Results section, we present all the different Protocols in detail and our main findings regarding their comparison. Finally, in the Discussion section, we discuss our findings and set this work in more general perspective, related to medical applications.

### MATERIALS AND METHODS

#### Model and Network Description

In this study we use the conductance-based Hodgkin-Huxley neuron model (Hodgkin and Huxley, 1952) for the description of an ensemble of spiking neurons. The set of equations and parameters read (see Hansel et al., 1993; Popovych and Tass, 2010):

$$C\frac{dV\_i}{dt} = \left[I\_i - \text{g}\_{\text{Na}}n\_i^3h\_i\left(V\_i - V\_{\text{Na}}\right) - \text{g}\_{\text{K}}n\_i^4\left(V\_i - V\_{\text{K}}\right)\right]$$

$$-\text{g}\_l\left(V\_i - V\_l\right) + \text{S}\_i + F\_{i,\*} \tag{1a}$$

$$\frac{d\mathbf{x}\_i}{dt} = \alpha\_{\mathbf{x}} \left( V\_i \right) \left( 1 - \mathbf{x}\_i \right) - \beta\_{\mathbf{x}} \left( V\_i \right) \mathbf{x}\_i. \tag{1b}$$

Variable V<sup>i</sup> denotes the membrane potential of neuron i (i = 1, . . . , N), while the variable x stands for the three gating variables m, n and h. The α<sup>x</sup> and β<sup>x</sup> variables are described in the standard model definition (see Manos et al., in review). The network consists of N = 200 neurons placed on a ring. The constant sodium, potassium and leak reversal potentials and the maximum conductance per unit area are (VNa, gNa) = (50 mV, 120 mS/cm<sup>2</sup> ), (VK, gK) = (−77 mV, 36 mS/cm<sup>2</sup> ) and (V<sup>l</sup> , gl ) = (−54.4 mV, 0.3 mS/cm<sup>2</sup> ), while the constant membrane capacitance is C = 1 µF/cm. I<sup>i</sup> denotes the constant depolarizing current injected into neuron i, regulating the intrinsic firing rate of the uncoupled neurons. For the realization of different initial networks, we used the same random initial conditions drawn from uniform distributions as used in Manos et al. (in review), i.e., I<sup>i</sup> ∈ [I<sup>0</sup> − σ<sup>I</sup> , I<sup>0</sup> + σ<sup>l</sup> ] (I<sup>0</sup> = 11.0 µA/cm<sup>2</sup> and σ<sup>l</sup> = 0.45 µA/cm<sup>2</sup> ), h<sup>i</sup> , m<sup>i</sup> , n<sup>i</sup> ∈ [0, 1 ] and V<sup>i</sup> ∈ [−65, 5 ] mV. In addition, in order to model variations of the model parameters (see Discussion and Supplementary Material), we add a sinusoidal external current input of the form Ivar = A·sin(2π ·f ·t) to the right-hand side of Equation 1a, where f and A are the frequency and the amplitude of the signal respectively.

The initial values of the neural synaptic weights cij are picked from a normal distribution N(µ<sup>c</sup> = 0.5 mS/cm<sup>2</sup> , σ<sup>c</sup> = 0.01 mS/cm<sup>2</sup> ) as in Manos et al. (in review, see Popovych and Tass, 2012; Zeitler and Tass, 2015 for details). Si(t) denotes the internal synaptic input within the network to neuron i. The neurons interact via excitatory and inhibitory chemical synapses si , by means of the post-synaptic potential (PSP) s<sup>i</sup> which is triggered by a spike of neuron i (Gerstner et al., 1996; Izhikevich, 2010) and modeled using an additional equation (see Golomb and Rinzel, 1993; Terman et al., 2002):

$$\frac{ds\_{\dot{j}}}{dt} = \frac{0.5(1 - s\_{\dot{j}})}{1 + \exp\left[-\left(V\_{\dot{j}} + 5\right)/12\right]} - 2s\_{\dot{j}}.\tag{1c}$$

Initially we draw s<sup>i</sup> ∈ [0, 1 ] (randomly from a uniform distribution). The coupling term S<sup>i</sup> from Equation 1a (see Popovych and Tass, 2012) contains a weighted ensemble average of all post-synaptic currents received by neuron i from the other neurons in the network: S<sup>i</sup> = N <sup>−</sup><sup>1</sup> P<sup>N</sup> j=1 Vr,<sup>j</sup> − V<sup>i</sup> cij <sup>M</sup>ij sj , where Vr,<sup>j</sup> is the reversal potential of the synaptic coupling (20 mV for excitatory and −40 mV for inhibitory coupling), and cij is the synaptic coupling strength from neuron j to neuron i. There are no neuronal self-connections within the network (cii = 0 mS/cm<sup>2</sup> ). The variable:

$$M\_{\vec{ij}} = \left(1 - d\_{\vec{ij}}^2 / \sigma\_1^2\right) \exp\left(-d\_{\vec{ij}}^2 / \left(2\sigma\_2^2\right)\right) \tag{2}$$

describes the spatial profile of coupling between neurons i and j and is of a Mexican hat-type (Wilson and Cowan, 1973; Dominguez et al., 2006; de la Rocha et al., 2008) with strong short-range excitatory Mij > 0 and weak long-range inhibitory interactions Mij < 0 . Here dij = d i − j is the distance between neurons i and j, while d = d0 N−1 determines the distance on the lattice between two neighboring neurons within the ensemble. d<sup>0</sup> is the length of the neuronal chain (d<sup>0</sup> = 10). σ<sup>1</sup> = 3.5, and σ<sup>2</sup> = 2.0. In order to limit boundary effects, we consider that the neurons are distributed in such a way that the distance dij is taken as: d · min( i − j , N − i − j ) for <sup>i</sup>, <sup>j</sup> <sup>&</sup>gt; <sup>N</sup>/2.

#### Spike-Timing-Dependent Plasticity

The synaptic weights cij are dynamical variables that depend on the time difference, 1tij = t<sup>i</sup> − t<sup>j</sup> , between the onset of the spikes of the post-synaptic neuron i and the pre-synaptic neuron j, denoted by t<sup>i</sup> and t<sup>j</sup> , according to Bi and Poo (1998) and Popovych and Tass (2012):

$$
\Delta \ c\_{\vec{\eta}} = \begin{cases}
\ \beta\_1 e^{\frac{-\Delta t\_{\vec{\eta}}}{\beta\_1}}, & \Delta t\_{\vec{\eta}} \ge 0 \\
\beta\_2 \frac{\Delta t\_{\vec{\eta}}}{\pi} e^{\frac{\Delta t\_{\vec{\eta}}}{\beta\_2}}, & \Delta t\_{\vec{\eta}} < 0
\end{cases},\tag{3}$$

with parameters β<sup>1</sup> = 1, β<sup>2</sup> = 16, γ<sup>1</sup> = 0.12, γ<sup>2</sup> = 0.15, τ = 14 ms and with learning rate δ = 0.002, while the values of cij are confinded to the interval [0, 1] mS/cm<sup>2</sup> for both excitatory and inhibitory synapses and, hence, remain bounded.

#### Coordinated Reset Stimulation

The term F<sup>i</sup> in Equation 1a represents the current induced in neuron i by the CR stimulation delivered at N<sup>s</sup> = 4 stimulations sites, equidistantly placed at the positions of neurons i = 25, 75, 125, 175 (Tass, 2003b). One stimulation site was active during Ts/N<sup>s</sup> , while the other stimulation sites were inactive during that time window. After this, another stimulation site was active during the next Ts/N<sup>s</sup> window. All N<sup>s</sup> stimulation sites were stimulated exactly once within one CR stimulation period of duration T<sup>s</sup> . The spatiotemporal activation pattern of stimulation sites is represented by the indicator functions ρ<sup>k</sup> (t) (kǫ {1, . . . , N}), taking the value 1 when the kth stimulation site is active at t and 0 else. The stimulation signals induced single brief excitatory post-synaptic currents. The evoked time-dependent normalized conductances of the post-synaptic membranes are represented by α-functions given in Popovych and Tass (2012) as Gstim (t) = t−t<sup>k</sup> τstim e −(t−t<sup>k</sup> )/τstim , <sup>t</sup><sup>k</sup> <sup>≤</sup> <sup>t</sup> <sup>≤</sup> <sup>t</sup>k+<sup>1</sup> . τstim = Ts 6Ns denotes the time-to-peak of Gstim, and t<sup>k</sup> is the onset of the kth activation of the stimulation site. Note, τstim determines the onset timing of each single signal as well as its duration. The spatial spread of the induced excitatory post-synaptic currents in the network is defined by the quadratic spatial decay profile D (i, xk) = 1 1+d 2 (i−x<sup>k</sup> ) 2 /σ2 d , a function of the difference between the index of neuron i and the index x<sup>k</sup> of the neuron at stimulation site k. d is the lattice distance between two neighboring neurons, and σ<sup>d</sup> = 0.8 the spatial decay rate of the stimulation current (see Popovych and Tass, 2012 for details). Thus, the total stimulation current from Equation 1 reads F<sup>i</sup> = - V<sup>r</sup> − Vi(t) · K PN<sup>s</sup> k=1 D (i, xk) ρ<sup>k</sup> (t)Gstim (t), where V<sup>r</sup> = 20 mV is the excitatory reverse potential, and K the stimulation intensity.

#### Macroscopic Measurements

We measure the strength of the coupling within the neuronal population at time t by calculating their total synaptic weight (averaged over the neuron population) Cav (t) = N <sup>−</sup><sup>2</sup> P i,j sgn Mij cij (t), where Mij is defined in Equation 2, sgn is the sign-function, while Cav is calculated by averaging over the last 100 · T<sup>s</sup> . The extent of in-phase synchronization within the network is assessed by the order parameter (Haken, 1983; Kuramoto, 2012) R (t) = N <sup>−</sup><sup>1</sup> P j e iϕj(t)  , where <sup>ϕ</sup><sup>j</sup> (t) <sup>=</sup> 2π(t−tj,m) (tj,m+1−tj,m) for tj,<sup>m</sup> ≤ t < tj,m+<sup>1</sup> is a linear approximation of the phase of neuron j between its mth and (m + 1)th spikes at spiking times tj,<sup>m</sup> and tj,m+1. R (t) = 1 for complete inphase synchronization, and R (t) = 0 in the absence of in-phase synchronization. Because of strong fluctuations of the order parameter, we calculate the moving average < R > over a time window of 400 · T<sup>s</sup> , to investigate the time evolution of the order parameter. Moreover, we use the quantity Rav , which is the order parameter R (t) averaged over the last 100·T<sup>s</sup> of a pause following a CR shot or of the end of the post-stim epoch. For the statistical description and analysis of the non-Gaussian distributed Rav data (n = 11 samples), we use boxplots (Tukey, 1977). Their Inter-Quartile Range measures the statistical dispersion around the median, which is defined as width of the middle 50% of the distribution and is represented by a box. It is also used to determine outliers in the data: an outlier falls more than 1.5 times IQR below the 25% quartile or more than 1.5 times IQR above the 75% quartile.

#### Dependence of CR Stimulation Outcome on CR Stimulation Frequency and Intensity

This section provides a short overview of the results of a study where CR stimulation frequency and intensity were varied in detail (Manos et al., in review). That study revealed the dependence of the outcome of RVS and SVS CR on the CR stimulation frequency and intensity and, in particular, possible limitations thereof, especially for SVS CR stimulation. Based on these limitations, the present study presents an approach that enables to overcome these issues.

In the present study, for each initial network condition and its corresponding parameters (simply denoted as network), we apply RVS and SVS CR stimulation with different realizations of the CR sequence orders per network. We start the simulations with an equilibration phase without STDP, which lasts for 2 s. From this point on, the network evolves in the presence of STDP, starting with a 60 s integration with STDP only (i.e., without stimulation), where a rewiring of the connections takes place, resulting in a strongly synchronized state with intrinsic firing rate fint ≈ 71.4 Hz (corresponding to a period of Tint = 14 ms). We then run four different CR stimulation protocols, resetting the starting time to t = 0 s. We use 3:2 ON-OFF CR stimulation, where three stimulation ON-cycles (with stimulation on) alternated with two OFF-cycles (without stimulation), with ON-/OFF-cycle duration of T<sup>s</sup> . 3:2 ON-OFF CR stimulation was used in a number of computational, pre-clinical and clinical studies, for details and motivation (see Manos et al., in review).

To study dosage regimens that potentially improve reliability and stimulation outcome of RVS and/or SVS stimulation, in the present study we focus on parameter ranges where RVS and/or SVS CR stimulation did not reliably elicit long-lasting desynchronization according to Manos et al. (in review). In Manos et al. (in review), we delivered CR single shots of 128 s duration followed by a 128 s CR-off period and varied the CR stimulation frequency and intensity over a wider range. In this way, we showed that RVS CR stimulation turned out to be more robust against variations of the stimulation frequency, while SVS CR stimulation can obtain stronger anti-kindling effects. This dependence on the CR stimulation intensity and frequency is summarized in **Figure 1**. **Figures 1A,E** show the boxplots for the time-averaged mean synaptic weights Cav (at

the end of the 128 s CR-off period) with values belonging to the same intensity value K for RVS and SVS CR stimulation, respectively, but CR stimulation frequency varying in the interval [25%f0, . . . , 175%f0], with f<sup>0</sup> as defined below. The motivation for restricting the CR stimulation intensity to the interval K ǫ [0.20, . . . , 0.50] is discussed in Manos et al. (in review). In a similar manner, **Figures 1B,F** show boxplots for the timeaveraged order parameter Rav (again at the end of the CR-off period) with values belonging to the same intensity value K for RVS and SVS CR stimulation respectively. **Figures 1C,G** depict the boxplots for the time-averaged mean synaptic weights Cav (at the end of the CR-off period) but now with values belonging to the same frequency ratio (fstim/f 0 ) · 100 for RVS and SVS CR stimulation, respectively, but intensity value K ∈ [0.20, . . . , 0.50]. The CR stimulation frequency fstim takes values in the interval [25%f0, . . . , 175%f0], where f<sup>0</sup> = 1/T<sup>0</sup> denotes the initial stimulation frequency. The choice of the frequency f<sup>0</sup> (or period T0) of the CR stimulation is made with respect of the intrinsic network's firing rate frequency (or period) fint(or Tint) and is meant to have a value close to that (in this case T<sup>0</sup> = 16 ms with f<sup>0</sup> = 62.5 Hz). More details can be found in Manos et al. (in review). **Figures 1D,H** show the boxplots for the time-averaged order parameter Rav (at the end of the CR-off period) with values belonging to the same frequency ratio (fstim/f 0 )· 100 for RVS and SVS CR stimulation, respectively.

#### RESULTS

#### Simulation Description

We investigate two singleshot and two multishot, spaced CR stimulation protocols (**Figure 2**). The multishot Protocols A and C consist of five single CR shots of 128 s duration, each followed by a pause of 128 s, respectively (**Figures 2A,C**). The CR singleshot Protocol B consists of a long singleshot of 5 × 128 s followed by a pause of 5 × 128 s (**Figures 2B**). The CR singleshot Protocol D consists of a long singleshot consisting of five single shots of 128 s duration, strung together without pauses in between, followed by a pause of 5 × 128 s (**Figures 2D**). The integral stimulation duration is identical for Protocols A and B. In Protocols A and B all stimulation parameters are kept constant. In contrast, in Protocol C at the end of each pause the amount of synchrony is evaluated in a time window of 100 stimulation periods length (**Figure 2**) and a three-stage control scheme is put in place: (i) If the amount of synchrony does not fall below a pre-defined threshold, the CR stimulation frequency is mildly varied. (ii) If the desynchronization effect is moderate, the CR stimulation frequency remains unchanged. (iii) If desynchronization is achieved, the stimulation intensity is set to zero for the subsequent shot. Analogously, in Protocol D at the end of each single shot the amount of synchrony is evaluated in a time window of 100 stimulation periods length (**Figure 2**) and the three-stage control scheme is executed. The difference between Protocol C and D is that the evaluation for the control intervention is performed in a pause subsequent to a single shot (Protocol C) as opposed to during a single shot (Protocol D).

For the stage (i) control, the variation of the CR stimulation frequency is not adapted to frequency characteristics of the neuronal network. Rather a minor variation of the CR stimulation frequency is performed to make a fresh start with the subsequent single CR shot. These minor changes of the CR stimulation frequency do not lead to changes of the neurons' intrinsic firing rates of more than ±3%.

Due to the stage (iii) control, the demand-controlled shutdown of CR stimulation, the maximum integral stimulation

duration of Protocol C and D can reach the level of Protocols A and B, but may well fall below. We use the order parameter to assess the amount of synchronization (see Materials and Methods).

#### Protocol A: Spaced Multishot CR Stimulation With Fixed Stimulation Parameters

For this stimulation protocol all stimulation parameters are kept constant (**Figure 2**). Accordingly, the CR stimulation period T<sup>s</sup> remains constant, too. We study the stimulation outcome of only five symmetrically spaced consecutive single CR shots. To this end, for both RVS CR and SVS CR stimulation we consider two unfavorable parameter pairs of fixed CR stimulation period and intensity, respectively. One example refers to cases where CR stimulation induces acute effects, but no long-lasting desynchronizing effects (Cases I and IV). The other example concerns the case where CR stimulation causes neither acute nor long-lasting desynchronizing effects in a reliable manner (Cases II and III).

#### **RVS CR stimulation: Case I:** (**K**, **Ts**) = (**0.30**, **11**)

At a stimulation duration of 128 s these parameters caused only an acute, but no long-lasting desynchronization in the majority of networks studied [Figure 4B of Manos et al. (in review), where T<sup>s</sup> = 11 ms corresponds to ∼127% of the intrinsic firing rate (or ∼91 Hz)]. Case II: (K, Ts) = (0.20, 28). In the

text).

majority of networks tested, these parameters did neither lead to acute nor long-lasting desynchronization after administration of a single CR shot [Figure 5B of Manos et al. (in review), where T<sup>s</sup> = 28 ms corresponds to ∼50% of the intrinsic firing rate (or ∼36 Hz)]. For both cases, we investigate the order parameter < R > averaged over a sliding window for 11 different networks (marked with different color/line types) (**Figures 2A,C**). Boxplots of the order parameter Rav averaged over a window of length 100 · T<sup>s</sup> at the end of each pause demonstrate the overall stimulation outcome for all tested 11 networks (**Figures 2B,D**).

#### **Case I:**

RVS CR stimulation induces a desynchronization during the CR shots (**Figure 3A**), but no reliable, long-lasting desynchronization in the subsequent pauses (**Figure 3B**). The spacing protocol with five identical RVS CR shots does not significantly improve the desynchronizing outcome of a single RVS CR shot. In fact, in the boxplots the large dispersion around the median value remains almost unchanged in the course of this protocol (**Figure 3B**). Case II: Neither during the RVS CR shots nor during the subsequent pauses a sufficient desynchronization is observed (**Figures 2C,D**). The spacing protocol does not cause an improvement of the stimulation outcome in this case, too (**Figure 3D**).

#### **SVS CR stimulation: Case III:** (**K**, **Ts**) = (**0.20**, **9**)

Single shot SVS CR stimulation with these parameters caused neither pronounced acute nor long-lasting desynchronization [Figure 6D of Manos et al. (in review), where T<sup>s</sup> = 9 ms corresponds to ∼156% of the intrinsic firing rate (or ∼111 Hz)]. Case IV: (K, Ts) = (0.20, 14). Single shot SVS CR stimulation with these parameters led to acute, but no longlasting desynchronization in the majority of networks tested [Figure 7B of Manos et al. (in review), where T<sup>s</sup> = 28 ms corresponds to ∼50% of the intrinsic firing rate (or ∼36 Hz)]. For both cases we performed the same analysis as shown in **Figure 3**.

#### **Case III:**

SVS CR stimulation neither induces a pronounced and reliable desynchronization during the CR shots (**Figure 4A**) nor during the pauses (**Figure 4B**). In fact, the dispersion around the median value is increased during the fourth and fifth pause (**Figure 4B**).

#### **Case IV:**

During the SVS CR shots a desynchronization occurs (**Figure 4C**). However, no reliable and pronounced desynchronization is observed during the pauses (**Figure 4D**).

In summary, for both RVS CR and SVS CR stimulation the spacing protocol with five consecutive CR shots does not cause an improvement of the long-lasting desynchronization (assessed after cessation of stimulation). We performed the same analysis

for a larger set of (K, Ts) pairs in the parameter plane analyzed in Manos et al. (in review), with K ranging from 0.2 to 0.3 (weak intensities) and T<sup>s</sup> ranging from 9 to 28 ms (around the intrinsic period). For all parameter pairs tested, the spacing Protocol A did not improve the long-term desynchronization effect.

#### Protocol B: Long Singleshot CR Stimulation With Fixed Stimulation Parameters

For this stimulation protocol all stimulation parameters are kept constant, too (**Figure 2**). Instead of five single CR shots of 128 s duration each (**Figures 1**–**3**), we deliver one fivefold longer singleshot of 5 × 128 s duration (**Figure 2**). For both RVS CR and SVS CR stimulation we consider the corresponding two unfavorable parameter pairs of fixed CR stimulation period and intensity already studied above (Cases I-IV). This is to study whether a fivefold prolongation of the stimulation duration leads to an improvement of the stimulation outcome.

#### **RVS CR stimulation:**

For comparison, we consider the cases studied above. Case I: (K, Ts) = (0.30, 11). Case II: (K, Ts) = (0.20, 28). We study the order parameter < R > averaged over a sliding window for 11 different networks (marked with different color/line types in **Figure 5A**). The overall stimulation outcome for all tested 11 networks is illustrated with boxplots of the order parameter Rav averaged over a window of length 100 · T<sup>s</sup> at the end of the post-stimulus epoch (**Figure 5B**).

#### **Case I:**

RVS CR stimulation induces a desynchronization during the long RVS CR singleshot (**Figure 5A**), but no reliable, longlasting desynchronization in the subsequent pauses (**Figure 5B**). The median of the order parameter of the long-term outcome hardly changes, but the dispersion around the median value is greater for the post-stim order parameter (as the IQR of the boxplots in **Figure 5B** show). Note, the overall longterm desynchronization for the long singleshot (**Figure 5B**) is more pronounced compared to the spaced RVS CR stimulation Protocol A (**Figure 3B**).

#### **Case II:**

Neither during the long RVS CR singleshot nor during the subsequent stimulation-free epoch a reliable and pronounced desynchronization is observed (**Figures 4C,D**). Interestingly, the network that undergoes an acute desynchronization during the singleshot relaxes back to a synchronized state (**Figure 5B**, green curve). Conversely, the only network that displays a long-term desynchronization does not undergo a pronounced desynchronization during the singleshot (**Figure 5B**, magenta curve).

#### **SVS CR stimulation:**

For comparison, we consider the time course of the timeaveraged order parameter < R > (**Figures 5A,C**) and the corresponding boxplots of the order parameter Rav averaged over

a window of length 100 · T<sup>s</sup> at the end of the post-stimulus epoch (**Figures 5B,D**) for 11 different networks (marked with different color/line types) for the cases studied above. Case III: (K, Ts) = (0.20, 9). Case IV: (K, Ts) = (0.20, 14).

#### **Case III:**

In the majority of networks SVS CR stimulation does not induce a pronounced and reliable desynchronization during the fivefold longer SVS CR singleshot as well as in the post-stim epoch (**Figure 6A**). Three out of 11 networks display a pronounced acute and long-lasting desynchronization (**Figure 6A**). Accordingly, the dispersion around the median is large during and after the singleshot (**Figure 6B**).

#### **Case IV:**

During the long SVS CR singleshot a pronounced desynchronization occurs (**Figure 6C**), as reflected by the small dispersion around the small median in the corresponding boxplot (**Figure 6D**). However, in the post-stimulation epoch most of the networks relax to a synchronized state, with only a few networks remaining in a long-term desynchronized state (**Figure 6D**). Accordingly, there is a large dispersion around a large median in the boxplot (**Figure 6D**).

In summary, for both RVS CR and SVS CR stimulation the fivefold increase of the stimulation duration does not lead to a reliable and pronounced long-lasting desynchronization. Again, we performed the same analysis for a larger set of (K, Ts) pairs in the parameter plane analyzed in Manos et al. (in review), with K ranging from 0.2 to 0.3 (weak intensities) and T<sup>s</sup> ranging from 9 to 28 ms (around the intrinsic period value). For all parameter pairs tested, the spacing Protocol B did not lead to a reliable and pronounced long-term desynchronization.

#### Protocol C: Spaced Multishot CR Stimulation With Demand-Controlled Variation of Stimulation Period **T<sup>s</sup>** and Intensity

We study the stimulation outcome of only five symmetrically spaced consecutive single CR shots with stimulation period T<sup>s</sup> and intensity varied according to a three-stage control scheme. To this end, for both RVS CR and SVS CR stimulation we consider two unfavorable parameter pairs of fixed CR stimulation period and intensity, respectively. One example refers to cases where CR stimulation induces acute effects, but no long-lasting desynchronizing effects (Cases I and IV). The other example concerns the case where CR stimulation causes neither acute nor long-lasting desynchronizing effects in a reliable manner (Cases II and III). We consider a regular and a random type of demandcontrolled variation of the CR stimulation period T<sup>s</sup> . Note, in both cases the CR stimulation period is not adapted to frequency characteristics of the network. We consider the time courses of the time-averaged order parameter < R > and Rav, the order parameter averaged over a window of length 100 ·T<sup>s</sup> at the end of pause.

#### **Demand-controlled regular variation of the CR stimulation period and demand-controlled variation of the intensity**

At the end of each pause we calculate the order parameter Rav averaged over a window of length 100 · T<sup>s</sup> . We vary the CR stimulation period and intensity according to the amount of synchrony, based on a three-stage control scheme:


#### **Spaced Multishot RVS CR Stimulation With Demand-Controlled Regular Variation of the Stimulation**

**Period T**s **and Demand-Controlled Variation of the Intensity** In both Cases (I and II) this protocol reliably induces a desynchronization for all networks tested (**Figures 7A,C**). After the second RVS CR shot the median of the time-averaged order parameter Rav at the end of the corresponding pauses falls below 0.4, with moderate dispersion (**Figures 7B,D**). Note, already after the first mild variation of the CR stimulation period T<sup>s</sup> the amount of synchrony is strongly reduced. In several networks and pauses, the desynchronization criterion, Rav < 0.2, is fulfilled, so that during the subsequent CR shots no stimulation is delivered (**Figures 7A,C**). Accordingly, Protocol C enables to reduce the integral amount of stimulation.

#### **Spaced Multishot SVS CR Stimulation With Demand-Controlled Regular Variation of the Stimulation**

**Period T**s **and Demand-Controlled Variation of the Intensity** This protocol causes a desynchronization for all networks tested in both Cases (III and IV) (**Figures 8A,C**). After the third (Case III, **Figure 8B**) or the second SVS CR shot (Case IV, **Figure 8D**) a pronounced desynchronization is achieved, as reflected by a median of Rav close to 0.2 (**Figures 8B,D**). Accordingly, about half of the networks fulfilled the desynchronization criterion Rav < 0.2 after the third SVS CR shot and, hence, did not require further CR stimulation. The mean firing rate,

duration. (B,D) Boxplots for the time-averaged order parameter Rav at the end of each pause, illustrate the overall outcome for all tested 11 networks. Case I: (K, Ts) = (0.30, 11). Case II: (K, Ts) = (0.20, 28).

measured at the end of each pause did not deviate from the baseline firing rates by more than ±3%, irrespective of the extent of protocol-induced variation of the stimulation period T<sup>s</sup> (data not shown).

We do not adapt the CR stimulation period T<sup>s</sup> to frequency characteristics of the stimulated network. To further illustrate this aspect, we replace a regular, increasing or decreasing variation of the stimulation period by a random variation.

#### **Demand-Controlled Random Variation of the CR Stimulation Period and Demand-Controlled Variation of the Intensity**

Again, at the end of each pause we calculate the order parameter Rav averaged over a window of length 100·T<sup>s</sup> . A random variation of the CR stimulation period is performed, depending on the amount of synchrony detected. To this end, we select the interval [T<sup>s</sup> (1) − 4 ms, T<sup>s</sup> (1) + 4 ms], where T<sup>s</sup> (1) denotes the CR stimulation period of the first shot. By design, this interval has a lower bound at 9 ms. The three-stage control scheme is governed by:

(i) Insufficient desynchronization: If Rav > 0.4 at the end of the pause of the j-th CR shot, we randomly pick T<sup>s</sup> j + 1 and skip inefficient values used before.


The feasibility of this protocol is demonstrated by considering one example for RVS CR stimulation (Case II, **Figure 9**) and one for SVS CR stimulation (Case IV, **Figure 10**). For both cases we additionally provide the mean firing rate of the networks at the end of each shot and at the end of each subsequent pause to demonstrate that deviations do not exceed ±3% (**Figures 9C**, **10C**). In the RVS case (**Figure 9**), the time course of the order parameter < R > (**Figure 9A**) and the corresponding boxplots of Rav (**Figure 9B**) display a similar pattern of reliable desynchronization as obtained by Protocol C with regular variation of the CR stimulation duration T<sup>s</sup> (**Figure 7**). In principle, the SVS case (**Figure 10**) provides similar findings as with a regular variation of the CR stimulation duration T<sup>s</sup> (**Figure 8**). However, one network relaxes back to a strongly synchronized state (**Figure 10A**, dashed blue line). Delivering a sixth SVS CR shot with randomly varied T<sup>s</sup> to that network caused a desynchronization (data not shown). This example

illustrates that a sequence of five SVS CR shots might not be sufficient to induce desynchronization in all possible networks.

In summary, for the five-shot RVS CR as well as SVS CR stimulation Protocol C with regular as well as random variation of the CR stimulation duration T<sup>s</sup> we observed a pronounced desynchronization, with the exception of one network (**Figure 10A**, dashed blue line). Our analysis was performed for a larger set of (K, Ts) pairs in the parameter plane analyzed in Manos et al. (in review), with K ranging from 0.2 to 0.3 (weak intensities) and T<sup>s</sup> ranging from 9 to 28 ms (around the intrinsic period). For all parameter pairs tested, the spacing Protocol C with regular and random variation of T<sup>s</sup> led to a reliable and pronounced long-term desynchronization in the vast majority of networks tested.

#### Protocol D: Long Singleshot CR Stimulation With Demand-Controlled Variation of the Stimulation Frequency

Protocol D consists of five consecutive shots. Unlike in Protocol C, there are no pauses between the five consecutive shots, so that they form one long singleshot.

#### **Demand-controlled regular variation of the CR stimulation period and demand-controlled variation of the intensity**

At the end of each shot we calculate the order parameter Rav averaged over a window of length 100 · T<sup>s</sup> . We vary the CR stimulation period and intensity according to the amount of synchrony, based on the three-stage control scheme as used for Protocol C (see above).

#### **Long singleshot RVS CR stimulation with demand-controlled variation of the stimulation frequency**

In Case I this protocol seems to perform similarly well (**Figures 11A,B)** as Protocol C (**Figures 7A,B**) and Protocol B (**Figures 5A,B**) which is also an alternative long singleshot but with fixed T<sup>s</sup> . After the second RVS CR shot almost all networks reach a moderate or sufficient desynchronization which is maintained fairly well after the RVS CR is ceased. Nonetheless, this particular protocol does not perform equally well for Case II (**Figures 11C,D)**. Even in the cases, where the variation of T<sup>s</sup> leads to some improvement, the overall long-lasting effect is worse than with Protocol C (**Figures 7C,D**).

#### **Long singleshot SVS CR stimulation with demand-controlled variation of the stimulation frequency**

In Case III this protocol does not show any systematical improvement (**Figures 12A,B)**. In fact, this is partly due to the fact that in some cases the network gets trapped in an unfavorable parameter variation loop, bouncing between T<sup>s</sup> = 9 ms and T<sup>s</sup> = 10 ms. In Case IV (**Figures 12C,D)** the global evolution is quite similar to the one found for Protocol B (**Figures 6C,D**), i.e., pronounced desynchronization during a single shot, with a

indicate the CR shots, while the horizontal dashed gray lines highlight the two control thresholds (see text). (B) Boxplots for the time-averaged order parameter Rav at the end of each pause, illustrate the overall outcome for all tested 11 networks. (C) For each (vertically aligned) network the table presents CR intensity (K = 0.2 if CR is ON or K = 0 if CR is OFF during a CR shot) and stimulation period T<sup>s</sup> used for each CR shot (indicated by red bars) together with the mean firing rate of the network at the end of each CR shot ("ON") and at the end of the subsequent pause ("OFF"). The mean firing rate was strongly fluctuating and, hence, calculated in a window of length 200 · Ts. Case II stimulation parameters are unfavorable for anti-kindling: (K, Ts) = (0.20, 28) (see text).

tendency to relapse back to the synchronized state while some of the networks remain desynchronized. However, the overall final outcome is rather poor as the corresponding boxplot (blue color) at the end of the CR-off period indicates.

In summary, for the singleshot RVS CR as well as SVS CR stimulation Protocol D with regular variation of the CR stimulation duration T<sup>s</sup> (without pauses between two consecutive shots) did not lead to a reliable and systematic long-lasting desynchronization.

## DISCUSSION

By comparing spaced CR stimulation with fixed stimulation parameters (Protocol A) and massed, continuous CR stimulation with equal integral duration (Protocol B) with a flexible spaced CR stimulation with demand-controlled variation of CR stimulation frequency and intensity (Protocol C), and with a flexible non-spaced CR stimulation with demand-controlled variation of CR stimulation frequency and intensity (Protocol D), we demonstrated that Protocol C enables to significantly improve the long-term desynchronization outcome of both RVS and SVS CR stimulation, even at comparatively short integral stimulation duration. Remarkably, spacing alone (Protocol A) is not sufficient to provide an efficient short-term dosage regimen (**Figures 2**, **3**). In fact, in particular cases fivefold longer stimulation duration might even be more efficient than five consecutive single CR shots with identical integral stimulation duration, at least for RVS CR stimulation (**Figure 2B** vs. **Figure 4B**). The low performance of pure spacing (Protocol A) might be due to the low number of single CR shots, here five, as opposed to slightly larger numbers of CR shots, say eight, tested for the case of subcritical CR stimulation before (Popovych et al., 2015). However, more important might be the approximately fiftyfold longer stimulation and pause duration used for the spaced subcritical CR stimulation protocol (Popovych et al., 2015). The long spaced subcritical CR stimulation protocol might be

beneficial for invasive application, such as DBS, and help reduce side-effects by substantially reducing stimulation current intake of the issue.

However, computationally we show that a spacing with rigid five-shot timing structure, but flexible, demand-controlled variation of stimulation frequency and intensity (Protocol C) provides a short-term dosage regimen that significantly improves the long-term desynchronization outcome of RVS and SVS CR stimulation (**Figures 6**–**9**). At the end of each pause between CR shots, the stimulus after-effect is assessed. If the desynchronization is considered to be insufficient, a mild variation of the CR stimulation frequency is performed to possibly provide a better fit between network and CR stimulation frequency, without actually adapting the stimulation frequency to frequency characteristics of the network stimulated. If desynchronization is considered to be moderate, the subsequent CR shot is delivered with parameters unchanged. If desynchronization is sufficient, CR stimulation is suspended during the subsequent shot. Intriguingly, in the vast majority of parameters and networks tested, this short-term dosage regimen induces a robust and reliable long-lasting desynchronization (**Figures 6**–**9**). This protocol might be a candidate especially for non-invasive, e.g., acoustic (Tass et al., 2012a) or vibrotactile (Tass, 2017; Syrkin-Nikolau et al., 2018), applications of CR stimulation to increase desynchronization efficacy, while keeping the stimulation duration at moderate levels.

Demand-controlled variation of CR stimulation frequency and intensity (Protocol D) alone (i.e., without inserting pauses) is not sufficient to significantly improve the outcome of RVS and SVS stimulation (**Figures 11**, **12**). Hence, introducing pauses significantly improves the effect of the demand-controlled variation of CR stimulation frequency and intensity.

In principle, stimulation parameters other than the CR stimulation frequency might be varied depending on the stimulation outcome. However, in this study we have chosen to vary the CR stimulation frequency, since the latter turned out to be a sensitive parameter, especially for SVS CR stimulation (see Manos et al., in review). In fact, the short-term dosage regimen

with demand-controlled variation of stimulation parameters (Protocol C) might help to turn SVS CR stimulation in a method that causes a particularly strong anti-kindling in a robust and reliable manner.

Protocol C does not require a direct adaption of the CR stimulation frequency to measured quantities reflecting frequency characteristics of the network. We have chosen this design, since it might be an advantage not to rely on specific biomarker-type of information. For instance, in the case of PD a number of relevant studies were devoted to closed-loop DBS (Graupe et al., 2010; Rosin et al., 2011; Carron et al., 2013; Little et al., 2013; Priori et al., 2013; Yamamoto et al., 2013; Hosain et al., 2014; Rosa et al., 2015). A relevant issue in this context is the availability of a biomarker adequately reflecting the individual patient's extent of symptoms (Beudel and Brown, 2016; Kühn and Volkmann, 2017). In fact, it is not clear whether low or high frequency beta band oscillations might be more appropriate as biomarker-type of feedback signal (Beudel and Brown, 2016). For several reasons, beta band oscillations might possibly not be an optimal feedback signal (Johnson et al., 2016; Kühn and Volkmann, 2017). Enhanced beta band oscillations are not consistently found in all PD patients (Kühn et al., 2008; Kühn and Volkmann, 2017). The clinical score of PD patients might more appropriately be reflected by the power ratio of two distinct bands of high frequency oscillations around 250 and 350 Hz (Özkurt et al., 2011). Appropriate biomarkers might depend on the patient phenotype (Quinn et al., 2015): In tremor dominant (compared to akinetic rigid) PD patients resting state beta power may decrease during tremor epochs (Bronte-Stewart et al., 2009; Quinn et al., 2015). By a similar token, theta and beta oscillations interact with high-frequency oscillations under physiological (Yanagisawa et al., 2012) as well as pathological (Yang et al., 2014) conditions. Also, quantities assessing the interaction of brain oscillation, e.g., phase amplitude coupling (PAC) might be used as biomarker to represent the amount of symptoms (Beudel and Brown, 2016). Also, activity in the beta band might be relevant for compensatory purposes, as recently shown in a parkinsonian monkey study with sensorimotor rhythm neurofeedback (Philippens et al., 2017).

It might be another potential advantage for clinical applications that the three-stage control of the proposed short-term dosage regimen (Protocol C) could possibly be approximated by scores reflecting the patient's state or the amount symptoms. A simple three-stage rating of the patient's state (bad, medium, and good) might replace the feedback signal-based stages (i), (ii), and (iii). Assessments of the patient's state might be performed in a pause after a CR shot. Depending on the rating, the CR stimulation frequency or intensity of the subsequent CR shot may be varied. In particular, for non-invasive application of CR stimulation a non-invasive

assessment of the stimulation effect might straightforwardly be realized.

In realistic biological systems intrinsic (model) parameters typically vary over time. These variations may be of complex dynamical nature (see e.g., Timmer et al., 2000; Yulmetyev et al., 2006). To obtain some indication as to whether Protocol C is robust against low-amplitude intrinsic variations of the neuronal firing rates, we added a low-amplitude term Ivar = A·sin(2π ·f ·t) to the right-hand side of Equation 1a. In the stimulation-free case, Ivar causes variations of the neurons' firing rates in the order of ±3% and no qualitative changes of the network dynamics (data not shown). For different frequencies f this type of variation does not significantly affect the long-term desynchronization outcome of Protocol C (f = 0.004, 4, and 20 Hz in Supplementary Figure 1). By the same token, the neuronal firing rates are not significantly altered by the additional periodic force (data not shown).

Note, this is not intended to be a comprehensive study of the impact of periodic forcing of arbitrary frequency on the spontaneous or stimulation-induced dynamics of the model network under consideration. Rather, the slow oscillatory forcing is meant to model slow physiological modulatory processes in an illustrative manner. In the extreme case of f = 0.004 Hz the slow oscillatory modulation acts on the same time scale as a cycle comprising shot and pause and, hence smoothly emulates the step-wise modulation of the CR stimulation frequency in Protocol C.

Conversely, intrinsic variations of sufficient size might naturally mimic variations of the relationship between CR stimulation frequency and intrinsic neuronal firing rates as introduced on purpose in Protocol C. Accordingly, already the purely spaced stimulation without demand-controlled variability (Protocol A) might display some variability of the relationships between intrinsic firing rates and CR stimulation frequency simply due to the intrinsic variability. However, at least with the frequencies 0.004, 4, and 20 Hz in the low-amplitude term Ivar = A · sin(2π · f · t) added to the right-hand side of Equation 1a, we were not able to observe any substantial improvement of the desynchronizing outcome of Protocol A (Supplementary Figure 2). However, more physiological patterns of firing rate modulations might have a more significant impact on the stimulation outcome of Protocol A. In future studies typical variations of the signals relevant to a particular preclinical or clinical application might be taken into account to further improve desynchronizing short-term dosage regimen. The additional periodic forcing considered here was meant to illustrate the stability of the suggested control approach. However, future studies could also provide a detailed analysis of the interplay of one or more periodic inputs and noise, thereby focusing on stochastic resonance and related phenomena (e.g., Pikovsky and Kurths, 1997; Gammaitoni et al., 1998; Manjarrez et al., 2002; Torres et al., 2011; Bordet et al., 2015; Yu et al., 2016; Guo et al., 2017; Uzuntarla et al., 2017 and references therein). The number of stimulation sites and CR stimulation spatial decay was based on Lysyansky et al. (2011). In accordance to that study, adding more stimulation sites does neither lead to qualitatively different results nor does it improve the stimulation outcome (see Supplementary Figure 3).

The short-term dosage regimen proposed here provides a closed-loop CR stimulation concept that enables to significantly increase the robustness and reliability of the stimulation outcome. Our results motivate to further improve the CR approach by closed loop or feedback-based dosage regimen. Compared to the computationally developed initial concept of demand-controlled CR-induced desynchronization of networks with fixed coupling constants (Tass, 2003a,b), the focus will now be on a feedback-adjusted modulation of synaptic patterns to induce long-lasting therapeutic effects. Currently, clinical proof of concept (phase IIa) is available for deep brain CR stimulation for the therapy of Parkinson's disease (Adamchic et al., 2014) and acoustic CR stimulation for the treatment of chronic subjective tinnitus (Tass et al., 2012a). In addition, promising first in human (phase I) data are available for vibrotactile CR stimulation for the treatment of Parkinson's disease showing pronounced and highly significant sustained therapeutic effects (Syrkin-Nikolau et al., 2018). For the clinical development of these treatments it is mandatory to perform dose-finding studies (phase IIb) to reveal optimal stimulation parameters and dosage regimens, for comparison (see Friedman et al., 2010). The latter are required to get properly prepared for large efficacy (phase III) trials (Friedman et al., 2010). Since

#### REFERENCES


CR stimulation modulates complex neuronal dynamics, dosefinding studies are sophisticated, since stimulation parameters as well as dosage patterns have to be chosen appropriately. Selecting appropriate stimulation parameters and dosage regimens by trial and error may neither be effective nor affordable, since it would require a huge number of patients. In contrast, our manuscript illustrates the important role of computational medicine in generating hypotheses for dose-finding studies. Specifically, we show that spacing (i.e., adding pauses in between stimulation epochs) as well as moderate and unspecific parameter variations adapted in the course of the therapy are not sufficient to overcome limitations of CR stimulation. Intriguingly, the combination of both, spacing plus adaptive moderate parameter variation increases the robustness of the stimulation outcome in a significant manner. This computational prediction can immediately be tested in dose-finding studies and, hence, help to optimize the CR therapy, shorten the development time and reduce related costs.

### AUTHOR CONTRIBUTIONS

TM, MZ, and PT: Conceived and designed the experiments; TM: Performed the experiments; TM, MZ, and PT: Analyzed the data; TM and MZ: Contributed reagents, materials, and analysis tools; TM, MZ, and PT: Wrote the paper.

#### SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fphys. 2018.00376/full#supplementary-material


parallel with improvement in motor performance. J. Neurosci. 28, 6165–6173. doi: 10.1523/JNEUROSCI.0282-08.2008


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Manos, Zeitler and Tass. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Computationally Developed Sham Stimulation Protocol for Multichannel Desynchronizing Stimulation

Magteld Zeitler <sup>1</sup> and Peter A. Tass <sup>2</sup> \*

<sup>1</sup> Research Center Jülich, Institute for Neuroscience and Medicine, Brain and Behaviour (INM-7), Jülich, Germany, <sup>2</sup> Department of Neurosurgery, Stanford University, Stanford, CA, United States

A characteristic pattern of abnormal brain activity is abnormally strong neuronal synchronization, as found in several brain disorders, such as tinnitus, Parkinson's disease, and epilepsy. As observed in several diseases, different therapeutic interventions may induce a placebo effect that may be strong and hinder reliable clinical evaluations. Hence, to distinguish between specific, neuromodulation-induced effects and unspecific, placebo effects, it is important to mimic the therapeutic procedure as precisely as possibly, thereby providing controls that actually lack specific effects. Coordinated Reset (CR) stimulation has been developed to specifically counteract abnormally strong synchronization by desynchronization. CR is a spatio-temporally patterned multichannel stimulation which reduces the extent of coincident neuronal activity and aims at an anti-kindling, i.e., an unlearning of both synaptic connectivity and neuronal synchrony. Apart from acute desynchronizing effects, CR may cause sustained, long-lasting desynchronizing effects, as already demonstrated in pre-clinical and clinical proof of concept studies. In this computational study, we set out to computationally develop a sham stimulation protocol for multichannel desynchronizing stimulation. To this end, we compare acute effects and long-lasting effects of six different spatio-temporally patterned stimulation protocols, including three variants of CR, using a no-stimulation condition as additional control. This is to provide an inventory of different stimulation algorithms with similar fundamental stimulation parameters (e.g., mean stimulation rates) but qualitatively different acute and/or long-lasting effects. Stimulation protocols sharing basic parameters, but inducing nevertheless completely different or even no acute effects and/or after-effects, might serve as controls to validate the specific effects of particular desynchronizing protocols such as CR. In particular, based on our computational findings we propose a multichannel sham (i.e., inactive) stimulation protocol as control condition for phase 2 and phase 3 studies with desynchronizing multichannel stimulation techniques.

Keywords: sensory neurostimulation, non-invasive neuromodulation, coordinated reset, spike timing-dependent plasticity, desynchronization, anti-kindling, sham stimulation, placebo

### INTRODUCTION

To establish a pharmacological therapy for clinical use, clinical trials are performed in humans that are typically classified into four phases (Friedman et al., 2015): First, in pre-clinical studies pharmacokinetic, toxicity, and efficacy are studied in non-human subjects. In phase I trials, so-called first in human-studies, safety and tolerability of a drug are investigated in

#### Edited by:

Alexey Zaikin, University College London, United Kingdom

#### Reviewed by:

Anuj Agarwal, Signal Solutions LLC, United States Elena Mitroshina, N. I. Lobachevsky State University of Nizhny Novgorod, Russia

> \*Correspondence: Peter A. Tass ptass@stanford.edu

#### Specialty section:

This article was submitted to Computational Physiology and Medicine, a section of the journal Frontiers in Physiology

Received: 29 November 2017 Accepted: 20 April 2018 Published: 08 May 2018

#### Citation:

Zeitler M and Tass PA (2018) Computationally Developed Sham Stimulation Protocol for Multichannel Desynchronizing Stimulation. Front. Physiol. 9:512. doi: 10.3389/fphys.2018.00512

**86**

healthy volunteers. Phase II trials aim to determine whether a drug can have any efficacy. More specifically, phase IIA trials typically aim at demonstrating clinical efficacy or biological activity ("proof of concept" studies), whereas phase IIB trials are dose-finding studies, performed to reveal optimum dose at which a drug has biological activity with minimal side-effects. Phase III trials investigate effectiveness and the clinical value of a new intervention in a larger patient group. In a randomized controlled trial the effect size of a new intervention is compared with state of the art treatment, if available. Finally, a phase IV trial is a postmarketing surveillance trial, performed e.g., to study whether any rare or long-term adverse effects occur within a much larger patient population and over longer time periods. Individual trials may actually comprise more than only one phase. For instance, there are combined phase I/II or phase II/III trials. Accordingly, given the different purpose of clinical trials, one may also distinguish between early phase studies and late phase trials (Friedman et al., 2015—see above). In principle, this 4-phase pattern also holds for medical technology, e.g., neuromodulation technologies.

Apart from investigating safety and tolerability, it is key to study whether a new therapeutic intervention is superior to pre-existing therapeutic options (Friedman et al., 2015). To this end, one has to take into account non-specific, placebo effects. A placebo effect is a psychobiological phenomenon that causes symptom relief after delivery of inert substances or other types of sham treatment, such as sham surgery or sham stimulation, in combination with verbal instructions suggesting clinical benefit (Price et al., 2008; Benedetti et al., 2011). Note, in clinical trials the terms placebo and sham are basically synonymous, while a placebo typically refers to an inactive substance used in pharmacological trials, whereas a sham stimulation/operation refers to a stimulation/operation without specific therapeutic effect (Price et al., 2008; Benedetti et al., 2011; Friedman et al., 2015). Real placebo effects go beyond spontaneous remission due to the natural history of a disease, regression to the mean induced by selection biases or expectation-related biases of patients and doctors (Benedetti, 2008a).

There are many different placebo effects, caused by different mechanisms and related to different types of interventions and different diseases (Benedetti, 2008b; Enck et al., 2008). Expectation, anxiety, reward, and different types of learning mechanisms may contribute to placebo effects (Benedetti et al., 2011). For instance, according to the Hawthorne effect, patients may simply improve because they are enrolled in a clinical trial (Last, 1983). Placebo treatments can decrease anxiety levels (Vase et al., 2005) and, in general, modulate emotions (Petrovic et al., 2002). Conversely, an inert substance combined with an instruction inducing negative expectations may cause a nocebo effect (Enck et al., 2008), e.g., an increase of pain (Colloca et al., 2008).

Placebo effects may actually be related to objective changes of brain action (Benedetti et al., 2011), e.g., release of endogenous dopamine (de la Fuente-Fernández et al., 2001), changes in brain glucose metabolism (Mayberg et al., 2002) or changes of the activity of specific neuronal populations (Benedetti et al., 2004).

Different types of learning mechanisms, e.g., conditioning, may play important roles in placebo mechanisms (Benedetti et al., 2011). For instance, administration of a placebo after delivery of active drugs may be more effective than placebo administration without the previous experience with the corresponding active drug (Sunshine et al., 1964; Batterman, 1966; Batterman and Lower, 1968; Laska and Sunshine, 1973; Amanzio and Benedetti, 1999; Colloca and Benedetti, 2006). Not only features related to a drug or therapeutic procedure may contribute to placebomediated clinical improvement, but also many other stimuli, related to medical environment, equipment, and personnel (Benedetti et al., 2011). From a clinical trials standpoint it is, hence, important to mimic the entire procedure of treatment delivery as well as possible, since even instructions and rituals of the treatment delivery and procedure may cause actual changes in brain activity that may be the same as those induced by the specific treatment (Benedetti et al., 2011). Accordingly, a vast majority (97%) of surveyed Parkinson's disease (PD) clinical researchers in the United States and Canada believe that even in the case of neurosurgical cell-based and gene therapies for PD double-blind, placebo-controlled trails have to be performed to assess safety and efficacy (Kim et al., 2005; Olanow, 2005). Ninety percentage of PD clinical researchers consider burr holes as justified for sham neurosurgery procedures, and a minority (<22%) even consider penetration of brain tissue to be justified for the neurosurgical sham control (Kim et al., 2005; Olanow, 2005). Hence, even in the case of clinical trials performed according to highest quality standards, e.g., in the field of deep brain stimulation (DBS) (Schuepbach et al., 2013), the comparison between qualitatively different therapeutic regimes, e.g., invasive neuromodulation plus medication vs. medication only, caused debates on whether the study design could reliably rule out placebo effects (Schüpbach et al., 2014).

There is a variety of strategies for the development of sham stimulation protocols. For instance, in the context of transcranial current stimulation a number of studies were devoted to the development of appropriate sham stimulation protocols, since current flow can elicit tingling or itching skin sensations, where different transcranial electrical stimulation methods have different cutaneous perception thresholds (Ambrus et al., 2010). In a comparative transcranial electrical stimulation study, a short-duration active protocol was used as sham, where the active stimulation was turned on only for a brief period, during which stimulation-related unwanted effects/perceptions were elicited (Inukai et al., 2016). Accordingly, the dose should be insufficient, but the patient should get the impression of receiving stimulation. Alternatively, off-target stimulation strategies were developed. In that case, the patient perceives stimuli and/or side effects thereof, but stimulation is directed to targets putatively rendering stimulation ineffective. For instance, for a sham condition for transcranial direct current stimulation a current configuration was chosen such that the current primarily traversed across the scalp, through adjacent pairs of electrodes of opposite polarity, in this way sparing cortical tissue (Richardson et al., 2014; Garnett and den Ouden, 2015). Another offtarget stimulation sham strategy is used in the field of tinnitus, where stimulation tones are delivered at sufficiently detuned pitch compared to the tonal tinnitus, putatively activating brain sites sufficiently remote from the brain regions engaged in the tinnitus-related abnormal neuronal synchrony (Tass et al.,

2012a; Adamchic et al, 2017). In contrast, we here consider the situation when clinical constraints are not permitting an off-target stimulation for sham purposes. We hypothesize that an appropriate stimulation pattern may render stimulation ineffective, although the single stimuli are delivered to target sites.

This computational study is dedicated to the development of sham stimulation protocols for desynchronizing multichannel stimulation techniques, specifically coordinated reset (CR) stimulation (Tass, 2003a). The latter was computationally designed to specifically antagonize abnormal neuronal synchrony by desynchronization (Tass, 2003a,b). To this end, sequences of stimuli are administered to different neuronal sub-populations engaged in abnormal neuronal synchronization (Tass, 2003a). In computational studies it was shown that in the presence of spike-timing-dependent plasticity (STDP) (Gerstner et al., 1996; Markram et al., 1997; Bi and Poo, 1998) CR stimulation may have long-lasting, sustained effects (Tass and Majtanik, 2006; Hauptmann and Tass, 2007; Popovych and Tass, 2012). This antikindling effect (Tass and Majtanik, 2006) is caused by a CRinduced reduction of the rate of coincidences which, in turn, induces a decrease of synaptic weights, ultimately shifting the stimulated network from an attractor with abnormal synaptic connectivity and abnormal neuronal synchrony to an attractor with weak connectivity and synchrony (Tass and Majtanik, 2006; Hauptmann and Tass, 2007; Popovych and Tass, 2012).

Abnormal neuronal synchronization was found in a number of brain diseases, e.g., Parkinson's disease (Lenz et al., 1994; Nini et al., 1995; Hammond et al., 2007), tinnitus (Ochi and Eggermont, 1997; Llinas et al., 1999; Weisz et al., 2005; Eggermont and Tass, 2015), migraine (Angelini et al., 2004; Bjørk and Sand, 2008). Standard high-frequency (HF) DBS is the standard treatment of medically refractory movement disorders, such as PD (Benabid et al., 1991; Krack et al., 2003; Deuschl et al., 2006). Standard HF DBS only has acute clinical (Temperli et al., 2003) and acute electrophysiological (Kühn et al., 2008; Bronte-Stewart et al., 2009) effects, which are present only during stimulation and vanish after cessation of stimulation. In contrast, in parkinsonian nonhuman primates it was shown that electrical CR-DBS of the subthalamic nucleus (STN) has sustained, long-lasting after-effects on motor function (Tass, 2003b; Wang et al., 2016). Analogously, cumulative and lasting after-effects of electrical CR-DBS of the STN were also observed in PD patients (Adamchic et al., 2014).

For the clinical development, in particular, of non-invasive applications of CR stimulation (Popovych and Tass, 2012), such as acoustic CR stimulation for tinnitus (Tass et al., 2012a) or vibrotactile stimulation for PD (Tass, 2017; Syrkin-Nikolau et al., 2018), it is key to compare the effects of CR stimulation with an appropriate sham stimulation protocol in phase II and phase III clinical trials. The sham stimulation protocol should be reasonably similar to the CR stimulation pattern, to prevent patients from being able to distinguish between actual treatment and control. Accordingly, performing double-blind, placebo-controlled trails for non-invasive, sensory multichannel stimulation therapies requires multichannel sham stimulation protocols.

We here computationally develop a multichannel sham stimulation protocol. To this end, we investigate the anti-kindling effect of several multichannel stimulation protocols that share basic features with CR stimulation. We apply the different stimulation protocols to a one-dimensional computational network model with spiking neurons and study the stimulation effects at different levels, ranging from the macroscopic network level, via subpopulations down to the single neuron level. We obtain an inventory of qualitatively different stimulation effects elicited by the different stimulation protocols. Intriguingly, we found an inert stimulation protocol which caused only weak acute and hardly any long-lasting effects. The latter is a potential sham candidate to be tested for clinical studies in the context of desynchronizing sensory multichannel stimulation techniques.

#### MATERIALS AND METHODS

In this section, we describe the equations used to model the dynamics of our one-dimensional neuronal network, the plasticity of the synapses, and the different stimulation protocols as well as the data analysis methods.

#### Neuronal Network

The model we use is a one-dimensional ring composed of N spiking Hodgkin-Huxley neurons which interact via strong excitatory short-range and weak inhibitory long-range synapses (Popovych and Tass, 2012). The membrane potential V<sup>i</sup> of the i-th neuron (i = 1: N) is given by:

$$\begin{aligned} C\dot{V}\_i &= I\_i - \operatorname{g\_{Na}m}\_i^3 h\_i \left( V\_i - V\_{\text{Na}} \right) \\ &- \operatorname{g\_k} n\_i^4 \left( V\_i - V\_{\text{K}} \right) - \operatorname{gl} \left( V\_i - V\_l \right) + \operatorname{S\_i} + F\_{i\text{,}} \\ \dot{\mathbf{x}}\_i &= \alpha\_\text{x} \left( V\_i \right) \left( 1 - \mathbf{x}\_i \right) - \beta\_\text{x} \left( V\_i \right) \mathbf{x}\_i, \\ \dot{s}\_i &= \frac{0.5 \left( 1 - s\_i \right)}{1 + \exp \left[ - \left( V\_i + 5 \right) / 12 \right]} - 2s\_i, \end{aligned} \tag{1}$$

where C denotes the membrane capacitance, the injected constant currents I<sup>i</sup> are uniformly distributed (Iiǫ [I<sup>0</sup> − 1<sup>I</sup> , I<sup>0</sup> + 1I]) and F<sup>i</sup> represents the current induced by an external stimulation signal (see section Simulation Details for more details). The voltage-dependent rate constants α<sup>x</sup> and β<sup>x</sup> of the time-varying ion gate variables x ǫ {m, n, h} are given by α<sup>m</sup> (V) = (0.1V + 4) / - 1 − exp (−0.1V − 4) , β<sup>m</sup> (V) = 4 exp [(−V − 65) /18], α<sup>h</sup> (V) = 0.07 exp [(−V − 65) /20], β<sup>h</sup> (V) = 1/ - 1 + exp (−0.1V − 3.5) , α<sup>n</sup> (V) = (0.01V + 0.55) / - 1 − exp (−0.1V − 5.5) , and β<sup>n</sup> (V) = 0.125 exp [(−V − 65) /80].

The coupling term S<sup>i</sup> in Equation (1) stands for the weighted ensemble average of all postsynaptic currents received by neuron i from the other neurons in the network and can be given in terms of the synaptic variable s<sup>j</sup> as:

$$S\_{\bar{i}} = N^{-1} \sum\_{j=1}^{N} \left( V\_{r\_{\bar{j}}} - V\_{\bar{i}} \right) c\_{\bar{i}\bar{j}} |M\_{\bar{i}\bar{j}}| s\_{\bar{j}}, \tag{2}$$

where N is the number of neurons within the network, Vr,jthe reversal potential of the synaptic coupling between neurons j and i and cij is the synaptic coupling from neuron j to neuron i. Mij has the form of a Mexican hat (Wilson and Cowan, 1973; Dominguez et al., 2006; de la Rocha et al., 2008) and determines the type of the neuronal connection between neurons i and j: Mij < 0 represents an inhibitory coupling, Mij > 0 an excitatory coupling. The value of Mij determines the distance dependent maximal strength between those neurons:

$$M\_{ij} = \left(1 - d\_{ij}^2 / \sigma\_1^2\right) \exp\left(-d\_{ij}^2 / \left(2\sigma\_2^2\right)\right) \tag{3}$$

with σ<sup>1</sup> = 3.5 and σ<sup>2</sup> = 2.0 as in Popovych and Tass (2012), and dij = d·min  i − j , N − i − j is the shortest distance between neurons i and j. To avoid boundary effects, the neurons form a one-dimensional ring. Therefore, the shortest distance between the neurons with indices 1 and N is 1 instead of N −1. The lattice distance between two adjacent neurons is given by:

$$d = \, d\_0 / \, (N - 1) \tag{4}$$

with d<sup>0</sup> the length of the neuronal chain.

Values used in this study are N = 200, C = 1 µF/cm<sup>2</sup> , maximum conductance per unit area for the sodium, potassium and leak currents gNa = 120 mS/cm<sup>2</sup> , g<sup>K</sup> = 36 mS/cm<sup>2</sup> , g<sup>l</sup> = 0.3 mS/cm<sup>2</sup> , sodium, potassium, and leak reversal potentials VNa = 50 mV, V<sup>K</sup> = −77 mV, and V<sup>l</sup> = −54.4 mV, reversal potential for excitatory respectively inhibitory coupling Vr,<sup>j</sup> = 20 mV respectively−40 mV. For the constant injected current we used I<sup>0</sup> = 11.0 µA/cm<sup>2</sup> and <sup>I</sup> = 0.45 µA/cm<sup>2</sup> . The length of the neuronal chain is defined as d<sup>0</sup> = 10.

#### Spike Timing-Dependent Plasticity

The dynamical synaptic weights cij are influenced by the precise timing of the pre- and postsynaptic spikes and are updated in an event-based manner every time a neuron spikes. This is realized by adding δ · 1cij to the excitatory and −δ · 1cij to the inhibitory synaptic weights cij with learning rate δ > 0 every time neuron i or j spikes. According to the spike timing-dependent plasticity (STDP) rule (Bi and Poo, 1998) the change in synaptic weight is given by (Popovych and Tass, 2012):

$$
\Delta c\_{ij} = \begin{cases}
\ \beta\_1 e^{\frac{-\Delta t\_{ij}}{\beta\_1}}, & \Delta t\_{ij} \ge 0 \\
\beta\_2 \frac{\Delta t\_{ij}}{\tau} e^{\frac{\Delta t\_{ij}}{\beta\_2}}, & \Delta t\_{ij} < 0
\end{cases} \tag{5}
$$

with 1tij = t<sup>i</sup> − t<sup>j</sup> and t<sup>i</sup> is the spike time of the postsynaptic neuron i and t<sup>j</sup> the spike time of the pre-synaptic neuron j.

#### External Stimulation

The aim of this study is to compare the effects of different stimulation algorithms on the neuronal connectivity as well as on the synchronization of the neuronal activity. In this section, we describe six different stimulation protocols as well as a no-stimulation control protocol to investigate the influence of the different stimulation protocols on the connectivity and synchronization.

#### Stimulation Implemented in the Model

Each stimulation onset induces single brief excitatory postsynaptic currents, with spatial spread in the network given by a quadratic spatial decay profile:

$$D\left(i, \chi\_k\right) = \frac{1}{1 + d^2(i - \chi\_k)^2 / \sigma\_d^2} \tag{6}$$

where d is the lattice distance between adjacent neurons (Equation 4), i − x<sup>k</sup> the difference in index of neuron i and index x<sup>k</sup> of the neuron at stimulation site k, and σ<sup>d</sup> the spatial decay rate of the stimulation current (Popovych and Tass, 2012).

The total stimulation current induced in neuron i is given by:

$$F\_i = \left[V\_r - V\_i\left(t\right)\right] \cdot K \sum\_{k=1}^{N\_i} D\left(i, \chi\_k\right) G\_{s,k}\left(t\right) \tag{7}$$

with the excitatory reversal potential V<sup>r</sup> = 20 mV, V<sup>i</sup> (t) the membrane potential of neuron i (Equation 1), K the stimulation intensity and D the spatial decay profile (Equation 6). Gs,<sup>k</sup> is the stimulation (at stimulation site k) evoked time-dependent normalized conductance of the post-synaptic membranes defined by α-functions. Since in this study the minimal time difference between two stimulation onsets within the network is not restricted to Ts/N<sup>s</sup> = 4 ms, we have adapted G<sup>s</sup> from Popovych and Tass (2012) by allowing a summation of two stimulation evoked time-dependent normalized conductances of the post-synaptic membranes if the two stimulations occur within a certain time interval. We have set this time interval to 2 · Ts/N<sup>s</sup> = 8 ms, since at 8 ms the value of the αfunction is marginal (only 0.02% of the peak value). Our adapted stimulation (at stim site k) evoked time-dependent normalized conductance of the post-synaptic membranes is now given by:

$$G\_{s,k}(t) = \begin{cases} \frac{t - t\_k^n}{\tau} \exp\left[-\frac{t - t\_k^n}{\tau}\right], & t\_k^n \le t \le \min(t\_k^n + \frac{T\_k}{2}, t\_k^{n+1})\\ \frac{t - t\_k^n}{\tau} \exp\left[-\frac{t - t\_k^n}{\tau}\right] + \frac{t - t\_k^{n+1}}{\tau} \exp\left[-\frac{t - t\_k^{n+1}}{\tau}\right], & t\_k^{n+1} \le t \le t\_k^n + T\_s/2\\ 0, & \text{otherwise} \end{cases} \tag{8}$$

Synaptic weights are restricted to the interval cij ∈ [0, 1] mS/cm<sup>2</sup> to avoid unbounded strengthening and weakening. Other values used in relation to the STDP learning rule are as in Popovych and Tass (2012): β<sup>1</sup> = 1, β<sup>2</sup> = 16, γ<sup>1</sup> = 0.12, γ<sup>2</sup> = 0.15, τ = 14 ms, and δ = 0.002. With these parameter values, the plastic neuronal network under study is multistable, comprising stable desynchronized and stable synchronized states (Popovych and Tass, 2012).

Here t n k is the onset of the nth activation of the kth stimulus site, τ = Ts/(6Ns) represents the time-to-peak of Gs,<sup>k</sup> , and min(t1, t2) is the minimum value of t1, t<sup>2</sup> and thus represents the earliest time event of t<sup>1</sup> and t2.

#### Stimulation Signal Features

As a control condition we use the situation where no stimulation signal is applied and thus no stimulation current is delivered to the N neurons: F<sup>i</sup> = 0 for all i ∈ {1, .., N}. We denote the control condition as no stimulation (no-stim) protocol. The other, active stimulation protocols last 128 s and consist of stimulation ON and stimulation OFF cycles (briefly ON- and OFF-cycles), of duration T<sup>s</sup> each. N<sup>s</sup> equidistantly spaced stimulation sites are activated exactly once during one ON-cycle. After three consecutive ON-cycles, two OFF-cycles follow, before the next three ON-cycles take place. At the end of the stimulation period (briefly stim-on period) each stimulation site will have been activated exactly 4,800 times (=4,800 ON-cycles). In this study, each cycle lasts for T<sup>s</sup> = 16 ms, and for all stimulation signals the same N<sup>s</sup> = 4 equidistantly spaced stimulation sites are activated. The four stimulation sites are located at the neurons with index 25, 75, 125, and 175.

The first of the six stimulation signal approaches is the purely periodic multichannel stimulation (PPMS). The first stimulation onset of the k-th stimulation site, t 1 k , is drawn randomly (with equal distribution within an ON-cycle), the next stimulation onsets occur exactly a multiple of T<sup>s</sup> later: t n <sup>k</sup> <sup>=</sup> <sup>t</sup> 1 <sup>k</sup> <sup>+</sup> <sup>n</sup> · <sup>T</sup><sup>s</sup> (with k ∈ {1, 2, . . . , Ns} and n ∈ N). Another feature of this PPMS approach is that all four stimulation sites are activated simultaneously (see **Figure 1A**), which implies that t n <sup>k</sup> <sup>=</sup> <sup>t</sup> n 1 for all stimulation sites k.

The correlated multichannel noisy stimulation (CMNS) activates, like the PPMS, all four stimulation sites simultaneously, but the stimulation onsets are no longer periodic, but rather noisy: For each ON-cycle the stimulation onset is drawn randomly, with equal probability within the ON-cycle (**Figure 1B**). In this case, the four stimulation sites are also active simultaneously: t n <sup>k</sup> <sup>=</sup> <sup>t</sup> n 1 for all stimulation sites k.

For the uncorrelated multichannel noisy stimulation (UMNS) the stimulation onsets t n k are determined for each stimulation site k separately. To this end, for each stimulation site k and ON-cycle n the stimulation onset t n k is drawn randomly, with equal distribution within the n-th ON-cycle. The random processes that generate the stimulation onsets t n k for the different stimulation sites k ∈ {1, 2, . . . , Ns} are completely uncorrelated between stimulation sites and, hence, typically do not coincide (**Figure 1C**).

The other three stimulation protocols are different variants of coordinated reset (CR) stimulation (Tass, 2003a,b). For CR stimulation, within each ON-cycle the activations of the N<sup>s</sup> = 4 different stimulation sites are equidistantly spaced in time, with a time shift of Ts/N<sup>s</sup> (**Figures 1D–F**). The different stimulation onsets are at the beginning of the ON-cycle, 4, 8, and 12 ms later. The order in which the stimulation sites are activated exactly once during an ON-cycle is called a stimulation sequence, briefly sequence. For the RVS CR the sequence randomly changes for one ON-cycle to the next. In contrast, for the fixed CR the same sequence is maintained for all ON-cycles (**Figure 1E**). For the slowly varying sequences (SVS-n) CR a sequence is applied during n ON-cycles, before randomly switching to another sequence which is, in turn, used for the next n ON-cycles etc (Zeitler and Tass, 2015). **Figure 1F** illustrates SVS-4 CR, where each sequence is repeated four times before another sequence is drawn randomly. In the SVS CR protocols of this study we will only apply n = 100 consecutive repetitions of a sequence before

FIGURE 1 | Schematic of the spatio-temporally patterned stimulation protocols as used in this study. (A) The purely period multichannel stimulation (PPMS) simultaneously activates all stimulation sites at the same time instance within each stimulation ON-cycle. (B) The correlated multichannel noisy stimulation (CMNS) activates all stimulation sites simultaneously, but at different, random time instances, equally distributed within each stimulation ON-cycle. (C) The uncorrelated multichannel noisy stimulation (UMNS) activates the stimulation sites sequentially in a random order at different time instances within different stimulation ON-cycles. Stimulation onsets are equally distributed within each stimulation ON-cycle, where the random processes for the different channels are uncorrelated. (D) The rapidly varying sequences (RVS) CR stimulation activates the stimulation sites in a random temporal order at times n · Ts/4 (n ∈ {0, 1, 2, 3}) within each stimulation ON-cycles. (E) The fixed CR stimulation approach employs a fixed sequence during the entire CR-on period, and stimulation onsets times within the stimulation ON-cycles are at multiples of n · Ts/4 (n ∈ {0, 1, 2, 3}). (F) In this example, for the sake of illustration, the slowly varying sequences (SVS) CR stimulation repeats the first sequence four times, before a new sequence is drawn. Different stimulation sequences are randomly drawn with equal probability. For the remainder of this study each sequence is repeated 100 times and not just four times (SVS-100 instead of SVS-4). Stimulation onsets are as for the RVS CR at n · Ts/4 (n ∈ {0, 1, 2, 3}) within the stimulation ON-cycles. All stimulation protocols have four equidistantly distributed stimulation sites, stimulation ONand OFF- cycles of 16 ms, a total stimulation on period of 128 s, in which three ON-cycles are alternated by two OFF-cycles. No stimulation is applied during OFF-cycles. Stimulation ON- and OFF-cycles are separated by dashed vertical lines. A change of color indicates a change of the pattern of stimulus onsets compared to the previous ON-cycle.

we draw the next sequence. Therefore, we will use the term SVS instead of SVS-100 CR for the remainder of this work.

#### Simulation Details

A simulation contains four different simulation periods: first an initializing period of 2 s, then a 60 s period with STDP and without external stimulation (denoted as STDP-only period) which is followed by a stim-on period of 128 s in which the network receives an external stimulation and after withdrawal of the stimulation follows a stimulation off period (denoted as stim-off period) of 128 s. During the initializing period - the only period without STDP - a network is built by drawing random numbers for each of the N = 200 neurons from uniform distributions for the injected constant current I<sup>i</sup> ∈ [I<sup>0</sup> − 1<sup>I</sup> , I<sup>0</sup> + 1I], for the membrane potential V<sup>i</sup> ∈ [−65, +5] ms, for the time-varying ion gate variables x<sup>i</sup> ∈ [0, 1] and the synaptic variable s<sup>i</sup> ∈ [0, 1]. The initial synaptic weights cij are drawn from a Gaussian distribution with mean 0.5 µA/cm<sup>2</sup> and standard deviation 0.01 µA/cm<sup>2</sup> . During the initializing period the neuronal network evolves without influences of external stimulation signals or STDP.

For this selected initial distribution of the synaptic weights, during the STDP-only period the network can develop into a strongly connected network with strongly synchronized neuronal activity. The time at which the first stimulation signal is delivered to the network is defined as t = 0 s. During the stim-on period the stimulation signals are applied to the network as described in section Stimulation Implemented in the Model. After 128 s no stimulation signals are applied anymore and the evolution of the network is monitored for another 128s (stim-off period). After going through this whole process, the procedure is repeated from t = 0 s on for a different stimulation intensity, K, and/or for a different stimulation signal approach.

From previous studies (Zeitler and Tass, 2015, 2016) we know that different initial network conditions and different stimulation signal realizations have an influence on the antikindling effects. Therefore, we draw eleven different sets of initial values from the distributions as described above at the start of the initializing period. For each of these eleven different initial network conditions we generate one realization for each stimulation signal approach. In the remainder of this study the combination of a stimulation signal realization with one set of initial network conditions is referred to as sample. All simulations were executed in Matlab R2007a. The differential equations were solved by the built-in function ODE45 with a relative tolerance of 10−<sup>5</sup> .

### Data Analysis

In this computational study we came up with a larger set of acute and long-lasting effects of different stimulation protocols applied to the plastic neural network as described in section Neuronal Network. We are particularly interested in whether the different stimulation protocols might induce qualitatively different antikindling effects. In this section, we will discuss the methods used to investigate the neuronal connectivity at several levels, the synchronization and a quantification of the acute stimulation and acute after-effects, as well as the phase resetting and entrainment induced by the different stimulation protocols. Matlab R2015a was used for the data analysis and for plotting the results.

#### Connectivity

In this study, the synaptic weights can change according to the STDP-rule (see Equation 5). On the network level the dynamics of the synaptic connectivity is monitored by the synaptic weight averaged over all synapses within the network:

$$C\_{a\nu}(t) = N^{-2} \sum\_{i=1}^{N} \sum\_{j=1}^{N} \text{sign}\left(M\_{ij}\right) c\_{ij}\left(t\right),\tag{10}$$

where N is the number of neurons within the network, sign(Mij ) is negative for inhibitory synapses and positive for excitatory synapses (with Mij defined as in Equation 3), and cij is the synaptic coupling strength from neuron j to i. A decrease of Cav over time may indicate that there is mainly a decrease in the average excitatory synaptic weights or an increase in average inhibitory weights or a combination of both. To unravel the contributions of the excitatory and the inhibitory synaptic weights, we introduce the average excitatory synaptic weight

$$c\_{EE}(t) = N\_{EE}^{-2} \sum\_{i=1}^{N} \sum\_{j=1}^{N} \left[ \text{sign} \left( M\_{ij} \right) \right]\_+ c\_{ij}(t), \tag{11}$$

and the average inhibitory synaptic weight

$$c\_{II}(t) = N\_{II}^{-2} \sum\_{i=1}^{N} \sum\_{j=1}^{N} \left[ \text{sign} \left( -M\_{ij} \right) \right]\_+ c\_{ij}(t) \,, \tag{12}$$

where [z]+ stands for the half-wave rectification operation ([z]<sup>+</sup> = z if z > 0 and [z]<sup>+</sup> = 0 otherwise), NEE is the number of excitatory synapses, and NII is the number of inhibitory synapses within the whole neuronal network.

On the neuronal level the connectivity matrices are analyzed at the end of the stim-on and at the end of the stim-off period, since we are interested in the acute and the long-lasting effects. Instead of considering the cij-values, we first multiplied each cij-value by the sign-function of the Mexican Hat, sign(Mij) (see Equation 3 for Mij). This allows to recognize the type of each synapse in a color plot of the connectivity matrix (negative values for inhibitory and positive values for excitatory synapses). For each synapse eleven cij-values exist since each stimulation protocol is applied at stimulation intensity K to eleven different initial networks. By determining the median and the inter-quartilerange (IQR) of these eleven cij-values for all i, j ∈ {1, 2, . . . , N] we obtained a larger reduction of the amount of data. Unfortunately, due to the different initial network conditions the general connectivity pattern induced by a stimulation protocol does not straightforwardly reveal at which locations neurons are coupled mainly by bidirectional weak as opposed to unidirectional strong synapses.

To get a better idea of what happens due to a stimulation signal, we first sorted for each sample the cij(t) values such that the strongest synaptic weight between two neurons i and j is placed in the lower right triangle at location (i ′ ,j ′ ) = (min(i,j), max(i,j)) and the weaker synaptic weight between those two neurons i and j in the upper left triangle of the connectivity matrix at location (i ′ ,j ′ ) = (max(i,j), min(i,j)). This is repeated for all combinations of two neurons. After that the result is multiplied by sign(Mij). We call this newly defined matrix the sorted connectivity matrix. After calculating the median of the eleven sorted connectivity matrices the general pattern is more evident for all stimulation protocols and the IQRs provide a clearer picture than for the unsorted connectivity matrices. By sorting the data like this, one can determine from the median if the synapses are in general strong in both directions or just in one direction or maybe even weak in both directions between two neurons. Furthermore, the IQR shows which type of synapses and at which spatial location the largest differences occur as an effect of the different samples. Note that the location (i, j) in this sorted connectivity matrix does not indicate that neuron i is the postsynaptic neuron and neuron j the pre-synaptic one as is the case for the unsorted connectivity matrices, but that if i < j (lower right triangle of the sorted matrix) than c sorted ij = max cij,cji with cij, cji elements of the unsorted connectivity matrix. For the upper left triangle of the sorted matrix where i > j this means that c sorted ij = min cij,cji .

#### Synchronization

The effect of the external stimulation signals on the strongly synchronized neuronal activity is investigated on the network level by the order parameter R (Haken, 1983; Kuramoto, 1984) defined by

$$R\left(t\right)\exp\left[i\Phi\right] = N^{-1}\sum\_{j=1}^{N} \exp\left[i\varphi(t)\right] \tag{13}$$

where Φ(t) is the circular mean phase of the entire group of N neurons in the network, and

$$\varphi\_{\mathbf{j}}(t) = 2\pi (t - t\_{\mathbf{j},m}) / (t\_{\mathbf{j},m+1} - t\_{\mathbf{j},m}) \text{ for } t\_{\mathbf{j},m} \le t < t\_{\mathbf{j},m+1} \tag{14}$$

is a linear approximation of the phase of neuron j between its mth and (m + 1)-th spikes at spike timestj,<sup>m</sup> and tj,m+<sup>1</sup> (Rosenblum et al., 2001). The minimum value of R is zero and indicates a complete lack of in-phase synchronization, whereas its maximum value (R = 1) indicates perfect in-phase synchronization.

For our analysis, we have calculated R, Φ, and all ϕ<sup>j</sup> at each ms of the stim-on and stim-off period. In case an R-value is shown at a certain time instance t the R-values of the preceding 5 s period are averaged and denoted as Rav at t.

On a mesoscopic level the amount of synchronization of the k-th subpopulation can be defined as

$$
\overline{R}\_k^{\text{pre}} = < \ R\_k >\_{last\ 5\ s\ before\ time - ON}
$$

$$
\overline{R}\_k^{\text{on}} = <\ R\_k >\_{last\ 5\ s\ of\ time - ON}
$$

$$
\overline{R}\_k^{\text{off}} = <\ R\_k >\_{last\ 5\ s\ of\ time - OFF}
$$

were R<sup>k</sup> denotes the synchronization order parameter of the k-th subpopulation (k ∈ {1, 2, . . . , Ns}) as determined by

$$R\_k\left(t\right)\exp\left[i\Phi\_k\left(t\right)\right] = N\_k^{-1}\sum\_{j=1}^{N\_k} \exp\left[i\phi\_j\left(t\right)\right] \tag{15}$$

with N<sup>k</sup> = 49 the number of neurons within subpopulation k, and ϕj(t) the linear approximation of the phase of neuron j in subpopulation k at time t as determined by Equation (14). Using this definition, we can determine the acute stimulation effect on the k-th subpopulation by

$$1 - \frac{\overline{R}\_k^{on}}{\overline{R}\_k^{pre}} \tag{16}$$

and the acute after-effect on the k-th subpopulation by

$$1 - \frac{\overline{R}\_k^{off}}{\overline{R}\_k^{pre}} \tag{17}$$

A negative outcome indicates a synchronizing effect, a zero outcome indicates that there is no acute stimulation effect or after-effect, respectively, and a positive outcome means a desynchronizing effect within the k-th subpopulation due to application of that particular stimulation protocol.

#### Stimulus-Locked Phase Dynamics of a Subpopulation

To shed more light on the mechanisms of the different stimulation protocols and reveal, e.g., phase resetting or entrainment processes, we investigate the stimulus-locked dynamics on a mesoscopic scale. We do this by considering subpopulations of neurons as given by their proximity to the N<sup>s</sup> = 4 different stimulation sites located at neuron indices 25, 75, 125, and 175. This implies that each subpopulation k contains N<sup>k</sup> = N/N<sup>s</sup> −1 = 49 neurons, since the subpopulations are separated by a neuron which has an equal distance to the stimulation sites in the two neighboring subpopulations, e.g., the neuron at index 50 has the same distance to the stimulation site at neuron index 25 as well as to the stimulation site at neuron index 75 and is therefore excluded from subpopulation 1 as well as from subpopulation 2. The mean phase Φ<sup>k</sup> (t) of subpopulation k (k ∈ {1, 2, . . . , Ns}) is determined by Equation 15. We focus on the distribution of the stimulus-locked phase dynamics within a time window Wof 32 ms before and up to 32 ms after each of the L stimulation onsets t n k (n = 1, .., L) for each subpopulation k separately (Tass, 2003c)

$$\left\{\Phi\_k\left(t\_k^n + \Delta t\right) \left(\operatorname{mod} 2\pi\right)\right\}\_{n=1,\ldots,L}.\tag{18}$$

These distributions of the stimulus-locked phase dynamics of a subpopulation will be displayed in color plots as a function of 1t ∈ W. To quantify the amount of stimulus locking of the phase dynamics of subpopulation k, we use the resetting index Ek (t) (Tass, 2003c; see also Tallon-Baudry et al., 1996) as given by

$$E\_k \left(\Delta t\right) = \left| L^{-1} \sum\_{n=1}^{L} \exp\left[i\Phi\_k(t\_k^n + \Delta t)\right] \right|. \tag{19}$$

In case of a phase entrainment (a permanent stimulus-locking of the phase dynamics) E<sup>k</sup> (1t) results in a uniform distribution throughout the window W = [−32 , +32] ms. For a stimulusinduced phase reset, E<sup>k</sup> (1t) will be small in the period before stimulus onset (corresponding to a uniform phase distribution) and will increase after stimulus onset, reflecting the emergence of a unimodal phase distribution (Tallon-Baudry et al., 1996; Tass, 2003c). For a stimulus-locked disruption of a phase entrainment the phase distribution will be unimodal before stimulus onset and decrease due to stimulus onset. Since all our stimulation protocols have the same ratio of ON: OFF cycles, namely 3:2, we can split each set of the L stimulation onsets t n k (n = 1, .., L) for each subpopulation k into three subsets. The first subset contains only the L/3 stimulation onsets t n k (n = 1, 4, .., L − 2) within the first of the three consecutive ON-cycles, the second one contains the L/3 stimulation onsets t n k (n = 2, 5, .., L − 1) during the middle of each block of three consecutive ON-cycles and the last subset contains the L/3 stimulation onsets t n k (n = 3, 6, .., L/3) within the third ON-cycle of each three consecutive ON-cycles. By using these three subsets separately instead of the three subsets together, the corresponding three resetting indices can be determined in a similar way as in Equation 19. This detailed analysis shows how the phase reset and entrainment processes evolve depending on the rank order of the onset after the OFF-cycles.

#### Statistics

To test whether an increase of the median of the eleven Cav or Rav values obtained by stimulations with protocol A at stimulation intensity K<sup>A</sup> compared to stimulation protocol B applied at K<sup>B</sup> is statistically significant, we used the Matlab R2015a built-in left-sided Wilcoxon rank sum test with significance level α: [p, h] = ranksum(A,B, "alpha",0.05, "tail," "left," "method," "exact"). This test is equivalent to the Mann-Whitney U-test. The built-in right-sided Wilcoxon rank sum test tests if a decrease in medians is statistically significant. In this study, we use significance level α = 0.05 and n<sup>A</sup> = n<sup>B</sup> = 11 samples unless stated otherwise.

### RESULTS

#### Acute Effects

In this section, we study the acute stimulation effects of the different spatio-temporally patterned stimulation protocols at K = 0.25 by comparing their effects on the connectivity as well as the synchronization at the end of the stim-on period (t = 128 s).

External stimulation signals can change the network's connectivity (due to STDP) and the amount of synchronized neuronal activity. The control signal (no-stim) has no influence on the average synaptic weight, Cav (**Figure 2A**) and on the synchronization of the population activity (shown by the order parameter Rav in **Figure 2B**). For the same initial network conditions the other stimulation signals applied at intensity K = 0.25 show an acute reduction of Cav as well as of Rav (see results at t = 128 s in **Figures 2A,B**). Only the correlated multichannel noisy stimulation (CMNS) results in an increase of Cav and a small reduction of Rav. The boxplots in **Figures 2C,D** show that these acute effects are representative for all 11 samples: compared to the control signal all stimulation protocols induce a statistically significant decrease of Cav and Rav, except the CMNS which induces also a statistically significant decrease of Rav, but a statistically significant increase of Cav (left-sided Wilcoxon rank sum test with α = 0.05). The corresponding p-values are given in Supplementary Table 1.

The CMNS-induced increase of Cav is the result of a statistically significant increase of the average excitatory synaptic weight cEE in combination with a statistically significant decrease of the average inhibitory synaptic weight cII (see **Figures 3A,B** for one set of initial network conditions and **Figures 3C,D** for all 11 samples). The other stimulation protocols show an opposite behavior: the combination of a statistically significant decrease of cEE with a statistically significant increase of cII (**Figure 3**) explains the statistically significant decreases of Cav (**Figure 2C**).

According to the median of the unsorted connectivity matrices induced by the control protocol (no-stim) there are many strong excitatory (and inhibitory, respectively) synapses without a spatial pattern in relation to the stimulation sites at locations (25,25), (75,75), (125,125), and (175,175) (upper panel of **Figure 4A**). Note, the excitatory (respectively inhibitory) synapses have positive (respectively negative) values. The corresponding IQRs show that there are large differences between the synaptic weights induced by the different samples for almost all synapses (bottom panel of **Figure 4A**). By first sorting for each sample the cij(t = 128 s) values such that the strongest synaptic weight between two neuronsi and j is placed in the lower right triangle and the weaker synaptic weight between those two neurons i and j in the upper left triangle (see section Connectivity for more details) and then determining the median and IQR, it becomes clear that in general the control protocol results in only one strong synapse (|cij<sup>|</sup> ∼= 1) between two neurons and in the reverse direction the synaptic weight is approximately zero (upper panel of **Figure 4B**). The corresponding IQR shows that the (sorted) synaptic strengths are rather independent of the actual sample (bottom panel of **Figure 4B**). So, the general pattern is that due to the control signals the synaptic weights between any two neurons are unidirectional (strong in one direction, weak in the reverse direction). This is similar for the synaptic weights at the beginning of the stim-on period (t = 0 s, result not shown).

**Figure 4C** shows that if all N<sup>s</sup> = 4 stimulation sites are activated simultaneously and periodically (PPMS) more bidirectional strong inhibitory synapses exist at the end of the stim-on period and on the other hand some excitatory synapses are bidirectional and weak. Differences between the results induced by different samples are mainly found in the indices of the neurons which have bidirectional strong inhibitory synapses and in the indices of those neurons which have bidirectional weak excitatory synapses (**Figure 4C**).

In case all N<sup>s</sup> = 4 stimulation sites are simultaneously stimulated in a noisy manner (CMNS), the median of the sorted connectivity matrices shows that besides the induction of some bidirectional strong excitatory synapses the median is similar to the median of the control signals (**Figures 4B,D**). However, the IQRs vary: compared to the no-stim condition (**Figure 4B**), different samples with CMNS will induce bidirectional strong excitatory synapses between different neuronal locations and also the induced bidirectional weak inhibitory synapses differ in locations (**Figure 4D**).

If the stimulation sites are not activated simultaneously but sequentially and still noisy (UMNS) then the number of unidirectional strong excitatory neurons decreases in such a way that they exist between neighboring neurons and between neurons nearby the stimulation sites, but not between more distant neurons (**Figure 4E**). Further, all inhibitory synapses are bidirectional and strong. Differences between samples are mainly found in the size of the region with strong excitatory synapses nearby the stimulation sites and between neighboring neurons

value after withdrawal of the stimulation. (B) Stimulation effect on the synchronization of the population activity. All stimulation protocols induce a decrease of the amount of synchronization Rav, except the CMNS, which basically only causes fluctuations during the stim-on period. These fluctuations vanish after stimulation withdrawal. In contrast, all other stimulation protocols cause a long-lasting desynchronization. All stimulation protocols are applied to the same initial network conditions and with stimulation intensity K = 0.25 during the stim-on period (t = 0–128 s). The red horizontal bar represents the stim-on period. No stimulation signals are delivered during the subsequent 128 s stim-off period. (C) Boxplots of Cav (t = 128 s) for different stimulation protocols show a statistically significant decrease compared to the no-stim approach except for the CMNS, which induces an increase of Cav. (D) Boxplots of Rav (t = 128 s) show a statistically significantly desynchronization induced by the different stimulation protocols compared to the control condition (no-stim). Eleven samples (different combinations of initial network conditions and sequence orders) are used for each boxplot. The horizontal line within the box represents the median, the length of the box the IQR (middle 50%) and the whiskers below and above the box the first and last 25%. Outliers are defined as 1.5 times the length of the box below or above the box and are represented by open circles. P-values of the left-sided Wilcoxon rank sum test are given in Supplementary Table 1.

and in the occurrence of bidirectional strong excitatory synapses nearby the stimulation sites (**Figure 4E**).

To some extent, RVS CR is similar to UMNS, but stimulation onsets are equidistantly spaced at 0, 4, 8, or 12 ms after onset of the ON-cycle instead of at random time instances as for UMNS. The median and IQR of the sorted connectivity matrices induced by RVS CR (**Figure 4F**) is similar to the results induced by UMNS. Small differences are obtained for the RVS CR compared to the UMNS: the unidirectional strong excitatory neurons exist in a smaller neighborhood of each neuron as well as of the stimulation sites. These small differences are in agreement with the decrease of excitatory synaptic weights in **Figure 3C** and with the smaller Cav-values induced by RVS CR compared to those induced by UMNS in **Figure 2C**.

The fixed CR activates the stimulation sites sequentially at 0, 4, 8, and 12 ms after onset of the ON-cycle, while the sequence does not change during the stim-on periods. This repetition of sequence has several effects (compare **Figures 4F,G**). Most prominent is the fact that due to the fixed CR (**Figure 4G**) the neurons close to a stimulation site are not so strongly coupled as in case of the RVS CR (**Figure 4F**). Furthermore, in general, the neurons halfway between two stimulation sites are not strongly coupled any more with their direct neighbors. Due to the repetition of the sequence strong excitatory couplings between neurons surrounding two consecutively activated stimulation sites remain present. For the eleven different samples (each with another sequence), this leads to some large IQRs for couplings between more distant excitatory synapses. By changing now and then the sequence during the stim-on period (SVS CR), these large IQRs between more distant excitatory neurons disappear (**Figure 4H**).

In **Figure 2D** we have seen how the amount of synchronization of the complete network Rav, changes under influence of the different stimulation approaches at stimulation intensity K = 0.25. On a mesoscopic level we investigate the desynchronizing effect in the four subpopulations containing N<sup>k</sup> = 49 neurons near the stimulation sites. For each stimulation protocol **Figure 5A** illustrates at K = 0.25 the distribution of the N<sup>s</sup> · n = 44 determined acute stimulation effects. All stimulation approaches induce a statistically significant acute stimulation effect whereby the weakest effect is induced by the CMNS (right-sided Wilcoxon rank sum test, α = 0.05, n = 44; see Supplementary Table 2 for p-values). The order of the stimulation protocols in having a stronger desynchronizing

effect is clear for the macroscopic measure Rav: only RVS CR and fixed CR have a similar effect on Rav. On the mesoscopic level this order is slightly different: the PPMS turned out as good as RVS CR, while fixed CR caused a better desynchronization of the neuronal activity than RVS CR (compare **Figures 2D**, **5A**; onesided Wilcoxon rank sum test with α = 0.05; see Supplementary Table 1 respectively 2 for p-values corresponding by **Figure 1D** respectively **Figure 5A**).

**Figure 6** shows the raster plots and spike counts induced by the different stimulation protocols (K = 0.25) for the same samples as used in **Figures 2A,B**, **3A,B**, **4** during the last 100 ms of the stim-on period. Activating the stimulation sites simultaneously does not cause a pronounced desynchronization (compare the results for no-stim with K = 0.25 in **Figures 6A,B**). A sequential random activation of the stimulation sites can broaden the synchronized spike-volley (**Figure 6C**). RVS-CR stimulation counteracts in-phase synchronization, typically by causing cluster states, where the network forms several phaseshifted (synchronized) subpopulations (see e.g., **Figure 6D**). A more pronounced overall desynchronization is achieved by means of the fixed CR stimulation and the SVS-CR stimulation (e.g., **Figures 6E,F**).

The spike counts suggest that activating the stimulation sites simultaneously can result in stronger synchronization of the activity of the whole network, while sequential stimulation can divide the network in several synchronized, but mutually phaseshifted subpopulations, which in turn causes a pronounced overall (i.e., close to uniform) desynchronization, as reflected by Rav. Since each single stimulus synchronizes the nearby neurons, while desynchronizing the entire neuronal network, on a macroscopic scale the in-phase synchronization (and hence Rav) may vanish, while the order parameters of the different subpopulations may still attain high values. Put otherwise, the acute stimulation effect will be weaker on the mesoscopic than on the macroscopic level (e.g., for UMNS and RVS CR). For the PPMS and the CMNS, which activate all stimulation sites simultaneously, the order parameter of each subpopulation represents the order parameter of the macroscopic network quite well and, hence, the acute stimulation effect is comparable on the mesoscopic and macroscopic level.

Our analysis continues on the mesoscopic level to study the mechanisms by which the different stimulation protocols influence the synchronization. For this we use the cross-trial analysis, which investigates the subpopulations' phases timelocked to the corresponding stimulus onset, averaged over all stimulus onsets of stimuli delivered to a particular subpopulation during the stim-on period. For subpopulation 2 (comprising neurons 51-99), **Figures 7A–F** show a clear difference between the stimulation protocols with periodic delivery pattern, PPMS, fixed CR and SVS CR, and those which have no strictly periodic stimulus delivery pattern (CMNS, UMNS, and RVS CR). The latter protocols cause phase resets of the stimulated subpopulations: Before stimulus onset (for 1t < 0 m) the phase distributions are close to uniform, whereas after stimulus

between samples occur.

delivery a stereotypical restart of the subpopulation phase occurs, which is reflected by the emergence of a pronounced peak of the distribution (for 1t > 0 ms in **Figures 7B–D**). In contrast, in case of the periodic stimulus patterns, PPMS, fixed CR and SVS CR, a pronounced peak of the phase distribution increasingly vanishes in the absence of stimulation (for 1t < 0 m) and re-occurs after stimulus delivery (for 1t > 0 ms) (**Figures 7A,E,F**). The corresponding resetting indices E2(1t) show that the non-periodic stimulation protocols induce a phase reset (**Figures 7B–D**): Following stimulus onset, the initially quite homogeneously distributed subpopulation phase turns into a unimodal phase distribution. In contrast, for the periodic stimulation protocols, the resetting indices display a completely different time course: Starting at a large value, they first decrease, then re-increase due to the first stimulus (at 1t = 0 ms) and further increase due to the subsequent stimulus (1t ∼ 16 ms) (**Figures 7A,E,F**). More precisely, at 1t ∼ 16 ms in only 2/3 of all stimulation onsets there is indeed a stimulation, since the third ON-cycle of each block of three consecutive ON-cycles is not directly followed by an ON-cycle but instead by two OFFcycles. This implies that after three activations of the stimulation site within the subpopulation with an inter-stimulus-interval of 16 ms, the next inter-stimulus-interval is equal to 48 ms. Accordingly, by sorting the stimulation onsets according to their

order in the blocks of three consecutive ON-cycles, reveals the effect of the different stimuli. The first of the three stimuli destroys most of the phase entrainment which is present during the OFF-cycles and then builds up the entrainment up to a lower level than the initial entrainment (**Figure 7G**). The second of the three stimuli further increases the extent of phase entrainment (**Figure 7H**), and the third stimulus finally increases the phase entrainment to the initial level (**Figure 7I**). The maximum values of the resetting indices of the periodic stimulation protocols increase within a block of three consecutive stimuli (i.e., ONcycles) from one stimulus to the subsequent one, indicating a further increase of the phase entrainment between stimulated subpopulation and corresponding stimulus train (compare the maxima in **Figures 7G–I**). Accordingly, the effect of the three consecutive periodic stimuli on the phase builds up within each block. In contrast, each subsequent non-periodic stimulus causes a new phase reset (**Figures 7G–I**).

Sequential stimulation patterns (UMNS, RVS CR, fixed CR, SVS CR) cause a more pronounced reduction of synchrony and the mean synaptic weight on the network level (**Figure 2**) as well as on the subpopulation level (**Figures 3**, **5**) compared to simultaneous stimulation patterns (PPMS, CMNS). On the level of the individual neurons the sequential stimulation patterns induce strong bidirectional inhibitory synapses and subpopulations with strong unidirectional excitatory coupling of the neurons near the stimulation sites, whereas the simultaneous stimulation protocols only induce some strong bidirectional inhibitory synapses and no spatial pattern of the strong unidirectional excitatory synaptic weights (compare **Figures 4E–H** with **Figures 4C,D**).

Another important aspect refers to the sequential arrangement of stimulation sequences. Repeating a sequence many times in a row (by fixed CR or SVS CR) causes a more pronounced reduction of Cav than random variations of the sequences (by UMNS or RVS CR) (**Figure 2**). Sufficient repetition of stimulation sequences results in a stronger reduction of cEE (**Figure 3**) caused by the fact that less neurons are coupled within a subpopulation around each stimulation site together with a weaker synaptic strength for those who are coupled (**Figure 4G,H** compared with **Figures 4E,F**). Although on the network level no clear difference in the amount of desynchronization is observed (RVS CR and fixed CR induced similar Cav –values; **Figure 2D**), on the subpopulation level the repetitive sequences induce a stronger acute stimulation effect (**Figure 5**). This suggests that different mechanisms of action may cause different types of macroscopic desynchronization: desynchronization between subpopulations which themselves can still be highly synchronized as opposed to a more overall, uniform desynchronization, affecting the subpopulations as well. According to the raster plots in **Figure 6**, a more uniform desynchronization is typically observed as a result of repetitive sequence CR (fixed CR and SVS CR). **Figure 7** shows that on the subpopulation level there is an entrainment between the stimulation signal and the subpopulation activity in the networks exposed to repetitive sequence CR. In contrast, phase resets are the salient mechanism of those CR variants without sequence repetition.

Intriguingly, all stimulation protocols except the CMNS protocol induce a decrease of cEE and an increase in cII compared to the no-stim protocol (**Figure 3**). In contrast, the CMNS protocol induces exactly the opposite (**Figure 3**), resulting in an increase of Cav, in contrast to all other active stimulation protocols (**Figure 2C**). Since also the median of the sorted connectivity matrix induced by CMNS is similar to the one of the control (no-stim) signal and not for the other stimulation protocols (**Figures 4B,D**), the CMNS seems to be the best candidate for a sham stimulation protocol for sensory CR stimulation. While the stimulation shares some perceptual features with CR stimulation, the acute effects of CMNS on connectivity and desynchronization are minimal.

corresponding neuron. Each red diamond shows the stimulation onset of the corresponding stimulation site. Spike counts for the last 100 ms of the stim-on period

show how many neurons fire within each time interval of 1 ms. (B–F) as (A) for CMNS (B), UMNS (C), RVS CR (D), fixed CR (E), and SVS CR (F).

### Long-Term Effects

It is a key goal for non-invasive neuromodulation techniques to cause long-lasting, sustained effects that persist after cessation of stimulation. Sufficiently pronounced long-lasting effects may open up the possibility to deliver stimulation only regularly or occasionally e.g., for a few hours only, to maintain substantial relief. Therefore, we are particularly interested in the effects of the different stimulation protocols after withdrawal of stimulation. In this section, we study these long-term effects at the end of the stim-off period (t = 256 s) unless stated otherwise. Again, the stimulations during the stim-on period were applied at intensity K = 0.25.

For the same initial network configuration, a stimulation epoch of duration t = 128 s can have different acute and longterm effects on the average synaptic weight Cav, depending on the stimulation protocol selected (**Figure 2A**). To focus on the longterm effects, the distributions of the Cav (t = 256 s) at the end of the stim-off period are shown in **Figure 8A** for the different stimulation protocols. The SVS CR protocol induces the greatest reduction of Cav (t = 256 s), while the CMNS induces even a small increase. These long-term effects (**Figure 8A**) are qualitatively comparable with the acute effects (**Figure 2A**) and statistically significant. None of the stimulation protocols induces Cav effects statistically equivalent to the no-stim protocol. However, CMNS induces Rav (t = 256 s) values similar to the control signal (no-stim) (**Figure 8B**), and there are no statistically significant differences. In contrast, the SVS CR stimulation induces the smallest Rav values at t = 128 s (acute effect; **Figure 2B**) as well as at t = 256 s (long-term effect; **Figure 8B**). Therefore, the CMNS protocol turns out to be the best candidate for sham stimulation. The p-values obtained by the left-sided Wilcoxon rank sum tests for the long-term effects as shown in **Figures 8A–D** are given in Supplementary Table 3.

The strongest Cav is induced by the CMNS protocol (**Figure 8A**) and can be disentangled in the strongest average excitatory synaptic weight cEE of the tested protocols (**Figure 8C**) in combination with the weakest inhibitory synaptic weight cII of the tested protocols (**Figure 8D**). The weakest Cav as shown in

resetting index of subpopulation 2, E2, is shown by a white curve superimposed to each phase distribution diagram. The resetting index is also calculated for the distributions of Φ2(1t), averaged across the 1,600 stimulation onsets within the first (G), the second (H), and the third (I) of the three consecutive ON-cycles for the different stimulation protocols. K = 0.25 for all panels.

**Figure 8A** is induced by the SVS CR stimulation protocol and is a combination of the weakest cEE (**Figure 8C**) and the strongest cII (**Figure 8D**) of the tested protocols. In general, a weaker Cav value obtained by a particular stimulation protocol compared to another protocol can be explained by a significantly reduced cEE and increased cII of the first stimulation protocol except for the comparison between fixed CR and SVS CR. These two protocols induce a similar cII and thus only the stronger reduced cEE induced by the SVS CR stimulation contributes to the smaller Cav value compared to the fixed CR stimulation (**Figures 8A,C,D**).

Comparing the medians and IQRs of the unsorted connectivity matrices at t = 256 s (long-term effect) for CMNS (**Figure 9A**) with those for the no-stim protocol at t = 128 s (**Figure 4A**) show similar patterns. Note that the patterns do not change for the no-stim protocol at different times e.g., at t = 0 or 256 s (results not shown). At the end of the stim-off period the medians and IQRs of the sorted connectivity matrices of CMNS (**Figure 9D**) and of the no-stim protocol (**Figure 9B**) look also similar and are comparable to those of the no-stim protocol at the end of the stim-on period (**Figure 4B**): strong unidirectional synaptic connections, with only a very small difference between the sorted connectivity matrices of different samples. Compared to the acute effect induced by CMNS (**Figure 4D**) the network has lost its strong bidirectional excitatory synapses during the stim-off period (**Figure 9D**) for all samples.

In case the stimulation onsets of the simultaneous stimulation patterns are not random as for CMNS but periodic (PPMS),

causes no decrease of Cav. (B) Boxplots of Rav (t = 256 s) show a statistically significant desynchronization induced by the different stimulation protocols compared to the control signal (no-stim) except for the inert CMNS. All stimulation protocols, except CMNS, induce a long-lasting decrease in the average excitatory synaptic weight cEE (C) and a long-lasting increase in the average inhibitory synaptic weight cII (D). Each boxplot represents the results of 11 samples. The horizontal line within the box represents the median, the length of the box the IQR (middle 50%), whereas the whiskers below and above the box indicate the first and last 25%. Outliers are defined as 1.5 times the length of the box below or above the box and are represented by open circles. p-values of the left-sided Wilcoxon rank sum test are given in Supplementary Table 3. K = 0.25 for all panels.

bidirectional strong inhibitory synapses exist at the end of the stim-off period and most excitatory synapses are unidirectional and strong, while some of them are bidirectional and weak (**Figure 9C**). Indices of the neurons which have bidirectional strong inhibitory synapses or bidirectional weak excitatory synapses can differ for other samples (**Figure 9C**).

For the CR protocols, the medians of the sorted connectivity matrices have lost their connectivity patterns of the shortrange excitatory synapses during the stim-off period (compare **Figures 9E–H** with **Figures 4E–H**). Mainly the CR protocols without repetition (**Figures 9E,F**) show a loss of some bidirectional strong inhibitory synapses, which were formed during the stim-on period (**Figures 4E,F**). For the CR protocols the IQRs of the sorted connectivity matrices are different at the end of the stim-off period compared with those at the end of the stim-on period (compare the bottom panels of **Figures 9E–H** with those of **Figures 4E–H**): at the end of the stim-off period large IQR are not only found for the strongest synapse between neighboring neurons (diagonals) or nearby stimulation sites, but without a clear spatial pattern in the lower right triangle large IQRs can be found for the excitatory synapses and in the upper left triangle for the inhibitory synapses influenced mainly by UMNS or RVS CR (bottom panels of **Figures 9E–H**) .

Although on the macroscopic neural network level there is no significant difference between the desynchronization induced by the CMNS and by the no-stim protocol (**Figure 8B**), on the subpopulation level, there is a statistically significant acute aftereffect (**Figure 5B**): An increase in the amount of synchronization is induced by CMNS. All other stimulation protocols have a stronger desynchronizing effect than the no-stim protocol, whereby the acute after-effects of the RVS, fixed and SVS CR are increased compared to their acute stimulation effects (**Figures 5A,B**). These increases are statistically significant (see Supplementary Table 2 for the p-values of the right-sided Wilcoxon rank sum test with significance level α = 0.05).

Accordingly, we conclude that sequential activation of the stimulation sites also induces more pronounced long-term antikindling effects except for UMNS, which tends to induce better long-term anti-kindling effects than PPMS, but the improved medians are not statistically significant (network level see **Figures 8A,B**; subpopulation level see **Figures 5B**, **8C,D**). At the end of the stim-off period the results induced by consecutively repeating each sequence many times (fixed CR and SVS CR) are still better than without repetition (UMNS and RVS CR) on the network level (**Figures 8A,B**), as well as on the subpopulation level (**Figures 8C,D** for the connectivity; **Figure 5B** for the desynchronizing effect).

Even at the end of the stim-off period the CMNS protocol results in the strongest cEE and the weakest cII, and therefore gives rise to the largest Cav value of all stimulation protocols

(**Figures 8A,C,D**). The long-term Rav shows no difference with the control signals (no-stim; **Figure 8B**), but it does show a synchronizing effect on the subpopulation level, which implies that within the subpopulations the amount of synchronization has increased compared to the beginning of the stim-on period (**Figure 5B**). Despite these small differences the median and IQR of the sorted connectivity matrices induced by the no-stim and CMNS protocols appear to be similar. From all the investigated stimulation protocols at K = 0.25 the CMNS protocol turns out to be the one which induces the most similar results as the control signal (no-stim), despite small but statistically significant differences.

## Robustness Against Stimulation Intensity

In this section, we investigate acute and long-term effects elicited by stimulation intensities weaker than K = 0.25. From the previous sections, we can conclude that a small difference in stimulation protocols might result in completely different acute as well as long-term effects. For instance, the simultaneous and noisy stimulation (CMNS) does not decrease Cav as the simultaneous and period stimulation (PPMS) does. By the same token, at weaker intensities (K = 0.10, 0.15, and 0.20) PPMS induces statistically significantly smaller values of Cav and Rav values than CMNS (**Figures 10A,B**). See Supplementary Tables 1, 3, 5–7 for the corresponding p-values of the left-sided Wilcoxon rank sum test with significance level α = 0.05. The basic difference in the stimulation protocol feature between UMNS and fixed CR is the same as between PPMS and CMNS, namely noise or periodic stimulation. By comparing the pvalues for the UMNS and the fixed CR it follows that for K = 0.10 UMNS induces smaller Cav and Rav values than fixed CR (Supplementary Table 5), but for K = 0.25 it is just the other way around (Supplementary Tables 1,3). Since for the investigated K-range PPMS always induces smaller Cav and Rav values than the CMNS, we can only conclude that the effect of noise stimulation also depends on how the stimulation sites are activated: simultaneously or sequentially. In case of sequential stimulation it depends strongly on K.

Fixed CR and SVS CR differ in the number of different sequences applied to the network. For fixed CR one sequence is applied 4,800 time during the on-period, whereas for SVS CR each sequence is applied 100 times before the next sequence is applied. For weak stimulation intensities up to K = 0.15 there is no difference in the anti-kindling effects. In contrast, at higher intensities changing the sequence from time to time, as for SVS CR, decreases the Cav and Rav values even more (**Figures 10E,F**) than the fixed CR does.

Cross-trial analysis shows that already for K = 0.10 a weak phase reset is observed for CMNS, UMNS, and RVS CR and a weak entrainment followed by a stimulus-locked disruption of the weak phase entrainment for PPMS, fixed CR, and SVS CR (**Figure 11**). Increasing the intensity also increases the amount of entrainment as well as the strength of the phase reset (e.g., **Figures 11A,D,G**). During the first ON-cycle, PPMS, fixed CR as well as SVS CR stimulation destroy the entrainment of the subpopulation phase-dynamics with the stimulation signal in the entire investigated K-range. During the second ON-cycle, this entrainment is partly recovered by the stimulation except for stimulations at K = 0.15. In the latter case the entrainment is destroyed even further. During the third ON-cycle entrainment is restored in the entire K-range (K ∈ {0.10; 0.15; 0.20; 0.25}), where the amount of entrainment increases with increasing K. The phase rest seems to increase slightly from the first to the second and, finally, to the third ON-cycle, but clearly increases with increasing K (K ∈ {0.10; 0.15; 0.20; 0.25}).

### DISCUSSION

We studied acute and long-lasting effects of six different stimulation protocols and compared the observed effects on a plastic neural network with a no-stimulation control condition. While sharing the same average rate of stimuli per channel, the tested stimulation protocols differ with respect to their amount of periodicity as opposed to randomness, both between ON-cycles and between stimulation sites. One of the tested stimulation protocols, CMNS, turned out not to induce desynchronization. In fact, comparing CMNS with the no-stimulation control condition showed that CMNS is nearly inert. More precisely, during stimulation (**Figure 2**), CMNS caused a significant increase of the strength of the mean synaptic connectivity by 49% compared to the no-stimulation control condition. Intriguingly, during CMNS the overall synchrony nevertheless was 8% smaller than for the no-stimulation control. Remarkably, there was hardly any long-term post-stimulation effect of CMNS compared to the no-stimulation control condition (**Figure 8**): While the mean synaptic weight increased significantly, but only slightly by 7%, the spontaneous (i.e., stimulation-free) synchronization did not significantly differ between the CMNS and the nostimulation condition. Whether a slight, but significant increase of the mean synaptic weight might be relevant in a nonspontaneous context, where the network is, e.g., subjected to other types of stimuli, remains to be tested. Ultimately, clinical studies will provide the necessary tests.

In this study, we have shown that stimulation protocols, that differ by just one, putatively minor feature, may cause massively different anti-kindling effects, robustly over a range of stimulation intensities as well as for different samples at the end of the stim-on period and also at the end of the stimoff period (see e.g., **Figure 10**). Stimulating simultaneously or sequentially has a big influence on the synchronization and connectivity of the network at all levels (e.g., compare CMNS vs. UMNS in **Figures 2C,D**, **3C,D**, **4D,E**, **5**, **6B,C**, **8**, **9D,E**, **10**). Another influential feature of some stimulation protocols is repetition (compare e.g., PPMS vs. CMNS or RVS vs. fixed CR), which goes along with a different dynamical stimulation mechanism: entrainment caused by repetition vs. phase reset otherwise (**Figures 7**, **10**). In this study, we restricted ourselves to cycle durations of 16 ms, which is slightly above the intrinsic firing period of the individual neurons (14 ms). For a more pronounced mismatch of ON-cycle duration and firing periods, stimulus-locked entrainment of subpopulations may probably become more difficult if not completely impossible. Another drawback of the SVS and fixed CR is that they have to be applied at slightly stronger intensities than UMNS and RVS to be effective. However, SVS and fixed CR may, in principle, induce more pronounced long-term anti-kindling effects than UMNS and RVS CR (see e.g., **Figures 8A,B**).

Compared to all stimulation protocols tested in this study, the anti-kindling effects induced by the CMNS protocol are most similar to the control (i.e., no-stim) protocol (**Figures 2C,D**, **3C,D**, **4D,E**, **5**, **6B,C**, **8**, **9D,E**, **10**), although the similarity is not always statistically significant. The CMNS protocol is also the only protocol which induced opposite effects on the connectivity, both on the network as well as subpopulation level, compared to all other active protocols (**Figures 3**, **8**). Although the implementation of the stimulation in the model used here is for sensory stimulation, we expect similar results for electrical stimulation. For comparison between electrical and sensory stimulation effects and their qualitative similarities in our model, see Popovych and Tass (2012).

In addition, we also performed simulations which extended the finished trials with the CMNS protocol (K = 0.20) by adding a second, additional 128 s lasting stim-on period at t = 256 s with the RVS CR protocol as well as with the SVS CR protocol, each at K = 0.20. Anti-kindling effects induced at t = 384 s (i.e., at the end of the second stim-on period) as well as at t = 512 s (the end of the second stim-off period) were statistically significantly similar with the anti-kindling effects at t = 128 s

period (B). (C,D) For UMNS stimulation onsets are random and uncorrelated between different sites, whereas for RVS CR they are restricted to only four equidistant moments within the ON-cycle. RVS CR induces a stronger reduction of Cav and Rav than UMNS for most K-values at t = 128 s (C) as well as at t = 2 56 s (D). (E,F) Fixed CR applies the same sequence during the stim-on period, whereas SVS CR randomly draws a new sequence after 100 consecutive repetitions of a sequence. For K = 0.10 and K = 0.15 there are no statistically significant differences in the distributions of Cav and Rav at t = 128 s (E) and t = 256 s (F). However for K = 0.15 and K = 0.20 SVS CR reduces the medians and IQRs of Cav and Rav more than the fixed CR does. Each boxplots represents eleven samples. p-values of the left-sided Wilcoxon rank sum test are given in Supplementary Tables 1, 3, 5–7.

(the end of the first and only stim-on period) and t = 256 s (the end of the first and only stim-off period) for the RVS and SVS CR only protocols applied during only one stim-on period as shown in **Figures 10C–F** (data not shown). Accordingly, from a computational standpoint there is no reason to assume that application of CMNS sham stimulation might render subsequent delivery of RVS or SVS CR stimulation ineffective.

Apart from the stimulation-related aspects, our findings are relevant with respect to the assessment of changes of synchronization and synaptic connectivity patterns and their mutual interrelation. Although the macroscopic measures Cav and Rav were often strongly correlated, an increase in Cav did not necessarily imply an increase in Rav. For example, in Supplementary Table 4, we show that for the CMNS protocol

consecutive ON-cycles. (C) as (A) for the third of the three consecutive ON-cycles. (D) as (A) for K = 0.15. (E) as (B) for K = 0.15. (F) as (C) for K = 0.15. (G) as (A) for K = 0.20. (H) as (B) for K = 0.20. (I) as (C) for K = 0.20. The RVS and SVS CR results are shown by a dashed curve to improve visibility of the UMNS and fixed CR results.

applied at K = 0.25 during the stim-off period a statistically significant decrease in Cav is combined with a statistically significant increase in Rav. Accordingly, it is not sufficient to exclusively monitor the connectivity in our model. Rather, we also have to determine the amount of synchronization. By a similar token, comparable amounts of macroscopic synchronization/desynchronization, as assessed by the order parameter, may differ on the mesoscopic, i.e., subpopulation level (see e.g., **Figure 7**). Accordingly, there is not just one type of (e.g., uniform) desynchronization. In summary, the analysis of both synchrony and synaptic connectivity on macroscopic as well as mesoscopic network levels may further the development of both active and inactive (sham) stimulation protocols.

In the field of drug development, a placebo treatment is realized by delivering a substance with no active therapeutic effect (Friedman et al., 2015). Accordingly, placebo effects can, for instance, be assessed with a parallel group design by comparing a placebo group with a natural history (i.e., no-treatment) group (Wager and Atlas, 2015). According to our computational results, CMNS is a promising candidate for a sham (i.e., inactive) stimulation protocol in the field of desynchronizing multichannel stimulation. In the present study, we did not aim at developing a biophysical, microscopic model. Rather, we used a minimal model, equipped with robust spontaneous multistable dynamics, comprising synchronized and desynchronized states, that served as testbed for different stimulation protocols to generate first and experimentally testable hypotheses. By a similar token, several previous computational studies in the field of CR stimulation were carried out in minimal models and led to a number of clinically significant predictions, e.g., concerning cumulative effects and stimulation intensity (Hauptmann and Tass, 2009; Lysyansky et al., 2011), that were verified in preclinical and clinical studies (Tass et al., 2012b; Adamchic et al., 2014; Wang et al., 2016). In the same manner, the present

computational study yields the testable hypothesis that CMNS might serve as sham stimulation protocol. Accordingly, the effects of CMNS should be tested in pre-clinical and, in particular, in clinical studies in order to disentangle CMNS effects from placebo effects in the best possible way. For instance, in a phase 1 (first in man) study feasibility and tolerability could be tested. In addition, in a phase 2 study an assessment of effects should be performed, possibly in comparison to a no-stimulation control and/or an active control group. Obviously, this adds to the complexity of the clinical development of neuromodulation treatments.

Furthermore, CMNS may be useful in clinical studies focusing on revealing predictive EEG markers for optimizing stimulation parameters of desynchronizing neuromodulation interventions, see Adamchic et al (2017). CMNS does neither cause substantial acute nor long-lasting effects. Nevertheless, CMNS might have unspecific EEG effects, e.g., on brain rhythms and/or brain areas less important to the disease related network dynamics. In this way, CMNS might help to separate EEG responses related to core stimulation effects from concomitant EEG responses. In addition, CMNS may also be helpful in pre-clinical studies to elucidate mechanisms by which stimulation protocols cause a desynchronization. Furthermore, CMNS might also help to reveal mechanisms that might be related to, but go beyond desynchronization, such as therapeutic rewiring (Tass and Majtanik, 2006) or neuroprotective effects (Musacchio et al., 2017).

Wherever appropriate and possible, sham procedures and double-blind protocols should be developed to scrutinize specific effects of neuromodulation interventions by adequate clinical trials and rule out placebo effects. However, placebo effects should not just be considered as a nuisance, requiring cumbersome clinical study protocols. Rather, the mechanisms underlying different placebo effects (Benedetti et al., 2011) could actually be specifically exploited to better neuromodulation techniques. For instance, similar to conditioned immunomodulation (Metal'nikov and Chorine, 1926; Ader and Cohen, 1975; Ader, 2003), one could condition specific desynchronization stimulation delivered invasively and causing long-lasting effects with unspecific non-invasive, sensory stimuli (Tass, 2011).

For the clinical implementation of multichannel sham stimulation protocols, one should take into account possible side effects related to the technical generation of the stimuli. For instance, in the field of transcranial magnetic stimulation (TMS) the current gold standard for sham TMS appears to be the use of a shielded coil, generating characteristic stimulusrelated auditory stimuli, but no magnetic brain stimuli, together with surface electrodes for skin stimulation, mimicking magnetic skin stimulation (Duecker and Sack, 2015). Intriguingly, sham TMS may have specific side effects (Duecker and Sack, 2013, 2015). Obviously, TMS is not just a purely magnetic stimulation modality, but may constitute a compound stimulation approach which may cause specific effects caused by stimuli of different modality (Duecker and Sack, 2013, 2015). For the clinical development of multichannel sham stimulation protocols such aspects might be relevant, for instance, when vibrotactile mulitchannel stimulation causes auditory and possibly other sensory side effects.

This computational study is a first step for the development of a sham stimulation protocol for multichannel desynchronizing stimulation techniques. For comparison, for the development of CR stimulation, in computational studies predominantly minimal models were used (Tass, 2003a,b; Tass and Majtanik, 2006; Hauptmann and Tass, 2007, 2009; Lysyansky et al., 2011; Popovych and Tass, 2012; Zeitler and Tass, 2015, 2016), as opposed to biophysically realistic models (Ebert et al., 2014). These computational studies revealed non-trivial predictions, e.g., concerning the emergence of long-lasting, sustained (Tass and Majtanik, 2006) as well as cumulative (Hauptmann and Tass, 2009) effects and concerning the amplitude of the stimulation amplitude (Lysyansky et al., 2011). These predictions were verified in pre-clinical (Tass et al., 2012b; Wang et al., 2016) and clinical studies (Tass et al., 2012a; Adamchic et al., 2014; Syrkin-Nikolau et al., 2018). The computational predictions were used as hypotheses for the design of the corresponding study protocols. Analogously, the development of sham stimulation protocols requires a combined effort, comprising computational, pre-clinical, and clinical studies. The effect of a sham stimulation protocol may depend on the type of the stimulated neurons, the target area of the stimulation and the mechanism of the stimulation. Accordingly, future studies should also use other neuronal network models, e.g., network of FitzHugh-Rinzel bursting neurons (Rinzel, 1987; Izhikevich, 2001). By the same token, sham stimulation should ideally be inactive also in the presence of additional features and mechanisms, such as synaptic noise (Destexhe et al., 2003), propagation delays (Madadi Asl et al., 2017) as well for different stimulation mechanisms, e.g., excitatory vs. inhibitory stimulation (Popovych and Tass, 2012). Hence, future computational studies should take into account these refinements, too.

## AUTHOR CONTRIBUTIONS

PT came up with the initial ideas for this work; MZ designed and performed the simulations and analyzed the data; MZ and PT discussed findings and interpretations, and wrote the manuscript.

## ACKNOWLEDGMENTS

PT gratefully acknowledges the generous research funding support (donation) from the John A. Blume Foundation.

## SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fphys. 2018.00512/full#supplementary-material

## REFERENCES


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Zeitler and Tass. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# SQI Quality Evaluation Mechanism of Single-Lead ECG Signal Based on Simple Heuristic Fusion and Fuzzy Comprehensive Evaluation

#### Edited by:

Zhidong Zhao<sup>1</sup>

\* and Yefei Zhang<sup>2</sup>

Alexey Zaikin, University College London, United Kingdom

#### Reviewed by:

Mohammad Hasan Imam, American International University-Bangladesh, Bangladesh Paolo Melillo, Università degli Studi della Campania "Luigi Vanvitelli" Caserta, Italy

#### \*Correspondence:

Zhidong Zhao zhaozd@hdu.edu.cn Yefei Zhang valora.Zhang@gmail.com

#### Specialty section:

This article was submitted to Computational Physiology and Medicine, a section of the journal Frontiers in Physiology

Received: 07 February 2018 Accepted: 25 May 2018 Published: 14 June 2018

#### Citation:

Zhao Z and Zhang Y (2018) SQI Quality Evaluation Mechanism of Single-Lead ECG Signal Based on Simple Heuristic Fusion and Fuzzy Comprehensive Evaluation. Front. Physiol. 9:727. doi: 10.3389/fphys.2018.00727 <sup>1</sup> Hangdian Smart City Research Center of Zhejiang Province, Hangzhou Dianzi University, Hangzhou, China, <sup>2</sup> School of Communication Engineering, Hangzhou Dianzi University, Hangzhou, China

\*

For both the acquisition of mobile electrocardiogram (ECG) devices and early warning and diagnosis of clinical work, high-quality ECG signals is particularly important. We describe an effective system which could be deployed as a stand-alone signal quality assessment algorithm for vetting the quality of ECG signals. The proposed ECG quality assessment method is based on the simple heuristic fusion and fuzzy comprehensive evaluation of the SQIs. This method includes two modules, i.e., the quantification and extraction of Signal Quality Indexes (SQIs) for different features, intelligent assessment and classification. First, simple heuristic fusion is executed to extract SQIs and determine the following SQIs: R peak detection match qSQI, QRS wave power spectrum distribution pSQI, kurtosis kSQI, and baseline relative power basSQI. Then, combined with Cauchy distribution, rectangular distribution and trapezoidal distribution, the membership function of SQIs was quantified, and the fuzzy vector was established. The bounded operator was selected for fuzzy synthesis, and the weighted membership function was used to perform the assessment and classification. The performance of the proposed method was tested on the database from Physionet ECG database, with an accuracy (Acc) of 97.67%, sensitivity (Se) of 96.33% and specificity (Sp) of 98.33% on the training set. Testing against the test datasets resulted in scores of 94.67, 90.33, and 93.00%, respectively. There's no gold standard exists for determining the quality of ECGs. However, the proposed algorithm discriminates between high- and poor-quality ECGs, which could aid in ECG acquisition for mobile ECG devices, early clinical diagnosis and early warning.

Keywords: electrocardiogram(ECG), quality assessment, signal quality indexes (SQIs), heuristic fusion, fuzzy comprehensive evaluation

## INTRODUCTION

With the wide application of mobile ECG in the fields of financial safety, security monitoring, medical insurance, and data confidentiality, ECG recording devices are not limited to professional training staff.

From the perspective of mobile ECG collection, most of the available ECG recording devices lack real-time feedback about the signal quality. Thus, it is difficult for non-professionals to collect high-quality ECG signals. The mobile recorders record the ECG in normal lifestyle condition. So there is movement of the recorders electrodes (Clifford et al., 2006). The existence of noise prevents the accurate detection of important clinical characteristics, thus reducing the quality of the ECG signal (Tob'on and Falk, 2015).

Regarding the clinical application of ECG signals, an ECG signal contains abundant physiological and pathological information, which can help clinical staff observe the early warnings of diseases and make a diagnosis. For instance, for the diagnosis of cardiovascular disease (World Health Organization, 2016), the high costs of primary health care make followup treatment unaffordable. To circumvent this problem, many countries transmit real-time ECG data recorded by patients to clinical experts to diagnose patients. In addition to the professional judgment of clinical experts, the accuracy of remote diagnosis should reduce the number of low-quality ECGs sent to experts, so we need to determine whether the quality of recorded ECGs is sufficient. In addition, in the ICU early warning system, the high false positive rate of monitors is caused by noise and data loss (Lawless, 1994). A survey demonstrated that only 10% of the alerts are related to treatment (Allen and Murray, 1996; Chambrin et al., 1999), which increases the workload of ICU staff and ultimately desensitize them.

Therefore, establishing a suitable assessment mechanism that divides the signal results into several different levels is particularly important.

The technology for signal quality assessment is gradually emerging. Currently, numerous research reports regarding quality assessment technology for ECG signals are available. However, the gold standard of ECG quality has not been evaluated to date. According to the existing research results, the research ideas for evaluation methods of ECG signal quality can be roughly divided into the following four methods: the waveform shape of the time domain signal, the characteristics of each frequency band of the frequency domain signal, signal quality characteristics extracted using the nonlinear tool, and signal quality parameters.

In this paper, we discuss the correlation between ECG signal quality and noise and ECG waveform characteristics to obtain accurate assessment results using simple rules and complex classification techniques. Our method of assessing ECG quality is divided into two steps:

### Step1: Feature Extraction

Based on the noise characteristics and ECG waveform features, six quality assessment parameters are extracted and quantified: the matching degree of R peak detection, power spectrum distribution of QRS wave, variability in the R-R interval, kurtosis, skewness, and baseline relative power. The advantages and disadvantages as well as the accuracy, sensitivity, and specificity of each of the quality assessment parameters of ECG quality are obtained by conducting quality assessment on the six quality assessment parameters. Using a simple heuristic fusion operation, the best accuracy based on the combination of 2–6 parameters is selected.

### Step2: Intelligent Classification

Using the fuzzy comprehensive evaluation method that combines Cauchy distribution, rectangular distribution, and trapezoidal distribution, the membership of the signal to be evaluated is calculated based on the parameters of the logical combination of the best accuracy selected in step 1. By establishing the fuzzy vector and choosing bounded calculation Sub-fuzzy synthesis, the ECG signal is divided into the evaluation level set V = {E, B, U} through the principle of weighted membership decision-making division.

The above algorithm evaluation is based on a single-lead ECG signal. If ECG signals are collected for multiple leads that are independent of each other, each channel can be processed separately.

### METHODS AND MATERIALS

#### Databases

To determine the parameters of the fuzzy comprehensive evaluation (correlation matrix R, weight vector A, synthesis operator) and verify the effectiveness of our algorithm.In this work, the ECG signals were obtained from ECG database (Physionet ECG database). We adopt two of these databases, Physionet/Cinc Challenge 2017 (Physionet, 2017), marked as database D1 and Physionet/Cinc Challenge 2011 (Physionet, 2011), marked as database D2.

The database D1 is single-lead ECG records, which is coincided with the ever-evolving mobile measurement and wearable measurement methods, supplied by AliverCor, which was annotated by clinical experts and categorized into one of four groups (i.e., normal rhythm, atrial fibrillation, other rhythm, and noisy recordings). We randomly selected 150 groups of normal rhythm, 150 groups of noisy recordings, and the length of each group is 9,000 data points.

The database D2 is standard 12-lead ECG recordings, in which the quality of ECGs were reviewed and examined by a group of annotators with varying amounts of expertise in ECG analysis. In our present work, we have used only lead II(Set-a) and randomly selected 150 groups of acceptable, 150 groups of unacceptable, and the length of each group is 9,000 data points.

To maximize contrast and create a balanced database, a 10-fold cross-validation method was used 10 times to reduce the generalization error in the training set (Zhihua, 2016), **Figure 1** below shows the schematic diagram of 10-fold cross validation.

The database is first divided into 10 equally sized mutually exclusive subsets: D = D<sup>1</sup> ∪ D<sup>2</sup> ∪ ... ∪ D10. D<sup>i</sup> ∩ D<sup>j</sup> is empty. Each subsection maintains the consistency of the data distribution, which is obtained through hierarchical sampling from D. Then, every nine subsets of the union is considered as a training set, and the remaining subset serves as the test set. Thus, 10 training and test sets are obtained. Thus, we can conduct 10 training and testing assessments and obtained the mean of 10 final test results. In this paper, the mean of 10 test sets is used as the result of cross validation to evaluate the performance of the algorithm. Numerous methods are available to divide database D into 10 subsets. To reduce the difference caused by different sample divisions, we randomly apply different division methods by repeating the process 10 times. The resulting assessment is the average of the 10 replicates of the 10-fold crossvalidation results, the mean of the obtained results is the final performance indicator. Obviously, the stability and fidelity of the cross-validation method evaluation results are considerably improved compared with those of the commonly used singledivision leave-one-out method.

#### Method Outline

Two methods for fusing the signal quality information were compared. The first step is based on simple heuristic fusion. Through the ECG waveform characteristics, time-frequency characteristics, and the time-frequency characteristics of the noise source, we propose six quality evaluation parameters and adjust the reasonable range of SQI. ECG quality [accuracy (Acc), sensitivity (Se) and specificity (Sp)] was analyzed using a simple logical combination to obtain the best combination of signal quality indexes (SQIs) U = {u1, u2, u3, ...}. The second step applied fuzzy comprehensive evaluation, which represents a more accurate assessment and classification of U = {u1, u2, u3, ...}. A schematic representation of the proposed method is presented in **Figure 2**.

### Signal Quality Indexes (SQIs) Matching Degree of R Peak Detection qSQI

We next describe an evaluation index for ECG signal recognition ability. For a complete ECG signal, the R wave has the maximum amplitude and most obvious characteristic, so the existence of the ECG signal is often identified by R-wave detection. Therefore, using different algorithms to perform R-wave detection on the same ECG signal, the result is compared and analyzed to estimate the quality of the signal. In this paper, Hilbert and dynamic adaptive threshold based on R-wave detection (algorithm flow shown in **Figure 3** and denoted as Algorithm 1), wavelet transform (This process is denoted as Algorithm 2) are used to compare the same ECG signal detection results. As for wavelet transform, the input ECG signal was decomposed by discrete wavelet transform (DWT) with four layers, and the wavelet coefficients for each layer were obtained. The R point is the singularity of the ECG waveform. The extraction of the R

point is completed by the correspondence between the singularity of the signal and the positive and negative modulus maximum of the wavelet coefficients to the zero point (Zhen et al., 2008).

Any R-wave detection algorithm has certain shortcomings, which may lead to false positives. Accordingly, the ECG signal quality is evaluated by the same R-wave matching degree of the

two algorithms mentioned above. The following equation is used to obtain the QRS wave R peak detection matching degree:

$$qSQI = \frac{2N}{N\_a + N\_b} \tag{1}$$

where N indicates the correct number of R waves detected by the two algorithms and N<sup>a</sup> and N<sup>b</sup> denote the numbers of R waves measured by Algorithms 1 and 2, respectively. The identification criteria of qSQI are given as follows:

$$ECG \begin{cases} optimal, & qSQI > 90\%; \\ susceptibility, & qSQI \in [60\%, 90\%] \\ \text{unqualified, } qSQI < 60\% \end{cases} \tag{2}$$

#### Power Spectrum Distribution of QRS Wave pSQI

We next describe an evaluation index of QRS wave quality (Li and Clifford, 2006). A heartbeat cycle is mainly composed of a P wave, QRS complex wave, T wave, and other important eigenvectors, of which the QRS wave accumulates ∼99% of the energy of the ECG signal and is the most stable. The energy of the QRS wave is concentrated in a frequency band centered at 10 Hz and is 10 Hz in width. Therefore, pSQI is mathematically defined as follows:

$$pSQI = \frac{\int\_{f=5Hz}^{f=15Hz} P\left(f\right) df}{\int\_{f=5Hz}^{f=40Hz} P\left(f\right) df} \tag{3}$$

Spectrum analysis is performed, and the energy of the two bands is calculated. The numerator represents the energy of the QRS wave, and the denominator represents the overall energy of the ECG signal.

If EMG interference exists, the high-frequency component increases, and pSQI decreases. The identification criteria of pSQI are given as follows:

$$\text{ECG} \begin{cases} \text{optimal} & p \text{SQI} \in [l\_1, l\_2]; \\ \text{subpicious}, & p \text{SQI} \in [l\_3, l\_1]; \\ \text{unqualified} & p \text{SQI} > l\_2, or \text{p} \text{SQI} < l\_3; \end{cases} \tag{4}$$

l<sup>1</sup> and l2, which represent the lower and upper limits, respectively, vary slightly with the heart rate. Based on the experimental training set, we adjusted l<sup>1</sup> and l<sup>2</sup> as follows:

$$\text{Heart rate} \begin{cases} \in [60 \text{bmp}, 130 \text{bmp}], l\_1 = 0.5, l\_2 = 0.8; l\_3 = 0.4; \\ \in [130 \text{bmp}, 160 \text{bmp}], l\_1 = 0.4, l\_2 = 0.7; l\_3 = 0.3; \end{cases} \tag{5}$$

#### Variability in the R-R Interval cSQI

We next describe an evaluation index of normal and stable heart rhythm. The ECG signal is a periodic signal, and the interval of R-R interval is periodic. The calculation of cardiac cycle (single-cycle ECG length) is related to the heart rate, which differs depending on the exercise state. The measurement is very sensitive to motion artifacts in the human body; the ECG signal is collected in the active state. We need to ensure that the heart rhythm has a normal ECG signal to avoid affecting the clinical diagnosis. In addition, the presence of noise artifacts leads to reduced QRS detector performance. When an artifact is present, the QRS detector underperforms by either missing R-peaks or erroneously identifying noisy peaks as Rpeaks. The above two problems will lead to a high degree of variability in the distribution of R-R intervals; therefore, the coefficient of the variation of R-R interval proposed by Hayn (Hayn et al., 2012) was used to calculate the variability of R-R intervals:

$$cSQI = \frac{\hat{\sigma}\_{RR}}{\hat{\mu}\_{RR}}\tag{6}$$

where µˆ RR and σˆRR are the empirical estimates of the mean and standard deviation of the distribution of the R-R intervals within a segment of ECG. The identification criteria of cSQI are given as Equation (7), where the threshold was determined empirically.

$$\begin{array}{ll} \text{\_{cEGG}} & q \text{SQI} < 0.45; \\ \text{\_{subpounds}} & q \text{SQI} \in [0.45, 0.64]; \\ \text{\_{unquailified}} & q \text{SQI} > 0.64 \end{array} \tag{7}$$

#### Skewness sSQI and Kurtosis kSQI

We next describe an evaluation index of the de-noising effect of three disturbing noises, which is defined as follows:

$$\text{resQI} = \nu\_{\text{3}} = \frac{E\left\{(\mathbf{x} - \boldsymbol{\mu}\_{\text{x}})^{\text{3}}\right\}}{\sigma^{\text{3}}} \tag{8}$$

$$kSQI = \nu\_4 = \frac{E\left\{(\alpha - \mu\_x)^4\right\}}{\sigma^4} \tag{9}$$

The third and fourth standardized moments of a signal are measures of signal symmetry and Gaussianity, respectively. The central limit theorem indicates that random uncorrelated processes tend to have Gaussian distributions, such as thermal noise. Conversely, correlated signals tend to exhibit non-Gaussian distributions. The fourth standardized moment of a distribution, kurtosis, measures the relative peakedness of a distribution with respect to a Gaussian distribution. However, outliers will cause the asymmetric distribution of the signal, and the skewness is high. In this equation, µ<sup>x</sup> and σ are the mean and standard deviation of the signal, respectively.

**Figure 4** shows different quality ECG signals and their kurtosis and skewness values. Comparing Example 1 with the other three sets of signals, the noisy ECG signal has a smaller kurtosis value but a different skewness distribution. Example 2 contains a large amount of high-frequency noise. The kurtosis value is very low. However, given the nature of the noise, the

distribution is approximately symmetric, yielding a low skewness value. Therefore, skewness is less robust to noise than kurtosis, and we only use kurtosis in the ensuing noise detection. The kSQI in each case is described as follows:


The identification criteria of kSQI is given as follows:

$$\text{ECG} \begin{cases} \text{optimal} & k \text{SQI} > 5; \\ \text{unqualified,} & k \text{SQI} \le 5; \end{cases} \tag{10}$$

#### The Relative Power in the Baseline basSQI

We next describe an evaluation index of the de-noising effect of baseline drift (Zhihua, 2016). basSQI is difficult to filter,


The best performing SQI indicator is shown in bold and underlined.

TABLE 2 | Single-lead classification using combination of SQIs.


The results is performed on Database D1. The best result of SQIs combinations (Database D1) is shown in bold and underlined.



The results is performed on Database D2. The best result of SQIs combinations (Database D2) is shown in bold and underlined.

but its presence greatly impacts late pathological judgment and identification, as shown in **Figure 5**, which gives an example of baseline for a high-quality ECG sample (upper plot, basSQI = 0.966) and low-quality ECG sample (lower plot, basSQI = 0.5) obtained from Set-a of the PhysioNet/CinC 2011 database (Silva et al., 2011). Therefore, it is necessary to evaluate its de-noising effect, as defined below:

$$basSQI = \frac{1 - \int\_{f}^{f} = {}^{1Hz}\_{0Hz} P \left(f\right) df}{\int\_{f}^{f} = {}^{40Hz}\_{0Hz} P \left(f\right) df} \tag{11}$$

If no baseline drift interference is noted, the basSQI value is close to 1. A low basSQI means that the power within the band [0, 1Hz] is abnormally high with respect to the power in the [0, 40Hz] interval, which is likely to be caused by an abnormal shift in the baseline. The identification criteria of basSQI are given as follows:

$$ECG \begin{cases} \textit{optimal}, & \textit{basSQI} \in [0.95, 1]; \\ \textit{suspicions}, & \textit{basSQI} \in [0.9, 0.95]; \\ \textit{unqualified}, & \textit{basSQI} < 0.9; \end{cases} \tag{12}$$

## Data Fusion Approaches

#### Simple Heuristic Fusion of the SQIs

After the analysis in the previous section, five SQIs are reserved. The simple logic classifier model was built on database D, and the performance of the individual SQIs was evaluated. Then, we studied the contribution of each SQI to the classification performance and removed SQIs with an accuracy (Acc) <75%. Then, we evaluated of all other possible combinations of SQIs (pairs, triplets, etc.). Ten-fold cross-validation was performed on database D to assess the performance of the predictive model. Possible fusion equations were constructed in an ad hoc manner as follows:

$$\begin{aligned} \text{(SQI1, SQI2)} &: \text{ECG is} \\ & \begin{cases} \text{Excell(E)}, & \text{'optimal'} = 2; \\ \text{Barchy accepitalable(B)}, & \text{'aspicion's'} = 2 \text{ or} \\ & \text{'optimal'} = 1, \text{'s subpicous'} = 1; \end{cases} \\ & (\text{SQI1}\_1, \text{SQI2}\_2, \text{SQI3}\_1): \text{ECG is} \begin{cases} \text{'s:} & \text{'optimal'} \ge 2, \text{'unqualified} = 0; \\ \text{Barchy accepitalable(B)}, & \text{'simqualities} = 0; \end{cases} \\ & \begin{cases} \text{'s:} & \text{'simquable(B)}, \text{'} = 2, \text{'unqualified} = 2 \text{ or} \\ & \text{'uspidied(U)}, & \text{'unqualified} = 1; \end{cases} \\ & (\text{SQI1}\_1, \text{SQI2}\_2, \text{SQ1}\_3, \text{SQ1}\_4): \text{ECG is} \begin{cases} \text{'s:} & \text{'optimal'} \ge 3, \text{'unqualified} = 0; \\ \text{'s:} & \text{'simquable(E)}, \text{'} \ge 3 \text{ or} \\ & \text{'unaguchi}(U), & \text{'unqualified} \ge 3 \text{ or} \\ & \text{'unqualified} = 2, \text{'usquically} = 2; \end{cases} \\ & (\text{SQI1}\_1, \text{SQI2}\_2, \text{SQI1}\_3, \text{SQI1}\_4, \text{SQI1}\_5): \text{ECG is} \begin{cases} \text{'s:} & \text{'simquable'} = 4, \\ & \text{'simquable'} = 3; \end{cases} \\ & \begin{cases} \text{'s:} & \text{'simquable'} = 4, \\ \text{$$

 ′unqualified′ = 3,′ suspicious′ ≥ 1 or ′unqualified′ = 2,′ suspicious′ ≥ 2 or ′unqualified′ = 1,′ suspicious′ = 4; (16) Where the ECG quality corresponding to the number distribution of "optimal," "suspicious," and "unqualified" is arbitrary and set empirically through trial and error.

Although these coefficients could be optimized, it is unlikely that the logic is optimal. Thus, an exhaustive search of possible logical combinations and thresholds was not performed.

Multiple SQI metrics quantify different characteristics, and the simple fusion of the SQIs classifies the signal quality of each ECG into three levels: excellent (E), barely acceptable (B), and unacceptable (U). We obtained the best combination of quality assessment parameters U = {u1, u2, u3, ...} by comparing the accuracy(Acc), sensitivity (Se) and specificity (Sp) of the different logical combinations. The three indicators are defined as follows:

Se = TP/ (TP + FN) (17)

$$\text{Sp} \, = \text{ TN} / (\text{TN} + \text{FP}) \tag{18}$$

$$A\alpha \varepsilon = (T\mathbf{N} + \mathbf{T}\mathbf{P}) / (T\mathbf{P} + \mathbf{T}\mathbf{N} + F\mathbf{N} + F\mathbf{P}) \tag{19}$$

TP (true case) indicates the number of acceptable ECG signals correctly counted. TN (true negative example) indicates the number of unacceptable ECG signals correctly counted. FP (false positives) indicates the number of acceptable ECGs under error statistics. FN (false positives) indicates the number of unacceptable ECGs that were counted as errors.

**Table 1** shows the performance of five SQIs in ECG quality assessment. The table clearly shoes that qSQI and pSQI best distinguish between records of good and bad quality (the results obtained from the 300 sets in database D1 and D2). Then, the system is trained with all possible combinations of SQIs, using 10-fold cross-validation method repeated 10 times to train database D1 and D2, and merged with the upper (13)–(16). The results for the best pair, triplet, etc. of SQIs combinations are summarized in **Table 2** and **Table 3**. Analysis of the table clearly reveals that as SQIs increase, the accuracy rate exhibited a slowly increasing trend. As for database D1, the best accuracy was obtained when considering all SQIs ((Acc)D<sup>1</sup> = 87.00%). However, compared with 4 SQIs (with higher sensitivity, (Se4)D<sup>1</sup> = 94.67% is superior to (Se5)D<sup>1</sup> = 87.33%), the accuracy difference is negligible. As for database D2, it shows the best precision when only considering qSQI, pSQI, kSQI and basSQI. Accordingly, U = {u1, u2, u3, u4} = qSQI, pSQI, kSQI, basSQI is the best combination for evaluation factor aggregation in the fuzzy comprehensive evaluation mechanism.

#### SQI Quality Evaluation Mechanism Based on Fuzzy Comprehensive Evaluation

We selected the best SQIs combination in simple heuristic fusion, and the SQI quality evaluation mechanism based on fuzzy comprehensive evaluation was established. Fuzzy comprehensive evaluation is based on fuzzy mathematics by applying the principle of fuzzy relational synthesis, quantifying some undefined and unquantifiable factors, and using a number of factors to evaluate the affair level of a comprehensive evaluation of a method (Fengbiao, 2000).The fuzzy data fusion technology has mature applications in speech analysis (Song et al., 2016), image analysis (Wenqing and Yongjun, 2016), traffic network, and power grid risk assessment (Deng et al., 2017). The specific steps are as follows:

#### **First, determine the evaluation factor aggregation U**

For the evaluated object, select the main factors that reflect the evaluation object, measure with corresponding index, and form the evaluation factor aggregation U. The ECG signal's evaluation factor aggregation is U = {u1, u2, u3, u4} = qSQI, pSQI, kSQI, basSQI .

#### **Second, determine the rating hierarchy V**

For each evaluation factor, determine a number of levels. In this paper, the quality of ECG signal is divided into excellent (E), barely acceptable (B), and unacceptable (U). The evaluation rating set is V = {v1, v2, v3}.

#### **Then, establish the evaluation matrix R**

Analyze the membership function rij of each factor u<sup>i</sup> to the rating level v<sup>j</sup> , and obtain the single factor evaluation result of the ith factor: r<sup>i</sup> = (ri1,ri2,ri3). After the multiple single factor evaluation, a fuzzy matrix R is formed. In this paper, fuzzy matrix R, which has 4 factors and 3 evaluation levels, is described as follows:

$$R = \begin{bmatrix} r\_{11} & r\_{12} & r\_{13} \\ r\_{21} & r\_{22} & r\_{23} \\ r\_{31} & r\_{32} & r\_{33} \\ r\_{41} & r\_{42} & r\_{43} \end{bmatrix}$$

#### **Next, determine the weight vector W**

According to the various factors for the evaluation of the importance of the object, give the appropriate weight, which is marked as W = (w1,w2,w3,w4), Pw<sup>i</sup> = 1. The W in this paper is based on 100 sets of experimental training to adjust the experimental training set.

#### **Then, assess fuzzy synthesis S**

S = W◦R, S = (s1,s2,s3). Different operator symbols ◦ correspond to different fuzzy comprehensive evaluation models (Zimmermann, 2011).

#### **Finally, make a decision**

According to the assessment needs of the appraisers to process S, obtain the results.

The single factor evaluation process for each factor u<sup>i</sup> is shown in **Figure 6** below.

Therefore, with a rating hierarchy V, the key step of single factor evaluation is to calculate the membership function rij and determine the fuzzy operator symbols ◦ . After determining the single factor evaluation, the key step in multifactorial evaluation is the choice of weight vector W and decision-making methods.

#### **Determine the membership function of each single factor evaluation**

Single factor evaluation of qSQI. Assuming that q is the value of matching degree of R peak detection, q ∈ [0, 100], we construct the membership function of its quality level (E, B, and U) as UqH q , UqI q , and UqJ q , respectively. Due to the matching degree of R peak detection's performance level and because its corresponding evaluation object values approximate a Cauchy distribution, we choose the Cauchy distribution function to serve as the membership function of qSQI.

#### (a).UqH q

Based on the understanding of formula (1), the greater the value of q is, the greater the membership of H is. Therefore, we select the increasing half of the Cauchy distribution.

$$U\_{qH}\left(q\right) = \begin{cases} 0, & q \le a\\ \frac{1}{\left[1 + \left[\alpha\left(q - a\right)\right]^{-\beta}\right]}, & q > a \end{cases} \tag{20}$$

Specifically, α, β > 0. In practice, we often utilize β = 2.

Analysis Equation (20), take a = 80. If q is not >80, the membership function to H is zero. If q is > 80, there is a nonzero membership for H. The greater the q value is, the greater the membership for H is. When q = 100, UqH q = 1, thus improving Equation (20):

$$U\_{qH}\left(q\right) = \begin{cases} 0, & 0 \le q \le 80\\ \frac{1}{\left[1 + \left[a\left(q - 80\right)\right]^{-\beta}\right]}, & 80 < q < 90\\ \frac{\chi}{100}, & 90 \le q \le 100 \end{cases} \tag{21}$$

In addition, to ensure the continuity of UqH q to calculation a, lim x→90<sup>−</sup> 1 n <sup>1</sup>+[α(q−80)] −2 <sup>o</sup> <sup>=</sup> 0.9, yielding <sup>α</sup> <sup>=</sup> 0.3. Therefore, UqH q is as follows:

$$U\_{qH}\left(q\right) = \begin{cases} 0, & 0 \le q \le 80\\ \frac{1}{\left[1 + \left[0.3\left(q - 80\right)\right]^{-2}\right]}, & 80 < q < 90\\ \frac{x}{100}, & 90 \le q \le 100 \end{cases} \tag{22}$$

(b).UqJ q 

Considering the decreasing half of the Cauchy distribution

$$U\_{q^f}(q) = \begin{cases} 1, & q \le a\\ \frac{1}{\left[1 + \left[\alpha(q-a)\right]^\beta\right]}, & q > a \end{cases} \tag{23}$$

Specifically, α, β > 0. If a = 55, β = 2, then UqJ (60) = 0.5, yielding α = 0.2.

$$U\_{q^f}(q) = \begin{cases} 1, & q \le 55\\ \frac{1}{\left\{1 + \left(\frac{q - 55}{5}\right)^2\right\}}, & 55 \le q \le 100 \end{cases} \tag{24}$$

(c).UqI q 

Considering the Cauchy distribution directly, a = 75, α = 1/7.5 . The membership function is calculated as follows:

$$U\_{qI(q)} = \frac{1}{\left\{1 + \left(\frac{q - 75}{7.5}\right)^2\right\}}\tag{25}$$

When assessing the matching degree of R peak detection, we calculate the value of qSQI according to Equations (22), (24), and (25). We can obtain qSQI single factor evaluation results: r<sup>1</sup> = (r11,r12,r13).

Single factor evaluation of pSQI. This evaluation is similar to the structure of qSQI. According to the power spectrum distribution of pSQI's identification criteria (4), its performance level and its corresponding evaluation object value exhibit a trapezoidal distribution. Therefore, the trapezoidal distribution function is adopted as the membership function of pSQI. Assuming that p is the value of the power spectrum distribution, p ∈ [0, 1], we obtain the membership function of pSQI quality level UpH p , UpI p , and UpJ p :

$$U\_{pH}\left(\boldsymbol{\rho}\right) = \begin{cases} 0, & \boldsymbol{\chi} \le 0.25\\ 0.1\left(\boldsymbol{\chi} - 0.25\right), & 0.25 < \boldsymbol{\chi} < 0.35\\ 1, & \boldsymbol{\chi} \ge 0.35 \end{cases} \tag{26}$$

$$U\_{\mathcal{P}^I}(\boldsymbol{\rho}) = \begin{cases} 1, & \boldsymbol{\chi} < 0.15\\ 0.1 \left( 0.25 - \boldsymbol{\chi} \right), & 0.15 \le \boldsymbol{\chi} \le 0.25\\ 0, & \boldsymbol{\chi} > 0.25 \end{cases} \tag{27}$$

$$U\_{pI}\left(\boldsymbol{p}\right) = \begin{cases} 0, & \boldsymbol{x} < 0.18\\ 25\left(\boldsymbol{x} - 0.18\right), & 0.18 \le \boldsymbol{x} < 0.22\\ 1, & 0.22 \le \boldsymbol{x} < 0.28\\ 25\left(0.32 - \boldsymbol{x}\right), & 0.28 \le \boldsymbol{x} < 0.32\\ 0, & \boldsymbol{x} \ge 0.32 \end{cases} \tag{28}$$

We calculate the value of pSQI according to Equations (26)– (28), and pSQI's single factor evaluation results can be obtained: r<sup>2</sup> = (r21,r22,r23).

Single factor evaluation of kSQI. According to the identification criteria (10) of Kurtosis kSQI, its performance level and its corresponding evaluation object value exhibits a rectangular distribution. Accordingly, we choose the rectangular distribution function to be the membership function of kSQI. We calculate the value of kSQI, and the result of single factor evaluation r<sup>3</sup> = (r31,r32,r33) is obtained as follows:

$$\begin{cases} \; \text{if } kSQI > 5, \; r\_3 = (1, 0, 0) \\ \; \text{if } kSQI \le 5, \; r\_3 = (0, 0, 1) \end{cases} \tag{29}$$

Single factor evaluation of basSQI. This evaluation is the same as that reported for qSQI. For the analysis of Equation (12), the baseline relative powers performance level and corresponding evaluation object value approximate a Cauchy distribution. We select the Cauchy distribution function to construct UbH b , UbI b , and UbJ b . We calculate the value of basSQI according to Equations (30)–(32), and basSQI's single factor evaluation results can be obtained: r<sup>4</sup> = (r41,r42,r43).

$$U\_{bH}\left(b\right) = \begin{cases} 0, & 0 \le b \le 90\\ \frac{1}{\left[1 + \left[0.8718\left(b - 90\right)\right]^{-2}\right]}, & 90 < q < 95\\ \frac{\chi}{100}, & 95 \le q \le 100 \end{cases} \tag{30}$$

$$U\_{bf}\left(b\right) = \begin{cases} 1, & q \le 85\\ \frac{1}{\left\{1 + \left(\frac{b-85}{5}\right)^2\right\}}, & 85 < q \le 100 \end{cases} \tag{31}$$

$$U\_{bI}\left(b\right) = \frac{1}{\left\{1 + \left(\frac{b-92}{2.5}\right)^2\right\}}\tag{32}$$

After repeating the single factor evaluation four times, we obtain the fuzzy matrix R = [r<sup>1</sup> r<sup>2</sup> r<sup>3</sup> r4] T .

#### **Determine the weight vector W**

Different evaluation factors have different effects on the quality of ECG signals. Therefore, the selection of weight coefficients will have a great influence on the final quality assessment results. In this paper, different sets of weight vectors are selected to compare the four factors, which are verified according to 10 replicates of the 10-fold cross-validation test in the Database D1 and D2. The statistic is presented in **Figure 7**. Under different weight values of different SQI decision values, the accuracy of the quality of the assessment of ECG also differs. When the ratios of the four were set as follows (0.4, 0.4, 0.1, 0.1), whether the database D1 or D2, the accuracy of the ECG quality assessment under different test set was relatively high, with minimal fluctuation.

#### **Determine the fuzzy operator**

The principle of fuzzy comprehensive evaluation is fuzzy transformation. Numerous types of operation modes of fuzzy transformation are available. The commonly used fuzzy synthesis operators are classified as the following types:


Considering the role of reflection, the first step is more appropriate for a multiplicative operation. In contrast, from a comprehensive point of view, it is appropriate to use the "boundedness and" operation to ensure the full use of all aspects of the information provided by the fuzzy vector R. In this paper, we need to synthesize the four SQIs' indicators to evaluate the effect of ECG quality, so we use the operator M (·, ⊕), which is also known as a bounded operator.

#### **Determine the decision-making methods**

After fuzzy synthesis, the vector S = (s1, s2, s3) of the fuzzy comprehensive evaluation is obtained, which provides abundant information. In this paper, we need to weigh the four single-factor evaluations for each tester to obtain the numerical result of rating class V. Therefore, further processing is needed. Commonly used methods include the principle of maximum membership degree, the principle of weighted membership degree, and the fuzzy vector single-value method. We choose the principle of weighted

TABLE 4 | Performances of simple heuristic fusion of the SQIs and fuzzy comprehensive evaluation.


The best performing algorithm (on the independent Database D1) is indicated in bold.

TABLE 5 | Performances of simple heuristic fusion of the SQIs and fuzzy comprehensive evaluation.


The best performing algorithm (on the independent database D2) is indicated in bold.

average decision-making, which is expressed as follows:

$$V = \frac{\sum\_{j=1}^{3} s\_j^2 \cdot j}{\sum\_{j=1}^{3} s\_j^2} \tag{33}$$

After the above four-step parameter setting, we establish the SQI quality evaluation mechanism based on fuzzy comprehensive evaluation and obtain the final evaluation rating set V as follows:

$$\text{ECG} \begin{cases} \text{Excellent (E), } \nu \le 1.50; \\ \text{'Barely acceptible (B), } 1.50 < \nu < 2.40; \\ \text{'Unaccepted (U), } \nu \ge 2.40. \end{cases} \tag{34}$$

Then, for any ECG signal to be evaluated, the SQI quality evaluation mechanism based on fuzzy comprehensive evaluation can be used to obtain the ECG quality assessment results (shown in **Figure 8**) according to the above formula (33).

After the ECG quality assessment, the results were analyzed:


reevaluate the ECG quality. If pSQI or qSQI are unqualified, recollect the tester's ECG.

3. If B, ECG quality assessment should be performed again. If the result is E, the signal is treated as in method one. Otherwise, treated as in method two.

#### RESULTS AND DISCUSSION

The results of the comparison of Fuzzy comprehensive evaluation with the simple heuristic fusion of the SQIs with Database D test are shown in **Tables 4**, **5** below:

For simple heuristic fusion of the SQIs, when the number of SQIs increases from 4 to 5, the accuracy of the database D1 and D2 is not well optimized, even in its sensitivity (Se) and specificity (Sp). These values are not increasing but are decreasing. These findings indicate that the new evaluation parameter cSQI contains information that is complementary to the original (qSQI, pSQI, kSQI, basSQI), which affects the evaluation of ECG quality. Therefore, the selection (qSQI, pSQI, kSQI, basSQI) is more reasonable. Compared with simple heuristic fusion of the SQIs, although the same number of SQI indicators is used to quantify the different characteristics of ECG signals, the accuracy of database D1 and D2 is improved after it is synthesized by fuzzy comprehensive evaluation.

vth2).

TABLE 6 | Contrast tabulation of experimental results for different quality evaluation algorithms.


Best results are highlighted. The results of this article are underlined.

Simultaneously, the above **Tables 4**, **5** shows that the database D2 shows a better accuracy than D1. Using the database D2, we varied the value of v above for which data were considered to be excellent (E) or barely acceptable (B) quality and calculated the receiver operating characteristic (ROC) curve (**Figure 9**). The value v that yielded the best classification accuracy was vth<sup>1</sup> = 1.50, vth<sup>2</sup> = 2.40 (in the ROC curve, they were normalized to vth<sup>1</sup> = 0.15, vth<sup>2</sup> = 0.24), which resulted in an accuracy of 91.67% (275 correctly classified out of 300) on the database D2, using the threshold vth<sup>1</sup> . Then adding the threshold vth<sup>2</sup> , the accuracy on the test set was found to be 94.67% (284 correctly classified out of 300).

To perform a more extensive and accurate comparative performance evaluation, the base performance of the proposed system is compared with the four existing algorithms, (Johannesen, 2011; Clifford et al., 2012; Martínez-Tabares et al., 2012; Shahriari et al., 2017), all of which adopted the Database D2, used single-lead ECG signal, made the comparison more persuasive. The experimental results are presented in **Table 6**.


Compared with these two studies, we propose the SQI Quality Evaluation Mechanism Based on Fuzzy Comprehensive Evaluation, with only 4 SQI indexes, all of which are simply calculated, and obtain a good accuracy.


## REFERENCES


Compared with these two algorithms, we use the SQI index to quantify the quality of ECG which increased the readability of this algorithm, made it easy to be understood, and obtains an ideal accuracy. Therefore, the algorithm proposed in this paper has some advantages compared with other algorithms reported in the literature.

### CONCLUSION

We have described an effective system (with an accuracy of 92.67% on database D1 and 94.67% on database D2) that could be deployed as a stand-alone signal quality assessment algorithm to vet the clinical utility of ECG signals. Applications range from determining the quality of ECG signal collected to false alarm suppression. Moreover, the algorithm presented here is quite general and could be retrained and applied to any periodic or quasi-periodic signal, such as contraction signals.

Future work should focus on methods for expanding the feature space and on the further optimization of feature fusion.

## AUTHOR CONTRIBUTIONS

ZZ and YZ developed the concept of this review and wrote the manuscript.

## FUNDING

This research was supported by the National Natural Science Foundation of China (Grant No. 61571173), the Welfare Project of the Science Technology Department of Zhejiang Province (Grant No. 2015C31086) and the Smart City Collaborative Innovation Center of Zhejiang Province.

### ACKNOWLEDGMENTS

The authors would like to thank the editor and all the reviewers for their hard work.


the IEEE Engineering in Medicine and Biology Society (Florida, FL), 4295–4298.


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Zhao and Zhang. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Spectral and Multifractal Signature of Cortical Spreading Depolarisation in Aged Rats

#### Péter Makra\*, Ákos Menyhárt, Ferenc Bari and Eszter Farkas

Department of Medical Physics and Informatics, University of Szeged, Szeged, Hungary

Cortical spreading depolarisation (CSD) is a transient disruption of ion balance that propagates along the cortex. It has been identified as an important factor in the progression of cerebral damage associated with stroke or traumatic brain injury. We analysed local field potential signals during CSD in old and young rats to look for age-related differences. We compared CSDs elicited under physiological conditions (baseline), during ischaemia and during reperfusion. We applied short-time Fourier transform and a windowed implementation of multifractal detrended fluctuation analysis to follow the electrophysiological signature of CSD. Both in the time-dependent spectral profiles and in the multifractal spectrum width, CSDs appeared as transient dips, which we described on the basis of their duration, depression and recovery slope and degree of drop and rise. The most significant age-related difference we found was in the depression slope, which was significantly more negative in the beta band and less negative in the delta band of old animals. In several parameters, we observed an attenuation-regeneration pattern in reaction to ischaemia and reperfusion, which was absent in the old age group. The age-related deviation from the pattern took two forms: the rise parameter did not show any attenuation in ischaemic conditions for old animals, whilst the depression slope in most frequency bands remained attenuated during reperfusion and did not regenerate in this age group. Though the multifractal spectrum width proved to be a reliable indicator of events like CSDs or ischaemia onset, we failed to find any case where it would add extra detail to the information provided by the Fourier description.

Keywords: cortical spreading depolarisation, short-time Fourier transform, multifractal, detrended fluctuation analysis, local field potential

## 1. INTRODUCTION

Cortical spreading depolarisation or depression (CSD) is a self-propagating wave of depolarisation along the cortex (Leão, 1944; Somjen, 2001). In recent years, it has gained significance as it has been recognised as a key factor in the progression of secondary tissue damage after subarachnoid haemorrhage, stroke, or traumatic brain injury (Hartings et al., 2016). CSDs have also been put forward in neurocritical care as indicators of the degree of metabolic failure in the nervous tissue (Dreier et al., 2016). The influences that regulate the occurrence and propagation of CSDs have not yet been fully explored, which justifies further research.

A decisive factor that is under extensive study is aging. Age brings multiple changes to the biochemistry and cellular make-up of the cortex, and, in accordance with this, several age-related

#### Edited by:

Shangbin Chen, Huazhong University of Science and Technology, China

#### Reviewed by:

Tim David, University of Canterbury, New Zealand Yicheng Xie, Zhejiang University, China

> \*Correspondence: Péter Makra makra.peter@med.u-szeged.hu

#### Specialty section:

This article was submitted to Computational Physiology and Medicine, a section of the journal Frontiers in Physiology

Received: 07 June 2018 Accepted: 09 October 2018 Published: 08 November 2018

#### Citation:

Makra P, Menyhárt Á, Bari F and Farkas E (2018) Spectral and Multifractal Signature of Cortical Spreading Depolarisation in Aged Rats. Front. Physiol. 9:1512. doi: 10.3389/fphys.2018.01512

**121**

differences in CSD dynamics have come to light: for example, the speed of propagation is slower (Guedes et al., 1996) or the duration of CSDs is shorter in aged rats (Farkas et al., 2011). Also, the three subsequent elements of the CSD-related cerebral blood flow (CBF) response, namely an initial, transient drop in CBF, the subsequent marked hyperaemia, and the ensuing, long-lasting oligaemia were all found to be attenuated with aging as shown in rats (Farkas et al., 2011; Menyhárt et al., 2015). This observation is especially significant taken that an age-specific haemodynamic response to CSD has been proposed to play an important role in injury progression in the aged brain (Farkas and Bari, 2014).

One of the most direct and spectacular electrophysiological indicators of the onset of CSDs is the direct current (DC) potential, which is essentially the component of the local field potential (LFP) that remains after low-pass filtering. Most studies use the transient deflection in the DC potential as the electrophysiological signature of CSD and use its morphological parameters to characterise CSD evolution. Yet CSDs also cause the full-band LFP amplitude to drop, reflecting a period of highly attenuated function and a loss of excitability in the neural tissue, and making the full-band LFP as feasible a target in offline investigations as the DC potential.

Exploring the spectral fine structure of cortical electrophysiological signals in the established frequency bands (delta to gamma) may also contribute to our understanding of CSD dynamics. The alpha-to-delta ratio (ADR), for example, has proved to be a predictor of worse recovery from ischaemia in humans (Claassen et al., 2004). Following the time dependence of the spectral powers in the standard frequency ranges can yield some additional information on the evolution and age dependence of CSDs (Menyhárt et al., 2015; Hertelendy et al., 2016).

An emergent tool in the study of complex system dynamics is multifractal analysis. Heralded as a promising method to disentangle multi-scale interactions and phase transitions in complex systems, it has recently gained ground in several areas of biomedical research and psychology from the segmentation of medical images (Lopes and Betrouni, 2009) through neuroscience (Zheng et al., 2005; Ihlen and Vereijken, 2010; Zorick and Mandelkern, 2013) to even long-range correlations in narrative texts (Drozd˙ z et al., 2016 ˙ ).

In this paper, we set out to explore how aging impacts the evolution of the spectral power of LFP during CSDs in distinct frequency bands (delta, theta, alpha and beta). Furthermore, we seek to incorporate multifractal analysis into the investigation of CSD dynamics to see if it reveals anything beyond the insights provided by the Fourier technique.

#### 2. MATERIALS AND METHODS

#### 2.1. Experimental Protocol

The data we analyse in this paper originate from an earlier study reported, and all surgical and experimental procedures are, therefore, identical to those previously published (Menyhárt et al., 2017b). Briefly, the specimens were young adult (2 month-old, n = 20) and old (18–20 month-old, n = 18) male Sprague-Dawley rats. Our experimental procedures conformed to the guidelines of the Scientific Committee of Animal Experimentation of the Hungarian Academy of Sciences (updated Law and Regulations on Animal Protection: 40/2013. [II. 14.] Govt of Hungary), following the EU Directive 2010/63/EU on the protection of animals used for scientific purposes and were approved by the National Food Chain Safety and Animal Health Directorate of Csongrád County, Hungary.

The animals were anaesthetised with isoflurane. After a baseline period lasting 50 min, we induced global forebrain ischaemia with the bilateral occlusion of the common carotid arteries (two-vessel occlusion, 2VO). An hour later, we released the carotid arteries to allow the reperfusion of the forebrain. Reperfusion also lasted for an hour. In all experimental stages (i.e., baseline, ischaemia, and reperfusion), we elicited three CSDs with the topical application of 1 M KCl at even intervals of 15 min (see **Figure 1**). Experiments were terminated by an overdose of isoflurane.

In the rat, the bilateral occlusion of the common carotid arteries is a widely accepted procedure to induce incomplete global forebrain ischaemia (Farkas et al., 2007). The time window for ischaemia (i.e., 1 h) in our experiments was chosen for a number of reasons, including: (i) Ischaemia is the most consistent during the first few hours after the occlusion of the carotid arteries; (ii) Ischaemia during this period of time is similar to the penumbra typically evolving in focal ischaemic stroke, a region highly relevant for medical intervention; (iii) One hour of ischaemia offers a long enough period to trigger 3 CSD events, 15–20 min apart—this number of CSD events is necessary to confirm reproducibility and reliability; the inter-CSD interval is necessary for the tissue to recover from each CSD before the next event is provoked; (iv) The duration of the surgical procedures and the experimental protocol (together reaching 10 h) does not allow much longer period of ischaemia monitoring, taken that the animals are terminated at the end of the experimental protocol for ethical and other practical reasons. The experimental protocol to induce ischaemia was published repeatedly in our previous papers, which justifies its validity (Hertelendy et al., 2016; Varga et al., 2016; Menyhárt et al., 2017a,b).

We monitored the local field potential (LFP) in the cortex with a glass capillary electrode through a cranial window, relative to an Ag/AgCl reference electrode implanted under the skin of the neck of the animal. The LFP signal was amplified, filtered, conditioned and finally digitised at a sampling frequency of 1 kHz by a setup identical to that described in Hertelendy et al. (2016) and Menyhárt et al. (2017b).

### 2.2. Spectral Analysis

We carried out all signal analysis tasks (spectral and multifractal) in a self-developed .NET environment written in C#. Fast Fourier transforms were calculated using a .NET wrapper around FFTW (the Fastest Fourier Transform in the West, http://www.fftw.org/). We also leveraged the Task Parallel Library (TPL) included in .NET to speed up calculations.

#### 2.2.1. Artefact Filtering

Before all further analysis, we filtered the local field potential (LFP), removing excessive spikes that had likely resulted

from measurement artefacts, in order to prevent them from contaminating the spectrum. To avoid subjectivity in deciding what constitutes an artefact, we calculated the Bollinger bands (moving average ± moving standard deviation) in a window of 10,000 points, and marked segments where the magnitude of the signal exceeded a threshold of the local mean plus 4.2 times the local standard deviation as potential artefacts. We reviewed each automatic detection and only considered a segment as artefact if its slope was uncharacteristically steep. We removed confirmed artefacts and replaced them with a slope-corrected copy of the preceding interval of equal length. We applied cubic spline interpolation in a 5-point radius of the junctions between the original signal and the correction to ensure that the signal stays smooth. We found no more than 10 artefacts on average per CSD event, which altogether lasted <1 s, about 0.1–0.15% of the duration of a CSD event. The process is illustrated in **Figure 2**.

#### 2.2.2. Short-Time Fourier Transform (STFT)

The basis of the spectral investigations is the short-time Fourier transform (STFT) of the relevant LFP sequence {x<sup>k</sup> } N−1 k=0 . Advancing a Gaussian window w<sup>k</sup> of width W1t = 60 s (wherein 1t = 0.001 s denotes the sampling interval) along the LFP sequence in steps of 1τ = 1 s, one can obtain the STFT value at time t<sup>m</sup> = m1τ and frequency f<sup>n</sup> = n1f = n W1t as

$$X\left(t\_m, f\_n\right) = X\_{m,n} = \sum\_{k=0}^{W-1} \,\omega\_k \chi\_{m+k} \mathbf{e}^{-i\frac{2\pi}{N}kn}.\tag{1}$$

From the STFT, a time-dependent power spectral density S tm, f<sup>n</sup> can be calculated as follows:

$$\mathcal{S}\left(t\_m, f\_n\right) = \mathcal{S}\_{m,n} = \frac{\left|X\_{m,n}\right|^2 \Delta t}{W}.\tag{2}$$

#### 2.2.3. Spectral Power

The integrated spectral power P (tm) = P<sup>m</sup> of a given frequency range between fmin = nmin1f and fmax = nmax1f is the integral of the power spectral density between these limits, which, in discrete representation can be calculated as

$$P\left(t\_m\right) = P\_m = \sum\_{n=n\_{\min}}^{n\_{\max}-1} S\_{m,n} \Delta f = \sum\_{n=n\_{\min}}^{n\_{\max}-1} \frac{\left|X\_{m,n}\right|^2 \Delta t}{W} \cdot \frac{1}{W \Delta t}$$

$$= \sum\_{n=n\_{\min}}^{n\_{\max}-1} \frac{\left|X\_{m,n}\right|^2}{W^2}.\tag{3}$$

The four frequency ranges of brain electrical activity defined in **Table 1** were analysed.

#### 2.3. Multifractal Detrended Fluctuation Analysis (MFDFA)

In addition to the Fourier spectrum, we also applied multifractal detrended fluctuation analysis (MFDFA) to our LFP sequences. We followed the procedure laid out in Kantelhardt et al. (2002) and Ihlen (2012). For a given interval {x<sup>k</sup> } N−1 k=0 of the LFP, we first constructed a cumulative sum Y of the data series:

$$Y\_i = \sum\_{k=0}^{i} x\_k - \langle \mathbf{x} \rangle \text{ (} 0 \le i < N \text{)},\tag{4}$$

wherein hxi denotes the mean of the interval. Then we partitioned the cumulative sum sequence Y into disjunct segments of equal length s. Since the length N of the interval is usually not an integer multiple of the segment size s, we repeated the partitioning process from the opposite end, thus obtaining 2N<sup>s</sup> segments altogether, where N<sup>s</sup> = <sup>N</sup> s is the number of segments in a single direction (⌊. . . ⌋ denotes rounding down). Detrending meant the subtraction of a local polynomial trend yν , after which we calculated the local variance as

$$F\_{\upsilon}^{2}(s) = \frac{1}{s} \sum\_{i=0}^{s-1} \left\{ Y\_{\upsilon s + i} - \wp\_{\upsilon, i} \right\}^{2} \tag{5}$$

in the forward direction (0 ≤ ν < Ns), and as

$$F\_{\upsilon}^{2}(s) = \frac{1}{s} \sum\_{i=0}^{s-1} \left\{ Y\_{N-(\upsilon-N\_{i}+1)s+i} - \wp\_{\upsilon,i} \right\}^{2} \tag{6}$$

in the reverse direction (N<sup>s</sup> ≤ ν < 2Ns), where y<sup>ν</sup> is the local trend for the νth segment, obtained using m-order polynomial

least squares fitting. The central quantity of MFDFA is the qthorder fluctuation function defined as

$$F(s,q) = \left\{\frac{1}{2N\_s} \sum\_{\nu=0}^{2N\_s-1} \left[F\_\nu^2(s)\right]^{q/2}\right\}^{1/q} \langle q \neq 0 \rangle,\tag{7}$$

$$F(s,0) = \exp\left\{\frac{1}{4N\_s} \sum\_{\nu=0}^{2N\_s-1} \ln\left[F\_{\nu}^2(s)\right]\right\} (q=0). \tag{8}$$

The fractal properties of our time series x are reflected in how this fluctuation function depends on the scale s, whilst the multifractal order q serves to level the contributions made by components of small effective value and by those of large effective value: negative values of q amplify processes of small fluctuations and positive values enhance large fluctuations. The value q = 2 corresponds to standard monofractal analysis, wherein the effective value of the local variance is calculated as a function of the segment size s. We obtained the generalised Hurst exponent h(q) from the slope of the ln - F(s, q) vs. ln(s) graph using linear least-squares regression. As described in Ihlen (2012), Eke et al. (2002), and Eke et al. (2000), this slope yields the Hurst exponent h(q) directly for a class of signals called fractional Gaussian noise (fGn), whereas for another class, fractional Brownian motion (fBm), the slope equals 1 + h(q). Our preliminary classification showed that our LFP signals fall into the latter category (as most physiological signals do), so we subtracted one from the slope. Finally, we calculated the singularity strength as

$$
\alpha = h(q) + q \frac{\mathrm{d}h}{\mathrm{d}q},\tag{9}
$$

and the singularity dimension as

$$f(\alpha) = q\left[\alpha - h(q)\right] + 1.\tag{10}$$

What we called the multifractal spectrum was the f(α) function as a parametric curve that depends on the multifractal order q. We

TABLE 1 | The frequency ranges used in the analysis.


focused on the multifractal spectrum width 1α (see **Figure 3A**), defined as

$$
\Delta \alpha = \alpha\_{\text{max}} - \alpha\_{\text{min}}.\tag{11}
$$

In a similar way to STFT, we applied a windowed implementation of MFDFA: we calculated the multifractal properties in a 60-s window, then advanced the window by a step of 1 s and repeated the process, obtaining time-dependent functions comparable to the spectral powers discussed above.

In our investigations, the segment size varied from 16 to 512 as powers of 2. The maximum segment size was constrained by the requirement of scale invariance discussed in Ihlen (2012), and we decided upon a detrending order m = 1 as this choice yielded the widest range of approximate scale invariance. We varied the multifractal order q between −5 and 5 in 100 even steps. We tested the reliability of our calculations on white noise sequences and on the binomial multifractal series described in Kantelhardt et al. (2002).

#### 2.4. Depression Profiles

Cortical spreading depolarisation events appeared as periods of transient drop in all spectral powers and in the multifractal spectrum width. To quantify the properties of these intervals, we searched for the best 4th-order polynomial fit for the depression in the signal. The initial candidates for the beginning and the end of such depression intervals we selected manually, but then an automatic algorithm could override these if it found a better fit

with an endpoint within 10 s of the initial estimate. Polynomials which showed non-monotonicity in the neighbourhood of the endpoints were rejected. We designated five characteristic points to describe depression profiles (see **Figure 3B**):

1 – baseline; 2 – mid-depression; 3 – depression; 4 – mid-recovery; 5 – recovery; 6 – depression duration; 7 – drop; 8 – rise.


We standardised each depression profile by subtracting the mean and dividing by the standard deviation, where the mean and the standard deviation were calculated for a signal segment that lasted from the end of the previous CSD to the beginning of the next. We evaluated the following quantifiers for a standardised depression profile (see **Figure 3B**):


### 2.5. Statistics

We used R for all our statistical calculations. Except where otherwise noted, we divided our data into six subgroups according to age (young or old) and experimental stage (baseline, ischaemia, and reperfusion). We did not include the first CSD in the baseline group as it represents a markedly different physiological state to all subsequent CSDs, even those in the baseline group. In each experimental group, we applied a Grubbs test to decide whether extreme values are outliers. Proven outliers were removed. Then we used two-way ANOVA with age and experimental stage as factors, followed by Tukey's honest significant differences (HSD) as a post-hoc test to obtain pairwise comparisons. In the figures and the text, data are given as mean ± standard error of the mean.

## 3. RESULTS

Changes in the physiological state of the specimens were reflected unambiguously in the spectral power in all bands and also in multifractal spectrum width. CSDs and ischaemia induction (2VO) caused both spectral powers and multifractal spectrum width to drop (see **Figure 4**).

### 3.1. Duration of Depression

Ischaemia lengthened the LFP depression in all frequency bands (see **Figure 5**). This effect was most significant in the theta band (e.g., 323.47 ± 35.32 s for young–ischaemic v 165 ± 52.19 s for young–baseline, p = 0.0058). The multifractal spectrum

width did not show such a discernible pattern: for old animals, the depression tended to be longer during ischaemia, but for young ones, the relation was the opposite (see **Figure 6B**). These apparent differences in the multifractal spectrum width did not prove significant, however.

Though the duration of depression was clearly shorter in old animals throughout all experimental stages (most visibly in the baseline stage), this categorisation did not show any statistically significant difference according to age. To reveal potential aging effects, we run another two-way ANOVA restricted to the baseline stage, this time with age and frequency band as its factors (see **Figure 6A**). This test indicated a significant difference according to age overall (192.14 ± 10.68 s for old v 246.63 ± 12.31 s for young, p = 0.00089) but Tukey's HSD did not find any significant difference in the pairwise comparisons between individual groups.

### 3.2. Depression Slope

Of all the parameters we investigated, the slope of depression showed the effects of aging the most clearly (see **Figure 7**). In the beta, alpha and theta bands, old age resulted in a steeper decrease of the spectral power in the baseline stage. This was markedly significant for the beta band (−0.065 ± 0.005 s−<sup>1</sup> for old– baseline v −0.034 ± 0.004 s−<sup>1</sup> for young–baseline, p = 0.00026). In all spectral bands, ischaemia reduced the absolute value of the depression slope, most significantly again in the beta band (−0.037 ± 0.005 s−<sup>1</sup> for old–ischaemia v −0.065 ± 0.005 s−<sup>1</sup> for old–baseline, p = 0.00045). For young animals, reperfusion restored the slope of depression or made it even steeper in the lower frequency bands theta and delta (in the theta band: −0.072 ± 0.011 s−<sup>1</sup> for young–reperfusion v −0.035 ± 0.007 s−<sup>1</sup> for young–baseline, p = 0.049). This regeneration of slope also occurred in old animals in the middle frequency bands alpha and theta (in the alpha band: −0.060 ± 0.009 s−<sup>1</sup> for old–reperfusion v −0.042 ± 0.006 s−<sup>1</sup> for old–ischaemia, p = 0.51), but was clearly absent from the beta and delta bands. Again, the multifractal spectrum width did not show any significant difference according to age or experimental stage.

### 3.3. Drop

The drop in the spectral power did not show any significant difference between experimental groups. When we focused on the frequency bands in the baseline stage, however, we could discern some frequency dependence (see **Figure 8**). The drop was less at low frequencies, especially in the old age group (1.65 ± 0.22 for old–delta v 2.86 ± 0.21 for old–beta, p = 2.4 · 10−<sup>5</sup> ). In the multifractal spectrum width, one could observe a smaller drop during ischaemia than in the baseline state, which was significant for young animals (1.67 ± 0.07 for young–ischaemia v 2.36 ± 0.12 for young–baseline, p = 0.048, see **Figure 9A**). This was the only case where the multifractal spectrum width proved significantly different to that in any other experimental group.

## 3.4. Recovery Slope

The recovery slope followed the same attenuation-regeneration pattern as the depression slope in young animals, though this effect proved significant only in the theta (0.019 ± 0.004 s−<sup>1</sup> for young–ischaemia v 0.065 ± 0.019 s−<sup>1</sup> for young– baseline, p = 0.0080, see **Figure 10**) and delta bands. For this parameter, however, the regeneration brought about by reperfusion persisted in the old age group in all spectral bands except the alpha. One can also observe that the recovery slope is markedly higher in the lower bands theta and delta than in the higher bands beta and alpha (e.g., 0.068 ± 0.017 s−<sup>1</sup> for delta–young–reperfusion v 0.012 ± 0.002 s−<sup>1</sup> for beta–young– reperfusion, p = 0.0002). As **Figure 9B** shows, the attenuationregeneration pattern was absent from the multifractal spectrum width.

FIGURE 5 | Duration of depression in different frequency bands for all experimental stages. Significance levels are given as #p < 0.05, ##p < 0.01 v respective baseline; \$p < 0.05, \$\$p < 0.01 v respective ischaemia.

significant difference with respect to age (p = 0.00089) overall, though Tukey's HSD applied as a post-hoc test did not find significant pairwise difference between any two comparable age–band groups. (B) Duration of depression in the multifractal spectrum width.

#### 3.5. Rise

A similar attenuation-regeneration dynamics also seemed to manifest itself in the rise after the CSD-induced transient depression for young animals (e.g., in the alpha band: 1.93 ± 0.22 for young–reperfusion v 1.08 ± 0.14 for young–ischaemia, p = 0.046; see **Figure 11**). This was clearly absent from the old age group as the rise remained at about the same level during ischaemia and reperfusion (e.g., in the beta band: 2.02 ± 0.26 for old–reperfusion and 1.92 ± 0.24 for old–ischaemia, the latter v 1.07 ± 0.15 for young–ischaemia, p = 0.029). We could observe no difference whatsoever in the multifractal spectrum width between experimental groups.

## 4. DISCUSSION

Intraoperative electrocorticogram (ECoG) monitoring is an invasive approach to aid tumour resection or surgery for the alleviation of epilepsy (Yang et al., 2014; Alcaraz and Manninen, 2017), and it has lately been used for the postoperative monitoring of acute brain injury patients to follow the evolution of CSD events (Dreier et al., 2016). Although under most circumstances, full-band ECoG is sufficient to provide feedback to the neuro-surgeon or neuro-intensive care specialist, several studies have underpinned the clinical relevance of the analysis of brain electrophysiological signals by frequency band: in addition to alpha-to-delta ratio (Claassen et al., 2004), focal reduction in the alpha band of the electroencephalogram (EEG) can be associated with the occurrence of delayed cerebral infarction in subarachnoid haemorrhage (Gollwitzer et al., 2015); the slope of the EEG delta power correlates with stroke severity (Finnigan et al., 2004; Hartings et al., 2005); low beta power in the ECoG indicates a higher probability of CSD occurrence in patients with traumatic brain injury (Hertle et al., 2016) or the slope of theta power decline in EEG

is a good predictor of postinjury epilepsy (Milikovsky et al., 2017).

Though age-associated changes have been revealed in the spectral composition of ECoG in rats—whilst aging causes highfrequency power to decrease, it also enhances delta power (Bagetta et al., 1989),—and on the other hand, the traces of CSD in scalp EEG frequency bands have been analysed (Hartings et al., 2014), the spectral implications of CSD have been mapped out in the EEG of conscious rabbits (Roshchina et al., 2014) and of conscious and anaesthetised rats (Koroleva et al., 2016), we have had no information on the influences of aging on the spectral representation of LFP during CSDs in different frequency bands. Since the pattern of CSD is considerably influenced by age (i.e., lower frequency of occurrence in the intact and ischaemic cortex, longer duration, and more frequent association with inverse CBF response in focal ischaemia) (Farkas et al., 2011; Clark et al., 2014; Menyhárt et al., 2015), here we set out to explore whether the analysis of LFP by frequency band or multifractal analysis might deliver any potential LFP signature specific for age. We presented LFP spectral analysis of CSD in a preliminary form earlier, but the age range covered in our previous report spanned young adulthood (7–30 weeks of age in rats), and did not go beyond to include old age as well (Hertelendy et al., 2016).

What we have found extends on the power spectrum-related conclusions in our earlier report (Hertelendy et al., 2016). There we observed a shorter duration of CSD-associated depression in the lower frequency bands delta and theta in 30-weekold animals and argued that owing to a low-pass filtering effect present in the brain tissue (Buzsáki et al., 2012), the distance from which lower-frequency components can reach the electrode is greater than that for higher frequencies, so a reduced duration in the lower frequency bands of older animals might indicate that at more distant sites, regeneration has already taken place, that is, the CSD-related depression wave is narrower in space for the 30-week-old group. Here, for 72–80-week-old animals, the duration of depression is shorter even in the high frequency bands alpha and beta, which, using the same logic, can mean that as age advances further, the CSD-related depression wave shrinks in space even more, to the point where its spatial extension does not exceed the range within which the electrode can detect high-frequency signals.

The most significant difference we found between young and old animals appeared in the depression slope of spectral powers after the onset of CSD. Under physiological conditions, CSDinduced decline in the beta power was much steeper in the old

age group, whilst during reperfusion, the rate at which delta power decreased in reaction to CSDs was less in absolute value in old animals. The latter might be in accord with earlier findings where less negative or even positive delta slope (termed aDCI, acute delta change index) predicted worse outcomes in ischaemic stroke patients (Finnigan et al., 2004).

#### 4.1. Attenuation-Regeneration Pattern

Several parameters investigated here followed a pattern where values decreased during ischaemia as compared to baseline then were restored during reperfusion. This behaviour was most consistent in the recovery slope. These data are consistent with the profound differences in the pattern of CSDs that occur in the intact and ischaemic cerebral cortex. As such, CSD as indicated by the negative deflection of the DC potential lasts significantly longer under ischaemia as compared with the intact condition (Menyhárt et al., 2015), and the coupled hyperaemia is of substantially smaller amplitude but longer duration (Menyhárt et al., 2017b). The slower LFP recovery from CSD under ischaemia, found here especially in the low frequency bands, faithfully reflects the lack of metabolic resources to re-establish resting electrical activity of the nervous tissue.

Most age-related effects we found represented a deviation from this pattern. The attenuation of the depression slope of beta and delta spectral powers proved permanent in old animals and was not followed by regeneration. Another type of age-related deviation could be observed in the rise after CSD-induced depressions in the spectral powers: here the attenuation step was absent from the spectrum

of old animals and the rise values remained about the same throughout all experimental stages. Finally, we noted an isolated departure from attenuation-regeneration dynamics in the recovery slope of the alpha power of old animals, where again no attenuation occurred during ischaemia.

### 4.2. Multifractal Spectrum to Complement the Fourier Spectrum

One argument for the application of fractal analysis instead of or in addition to traditional linear investigation methods such as Fourier transform is the perceived inability of the latter to quantify the scale-dependent properties of complex biological systems that stem from the interplay of many levels of substructure (Chakraborty et al., 2016). In human EEG, the Fourier transform failed to show a response to olfactory stimuli, whilst the Hausdorff–Besicovitch fractal dimension proved sensitive to them (Murali and Vladimir, 2007). The fractal dimension increases after brain injury in rats (Spasic et al., 2005). The fractal dimension and the Hurst exponent differ before and after CSD (Santos et al., 2014) and monofractal detrended fluctuation analysis detected slight variations in the Hurst exponent before, during and after CSD (do Nascimento et al., 2010).

In addition to the monofractal studies above, several multifractal analyses have targeted the brain. The multifractal spectrum width calculated for the EEG database of epilepsy patients has been shown to be less in ictal periods than in interictal ones (Zhang et al., 2015), which was in agreement with the comparison of normal and epileptic rat EEG (Dutta, 2010).

To our knowledge, this paper is the first to extend earlier monofractal studies (do Nascimento et al., 2010; Santos et al., 2014) towards multifractality in the investigation of CSD. We demonstrated that CSDs cause a transient narrowing in the multifractal spectrum, signalling a temporary loss of multifractality and thus a suppression of the interplay between different scales in the LFP. This is in agreement with previous MFDFA-based findings: just like epilepsy (Dutta, 2010; Zhang et al., 2015), CSD is reflected in the multifractal spectrum as a reduction in width.

One goal of ours was to find out whether multifractal analysis yields any additional information on the dynamics of the local field potential during CSDs as compared to the Fourier spectrum. The MFDFA parameter we chose to follow in this study, the multifractal spectrum width, has failed to reveal anything more than our STFT-based profiles—in fact, it has proved largely insensitive to age or experimental stage. The only exception to this was the drop in the profile as a reaction to CSD, which, in young animals, was significantly less during ischaemia than the baseline, but it did not fit into any discernible pattern.

## AUTHOR CONTRIBUTIONS

PM: analysis and interpretation of data, drafting the article; ÁM: substantial contributions to conception and design, acquisition of data; FB: revising the manuscript critically for important intellectual content; EF: substantial contributions to conception and design, analysis and interpretation of data, drafting the article, and revising it critically for important intellectual content.

## FUNDING

The authors acknowledge support from the National Research, Development and Innovation Office of Hungary (Grant No K111923 and K120358); the Economic Development and Innovation Operational Programme in Hungary co-financed by the European Union and the European Regional Development Fund (No GINOP-2.3.2-15-2016-00048) and the EU-funded Hungarian grant No EFOP-3.6.1-16- 2016-00008.

## ACKNOWLEDGMENTS

PM thanks Szabina Tudja for ideas and thoughtful discussions.

#### Makra et al. CSD Spectral Bands in Aging

#### REFERENCES


spreading depolarization and the associated cerebral blood flow response. Sci. Rep. 6:31402. doi: 10.1038/srep31402


the wavelet transform modulus maxima technique. PLoS ONE 8:e68360. doi: 10.1371/journal.pone.0068360

**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Makra, Menyhárt, Bari and Farkas. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Dendritic and Axonal Propagation Delays May Shape Neuronal Networks With Plastic Synapses

Mojtaba Madadi Asl <sup>1</sup> \*, Alireza Valizadeh1,2 and Peter A. Tass <sup>3</sup>

*<sup>1</sup> Department of Physics, Institute for Advanced Studies in Basic Sciences (IASBS), Zanjan, Iran, <sup>2</sup> School of Cognitive Sciences, Institute for Research in Fundamental Sciences (IPM), Tehran, Iran, <sup>3</sup> Department of Neurosurgery, Stanford University School of Medicine, Stanford, CA, United States*

Biological neuronal networks are highly adaptive and plastic. For instance, spike-timing-dependent plasticity (STDP) is a core mechanism which adapts the synaptic strengths based on the relative timing of pre- and postsynaptic spikes. In various fields of physiology, time delays cause a plethora of biologically relevant dynamical phenomena. However, time delays increase the complexity of model systems together with the computational and theoretical analysis burden. Accordingly, in computational neuronal network studies propagation delays were often neglected. As a downside, a classic STDP rule in oscillatory neurons without propagation delays is unable to give rise to bidirectional synaptic couplings, i.e., loops or uncoupled states. This is at variance with basic experimental results. In this mini review, we focus on recent theoretical studies focusing on how things change in the presence of propagation delays. Realistic propagation delays may lead to the emergence of neuronal activity and synaptic connectivity patterns, which cannot be captured by classic STDP models. In fact, propagation delays determine the inventory of attractor states and shape their basins of attractions. The results reviewed here enable to overcome fundamental discrepancies between theory and experiments. Furthermore, these findings are relevant for the development of therapeutic brain stimulation techniques aiming at shifting the diseased brain to more favorable attractor states.

Keywords: propagation delays, spike-timing-dependent plasticity, synchronization, mathematical modeling, living systems

## 1. INTRODUCTION

Time delays play an important role in various fields of physiology (Glass et al., 1988; Batzel and Kappel, 2011). Neurophysiological time delays crucially affect generation, transmission, and processing of information among different components of a living system, and more specifically, between interconnected neurons in the nervous system. The time required for neuronal communication can be significantly prolonged due to the physical distance between sending and receiving units (Knoblauch and Sommer, 2003, 2004), finite velocity of signal transmission (Desmedt and Cheron, 1980), morphology of dendrites and axons (Manor et al., 1991; Boudkkazi et al., 2007) and information processing time of the cell (Wang et al., 2009). The physiological range of such time delays may vary from a few milliseconds in dendritic trees (Agmon-Snir and Segev, 1993; Schierwagen and Claus, 2001) to tens of milliseconds in axonal components of cortico-thalamic circuits (Swadlow and Weyand, 1987).

#### Edited by:

*Alexey Zaikin, University College London, United Kingdom*

#### Reviewed by:

*Jihwan Myung, Taipei Medical University, Taiwan Silvina Ponce Dawson, Universidad de Buenos Aires, Argentina*

\*Correspondence:

*Mojtaba Madadi Asl m.madadi@iasbs.ac.ir*

#### Specialty section:

*This article was submitted to Computational Physiology and Medicine, a section of the journal Frontiers in Physiology*

Received: *25 August 2018* Accepted: *07 December 2018* Published: *20 December 2018*

#### Citation:

*Madadi Asl M, Valizadeh A and Tass PA (2018) Dendritic and Axonal Propagation Delays May Shape Neuronal Networks With Plastic Synapses. Front. Physiol. 9:1849. doi: 10.3389/fphys.2018.01849*

The presence of such experimentally observed propagation delays can have significant impacts on the performance, structure, and function of the nervous system (Sirota et al., 2005; Joris and Yin, 2007; Chomiak et al., 2008; Spencer et al., 2012, 2018; Squire et al., 2012; Walters et al., 2013; Esfahani et al., 2016; Stoelzel et al., 2017). In fact, the diversity of dendritic and axonal propagation delays in the nervous system can underlie different response properties of the relevant neuronal populations (Sirota et al., 2005; Stoelzel et al., 2017). For instance, axonal propagation delays in visual and motor cortico-thalamic circuits correspond to different response functions associated with sensory, movement-related, or spontaneous activity of neurons (Sirota et al., 2005; Stoelzel et al., 2017). The auditory system employs compensatory delay mechanisms to modulate the asynchrony in inputs, in this way reducing the sensitivity of brainstem neurons to interaural time delays (Spencer et al., 2012, 2018). Propagation delays also can affect the communication between connected neurons by modulating the spatio-temporal properties of pre- and postsynaptic activity patterns (Chomiak et al., 2008). One major role of axonal propagation delays might be their involvement in the generation of nearly synchronous responses in postsynaptic neurons by regulating the outgoing impulses in axons with several postsynaptic target neurons (Chomiak et al., 2008).

Despite their inevitable physiological significance in living systems, propagation delays are usually overlooked in mathematical models, presumably to avoid further complexity. Although this assumption simplifies the theoretical calculations and reduces the computational cost of multiscale computer simulations, it renders mathematical models unable to provide insight into relevant physiological mechanisms. However, a number of theoretical and computational studies have shown that propagation delays modify weight and neuronal dynamics by affecting the co-evolution of synaptic strengths and neuronal activity, and therefore, shaping the emergent functional and structural properties of plastic neuronal networks (Lubenov and Siapas, 2008; Aoki and Aoyagi, 2009; Kozloski and Cecchi, 2010; Rubinov et al., 2011; Knoblauch et al., 2012; Babadi and Abbott, 2013; Kerr et al., 2013; Madadi Asl et al., 2017, 2018a), where the synaptic strengths are regulated by spike-timing-dependent plasticity (STDP) (Gerstner et al., 1996; Markram et al., 1997; Bi and Poo, 1998; Song et al., 2000). Hence, incorporation of time delays in mathematical models can significantly modify the dynamical properties of neuronal systems, such as the emergence of different connectivity patterns (Madadi Asl et al., 2017, 2018a), affecting the dynamics of fixed points and synchronization properties between interconnected neurons (D'Huys et al., 2008; Popovych et al., 2011), and the emergence of different multistable dynamical attractors (Song et al., 2009; Madadi Asl et al., 2018a).

Neglecting realistic time delays in mathematical models has led to discrepancies between theoretical and experimental findings over the past few years. In this manuscript, we review recent physiological and computational studies that have shown that a simple classic STDP rule enhanced by realistic dendritic and axonal propagation delays is able to explain some of the corresponding experimental results. We highlight the pivotal role of dendritic and axonal propagation delays in regulating the emergent activity and connectivity patterns in plastic neuronal networks under the influence of classic pair-based STDP which significantly affects the information transmission in neuronal populations. Ultimately, we point out the importance of propagation delays in the computation-based development of therapeutic brain stimulation techniques that are used for modulating plastic neuronal networks in diseased brains.

## 2. PROPAGATION DELAYS: PHYSIOLOGICAL ASPECTS

From a physiological standpoint, the measurement of propagation delays in dendrites or axons of neuronal populations requires complex experimental setups, stimulation protocols, or modern clinical instruments. Several experimental studies investigated dendritic and axonal propagation delays in neuronal populations of various species (Swadlow and Weyand, 1987; Swadlow, 1990; Agmon-Snir and Segev, 1993; Schierwagen and Claus, 2001; Ferraina et al., 2002; Briggs and Usrey, 2009; Stoelzel et al., 2017). The physiological range of dendritic and axonal propagation delays may attain a range of different values, based on different experimental model systems in which they were measured. For instance, the value of dendritic propagation delays may vary from sub-milliseconds to a few milliseconds (Agmon-Snir and Segev, 1993; Schierwagen and Claus, 2001). Axonal propagation delays, however, may take a wider range from a few milliseconds in corticotectal connections (Swadlow and Weyand, 1987) to tens of milliseconds in cortico-cortical (Swadlow, 1990) and corticothalamic circuits (Swadlow and Weyand, 1987). Axonal delays are typically greater than dendritic delays in a neuron, however, values of dendritic delays greater than the axonal delays were experimentally measured in distal dendrites of neocortical pyramidal neurons (Stuart and Spruston, 1998; Sjöström and Häusser, 2006).

In the auditory system, dendritic and axonal propagation delays modify the mechanisms of interaural time sensitivity by regulating coincident or lagged inputs from the two sides, and therefore, play a constructive/destructive role in binaural sound localization depending on the location of the sound source and the leading ear (Joris and Yin, 2007; Squire et al., 2012). Dendritic propagation delays are hypothesized to play a compensatory role for the input asynchrony in the auditory brainstem of mammals using plastic synaptic weights (Spencer et al., 2012, 2018). In the motor system, propagation delays can impose functional limitations on the efficiency of feedback control in situations where time-critical performance of the sensory feedback is vital for the biological system (Squire et al., 2012). The functional significance of diverse range of axonal propagation delays in cortico-thalamic circuits are shown to be strongly related to multiple visual response properties (Stoelzel et al., 2017). Axonal delays act as a timing mechanism in the neuronal networks responsible for path integration of head direction and were computationally shown to promote the accuracy of path integration in the absence of visual input (Walters et al., 2013). Experimentally delayed visual feedback was used as a tool to manipulate and disentangle different motor control regulatory brain mechanisms (Tass et al., 1996; Rougier, 2003; van den Heuvel et al., 2009).

The role of dendritic or axonal propagation delays has been implicated in a number of nervous system disorders such as Parkinson's disease (PD) (Hauptmann and Tass, 2007; Ebert et al., 2014; Shouno et al., 2017), epilepsy (Wendling et al., 2010), and multiple sclerosis (MS) (Waxman, 2006). Subthalamic nucleus (STN) parkinsonian oscillations are shown to be sensitive to feedback oscillatory inputs of cortical circuits in a delaydependent manner (Shouno et al., 2017). Neurophysiological latencies are hypothetically involved in the complex propagation mechanisms of epileptic activity in the brain (Wendling et al., 2010). In MS patients a demyelination of axonal components may lead to significant transmission delays along the axon of the cell (Waxman, 2006). This process reduces the conduction velocity of signals along the axon and can ultimately result in a blockage of information transmission and conduction failure of the axon (Waxman, 2006). Furthermore, propagation delays can have significant impact on methods used to record or modulate brain activity. For instance, time delays can affect procedures that estimate the degree of association and phase relationships between electroencephalogram (EEG) signals (Lopes da Silva F et al., 1989), or adjust the performance of therapeutic brain stimulation techniques (see below).

### 3. PROPAGATION DELAYS: COMPUTATIONAL ASPECTS

From a computational standpoint, one of the most important roles of propagation delays might be their potential to address the challenging inconsistencies between theoretical and computational studies regarding the functional, structural, and dynamical properties of plastic neuronal networks driven by the pair-based STDP (Abbott and Nelson, 2000; Song and Abbott, 2001; Pfister and Gerstner, 2006; Masuda and Kori, 2007; Lubenov and Siapas, 2008; Clopath et al., 2010; Kozloski and Cecchi, 2010; Knoblauch et al., 2012) on the one hand and relevant experimental observations (Bi and Poo, 1998; Van Rossum et al., 2000; Sjöström et al., 2001; Song et al., 2005; Wang et al., 2005; Lea-Carnall et al., 2017) on the other hand, e.g., the prevalence of strong bidirectional loops between pairs of neurons in cortical circuits (Song et al., 2005; Morishima and Kawaguchi, 2006) and the dependence of emergent synaptic structures on the firing rate of neurons (Sjöström et al., 2001; Wang et al., 2005; Lea-Carnall et al., 2017).

In fact, the classic pair-based STDP model (Gerstner et al., 1996; Markram et al., 1997; Bi and Poo, 1998; Song et al., 2000), through which the change of the synaptic strengths is induced by pairwise temporal interactions between pre- and postsynaptic spikes, has shown to be unable to account for the emergence of strong bidirectional connections and neuronal loops (Abbott and Nelson, 2000; Song and Abbott, 2001; Lubenov and Siapas, 2008; Kozloski and Cecchi, 2010; Knoblauch et al., 2012; Babadi and Abbott, 2013). Furthermore, it fails to address the experimentally measured dependency of weight dynamics on the frequency of oscillations (Sjöström et al., 2001; Wang et al., 2005; Lea-Carnall et al., 2017). Several attempts were made in order to overcome the limitations of the pair-based STDP model over the past few years via the introduction of variations or improvements of the STDP model, such as the tripletbased STDP (Pfister and Gerstner, 2006), STDP with shifted learning window (Babadi and Abbott, 2013), or application of independent noise (Popovych et al., 2013; Lücken et al., 2016). Furthermore, there are several biophysical models that attempt to identify variables with specific biophysical quantities and include them in biophysics-based models of STDP (Castellani et al., 2001; Shouval et al., 2002a,b, 2010; Abarbanel et al., 2003; Rachmuth et al., 2011). For instance, Shouval et al. developed a model of long-term potentiation/depression that includes the back propagating potential in the STDP model (Castellani et al., 2001; Shouval et al., 2002a,b). For a review of the shortcomings of pair-based STDP and its variations see (Morrison et al., 2008; Madadi Asl et al., 2018b).

A number of studies, however, have focused on the role of propagation delays to resolve the aforementioned discrepancies. Short axonal propagation delays were shown to decouple synchronous neurons in the presence of STDP (Knoblauch and Sommer, 2003, 2004), whereas long axonal propagation delays promote inter-areal synchronized activity and result in a potentiation of the synaptic strengths (Knoblauch and Sommer, 2004). Taking into account only dendritic propagation delays in the modeling can result in the emergence of strong twoneuron loops (Morrison et al., 2007). Also, it was shown that a combination of dendritic and axonal propagation delays along with an unbalanced STDP profile can lead to the emergence of self-organized states in recurrent neuronal networks (Lubenov and Siapas, 2008). The role of dendritic and axonal propagation delays on the dynamics of recurrent neuronal networks has also been pointed out by considering the effect of time delays in terms of a shift in the STDP temporal window (Babadi and Abbott, 2013). Pairwise interactions of STDP-driven recurrent neuronal populations with such shifts can explain mechanisms underlying loop formation and elimination in bidirectional synapses (Kozloski and Cecchi, 2010; Babadi and Abbott, 2013).

Recently, by presenting a theoretical framework comprising regular spiking neurons we showed that by taking into account dendritic and axonal propagation delays in the modeling of a STDP-driven two-neuron motif different patterns of synaptic connectivity may emerge (Madadi Asl et al., 2017). The synaptic strengths are modified according to the following pair-based STDP rule (Bi and Poo, 1998):

$$
\Delta \mathbf{g}\_{ij} = A\_{\pm} \operatorname{sgn}(\Delta t') \exp(-|\Delta t'|/\tau\_{\pm}), \tag{1}
$$

where A+(A−) and τ+(τ−) are the learning rate and the effective time window of synaptic potentiation (depression), respectively, and sgn(1t ′ ) is the sign function. 1t ′ = 1t + ξ is the effective delayed time lag between pre- and postsynaptic spikes at the synaptic site (Madadi Asl et al., 2017, 2018a). 1t = tpost − tpre is the original time lag between pre- and postsynaptic spike pairs, and ξ = τ<sup>d</sup> − τ<sup>a</sup> is the difference between dendritic and axonal propagation delays. The synaptic strengths are updated by an additive rule at each step gij → gij+1gij, and they are confined in the range (gmin, gmax) ∈ [0, 1] by using a hard bound saturation constraint.

When propagation delays are ignored or, equivalently, when dendritic and axonal delays are identical for both directions of the reciprocal synapses, ξ = τ<sup>d</sup> − τ<sup>a</sup> = 0, the original and the effective delayed time lags are equal, 1t ′ = 1t. Therefore, the type of synaptic modification is simply determined by the sign of the original time lag, i.e., 1t ≥ 0 leads to a potentiation of the synapse whereas 1t < 0 results in a depression. Hence, in the absence of propagation delays, the potentiation of one synapse is accompanied by the depression of the other synapse, leading to a unidirectional connection when the potentiation and depression amplitude of the STDP profile is balanced. However, in the presence of dendritic and axonal propagation delays and assuming that the spiking neurons are relatively phase-locked with a small time lag with respect to the propagation delays, |ξ | > |1t|, the effective delayed time lag 1t ′ perceived at the synaptic site may be different from the time lag of the spikes at the cell bodies. Hence, as shown in **Figure 1A**, when the dendritic delay is greater than the axonal τ<sup>d</sup> > τa, reciprocal synapses are both potentiated, which lead to the emergence of a strong bidirectional loop. On the contrary, greater axonal delays τ<sup>d</sup> < τ<sup>a</sup> result in a depression of both reciprocal synapses, in this way generating a loosely connected motif (see **Figure 1B**).

By assuming that the neurons remain phase-locked, it was illustrated that the two-neuron results can be extended to recurrent networks of spiking neurons (Madadi Asl et al., 2017, 2018a). Different combinations of dendritic and axonal propagation delays can lead to the emergence of symmetric connections, i.e., either two-neuron bidirectional loops, in the case that dendritic propagation delays are greater than the axonal delays (**Figure 2A**), or loosely connected motifs when axonal propagation delays are greater than the dendritic delays (**Figure 2C**) (Madadi Asl et al., 2017). As shown in **Figure 2C**, the disconnected network is highly unstable and ultimately leads to the emergence of unidirectional connections. However, we showed that the loosely connected motif can be stabilized by assigning a finite value to the lower bound of the synaptic strengths gmin (Madadi Asl et al., 2018a). In this framework, unidirectional connections can also arise when dendritic and axonal propagation delays are identical in both directions of the reciprocal synapses (**Figure 2B**).

Furthermore, we studied the emergence of delay-induced multistable dynamics in recurrent networks of spiking neurons attributed to the distribution of the initial synaptic strengths modified by STDP (Madadi Asl et al., 2018a). Such a multistability of the network evolution can be theoretically addressed by the emergence of different attractor states representing the two-dimensional space of the initial synaptic strengths in a two-neuron motif (Madadi Asl et al., 2018a). Moreover, it was shown that the basin of attraction of each dynamical state depends on the firing rate of the neurons in a way that higher firing frequencies favor the emergence of symmetric connections in expense of eliminating the unidirectional connections (Madadi Asl et al., 2018a). Intriguingly, the aforementioned nontrivial dynamics are only present when the dendritic and axonal propagation delays are considered in the neuronal networks models. In the simplest setting, characterized by ignoring dendritic and axonal propagation delays as well as the absence of independent noise, any initial preparation leads to the emergence of unidirectional connections regardless of the neuronal firing pattern and the initial synaptic strengths.

## 4. CONCLUDING REMARKS

Propagation delays are inevitable in living systems, and in particular, in the nervous system. The presence of propagation delays has significant impact on the performance, structure, and function of the nervous system. However, from a physiological as well as theoretical standpoint, systems with time delays are considerably more complex, and therefore, delays have typically not been taken into account in relevant studies to simplify the experimental setups in physiological measurements or the mathematical approach in theoretical studies. Incorporating time delays can impose significant levels of complexity and computational cost to the problem. Time delay differential equations are more complicated to deal with from an analytical standpoint. For this reason, in a first approximation, theoretical and computational studies typically ignored the effects of time delays in the modeling. This has led to some discrepancies between theoretical and computational studies with physiological measurements over the past few years (Bi and Poo, 1998; Abbott and Nelson, 2000; Van Rossum et al., 2000; Sjöström et al., 2001; Song and Abbott, 2001; Song et al., 2005; Wang et al., 2005; Pfister and Gerstner, 2006; Masuda and Kori, 2007; Lubenov and Siapas, 2008; Clopath et al., 2010; Kozloski and Cecchi, 2010; Lea-Carnall et al., 2017). However, in an attempt to overcome unphysiological simplifications, we recently demonstrated that incorporating dendritic and axonal propagation delays in STDP-driven networks of spiking model neurons can lead to the emergence of different synaptic connectivity patterns characterized by different dynamical attractors (Madadi Asl et al., 2017, 2018a).

The shortcomings of the pair-based STDP model can be resolved by several improvements proposed during the past decade: The experimentally demonstrated dependency of the weight dynamics on the frequency of the neuronal oscillations can be addressed by considering triplets of spikes (Pfister and Gerstner, 2006) or postsynaptic voltage (Clopath et al., 2010) in the plasticity model. In fact, the triplet-based STDP model is proposed to comply with the experimentally observed dependence of the weight changes on the firing frequency of the oscillations (Sjöström et al., 2001; Wang et al., 2005; Lea-Carnall et al., 2017), showing that bidirectional connections can be promoted at high firing rates (Pfister and Gerstner, 2006). Strong bidirectional loops can be retained by employing an unbalanced STDP model with a shifted learning window (Babadi and Abbott, 2013) or the application of independent noise (Popovych et al., 2013; Lücken et al., 2016). A pair-based STDP model with a rightward shifted learning window was shown to preserve bidirectional connections, provided that potentiation dominates

FIGURE 1 | Delay-induced connectivity patterns in a two-neuron motif. Spiking neurons are connected to each other via initially symmetric synapses with strengths *g*<sup>21</sup> (*g*12) of the synapse 1 −→ 2 (2 −→ 1) with a small time lag 1*t* = *t*post − *t*pre. 1*t* ′ = 1*t* + ξ is the effective delayed time lag perceived at the synapse which STDP employs to modify the synapse, where ξ = τ<sup>d</sup> − τ<sup>a</sup> and |ξ | > |1*t*|. Green and red dotted (solid) markers indicate the original, *t*<sup>1</sup> and *t*<sup>2</sup> (delayed) forward and backpropagated spike time of pre- and postsynaptic neurons at the synapse, respectively. (A) Emergence of a strong bidirectional loop: both synapses are reciprocally potentiated when τ<sup>d</sup> > τa. (B) A loosely connected motif: both reciprocal synapses are depressed when τ<sup>d</sup> < τa. Figure partly adopted from Madadi Asl et al. (2018b) with authors' permission.

over depression (Babadi and Abbott, 2013). Furthermore, STDPdriven neuronal populations subjected to independent noise counteract the desynchronizing effect of noise by reorganizing their synaptic connectivity (Popovych et al., 2013; Lücken et al., 2016). This ultimately leads to a self-organized noise resistance and promotes bidirectional connections between neurons.

The findings reviewed in this paper highlight the key role of the presence and the range of dendritic and axonal propagation delays in modifying the arising dynamics of synaptic connectivity patterns in recurrent networks of spiking neurons. In fact, shortrange propagation delays may favor strong two-neuron loops, whereas connections with long propagation delays may result in the stabilization of a loosely connected network. Hence, the difference of dendritic and axonal propagation delays play a crucial role in determining the final stable coupling regime selected by the network dynamics (Madadi Asl et al., 2017, 2018a). In this way, delay-induced dynamics can overcome the shortcomings of the pair-based STDP model: Strong two-neuron loops can be preserved even with a balanced STDP profile in the absence of independent noise, provided dendritic and axonal propagation delays are considered in the model, and furthermore, the experimentally observed dependency of the weight dynamics on the frequency of the oscillations can be addressed in this setting (Madadi Asl et al., 2017, 2018a).

Abnormal neuronal synchronization is a hallmark of several brain disorders (Lenz et al., 1994; Nini et al., 1995; Hammond et al., 2007). Coordinated reset (CR) stimulation is a computationally developed patterned multichannel stimulation (Tass, 2003) which aims at specifically counteracting abnormal synchrony by desynchronization (Tass, 2003), thereby causing a decrease of neuronal coincidences and, hence, a down-regulation of synaptic weights, ultimately shifting the affected neuronal networks from pathological attractor states (with strong synchrony and strong synaptic connectivity) to more physiological attractor states (with loose coupling and desynchronized activity) (Tass and Majtanik, 2006). The very goal of this approach is to induce long-lasting desynchronization which persists after cessation of stimulation (Tass and Majtanik, 2006). Computationally predicted desynchronizing effects (Tass, 2003), cumulative effects (Hauptmann and Tass, 2009) and longlasting effects (Tass and Majtanik, 2006) were experimentally validated in the field of deep brain stimulation for the treatment of Parkinson's disease in pre-clinical studies in Parkinsonian monkeys (Tass et al., 2012b; Wang et al., 2016) as well as in a proof of concept study in patients with Parkinson's disease (Adamchic et al., 2014a). As computationally predicted (Popovych and Tass, 2012; Tass and Popovych, 2012), CR stimulation can also be realized by sensory stimulation modalities. Acoustic CR stimulation caused a significant relief of symptoms in patients with chronic subjective tinnitus (Tass et al., 2012a), combined with a significant reduction of abnormal neuronal synchrony (Tass et al., 2012a; Adamchic et al., 2014b) and abnormal effective connectivity (Silchenko et al., 2013), as shown in a proof of concept study employing clinical scores and EEG recordings. By the same token, vibrotactile CR stimulation (Tass, 2017) caused long-lasting treatment effects, as observed in a first in man study in Parkinson's patients (Syrkin-Nikolau et al., 2018).

The findings reviewed above are relevant for the development of desynchronizing brain stimulation techniques. From a model perspective, long-lasting treatment effects are caused by shifting networks from pathological, strongly synchronized model attractor states to physiological, desynchronized attractor states (Tass and Majtanik, 2006). One the one hand, propagation delays determine which attractors actually emerge. On the other hand, propagation delays additionally shape the basins

FIGURE 2 | Emergence of different connectivity patterns in a recurrent network of spiking neurons mediated by STDP. (Left panels) Simulated order parameter and time course of three different mean couplings of weight distributions. (Middle panels) Initial Gaussian distribution around different mean values and final distribution of the synaptic strengths. (Right panels) Time course of the normalized number of closed loops of length 2 measuring the number of two-neuron loops in the network (Madadi Asl et al., 2017). (A) The synaptic strengths are potentiated and bidirectional connections are significantly enhanced in the inphase firing when τ<sup>d</sup> = 0.5 ms > τ<sup>a</sup> = 0.3 ms. (B) STDP breaks strong two-neuron loops and results in unidirectional connections in nearly inphase firing when τ<sup>d</sup> = τ<sup>a</sup> = 0.5 ms. (C) A loosely connected network is achieved where bidirectional loops are eliminated in the nearly antiphase firing when τ<sup>d</sup> = 0.5 ms < τ<sup>a</sup> = 1.0 ms. STDP parameters are *A*<sup>+</sup> = *A*<sup>−</sup> = 0.005, and τ<sup>+</sup> = τ<sup>−</sup> = 10 ms. Figure partly adopted from Madadi Asl et al. (2017) with authors' permission.

of attraction and, hence, determine to which extent attractors get accessible by appropriate stimulus protocols. Finally, propagation delays may favorably or unfavorably impact on multichannel stimulation protocols with dedicated stimulus sequences, since delays may counteract proper stimulus timing.

#### DATA AVAILABILITY STATEMENT

All datasets generated or analyzed for this study are included in the manuscript.

#### REFERENCES

Abarbanel, H. D., Gibb, L., Huerta, R., and Rabinovich, M. I. (2003). Biophysical model of synaptic plasticity dynamics. Biol. Cybern. 89, 214–226. doi: 10.1007/s00422-003-0422-x

### AUTHOR CONTRIBUTIONS

AV and PT conceived the study. MM conducted the numerical simulations and theoretical approximations. MM, AV, and PT analyzed the results. MM, AV, and PT wrote and reviewed the paper.

#### ACKNOWLEDGMENTS

PT gratefully acknowledges the generous research funding support from the John A. Blume Foundation.

Abbott, L. F., and Nelson, S. B. (2000). Synaptic plasticity: taming the beast. Nat. Neurosci. 3, 1178–1183. doi: 10.1038/81453

Adamchic, I., Hauptmann, C., Barnikol, U., Pawelcyk, N., Popovych, O. V., Barnikol, T., et al. (2014a). Coordinated reset has lasting aftereffects in patients with parkinson's disease. Mov. Disord. 29, 1679–1684. doi: 10.1002/mds.25923


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Madadi Asl, Valizadeh and Tass. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Features of Neural Network Formation and Their Functions in Primary Hippocampal Cultures in the Context of Chronic TrkB Receptor System Influence

Tatiana A. Mishchenko1,2 \*, Elena V. Mitroshina1,2, Alexandra V. Usenko<sup>1</sup> , Natalia V. Voronova<sup>1</sup> , Tatiana A. Astrakhanova<sup>1</sup> , Olesya M. Shirokova<sup>2</sup> , Innokentiy A. Kastalskiy<sup>1</sup> and Maria V. Vedunova<sup>1</sup> \*

#### Edited by:

Shangbin Chen, Huazhong University of Science and Technology, China

#### Reviewed by:

Jihwan Myung, Taipei Medical University, Taiwan Vincenzo Marra, University of Leicester, United Kingdom

#### \*Correspondence:

Tatiana A. Mishchenko saHarnova87@mail.ru Maria V. Vedunova MVedunova@yandex.ru

#### Specialty section:

This article was submitted to Computational Physiology and Medicine, a section of the journal Frontiers in Physiology

Received: 27 August 2018 Accepted: 20 December 2018 Published: 10 January 2019

#### Citation:

Mishchenko TA, Mitroshina EV, Usenko AV, Voronova NV, Astrakhanova TA, Shirokova OM, Kastalskiy IA and Vedunova MV (2019) Features of Neural Network Formation and Their Functions in Primary Hippocampal Cultures in the Context of Chronic TrkB Receptor System Influence. Front. Physiol. 9:1925. doi: 10.3389/fphys.2018.01925 <sup>1</sup> Department of Neurotechnology, Institute of Biology and Biomedicine, National Research Lobachevsky State University of Nizhni Novgorod, Nizhny Novgorod, Russia, <sup>2</sup> Molecular and Cell Technologies Group, Central Scientific Research Laboratory, Privolzhsky Research Medical University, Nizhny Novgorod, Russia

Discovering the mechanisms underlying homeostatic regulation in brain neural network formation and stability processes is one of the most urgent tasks in modern neuroscience. Brain-derived neurotrophic factor (BDNF) and the tropomyosin-related kinase B (TrkB) receptor system have long been considered the main regulators of neuronal survival and differentiation. The elucidation of methods for studying neural network activity makes investigating the complex mechanisms underlying neural network structure reorganization during development and detecting new mechanisms for neuronal activity remodeling possible. In this in vitro study, we investigated the effects of chronic BDNF (the main TrkB stimulator) and ANA-12 (a TrkB receptor system blocker) administration on the formation of neural-glial networks. The formation of spontaneous bioelectrical activity and functional neural structure depend on TrkB receptors, and blocking TrkB receptors inhibits full bioelectrical activity development. Cross-correlation analysis demonstrated the decisive role of TrkB in the formation and "strengths" of activity centers. Even though an appropriate ANA-12 concentration is non-toxic to nerve cells, numerous cells in culture medium containing this reagent do not exhibit metabolic activity and are not functionally involved in signal transmission processes. Electron microscopy studies revealed that chronically influencing the TrkB receptor system significantly alters synaptic and mitochondrial apparatus capture in cells, and functional analysis of mitochondrial activity confirmed these findings. Because knowledge of interactions between TrkB-mediated regulation and the mitochondrial state under normal conditions is rather limited, data on these relationships are particularly interesting and require further investigation. Thus, we assume that the molecular cascades mediated by TrkB actively participate in the formation of functionally complete brain neural networks.

Keywords: TrkB receptor, brain-derived neurotrophic factor (BDNF), primary hippocampal cell cultures, neural networks, multielectrode arrays, calcium imaging, mitochondrial functional activity

## INTRODUCTION

fphys-09-01925 January 8, 2019 Time: 16:17 # 2

Neural networks are currently considered the minimal functional unit of the central nervous system (CNS), and formation of the brain neural networks responsible for processing and transmitting information is a complex process characterized by a number of critical stages (Shirokova et al., 2013; Yuste, 2015). Each stage has its own peculiarities, and issues in any of these stages lead to the development of a functionally inferior structure. Even though the brain structure is formed before birth, its full development is mainly determined by the nature of postpartum stimuli from the environment. External stimuli modulate functional brain maturation and neurogenesis in adulthood (Lledo et al., 2006; de Almeida et al., 2013; Sailor et al., 2017). Notably, the formation of synaptic contacts between neurons continues throughout life. Organization of a stable neural network structure in the brain and reconsolidation of synaptic contacts under stress conditions are associated with the activation of numerous intracellular signaling cascades (de Almeida et al., 2013; Yu and Yu, 2017). Searching for endogenous compounds involved in the formation of complex spatial and functional neural networks is one of the most urgent tasks in modern neuroscience.

Brain-derived neurotrophic factor (BDNF) is a promising signaling molecule that may have a generalized effect on the formation and reconsolidation of neural networks. BDNF is involved in the regulation of neurogenesis, neuronal development and survival (Martin and Finsterwald, 2011; Park and Poo, 2013; Skaper, 2018); it also plays a crucial role in early neuronal differentiation, synaptic development, neural outgrowth, mature neuron survival, and synaptic plasticity (Douglas-Escobar et al., 2012; Leal et al., 2015; Kowianski et al., ´ 2018; von Bohlen Und Halbach and von Bohlen Und Halbach, 2018).

The main functions of BDNF are mediated by its interaction with the tropomyosin-related kinase B (TrkB) receptor (Patapoutian and Reichardt, 2001; Skaper, 2018) and the possibility of intracellular signaling cascade activation, which can indirectly affect synaptic transmission and synaptic contact formation, thus determining the neural network structure (Ohira and Hayashi, 2009; Guo et al., 2014; Mitre et al., 2017). TrkB receptors are normally localized within vesicles inside the cell and translocate to the plasma membrane through neuronal activity (Meyer-Franke et al., 1998; Du et al., 2000). These data are consistent with a central role of neurotrophins as mediators of activity-dependent plasticity (Poo, 2001; Lu et al., 2014). BDNF plays a key role in mediating activityinduced long-term potentiation (LTP) (Leal et al., 2014). The early effects of BDNF result from the modification (e.g., protein phosphorylation) of components already present at the synapse, while the long-term effects arise from the modification of translational activity at the synapse and changes in transcription. High-frequency stimulation that induces LTP increases BDNF production (Castrén et al., 1993). Furthermore, BDNF increases neurotransmitter release and promotes synaptic transmission and LTP (Panja and Bramham, 2014; Kuipers et al., 2016). Thus, it is reasonable to assume that the effects exerted by BDNF on synaptic plasticity are TrkB mediated.

Investigating neural network formation at different levels of neuron-glial system organization under chronic influence of the TrkB receptor system during development is of particular interest. Application of mathematical methods for biological data analysis may help reveal the features of neural network internal structure formation and the influence of TrkB signaling on the nerve impulse transmission process as well as predict the effects of endogenous BDNF dynamics on neural networks.

Our present study is devoted to investigating the features of neural network formation and functions in primary hippocampal cultures in the context of chronic BDNF application and TrkB receptor blockage.

## MATERIALS AND METHODS

### Ethics Statement

All experimental protocols utilized in this study were approved by the Bioethics Committee of Lobachevsky University and carried out in accordance to Act708n (23 08 2010) of the Russian Federation National Ministry of Public Health, which states the rules of laboratory practice for the care and use of laboratory animals, and the Council Directive 2010/63 EU of the European Parliament (September 22, 2010) on the protection of animals used for scientific purposes. C57BL/6J mice were killed by cervical vertebra dislocation, and their embryos were then surgically removed and sacrificed by decapitation.

### Experimental Scheme

Recombinant BDNF (1 ng/mL, Merck, GF301, Germany), ANA-12 (a selective TrkB receptor blocker, 1 µM, Sigma-Aldrich, SML0209, Germany) or BDNF (1 ng/mL) and ANA-12 (1 µM) in combination were added to the culture medium daily beginning on the third day of culture development in vitro (DIV) (**Figure 1**).

Analyses of spontaneous bioelectrical and calcium activities, which included defining the internal neural network structure,

morphological studies, and evaluation of the dynamics of functional characteristics of the mitochondrial respiratory chain in neural cells, were performed on DIV 7, 10, and 14.

### Cell Culture

Hippocampal cells were obtained from mouse embryos (day 18 of gestation) and cultivated on multielectrode arrays (MEAs) (Multichannel Systems, Germany) or coverslips pretreated with polyethyleneimine solution (1 mg/mL) (Sigma-Aldrich, P3143) according to a previously developed protocol (Vedunova et al., 2013). Hippocampi were dissected in Ca2+- and Mg2+-free phosphate buffered saline (PBS-minus) followed by 20 min of enzymatic treatment with 0.25% trypsin-ethylenediaminetetraacetic acid (EDTA, Invitrogen, 25200-056, United States). Next, the cells were carefully suspended and centrifuged at 1,000 rotations per min (rpm) for 3 min. The cell pellet was immediately resuspended in neurobasal medium (Invitrogen, 21103-049) supplemented with 2% B27 (Invitrogen, 17504-044), 0.5 mM L-glutamine (Invitrogen, 25030-024), and 5% fetal bovine serum (PanEco, K055, Russia). Dissociated cells were plated on MEAs and coverslips at an approximate initial density of 9,000 cells/mm<sup>2</sup> . After 24 h, 50% of the culture medium was replaced with neurobasal medium containing a lower concentration of fetal bovine serum (0.4%). Fifty percent of the medium was changed every third day, and cell viability was maintained under constant conditions of 35.5◦C, 5% CO<sup>2</sup> and a humidified atmosphere in a cell culture incubator.

### Cell Viability Detection

To determine the viability of dissociated hippocampal cells upon application of different concentrations of ANA-12, we estimated the ratio of the number of dead cells stained with propidium iodide (Sigma-Aldrich, P4170) to the total number of cells stained with bisbenzimide (Invitrogen, H3570). Propidium iodide and bisbenzimide at concentrations of 5 µg/mL and 1 µg/mL, respectively, were added to the culture medium 30 min before the viability was measured (Vedunova et al., 2015). Cells were observed under a Leica DMIL HC inverted fluorescence microscope (Leica, Germany) with a 10×/0.2Ph1 objective.

### Electrophysiological Methods

Extracellular potentials were collected using 59 planar TiN electrodes integrated into the USB-MEA-120 system (Multichannel system, Germany). The MEAs had 59 electrodes (8 × 8 grid) with a diameter of 30 µm and were spaced 200 µm apart. Data were recorded simultaneously from 59 channels at a sampling rate of 20 kHz/channel. All signaling and statistical analyses were performed using custom-made software (MATLAB <sup>R</sup> 6.0, Natick, MA, United States).

Small network bursts were detected by calculating the total spiking rate (TSR), which considered the total number of spikes from all electrodes within 50-ms time bins. The criterion for a small network burst was the rapid appearance of a large number of spikes over four electrodes within a small (50 ms) time bin (Pimashkin et al., 2011; Vedunova et al., 2013; Gladkov et al., 2018).

#### Spike Detection

The recorded extracellular action potentials were detected by threshold calculations using the signal median as follows:

$$T = \text{N}\_s \sigma, \sigma = median \left(\frac{|\chi|}{0.6745}\right) \tag{1}$$

where x is the bandpass-filtered (0.3–8 kHz) data signal, σ is an estimate of the standard deviation of the signal without spikes, and N<sup>s</sup> is the spike detection coefficient that determines the detection threshold. Threshold estimations based on the median of the signal in the form of Eq. (1) are less dependent on the frequency of the spikes than threshold estimates based on the standard deviation during signal processing. The coefficient 0.6745 in Eq. (1) was used to normalize the median of the absolute signal to the standard deviation. N<sup>s</sup> = 4 was used for all data, which allowed the reliable detection of spikes with amplitudes greater than 20 µV. The minimal interspike interval was set to 1 ms. Detected spikes were plotted using raster diagrams (Quiroga et al., 2004; Pimashkin et al., 2011).

#### Small Burst Detection

We recorded spontaneous burst activity to analyze the effects of chronic BDNF and ANA-12 application on the functional characteristics of neural networks in primary hippocampal cultures. The TSR was determined by counting the total number of spikes from all electrodes within 50-ms time bins for small network burst detection. The rapid emergence of a large number of spikes over multiple electrodes within a small (50 ms) time bin was used as the criterion for a small network burst. Spontaneous activity in the cultures corresponded to the basal stochastic activity, which was observed in fractions of cells together with short burst episodes. The spike trains (approximately 1 spike per 10–100 ms) were considered to represent basal activity. To reveal bursts, we used a threshold detection based on the statistical characteristics of the spontaneous activity TSR(t). The burst threshold was set to TBurst = 0.1 × σ TSR, where σ TSR was the standard deviation of TSR(t). The burst detection threshold coefficient was empirically set to 0.1, yielding the best estimate for the burst initiation and end points according to the raster diagram. Simulations of bursts with frequencies up to 5 Hz revealed that the estimated burst durations were within 10% of the actual values. Statistical analysis of the bursting activity characteristics was performed using analysis of variance (ANOVA, p < 0.05) (Pimashkin et al., 2011; Vedunova et al., 2013).

#### Cross-Correlation Method and Graphs

The dataset, obtained from spontaneous bioelectrical network activity recordings, was represented as a raster plot. The existence of functional connections between neuronal groups was not obvious based on visual analysis. The network graph method was then used to detect the neuronal groups.

First, to assess the degree of synchronization between all pairs of cells, considering axonal delays, we calculated the proportion of transmitted spikes. This measure is an analog of the cross-correlation coefficient of continuous signals. According to this method, the number of "delayed synchronous" spikes was

calculated. These spikes had to be recorded from both channels within a tolerance interval of δ for which the time delay τ between the centers of the spikes was proportional to the distance between the electrodes. The number of delayed synchronous spikes was normalized by the number of spikes received by the postsynaptic neuron n<sup>j</sup> . Thus, the cross-correlation matrix was calculated using the following formula:

$$C\_{ij} = \frac{n\_{synchr,ij}}{n\_j} \tag{2}$$

we then selected the largest 5% of Cij coefficients and defined a set of indices, i.e., hubs of cells with a maximum number of functionally active connections. In addition, for each hub "i," we calculated the number of connections to index i within the array Cij.

Next, the graph was constructed. The vertex size was proportional to the number of significant connections, and the edge of the graph corresponded to the functional connections of spikes transferred from one neuron to another at individual time points for each pair of axonal delays, i.e., τ ± δ/2 (Shishkina et al., 2018).

The cross-correlation method and graphs allow the detection of hubs, i.e., elements with the maximum number of functionally active connections, and show the dynamic changes occurring in the network in short- and long-term periods. The hub coefficient was calculated as the ratio of the number of connections of an electrode to its total number in the graph; therefore, it characterized the importance of a group of neurons located at one electrode to network activity. The hub coefficient allows an estimation of changes in the significance of each electrode in an MEA.

## Ca2<sup>+</sup> Imaging

To conduct functional calcium imaging, we dissolved Oregon Green 488 BAPTA-1 AM (OGB-1) (0.4 µM, Invitrogen, O-6807) in dimethylsulfoxide (DMSO) (Sigma-Aldrich, D8418) with 4% pluronic F-127 (Invitrogen, P-3000 MP) and then added it to the culture medium for 40 min. After incubation to allow full absorption of OGB-1 molecules by the cells, the cells were washed with dye-free medium for 15 min. A confocal laser-scanning microscope (Zeiss LSM 510, Germany) with a W Plan-Apochromat 20×/1.0 objective was used to visualize spontaneous calcium activity in the dissociated cultures.

Cytosolic Ca2<sup>+</sup> was visualized via OGB-1 excitation with an Argon laser at 488 nm and emission detection with a 500–530 nm filter. Time series of 256 × 256 pixel images capturing 420 µm × 420 µm fields of view were recorded at 4 Hz. A confocal pinhole of 1 airy unit was used to obtain an axial optical slice resolution of 1.6 µm.

Quantitative evaluation of Ca2<sup>+</sup> transients was performed off-line using custom-made software in C++ Builder. Cell regions from fluorescent images were manually selected. The Ca2<sup>+</sup> fluorescence of each cell in each frame was calculated as the average fluorescence intensity (F, relative units from 0 to 255) of the pixels within the defined cell region. Single Ca2<sup>+</sup> signals were identified using the following algorithm. First, each trace from all of the cells was filtered by averaging two neighboring points in the sample set. Next, we calculated a simple derivative of the signal by determining the difference between each pair of consequent points. The pulses were identified from the derivative of the trace using a threshold detection algorithm. The threshold was estimated as the detection accuracy coefficient multiplied by the standard deviation of the derivative of the trace. Suprathreshold points on the derivative of the trace were taken as the beginnings and ends of the pulses (Zakharov et al., 2013).

#### Electron Microscopy

Primary hippocampal cultures were fixed in 2.5% glutaraldehyde (Acros Organics, AC119980010, United States) on DIV 10 and DIV 14. The cultures were then washed three times with PBS and treated with 1% osmium tetroxide (Sigma-Aldrich, 20816-12-0) for 60 min. After additional washing steps, the dissociated hippocampal cells were dehydrated in a series of ethanol solutions of increasing concentration (30–100%) followed by 100% acetone and then embedded in a mixture of acetone/EPON resin (50:50). The culture was ultimately embedded in EPON resin (Fluka, United States).

For electron microscopy, the resin blocks were cut using a Leica EM UC7 ULTRA ultramicrotome (Leica, Germany). Ultrathin sections were contrasted with 4% uranyl acetate (SPI-chem, 02624-AB, United States), lead citrate and trihydrate (SPI-chem, 512-26-5). The ultrathin sections were examined with a Morgagni 268D transmission electron microscope (FEI Company, United States).

### Registration of Mitochondrial Functional Activity

Mitochondrial functional activity was analyzed on DIV 10 and DIV 14. Mitochondria were isolated using the standard differential centrifugation method (Pallotti and Lenaz, 2001; Schmitt et al., 2015), and the dissociated hippocampal cells were enzymatically [versine-trypsin (3:1) solution] removed from the cultivation substrate. The subsequent manipulations were performed on ice, and the equipment and isolation media were also cooled. The cells were placed into a precooled porcelain mortar; washed with an ice-cold isolation medium comprising 70 mM saccharose, 210 mM mannitol, 30 mM HEPES, and 0.1 mM EDTA (pH 7.4); and then subjected to homogenization in a glass homogenizer. An electrically driven Teflon pestle had a clearance excluding mitochondrial destruction. The obtained homogenate was centrifuged at 4,000 rpm (temperature ranging from −3 to 0◦C) for 10 min. The precipitate was resuspended in medium containing 210 mM mannitol, 70 mM saccharose, 0.1 mM EGTA, and 10 mM HEPES (pH 7.4). The obtained mitochondrial suspension was stored on ice to avoid freezing. The Bradford method was used to quantitate the protein in the isolated mitochondria.

Oxygen consumption by the isolated mitochondria was registered polarographically using the high-resolution respirometer OROBOROS Oxygraph-2k (OROBOROS Instruments, Austria) in 2 mL incubation medium (210 mM mannitol, 70 mM saccharose, 0.1 mM EGTA, 10 mM HEPES, pH

7.4) with constant stirring. The oxygen consumption rate was expressed in picomol/s/1 mg mitochondrial protein.

The oxygen consumption in the chamber was fixed using DatLab5 software (OROBOROS Instruments, Austria).

The state of the mitochondrial respiratory chain was evaluated according to the following parameters: the rate of oxygen consumption by mitochondria with a high substrate content, 5 mM glutamate and 5 mM malate (substrates of complex I), in the incubation medium; the oxidative phosphorylation rate of the respiratory chain in the presence of 5 mM adenosine diphosphate (ADP); the inhibition of complex I activity with 0.5 µM rotenone, and the work intensity of the respiratory chain after the stimulation of complex II with 10 mM sodium succinate.

#### Statistical Analysis

All quantified data are presented as the mean ± standard error of the mean (SEM). Statistical analyses were performed using two-way ANOVA implemented in Sigma Plot 11.0 software (Systat Software, Inc.). The Student–Newman–Keuls (SNK) test was used as a post hoc test following ANOVA. Differences between groups were considered significant if the corresponding p-value was less than 0.05.

## RESULTS

First, we selected an appropriate concentration for the selective TrkB receptor blocker ANA-12. ANA-12 is low-molecular-weight heterocyclic compound that affects the formation of a functional complex between BDNF and TrkB (Cazorla et al., 2011). ANA-12 binds the extracellular fifth subdomain of TrkB (TrkB-d5). Given the size of BDNF relative to ANA-12, although the N-terminal arm of the neurotrophin competes with ANA-12 for the TrkB-d5-binding pocket, it can be assumed that the small compound can easily bind to the active center of TrkB.

A single application of ANA-12 at a concentration of 24 µM exerted a pronounced toxic effect on primary hippocampal cultures. The number of viable cells treated with 24 µM ANA-12 was significantly decreased by 1.3-fold compared to that in the "sham" group (day 7 after the addition; sham: 91.32 ± 2.17; ANA-12: 70.24 ± 3.42). Moreover, even though single ANA-12 applications at concentrations of 12 µM, 10 µM, and 5 µM did not affect cell viability, this daily application caused high primary hippocampal cell mortality by DIV 7. Therefore, the concentration of ANA-12 was reduced to 1 µM. According to previous studies, the use of ANA-12 at this concentration completely negates the neuroprotective effect of BDNF (Cazorla et al., 2011; Longo and Massa, 2013; Saba et al., 2018). These findings indicate that such concentrations of ANA-12 are sufficient to block the majority of TrkB receptors. Application of BDNF, ANA-12 and BDNF and ANA-12 in combination was carried out according to the scheme presented in the "Materials and Methods" section.

To identify the role of TrkB receptors in the formation of neural network activity and the adaptive potential of brain cells in the context of chronic test substance usage, we performed analyses of spontaneous bioelectrical and calcium activities, morphological studies, and evaluations of the dynamics of mitochondrial respiratory chain functional characteristics in neural cells on DIV 7, 10, and 14.

### Features of the Spontaneous Bioelectrical Activity of Neural Networks in Primary Hippocampal Cultures in the Context of Chronic TrkB Receptor System Influence

Electrophysiological data analysis revealed that daily application of the tested substances to the culture medium modulated spontaneous bioelectrical activity in primary hippocampal cultures.

According to the classical concept, a network burst is considered an event comprising no fewer than four spikes simultaneously recorded from different electrodes in a 50 ms interval (Wagenaar et al., 2006; Pimashkin et al., 2011; Vedunova et al., 2013). As the main purpose of this study was to investigate the neural network complex structure, we focused on the events that simultaneously captured the prevailing portion of the functionally active cells. In this regard, all neural network bursts were conditionally divided into small (from 4 to 100 spikes in 50 ms) and large (101 or more spikes in 50 ms) groups (**Figure 2** and **Supplementary Table 1**). The detection of large network bursts allows the identification of the network structure by the cross-correlation method and graphs and the presentation of the neural network activation pattern.

The development of neural network activity is associated with the gradual formation of new contacts between neurons and synapse maturation.

The first network bursts were registered in the "sham" group on DIV 7. Although small network bursts prevailed (approximately 82.3 ± 10.76% of all network bursts), large neural network events were also observed. However, the number of spikes in a burst remained relatively small (number of large bursts/5 min: 34.49 ± 7.32; number of spikes in a large burst: 123.82 ± 18.97). On DIV 10, the number of large network bursts was decreased (number of large bursts/5 min: 21.04 ± 3.21), whereas the number of spikes in a large burst was significantly increased (number of spikes in a large burst: 226.78 ± 41.51). On DIV 14 on the maturing synaptic contact background, an insignificant increase in the number of large bursts and stabilization of bioelectrical parameters were observed (number of large bursts/5 min: 27.87 ± 5.21; number of spikes in a large burst: 506.54 ± 67.11).

Chronically blocking TrkB receptors leads to the modulation of neural network activity. On DIV 14, the number of large bursts was significantly decreased to 12.65 ± 4.65 after TrkB receptors were chronically blocked, and the number of spikes in a large burst was also decreased compared to that in the sham group and amounted to 218.54 ± 65.12.

Notably, the largest number of large network bursts was registered in cultures to which BDNF was applied chronically (number of large bursts/5 min on DIV 14: 49.56 ± 5.67), and the number of spikes in a burst was comparable to that in the

sham group (number of spikes in a large burst on DIV 14: 409.65 ± 78.32).

The combined application of BDNF and ANA-12 activated neural network activity, which is specifically related to the period of bioelectrical process reformatting and replacement of the main paths of information transmission from electrical to chemical synapses (DIV 10). In this period, the number of large bursts/5 min was 42.34 ± 8.45, and the number of spikes in a large burst was 320.78 ± 59.89. By DIV 14, despite the high number of large bursts (39.08 ± 8.76), no significant increase in the number of spikes in a large burst was observed; this parameter remained the same as that observed on DIV 10 (**Figure 3** and **Supplementary Figure 1**).

The characteristic neural network activation profile showed an increase in the activation time during culture development, which can be attributed to signal transmission through chemical synapses occurring on a time delay and creating complicated neural network structures during developmental processes. The use of BDNF exerts a more pronounced effect on increasing the transmission time of the first signal, which is probably associated with a change in the proportion of different synapses under chronic neurotrophin application. In another experimental group, changes in the pattern activation profile were not observed (see **Supplementary Figure 2**).

A cross-correlation analysis showed the complicated neural network structure and the appearance of hubs during the in vitro culture development (**Figure 4**). The gradual formation of a sustainable neural network without the redistribution of activity centers was observed in the "sham" group, and 77.56 ± 17.23% of the connections remained stable from DIV 10 to DIV 14 (**Table 1**).

Chronically influencing the TrkB receptor system leads to changes in the parameters of spontaneous bioelectrical activity and significantly affects neural network stability. During neural network formation, an active redistribution of the network structure was observed in all experimental groups (percentages of overlap between DIV 10 and DIV 14: BDNF: 57.37 ± 19.56%; ANA-12: 5.75 ± 2.82%; BDNF + ANA-12: 26.46 ± 9.8%) (**Table 1**).

In addition, an increased number of hubs were observed during the development of primary cultures in vitro. This change was apparently related to the complexity of the network structure. The gradual formation of the hub structure and increased significance of individual electrodes are shown for the "sham" and "BDNF" groups. Blocking TrkB receptors decreased the number of hubs in all observation periods (**Table 2**).

The same tendency was observed for the parameter involving the number of connections in a hub. During a period of active neural structure formation, increases in the number of connections in a hub were observed in the "sham" and "BDNF" groups (**Table 3**).

### Features of Spontaneous Calcium Activity in Neural Networks of Primary Hippocampal Cultures in the Context of Chronic TrkB Receptor System Influence

Next, we investigated the features of functional calcium activity in primary hippocampal cell cultures under chronic TrkB receptor system influence.

Registration of calcium dynamics in neural cells using the Ca2<sup>+</sup> imaging technique is considered the most informative method for studying metabolic neural network activity (Braet et al., 2004; Jercog et al., 2016; Carrillo-Reid et al., 2017). Ca2<sup>+</sup> is a key regulator of various metabolic processes, and registration of its concentration dynamics in the cytoplasm allows for fine analyses of both neuronal and glial activity.

On DIV 7, spontaneous calcium activity was observed in the "sham" culture, and the percentage of cells that exhibited Ca2<sup>+</sup> activity was 52.65 ± 3.69% (**Supplementary Table 2A**). In all experimental groups, this parameter was comparable to the intact values (percentages of cells that exhibited Ca2<sup>+</sup> activity on DIV 7: BDNF: 51.14 ± 3.05%, ANA-12: 47.98 ± 2.95%, BDNF + ANA-12: 42.97 ± 5.23%).

Beginning on DIV 10, the percentage of cells that exhibited Ca2<sup>+</sup> activity was significantly lower in the "ANA-12" group than in the "sham" group (percentages of cells that exhibited Ca2<sup>+</sup> activity on DIV 10: sham: 57.30 ± 3.88%; ANA-12: 35.37 ± 2.39%). The number of active cells in the "BDNF + ANA-12" group was comparable to that in the "BDNF" group (percentage of cells that exhibited Ca2<sup>+</sup> activity on DIV 10: BDNF: 65.92 ± 3.94%; BDNF + ANA-12: 71.50 ± 3.70%).

On DIV 14, the percentage of cells exhibiting Ca2<sup>+</sup> activity in the "BDNF" group was 79.39 ± 2.52%, which significantly exceeded the values in the "sham" (62.77 ± 3.84%), "ANA-12" (36.38 ± 6.06%) and "BDNF + ANA-12" (50.1 ± 5.67%) groups (**Figures 5**, **6**).

During the early stage of culture development (DIV 7), the frequencies of Ca2<sup>+</sup> oscillations in the "BDNF" and "ANA-12" groups did not differ from the intact values [frequencies of Ca2<sup>+</sup> oscillations (ocs/min): sham: 0.74 ± 0.09; BDNF: 0.85 ± 0.10; ANA-12: 0.76 ± 0.06]. In the "BDNF + ANA-12" group, this parameter exceeded the intact values by 1.87-fold and

TABLE 1 | Functional rearrangement alterations in neural networks during primary hippocampal culture development in vitro.


TABLE 2 | Number of hubs in the primary hippocampal neural network on different days of development in vitro.


∗ versus "Sham" p < 0.05, ANOVA, N = 9.

TABLE 3 | Average number of connections in a hub in the primary hippocampal neural network on different days of development in vitro.


∗ versus "Sham" p < 0.05, ANOVA, N = 9.

amounted to 1.39 ± 0.12 ocs/min (**Supplementary Table 2B**). A similar change dynamic was observed on DIV 10; however, this parameter was significantly decreased relative to that in the "sham" group on DIV 14 [frequencies of Ca2<sup>+</sup> oscillations (ocs/min): sham: 1.55 ± 0.08; BDNF + ANA-12: 0.82 ± 0.07]. At this stage of culture development, a decreased frequency of Ca2<sup>+</sup> events in the "ANA-12" group was also observed (0.81 ± 0.05 ocs/min). In the "BDNF" group, the frequency of Ca2<sup>+</sup> oscillations was increased on DIV 10. This parameter was normalized to the intact values by day DIV 14 (1.41 ± 0.31 ocs/min) (**Figures 5**, **6**).

The duration of Ca2<sup>+</sup> oscillations in all experimental groups did not significantly differ from the intact values on DIV 7 (**Supplementary Table 2C**). From DIV 10, a decrease in this parameter was observed in the "ANA-12" and "BDNF + ANA-12" groups [durations of Ca2<sup>+</sup> oscillations (s): sham: 9.67 ± 0.69; ANA-12: 7.21 ± 0.47; BDNF + ANA-12: 6.48 ± 0.30]. Chronic BDNF application did not affect the duration of Ca2<sup>+</sup> oscillations throughout the entire observation period (BDNF on DIV 7: 10.5 ± 0.28; BDNF on DIV 10: 8.46 ± 0.43; BDNF on DIV 14: 10.73 ± 0.71) (**Figures 5**, **6**).

Thus, activation of the TrkB receptor system during development promotes the stimulation of neural network functional calcium activity, which was manifested in the increased number of cells exhibiting calcium activity and the frequency of network calcium events. Blocking TrkB receptors leads to the irreversible suppression of spontaneous calcium activity in primary hippocampal cultures beginning on DIV 10.

### Ultrastructural Features of Neural Networks in Primary Hippocampal Cultures in the Context of Chronic TrkB Receptor System Influence

Qualitative and quantitative ultrastructural analyses of primary hippocampal cell cultures performed on DIV 10 and DIV 14 allowed the identification of culture organization features in the context of chronic TrkB receptor system influence (**Figures 7**, **8**).

Many axonal buds containing equal-sized synaptic vesicles and dendritic spines without contacts were observed in the "sham" group on DIV 10. At this stage of culture development, a few mature symmetric and asymmetric contacts were observed. Furthermore, glycogen granules were visualized in glial outgrowths, and many cones of growth were observed (**Figures 7a–e**).

Previous studies have shown that a single BDNF application affects the functional and ultrastructural characteristics of neural and glial brain mitochondria (Markham et al., 2012). The current study did not reveal significant changes in the mitochondrial apparatuses (occupied area or amount or ultrastructural organization) of primary hippocampal cells chronically administered BDNF, ANA-12 or BDNF and ANA-12 in combination. The mitochondrial structure typically exhibits smooth cristae and an osmiophilic matrix, and the organelle shape is oval. However, in the "BDNF" group, insignificant changes in these parameters were found. In addition, numerous glycogen granules in glial outgrowths as well as numerous ribosomes in the granular endoplasmic reticulum and free ribosomes were visualized (**Figures 7f–j**).

Significant changes in the mitochondrial apparatus structure were observed in the "ANA-12" group. Mitochondria with impaired internal organization were visualized in the view field. However, no quantitative changes in the occupied area or the number of mitochondria per µm<sup>2</sup> of neuronal soma were detected (**Figure 9**). Mitochondria with modified shapes and intermitochondrial contacts in the outgrowth were also observed. In this group of cultures, axonal cytoskeleton violations were visualized, and "empty" axons containing only synaptic vesicles were identified in the observation field. Delayed synaptogenesis was also observed.

Mature synapses had weak postsynaptic density (PSD) osmiophility, and "contacts" between the axonal bud and a dendrite or dendritic spine in the absence of PSD were often observed. A distinctive feature of the ultrastructural organization under a chronic TrkB receptor blockade was the cistern expansion in the smooth endoplasmic reticuli of glial and neural outgrowths. Reconstruction of the Golgi apparatus lamellar complex and its cistern overgrowth were observed in the cell body (**Figures 7k–o**).

In contrast to the "sham" group, the "BDNF + ANA-12" group exhibited disrupted mitochondria and mitochondria with impaired or vacuolated cristae. Nevertheless, numerous

synaptic contacts and a well-defined PSD were observed in "BDNF + ANA-12" cultures compared to those in "ANA12" cultures (**Figures 7p–t**).

No significant differences in the number of mature synaptic contacts were observed on DIV 14 in any experimental groups (**Figure 8**). Thus, delayed synaptogenesis in the "ANA12" group was overcome on DIV 14. This delay was presumably associated with the cell synthetic apparatus because weak PSD was detected on DIV 10, although PSD is normally formed first during cell culture ontogenesis (Meyer et al., 2014).

Analysis of PSD features in postsynaptic asymmetric contacts under chronic administration of the tested substances was performed on DIV 14 (**Figure 10**). We did not find significant changes in the area or extent of the PSD. However, the extent of the PSD in the "BDNF + ANA-12" group had a tendency to decrease (sham: 0.511 ± 0.31, BDNF + ANA-12: 0.32 ± 0.13). These findings could be explained by the high variability of synaptogenesis at this stage of development. Because the synaptogenic processes had not been completed by DIV 14, we observed synapses with various sizes and characteristics. Synapses in the "BDNF + ANA-12" group were short (**Figures 8p–t**), and changes in the presynaptic terminal were also observed.

An analysis of PSD osmiophility in asymmetric synapses revealed that the percentage of high osmiophilic PSD in the "BDNF" group was increased by 38.69% compared to that in the "sham" group, which is potentially indicative of more efficient signal transmission between neurons (**Figure 11**). By contrast, a delay in synaptogenesis was observed in the "ANA-12" group, and 16.16% fewer synaptic contacts were observed compared to intact cultures. Despite the reduced PSD length, the osmiophility in the "BDNF + ANA-12" group was comparable to that in the "sham" group.

The number of mitochondria in the neuronal soma in the "ANA-12" group was significantly decreased compared to that in the "sham" group and amounted to 43.73 ± 5.66 per 100 µm<sup>2</sup> (**Figure 9A**). These changes are associated with organelle destruction. In parallel, increased individual mitochondria areas were observed in the "ANA-12" group (**Figure 9B**).

Thus, TrkB receptor blockade led to significant ultrastructural changes in primary hippocampal cell cultures, which was primarily associated with alterations in the structure of the mitochondrial cell apparatus. The absence of PSD in the synaptic contact structure was also observed during the early stages of culture development. The number of mature synaptic contacts during synaptogenesis was normalized in later stages of cultivation.

### Mitochondrial Functional Activity in Primary Hippocampal Cell Cultures in the Context of Chronic TrkB Receptor System Influence

Changes in the number of mitochondria and their structure upon influencing the TrkB receptor system are of particular interest.

In this regard, the final stage of our study was devoted to the assessment of dynamic changes in the functional characteristics of the mitochondrial respiratory chain in primary hippocampal cells in response to chronic BDNF and ANA12 administration to the culture medium.

Daily BDNF application increased the mitochondrial oxygen consumption rate (**Figure 12A**). The basal oxygen consumption rate in the "BDNF" group significantly exceeded that in the "sham" and "ANA-12" groups and amounted to 272.2 ± 11.5 pmol/(s∗mL) on DIV 10 and 217.1 ± 20.5 pmol/(s∗mL) on DIV 14. The basal oxygen consumption rate in the "ANA-12" group did not differ from that in the sham group throughout the entire observation period.

In addition, neural-glial network development is associated with increased mitochondrial respiratory chain complex I activity (**Figure 12B**). On DIV 14, the mitochondrial respiratory chain activity in the "sham" group was 383 ± 17.1 pmol/(s∗mL), which was increased by 2.4-fold compared to that on DIV 10 [159.1 ± 3.9 pmol/(s∗mL)]. Similar tendencies were observed in the "BDNF" and "ANA-12" groups. However, on DIV 14, the mitochondrial respiratory chain complex I activities in these experimental groups were significantly lower than the intact values. Thus, blocking TrkB receptors leads to a marked decrease in mitochondrial respiratory chain activity. The oxygen consumption rate in the "ANA-12" group was 273.9 ± 23.8 pmol/(s∗mL), which was significantly lower than that in the "BDNF" [347.3 ± 4.0 pmol/(s∗mL)] and "sham" [383 ± 17.1 pmol/(s∗mL)] groups.

Alternatively, the intensity of mitochondrial respiratory chain complex II activity was decreased during culture development. In contrast, chronic BDNF administration increased the respiratory complex II activity (**Figure 12C**). The oxygen consumption rates in the "BDNF" group on DIV 10 and DIV 14 were 104.9 ± 5.1 pmol/(s∗mL) and 49.5 ± 7.5 pmol/(s∗mL), respectively. This parameter in the "ANA-12" group did not differ from the intact values.

Thus, BDNF application stimulates mitochondrial functional activity in primary hippocampal cell cultures and manifested as

FIGURE 7 | Representative electron microscopy images of dissociated hippocampal cells on DIV 10. (a–e) Sham, (f–j) BDNF, (k–o) ANA-12, (p–t) BDNF + ANA-12. (a) Axo-spiny synapse; the cristae of mitochondria in a dendrite are smooth. (b) Neuronal soma; the cytoplasm shows well-visualized mitochondria, Golgi apparatus, granular endoplasmic reticulum, and numerous free ribosomes. (c) Mitochondria in a glial cell; cristae are smooth, and the glia-glial gap junction is well visualized. (d) Cisterns of the endoplasmic reticulum in a dendrite are expanded, and the cristae in a mitochondrion are smooth. (e) Growth cone. (f) Mitochondrion in a neuronal axon. (g) Mitochondria in a neuronal soma, some are without cristae and have an enlightened matrix, a vacuole, and numerous free ribosomes. (h) Glial outgrowth, a few glycogen pellets and uneven cristae in a mitochondrion are visible. (i) Axo-spiny contact; mitochondria in a dendrite have swollen cristae. (j) Dendritic spine. (k) Axo-spiny synapse, short postsynaptic density (PSD), few synaptic vesicles near the active zone, and large osmiophilic bubbles in an axon. (l) Part of the cell body; a destroyed Golgi apparatus is visualized, mitochondria have normal structures, and an extremely low number of ribosomes are present on the endoplasmic reticulum. (m) Mitochondria in a glial cell, axonal outgrowth, and an enlightened axoplasm. (n) Mitochondria in a dendrite are densely packaged, have an irregular shape and exhibit cristae with small extensions. (o) Axo-dendritic synapse, destroyed mitochondria in a dendrite. (p) Mature axo-spiny synapse, a vacuole in an axon. (q) Increased Golgi apparatus area, a vacuole and destroyed mitochondria in the cytoplasm. (r) Mitochondria with impaired internal structures in a glial outgrowth, with a large number of osmiophilic bubbles. (s) Mitochondria in a large dendrite. (t) Mitochondria in a glial outgrowth have impaired internal structures. Scale bar – 0.5 µm.

increased mitochondrial respiratory chain complexes I and II activity and mitochondrial oxygen consumption rates. Blocking TrkB receptors leads to marked decreases in mitochondrial respiratory chain complex I activity during late stages of culture development.

## DISCUSSION

Brain activity is associated with complex signal transmission mechanisms and modulation in the neuron-glial network. This complex activation pattern and inhibitory signals summarized

FIGURE 8 | Representative electron microscopy images of dissociated hippocampal cells on DIV 14. (a–e) Sham, (f–j) BDNF, (k–o) ANA-12, (p–t) BDNF + ANA-12. (a) Axo-spiny synapse; the cristae of mitochondria in a dendrite are smooth. (b) Cytoplasm of a neuron; mitochondria, Golgi apparatus, granular endoplasmic reticulum, numerous ribosomes and part of the nucleus are well visualized. (c) Mitochondria in a glial cell; cristae are smooth, and intermitochondrial contacts are visible. (d) Axo-dendritic and axo-spiny asymmetric synapse. (e) Immature axo-spiny synapse; no postsynaptic density is observed. (f) Mitochondrion in the neuronal dendrite. (g) Mitochondria in a neuronal soma; some are without cristae and have an enlightened matrix and short cristae. (h) Glial outgrowth with few glycogen pellets and irregular cristae. (i) Axo-spiny synapse; osmiophilic synaptic bubbles in an axon. (j) Axo-dendritic and axo-spiny asymmetric synapses, osmiophilic mitochondrion in an axon; mitochondria in the axonal outgrowth are destroyed. (k) Axo-spiny asymmetric synapse, short postsynaptic density. (l) Part of a neuronal soma, destroyed Golgi apparatus, few ribosomes on the endoplasmic reticulum, and normal mitochondrial structure. (m) Mitochondria with disrupted cristae in a glial cell. (n) Axo-spiny asymmetric synapse, osmiophilic bubbles among synaptic vesicles, mitochondrion with an irregular shape and high osmiophility. (o) Part of a neuronal soma, Golgi apparatus, and mitochondria with impaired structures. (p) Mitochondrial cluster in a dendrite. (q) Golgi apparatus with an increased area, a vacuole and destroyed mitochondria in the cytoplasm. (r) Axo-dendritic synapse and vacuolated mitochondria in a glial outgrowth. (s) Glial cell with a smooth endoplasmic reticulum, synaptic vesicles of different sizes in the axon, and mitochondrion with a reduced area. (t) Mitochondrial clusters in the neuronal cytoplasm; many of these clusters have extended cristae; mitochondria without cristae were also visualized. Scale bar – 0.5 µm.

in the transmission of information through the neuron-glial network makes it possible to implement complex cognitive and behavioral responses. Understanding the fundamental laws of brain function is currently the main task in modern science and will contribute to solutions for a number of clinical and scientific issues.

In our study, we modeled chronic activation and inhibition of the TrkB receptor system, which is one of the most important systems for neurons. This approach can potentially be used to uncover the basic mechanisms underlying emergent information functions. The application of MEAs and the complexity of functional and ultrastructural studies allowed us to detect

changes in the neural network structure under the influence on the TrkB receptor system and to suggest the main intracellular targets for these implemented changes.

The TrkB receptor system is responsible for a number of functions associated with development and brain cell adaptation to unfavorable factors (Patapoutian and Reichardt, 2001; Skaper, 2018). In this case, identifying whether metabolic changes are possible at the neural network activity level is difficult. MEAs are considered the most adequate method for estimating the neural network structure both in vivo and in vitro (Martinoia et al., 2005; Obien et al., 2014). The application of this technology in combination with mathematical

methods of biological data analysis allowed us to identify the features of signal transmission throughout the network and reveal possible mechanisms underlying the systemic effect of chronic BDNF administration and a TrkB receptor system blockade.

Electrical synapses with a small number of immature chemical axo-dendritic and axo-spiny synapses are predominant in the early stages of primary hippocampal cell culture development (Shirokova et al., 2013). During excitation transmission through electrical synapses, signal modulation is limited and does not strongly depend on chemical stimuli (Hormuzdi et al., 2004). However, the influence on the TrkB receptor system can significantly impact chemical contact maturation and eventually affect neural network activity.

A sufficiently high level of spontaneous bioelectrical activity in primary hippocampal cultures in which chronic TrkB receptor blockade is implemented on DIV 14 is possibly associated with long-term adaptation effects related to changes in TrkB receptor expression under chronic influence (Proenca et al., 2016). ANA-12 is low-molecular-weight heterocyclic compound that affects the formation of a functional complex between BDNF and TrkB (Cazorla et al., 2011). The result of ANA-12 binding to the extracellular domain of TrkB is the prevention of BDNF-induced TrkB activation and negation of the biological effects of BDNF on TrkB-expressing cells. The N-terminal region of BDNF, which is critical for selectively binding and activating cognate Trk receptors, interacts with the "specificity patch" of the receptor, a binding pocket located in the fifth subdomain of TrkB (TrkB-d5) that may drive the selectivity of the interaction with BDNF. ANA-12 specifically binds to TrkB-d5 in a dosedependent manner. Based on previous binding experiments, even high concentrations of ANA-12 cannot overcome its displacement from TrkB by BDNF, suggesting a non-competitive mechanism (Banfield et al., 2001; Pattarawarapan and Burgess, 2003; Cazorla et al., 2011). Thus, the stereometric features of ANA-12 allow the rapid and reliable binding of this molecule to the active center of TrkB, which prevents the biological effect of BDNF.

However, the cross-correlation method and graphs revealed the nearly complete absence of hubs in the network structure under chronic blockade of TrkB receptors. This result is especially noteworthy in the context of high absolute values of spontaneous bioelectrical activity.

Analysis of functional calcium activity showed that changes in the percentages of working cells in the different groups were proportional to changes in the number of spikes in a large burst. The highest percentage of cells exhibiting Ca2<sup>+</sup> activity was found in cultures chronically administered BDNF. In this experimental group, the highest number of spikes in a large burst was detected on DIV 14. In addition, functional Ca2<sup>+</sup> imaging confirmed that the marked effects on the TrkB receptor system were manifested in the developmental period associated with chemical synapse predominance.

Ultrastructural changes demonstrated that blocking TrkB receptors affected the structures of synapses (empty axons in neurons, absence of PSD in a synapse) and the mitochondrial apparatus. For a long period, mitochondria were not considered capable of participating in synaptic signal modulation. Mitochondria are considered responsible for only energy metabolism in cells, and their effects are limited by the amount of adenosine triphosphate (ATP) produced. Whether BDNF influences mitochondria remains unclear. According to the present study, TrkB-mediated signaling can affect both the ultrastructural and functional parameters of brain mitochondria, even in normal oxygen and nutrient supply conditions. Chronic TrkB receptor blockade leads to destructive ultrastructural changes in mitochondria wherein the functional activity of organelles is not significantly decreased. Long-term application of BDNF increases the enzymatic activity of the mitochondrial apparatus, but this alteration is not associated with changes in the ultrastructure of organelles. Our findings likely indicate that TrkB-mediated mitochondrial regulation is related to functional modifications of the enzymatic systems in the respiratory chain but not to structural rearrangements of organelles. In this regard, the fundamental question for further research is whether the TrkB-mediated pathway for influencing mitochondria is generalized (that is, carried out through nuclear genes) or directed toward an isolated organelle.

Notably, chronic BDNF application increased the basal oxygen consumption rate via activating respiratory chain complex II. However, according to classical ideas, this pathway

#### REFERENCES


is typical utilized in hypoxic states (Kristián, 2004). Under oxygen stress conditions, BDNF increases the adaptive potential by impacting the functional parameters of the mitochondrial apparatus. In the current study, the percentage of osmiophilic PSDs in asymmetrical synapses in the "BDNF" group was significantly higher than that in the "sham" group, which potentially indicates more efficient signal transmission between neurons.

A genetically determined high level of BDNF can provide significant adaptive potential to the nervous system, and its neuroprotective effects are induced via the TrkB receptor system (Yu et al., 2013; Katsu-Jiménez et al., 2016). The current study showed that BDNF-mediated TrkB system activation is responsible for the formation of more complex functionally active neural networks with a high level of synaptic transmission efficiency. Thus, the TrkB signaling system can play a key role in inducing higher cognitive functions.

#### AUTHOR CONTRIBUTIONS

TM, EM, and MV conceived and designed the experiments, analyzed the data, and wrote the manuscript. AU, NV, TA, IK, and OS carried out the experiments. MV supervised the project, conceptualized the original idea, and ensured the financing of the project.

### FUNDING

This research was funded by a grant from Russian Science Foundation (RSF), project no. 18-75-10071.

#### ACKNOWLEDGMENTS

The authors thank Alexey S. Pimashkin for program technical support.

#### SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fphys. 2018.01925/full#supplementary-material

Rev. Biophys. 46, 271–293. doi: 10.1146/annurev-biophys-070816- 033647


in hippocampal BDNF, inflammatory cytokines and cell proliferation in rats during the postnatal brain development. Neurosci. Lett. 553, 1–6. doi: 10.1016/ j.neulet.2013.08.015



recovery in stroke rats. PLoS One 8:e81750. doi: 10.1371/journal.pone. 0081750


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2019 Mishchenko, Mitroshina, Usenko, Voronova, Astrakhanova, Shirokova, Kastalskiy and Vedunova. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# TREC and KREC Levels as a Predictors of Lymphocyte Subpopulations Measured by Flow Cytometry

#### Edited by:

Shangbin Chen, Huazhong University of Science and Technology, China

#### Reviewed by:

Igor Vladimirovich Kudryavtsev, Institute of Experimental Medicine (RAS), Russia Maria Giulia Bacalini, University of Bologna, Italy

#### \*Correspondence:

Ilya Korsunskiy iliakors@gmail.com Daniel Munblit daniel.munblit08@imperial.ac.uk

#### Specialty section:

This article was submitted to Computational Physiology and Medicine, a section of the journal Frontiers in Physiology

Received: 29 September 2018 Accepted: 11 December 2018 Published: 21 January 2019

#### Citation:

Korsunskiy I, Blyuss O, Gordukova M, Davydova N, Gordleeva S, Molchanov R, Asmanov A, Peshko D, Zinovieva N, Zimin S, Lazarev V, Salpagarova A, Filipenko M, Kozlov I, Prodeus A, Korsunskiy A, Hsu P and Munblit D (2019) TREC and KREC Levels as a Predictors of Lymphocyte Subpopulations Measured by Flow Cytometry. Front. Physiol. 9:1877. doi: 10.3389/fphys.2018.01877

Frontiers in Physiology | www.frontiersin.org

Ilya Korsunskiy1,2,3 \*, Oleg Blyuss<sup>4</sup> , Maria Gordukova<sup>1</sup> , Nataliia Davydova1,3 , Susanna Gordleeva<sup>5</sup> , Robert Molchanov<sup>6</sup> , Alan Asmanov<sup>7</sup> , Dmitrii Peshko<sup>2</sup> , Nataliia Zinovieva<sup>1</sup> , Sergey Zimin<sup>1</sup> , Vladimir Lazarev<sup>2</sup> , Aminat Salpagarova<sup>2</sup> , Maxim Filipenko<sup>8</sup> , Ivan Kozlov<sup>3</sup> , Andrey Prodeus1,3,9, Anatoliy Korsunskiy1,2 , Peter Hsu10,11 and Daniel Munblit2,11,12,13 \*

<sup>1</sup> Speransky Children's Hospital, Moscow, Russia, <sup>2</sup> Department of Paediatrics, Sechenov University, Moscow, Russia, <sup>3</sup> Dmitry Rogachev National Research Center of Pediatric Hematology, Oncology and Immunology, Moscow, Russia, <sup>4</sup> Wolfson Institute of Preventive Medicine, Queen Mary University of London, London, United Kingdom, <sup>5</sup> Lobachevsky State University of Nizhny Novgorod, Nizhny Novgorod, Russia, <sup>6</sup> State Institution "Dnipropetrovsk Medical Academy of the Ministry of Health of Ukraine", Dnipro, Ukraine, <sup>7</sup> The Research and Clinical Institute for Pediatrics named after Academician Yuri Veltischev of the Pirogov Russian National Research Medical University, Moscow, Russia, <sup>8</sup> Pharmacogenomic Laboratory, Institute of Chemical Biology and Fundamental Medicine SB RAS, Novosibirsk, Russia, <sup>9</sup> Immanuel Kant Baltic Federal University, Kaliningrad, Russia, <sup>10</sup> Allergy and Immunology, The Kids Research Institute, The Children's Hospital at Westmead, Sydney, NSW, Australia, <sup>11</sup> The In-VIVO Global Network, An Affiliate of the World Universities Network, New York, NY, United States, <sup>12</sup> Department of Paediatrics, Imperial College London, London, United Kingdom, <sup>13</sup> Solov'ev Research and Clinical Center for Neuropsychiatry, Moscow, Russia

Primary immunodeficiency diseases (PID) is a heterogeneous group of disorders caused by genetic defects of the immune system, which manifests clinically as recurrent infections, autoimmune diseases, or malignancies. Early detection of other PID remains a challenge, particularly in older children due to milder and less specific symptoms, a low level of clinician PID awareness and poor provision of hospital laboratories with appropriate devices. T-cell recombination excision circles (TREC) and kappadeleting element recombination circle (KREC) in a dried blood spot and in peripheral blood using real-time polymerase chain reaction (PCR) are used as a tool for severe combined immune deficiency but not in PID. They represent an attractive and cheap target for a more extensive use in clinical practice. This study aimed to assess TREC/KREC correspondence with lymphocyte subpopulations, measured by flow cytometry and evaluate correlations between TREC/KREC, lymphocyte subpopulations and immunoglobulins. We carried out analysis of data from children assessed by clinical immunologists at Speransky Children's Hospital, Moscow, Russia with suspected immunodeficiencies between May 2013 and August 2016. Peripheral blood samples were sent for TREC/KREC, flow cytometry (CD3, CD4, CD8, and CD19), IgA, IgM, and IgG analysis. A total of 839 samples were analyzed for using TREC assay and flow

cytometry and 931 KREC/flow cytometry. TREC demonstrated an AUC of 0.73 (95% CI 0.70–0.76) for CD3, 0.74 (95% CI 0.71–0.77) for CD4 and 0.67 (95% CI 0.63–0.70) for CD8, respectively, while KREC demonstrated an AUC of 0.72 (95% CI 0.69–0.76) for CD19. Moderate correlation was found between the levels of TREC and CD4 (r = 0.55, p < 0.01) and KREC with CD19 (r = 0.56, p < 0.01). In this study, promising prediction models were tested. We found that TREC and KREC are able to moderately detect abnormal levels of individual lymphocyte subpopulations. Future research should assess associations between TREC/KREC and other lymphocyte subpopulations and approach TREC/KREC use in PID diagnosis.

Keywords: TREC, KREC, primary immune deficiency, PID, flow cytometry, lymphocyte subpopulations, immunoglobulins

### INTRODUCTION

Primary immunodeficiency diseases (PID) is a heterogeneous group of disorders caused by genetic defects of the immune system, which manifests clinically as recurrent infections, autoimmune diseases or malignancies. Severe forms of PID – Severe Combined Immune Deficiency (SCID) – are associated with inherited lack of cellular and humoral immunity caused by mutations in various genes (Chan and Puck, 2005) and associated with a significant mortality rates in the first 2 years of life (Dvorak et al., 2013; Yao et al., 2013).

Severe combined immune deficiency can be detected by T-cell receptor excision circles (TRECs) measurement in a dried blood spot using real-time polymerase chain reaction (PCR) (Chan and Puck, 2005). TREC measurement became a part of neonatal screening in the United States and some other countries (Verbsky et al., 2012; Kwan et al., 2013). Despite great predictive value TREC can detect T-cells production defects, but not isolated B-cell defects. Some experts suggested that kappa-deleting element recombination circle (KREC) may add value in PID diagnosis (Nakagawa et al., 2011) and multiplex techniques for simultaneous quantitation of TREC/KREC were piloted (Borte et al., 2012).

Outside of neonatal screening, TREC/KREC measurement is not commonly used in routine clinical practice, with flow cytometry being a traditional, but more expensive diagnostic technique for PID detection, when compared with the PCR (Puck and SCID Newborn Screening Working Group, 2007). It requires a significant amount of training and not readily available in many developing countries. TREC and KREC assessment both in PID diagnosis and in therapy monitoring represent great potential (Serana et al., 2013).

TREC and KREC predictive ability in SCID has been extensively studied, but not much research was done in relation to physiological aspects of relationships between TREC/KREC and lymphocyte subpopulations. In this pilot study we assessed correlations between TREC/KREC levels, lymphocyte subpopulations and immunoglobulins and evaluated TREC/KREC ability to predict reduced levels of lymphocyte subpopulations.

### MATERIALS AND METHODS

### Study Setting, Eligibility Criteria, and Ethics

We carried out a retrospective analysis of data from all children assessed by clinical immunologists at Speransky Children's Hospital, Moscow, Russia with suspected immunodeficiencies between May 2013 and August 2016. The diagnosis of different types of PID was based on IUIS Phenotypic Classification for Primary Immunodeficiencies (Bousfiha et al., 2015). The investigations and sample collection have been conducted following ethical approval by the Speransky Children's Hospital Ethics Committee. Parental written consent was obtained for all participants as a part of routine procedure at Speransky Children's Hospital. Parents/guardians were informed of the procedures in lay terms.

### Sample Analysis

Peripheral blood samples were taken by venipuncture during morning hours, aliquoted and sent for CBC, flow cytometry, immunoglobulins (IgA, IgM, and IgG) and TREC/KREC analysis. All blood samples were EDTA-anticoagulated and analyzed on the same day of the collection in order to avoid cellular death.

#### Immunophenotyping

Three-Four color flow cytometric immunophenotyping with directly labeled monoclonal antibodies was used to determine the following immune cell subsets: CD3, CD4, CD8, CD19 following manufacturer's protocol. In brief, 50 µl aliquots of blood were incubated for 15 min in the dark at room temperature with combinations of optimally titrated fluorochrome-conjugated monoclonal antibodies. After surface staining, erythrocytes were lysed using 1 ml of BD FACSLysing Solution, diluted according to manufacturer's instructions. Remaining cells were washed twice and suspected in CELL WASH buffer for further analysis for a FACS Canto II flow cytometer using FACSDiva v7.0 software (Becton Dickinson). Cell suspension for staining of naïve and memory B-lymphocytes were prepared as described by Ferry et al. (2005). Briefly, 300 µl blood aliquots were washed three times using CELL WASH buffer (Becton Dickinson) supplemented

with 2% bovine serum albumin to eliminate any cell-bound antibodies non-specifically inhibiting the staining effect.

Acquisition was run until 10000–50000 events were detected. First the viable part of the sample was selected by physical gating based on forward scatter (FS) and side scatter (SS); the lymphocyte population was identified by the low forward and side scatter and checked for purity by the positivity for CD45. Then the different lymphocyte subpopulations were identified by immunophenotype markers. At least 5000 events from B-lymphocyte gate set based on CD19 expression and side scatter characteristics were acquired.

The total leucocytes count and differential was measured with Advia 2120i hematology analyzer (Siemens). The absolute size of each lymphocyte subpopulation was calculated by multiplying the relative size of the lymphocyte subpopulation and the absolute lymphocyte count.

#### Immunoglobulins Assay

Immunoglobulin levels were assessed by immunoturbidimetry method using biochemical analyzer Architect C8000 (Abbott, United States, Abbott kits) in accordance with manufacturers' protocol.

#### TREC and KREC Assay

TREC and KREC assay was performed using real-time PCR with fluorescent hybridization probes with reagents for TREC/KREC assay T&B PCR kit (ABV-test, Russia) (Deripapa et al., 2017) in whole blood and dry blood stain DNAs.

The TREC/KREC levels were assayed in whole blood samples as described previously (Sottini et al., 2010; Deripapa et al., 2017). In brief, DNA was extracted from 100 µl EDTA anticoagulated whole blood by using RIBO-prep nucleic acid extraction kit (Amplisense <sup>R</sup> , Russia). The Real-time qPCR was performed by using CFX 96 Real-Time PCR System (Bio Rad, United States).

#### Statistical Analysis

Shapiro-Wilk test has been used to assess the normality of the distribution of variables analyzed in this paper. Since the null hypothesis about the normality was rejected, Spearmen correlation coefficient was used to assess the strength of the correlation between the variables. Sensitivity, specificity and their 95% confidence intervals were computed with stratified bootstrap replicates (Carpenter and Bithell, 2000). Area under Receiver Operating Characteristic (ROC) -curve (AUC) calculation was followed by 95% confidence interval as suggested by DeLong (DeLong et al., 1988). To account for non-linear trend, level of TREC, KREC, and lymphocyte subpopulations were substituted by the ratio of their concentrations to corresponding reference values for a given patient age.

Results were considered statistically significant if p-value was smaller than 0.05. All calculations were done using R package version 3.4.1.

#### TREC/KREC and Lymphocyte Subpopulations

Primary analysis consisted of two stages.

At stage 1 we assessed 4 paired relationships between the levels of TREC with CD3, CD4, CD8, and KREC with CD19. These


Provided codes are in accordance with International Classification of Diseases, 10th revision (ICD-10).

were presented as a proportion of patients with normal levels of one of the paired variable among patients with normal levels of another immunological marker.

We also assessed correlations between TREC/KREC and lymphocyte subpopulations.

At stage 2 we assessed ability of TREC and KREC to predict abnormality in lymphocyte subpopulation levels. Using ROCanalysis the predictivity of TREC, KREC and their combination was tested providing: (a) the sensitivity (proportion detected of those with lower lymphocyte subpopulation levels) at a fixed specificity (proportion of controls correctly detected not to have reduced lymphocyte subpopulation levels) and (b) AUC.

#### RESULTS

#### Study Population

The data was extracted from the clinical notes and laboratory database of Speransky Children's Hospital. Out of all 3055 patients requiring flow cytometry within the given period of time, due to financial restrictions, a total of 839 samples were analyzed using flow cytometry and TREC assay and 931 samples were analyzed using flow cytometry and KREC assay and were included into the statistical analysis. Data on TREC/KREC levels of 2050 children were unavailable and were not evaluated further.



Optimum cut-off point based on maximum value of the J index is presented in bold.

TABLE 3 | Model performance for different cutoff points of the predicted probabilities for TREC with regards to CD4.


Optimum cut-off point based on maximum value of the J index is presented in bold.

TABLE 4 | Model performance for different cutoff points of the predicted probabilities for TREC with regards to CD8.


Optimum cut-off point based on maximum value of the J index is presented in bold.

TABLE 5 | Model performance for different cutoff points of the predicted probabilities for KREC with regards to CD19.


Optimum cut-off point based on maximum value of the J index is presented in bold.

Data on clinical diagnosis was available in 471 participant and presented in **Table 1**.

### Comparison of Flow Cytometry Parameters With TREC and KREC

At first stage ability of TREC/KREC test to predict CD19, CD3, CD4 and CD8 flow cytometry results was assessed. We found that 667 out of 863 (77.3%) patients with normal KREC levels [as reported earlier (Gordukova et al., 2015)] had normal CD19, while 667 out of 682 (97.8%) individuals with normal CD19 had normal KREC.

In patients with normal TREC levels, 462 out of 548 (84.3%) had normal CD3, 440 out of 548 (80.3%) normal CD4, and

FIGURE 2 | Receiver operating characteristic (ROC) curves for TREC for the ability to predict abnormal values of lymphocyte subpopulations (CD3, CD4, and CD8), (n = 839).

473 out of 548 (86.3%) normal CD8 counts. Individuals having normal levels of CD3, CD4, and CD8 had normal levels of TREC in 462/548 (84.3%), 440/548 (80.3%), and 473/548 (86.3%), respectively.

Values of TREC, KREC and lymphocyte subpopulations which were considered abnormally low for the purpose of this study are reported in **Supplementary Table S1**.

TABLE 6 | Model performance for different cutoff points of the predicted probabilities for TREC with regards to CD3,4, and 8.


Optimum cut-off point based on maximum value of the J index is presented in bold.

TABLE 7 | Model performance for different cutoff points of the predicted probabilities for combined TREC and KREC with regards o all lymphocyte subpopulations measured.


Optimum cut-off point based on maximum value of the J index is presented in bold.

### TREC/KREC Ability to Predict Abnormal Levels in Lymphocyte Subpopulations

We assessed TREC/KREC ability to predict each lymphocyte subpopulation individually, using area under the curve (AUC), which are shown in **Figure 1**. TREC demonstrated an AUC of 0.73 (95% CI 0.70–0.76) for CD3, 0.74 (95% CI 0.71–0.77) for CD4 and 0.67 (95% CI 0.63–0.70) for CD8, respectively, while KREC demonstrated an AUC of 0.72 (95% CI 0.69–0.76) for CD19.

The following cutoff points of a probability showed the best prognostic accuracy with regards to sensitivity and specificity for TREC: 0.6 (67% for both, sensitivity, and specificity; **Table 2**), Youden's index (J) = 32.9 in CD3; 0.55 (77 and 58%; **Table 3**), J = 35.1 in CD4; 0.5 (80 and 44%; **Table 4**), J = 24.6 in CD8, respectively. A cutoff point of a probability of 0.55 showed the best diagnostic accuracy with regards to sensitivity and specificity for KREC (67% and 64%; **Table 5**), J = 30.4 in predicting abnormal levels of CD19.

We also assessed AUC for TREC ability to predict the reduced levels of CD3, CD4 and CD8 (**Figure 2**), and a combination of TREC and KREC (**Figure 3**) to predict the reduced levels of all lymphocyte subpopulations analyzed. TREC demonstrated an AUC of 0.66 (95% CI 0.63–0.70) while a combination of TREC and KREC resulted in an AUC of 0.65 (95% CI 0.62–0.69). A cutoff point of a probability of 0.4 showed the best diagnostic accuracy with regards to sensitivity and specificity for TREC (59% and 65%; **Table 6**), J = 23 and 0.35 for a combination of TREC and KREC (61 and 59%, respectively; **Table 7**), J = 20.2.

### Correlation Between TREC/KREC and Lymphocyte Subpopulations

We observed a moderate correlation (**Table 8**) between the levels of TREC and CD4 (r = 0.55, p < 0.01) and KREC with CD19 (r = 0.56, p < 0.01). Moderate to low correlation was found between TREC with CD19, CD3 and lymphocytes (r between 0.28 and 0.46, p < 0.01) and KREC with CD4 and lymphocytes (r = 0.33, p < 0.01). Neither TRECs nor KRECs levels correlated with the concentration of immunoglobulins (IgA, IgG).

### DISCUSSION

In this study, we assessed associations between TREC/KREC and lymphocyte subpopulations. TREC and KREC models showed good ability to predict abnormal levels of certain lymphocyte subpopulations and modest correlations between TREC and CD4, KREC and CD19 were found.

PID is a large group of disorders encompassing a few hundred various conditions affecting development and/or functioning of the immune system (Picard et al., 2015). Flow cytometry is a sensitive and important tool in immune system functioning evaluation and PID diagnosis (Kanegane et al., 2018), however, it is expensive, not easily available and complexity of the method requires appropriate training. TREC and KREC may represent a cheaper alternative and/or add value to PID diagnosis and screening. Low cost methodology can be used in small laboratories and rural settings, where complex and expensive tools are unavailable, to provide access to primary PID evaluation. TREC/KREC evaluation may also serve as a prerequisite to flow cytometry.

We found significant correlations between the levels of TREC and lymphocyte subpopulations with the strongest correlations were observed for TREC/CD3, TREC/CD4. This finding is consistent with previous reports (Mensen et al., 2013; Gul et al., 2015), suggesting that low TREC levels correlate with low values of CD3+ and CD4+. The observed positive correlation could be attributed to TRECs being a direct marker for thymic output (Ravkov et al., 2017). We also observed a statistically significant moderate KREC levels correlation with CD19 levels and we are not aware of other studies reporting this finding, however

TABLE 8 | Heatmap of correlation between TRECs/KRECs level with immunoglobulins, CDs and blood cells.


More intense green color means stronger positive correlations, while more intense red color means stronger negative correlation. <sup>∗</sup>p < 0.05; ∗∗p < 0.01; NS, statistically not significant result.

correlation is plausible as both KREC and CD19 are linked with B-lymphocytes.

When proportion of patients with both normal TREC/KREC and lymphocyte subpopulations was assessed, we found that almost every individual with CD19 within the reference range had normal KREC levels. Most of individuals (80–85%) with CD3, CD4 and CD8 within the reference range had normal TREC levels. Given moderate correlations between TREC/KREC and lymphocyte subpopulations and promising proportion results, we expected a decent ability of TREC and KREC predictive models with regards to lymphocyte subpopulations abnormal levels. Positive predictive values for TREC ability to predict abnormal levels of CD3 and CD4, and KREC abnormal levels of CD19 varied between 75 and 79%, when optimum cut-off point was selected based on maximum value of the J index. TREC ability to predict abnormal level of CD8, in contrast, was much lower and reached a PPV of 65% only. This result was not surprising as negligible correlation between TREC and CD8 levels was detected.

Neither TREC, nor a combination of TREC and KREC reached impressive AUC values when predictivity of cumulative lymphocyte subpopulations was modeled. A cut-off points of a probability of 0.4 for TREC and 0.35 for a combination of TREC and KREC showed the best diagnostic accuracy according to Youden's index but positive predictive value of the models was very low. We may speculate that lack of individual and multiple correlations between TREC and CD8; KREC and CD3, CD8, may explain lack of consistency in the model performance, when predictivity in relation to cumulative lymphocyte subpopulations was tested.

Models showed promising ability of TREC to predict abnormal levels of CD3 and CD4, and KREC abnormal levels of CD19. Although combined use of TREC and KREC did not result in good predictivity when cumulative lymphocyte

### REFERENCES


subpopulations were assessed, further research may improve predictive ability, adding other subpopulations, such as naïve B-lymphocytes CD19+CD27-IgD+, recent thymic emigrants (RTE) and CD31+CD45RA+ T-lymphocytes. PID is a very heterogenous group of diseases, and TREC/KREC predictive abilities should be further tested in individuals with separate PID conditions. Future research should also target investigation of TREC/KREC diagnostic abilities in PID patients, which is an existing unmet need.

### AUTHOR CONTRIBUTIONS

IK, MF, IK, AP, AK, and DM conceived and designed the experiments and study analysis. MG and ND performed the experiments. IK, NZ, SZ, AS, and DP collected, extracted, and sorted the data. OB and RM analyzed the data. VL and SG reviewed available evidence on the matter. AA, DP, IK, PH, and DM wrote the manuscript.

### FUNDING

SG acknowledges the support from the Russian Science Foundation grant no. 16-12-00077. Some data from this manuscript has been presented at European Academy of Allergy and Clinical Immunology meeting (Korsunskiy et al., 2018).

### SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fphys. 2018.01877/full#supplementary-material



**Conflict of Interest Statement:** MG, MF, and IK are board members for MG, MF, AP, and IK has a patent with ABV-test.

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2019 Korsunskiy, Blyuss, Gordukova, Davydova, Gordleeva, Molchanov, Asmanov, Peshko, Zinovieva, Zimin, Lazarev, Salpagarova, Filipenko, Kozlov, Prodeus, Korsunskiy, Hsu and Munblit. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Identification of Functional Gene Modules Associated With STAT-Mediated Antiviral Responses to White Spot Syndrome Virus in Shrimp

#### Guanghui Zhu1,2, Shihao Li3,4,5, Jun Wu<sup>2</sup> , Fuhua Li3,4,5 \* and Xing-Ming Zhao<sup>1</sup> \*

1 Institute of Science and Technology for Brain-Inspired Intelligence, Fudan University, Shanghai, China, <sup>2</sup> Department of Computer Science and Technology, Tongji University, Shanghai, China, <sup>3</sup> Laboratory for Marine Biology and Biotechnology, Qingdao National Laboratory for Marine Science and Technology, Qingdao, China, <sup>4</sup> Institute of Oceanology, Chinese Academy of Sciences, Qingdao, China, <sup>5</sup> Center for Ocean Mega-Science, Chinese Academy of Sciences, Qingdao, China

#### Edited by:

Shangbin Chen, Huazhong University of Science and Technology, China

#### Reviewed by:

Yin-Ying Wang, The University of Texas Health Science Center at Houston, United States Haiying Wang, Ulster University, United Kingdom

#### \*Correspondence:

Fuhua Li fhli@qdio.ac.cn Xing-Ming Zhao xmzhao@fudan.edu.cn

#### Specialty section:

This article was submitted to Systems Biology, a section of the journal Frontiers in Physiology

Received: 07 November 2018 Accepted: 19 February 2019 Published: 11 March 2019

#### Citation:

Zhu G, Li S, Wu J, Li F and Zhao X-M (2019) Identification of Functional Gene Modules Associated With STAT-Mediated Antiviral Responses to White Spot Syndrome Virus in Shrimp. Front. Physiol. 10:212. doi: 10.3389/fphys.2019.00212 White spot syndrome virus (WSSV) is one of the major threats to shrimp aquaculture. It has been found that the signal transducer and activator of transcription (STAT) protein plays an important role in the antiviral immunity of shrimp with a WSSV infection. However, the mechanism that underlies the STAT-mediated antiviral responses in shrimp, against WSSV infection, remains unclear. In this work, based on the gene expression profiles of shrimp with an injection of WSSV and STAT double strand RNA (dsRNA), we constructed a gene co-expression network for shrimp and identified the gene modules that are possibly responsible for STAT-mediated antiviral responses. These gene modules are found enriched in the regulation of the viral process, JAK-STAT cascade and the regulation of immune effector process pathways. The gene modules identified here provide insights into the molecular mechanism that underlies the STAT-mediated antiviral response of shrimp, against WSSV.

Keywords: antivirus response, co-expression network, gene module, STAT dsRNA, white spot syndrome virus, shrimp

### INTRODUCTION

White spot syndrome virus (WSSV) is a highly lethal and contagious virus in penaeid shrimp, with huge economic consequences in commercial fishery and farming of the Pacific white shrimp, tiger prawn, Atlantic white shrimp, and so on. Once an outbreak of WSSV occurs, it wipes out entire populations in many shrimp farms within a few days and leads to enormous economic losses (Flegel et al., 2008). Due to the serious impact of WSSV on shrimp aquaculture, it is urgent to understand the molecular mechanisms that underlie WSSV pathogenesis in shrimp.

It has been reported that there are some genes that have revealed WSSV pathogenesis based on the transcriptome of shrimp with WSSV infection. For example, García et al. (2009) employed PCR technology to compare the transcriptomes in hemocytes of WSSV-infected shrimp with uninfected ones. They found that penaeidin-3 isoforms and crustin were over-expressed in hemocytes of WSSV-infected pre-challenged Penaeus vannamei (García et al., 2009). Shekhar et al. (2015) utilized DNA microarray technology to explore the genes of host immune responses to WSSV pathogenesis.

**166**

They found some up- or down-regulated genes during WSSV infection in shrimp (Shekhar et al., 2015). Yu et al. (2017) found that a combination of single nucleotide polymorphisms in three genes (TRAF6, Cu/Zn SOD, and nLvALF2) were significantly associated with resistance to WSSV infection. The SNP loci in TRAF6, Cu/Zn SOD, and nLvALF2 were found to exhibit a significant effect on the resistance of shrimp to WSSV, while the expression of the three immune-related genes were affected by those SNPs (Yu et al., 2017). Li et al. (2013) used RNA-Seq technology to investigate the transcriptome of the shrimp between latent infection stage and acute infection stage. The genes that played an important role in host defense against WSSV and the genes that were possibly responsible for the rapid proliferation of WSSV, were identified (Li et al., 2013). Recently, the Toll, IMD, and JAK-STAT pathways were reported as the main pathways in the antiviral immunity against WSSV (Li and Xiang, 2013). It was found that the regulation of Toll and IMD pathways improved the anti-WSSV response in shrimp, while the silencing of the signal transducer and activator of transcription (STAT) gene, an important part of the JAK-STAT pathway, was proved effective both in reducing the WSSV-DNA copy number and the mortality of shrimp (Liu et al., 2007; Chen et al., 2008; Wen et al., 2014).

The above studies have provided a preliminary description of host responses against WSSV infection at the transcriptional level in shrimp, where the differentially expressed genes (DEGs) between WSSV-infected shrimp with and without the treatment of STAT. However, few of those DEGs are related to the antiviral response and the antiviral response genes may not be differentially expressed. This makes it difficult to understand the antiviral process mediated by the STAT gene. Biological networks can provide valuable information to better understand the mechanism of antiviral responses in a comprehensive and systematic way (Behura et al., 2011; Doering et al., 2012; Gupta et al., 2014; Li et al., 2014; Dai et al., 2017). In this work, based on the gene expression profiles of shrimp with an injection of WSSV and STAT double strand RNA (dsRNA), the gene coexpression network was constructed, where the network provided the functional relationships between genes. The gene modules, representing components consisting of densely connected genes in a co-expression network, were found to be suitable units to describe the metabolic disorders associated with WSSV infections. In the modules enriched with genes associated with STAT-mediated antiviral response against WSSV, infections were found related to biological processes such as the regulation of the viral process, modulation by host of symbiont transcription, JAK-STAT cascade, and the regulation of the immune effector process pathways. Furthermore, the network topology of these modules associated with the STAT-mediated antiviral response, provided clues to identify important genes and pathways in the antiviral response.

## RESULTS AND DISCUSSION

The schematic for the analysis pipeline is shown in **Figure 1**. The gene co-expression network was constructed based on gene expression profiles across 14 shrimp samples injected with STAT dsRNA and WSSV (**Table 1**). The densely connected components, i.e., gene modules, of the network were further detected as functional units in the antiviral process. These modules were found enriched with metabolic functions, previously reported as dysfunctional in WSSV-infected shrimps. Furthermore, the information of network topology of those modules affected by STAT during WSSV infection, was utilized to identify important genes and pathways in the antiviral response affected by STAT against WSSV infection.

### Identification of Gene Modules Affected by WSSV Infection and STAT dsRNA

To identify the gene modules affected by STAT during WSSV response, the gene co-expression network was constructed based on the gene expression across 14 samples injected with STAT dsRNA and WSSV. Only genes with top a 30% variance of the gene expression were used to construct the network. 0.01 was chosen as the p-value threshold for the Pearson correlation coefficient (PCC) between two genes, to further screen out the edges of the network. The co-expression network consisted of 15,167 genes and 2,288,537 edges, representing significant correlations between these genes. In total, 1873 modules with default parameters were detected by the ClusterOne algorithm (Nepusz et al., 2012). To investigate the role of these modules in the antiviral process, the functional enrichment analysis was performed to identify the biological processes that were significantly enriched in every module. In particular, the twotailed Fisher's exact test with a p-value threshold of 0.05 was used to identify processes that were significantly enriched in each module. Referring to the report from a previous study on the metabolic changes in WSSV-infected shrimp, the metabolic system of WSSV-infected shrimp was mainly changed in glucose consumption, plasma lactate concentration, activity of glucose-6-phosphate dehydrogenase, ADP/ATP ratio, oxidative stress, triglyceride concentration, the cell death process, mitochondrial membrane permeabilization, energy production, and upregulation of the voltage-dependent anion channel (Chen et al., 2011). The identified modules were significantly enriched within biological processes relevant to almost all of these reported abnormalities. The number of modules enriched with the processes relevant to these reported disorders is shown in **Figure 2**. Additionally, there were 12 modules enriched with the biological processes of regulation of the JAK-STAT cascade and 13 modules enriched with modulation by the host of symbiont transcription. Considering the above, these modules can be used as a signature to characterize the influence of WSSV infection and STAT dsRNA interference on the metabolism system.

## Identification of Genes Affected by STAT During WSSV Infection

To find network modules affected by STAT during WSSV infection, the DEGs were identified based on gene expression profiles across the 14 samples. Three sets of DEGs were obtained by comparing three groups of samples. The WSSV-infected samples (EW\_48 and EW72) were compared with controls (EP48

and EP72) to identify DEGs affected by WSSV infection. The samples injected with STAT dsRNA (SP48 and SP72) were compared with the controls (EP48 and EP72) to identify DEGs affected by STAT. The samples injected with both WSSV and STAT dsRNA (SW48 and SW72) were compared with the controls (EP48 and EP72) to identify DEGs affected by both WSSV and STAT dsRNA. A functional enrichment analysis was performed on each set of DEGs, to evaluate whether these gene

TABLE 1 | The 14 samples used to generate gene expression profiles.


sets were associated with the regulation of STAT and infection of WSSV. As shown in **Table 2**, the 3144 genes affected by WSSV infection were significantly enriched with biological processes such as the positive regulation of JUN kinase activity, natural killer cell activation, B cell activation, T cell activation, and the regulation of an adaptive immune response. Other than the four processes associated with activity of immunocytes, JUN kinase activity has been reported to promote viral replication and plays a role in WSSV infection in shrimp (Shi et al., 2012; Wang et al., 2017). The 2685 genes affected by STAT dsRNA were significantly enriched with processes such as hemocyte differentiation, embryonic hindlimb morphogenesis, positive regulation of an inflammatory response, and the viral process. These processes have been reported to be associated with STAT, indicating that the STAT dsRNA interference indeed affected the cascades of STAT signaling in the STAT dsRNA-injected samples (Luo and Dearolf, 2001; Grebien et al., 2008; Kiu and Nicholson, 2012). The 2785 genes affected by both WSSV and STAT dsRNA were significantly enriched with processes such as the negative regulation of JAK-STAT cascade, cell death, the viral process, and the oxidation-reduction process. There were accidental factors in the identified genes affected by both WSSV and STAT dsRNA, such as technical noise and genes that were affected only by either WSSV or STAT dsRNA. To further exclude accidental factors, the genes specially affected by STAT during WSSV infection were obtained as the remaining part of the genes affected by both WSSV and STAT dsRNA, after removing the genes in the gene set

associated with WSSV infection and the gene set affected by STAT interference (**Supplementary File S1**). As shown in the infection, 1723 genes particularly affected by STAT during WSSV infection were significantly enriched with the negative regulation of the JAK-STAT cascade, lymphocyte differentiation, and positive regulation of immune effector process. It suggested that the part of the genes annotated with lymphocyte differentiation and immune effector process were particularly regulated by the mutual effect of STAT dsRNA interference and WSSV infection, rather than individually regulated by the WSSV or STAT dsRNA alone. The genes particularly affected by STAT during WSSV infection were also significantly enriched with the regulation of an adaptive immune response. While shrimp are generally assumed to have no adaptive immune response, recent studies have shown that shrimp can obtain immune responses to specific pathogens including bacterium and viruses (Yang et al., 2012; Lin et al., 2013).

### Identification of Modules Specially Affected by STAT During WSSV Infection

The 381 modules significantly enriched with genes particularly affected by STAT during WSSV infection, were identified as the candidate modules particularly affected by STAT during WSSV infection (CMASWs) (**Supplementary File S2**). These CMASWs were enriched with all the biological processes relevant to the metabolic changes in WSSV-infected shrimp (**Figure 2**). To obtain a comprehensive view of the metabolic disorders affected by WSSV and STAT dsRNA, the number of CMASWs significantly enriched with every relevant biological process was compared with that of candidate modules affected by WSSV and the modules affected by both WSSV and STAT. The 531 modules enriched with genes affected by WSSV and 574 modules enriched with genes affected by both WSSV and STAT were identified as candidate modules affected by WSSV and affected by both WSSV and STAT, respectively (**Supplementary Files S3, S4**). The number of CMASWs enriched with every process relevant to glucose consumption was more than that of candidate modules affected by WSSV, suggesting that glucose consumption was the potential aspect of the metabolic system affected by STAT dsRNA interference in WSSV-infected shrimps. The number of CMASWs significantly enriched with every process related with ATP synthesis, triglyceride concentration, and the cell death process was less than that of candidate modules affected by WSSV. In particular, the number of candidate modules affected by both WSSV and STAT, enriched with every process related with ATP synthesis and cell death, was also less than that of candidate modules affected by WSSV, indicating that these two aspects of the metabolic disorder were alleviated by STAT dsRNA interference. Additionally, there were nine CMASWs significantly enriched with the regulation of the JAK-STAT cascade biological process, 13 CMASWs enriched with the regulation of the viral process, and 24 CMASWs enriched with the regulation of the immune response, suggesting that these CMASWs were indeed associated with the antiviral response affected by STAT against WSSV. Considering that the densely

TABLE 2 | The biological processes significantly enriched in the genes affected by WSSV infection, genes affected by STAT, genes affected by both WSSV and STAT dsRNA, and genes specially affected by STAT during WSSV infection.


The p-values obtained through two-tail Fisher's exact test were represented in the brackets.

connected genes in a CMASW tended to achieve functions together, the topology of a CMASW provided clues to identify important genes in the STAT-affected antiviral response against WSSV. As shown in the **Figure 3A**, the gene Unigene11346 in the CMASW module\_233 was not differentially expressed between shrimp injected with both STAT dsRNA and WSSV (samples SW48 and SW72) and the control (samples EP48 and EP72), but the gene Unigene11346 was associated with three genes, CL88.Contig9, Unigene39525, and CL4771.Contig2, that were particularly affected by STAT during WSSV infection in the module. Despite this, Unigene11346 was not differentially expressed during the STAT-mediated antiviral response. Its relevant functions, such as the regulation of the immune effector process, regulation of the defense response to a virus by the host, and the positive regulation of the immune system process, were potentially regulated by the three associated DEGs. The CL2191.Contig3 (STAT) was a hub gene connected with 13 (44.8%) out of 29 genes in the module, suggesting that STAT had significant influence on this module. While STAT was annotated with the immune effector process, positive regulation of the immune system process, regulation of the immunoglobulin mediated immune response, natural killer cell activation, and the response to interleukins, other genes in the modules were potentially involved in these processes. The other gene CL4749.Contig3 was associated with 23 (79.3%) genes including all three genes particularly affected by STAT during WSSV infection and the other two important genes, Unigene11346 and CL2191.Contig3, suggesting the important role of CL4749.Contig3 in the STAT-mediated antiviral response. As shown in **Figure 3B**, the module\_1253 was associated with a modification by the host of symbiont morphology or physiology, modulation by host of viral transcription, regulation of viral transcription, and interspecies interaction between organisms. Both the genes Unigene3981 and CL3235.Contig2 were annotated with these biological processes. While these two genes were not identified as DEGs, the three associated genes, CL1462.Contig1, CL3940.Contig2, and Unigene31272, that were identified as genes particularly affected by STAT during WSSV infection, tended to participate in these processes. Among these three genes particularly affected by STAT during WSSV infection, Unigene31272 was a hub gene connecting 17 (70.8%) genes out of 24 in the module, indicating they played an important role in the antiviral process. Unigene3981 was also annotated with the regulation of the glucose metabolic process, glucose homeostasis, and the regulation of the glucose metabolic process, which are relevant to the disordered aspects in the metabolic system in WSSV-infected shrimp.

#### CONCLUSION

Aiming to better understand the antiviral responses affected by STAT against WSSV in shrimp, the gene modules relevant to the antiviral responses mediated by STAT were identified from the gene co-expression network. These modules were found to be associated with the biological processes that underlie the metabolic changes during WSSV infection. The important genes and antiviral responses were further identified based on the network topology of these modules. For example, Unigene11346, CL4749.Contig3, Unigene3981, and CL3235.Contig2 were identified as important genes in the antiviral response. Functional enrichment analysis suggests that these genes are enriched with biological processes that potentially underlie antiviral responses, e.g., the immune effector process, regulation of the defense response to a virus by the host, regulation of the immune system process, modification by the host of symbiont morphology or physiology, modulation by the host of viral transcription, regulation of the viral transcription, and the interspecies interaction between organisms. These findings provide insights into the molecular mechanisms that underlie shrimp antiviral responses.

#### MATERIALS AND METHODS

### Shrimp Maintenance and Virus Preparation

Healthy shrimp Litopenaeus vannamei, with a body weight of 4.4 ± 1.0 g were acquired from an aquaculture farm and reared in the lab. They were maintained in natural, aerated seawater at 25 ± 1 ◦C and fed with commercial feed twice a day. WSSV virions were purified from the

tissues of infected shrimp, following the method reported by Sun et al. (2013) and the stock solution was 3.5 × 10<sup>4</sup> copies/µL in PBS.

edges connecting important genes to STAT-WSSV-jointly affected genes were marked as red thick lines.

#### Preparation of Samples

Synthesis and optimization of the dose of the dsRNA of the target gene LvSTAT (dsLvSTAT) for gene silencing, fully followed the methods reported by Wen et al. (2014). Ten micrograms dsLvSTAT was injected into each shrimp for gene silencing. At the same time, a fragment of enhanced green fluorescent protein (EGFP) gene from the pEGFP-N1 plasmid was used for synthesis of control dsRNA (dsEGFP) following the same method (Wen et al., 2014).

Four groups including SP, EP, SW, and EW were set in the following experiments. Each group contained 20 shrimps. In SP and EP, each shrimp was injected with 10 µg dsLvSTAT and 10 µg dsEGFP, respectively, and then each shrimp was injected with 10 µL PBS 48 h later. In SW and EW, each shrimp was injected with 10 µg dsLvSTAT and 10 µg dsEGFP, respectively, and then each shrimp was injected with 10 µL WSSV solution containing 8.0 × 10<sup>3</sup> copies, 48 h later. The time of PBS or WSSV injection was recorded as 0 h. Before PBS or WSSV injection, 48-h post dsLvSTAT or dsEGFP injection, cephalothorax samples of three shrimp from each group were collected separately for RNA extraction. They were named as S0 and E0. At 24, 48, and 72-h post WSSV or PBS injection, cephalothorax samples of three shrimp from each group were collected separately for RNA extraction. These samples were named EP24, SP24, EW24, SW24, EP48, SP48, EW48, SW48, EP72, SP72, EW72, and SW72, respectively.

### Illumina Sequencing

fphys-10-00212 March 7, 2019 Time: 16:58 # 7

Paired-end RNA sequencing was performed to generate the transcriptome for each sample. In brief, the total RNA of each sample was extracted with a RNAisol reagent (Takara, Japan) and treated with DNase I. The RNA amounts were estimated spectrophotometrically by a NanoDrop 2000 spectrophotometer (Thermo Fisher Scientific, United States). Polyadenylated (polyA+) RNA was purified from the total RNA using Sera-mag oligo(dT) beads, fragmented to a length of 100–500 bases, reverse transcribed using random hexamers, and end repaired and adaptor-ligated according to the manufacturer's protocol (Illumina). Ligated products of 300–500 bp were excised from agarose and PCR-amplified (15 cycles). Products were cleaned using a MinElute column (Qiagen) and single-end sequenced on a Genome Analyzer II (Illumina), according to manufacturer's instructions. The raw sequencing data has been deposited in the Sequence Read Archive (SRA) database (SRA accession: SRP159438).

#### Preprocessing of RNA-Seq Data

The reference assembly from a previous work was used as the reference transcriptome (Wei et al., 2014). More detailed information about the sequencing data can be found in **Supplementary File S5**. The reads from each sample were mapped to the reference assembly with RSEM software (Li and Dewey, 2011). The percentage of total mapped reads ranged from 78.34 to 85.40% across the 14 samples (**Supplementary File S6**). The expression of each gene was calculated as the RPKM with HTSeq software (Anders et al., 2015). After the gene expression values were log2 transformed and normalized with the quantile function from the Limma R package, the gene expression were further normalized with the median subtracted within each sample (Gentleman et al., 2004).

### Identification of Differentially Expressed Genes

The genes that were differentially expressed between distinctive conditions were identified as those genes whose expression had changed more than fourfold. Specifically, only samples affected by WSSV infection and STAT dsRNA interference at both 48 and 72 h post-infection (hpi) were considered here, since the copy number of WSSV was reported with no difference between shrimps injected with STAT dsRNA at 12 hpi and the control (Wen et al., 2014). For each time point, a set of DEGs were identified between two different conditions, and the intersection of the two sets of DEGs obtained at two time points were used for further analysis. In detail, the samples obtained after injection of WSSV (EW\_48 and EW72) were compared with the controls (EP48 and EP72) to identify the genes affected by WSSV infection, and the same for samples injected with STAT dsRNA (SP48 and SP72) and samples injected with both WSSV and STAT dsRNA (SW48 and SW72) (**Supplementary Files S7, S8**).

### Construction of Gene Co-expression Network

For the gene co-expression network, the PCC was calculated to quantify the association between each pair of genes. Only genes with the top 30% variance of gene expression across samples were included for further analysis. To keep only significant correlations between genes, and to further reduce the noise in the network, each association was required to have a p-value no larger than 0.01. As a result, the degree distribution of the gene co-expression network followed a power-law distribution with the parameter alpha equal to 2.12, consistent with a previous conclusion that biological networks were scale-free networks (Albert, 2005; Clauset et al., 2009).

### Identification of Modules From Gene Co-expression Network

The modules in the gene co-expression network were detected by ClusterOne, which is a popular tool widely used in the bioinformatics field, where the default parameter was employed for ClusterOne (Nepusz et al., 2012; Giovannetti et al., 2014; Fuchsberger et al., 2016; Wojtuszkiewicz et al., 2016; Yang et al., 2016). The gene components of each module can be found in **Supplementary File S9**. The modules significantly enriched with DEGs were detected with a two-tail Fisher's exact test with a p-value smaller than 0.05, and the same for the detection of biological processes enriched in each module. The functional annotation of shrimp genes was obtained with the functional annotation transferred from their homologous genes. The shrimp gene sequences were queried against the NCBI non-redundant protein sequence database, the NCBI nucleotide sequence database and the EggNOG database with BLAST, where a gene was regarded as a homologous gene with an E-value smaller than 1e-5 (Huerta-Cepas et al., 2015). Consequently, 665,531 biological processes were annotated to 12,505 shrimp genes (available in **Supplementary File S10**).

## DATA AVAILABILITY

The datasets generated for this study can be found in sequence read archive (SRA), SRP159438.

## AUTHOR CONTRIBUTIONS

FL and X-MZ conceived the study and designed the experiments. SL performed the experiments. GZ, JW, and X-MZ analyzed and interpreted the data. GZ and SL wrote the manuscript. SL, GZ, JW, FL, and X-MZ discussed the work and revised the manuscript.

### FUNDING

fphys-10-00212 March 7, 2019 Time: 16:58 # 8

This work was partly supported by the National Natural Science Foundation of China (61772368, 61572363, and 91530321), the Natural Science Foundation of Shanghai (17ZR1445600), and the Open Fund of Laboratory for Marine Biology and Biotechnology, Qingdao National Laboratory for Marine Science and Technology, Qingdao, China (No. OF2015NO13).

#### SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fphys. 2019.00212/full#supplementary-material

### REFERENCES


of EpoR and Jak2. Blood 111, 4511–4522. doi: 10.1182/blood-2007-07- 102848



**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2019 Zhu, Li, Wu, Li and Zhao. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Understanding Molecular Mechanisms of the Brain Through Transcriptomics

Wei Wang and Guang-Zhong Wang\*

Key Laboratory of Computational Biology, CAS-MPG Partner Institute for Computational Biology, Shanghai Institute of Nutrition and Health, Shanghai Institutes for Biological Sciences, University of Chinese Academy of Sciences, Chinese Academy of Sciences, Shanghai, China

The brain is the most complicated organ in the human body with more than ten thousand genes expressed in each region. The molecular activity of the brain is divergent in various brain regions, both spatially and temporally. The function of each brain region lies in the fact that each region has different gene expression profiles, the possibility of differential RNA splicing, as well as various post-transcriptional and translational modification processes. Understanding the overall activity of the brain at the molecular level is essential for a comprehensive understanding of how the brain works. Fortunately, the development of next generation sequencing technology has made it possible to measure the molecular activity of a specific tissue as a daily routine approach of research. Therefore, at the molecular level, the application of sequencing technology to investigate the molecular organization of the brain has become a novel field, and significant progress has been made recently in this field. In this paper, we reviewed the major computational methods used in the analysis of brain transcriptome, including the application of these methods to the research of human and non-human mammal brains. Finally, we discussed the utilization of transcriptome methods in neurological diseases.

Keywords: brain transcriptome, WGCNA, neurodevelopmental disorders, differentially expressed genes, cerebral cortex

## INTRODUCTION

Humans and other mammalian species are very different in the aspect of several advanced behaviors, such as language, cognition and sleep. How to explain the differences of these behaviors at the molecular level remains a mystery. The most straightforward idea is that these behavioral differences are the result of many behavior related genes in the human genome, that are not found in other primates, or that the genes responsible for some human specific behaviors and other mammalian animals are quite different in structure. Those differences, at the gene level, lead to different functions responsible for the regulation of behavior. This idea was rejected after obtaining some DNA and protein sequences from humans and humanoid primates such as chimpanzees. For example, by comparing the cytochromes c protein sequence of a human with that of a chimpanzee, most sequences were found to be identical. This finding leads to a conjecture that the difference between humans and other species is

#### Edited by:

Shangbin Chen, Huazhong University of Science and Technology, China

#### Reviewed by:

Jihwan Myung, Taipei Medical University, Taiwan Jingpeng Wu, Princeton University, United States

> \*Correspondence: Guang-Zhong Wang guangzhong.wang@picb.ac.cn

#### Specialty section:

This article was submitted to Computational Physiology and Medicine, a section of the journal Frontiers in Physiology

Received: 07 September 2018 Accepted: 20 February 2019 Published: 15 March 2019

#### Citation:

Wang W and Wang G-Z (2019) Understanding Molecular Mechanisms of the Brain Through Transcriptomics. Front. Physiol. 10:214. doi: 10.3389/fphys.2019.00214

not due to differences in their genomic sequences, but mainly because of differences in their regulation and expression (King and Wilson, 1975).

Although there are roughly 20,000 genes in the mouse and human genome (Salzberg, 2018), and about 80% of these genes have significant transcription signatures in the brain (Lein et al., 2007). In the past decade, several important studies have explored the spatio-temporal regulation of gene expression during the brain development of mammals such as mice (Thompson et al., 2014), humans (Colantuoni et al., 2011; Kang et al., 2011; Hawrylycz et al., 2012; Miller et al., 2014) and non-human primates (Bakken et al., 2016; **Table 1**), using multiple dimensions of brain transcriptomes. Several brain gene expression datasets have been released by different labs or organizations (Lein et al., 2007; Johnson et al., 2009; Kang et al., 2011; Shimogori et al., 2010; Hawrylycz et al., 2012; Bakken et al., 2016). Brain transcriptome atlases have offered great resources to understand the gene expression patterns among different brain regions or during different development stages of a mammalian brain (Mahfouz et al., 2017). With the accumulation of microarray and next-generation sequencing (NGS) data, it is time to explore how the brain is organized at the molecular level. Furthermore, analyses of the transcriptional dynamics of the human brain will afford valuable information to illuminate the molecular activities of gene related brain disorders such as autism (Wu et al., 2016).

Here, we reviewed the computational methods employed to investigate the patterns of gene expression and functional organization of the mammalian brain. We focus our discussion on the analysis of spatio-temporal brain transcriptomes, and we first describe different computational methods such as differential expression (DE) analysis and network analysis. We then describe normal gene networks identified in the brains of mice, non-human primates and humans. Finally, we discuss their potential application to better understand brain diseases. These latest advances have provided a deeper understanding of molecular activities in the brain. Due space constraints, the discussion of this article does not include the single-cell transcriptome, which is a very important emerging field for brain transcriptome analysis. Note that in this review we refer to "transcriptome" as the expression profile of all sets of RNA molecules in one cell or a population of cells and "gene expression analysis" as the investigation of expression profiles using computational approaches.

#### METHODS OF TRANSCRIPTOME ANALYSIS OF THE BRAIN

Most methods of brain transcriptome analysis involve identifying differentially expressed genes, either among normal tissues of various brain regions, or between normal tissues and disease tissues such as autism or schizophrenia. The next step is to study the functions of these differentially expressed genes, as well as network properties, such as their features in the co-expression network. The characterization of these properties lay the foundations to understanding the role of these molecules in the brain.

#### Differential Gene Expression Analysis

The aim of differential gene expression analysis is to detect changes of expression levels under different conditions using statistical methods. For microarray data, there are wellestablished methods such as limma (Ritchie et al., 2015), which uses linear models to detect DE of transcriptomic data, as well as to correct batch effects. For RNA-seq data, two models based on the Poisson distribution and the Negative Binomial distribution are frequently used (Soneson and Delorenzi, 2013). The detailed comparisons of different methods, including the well-designed R package "DESeq" (Anders and Huber, 2010) and "edgeR" (Robinson et al., 2010), was discussed in a previous review (Soneson and Delorenzi, 2013).

The rapid development of high-throughput techniques, such as microarrays and NGS, makes it possible to assess the status of a cell's transcriptome at any given time (Barabási and Oltvai, 2004). Several methods are applied to analyze the transcriptome data. Traditional methods involve comparisons of knockout with wildtype samples, or of diseases with control groups. Several pilot studies have provided a first glimpse of the brain transcriptome, mainly with the DE gene methods, to compare knockout with wildtype mice (Geschwind and Konopka, 2009). In the first step, an analysis of DE is performed, and DE genes are identified. Next, functional annotation of these DE genes can be assessed by gene ontology (GO) enrichment or KEGG pathway analysis, and enrichment of disease candidate genes can also be performed. However, as the brain is a complex network system composing of multiple cell types, it is claimed that DE analysis may not be sufficient to obtain the underlying structure of gene expression data from the central nervous system (Miller et al., 2008; Oldham et al., 2008; Winden et al., 2009; Konopka, 2011).

#### Network Analysis

Network-based methods are proven to be more powerful than absolute magnitudes of expression levels, in revealing gene expression patterns (Oldham et al., 2006; Miller et al., 2014; Hawrylycz et al., 2015, 2012), and have been found useful in analyzing the inner workings of a cell (Wang and Huang, 2014). Using network analysis, we can study higher order properties of brain transcriptome.

Gene expression profiling data can be modeled as a network, in which each gene corresponds to a node and gene pairs are connected by an edge if their expression values are highly correlated (Parikshak et al., 2015). In a network, degree is an elementary characteristic of a node, and the degree distribution indicates the probability that a selected node has exactly N links. The nodes' degrees in random networks follow a Poisson distribution, while most biological networks approximate a scalefree topology, which means that fewer nodes are highly connected and most nodes have low connectivity. Biological networks exhibit a high clustering feature and consist of a set of modules, where several nodes form a densely connected community have sparser connections with the rest of the network. Within

#### TABLE 1 | Typical researches on the brain transcriptome of mammals.


functional modules, cellular functions are executed by clustered molecules (Barabási and Oltvai, 2004).

Co-expression of genes is defined as genes with similar expression patterns. Common measures of gene co-expression include Pearson correlation, Spearman correlation, Euclidean distance, and the angle between a pair of observed vectors (D'haeseleer et al., 2000; Horvath and Dong, 2008). In a gene coexpression network, modules refer to sets of highly co-regulated genes (Barabási and Oltvai, 2004). To identify gene modules, several clustering methods have been employed, including hierarchical clustering, model-based clustering, k-means, etc. (**Figure 1**). Genes within a module work together to achieve a distinct function.

One major goal of co-expression network analysis is to identify gene modules (Barabási and Oltvai, 2004). Gene coexpression patterns of the brain are mainly evaluated by correlation-based measurements (Mahfouz et al., 2017). By detecting similar gene expression patterns to disease genes, in silico prediction can be made with the gene co-expression approach. To discover clusters of co-expressed genes within a set of samples, a commonly used unsupervised method is hierarchical clustering (Mahfouz et al., 2017). One method used to identify co-expression modules is Pearson correlation, the most popular co-expression measure (Wang and Huang, 2014), as the distance measurement for hierarchical clustering. Hard thresholding is then applied to produce a network (Li et al., 2016).

One widely used method for co-expression network construction is weighted correlation network analysis (WGCNA), which was first introduced by Zhang and Horvath (2005). It is an informative method for detecting biologically relevant patterns using high-dimensional data sets,

and it allows for the assessment of the relation of modules to experimental traits (Zhang and Horvath, 2005). Genes with strongly covarying patterns are grouped into modules across the sample set. Identified modules are characterized by module eigengenes, and hub genes refer to genes that are highly correlated with the eigengenes. WGCNA is a systems biology method used to construct modules of gene co-expression with an unsupervised clustering approach and has been broadly applied to transcriptome analysis of the mammalian brain (Oldham et al., 2006; Hawrylycz et al., 2012; Thompson et al., 2014; Bakken et al., 2016). WGCNA searches for gene modules of co-expression with high topological overlap (Zhang and Horvath, 2005). First, a soft thresholding power is chosen to calculate adjacency, which is further transformed into a topological overlap matrix. Then, the dendrogram of genes can be produced through hierarchical clustering. Finally, modules are identified using a dynamic tree cut method for branch cutting (Langfelder and Horvath, 2008).

Besides the popular WGCNA, there are also a number of different methods that have been developed for cluster analysis and further detecting network modularity analysis (**Figure 1**). The K-mean clustering method sets the number of clusters (K) before clustering, and then, based on the calculation of distance (typically Euclidean distance), all different modules are detected (Jain, 2010). However, different cluster initialization may lead to different final clustering. Another plausible approach is based on probability models, the network nodes of which are calculated based on the probability distribution of the genes (Yeung et al., 2001). The model-based method can capture correlation and dependence between attributes, and is implemented in the R package "mclust" (Yeung et al., 2001).

## ANALYSIS OF NORMAL MAMMALIAN BRAINS

### Mouse Brain

For decades, mice have been used as a model organism to study human biology and diseases (Breschi et al., 2017). It is claimed that transcriptional patterns between orthologous organs of different species are more similar than those between different organs from the same species (Brawand et al., 2011; Barbosa-Morais et al., 2012; Merkin et al., 2012; Breschi et al., 2017). Therefore, mouse brain transcriptome data are very useful to complement the study of the human brain and neuronal disorders, as a series of processes of primate brain development are conserved across mammals (Bakken et al., 2016).

Using voxel expression data, Thompson et al. (2014) explored the temporal co-expression patterns of the mouse brain in the diencephalon over three time periods: "embryonic," "postnatal" and "all." They analyzed the "all" period and found that genes in two modules showed strong upregulation in the diencephalon at P14 and P28. They further examined the postnatal cluster and found that a set of well-known oligodendrocyte genes were not widely distributed until P14. An especially interesting temporal expression pattern was that P14 exhibited strong thalamus-specific expression of predominantly TF genes. The authors inferred that this may coincide with eye opening and the initial reception of visual stimulation by the thalamus.

### Non-human Primate Brain

fphys-10-00214 March 13, 2019 Time: 18:17 # 5

Despite the fact that humans and mice share many core biological processes and genetic elements, many human brain features are poorly modeled in rodents (Bakken et al., 2016) due to the extended periods of primate brain development. Compared with rodents, humans and monkeys are more similar on expression trajectories of brain development (Bakken et al., 2016). In addition, the comparison of co-expression patterns between human and chimpanzee brains, showed that many hub genes in the human brain are conserved in the chimpanzee brain (Oldham et al., 2006).

Bakken et al. (2015) explored the spatio and temporal expression patterns of a postnatal brain of a rhesus monkey. Five brain regions were considered for the genome-wide gene expression at birth, infancy, childhood and young adulthood. They identified 27 modules in total. Correlating each module eigengene with age and brain region, they found several age-related modules, with a gradual shift of gene expression postnatally. They also identified cortical area-specific expression modules such as the primary visual cortex enriched module (M6). They explored the expression of M6 genes, and confirmed the previous finding that, in rhesus monkey and adult human brains, the gene expression pattern in the primary visual cortex is distinct from that of other brain regions.

However, in the cerebral cortex, there are prominent differences between humans and chimpanzees, consistent with the expansion of the cortex in the human lineage (Oldham et al., 2006). Moreover, Zhu et al. (2018) compared the development of the nervous system between humans and macaques, and detected a cup-shaped pattern of transcriptomic differences between the two species. In addition, they also identified human-distinct gene co-expression modules, indicating the difference of molecular mechanisms for species divergence, which could play a role in mental disorders. Therefore, to reveal human-specific features of the brain at the molecular level, it is necessary to use human brain transcriptome instead of a non-human primate brain transcriptome.

### Human Brain

Human brain development is a complex process and depends on the precise regulation of gene expression (Rakic, 2009; Rubenstein, 2011). Using transcriptome data of highly differential stability genes, Hawrylycz et al. (2015) constructed a consensus gene co-expression network and found several modules with the most neuronal function-related annotations. Allocating genes to each of the identified modules according to the gene's correlation to the corresponding module eigengene, they detected a number of modules which were remarkably selective for certain brain regions. Interestingly, when assessing the module preservation between humans and mice, they found that some neuronrelated modules were well preserved, whereas many of the most non-neuronal modules were poorly preserved. Nevertheless, several genes differ in their expression patterning across species even in highly preserved modules. Modules associated with neurons were better conserved than modules associated with glia.

Using data from 16 regions comprised of six brain structures across pre- and postnatal development periods, Kang et al. (2011) created a gene co-expression network and identified 29 modules related to different spatio-temporal profiles. They found that 90% of the expressed genes were differentially regulated at the whole-transcript or exon level across brain regions or brain development periods. Among these modules, M8 showed the highest expression levels in the early fetal neocortex and hippocampus, and then a progressive drop in expression levels until infancy. The hub genes of M8 are involved in the development of the neocortex and the hippocampus projection neurons. In addition, they identified two temporally regulated modules, with opposite developmental trajectories: M20 showed decreased expression while M2 showed increased expression, with the shift just before birth, which indicates that environmental influences are probably associated with the transcriptional changes at this period of brain development.

Furthermore, using gene expression data of 11 neocortex areas in human and macaque brains, Pletikos et al. (2014) analyzed the spatial expression patterns among areas across development periods. They first applied the ANOVA approach to identify differentially expressed genes among neocortex regions, at each development period and proposed an hourglass model of interareal transcriptional divergence over time, indicating that the spatial pattern of interareal divergence is primarily driven by a number of functional areas. In addition, to gain insight into the organization of the neocortex transcriptomes, they further performed WGCNA with samples from two periods (fetal development period and from adolescent period onward) of increased interareal differences and identified 122 modules and 207 modules, respectively. Most of the fetal modules showed temporally specified areal patterns and lost their prominent areal differences postnatally. In contrast, adolescent and adult modules were more stable over time, and showed less complex spatial patterns.

Moreover, Li et al. (2018) integrated transcriptome, DNA methylation, and histone modifications data from 16 brain regions, and revealed a cup-shaped pattern of regional divergence during prenatal and postnatal development. Specifically, they identified a group of gene co-expression modules associated with dynamic spatiotemporal trajectories and uncovered that many modules are enriched with specific cell types or diseaseassociated genes.

Using organoids from human pluripotent cells, Amiri et al. (2018) modeled the cerebral cortical development between 5 and 16 weeks post-conception. They identified the networks of genes and enhancer modules and found that some enhancer modules converged with gene modules, indicating the regulation of co-expressed genes by enhancers across time.

### MENTAL DISORDERS

fphys-10-00214 March 13, 2019 Time: 18:17 # 6

Integrating co-expression network analysis to traditional differential gene expression analysis uncovered features of normal mammalian brains and expanded our knowledge of the spatio-temporal event in mammalian brain development over the last decade. Moreover, in order to reveal the molecular mechanisms of neuronal disorders, such as Autism spectrum disorder (ASD), Alzheimer's disease (AD), Schizophrenia, etc., co-expression networks are applied to compare healthy and diseased brains, which would also reveal important biological pathways in these disorders and provide potential biomarkers or therapeutic targets (Keo et al., 2017; Seyfried et al., 2017; Mostafavi et al., 2018; Rajarajan et al., 2018).

Utilizing gene expression analysis to decode the mechanism of mental disorders is a powerful tool as it is largescale, high-throughput and cost-efficient. ASD is a group of neurodevelopmental disorders characterized by deficits in social functioning and repetitive, restricted behaviors or interests (Bourgeron, 2015). Previous findings show that ASD genes are enriched only in pathways during early fetal development (Parikshak et al., 2013). In the networks of a postnatal rhesus brain, Bakken et al. (2015) found that ASD gene enriched modules show significant enrichment in the neocortex. Gene expression in one of these modules was high in the neonatal cortex and striatum but low during infant and juvenile development periods. Combining dense temporal sampling of prenatal and postnatal periods, Bakken et al. (2016) demonstrated a high-resolution transcriptional atlas of macaque (Macaca mulatta) brain development with fine anatomical division of cortical and subcortical regions associated with human neuronal disease. They found that many ASD genes exhibited a coordinated expression in postmitotic neurons both prenatally and postnatally. They also found that in neuronal progenitor-enriched modules, MCPH genes were enriched in early- to mid-fetal ages. No enrichment of intellectual-disabilityassociated genes was observed in any modules. Using 109 cortex miRNA samples, Wu et al. (2016) applied WGCNA and identified 11 modules. By examining the relationship between module eigengene and ASD traits, they detected three modules significantly correlated with ASD, and successfully predicted and validated two transcription factors which regulate neuronal genes in ASD.

Alzheimer's disease is the most common cause of neurodegenerative dementia (Verheijen and Sleegers, 2018). Using 19 cortical regions, Wang et al. (2016) constructed region-specific co-expression networks, and rank-ordered co-expression modules and brain regions based on their association with AD pathological traits. They found that temporal lobe gyri exhibited the largest and earliest gene expression abnormalities. Mostafavi et al. (2018) applied a network-based method and identified specific genes that were associated with AD-related traits. By integrating clinical, neuropathology and gene expression data, they detected a co-expression module which is related to both cognitive decline and β-amyloid burden. Furthermore, they identified two genes in the module, INPPL1 and PLXNB1, as potential AD therapeutic targets.

Gandal et al. (2018a) analyzed the transcriptome of five major psychiatric disorders, including ASD and schizophrenia, and identified a number of shared and disorder-specific coexpression modules. They found an up-regulated module, which is associated with astrocyte, and several down-regulated modules, which are annotated as neuronal or mitochondrial, across ASD, schizophrenia, and bipolar disorder, suggesting pathways of molecular convergence of major neuropsychiatric illness.

Nevertheless, the PsychENCODE consortium integrated multiomics data and provides a comprehensive resource for the functional genomics of the human brain (Wang et al., 2018). For example, Gandal et al. (2018b) integrated RNA-seq and genotypes in brain samples with ASD, schizophrenia, and bipolar disorder, and detected gene co-expression modules related to each disorder. They found that one module, associated with the microglial cell marker, is upregulated in ASD, and down-regulated in schizophrenia and bipolar disorder, suggesting a previously unrevealed neural-immune mechanism.

Integration of co-expression data with clinical traits enables the identification of novel disease related modules and hub genes, which provide potential therapeutic targets for related neuronal disorders.

#### CONCLUSION AND FURTHER DIRECTION

Transcriptomic data of the mammalian brain provides eminent opportunities to illuminate how the brain works in the molecular level. The current status of this field has provided us with great insight on the molecular developmental patterns of the brain, and we expect more primate brains to be included in future research. Additionally, other molecular activities such as microRNA and non-coding RNAs should be profiled at the brain-wide scale as well. In this article, we summarized the progress made by various researchers in the analysis of brain transcriptome in recent years. In addition to traditional DE analysis, network-based methods offer an unsupervised perspective to analyze large scale data from mouse to human brains, as well as data of different developmental stages of each species. Moreover, systems-level analysis assembles correlates single genes and enables the discovery of key pathways. As the rapid development of NGS in the past decade has accelerated the research on transcriptomics of the brain, the knowledge obtained from this field can facilitate deciphering the complexity of the brain and help us gain valuable insight into the organization of the brain's functions. Nevertheless, the use of network-based methods integrated with clinical traits and experimental validation (Mostafavi et al., 2018) demonstrates a blueprint for investigating complex neuronal diseases.

One major limitation of bulk sample transcriptome analysis is that it can't provide insight into the behavior of different cell types, which is a critical aspect of brain research. Similarly,

the analysis methodologies developed for bulk samples may not be suitable for analyzing single-cell data with algorithms of a network. In this mini-review, the recent emerging single-cell sequencing data is not covered due to the space constraints. The analysis of differentially expressed genes between different cell types or of the marker between different cell types would be an important topic in the future. Research in this area is progressing rapidly (Zeisel et al., 2015; Tasic et al., 2016, 2018; Fan et al., 2018; Kelley et al., 2018; Zhong et al., 2018), and we look forward to some critical improvements for the identification of cell types related to differentially expressed genes in the future.

#### REFERENCES


#### AUTHOR CONTRIBUTIONS

WW drafted the manuscript. WW and G-ZW finalized the manuscript.

## FUNDING

We thank the National Key R&D Program of China (2016YFC1303100 and 2016YFC0901700) and the National Natural Science Foundation of China (Nos. 31600960, 31871333, and 81827901) for their support.



**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2019 Wang and Wang. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Astrocyte as Spatiotemporal Integrating Detector of Neuronal Activity

Susan Yu. Gordleeva\*, Anastasia V. Ermolaeva, Innokentiy A. Kastalskiy and Victor B. Kazantsev

*Department of Neurotechnology, Lobachevsky State University, Nizhny Novgorod, Russia*

The functional role of astrocyte calcium signaling in brain information processing was intensely debated in recent decades. This interest was motivated by high resolution imaging techniques showing highly developed structure of distal astrocyte processes. Another point was the evidence of bi-directional astrocytic regulation of neuronal activity. To analyze the effects of interplay of calcium signals in processes and in soma mediating correlations between local signals and the cell-level response of the astrocyte we proposed spatially extended model of the astrocyte calcium dynamics. Specifically, we investigated how spatiotemporal properties of Ca2<sup>+</sup> dynamics in spatially extended astrocyte model can coordinate (e.g., synchronize) networks of neurons and synapses.

#### Edited by:

*Alexey Zaikin, University College London, United Kingdom*

#### Reviewed by:

*Shivendra Gajraj Tewari, Biotechnology HPC Software Applications Institute (BHSAI), United States Ekkehard Ullner, University of Aberdeen, United Kingdom*

#### \*Correspondence:

*Susan Yu. Gordleeva gordleeva@neuro.nnov.ru*

#### Specialty section:

*This article was submitted to Computational Physiology and Medicine, a section of the journal Frontiers in Physiology*

Received: *30 September 2018* Accepted: *06 March 2019* Published: *18 April 2019*

#### Citation:

*Gordleeva SY, Ermolaeva AV, Kastalskiy IA and Kazantsev VB (2019) Astrocyte as Spatiotemporal Integrating Detector of Neuronal Activity. Front. Physiol. 10:294. doi: 10.3389/fphys.2019.00294* Keywords: astrocyte, synaptic transmission, neuron–astrocyte interaction, neuron–astrocyte network, calcium

### INTRODUCTION

The functional role of astrocyte calcium signaling remains intensely debated. One of the principal reasons for such a debate is that the astrocytic Ca2<sup>+</sup> dynamics possesses high complexity which was confirmed by new experimental approaches to study the signaling of astrocytes at qualitatively new spatial-temporal resolutions (Volterra et al., 2014; Bindocci et al., 2017; Wu et al., 2018). Another reason was the evidence of bi-directional astrocytic regulation of neuronal activity referred as gliotransmission [Ca2+-dependent release of neurotransmitters (glutamate, D-serine, ATP) by astrocytes] (Araque et al., 2014; Bazargani and Attwell, 2016; Fiacco and McCarthy, 2018; Savtchouk and Volterra, 2018).

Astrocytes respond to synaptic activity by intracellular Ca2<sup>+</sup> elevations (Verkhratsky et al., 2012). Synaptically-released neurotransmitters (e.g., glutamate) can activate G-coupled receptors [e.g., the metabotropic glutamate receptors (mGluRs)] (Porter and McCarthy, 1996; Pasti et al., 1997; Perea and Araque, 2005), which, upon activation, promote inositol 1,4,5-triphosphate (IP3) production by phosphoinositide-specific phospholipase C β (PLCβ) (Zur Nieden and Deitmer, 2006). In turn, the elevation of cytosolic concentration of the second messenger IP<sup>3</sup> promotes the Ca2+-induced Ca2<sup>+</sup> release (CICR) from the astrocyte's endoplasmic reticulum (ER) stores. Clustering of astrocytic receptors, targeted by synaptically released neurotransmitters at points of contact of synapses with astrocytic processes (Di Castro et al., 2011; Panatier et al., 2011; Arizono et al., 2012) provides spatially confined sites of IP<sup>3</sup> production, whose differential activation could result in rich spatiotemporal IP<sup>3</sup> and Ca2<sup>+</sup> dynamics (Volterra et al., 2014).

There are two main types of IP3-induced CICR observed in astrocytes (Volterra et al., 2014; Rusakov, 2015; Bindocci et al., 2017): small, fast Ca2<sup>+</sup> events that are confined to their (primary) processes and caused by minimal synaptic activity; and Ca2<sup>+</sup> elevations propagating

**183**

along astrocytic processes that can reach the soma and trigger whole-cell global Ca2<sup>+</sup> signal. The latter slow calcium events were induced by intense neuronal activity. This circumstance can be interpreted as the ability of astrocytes to perceive the information contained in the repetition rate of action potentials, however, this occurs on time scales larger than the characteristic times inherent in the synaptic activity.

The elevation of intracellular calcium concentration in astrocyte can trigger the release of various active chemicals, gliotransmitters, such as glutamate, GABA, ATP, and D-serine (Bezzi and Volterra, 2001). The concept of "tripartite synapse" (Araque et al., 1999) is based on the ability of gliotransmitters to regulate synaptic transmission and plasticity on time scales from seconds to minutes (see Araque et al., 2014 for a recent review).

Since effect of single gliotransmitter depends on the type of circuit and targeted neurons (Araque et al., 2014), we focused in our study on the excitatory transmission in hippocampus. It was shown that hippocampal astrocyte can release ATP (Zhang et al., 2003), D-serine–co-agonist of the NMDA receptor (NMDAR) (Henneberger et al., 2010; Zhuang et al., 2010) and glutamate. After conversion to adenosine, ATP can depress, or facilitate excitatory synaptic transmission acting on either A1 or A2A receptors, respectively (Serrano et al., 2006; Pascual et al., 2012). At CA3-CA1 synapses astrocytic glutamate acts on presynaptic NMDARs (Jourdain et al., 2007) or mGluRs (Navarrete and Araque, 2010; Navarrete et al., 2012) to potentiate or decrease release probability, respectively. In the CA1 region a long-term potentiation (LTP) which required presynaptic mGluR activation (Perea and Araque, 2007) can be induced by the postsynaptic activity accompanying by glutamate release from astrocyte due to Ca2<sup>+</sup> elevation. Also, it was shown (Henneberger et al., 2010) that D-serine, released from astrocyte and being a coagonist of postsynaptic NMDARs, can trigger NMDAR-mediated LTP at synapses nearby of the astrocyte.

Intense modeling efforts have been devoted in recent years to understand the functional role of astrocytic modulation of the neuronal communication (see Oschmann et al., 2018 for a recent review, Kanakov et al., 2019). There are several biophysical studies of astrocytic influence on post- and presynaptic neuronal activity (De Pittà et al., 2011; Gordleeva et al., 2012; Tewari and Majumdar, 2012a,b; Tewari and Parpura, 2013; De Pittà and Brunel, 2016; Flanagan et al., 2018). However, these works did not account for the impact of spatiotemporal patterns of calcium dynamics in astrocyte, i.e., the spatial distribution of the calcium activity was neglected. Meanwhile, under the action of highfrequency action potentials in one synapse or coherent activity of several synapses, the calcium signal originating in astrocyte can spread along the processes and throughout the astrocyte and cause the release of the gliotransmitter in remote locations, affecting other synapses.

There are also a few recent computational works which investigated mechanisms underlying IP3-triggered CICRmediated spatiotemporal dynamics in single astrocyte (Wu et al., 2014; Gordleeva et al., 2018; Savtchenko et al., 2018).

To analyze the principles of generation of calcium signals in processes and soma of the astrocyte and to seek mechanisms of correlations between local signals and the global signalization response of the astrocyte including its spatially distributed structure, we propose a spatially extended model of astrocyte calcium dynamics. In this paper we investigate how spatiotemporal Ca2<sup>+</sup> dynamics in spatially extended astrocyte model can coordinate and synchronize network signaling.

### METHODS

For our purpose we designed a neuron-astrocyte network composed of 100 synaptically coupled Hodgkin-Huxley excitatory neurons (Hodgkin and Huxley, 1952) and two astrocytes connected via gap junction. Each astrocyte has spatially distributed structure that repeated the morphology of real astrocyte (Bindocci et al., 2017). For illustration, we assumed that each process of the astrocyte, interacting with presynaptic and postsynaptic neurons, forms one tripartite synapse providing the connectivity between neuronal and astrocytic parts of the network. The schematic architecture of the model is shown in **Figure 1**. Our model of the tripartite synapse describes the effects of the astrocytic modulation of synaptic transmission in the CA1-CA3 area of hippocampus. We consider glutamate as the neurotransmitter, and glutamate and D-serine as the gliotransmitters. We describe three effects resulting from the influence of the gliotransmitters on the synapse: (i) potentiation of presynaptic release probability due to glutamate acting on presynaptic NMDARs; (ii) depression of presynaptic release probability due to glutamate acting on presynaptic mGluRs; and (iii) increase of the postsynaptic currents due to D-serine modulation of the postsynaptic NMDA receptors.

#### Neural Network

Neural network consists of 100 excitatory synaptically coupled Hodgkin-Huxley neurons (Hodgkin and Huxley, 1952; Esir et al., 2018). We use random coupling topology with connection probability for each pair of neurons equal to 20% (Braitenberg and Schüz, 1998). The dynamics of the neuronal membrane potential is described by the following ionic current balance equation:

$$C\frac{dV^{(n)}}{dt} = I\_{channel}^{(n)} + I\_{app}^{(n)} + \sum\_{m} I\_{ym}^{(mn)} + I\_{P}^{(n)},\tag{1}$$

where capacitance, C, is 1 µF/cm<sup>2</sup> , the superscript (n = 1,. . . , M) corresponds to a neuronal index and (m) corresponds to an index of input connection. The Na+, K+, and leak currents are expressed as follows:

$$I\_{channel} = -\text{g}\_{\text{Na}} m^3 h \left( V - E\_{\text{Na}} \right) - \text{g}\_K n^4 \left( V - E\_K \right)$$

$$- \text{ g}\_{leak} \left( V - E\_{leak} \right), \tag{2}$$

where gNa and g<sup>K</sup> are the potassium and sodium conductances (mS/cm<sup>2</sup> ), ENa, and E<sup>K</sup> are the potassium and sodium reversal potentials (mV), gleak and Eleak are the leak conductances and leak reversal potential, respectively.

The kinetics of the potassium channel is determined by:

$$\frac{dm}{dt} = \alpha\_m (1 - m) - \beta\_m m,$$

$$\alpha\_m = \frac{0.1(V+40)}{1 - \exp(-(V+40)/10)},$$

$$\beta\_m = 4 \exp(-(V+65)/18), \tag{3}$$

$$\frac{dh}{dt} = \alpha\_h (1 - h) - \beta\_h h,$$

$$\alpha\_h = 0.07 \exp(-(V+65)/20),$$

$$\beta\_h = \frac{1}{1 + \exp(-(V+35)/10)}. \tag{4}$$

The kinetics of the sodium channel is determined by:

$$\frac{dn}{dt} = \alpha\_n (1 - n) - \beta\_n n,$$

$$\alpha\_n = \frac{0.01(V + 55)}{1 - \exp(-(V + 55)/10)},$$

$$\beta\_n = 0.125 \exp(-(V + 65)/80). \tag{5}$$

The applied currents I (n) app are fixed at constant value controlling the depolarization level and dynamical mode of the neuron (Kazantsev and Asatryan, 2011). We use I (n) app = 4.5 µA/cm<sup>2</sup> which corresponds to the neuron's excitable mode. The synaptic current Isyn (µA/cm<sup>2</sup> ) simulating interactions between the neurons. Each neuron is stimulated by a Poisson pulse train mimicking external spiking inputs I (n) P (µA/cm<sup>2</sup> ) with a certain average rate λ. Each Poisson pulse has rectangular shape with fixed width of 10 ms and variable height, which is sampled independently for each pulse from uniform random distribution on interval [0, 1.5]. Sequences of Poisson pulses applied to different neurons are independent.

#### Synaptic Dynamics

Each spike on presynaptic neuron results in the release of the glutamate quant. We describe presynaptic dynamics of the glutamate, G, using a mean field approach from Gordleeva et al. (2012):

$$\frac{dG}{dt} = -\alpha\_G(G - k\_{pre}\delta(t - t\_k)),\tag{6}$$

where α<sup>G</sup> denotes the clearance rate of the neurotransmitter, kpre denotes the efficacy of the presynaptic release, δ denotes the Dirac delta function and t<sup>k</sup> is spike time.

The release of the glutamate leads to excitatory postsynaptic current (EPSC). For description of the EPSCs dynamics we use the approach from our previous work (Gordleeva et al., 2012):

$$\frac{dI\_{EPS}}{dt} = -\alpha\_I \left( I\_{EPS} - A \delta \left( t - t\_k \right) \right), \tag{7}$$

$$P(A) = \frac{2A}{b^2} \exp(-\frac{A^2}{b^2}), \int\_0^{+\infty} P(A) dA = I'(1) = 1,$$

where α<sup>I</sup> is rate constant. According to the experimental data (Fernández-Ruiz et al., 2012; Guzman et al., 2016) we supposed that amplitude of the EPSCs, A, is gamma-distributed with probability density function P(A), where b is the scaling parameter of gamma-distribution that denotes the impact of the synaptic input.

Integrated synaptic current of the neuron, Isyn, is described by the following equation (Gordleeva et al., 2012):

$$I\_{\rm syn} = \frac{I\_{\rm EPSC}}{1 + \exp(-(G - \theta\_G)/k\_G)},\tag{8}$$

where θ<sup>G</sup> denotes the midpoint and k<sup>G</sup> denotes the slope of the neuronal activation function.

### Geometry of the Astrocytic Model and Astrocytic Ca2<sup>+</sup> and IP<sup>3</sup> Dynamics

To design the architecture of the spatially distributed astrocyte model, we followed available experimental facts (Bindocci et al., 2017) (see **Figure 2A**). Specifically, we consider the astrocyte as network of inter-coupled small compartments, which have a cylindrical shape (Gordleeva et al., 2018). Each element is a unitlength cylinder with a finite radius r containing ER (**Figure 2B**). Compartments are coupled through the diffusion of calcium and IP<sup>3</sup> controlling the calcium exchange between the cytoplasm and intracellular stores of calcium (in particular, ER).

The dynamics of each compartment is described by the following set equations (modified from Li and Rinzel, 1994; Gordleeva et al., 2018). The balance of calcium fluxes in cytosol for particular compartment is described by

$$\frac{d[Ca\_c]\_i}{dt} = \frac{S\_{ERi}}{F \cdot V\_i} (J\_{IP3R} - J\_{SERCA} + J\_{ERleak}) + J\_{Cadiff}.\tag{9}$$

Changing free calcium concentration in the cytosol of the compartment i is defined by calcium exchange with the ER involving the calcium release from the ER to the cytosolic volume through IP<sup>3</sup> receptors, JIP3R, the ATPase Ca2+- pump "SERCA" by JSERCA, and by calcium leak from the ER, JERleak. SERi = S<sup>i</sup> √ rERi denotes the surface of the ER. The volume and the surface of the intracellular space are defined as V<sup>i</sup> and S<sup>i</sup> , respectively. rER is the ratio of the volume of ER to the volume of cytoplasm in the considered compartment of the astrocyte. The distribution of values over compartments in the developed model was chosen according to experimental data (Patrushev et al., 2013; Oschmann et al., 2017) and can be found in **Supplementary Table 2**.

The current JIP3<sup>R</sup> is expressed by the following equations (Li and Rinzel, 1994) approximating the kinetics of ER IP3Rs:

$$\begin{aligned} \label{eq:1} \quad & \frac{FVi}{S\_i} \nu\_1 m\_{\infty}^3 n\_{\infty}^3 h^3 ([Ca\_{ER}]\_i - [Ca\_c]\_i), \\ \frac{dh\_i}{dt} &= \frac{h\_{\infty} - h\_i}{\tau\_h}, \\ \space{1.} \quad & h\_{\infty} = d\_2 \frac{([IP\_3]\_i + d\_1)([IP\_3]\_i + d\_3)}{(d\_2([IP\_3]\_i + d\_1) + [Ca\_c]\_i([IP\_3]\_i + d\_3))}, \\ \space{1.} \quad & \frac{[Ca\_c]\_i}{[Ca\_c]\_i + d\_5}, \space{1.} \quad & m\_{\infty} = \frac{[IP\_3]\_i}{[IP\_3]\_i + d\_1}, \\ \space{1.} \quad & \frac{([IP\_3]\_i + d\_3)}{d\_2(d\_2([IP\_3]\_i + d\_1) + [Ca\_c]\_i([IP\_3]\_i + d\_3))}. \end{aligned}$$

Here [Cac]<sup>i</sup> and [CaER]<sup>i</sup> are calcium concentrations in the cytosol and in the ER of the compartment i, respectively. The dynamics of the Ca2<sup>+</sup> concentration in the ER is described by:

$$\frac{d[\text{Ca}\_{\text{ER}}]\_i}{dt} = \frac{S\_{\text{ERi}}}{F \cdot V\_{\text{ERi}}} (-J\_{IP3R} + J\_{\text{SERCA}} - J\_{\text{ERleak}}) + J\_{\text{CaERdiff}} \tag{11}$$

where SERi and VERi denote the area and the volume of the ER, respectively.

The variable h denotes the fraction of activated IP<sup>3</sup> receptors and the other gating variables for IP3Rs are set to their equilibrium values m∞ and n∞. Active ATP-dependent current JSERCA pumping calcium back to the ER and the Ca2<sup>+</sup> leak current from the ER, JERleak, are given by the following equations:

$$J\_{\rm SERCA} = \frac{FV\_i}{S\_i} \nu\_3 \frac{[\rm Ca\_c]\_i^2}{[\rm Ca\_c]\_i^2 + k\_3^2},$$

$$J\_{\rm ERleak} = \frac{FV\_i}{S\_i} \nu\_2 ([\rm Ca\_{ER}]\_i - [\rm Ca\_c]\_i). \tag{12}$$

The change of the IP<sup>3</sup> concentration is defined by production and degradation that are regulated by cytosolic Ca2+. These include Ca2+-dependent phospholipase C δ (PLCδ) mediated IP<sup>3</sup> synthesis and Ca2+-dependent IP<sup>3</sup> degradation by the IP<sup>3</sup> 3 kinase (IP3-3K) and the inositol polyphosphate 5-phosphatase (IP-5P) (De Pitta et al., 2009):

$$\frac{d[IP\_3]\_i}{dt} = J\_{PLC\beta} + J\_{PLC\delta} - J\_{\text{deg }3K} - J\_{\text{deg }5P} + J\_{IP 3d} \text{f.} \tag{13}$$

The first current, JPLCβ, describes agonist dependent IP<sup>3</sup> production by PLCβ. The activation of PLCβ by G-protein is controlled by the glutamate concentration, G, (Equation 6) (De Pitta et al., 2009):

$$J\_{\rm PLC} = \nu\_{\beta} \frac{G^{0.7}}{G^{0.7} + (K\_R + K\_{\rho} \cdot \frac{[Ca\_c]\_i}{[Ca\_c]\_i + K\_{\pi}})^{0.7}},\tag{14}$$

where v<sup>β</sup> is the rate of IP<sup>3</sup> production by PLCβ, K<sup>R</sup> is the glutamate affinity of the receptor, K<sup>p</sup> is Ca2+/PLC- dependent inhibition constant, and K<sup>π</sup> is Ca2<sup>+</sup> affinity of PLC.

The second term in Equation (13) describes cytosolic calcium dependent PLCδ activation (De Pitta et al., 2009):

$$J\_{\rm PLC\delta} = \frac{\nu\_{\rm \\$}}{1 + \frac{[IP\_3]\_i}{k\_{\rm \\$}}} \frac{[\rm Ca\_c]\_i^2}{[\rm Ca\_c]\_i^2 + K\_{\rm PLC\\$}^2},\tag{15}$$

where v<sup>δ</sup> described the maximal rate of IP<sup>3</sup> production by PLCδ, kδ– the inhibition constant, and KPLCδ– the Ca2<sup>+</sup> affinity of PLCδ.

The IP<sup>3</sup> degradation by IP3-3K and IP-5P described by the following equations (De Pitta et al., 2009):

$$J\_{\rm deg\,3K} = \nu\_{\rm 3K} \frac{[Ca\_{\rm c}]\_i^4}{[Ca\_{\rm c}]\_i^4 + K\_D^4} \cdot \frac{[IP\mathfrak{J}]\_i}{[IP\mathfrak{J}]\_i + K\_{\rm 3}}, \quad J\_{\rm deg\,5P} = r\_{\rm 5P}[IP\_{\rm 3}]\_i. \tag{16}$$

The whole process dynamics is formed by the intracellular diffusion of calcium and IP<sup>3</sup> accounted by the following fluxes:

$$\begin{aligned} \textit{JIP3diff} &= \textit{dIP3}(i, i+1) \{ [IP3]\_{(i+1)} - [IP3]\_{i} \} \\ &+ \textit{d}\_{IP3(i, i-1)} ([IP3]\_{(i-1)} - [IP3]\_{i}), \\ \textit{J}\_{\textit{Cadiff}} &= \textit{d}\_{\textit{Ca(i, i+1)}} ([\textit{Ca}\_{\textit{c}}]\_{(i+1)} - [\textit{Ca}\_{\textit{c}}]\_{i}) \\ &+ \textit{d}\_{\textit{Ca(i, i-1)}} ([\textit{Ca}\_{\textit{c}}]\_{(i-1)} - [\textit{Ca}\_{\textit{c}}]\_{i}). \end{aligned}$$

The diffusion flux through compartments of ER described by the following:

$$J\_{\text{CaERdiff}} = d\_{\text{CaER}}([\text{Ca}\_{\text{ER}}]\_{(i+1)} + [\text{Ca}\_{\text{ER}}]\_{(i-1)} - 2[\text{Ca}\_{\text{ER}}]\_{i}). \tag{18}$$

Note, that the values of the diffusion rates from compartment j (j = i+1; j = i−1) to compartment i for IP3, dIP3ij, and for calcium, dCaij, depend on the compartment geometry (e.g., compartment volume) and are different the inward and outward fluxes at the process branching sites:

$$d\_{IP3\bar{i}\bar{j}} = \frac{D\_{IP3}A\_{\bar{i}\bar{j}}}{V\_i \cdot \varkappa\_{i\bar{j}}}, \quad d\_{Ca\bar{i}\bar{j}} = \frac{D\_{Ca}A\_{i\bar{j}}}{V\_i \cdot \varkappa\_{i\bar{j}}},\tag{19}$$

where Aij is the cross-section area between compartments, V<sup>i</sup> is the volume of compartment i, xij is the distance between

FIGURE 3 | The dynamics of the tripartite synapse without astrocytic influence on synaptic transmission for two frequencies of spiking presynaptic neuron: (A) 5 Hz and (B) 10 Hz. The *G(t)* is the mean field concentration of glutamate released for each spike on the presynaptic neuron (*Vpre(t)*). Released into synaptic cleft glutamate induced firing of the postsynaptic neuron (*Vpost(t)*) and rise of Ca2<sup>+</sup> in the cytosol of the perisynaptic process compartment (*Ca*2+*(t)*). The elevation of intracellular concentration of Ca2<sup>+</sup> in the astrocytic compartment trigger release gliotransmitter (*Y(t)*) and can propagate along the process to the soma. Time realizations of the intracellular calcium concentrations are marked by the different color in different compartments of the process and shown the propagation of the calcium signals. Axis designation *(Ca*2+*)* corresponds to the model variable *[Cac]* described by Equation (9).

the centers of the nearest-neighbor compartments, which is equal to the unite length of compartment. DIP<sup>3</sup> and DCa is the diffusion constant for IP<sup>3</sup> and Ca2+, respectively. For simplicity, we assume that the size of the ER is the same in all compartments and therefore the diffusion coefficient of Ca2<sup>+</sup> in the ER does not depend on the geometry of the compartment and is constant. Values of model parameters can be found in **Supplementary Table 1**. Note that the time unit in the neuronal model (1–5) is 1 ms. Due to a slower time scale, in the astrocytic model empirical constants are indicated using seconds as time units. When integrating the joint system of differential equations, the astrocytic model time is rescaled so that the units in both models match up.

#### Dynamics of Gliotransmitters

When the [Cac] in astrocytic processes exceeds a threshold [Cac]th concentration, gliotransmitter is released by the astrocyte into the extra-synaptic space. For illustration, we consider that the gliotransmitter is released only from the distal compartment on each astrocytic process forming the tripartite synapse. We describe the concentration of gliotransmitter by the following equations (Gordleeva et al., 2012):

$$\frac{dY\_k}{dt} = -\alpha\_k (Y\_k - H\_k([Ca\_c])),$$

$$H\_k([Ca\_c]) = \frac{1}{1 + \exp(-\frac{[Ca\_c] - [Ca\_c]\_{\text{fl}}}{k\_k})},\tag{20}$$

where index k denotes the type of gliotransmitter released from astrocyte: k = 1 for glutamate and k = 2 for D-serine. α<sup>k</sup> denotes the gliotransmitter clearance rate. The amount of the gliotransmitter released from astrocyte if the Ca2<sup>+</sup> concentration exceeds a threshold accounted by the function H<sup>k</sup> ([Cac]).

Glutamate released from astrocyte can modulate presynaptic release. Equation for presynaptic dynamics (6) considering the astrocytic modulation should be modified to:

$$\frac{dG}{dt} = -\alpha\_G \left( G - k\_0 (1 + \gamma\_1 Y\_1) \delta(t - t\_k) \right),\tag{21}$$

where the influence of glutamate released from astrocyte on the amount of neurotransmitter describes by parameter γ1. γ<sup>1</sup> > 0 for the potentiation and γ<sup>1</sup> < 0 for the depression of neurotransmitter release, respectively.

Astrocytic D-serine modulates the response of the NMDARs on the postsynaptic terminal. This modulation leads to increase the amplitudes of postsynaptic currents. In the model it means the increase of the scaling parameter, b, of the probability density function P(A) (7):

$$b = b\_0(1 + \wp\_2 Y\_2),\tag{22}$$

where γ<sup>2</sup> is the parameter which describe impact of the astrocytic D-serine on the amplitudes of the EPSCs.

#### RESULTS

First, let us consider the dynamics of single tripartite synapse without influence of the gliotransmitters on the synaptic strength. The dynamics of synaptic transmission obtained in model (1–20) is shown in **Figure 3**. We consider quite low frequency of presynaptic firing (**Figure 3A**). The model has been tuned to follow recent experimental data on the calcium activity of astrocyte taken in vivo on a subcellular scale (Bindocci et al., 2017). They showed that astrocyte could response even on the low frequency of neuronal activity. According to experimental data the parameters values were chosen so that even individual action potential on the presynaptic neuron could induce small calcium event in the most distant astrocytic compartment. In response to the glutamate release from presynaptic terminal, the calcium concentration in the distal compartment increases. This increase, however, is not sufficient to trigger the gliotransmission (e.g., Y(t) is close to zero), because the intracellular diffusion calcium concentration consequently increases in all elements of this process. However, the amplitude of these pulses is smaller than the amplitude of the basic response. With increase of presynaptic firing rate the amplitudes of the calcium signals in the distal compartment substantially increase and exceed the threshold of gliotransmission (**Figure 3B**).

Next, we consider the interaction of whole astrocyte and neural network. All neurons of the network are stimulated by Poisson process with fixed frequency. The model architecture of the astrocyte model includes 14 processes, and, hence, the astrocyte interacts with 14 synapses from neural network of 36 neurons. **Figure 4A** illustrates the space-time diagram of calcium signals in the compartmental model. Note that the frequency of calcium signals in soma is much lower than in all compartments of the astrocyte. Calcium signals generated in the distal elements of different processes of the astrocyte propagate to the soma due to diffusion. An increase in the calcium concentration in the soma of the cell induces the propagation of the Ca2<sup>+</sup> signal back through all processes of the model. If one compares the astrocytic calcium activity firing rate (the number of calcium

signals in all compartments in the 100-ms time window) (**Figure 4C**) and corresponding time trace of the intracellular calcium concentration in soma (**Figure 4D**), then it happens that the calcium response in the soma occurs as the result of a space–time integration of calcium fluctuations in the processes of the astrocyte. To estimate the level of functional connectivity between activity of neurons and calcium activity in astrocyte, we calculated the cross correlation (CC) between neuronal firing rate (**Figure 4B**) and astrocytic firing rate (**Figure 4C**). The peak of the CC (**Figure 4E**) indicates the presence of cross correlation (e.g., a synchrony) and estimates the communication delay time τ equal about to 2 s. It is important to note that the peak of the CC exists only for firing rate of presynaptic neurons not all neurons in network (red line on the **Figure 4E**). Thus, the model verified that activation of the astrocyte is stimulated by synchronous in time and in space neuronal activity. This correspond to the experimental data presented in Bindocci et al. (2017). They found very large events in vivo, which they called global Ca2<sup>+</sup> events, that spread spatially to most of the astrocytic structures. Most of global calcium events were registered during movement of the mouse associated intense neuronal discharges. The dependences of the average frequencies of generation of

FIGURE 6 | The dynamics of the tripartite synapse with astrocytic influence on synaptic transmission. (A) Time traces of membrane potentials of pre- (*Vpre(t)*) and post-synaptic (*Vpost(t)*) neurons and calcium concentration (*Ca*2+*(t)*) in distal compartment of the astrocytic process without impact of astrocyte (γ<sup>1</sup> <sup>=</sup> <sup>γ</sup><sup>2</sup> <sup>=</sup> 0). (B) Astrocyte-mediated potentiation of presynaptic release. Red line corresponds to the post-synaptic activity with astrocytic influence (γ<sup>1</sup> = 0.1). Black—without. *Y(t)* is the time trace of gliotransmitter concentration. (C) Astrocyte-mediated inhibition of presynaptic release. Red line corresponds to the postsynaptic activity without astrocytic influence (γ<sup>1</sup> = −0.4). (D) Astrocyte-mediated increasing of PSCs amplitudes. Red line corresponds to the postsynaptic activity with astrocytic influence (γ<sup>2</sup> = *1*). λ = 9 Hz.

calcium signals in the soma and processes of the astrocyte on the firing frequency of the presynaptic neurons are shown on the **Figure 5**. Note that the frequency at the distal compartments reaches the highest values being monotonically dependent on the firing rate of presynaptic neurons and accordingly on the release rate of the neurotransmitter. Calcium signals on the astrocyte soma occur less often when exceeding a certain threshold of the spiking frequency of the presynaptic neurons.

Let us now consider the impact of gliotransmitter on the synaptic transmission. **Figure 6** shows dynamics of single tripartiate synapse. We stimulate the presynaptic neuron by Poisson process and register the activities of postsynaptic neuron, gliotransmitter, and Ca2<sup>+</sup> concentration in perisynaptic process. We analyze the following astrocyte-mediated modulations of synaptic transmission: (i) astrocytic glutamate potentiates of neurotransmitter release by acting on presynaptic NMDARs, γ<sup>1</sup>

> 0 in (21) (Jourdain et al., 2007) (**Figure 6B**), (ii) astrocytic glutamate can target presynaptic mGluRs which decrease release probability, γ<sup>1</sup> < 0 in (21) (Semyanov and Kullmann, 2000; Perea and Araque, 2007; Navarrete et al., 2012) (**Figure 6C**); (iii) the gliotransmitter D-serine triggers increase of the amplitudes of post synaptic currents by acting as the coagonist of postsynaptic NMDARs, γ<sup>2</sup> > 0 in (22) (Henneberger et al., 2010) (**Figure 6D**).

Gliotransmission can co-ordinate several synapses and networks of neurons (Serrano et al., 2006; Pascual et al., 2012). **Figure 7** illustrates the heterosynaptic astrocyte-induced modulation of neurotransmission in our model. Ca2<sup>+</sup> signals evoked locally by high-frequency discharge of synapse 1 (**Figure 7A**) can propagate intracellularly from their initial source toward different process and trigger gliotransmitter release at nearby synapse (synapse 2) facilitating (**Figure 7B**) or depressing (**Figure 7C**) synaptic transmission.

Next, we study a bidirectional regulation of the signal transmission in neural ensemble by astrocytes. According to the

FIGURE 8 | A raster plots of the neuronal and calcium activities in astrocyte. λ = 7 Hz. We consider astrocyte-mediated potentiation of presynaptic release (γ<sup>1</sup> = 0.1). Neuron-astrocyte network under study consists of (A) one astrocyte and 36 neurons. Neurons 0–14 are postsynaptic; (B) two connected via gap-junction astrocytes (color of dots correspond to different cells) and 100 neurons. Neurons 0–28 are postsynaptic.

experimental data astrocytes occupy non-overlapping territories (Halassa et al., 2007). Thus, only one astrocyte can impact on transmission in a determined set of thousands synapses. We consider neuron-astrocyte network consists of two astrocytes connected via gap junction and 100 synaptically coupled neurons (**Figure 8**). **Figure 8A** shows communication of one astrocyte and small neuronal network of 36 neurons the same as on **Figure 4** but with taking into consideration astrocytic modulation of synaptic transmission. High frequency of neuronal firing rate increases the probability of synchronous activity in neighboring to astrocyte synapses. Such coherent synaptic activity induces Ca2<sup>+</sup> elevations in different astrocytic processes, which due to spatial-temporal integration results in global, long lasting Ca2<sup>+</sup> events. This large calcium signal reaching the cell soma and triggering whole-cell Ca2<sup>+</sup> signaling can stimulate the release of gliotransmitter to multiple synapses coordinating activity of the neuronal circuit (**Figure 8A**) defined by the morphological territory of individual astrocyte. Thus, astrocytemediated potentiation of presynaptic release results in longterm increasing of neuronal activity from domain defined by the astrocytic morphology (**Figure 8A**). This kind of response can even propagate to neighboring astrocytes, through gap junction channels. **Figure 8B** shows the simulation of the calcium signal propagation through the processes of one astrocyte to another triggering modulation of communication in large neuronal ensembles.

### DISCUSSION

The majority of known data is extracted from Ca2<sup>+</sup> signals monitoring in astrocyte soma. Slow Ca2<sup>+</sup> events in astrocyte used to be associated with the high level of neuronal activity (Pasti et al., 1997; Sul et al., 2004). Recent studies, indeed, revealed that even a minimal synaptic activity is capable of small, rapid, and localized Ca2<sup>+</sup> response excitation in astrocyte (Volterra et al., 2014; Bindocci et al., 2017). These data gave a ground to assume that astrocytes generate large calcium signal by integrating the activity of several individual synapses. Thus, the astrocyte Ca2<sup>+</sup> signaling represents self-coordinated spatio-temporal patterns including local fast responses as well as, respectively, slow global responses resulting from the integration of the signals. The integration can encapsulate the mechanism of the global responses control via local changes in neuronal activity.

Our model accounting for spatial morphology of the tripartite synapses revealed interesting functional features of calcium activity in astrocyte-mediated modulation of signal transmission. It was shown that astrocyte can act as temporal and spatial integrator, hence, detecting the level of spatio-temporal coherence in the activity of accompanying neuronal network.

### REFERENCES

Araque, A., Carmignoto, G., Haydon, P. G., Oliet, S. H., Robitaille, R., and Volterra, A. (2014). Gliotransmitters travel in time and space. Neuron 81, 728–739. doi: 10.1016/j.neuron.2014.02.007

Specifically, such time and space integration based on rapid and local events of activation of small compartments along the astrocytic processes results in the long-term astrocyte-mediated changes of the synaptic functionality of the neuronal network. Revealed by a correlation analysis of obtained numerical simulations, a presence of the synchrony between neuronal and astrocytic activity has verified that activation of the astrocyte is stimulated by neuronal activity, which is synchronous in time and in space.

In this study we show that different level of the neuronal activity can trigger Ca2<sup>+</sup> dynamics in astrocyte with various spatio-temporal characteristics which can lead to different astrocytic-induced regulatory effects on synaptic transmission. The minimal synaptic activity causes the fast and local Ca2<sup>+</sup> elevation in astrocytic process. This small Ca2<sup>+</sup> signal triggers the gliotransmission in the active synapse induces localized regulatory astrocytic feedback of the synapse (**Figure 6**). Increasing frequency of synaptic activity can produce Ca2<sup>+</sup> signal which can spread to another astrocytic process (**Figure 7**) and to the whole cell (**Figure 8**). Therefore, it can result in modulation of activity in neighboring synapses (**Figure 7**) and domain of synapses restricted by the territory of astrocytic morphology (**Figure 8**). In other words astrocyte can induce spatial synchronization in neuronal circuits defined by the morphological territory of the astrocyte. It is known that spatial synchronization in the brain is responsible for various cognitive functions (attention, recognition, navigation, making decisions, etc.) and for various pathologies (epileptic discharges, etc.).

## AUTHOR CONTRIBUTIONS

SG, AE, and VK: conceptualization. SG and AE: data curation. SG, AE, and IK: formal analysis, investigation, and software. SG and VK: supervision and writing original draft.

### FUNDING

This work was supported by The Russian Science Foundation (Grant No. 18-11-00294). Simulations for **Figure 8** was carried out with financial support of the Russian Foundation for Basic Research (Grant No. RFBR 18-29-10068).

### SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fphys. 2019.00294/full#supplementary-material

Araque, A., Parpura, V., Sanzgiri, R. P., and Haydon, P. G. (1999). Tripartite synapses: glia, the unacknowledged partner. Trends Neurosci. 22, 208–215. doi: 10.1016/S0166-2236(98)01349-6

Arizono, M., Bannai, H., Nakamura, K., Niwa, F., Enomoto, M., Matsu-ura, T., et al. (2012). Receptor-selective diffusion barrier enhances sensitivity of astrocytic processes to metabotropic glutamate receptorstimulation. Sci. Signal. 5:ra27. doi: 10.1126/scisignal.2002498


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2019 Gordleeva, Ermolaeva, Kastalskiy and Kazantsev. This is an openaccess article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

digital media

of impactful research

article's readership