# MATHEMATICS FOR HEALTHCARE

EDITED BY : Krasimira Tsaneva-Atanasova and Vanessa Diaz-Zuccarini PUBLISHED IN : Frontiers in Physiology

#### Frontiers Copyright Statement

© Copyright 2007-2018 Frontiers Media SA. All rights reserved. All content included on this site, such as text, graphics, logos, button icons, images, video/audio clips, downloads, data compilations and software, is the property of or is licensed to Frontiers Media SA ("Frontiers") or its licensees and/or subcontractors. The copyright in the text of individual articles is the property of their respective authors, subject to a license granted to Frontiers.

The compilation of articles constituting this e-book, wherever published, as well as the compilation of all other content on this site, is the exclusive property of Frontiers. For the conditions for downloading and copying of e-books from Frontiers' website, please see the Terms for Website Use. If purchasing Frontiers e-books from other websites or sources, the conditions of the website concerned apply.

Images and graphics not forming part of user-contributed materials may not be downloaded or copied without permission.

Individual articles may be downloaded and reproduced in accordance with the principles of the CC-BY licence subject to any copyright or other notices. They may not be re-sold as an e-book.

As author or other contributor you grant a CC-BY licence to others to reproduce your articles, including any graphics and third-party materials supplied by you, in accordance with the Conditions for Website Use and subject to any copyright notices which you include in connection with your articles and materials.

All copyright, and all rights therein, are protected by national and international copyright laws.

The above represents a summary only. For the full conditions see the Conditions for Authors and the Conditions for Website Use. ISSN 1664-8714 ISBN 978-2-88945-577-5 DOI 10.3389/978-2-88945-577-5

#### About Frontiers

Frontiers is more than just an open-access publisher of scholarly articles: it is a pioneering approach to the world of academia, radically improving the way scholarly research is managed. The grand vision of Frontiers is a world where all people have an equal opportunity to seek, share and generate knowledge. Frontiers provides immediate and permanent online open access to all its publications, but this alone is not enough to realize our grand goals.

#### Frontiers Journal Series

The Frontiers Journal Series is a multi-tier and interdisciplinary set of open-access, online journals, promising a paradigm shift from the current review, selection and dissemination processes in academic publishing. All Frontiers journals are driven by researchers for researchers; therefore, they constitute a service to the scholarly community. At the same time, the Frontiers Journal Series operates on a revolutionary invention, the tiered publishing system, initially addressing specific communities of scholars, and gradually climbing up to broader public understanding, thus serving the interests of the lay society, too.

#### Dedication to Quality

Each Frontiers article is a landmark of the highest quality, thanks to genuinely collaborative interactions between authors and review editors, who include some of the world's best academicians. Research must be certified by peers before entering a stream of knowledge that may eventually reach the public - and shape society; therefore, Frontiers only applies the most rigorous and unbiased reviews.

Frontiers revolutionizes research publishing by freely delivering the most outstanding research, evaluated with no bias from both the academic and social point of view. By applying the most advanced information technologies, Frontiers is catapulting scholarly publishing into a new generation.

#### What are Frontiers Research Topics?

Frontiers Research Topics are very popular trademarks of the Frontiers Journals Series: they are collections of at least ten articles, all centered on a particular subject. With their unique mix of varied contributions from Original Research to Review Articles, Frontiers Research Topics unify the most influential researchers, the latest key findings and historical advances in a hot research area! Find out more on how to host your own Frontiers Research Topic or contribute to one as an author by contacting the Frontiers Editorial Office: researchtopics@frontiersin.org

# MATHEMATICS FOR HEALTHCARE

Topic Editors:

Krasimira Tsaneva-Atanasova, University of Exeter, United Kingdom Vanessa Diaz-Zuccarini, University College London, United Kingdom

"Instantaneous velocity streamlines at mid-systole in a patient-specific model of a femoropopliteal bypass graft". Courtesy of: Francesca Donadoni, UCL. Image licensed under CC-BY.

In 1996, and with extraordinary prescience, Panfilov and Holden had highlighted in their seminal book 'Computational Biology of the Heart' that biology was, potentially, the most mathematical of all sciences. Fast-forward 20 years and we have seen an explosion of applications of mathematics in not only biology, but healthcare that has already produced significant breakthroughs not imaginable 20 years ago. Great strides have been made in explaining through quantitative methods the

underlying mechanisms of human disease, not without considerable ingenuity and effort. Biological mechanisms are bewildering: complex, ever evolving, multi-scale, variable, difficult to fully access and understand. This poses immense challenges to the computational physiology community that, nevertheless, has developed an impressive arsenal of tools and methods in a vertiginous race to combat disease with the tall order of improving human healthcare. Mechanistic models are now contending with the advent of machine learning in healthcare and the hope is that both approaches will be used synergistically since the complexity of human patophysiology and the difficulty of acquiring human datasets will require both, deductive and inductive methods. This Research Topic presents work that is currently at the frontier in computational physiology with a striking range of applications, from diabetes to graft failure and using a multitude of mathematical tools. This collection of articles represents a snapshot in a field that is moving at a dizzying speed, bringing understanding of fundamental mechanism and solutions to healthcare problems experienced by healthcare systems all over the world.

Citation: Tsaneva-Atanasova, K., Diaz-Zuccarini, V., eds (2018). Mathematics for Healthcare. Lausanne: Frontiers Media. doi: 10.3389/978-2-88945-577-5

# Table of Contents

*06 Editorial: Mathematics for Healthcare as Part of Computational Medicine* Krasimira Tsaneva-Atanasova and Vanessa Diaz-Zuccarini

# OPINION ARTICLES

*09 Patient Similarity: Emerging Concepts in Systems and Precision Medicine* Sherry-Ann Brown

# PERSPECTIVE ARTICLES

*15 Toward Precision Healthcare: Context and Mathematical Challenges* Caroline Colijn, Nick Jones, Iain G. Johnston, Sophia Yaliraki and Mauricio Barahona

#### METHODS ARTICLES

*25 Personalized Medication Response Prediction for Attention-Deficit Hyperactivity Disorder: Learning in the Model Space vs. Learning in the Data Space*

Hin K. Wong, Paul A. Tiffin, Michael J. Chappell, Thomas E. Nichols, Patrick R. Welsh, Orla M. Doyle, Boryana C. Lopez-Kolkovska, Sarah K. Inglis, David Coghill, Yuan Shen and Peter Tiño

*46 Systematic Characterization of Dynamic Parameters of Intracellular Calcium Signals*

Laurent Mackay, Nicholas Mikolajewicz, Svetlana V. Komarova and Anmar Khadra

*68 Cancer Markers Selection Using Network-Based Cox Regression: A Methodological and Computational Practice*

Antonella Iuliano, Annalisa Occhipinti, Claudia Angelini, Italia De Feis and Pietro Lió

#### ORIGINAL RESEARCH ARTICLES


Wenguang Li, Nigel C. Bird and Xiaoyu Luo

*141 Input Estimation for Extended-Release Formulations Exemplified With Exenatide*

Magnus Trägårdh, Michael J. Chappell, Johan E. Palm, Neil D. Evans, David L. I. Janzén and imagePeter Gennemark

*153 Modeling Disease Progression: Angiotensin II Indirectly Inhibits Nitric Oxide Production via ADMA Accumulation in Spontaneously Hypertensive Rats*

Haidong Wang, Hao Jiang, Haochen Liu, Xue Zhang, Guimei Ran, Hua He and Xiaoquan Liu

*166 Development of a Patient-Specific Multi-Scale Model to Understand Atherosclerosis and Calcification Locations: Comparison With* In vivo *Data in an Aortic Dissection*

Mona Alimohammadi, Cesar Pichardo-Almarza, Obiekezie Agu and Vanessa Díaz-Zuccarini

*181 Signal Reconstruction of Pulmonary Vein Recordings Using a Phenomenological Mathematical Model: Application to Pulmonary Vein Isolation Therapy*

Harry D. Green, Glyn Thomas and John R. Terry


Elisa Domínguez-Hüttinger, Neville J. Boon, Thomas B. Clarke and Reiko J. Tanaka


Francesca Donadoni, Cesar Pichardo-Almarza, Matthew Bartlett, Alan Dardik, Shervanthi Homer-Vanniasinkam and Vanessa Díaz-Zuccarini

# Editorial: Mathematics for Healthcare as Part of Computational Medicine

#### Krasimira Tsaneva-Atanasova1,2 \* and Vanessa Diaz-Zuccarini <sup>3</sup>

*<sup>1</sup> Department of Mathematics and Living Systems Institute, University of Exeter, Exeter, United Kingdom, <sup>2</sup> EPSRC Centre for Predictive Modelling in Healthcare, University of Exeter, Exeter, United Kingdom, <sup>3</sup> Multiscale Cardiovascular Engineering Group, Department of Mechanical Engineering, University College London, London, United Kingdom*

Keywords: mathematical modeling, computer simulation, precision medicine, patient specific modeling, digital health

**Editorial on the Research Topic**

#### **Mathematics for Healthcare as Part of Computational Medicine**

#### Edited by:

*Raimond L. Winslow, Johns Hopkins University, United States*

#### Reviewed by:

*Jason Papin, University of Virginia, United States Henggui Zhang, University of Manchester, United Kingdom*

#### \*Correspondence:

*Krasimira Tsaneva-Atanasova k.tsaneva-atanasova@exeter.ac.uk*

#### Specialty section:

*This article was submitted to Computational Physiology and Medicine, a section of the journal Frontiers in Physiology*

> Received: *14 April 2018* Accepted: *04 July 2018* Published: *24 July 2018*

#### Citation:

*Tsaneva-Atanasova K and Diaz-Zuccarini V (2018) Editorial: Mathematics for Healthcare as Part of Computational Medicine. Front. Physiol. 9:985. doi: 10.3389/fphys.2018.00985* Appropriate mathematical tools and methodologies are critical for ensuring robust and reliable computational model predictions based on medical and healthcare data in the era of the digital health revolution (Duggal et al., 2018). Patient-specific approaches are being increasingly pursued, with simulations benchmarked by clinical data (e.g., brain activity recordings; Breakspear, 2017) obtained in non-invasive manner on individual level (e.g., resting state; Spetsieris et al., 2015). Precision Medicine, although not a new concept, is gaining momentum (Hodson, 2016) powered by the ever increasing volume of patients data (Colijn et al.). Quantifying patient similarity is an important challenge that is critical in predicting patients' disease trajectories (Sharafoddini et al., 2017). In an opinion article (Brown) patient similarity concept has been introduced as a paradigm shift in optimizing personalisation of patient care.

Applications of mathematics in healthcare are achieving unprecedented growth at vertiginous speed in a vast number of areas. Mental health presents a formidable challenge in our modern society and computational psychiatry has recently emerged (Huys et al., 2016) as a field combing computational models and patients' data in an attempt to enhance the prognosis, diagnosis and treatment of mental health conditions. This special issue offers an example (Wong et al.) of statistical learning in the model space for Attention-Deficit Hyperactivity Disorder medication response prediction at individual patient level. Mental health is intimately related to neurological diseases and brain modeling for neurological disease treatments (Rubin, 2017) has found applications to setting deep-brain stimulation parameters in Parkinson's disease treatment (Mandali et al.).

Non-negligible effort is currently being devoted to capturing disease progression, a real challenge in this field. Disease progression modeling involves simulations of disease evolution based on available biomarkers or other time-dependent measures of disease status. This is particularly important in the case of chronic (non-communicable) diseases as demonstrated in this special issue in the case of hypertension (Wang et al.) as well as type 1 diabetes (Wedgwood et al.). Chronic diseases are often extremely complex leading to computational models formulated in high-dimensional space, which poses a challenge for characterizing the pathways

**6**

of disease progression or patient-specific disease progression trajectories as discussed in Colijn et al. Computational cardiology encompasses mathematical modeling and computer simulation of dynamical processes in the heart and the cardiovascular system in health and disease (Trayanova et al., 2012). An example of an application of computational cardiology is the use of subjectspecific computer models to predict neointimal hyperplasia in vein grafts (Donadoni et al.). Computational cardiology applications could help in improving clinical decision support systems in cardiac ablation therapy for example (Green et al.). Moreover, the use of collections of patient-specific models could provide a tool for pre-clinical and clinical assessment of disease pathology such as atherosclerosis and associated calcification (Alimohammadi et al.).

Computational oncology broadly refers to computational modeling and simulations of biological process underlying tumors' development and progression as well as cancer therapy including cancer biomarkers and drug effects (Barbolosi et al., 2016). An example of computational modeling of signaling pathways involved in multiple myeloma is presented in Kendrick et al. whereas (Iuliano et al.) presents a networkbased statistical methodology for cancer biomarker selection. Computational and mathematical pharmacology is becoming increasingly relevant for drug development. Mechanistic models has become more and more widely used and our understanding of the models' qualitative and quantitative behavior has improved (Krzyzanski and van Hasselt, 2018). There are outstanding challenges, however, associated with parameter identifiability of pharmacodynamics models (Janzén et al.) as well as the estimation of drug absorption profiles in-vivo (Trägårdh et al.).

Robust quantitative methods for identifying biologically/physiologically relevant computational model parameters from experimental data are critical for the successful applications of computational medicine in precision healthcare (Colijn et al.). This special issue presents several examples of such methodological developments in the case of quantifying: the biomechanical properties of human gallbladder (Li et al.); the forces involved in abrasion damage to skin (Jayawardana et al.); and intracellular calcium signals (Mackay et al.).

#### REFERENCES


Infectious disease modeling including the underlying mechanisms is becoming increasingly important in the face of the anti-microbial resistance and its associated clinical and public health burden (de Kraker et al., 2016). Grasping the complexity of host-pathogen interactions remains a challenge and mathematical modeling and analysis could help designing appropriate disease management strategies at patient-specific level (Domínguez-Hüttinger et al.) that are necessary for implementation of precision healthcare (Colijn et al.) as well as to inform public policies related to vaccination, for example see (Hamami et al.).

By no means the topics included in this special issue are exhaustive. They are rather indicative of a wider range of problems specific to computational medicine that not only can be tackled by available mathematical approaches but also inspire the development of novel tools and techniques. Examples of methods that have not been included are machine learning and artificial intelligence for electronic health records analysis and usage (Callahan and Shah, 2018). We hope that future Frontiers Research Topics will contain an increasing number of contributions within the scope of mathematics for healthcare as part of computational medicine.

#### AUTHOR CONTRIBUTIONS

KT-A and VD-Z conceived and wrote the editorial.

#### FUNDING

01/01/2016–31/12/2019 EP/N014391/1 EPSRC Centre for Predictive Modeling in Healthcare at the University of Exeter (£2,008,955).

#### ACKNOWLEDGMENTS

We would like to thank all reviewers, particularly those of interdisciplinary background, for their constructive comments in an attempt to initiate a dialogue between theoreticians and practitioners in the articles of this special issue. KT-A gratefully acknowledges the financial support of the EPSRC via grant EP/N014391/1.

Duggal, R., Brindle, I., and Bagenal, J. (2018). Digital healthcare: regulating the revolution. Br. Med. J. 2018:360. doi: 10.1136/bmj.k6


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Tsaneva-Atanasova and Diaz-Zuccarini. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Patient Similarity: Emerging Concepts in Systems and Precision Medicine

#### Sherry-Ann Brown\*

*Department of Cardiovascular Diseases, Mayo Clinic, Rochester, MN, USA*

Keywords: patient similarity, patient similarity analytics, computational medicine, big data analytics, clinical decision support

# INTRODUCTION

Healthcare data generates a huge volume of information in various formats at high velocity with sometimes questionable veracity (Barkhordari and Niamanesh, 2015) (4V). As a result, big data tools such as patient similarity are necessary to facilitate analytics, which reduces costs (Srinivasan and Arunasalam, 2013) and improves healthcare systems (Jee and Kim, 2013). Patient similarity investigates distances between a variety of components of patient data, and determines methods of clustering patients, based on short distances between some of their characteristics. Although patient similarity is in its early stages, ultimately information about diseases, risk factors, lifestyle habits, medication use, co-morbidities, molecular and histopathological information, hospitalizations, or death are compared with laboratory investigations, imaging, and other clinical data assessing medical evidence of human behavior (**Figure 1**). Such analytics consist of efficient computational analyses with patient stratification by multiple co-occurrence statistics, based on clinical characteristics. Algorithms create subgroups of patients based on similarities among their electronic avatars. Among electronic avatars found to be similar, subgroups of patients can be evaluated by further stratification guided by individual diagnoses, risk factors, medications, and so on. Because of the multiple networks of subgroups of patients, patient similarity can be considered an application of network medicine, with the output termed "patient similarity networks." Thus, data mining extracts clinically relevant information hidden in clinical notes and embedded in other areas of the electronic health record (EHR) coupled with International Classification of Disease codes. The result is a systematic individualized analysis of a subset of patients that can improve outcome prediction and help guide management for a particular patient currently being cared for by a clinician (Lee et al., 2015). The communication or output from the algorithms can be used to identify and predict disease correlations and occurrence, and potentially for clinical decision support at the point of care. Patient similarity analytics are not restricted to global findings from large clinical trials consisting of somewhat heterogeneous patient populations (Roque et al., 2011). In this way, patient similarity represents a paradigm shift that introduces disruptive innovation to optimize personalization of patient care. Some promising examples are regarding mental and behavioral disorders (Roque et al., 2011), infectious diseases (Li et al., 2015), cancers (Wu et al., 2005; Teng et al., 2007; Chan et al., 2010, 2015; Klenk et al., 2010; Cho and Przytycka, 2013; Li et al., 2015; Wang, 2015; Bolouri et al., 2016; Wang et al., 2016), endocrine (Li et al., 2015; Wang, 2015), and metabolic diseases (Zhang et al., 2014; Ng et al., 2015). Others involve diseases of the nervous system (Lieberman et al., 2005; Carreiro et al., 2013; Cho and Przytycka, 2013; Qian et al., 2014; Buske et al., 2015a; Li et al., 2015; Bolouri et al., 2016; Wang et al., 2016), eyes (Buske et al., 2015a; Li et al., 2015), skin (Buske et al., 2015a; Li et al., 2015), heart (Wu et al., 2005; Tsymbal et al., 2007; Syed and Guttag, 2011; Buske et al., 2015a; Li et al., 2015; Panahiazar et al., 2015a,b; Wang, 2015; Björnson et al., 2016), liver (Chan et al., 2015), intestines (Buske et al., 2015a), musculoskeletal system

#### Edited by:

*Krasimira Tsaneva-Atanasova, University of Exeter, UK*

#### Reviewed by:

*Pietro Lio, University of Cambridge, UK Gustavo Glusman, Institute for Systems Biology, USA*

> \*Correspondence: *Sherry-Ann Brown brown.sherryann@mayo.edu*

#### Specialty section:

*This article was submitted to Computational Physiology and Medicine, a section of the journal Frontiers in Physiology*

Received: *31 August 2016* Accepted: *07 November 2016* Published: *24 November 2016*

#### Citation:

*Brown S-A (2016) Patient Similarity: Emerging Concepts in Systems and Precision Medicine. Front. Physiol. 7:561. doi: 10.3389/fphys.2016.00561*

**9**

(Buske et al., 2015a), congenital malformations (Buske et al., 2015a), and various other conditions or factors influencing health status (Gotz et al., 2012; Subirats et al., 2012; Ng et al., 2015).

# PATIENT SIMILARITY IN SYSTEMS MEDICINE

Patient similarity is just starting to spread its wings and has the potential to transform Systems Medicine, which is Systems Biology applied to health care. Systems Biology studies the characteristics of cells, tissues, organisms, or other comprehensive biological units as whole systems. Systems Biology seeks to determine how changes in one part of the system can affect the behavior of the whole system, and often focuses on predictive modeling of the system in a perturbed state. Patient similarity analytics could be developed to bring together characteristics of the patient as a whole human system, and compare these to a multitude of similar patients. Accordingly, patient similarity analytics should in the near future incorporate genomics, transcriptomics, proteomics, microbiomics, and other "omics" and diverse components of systems medicine. In addition, simulation of physiology at the level of the molecule, cell, tissue, organ, and organism should be consolidated as a comprehensive similarity feature to give a broader view of interactions among organ systems. Patient similarity analytics could provide predictive models of a patient's outcome in the setting of disease perturbations or diagnoses relevant to the index patient. Making adjustments in the query data that serve as input for the predictive models would allow for assessment of how new diagnoses or therapies could impact the overall behavior and phenotype of the whole patient.

Beyond the reasoning above, integrating the majority of these systems medicine tools into patient similarity analytics is potentially the next frontier in Systems Medicine, for at least a few reasons. First, patient similarity analytics embrace a systems view by assessing a myriad of characteristics for hundreds or thousands of patients to produce a meaningful and useful result. Second, patient similarity analytics are analogous to various "omics" that in part compose Systems Biology. Just as transcriptomics refers to generation of messenger RNA expression profiles (Briefing, 1999), one could consider a term similaromics referring to generation or identification of patients similar to an index patient. Similaromics is also akin to phenomics, proteomics, and genomics, among others. Phenomics refers to cataloging the observable characteristics conferred by a gene and proteomics describes the generation of proteins expressed by a cell (Briefing, 1999). One might argue that patient similarity is not quite analogous to genomics, since an individual's genome is thought to be constant throughout their lifetime. However, this is no longer necessarily the case, due to the current progress of genome editing tools. Indeed, patient similarity is analogous to these various omics, all with the potential to change over the lifetime of the individual. Thus, just as a genome is the complement of all DNA within a cell, a similarome is the complement of patients found to be similar to an index patient. Within the similarome, one can further

comorbidities/medications, etc.). Step 6 involves data collating and information retrieval. In step 7, the similarome (which includes subsimilaromes) is presented to the patient-clinician partnership via the clinical decision support tool interface for clinical decision-making at the point-of-care. C, Clinical information; G, Genomics; O, Other systems medicine data types or tools; P, Proteomics; S, Social network data; T, Transcriptomics.

distinguish subgroups of patients that are most similar to an index patient, based on preferentially assigning preeminence to comorbidities or medications of most interest or relevance to the index patient, e.g., during a focused shared decision-making session with a clinician. Similar to genotyping then, which determines the presence or absence of a particular gene feature, simotyping would allocate the presence or absence of a particular similarity feature, for example, a diagnosis of diabetes. In this context then, a similarity-wide association study (SiWAS) has the goal of discovering clusters of patients similar to an index patient and identifying similar features that associate with specific outcomes, such as complications, procedures, hospitalizations, or death. For example, investigating whether in patients most similar to an index patient diabetes is more likely to associate with non-healing leg ulcers, critical limb ischemia, or gangrene leading to limb amputation.

Third, patient similarity analytics have the potential to bring together a variety of omics and other systems medicine tools, if we can do so in a way that is effective, accurate, consistent, and computationally efficient (Brown, 2015a). Indeed, several groups have proposed methods of aggregating omics and monitoring these over time for individual patients, and perhaps even using comprehensive patient avatars. Integrating these methods with patient similarity has the potential to launch systems medicine further into a future where medicine is even more precisely individualized. Patient similarity will likely become and persist as a useful tool in systems medicine.

# MATHEMATICS IN PATIENT SIMILARITY ANALYTICS

For illustration of the utility of patient similarity in medicine, only briefly presented here are a few selected examples of patient similarity analytics used for diabetes and cancer, which are common chronic or terminal diseases, respectively, currently addressed in public health. In some studies, a patient similarity metric is determined as follows (Lee et al., 2015; Li et al., 2015). A patient can be represented by a Euclidean vector. Predictor variables such as laboratory test results or vital signs can define a multi-dimensional feature space. The cosine of the angle between two patients' vectors can define the associated patient similarity metric. A dot product can facilitate the calculation. This can be termed the "cosine similarity," defining the patient similarity metric as follows:

$$\begin{aligned} PSM(P\_1, P\_2) &= \frac{P\_1 \cdot P\_2}{||P\_1|| \, ||P\_2||}, \\ &= \frac{\sum\_{i=1}^n P\_{1i} \times P\_{2i}}{\sqrt{\sum\_{i=1}^n P\_{1i}^2} \times \sqrt{\sum\_{i=1}^n P\_{2i}^2}} \end{aligned}$$

where P1<sup>i</sup> and P2<sup>i</sup> represent a single predictor variable vector for two separate patients, · represents the dot product, and || || represents the Euclidean vector magnitude, as shown. Since the patient similarity metric is an angle cosine, it normalizes between −1 (considered minimum possible similarity) and 1 (considered maximum possible similarity). As expected, two predictor variable vectors pointing in the exact opposite direction to each other would have a 180◦ angle between them, and would therefore calculate to a patient similarity metric of −1. Conversely, two perfectly overlapping vectors would have an angle of 0◦ between them, and would therefore calculate to a patient similarity metric of 1. Accordingly, before calculating the total patient similarity metric, the product for each predictor variable vector would be normalized to the range of −1 to 1 in the multidimensional feature space, if continuous (Lee et al., 2015). The product for categorical/binary predictor variable vectors would be assigned a value of −1 or 1. The patient similarity metric would be calculated for each patient in a given data set, relative to an index patient P1. The N most similar patients to the index patient would be utilized as a training data set for testing in a validation data set, with prediction of prognosis, morbidity, or mortality. After successful validation, the predictive model could be used for epidemiologic or clinical studies. For example, am algorithm using cosine similarity successfully identified three subgroups of patients with diabetes (Li et al., 2015). The first subgroup included patients with diabetic nephropathy (diabetesrelated kidney disease) and diabetic retinopathy (diabetesrelated eye disease). The second subgroup included several patients with cancer and cardiovascular diseases. The third subgroup included many patients who also had cardiovascular diseases, along with neurological diseases, allergies, and HIV infection. Various single nucleotide polymorphisms mapped to these three subgroups that were confirmed in the EHR, suggesting clinical relevance for patient similarity in precision medicine. Jaccard similarity, another metric that can be leveraged after assigning binary attributes to each patient's multifeature vector space, was useful to analyze features underlying deviant responses to therapeutics in patients with diabetes (Zhang et al., 2014).

Alternatively, unsupervised clustering of patients based on their clinical predictor variables could be used to produce a patient-patient network. The network could be organized using L-infinity centrality, which is the maximum distance from each point from any other point in a given data set. L-infinity centrality produces a detailed and succinct description of any data set yielding more information than scatter plots (Lum et al., 2013). Large values for L-infinity centrality correspond to data points at large distances from the center of the data set (Li et al., 2015). Other pattern analysis and cluster algorithms (Daemen and De Moor, 2009; Chan et al., 2010; Liu et al., 2013a; Mabotuwana et al., 2013; Sundar et al., 2014), or algorithms incorporating distance metric learning (Wang et al., 2011; Bian and Tao, 2012), locally supervised metric learning (Sun et al., 2012; Ng et al., 2015), local spline regression (Wang et al., 2012), or visual analytics (Tsymbal et al., 2009; Ebadollahi et al., 2010; Gotz et al., 2011; Perer, 2012; Heer and Perer, 2014; Bolouri et al., 2016; Ozery-Flato et al., 2016), can also be used for patient similarity to predict diabetes onset, develop treatment recommendations tailored to each patient, or predict survival after chemotherapy (Chan et al., 2010; Liu et al., 2013a; Ng et al., 2015; Ozery-Flato et al., 2016), among other applications. SNOMED CT and other medical terminology frameworks can be used to facilitate communication across platforms in various studies (Melton et al., 2006). There are also algorithms to incorporate a time series into patient similarity analysis, to predict trends over time among patients (Wu et al., 2005; Hartge et al., 2006; Ebadollahi et al., 2010; Carreiro et al., 2013; Alaa et al., 2016). For example, a patient similarity time series algorithm has been used to fine-tune radiation treatment planning for patients with head and neck cancers (Wu et al., 2005).

# CHALLENGES IN PATIENT SIMILARITY

There are certain challenges in patient similarity, such as network bottlenecks, low hardware performance (processing power and memory), and data locality (Osman et al., 2013; Karapiperis and Verykios, 2014; Barkhordari and Niamanesh, 2015). Given the observational or retrospective nature of patient similarity, interpretation of data analysis will be imperfect. Confounder control and treatment selection bias are inherent limitations in such studies. However, groups have developed strategies to manage the potential for confounders, such as restriction, stratification, matching, inverse probability weighting, and covariate adjustment (Gallego et al., 2015). Several groups have also proposed solutions for other challenges that enable large scale patient indexing and accurate and efficient clinical data retrieval (Wang, 2015). Some have devised algorithms to address the complexity of clinical data and limited transparency of many existing clinical case retrieval decision support systems (Tsymbal et al., 2009), as well as integration of data from various heterogeneous omics studies (Wang et al., 2014, 2016; Gligorijevic´ et al., 2016) and physician input and feedback (Wang et al., 2011; Sun et al., 2012; Fei and Sun, 2015). Others have produced algorithms that address scalability and uncertainty, by requiring parallel or distributed algorithm implementations built to scale, and enhancing interpretability by conveying the certainty of results presented (Feldman et al., 2015). One such algorithm or platform is scalable and distributable patient similarity (ScaDiPaSi), a dynamic method for investigating patient similarity that spreads the algorithm over several selfsufficient hardware nodes to process query data from various sources of different formats simultaneously (Barkhordari and Niamanesh, 2015). Another tool, MapReduce, employs several optimization techniques, such as job scheduling and cascading work flows over multiple interdependent hardware nodes (Dean and Ghemawat, 2008). Use of all of these technological solutions for patient similarity in precision medicine will be facilitated by bridging gaps among different scientific, technological, and medical cultures, through interdisciplinary collaborations among experts in medicine, biology, informatics, engineering, public health, economics, and the social sciences (Kuhn et al., 2008).

# REFERENCES


# CONCLUSION

Various patient similarity algorithms have been deployed and have been found beneficial by improving clinical efficiency (Wang et al., 2015), enabling secure identification of similar patients and records sharing by clinicians and rare disease scientists (Buske et al., 2015a,b), predicting patients' prognosis or trajectory over time (Ebadollahi et al., 2010; Subirats et al., 2012; Wang et al., 2012; Gallego et al., 2015), providing clinical decision support (Daemen et al., 2009; Wang et al., 2011; Subirats et al., 2012; Sun et al., 2012; Gottlieb et al., 2013; Liu et al., 2013b; Gallego et al., 2015), tailoring individual treatments (Zhang et al., 2014), preventing unexpected adverse drug reactions (Hartge et al., 2006; Yang et al., 2014), flagging patients deserving more attention due to poor response to therapies (Zhang et al., 2014; Ozery-Flato et al., 2016), and pursuing comparative effectiveness studies (Wang et al., 2011), among other applications. In general, clinical guidelines often do not supply evidence on risks, secondary therapy effects, and long-term outcomes (Gallego et al., 2015). In this setting, patient similarity analytics can provide a cheaper, portable alternative or in fact adjunct to evidence-based clinical guidelines and randomized controlled trials, particularly if trial data are unavailable for conditions or patient characteristics specific to a query individual (Longhurst et al., 2014; Gallego et al., 2015). Synthesizing current patient similarity algorithms with systems medicine tools could provide actionable insights in precision medicine.

# AUTHOR CONTRIBUTIONS

SB conceived, analyzed, designed, drafted, critically revised, approved, and agreed to be accountable for this submitted work.

# ACKNOWLEDGMENTS

The author is grateful to Dr. Joerg Herrmann of Mayo Clinic in Rochester, Minnesota for reading the manuscript.


Carreiro, A., Madeira, S., and Francisco, A. (2013). "Unravelling communities of ALS patients using network mining," in KDD-DMH'13 (Chicago, IL).


Identification of Cancer Subtypes. IEEE Transact. Nanobiosci. 15, 335–342. doi: 10.1109/TNB.2016.2556640


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2016 Brown. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Toward Precision Healthcare: Context and Mathematical Challenges

Caroline Colijn1, 2 \*, Nick Jones 1, 2, Iain G. Johnston2, 3, Sophia Yaliraki 2, 4 and Mauricio Barahona1, 2 \*

*<sup>1</sup> Department of Mathematics, Imperial College London, London, UK, <sup>2</sup> EPSRC Centre for Mathematics of Precision Healthcare, Imperial College London, London, UK, <sup>3</sup> School of Biosciences, University of Birmingham, Birmingham, UK, <sup>4</sup> Department of Chemistry, Imperial College London, London, UK*

Precision medicine refers to the idea of delivering the right treatment to the right patient at the right time, usually with a focus on a data-centered approach to this task. In this perspective piece, we use the term "precision healthcare" to describe the development of precision approaches that bridge from the individual to the population, taking advantage of individual-level data, but also taking the social context into account. These problems give rise to a broad spectrum of technical, scientific, policy, ethical and social challenges, and new mathematical techniques will be required to meet them. To ensure that the science underpinning "precision" is robust, interpretable and well-suited to meet the policy, ethical and social questions that such approaches raise, the mathematical methods for data analysis should be transparent, robust, and able to adapt to errors and uncertainties. In particular, precision methodologies should capture the complexity of data, yet produce tractable descriptions at the relevant resolution while preserving intelligibility and traceability, so that they can be used by practitioners to aid decision-making. Through several case studies in this domain of precision healthcare, we argue that this vision requires the development of new mathematical frameworks, both in modeling and in data analysis and interpretation.

#### Edited by:

*Krasimira Tsaneva-Atanasova, University of Exeter, UK*

#### Reviewed by:

*Biswapriya Biswavas Misra, Texas Biomedical Research Institute, USA Marcus Kaiser, Newcastle University, UK*

#### \*Correspondence:

*Caroline Colijn c.colijn@imperial.ac.uk Mauricio Barahona m.barahona@imperial.ac.uk*

#### Specialty section:

*This article was submitted to Computational Physiology and Medicine, a section of the journal Frontiers in Physiology*

Received: *01 December 2016* Accepted: *22 February 2017* Published: *21 March 2017*

#### Citation:

*Colijn C, Jones N, Johnston IG, Yaliraki S and Barahona M (2017) Toward Precision Healthcare: Context and Mathematical Challenges. Front. Physiol. 8:136. doi: 10.3389/fphys.2017.00136* Keywords: precision medicine, precision healthcare, data science, precision public health, mathematical modeling

# INTRODUCTION: PRECISION MEDICINE AND ITS CHALLENGES

The phrase "precision medicine" describes the idea of delivering the right treatment to the right person at the right time. Precision approaches aim to achieve a medical revolution: individualized therapies based on quantitative, patient-specific datasets, integrated via algorithmic analyses that can aid patient stratification, monitoring, and treatment design. These approaches have been broadly supported in the US under President Obama's Precision Medicine Initiative (Collins and Varmus, 2015; White House Precision Medicine Initiative, 2016), by the Gates Foundation (Cisneros, 2016), and by the Chan Zuckerberg Initiative (Chan Zuckerberg Initiative — Advancing human potential and promoting equal opportunity, 2017). Although some of the aspects of this vision date back to the inception of the Human Genome Project, precision medicine now expands beyond the restrictions of genomics to encompass a wide range of data sources increasingly available to clinicians. The idea of embedding diagnostics and treatment with omics and other medical and physiological datasets at the heart of medicine has been variously described as systems medicine, personalized medicine, computational systems biomedicine, P4 (Predictive, Preventative, Personalized, Participatory) medicine, and

precision medicine, to name just a few (Duffy, 2016). The development of the field has been underpinned by some striking successes, particularly in cancer (Derks et al., 2014; Hiley et al., 2014; Arnedos et al., 2015; Friedman et al., 2015; Navin, 2015; Rubin, 2015; Stover and Wagle, 2015; Wills and Mead, 2015; Cloney, 2017), where molecular profiling is increasingly routine in lung, breast, and colorectal cancers, as well as in leukemias and melanomas (Larry Jameson and Longo, 2015). In asthma, the heterogeneity in clinical response has been shown to overlap with differences in a number of predictive biomarkers, allowing patient stratification for tailored therapies (Muraro et al., 2016). Precision approaches can have immediate benefits for drug repurposing and treatment: the link between type 2 diabetes and early stage Alzheimer's, in which there is often impaired glucose metabolism in the brain, is giving rise to a body of research for new therapeutics that includes repurposing existing drugs (Yarchoan and Arnold, 2014). Cardiovascular disease is another natural domain for precision medicine, as chronic, pervasive problems like diabetes, obesity, and hypertension (with a significant socio-economic and life style component) are directly linked to severe disease including heart failure (Antman and Loscalzo, 2016). These highly prevalent conditions are themselves diverse, multifactorial, and co-occurrent in many individuals, yet mechanism-based markers that predict the development of hypertension can already be identified based on functional genetic and epi-genetic markers (El Shamieh and Visvikis-Siest, 2012; Zhang et al., 2015). In the domain of infectious diseases, precision technologies can also be used to identify pathogens and to determine susceptibility to antimicrobial agents, guiding prescription, e.g., CD4+ cell counts and viral loads can guide HIV therapies (Barnett et al., 2008). Beyond single infections, the function of the microbiome is being probed for disease associations (Gilbert et al., 2016) and metabolomics and integrated omics' tools are revealing disease phenotypes (Chen et al., 2012; Dorrestein et al., 2014).

While there are plenty of potential "low hanging fruits" yet to be plucked, for precision medicine to maximize its impact as envisioned, a number of significant challenges need to be met across multiple domains. Some of these challenges are technical and relate to data collection, processing, storing, and sharing (Garber and Tunis, 2009; Servant et al., 2014; Palmisano et al., 2016; Sboner and Elemento, 2016), and have broad scientific, clinical, social, and ethical ramifications (Juengst et al., 2012; Khoury et al., 2012; Castaneda et al., 2015; Schork, 2015; Cohn et al., 2016). Indeed, advances in sequencing, metabolomics, biomarker discovery, genetics and single-cell technologies, alongside computing, and data science, have brought a strong impetus to the development of the scientific toolkit, data management systems, and regulatory framework for precision medicine. Data collection is currently taking place across the traditional channels of hospitals, community health care settings, and public health bodies, but also increasingly in a decentralized manner via social media analytics and wearable devices. The adoption of systematic formats for Electronic Health Records has improved data collection and consistency, but a considerable effort in data processing and integration still needs to take place (Garber and Tunis, 2009; Servant et al., 2014; Palmisano et al., 2016; Sboner and Elemento, 2016). Storing and accessing extremely high volumes of data is difficult and a concerted effort must be developed to enable clinicians, policy-makers, and academics to access these datasets, thus reducing the need for custom bioinformatics expertise. The question of whether data management is done by public or private organizations, and whether researchers and other users will need to pay to use data is an additional area of concern. Further, efforts to harness large datasets will require the development of sophisticated graphical user interfaces and visualization, data quality management, and data storage (Duffy, 2016). Programmes like the UK Biobank (2016), an open resource collecting de-identified data on health and well-being from 500,000 volunteers, and making it available for research, will be instrumental in meeting these challenges. There is also the issue of obtaining informed consent about the storage and use of data, when the uses are dynamic and expanding (Khoury et al., 2016). Finally, as yet there are no centralized resources collecting datasets, modeling and software analysis tools, and pipelines for precision medicine, which would facilitate method-sharing and allow interested researchers to join the effort.

Precision approaches, as they develop, must also accommodate the ethical and transparent use of data. Recently, O'Neil has coined the phrase "Weapons of Math Destruction" (WMD) (O'Neil, 2016) to describe how black-box algorithms can create pernicious and damaging feedback loops, with unfair consequences to individuals, often without much effort placed on identifying and correcting errors (O'Neil, 2016). Hood and Friend (2011) present the vision that "in the not-too-distant future, each patient will be surrounded by a 'virtual cloud' of billions of data points that will uniquely define their past medical history and current health status. Furthermore, it will be possible to mine the billions of data points from hundreds of millions of individuals to generate algorithms to help predict the future clinical needs for each patient." Hence, although precision medicine under this broad vision could have the beneficial potential to identify diseases earlier, to reduce burdens of treatment, and to improve screening by reducing false positives, and ultimately improve health, a sceptic might imagine a scenario in which these same predictions are used to produce quality-adjusted life year estimates, affecting which treatments are covered for whom, and guiding hiring, lending or health insurance decisions (O'Neil, 2016). With the amount and breadth of data available, there is the danger that such decisions could yield negative discrimination according to e.g., postcode lotteries, socio-economic factors, social network data, past healthcare interactions, judicial and law enforcement history.

How can such scenarios be avoided? Part of the answer must come from policy and regulation to ensure openness and fair use of data (Noveck, 2015). Yet, in addition, we need to develop the kind of mathematics and statistics for data science that will keep the "human in the loop" so that decision-making can be transparent and based on interpretable features and evidence. In doing so, we will need to develop methods that can track back and be updated in response to errors, taking full account of uncertainties, thus avoiding the over-reliance on complex computational decision black boxes. With this aim of model intelligibility, an important goal is to extract improved data-driven descriptions at the appropriate intermediate scales between the fully individualized level, which carries the risk of non-transparent and damaging over-use of data, and descriptions that are too coarse, which lead to insufficient precision in the face of individual variation. There is virtue in reaching a nuanced, data-informed middle-ground between these extremes: one that considers the individual in a population context and includes the role of human judgment.

It is thus essential to build theoretical understanding at the appropriate scale. One of the advantages of an integrated precision approach in medicine is to refine disease classification, increasing and finessing the number of groups of patients to reflect the true diversity of major diseases like cancers, so as to target treatment appropriately. While biomarker information can substantially improve clinical trial design as well as treatment (Trusheim et al., 2011), stratification also greatly increases the number of categories for which clinical trials may need to be carried out, reducing reproducibility and posing challenges to evidential policy (Khoury et al., 2012). Conversely, datacentric approaches may identify common mechanisms and treatments across disparate diseases, reducing stratification. Such approaches could potentially deliver dramatic cost efficiency. For these reasons, precision approaches must act at the right scale, which will often be intermediate–between "one size fits all" medicine and fully individualized therapies.

However, we do not yet have the ideal tools at hand to identify relevant features and integrate them to obtain interpretable predictions, optimized therapies, and new policies– even if merged datasets describing multivariate aspects of individuals' health across time (including, e.g., genomic, proteomic, metabolomic, brain images, social, and behavioral data) collected with informed consent were available. In addition, such combined genetic, genomic, proteomic, metabolomic, or single-cell data will only provide highly enriched and noisy snapshots taken at a few times—at best, we will have sparse noisy samples of the underlying process of disease, and sparse samples of the context of each patient.

To reveal the potential of such datasets in medicine, we must thus develop mathematical frameworks that are able to describe high-dimensional, dynamic, noisy, sparsely-sampled processes. Ideally, we must then be able to extract concise descriptions (coarse-grained at the right resolution) which are intelligible and actionable, and which link co-occurrences of events, comorbidities, and time patterns in disease and in health-related processes. This area poses a set of core mathematical challenges: creating transparent, replicable descriptions in healthcare, which make use of large diverse datasets, placing individuals in context, and which use dynamical information across time at the correct scales. These mathematical challenges must be researched in parallel with precision medicine, ideally spanning the individualand population-level perspectives.

In our view, these constitute deep additional challenges to mathematical modeling and data analysis that will need to be met in order for precision approaches to meet their promise. In the remainder of this perspective, we lay out a vision for what we term precision healthcare, its aims and its mathematical challenges. We do not aim to write a review of precision medicine; many reviews of tools and methods in different medical domains are available (see for example Chen and Snyder, 2013; Rosell and Karachaliou, 2013; Hiley et al., 2014; Ignatiadis and Dawson, 2014; Arnedos et al., 2015 among many others), as well as perspectives from a variety of viewpoints (Mirnezami et al., 2012; Roychowdhury and Chinnaiyan, 2013; Ciardiello et al., 2014; Ignatiadis and Dawson, 2014; Servant et al., 2014; Arnett and Claas, 2016; Rost et al., 2016; Vargas and Harris, 2016).

# Why Precision Healthcare?

For most of these challenges, population-level thinking coupled with mathematical data science analytics can help translate the benefits of precision medicine to address broader effects at the group level, including concerns regarding health equity and ethics. We use the phrase "precision healthcare" to encompass this vision that integrates the population and individual perspectives. Precision healthcare thus aims to build tools that make use of the increasing array of data sources, allowing for their continuous refinement in the face of new data, and whose predictions are aimed at and respond to the requirements of healthcare practitioners (clinicians, the public, policy thinkers, and other stakeholders).

This vision will require the use of an array of mathematical tools to unify individual-level precision medicine with public health, placing high-dimensional individual data and refined interventions in their social network context. Indeed, in many instances, individual health cannot be separated from its behavioral and social context. For example, highly targeted interventions against a cancer can be undermined by metabolic diseases caused by dietary behaviors which, in turn, co-vary with social network structure and other societal constructs. An adjuvant therapy for cancer might thus be to influence the diet and behavior of the patient taking into account their close social contacts.

The scenario by Hood and Friend (2011) mentioned above can thus be thought of as the analysis of a virtual cloud of a large number of high-dimensional feature vectors corresponding to the different individuals. Dynamical datasets in this scenario would correspond to a large collection of paths in such a space. If the technical and policy challenges to collect and integrate such data into a single accessible point of access were surmounted, methods for dimensionality reduction could be applied to reduce the relevant features to a few "components" which could then be used to "cluster" (or classify) the data into groups of similar individuals according to their paths. This is an area of current active research, ranging from the direct application of classic methods such as principal components analysis (PCA), support vector machines (SVMs), and independent component analysis (ICA) with all their myriad of variants, through manifold learning to the revivified use of neural networks for such classification tasks (Mallat, 2016). Developing ways to cope with noisy data and noisy labels is an ongoing challenge in machine learning (Xiao et al., 2015) and across precision medicine, as omics datasets can be extremely noisy.

However, specific requirements in the precision healthcare setting make such tasks especially difficult. The datasets are dynamic and usually sparsely sampled. The processes involved are high-dimensional, highly nonlinear, noisy, and uncertain. The dimensionality reduction framework for such datasets should ideally achieve competing objectives: preserve, to some extent, the meaning of the original descriptive variables (without mixing all features into conglomerates) while extracting concise (i.e., sparse) representations in terms of few relevant extracted features. Ideally, it should be possible to adjust the level of detail (i.e., the resolution scale) of such models depending on the quality of the data and the needs of the practitioner. Finally, the mathematical framework should deliver robust outcomes, and include the possibility of restricting and conditioning the extracted models to incorporate additional and complementary data without the need for refitting.

Indeed, in the process of harnessing these large-scale data, a great degree of caution is required. Most biomedical research is plagued by a flood of false positive results due to experiments of insufficient discriminatory power (Ioannidis, 2005). The translational impact of this trend is starkly illustrated by recent failures to reproduce landmark cancer studies and low success rates in clinical trials (Prinz et al., 2011; Begley and Ellis, 2012). In particular, the quest for (publishable) p-values over (meaningful) effect sizes (Goodman, 1999; Ziliak and McCloskey, 2008) has led to the likely incorrect linking of many genetic features with diseases (Johnston, 2016). Selecting appropriate mathematical models can help increase the statistical power of large-scale experimental data, allowing rigorous statistical treatments to discriminate likely from spurious effects, and quantifying the sizes of effects so that the scientific, as well as the nominally statistical, significance of observations can be better understood.

The interface of individual-level personalized medicine and public health will thus need to develop new mathematical tools to formulate and analyse mathematical questions for datarich characterization of disease progression and transmission, controlled intervention, and healthcare provision. Key areas that we see in the remit of precision healthcare include: statistics for noisy, incomplete, heterogeneous data; stochastic modeling; inference and control of network dynamics; mathematical approaches to exploit complex structure in large datasets, and methods to couple imaging and omics. More broadly, a central distinction between precision medicine and precision healthcare is that the former treats individuals, whereas the latter treats individuals explicitly embedded in a society or broader context. Precision healthcare thus aims to link "big data" tools to explore individual agents with an understanding of how those individuals behave collectively and respond to society-wide initiatives.

# Some Proposed Case Studies in Precision Healthcare

We now describe a number of demonstrative examples, illustrating some of the tools that come under the umbrella of precision healthcare. These range from systems precision medicine approaches focusing on the representation of complex dynamic data, to precision healthcare approaches including both retrospective analysis and real-time interventions that are rooted in complex individual and population data.

#### Gene Therapies for Mitochondrial Diseases

A combination of new maths, statistics, and large-scale experimental data has led to recognition of the importance of personalized therapeutic approaches in cutting-edge gene therapies addressing the inheritance of mitochondrial diseases. These diseases (e.g., mitochondrial encephalomyopathy, lactic acidosis, and stroke-like episodes—MELAS, myoclonic epilepsy with ragged red fibers—MERRF, Leber's hereditary optic neuropathy—LHON) result from mutations in mitochondrial DNA (mtDNA) which are passed from mother to child (DiMauro and Davidzon, 2005). Mitochondrial replacement therapies aim to prevent this inheritance by replacing mutated mother mtDNA with mtDNA from a third party woman, but technological limitations in the procedure can lead to small amounts of mother mtDNA being amplified leading to disease (Burgstaller et al., 2015). Classically the risk of differential proliferation has been considered minimal, but evidence harnessed with statistical modeling and large-scale data from mouse models has shown it is common (Burgstaller et al., 2014). Further, quantitative modeling on large-scale human mtDNA datasets has confirmed that this risk is present in heterogeneous human populations (Røyrvik et al., 2016), as supported by experimental observations (Hyslop et al., 2016; Yamada et al., 2016). The personalized aspect stems from the fact that the risk of differential proliferation depends on the genetic details of the mother's and third-party's mtDNA, which vary throughout global human populations according to geography and ancestry. Appropriate modeling can elucidate the biological details of why these proliferative differences arise, make probabilistic statements about the probability and timescales of therapeutic outcomes, and describe the mtDNA differences likely to arise in human populations. In the future, precision healthcare strategies could allow us to propose suitable third-party donors to optimize successful fertility strategies.

#### Pathways of Disease Progression in High-Dimensional Spaces

Recent mathematical and statistical developments in the study of evolution have shed light on the emergence of efficient photosynthesis (Williams et al., 2013) and the reduction of organelle genomes (Johnston and Williams, 2016) by modeling evolution as the acquisition (or loss) of a set of L-discrete traits. Evolution in this picture takes place on an L-dimensional hypercube, with each vertex corresponding to a given pattern of trait presence/absence and each edge corresponding to an evolutionary innovation. Observations of evolutionary intermediates can then be used, as in a hidden Markov model, to infer likely trajectories through this space. This paradigm can be developed to infer likely pathways of disease progression (generalizing statistical studies on disease progression; Hjelm et al., 2006; Pagel and Meade, 2006; Loohuis et al., 2014; Beerenwinkel et al., 2015), picturing the "space of symptoms" as an analogous hypercube, and disease progression as paths over its edges. Large-scale and longitudinal patient datasets can be used to infer likely sets of "evolutionary" trajectories through this space, so that probabilistic statements can be made about the likely next step for any given individual patient–and thus a personalized optimal therapeutic strategy. Interestingly, this approach can be linked with descriptions based on continuous variables, where similarity graphs are obtained from distance matrices by using graph-theoretical sparsifications that preserve the topological and geometrical structure of the data (Beguerisse-Diaz et al., 2013). The structure of the similarity graphs from the data can then be analyzed using multiscale community detection algorithms leading to highly nonlinear clustering of symptoms and individuals describing the observed pathways of disease progression (Schaub et al., 2012).

#### Social Networks in Health Policy

Twitter provides a platform to interact directly with a large audience, and to sample and address public opinion and responses around specific issues and questions. However, it is critical to understand the different communities and conversations on Twitter, so as to target them appropriately. For example, a recent example following conversations on diabetes (Beguerisse-Díaz et al., 2017) used a unified mathematical framework (Delvenne et al., 2010; Beguerisse-Diaz et al., 2013; Beguerisse-Díaz et al., 2014; Lambiotte et al., 2015) that brings notions from stochastic processes on graphs and optimization to the analysis of Twitter networks. In this particular study, 2.5 million diabetes-related tweets were analyzed and found to fall within five broad thematic groups: health information, news, social interaction, commercial, and humor. Indeed, humorous messages and references to popular culture appear consistently, more than any other type of tweet, revealing the specific characteristics of social media interactions. The analysis of the temporal "hub" and "authority" scores of Twitter users revealed that the hub landscape is diffuse whereas the landscape of authorities is highly persistent. The Twitter authorities comprise not only bloggers, advocacy groups and NGOs related to diabetes, but also for-profit entities without specific diabetes expertise which influence the online exchanges. The top authorities fall into seven interest groups, as derived from their Twitter follower network revealing the flow of information with specific audiences. A similar analysis was carried out on the network of retweets generated by the debate surrounding the proposed adoption of the "care.data" (https://www.england.nhs.uk/ourwork/tsd/care-data/) scheme of personalized health care records by NHS England (Amor et al., 2016). In that case, a series of interest groups and conversations were identified revealing the different roles of users within and across communities, including the limited reach of some of the public policy accounts in the debate. Such findings could be used by public health professionals and policy makers to use social media as an engagement tool and to inform policy design. A similar analysis have been carried out in Beguerisse-Díaz et al. (2017) following other social movements.

#### Spreading of Vaccine Sentiment and Spreading of Vaccine Preventable Disease

Vaccine hesitancy and a vaccine preventable disease can be thought of as two distinct types of processes and they propagate through distinct media. We suppose that sentiment is spread socially (and is influenced by media outlets) but involves the slow evolution of beliefs rather than something as simple as the infectious propagation of a meme. In contrast, disease spread need not respect social network structure. An integrated intervention would not only target vaccination where the disease has been reported and vaccine coverage rates are low (de Figueiredo et al., 2015) but also where it is predicted to appear. Predictions would be based on integrated multi-variate "precision" data. Similarly, negative vaccine sentiment (Larson et al., 2016) could be targeted not only where it has been reported but also where it is predicted to appear given the social network structure. The coupling between belief dynamics and epidemiology now has an established theoretical presence (Wang et al., 2016) and importantly it has been observed that antivaccination behavior is socially clustered (Onnela et al., 2016) thereby undermining herd immunity (Salathé and Bonhoeffer, 2008); it is important to coordinate a public health response that can incorporate belief and behavior dynamics as well as the spread of infection.

#### Influencing Beliefs and Influencing Networks

Health outcomes for chronic conditions are modulated by health behavior, which in turn might be expected to show covariation sensitive to underlying social network structure (Centola, 2010; Shalizi and Thomas, 2011; Christakis and Fowler, 2013). It has further been suggested, independent of unhealthy behavior, that social position can modulate health outcomes (Snyder-Mackler et al., 2016). There are thus a number of possible types of social interventions to improve health: (1) influencing modes of thinking to encourage critical appraisal of apparently acceptable but unhealthy behavior (changing the models that individuals use) (2) influencing health beliefs about particular topics (changing the data individuals access) (3) influencing network structure (but not social co-ordinates) to build bridges between communities for the exchange of health behavior (4) influencing the social co-ordinates of individuals (or sectors of society) and thereby altering their network neighborhood (or the gross social network structure). Changes to (3) and (4) might also affect possible physiological consequences of status comparisons (Pickett and Wilkinson, 2015). While we can cite examples of each class of intervention, these can be remarkably challenging to effect: for example showing some extreme vaccine sceptics information about the consequences of vaccine preventable disease can increase their vaccine scepticism (Nyhan et al., 2014); overwhelming evidence has been presented of health inequities (Marmot and Commission on Social Determinants of Health, 2007; Adler et al., 2016) but the problem persists. Challenges (1–4) constitute challenges in contemporary network science and its interface with optimal control: ideal interventions will optimally control processes on networks and optimally influence the network structure itself (Liu and Barabási, 2016).

#### Genomic Epidemiology for Outbreak Reconstruction

Recent advances in sequencing technologies have driven changes in many biological domains, including epidemiology (Jombart et al., 2014; Kao et al., 2014; Colijn and Cohen, 2016). It is now feasible to obtain DNA or RNA sequences from Colijn et al. Toward Precision Healthcare

viruses, bacteria and other pathogens, and to use these data to detect drug resistance, optimize treatments for individual patients (Vanderkooi et al., 2005; Perez et al., 2016), and to understand how pathogens are spreading and evolving by tracking small variations in the pathogen as it moves between individuals. To understand transmission, isolates are collected from patients alongside clinical data such as times of symptom onset. The isolates are sequenced and processed with bioinformatics tools, capturing even small levels of variation between patients (e.g., in a multiple sequence alignment). These can be integrated with evolutionary models to infer phylogenetic trees, describing patterns of shared ancestry among the isolates. An epidemiological model is used to define how likely a set of infection events are. This incorporates clinical information–for example, it is very unlikely that an individual would transmit an infection years before showing any symptoms, or while living in another area. Finally, mathematical models that link the phylogenetic and epidemiological information are used to compute the joint likelihood of the genetic data and the set of transmission events. This is embedded in a Bayesian approach, so the result is a posterior collection of transmission trees (who infected whom, and when), consistent with the data. There is a rapidly-growing body of work on these inference problems (Hall et al., 2015; Worby et al., 2015; De Maio et al., 2016; Klinkenberg et al., 2016; Worby et al., 2016; Didelot et al., 2017); **Figure 1** is based on the approach in Didelot et al. (2017). There are natural precision healthcare applications of these tools: if more transmission is inferred to have occurred in particular locations, interventions such as improved ventilation and cleaning, early screening and active case fining can be directed there. If risk factors such as community membership, age, or co-morbidities are identified, these can be managed similarly. But perhaps the most exciting applications of these tools will happen when sequencing can be done in a matter of hours or even days. Identifying where there are likely missing cases could allow us to identify cases early, treat them, and prevent onward transmission. Real-time sequencing and infection-tracing has already had impact in the recent Ebola epidemic (Quick et al., 2016), setting the stage for this direction in public health (Gardy et al., 2015).

# DISCUSSION

We have framed precision healthcare to describe the development of precision approaches which, while capturing the complexity of individual data and its societal context, extract reduced dimensionality descriptions at the relevant resolution while preserving a measure of intelligibility of the models. This can enable practitioners in the loop to use these precision approaches effectively. Such methods should be transparent, robust and able to adapt to errors and uncertainties. In bridging from the individual to the population, the methodologies should take advantage of the multivariate data sources at the heart of precision medicine, yet take the social context and population levels into account. Through several case studies in this domain of precision healthcare, we argue that this vision requires the development of new mathematical frameworks, both in modeling and in data analysis and interpretation.

Recently, "precision public health" has been characterized as delivering the right intervention to the right population at the right time (Desmond-Hellmann, 2016; Khoury et al., 2016), mirroring the oft-cited characterization of precision medicine. With support from the Gates Foundation (Cisneros, 2016), precision public health aims to apply precision (data-centered) approaches to improve the health of populations and to reduce health disparities. Public health thinkers are concerned about precision medicine's current emphasis on individual approaches, its focus on extending the use of costly genetics and other omics', and the development of tailored drug treatments (Khoury et al., 2016). Bayer and Galea report that the number of NIH projects with "public" or "population" in the title has dramatically declined, and that in 2014, research areas described with the words "genetic," "genome," or "gene" received 50% more funding than those with "prevention". They are concerned that the focus on precision medicine is misguided (Bayer and Galea, 2015), and argue that improving health requires addressing persistent social realities that are not covered by access to clinical medicine (Adler et al., 2016). Persistent social inequalities can also be expected to be a major barrier in bringing advances from omics-based precision medicine to low-income countries, although recent use of rapid genomics-based tools in the Ebola outbreak (Quick et al., 2016) points to the potential to develop precision-based approaches for low-income settings.

Precision public health places emphasis on addressing such disparities, and (as with public health more generally) on prevention. In many ways, data-centered approaches have already been adopted by epidemiologists and public health practitioners and, as precision public health incorporates more individual-level data, it will require the envisaged scientific tools of precision healthcare. These methodologies will allow public health methods to integrate data on vaccine belief and social context with individual health records, genetic data, other biomarkers, and individual risk factors. Importantly, it is realistic to envision that the use of mobile and social network technologies will enable public health interventions typically considered at the level of populations to instead be tailored to individuals. We believe that an important aspect of the success of precision public health will depend on meeting the mathematical challenges we have outlined as precision healthcare. Identifying the right population for the right intervention will require data analysis, stratification, and modeling at the right scale: too fine, and there would be impractically many populations; too coarse, and the precision advantage is lost. It will require intelligible, transparent methods that can be communicated to public health practitioners, easily updated in the face of new data and human judgment. It will require using the right data to answer the right question, and avoiding mis-use of data to treat some populations unfairly.

It is no longer the case that the timescales of individual disease progression and the timescale of changes in health policy or social behavior are distinct. Chronic conditions from cancer to diabetes are managed over years and decades. Years and decades

are equally the timescales on which other chronic problems are resolved: detrimental individual beliefs about healthy behavior, or disadvantageous social policies. The comparability of timescales of chronic diseases and chronic social problems, combined with the increase of chronic disease in the population, presents both policy, and mathematical challenges: parsimonious and predictive model choice for these slow coupled processes is an open challenge with important implications for the design of public health protocols and policies. Such problems are specific to precision healthcare: While precision medicine might integrate multiple individual-level datasets to improve treatment for a diabetic patient, it does not aim to consider the changing relevant environment and behavior (including beliefs about diet and obesity, food quality and availability, urban environments, and access to exercise). The research outlined above on social networks and health policy also exemplifies precision healthcare: it has a core set of mathematical challenges that are directly linked to healthcare (vs. medicine) and integrates opinion, engagement, delivery, and policy. While precision medicine and healthcare naturally have some overlap, the coupling of scales from individual information to societal behavior and intervention will be characteristic of precision healthcare. However, even with the best intentions, a version of precision healthcare that is highly dependent on advanced tools might be used to reduce, rather than enhance, health equity. A key challenge for precision healthcare is thus to create technologies and practices to drive us toward health equity.

# AUTHOR CONTRIBUTIONS

CC, wrote first draft. NJ, MB, IJ, SY contributed text, edited drafts, contributed to the planning stages and co-authored the paper.

# ACKNOWLEDGMENTS

We thank Paul Matthews, Beth Noveck, Gianluca Fontana, Sabine Vuik, Michael Schneider, Charles Coombes, and Ara Darzi for their insightful comments and ideas. All authors acknowledge funding from EPSRC grant EP/ N014529/1.

#### REFERENCES


paper by the european society for medical oncology (ESMO). Ann. Oncol. 25, 1673–1678. doi: 10.1093/annonc/mdu217

Cisneros, L. (2016). White House and Gates Foundation to Convene Precision Public Health Summit at UCSF.

Available online at: https://www.ucsf.edu/news/2016/05/403091/whitehouse-and-gates-foundation-convene-precision-public-health-summit-ucsf (Accessed November 30, 2016).


treatments. Curr. Oncol. Rep. 17:15. doi: 10.1007/s11912-015- 0438-0


communicable diseases using densely sampled genetic data. Ann. Appl. Stat. 10, 395–417. doi: 10.1214/15-aoas898


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2017 Colijn, Jones, Johnston, Yaliraki and Barahona. This is an openaccess article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Personalized Medication Response Prediction for Attention-Deficit Hyperactivity Disorder: Learning in the Model Space vs. Learning in the Data Space

Hin K. Wong<sup>1</sup> , Paul A. Tiffin<sup>2</sup> \*, Michael J. Chappell <sup>3</sup> , Thomas E. Nichols <sup>1</sup> , Patrick R. Welsh<sup>4</sup> , Orla M. Doyle<sup>5</sup> , Boryana C. Lopez-Kolkovska<sup>1</sup> , Sarah K. Inglis <sup>6</sup> , David Coghill <sup>7</sup> , Yuan Shen<sup>8</sup> and Peter Tiño<sup>8</sup>

<sup>1</sup> Warwick Manufacturing Group, Institute of Digital Healthcare, University of Warwick, Coventry, UK, <sup>2</sup> Mental Health and Addiction Research Group, Department of Health Sciences, University of York, York, UK, <sup>3</sup> School of Engineering, University of Warwick, Coventry, UK, <sup>4</sup> School of Psychology, Newcastle University, Newcastle upon Tyne, UK, <sup>5</sup> Centre for Neuroimaging Sciences, King's College London, London, UK, <sup>6</sup> Division of Maternal and Child Health Sciences, Ninewells Hospital and Medical School, University of Dundee, Dundee, UK, <sup>7</sup> Departments of Paediatrics and Psychiatry, University of Melbourne, Melbourne, VIC, Australia, <sup>8</sup> School of Computer Science, University of Birmingham, Birmingham, UK

#### Edited by:

Zbigniew R. Struzik, University of Tokyo, Japan

#### Reviewed by:

Joaquim Radua, Fidmag Sisters Hospitallers, Spain Qibin Zhao, RIKEN Brain Science Institute, Japan

> \*Correspondence: Paul A. Tiffin paul.tiffin@york.ac.uk

#### Specialty section:

This article was submitted to Computational Physiology and Medicine, a section of the journal Frontiers in Physiology

Received: 16 November 2016 Accepted: 17 March 2017 Published: 11 April 2017

#### Citation:

Wong HK, Tiffin PA, Chappell MJ, Nichols TE, Welsh PR, Doyle OM, Lopez-Kolkovska BC, Inglis SK, Coghill D, Shen Y and Tiño P (2017) Personalized Medication Response Prediction for Attention-Deficit Hyperactivity Disorder: Learning in the Model Space vs. Learning in the Data Space. Front. Physiol. 8:199. doi: 10.3389/fphys.2017.00199 Attention-Deficit Hyperactive Disorder (ADHD) is one of the most common mental health disorders amongst school-aged children with an estimated prevalence of 5% in the global population (American Psychiatric Association, 2013). Stimulants, particularly methylphenidate (MPH), are the first-line option in the treatment of ADHD (Reeves and Schweitzer, 2004; Dopheide and Pliszka, 2009) and are prescribed to an increasing number of children and adolescents in the US and the UK every year (Safer et al., 1996; McCarthy et al., 2009), though recent studies suggest that this is tailing off, e.g., Holden et al. (2013). Around 70% of children demonstrate a clinically significant treatment response to stimulant medication (Spencer et al., 1996; Schachter et al., 2001; Swanson et al., 2001; Barbaresi et al., 2006). However, it is unclear which patient characteristics may moderate treatment effectiveness. As such, most existing research has focused on investigating univariate or multivariate correlations between a set of patient characteristics and the treatment outcome, with respect to dosage of one or several types of medication. The results of such studies are often contradictory and inconclusive due to a combination of small sample sizes, low-quality data, or a lack of available information on covariates. In this paper, feature extraction techniques such as latent trait analysis were applied to reduce the dimension of on a large dataset of patient characteristics, including the responses to symptom-based questionnaires, developmental health factors, demographic variables such as age and gender, and socioeconomic factors such as parental income. We introduce a Bayesian modeling approach in a "learning in the model space" framework that combines existing knowledge in the literature on factors that may potentially affect treatment response, with constraints imposed by a treatment response model. The model is personalized such that the variability among subjects is accounted for by a set of subject-specific parameters. For remission classification,

**25**

this approach compares favorably with conventional methods such as support vector machines and mixed effect models on a range of performance measures. For instance, the proposed approach achieved an area under receiver operator characteristic curve of 82–84%, compared to 75–77% obtained from conventional regression or machine learning ("learning in the data space") methods.

Keywords: attention-deficit hyperactivity disorder, Bayesian inference, machine learning, methylphenidate, mixed effects model, personalized medicine, prognosis, treatment response

# 1. INTRODUCTION

The ability to predict treatment response (or non-response) in patients with mental health issues is potentially beneficial to both clinicians and patients in a number of ways. First, any treatment is accompanied by the risk of adverse effects—where non-response is a probable outcome then the risks of treatment may outweigh the benefits. Second, prediction of treatment response may guide both the dose and choice of medication. For example, where adverse events are dose-dependent then a clinician may chose to abandon a treatment course if a patient was a probable non-responder. Third, response prediction helps to calibrate both clinician and patient expectations of treatment outcomes. Finally, identifying non-responders may prompt a re-appraisal of the diagnosis and formulation of a patient's problem—misdiagnosis being one potential cause of non-response. These benefits certainly apply to Attention-Deficit Hyperactive Disorder (ADHD), which is one of the most common developmental disorders among school-aged children with an estimated prevalence of 5% in the general population worldwide (American Psychiatric Association, 2013). Stimulants, particularly methylphenidate (MPH), are the first-line option in the treatment of ADHD (Reeves and Schweitzer, 2004; Dopheide and Pliszka, 2009). Stimulants are prescribed to an increasing number of children and adolescents in the US and the UK every year (Safer et al., 1996; McCarthy et al., 2009), though recent studies suggest that this trend is tailing off e.g., Holden et al. (2013). The beneficial effects of stimulant medication on the core symptoms of ADHD have been demonstrated by numerous clinical trials, reviews and meta-analyses (Banaschewski et al., 2006; Greenhill et al., 2006; van der Oord et al., 2008; Storebø et al., 2015). Nevertheless, adverse effects of the medications are also common (Storebø et al., 2015). The findings from previous research suggest that around 70% of children demonstrate a clinically significant treatment response to stimulant medication (Spencer et al., 1996; Schachter et al., 2001; Swanson et al., 2001; Barbaresi et al., 2006). However, it is unclear which patient characteristics may moderate treatment effectiveness and whether non-response can be predicted.

To date, achieving accurate predictions of the clinical outcomes for patients with ADHD has proven elusive—most of the literature has focused on investigating the potential correlations between a set of patient characteristics and the outcome following treatment with one or more types of medication. Information relating to patient characteristics has mostly been in the form of subjective questionnaire ratings, clinical notes and qualitative psychometric data; for example, the ratings from symptom-based questionnaires such as the Swanson, Nolan, and Pelham (SNAP) questionnaire (Swanson et al., 1983; Atkins et al., 1985; Swanson, 1992; Bussing et al., 2008), along with demographic variables such as age, sex and social economic background. The results from such studies are often contradictory and inconclusive due to small sample sizes and/or limited availability and quality of data, especially in the temporal (longitudinal) domain.

Along with more conventional statistical approaches, machine learning has also shown promise in predicting treatment response or prognosis in healthcare applications. Indeed, recently a random forest regression analysis was used to predict outcome in a group of patients affected by Obsessive Compulsive Disorder (OCD) from a relatively small pool of questionnaire items, with a reported error rate of 24.6% (Askland et al., 2015). Likewise, there has been a previous attempt to use machine learning techniques to predict treatment response in ADHD (Kim et al., 2015); support vector machine classification from this study was reported as 84.6% accurate (not to be confused with the balanced accuracy measure used in this paper). However, in addition to demographic and clinical questionnaire-derived data, the study used genetic as well as neuroimaging and neuropsychological information as inputs. Such data are unlikely to be readily available to clinicians in routine practice.

In this paper we investigate whether the inclusion of prior knowledge relating to the potential mechanism behind the presentation of a mental health condition and characteristics of individual patients can add value in predicting treatment. Thus, we hypothesized that a pragmatic machine learning approach based on a mechanistic or parametric model (a "learning in the model space" framework) for treatment response prediction may offer an advantage over more conventional methods (Brodersen et al., 2011; Doyle et al., 2013; Chen et al., 2014; Shen et al., 2016). This method represents each newly observed patient through a model; the models are personalized such that individual differences are accounted for by a set of subjectspecific parameters. In the case of ADHD, developing a plausible mechanistic model is not straightforward—despite decades of research, the underlying mechanism for the disorder is not well understood. In addition, any mechanistic model would have to be based on data that are likely to be available in good, but routine, clinical practice.

This paper documents, within the "learning in the model space" framework, a Bayesian linear regression model for the prediction of treatment response in a cohort of children diagnosed and treated for ADHD in the UK. The performance of this new approach is then compared with conventional regression and machine learning methods ("learning in the data space") to assess whether or not the new approach offers benefits, and if so under what circumstances.

# 2. MATERIALS AND METHODS

# 2.1. Participants

The children enrolled in the study were drawn from the ADHD Drug Use and Chronic Effects (ADDUCE) cohort study (The ADDUCE Consortium, 2016), covered by a data sharing agreement with patient consent. The participants were from the UK NHS Tayside region who had attended the ADHD treatment clinics held at Dundee and Perth, UK. 262 families of eligible children were contacted, of which 181 (70%) were recruited and data on 173 of them were obtained for the purpose of this study. In addition, data were available on 94 healthy controls. Out of the 173 patients (whose baseline data are available), 157 of them started dose optimization studies and therefore longitudinal (temporal) data are available (See Section 3.1). To be eligible for the ADHD group, children had to be 6–17 years of age, have a clinical diagnosis of ADHD (see below), have had no previous medical history of methylphenidate use (medicationnaïve) and have parental and child consent/assent to commence. The criteria for the healthy control group were similar apart from them having no current or previous psychiatric diagnoses. The recruitment was carried out over a 30-month period from January 2012–August 2014.

All patients in the ADHD group had already been clinically diagnosed with ADHD; this diagnosis was based on the clinical judgment of the assessing physician, informed by structured interviews with parents/carers, information provided by the child's school, direct observation of the child at the clinic, and at times, in their educational setting. Thus, the physician had to be satisfied that the child fulfilled the diagnostic criteria for a hyperkinetic disorder according to the International Classification of Diseases 10th edition (ICD-10) (World Health Organization, 2010), or ADHD as defined by the Diagnostic and Statistical Manual 4th edition (DSM-IV) (American Psychiatric Association, 2000). This means that the child had to demonstrate disabling and pervasive inattentiveness, hyperactivity, and impulsivity across a range of settings. The clinic was designed to implement a "dose optimization titration" scheme of medication in children diagnosed with ADHD. This involved giving increasing doses of methylphenidate (as the first line medication) at roughly weekly intervals until remission from symptoms was achieved or problematic adverse effects were encountered. If remission was not achieved with a first line medication within recommended dosage limits, or if problematic side-effects were encountered then a second line drug was initiated, and again, increased in dosage, as before.

#### 2.2. Assessment

A range of baseline social and demographic factors was recorded, including parental marital status, family composition, and socioeconomic status as indicated by the Scottish Index of Multiple Deprivation (SIMD) 2012 (APS Group Scotland, 2012) derived from the family home postcode. A history of any previous psychiatric or non-psychiatric medication exposure was recorded, as were any physical health issues. Verbal and non-verbal intellectual functioning was estimated from parental reports and any educational issues noted. Problems with anxiety and low mood were rated using the short form of the Mood and Feelings Questionnaire (MFQ) with the parents, and where appropriate, the child as informants (Angold et al., 1996). Dystonia and abnormal movements were recorded using the Abnormal Involuntary Movement Scale (AIMS) (Guy, 1974, pp. 534–537). Oppositional and ADHD symptoms and behaviors were rated, according to parental report, using the Swanson, Nolan, and Pelham (SNAP-IV) questionnaire (Swanson et al., 1983). Any substance used by the participants was recorded using the Substance Use Questionnaire (SUQ). Fine motor issues were recorded using the Developmental Coordination Disorder Questionnaire 2007 (DCDQ'07). Several sections of the Development and Well-Being Assessment (DAWBA) were used (Goodman et al., 2000); these were (1) Rapidly Changing Mood (child and parent versions), (2) Tic disorders, including the Tourette syndrome, (3) Awkward and troublesome behavior. Tic severity (where present) was also rated using the Yale Global Tic Severity Scale (YGTSS) (Leckman et al., 1989). Possible behaviors associated with an underlying Autism Spectrum Disorder (ASD) were evaluated using the Social Communication Questionnaire (SCQ) (Rutter et al., 2003). The Strengths and Difficulties Questionnaire (SDQ) (Goodman, 1997-07) was used to rate parental perceived levels of pro-social behavior, hyperactivity/ impulsivity, conduct problems, emotional symptoms and peer relationship problems. The overall clinical impression was recorded using the Clinical Global Impression—Severity scale (CGI-S) (Guy, 1974, pp. 218–222) and Children's Global Assessment Scale (CGAS) (Shaffer et al., 1983).

Responses to medication, in terms of levels of ADHD symptoms, were reported by parents and recorded using the SNAP-IV questionnaire at each visit. Likewise, any potential adverse effects and co-morbidity problems were reported using the standard clinic proforma, along with weight, height and blood pressure of the child at each visit.

# 2.3. Feature Extraction/Factor Analysis

The aforementioned questionnaires included a large number of items with categorical (binary or ordinal) response formats. Thus, in order to facilitate model development by reducing the dimensionality of the data whilst minimizing the loss of information, a series of factor (latent trait) analyses were conducted.

The key questionnaires used in the modeling process were the SCQ, the SDQ, and the SNAP-IV (see the previous section). In particular, the SNAP-IV scores served as the outcome variables, which indicated whether symptomatic remission had been achieved, following the dose-optimized titration of medication. The factor analyses sought to identify the dimensionality underlying the responses to the questionnaires and, consequently, the standardized factor scores represented the level of trait for each patient in that underlying dimension or construct.

In order to estimate the dimensionality, the sample of 173 patients and 94 healthy controls was randomly divided into two roughly equal exploratory and confirmatory datasets. A parallel analysis (Horn, 1965), adapted for categorical data, was then implemented in the freeware FACTOR (Lorenzo-Seva and Ferrando, 2006) using unweighted least squares (ULS) estimation method. A weighted "promax" rotation was deployed to achieve factor simplicity (Abdi, 2003). The maximum number of plausible factors (latent variables) was assumed to be indicated at the point where the eigenvalues of the factors in randomly generated data exceeded those observed in the real data. A series of exploratory factor analyses (EFAs–adapted for categorical dependent variables) were then conducted to aid interpretation of the factors. Oblique "geomin" rotation was used (Asparouhov and Muthén, 2009), assuming that, as in almost all psychological measures, underlying latent traits would be correlated with each other to some extent (Thurstone, 1931). A series of confirmatory factor analyses (CFAs) were then conducted using the heldback, confirmatory data (see Section 3.1 on cross-validation), in order to ensure that the factor structures derived fitted

the data adequately. All EFAs and CFAs were conducted in the Mplus software version 7.1, using robust weighted least squares with mean and variance adjustment (WLSMV) as the estimation method (Muthén et al., 1997). Remission was defined by a child having a reported factor score in the hyperactive and inattentive dimensions (both elicited from factor analysis) equivalent to a mean item score in the SNAP-IV of one or less, which is conventionally taken to indicate symptomatic remission (Hechtman, 2005; Chou et al., 2012). The resulting symptom score thresholds are only slightly different for inattentiveness and hyperactivity (−0.97 vs. −0.92).

# 3. MODELING APPROACH

The causal factor model, shown in **Figure 1**, was derived using a rapid review approach to appraise and synthesize the existing evidence (Khangura et al., 2012). This model also took into account the nature of the data available in the cohort and was modified accordingly. The goal is not for the causal model to be comprehensive or definitive, but to identify from the literature

as many potential factors relating to treatment response as there are available from the dataset, as well as helping to elicit the Bayesian prior distributions (Section 3.2.1). Model development was based on a literature review. This involved running searches in the EMBASE, MEDLINE and PsycINFO databases using the synonyms for ADHD (e.g., hyperkinesis) combined with terms relating to treatment outcome or response, and the names of the medications (both scientific and trademarks, full and abbreviated) prescribed in the cohort. The medications include 1) immediate release methylphenidate (IR-MPH, e.g., Ritalin <sup>R</sup> ), 2) long-acting methylphenidate (XR-MPH, e.g., Concerta XL <sup>R</sup> , Equasym XL <sup>R</sup> , Medikinet XL <sup>R</sup> ), 3) dextroamphetamine (DEX, e.g., Dexedrine <sup>R</sup> ) including its prodrug lisdexamfetamine dimesylate (e.g., Elvanse <sup>R</sup> ), and 4) atomoxetine (ATOM, e.g., Strattera <sup>R</sup> ). Secondary sources were followed up. The quality of trial-based studies could be appraised using the CONSORT checklist (Schulz et al., 2010) and observational studies via the STROBE guidance (von Elm et al., 2007). Two of the authors (HKW and PAT) then made a judgment, based on the findings reported in the literature and the perceived likelihood of bias or uncertainty as to what extent variables in the model might be related to treatment response in ADHD. The model derived was consequently used to populate prior distributions for the patient-specific model parameters (i.e., the hyperpriors). Where the evidence was uncertain or inconsistent, the variances (i.e., imprecision) of the hyperpriors were increased.

Not every piece of information mentioned in Section 2.2 was used for the purpose of modeling, because of insufficient data or multicollinearity between the variables. The causal factor model was then simplified based on the breadth of available data from the cohort, leading to a much reduced model as shown in **Figure 2**. Some factors were combined through another layer of feature extraction; for example, the motor and control latent factors, themselves also obtained from applying feature extraction to the DCDQ'07 questionnaire data (see Section 2.2), were combined with the non-verbal communication factor from the SCQ questionnaire to obtain a developmental adversity factor. Some factors were not obtained from standard questionnaires; for example, the perinatal adversity factor (see **Figure 2**) was constructed from birth weight and gestation age; the family size and socioeconomic status factor combined the number of siblings, parental house ownership (owned, mortgaged or rented) and the SIMD 2012 index (APS Group Scotland, 2012).

#### 3.1. Data

There were 267 subjects whose baseline characteristics were measured (173 clinically diagnosed with ADHD and 94 healthy controls) at the first clinical appointment. Of the 173 noncontrols, 157 were enrolled in dose optimization titration studies with parental consent, for whom longitudinal (temporal) data are available. The 157 patients with longitudinal data were randomized and 10-fold cross-validation partitions were constructed. Subjects were partitioned into 10 subgroups of roughly equal size in a patient-coherent fashion, i.e., data from a single patient only appeared in a single fold.

For all models investigated in this paper, a single fold was used as the validation dataset and the remaining nine folds were combined to serve as the training dataset. This process was iterated until each fold had served as validation data exactly once.

#### 3.1.1. Baseline Characteristics

We labeled the patient subjects by the indexing variable s = 1, 2, . . . , N. A set of L patient-specific baseline continuous latent factors, encoded in a row vector **b**<sup>s</sup> ∈ R1×<sup>L</sup> was obtained by performing feature extraction as described in Section 2.3 over the questionnaires detailed in Section 2.2. Referring to **Figure 2**, L = 14 factors were used for the baseline. Data from the controls in addition to the training dataset were utilized during feature extraction to ensure that the resulting latent factor models can sufficiently encompass the entire range of characteristics from ADHD patients to normal children. The resulting continuous latent factors would, in theory, be sufficiently representative of the information conveyed by the categorical questionnaire response variables.

To ensure that validation data were strictly not used for the model building, feature extraction was first performed using only training data from each of the folds (plus all the controls). This resulted in 10 sets of factor scores corresponding to each fold. The factor model structures (e.g., the number of factors per questionnaire) over the folds did not change across the folds, as statistical fit indices and Chi-square difference tests did not suggest that any changes were necessary. The factor models were then used to estimate the baseline factor scores for the validation sets in each of the folds.

Each of the 10 cross-validation runs resulted in a set of corresponding continuous latent factors, which were used as inputs to subsequent models. The models were trained and validated using the same training-validation partitioning used in the feature extraction process.

#### 3.1.2. Longitudinal Data

Each of the 157 subjects with longitudinal data visited the clinic a varying number of times—from titration, stabilization to continuing care; the number of doctor's appointments, A<sup>s</sup> , varies from 1 to 22. At each appointment, the parent or guardian of the patient was asked to fill in an 18-item SNAP-IV questionnaire, which measures the degree of inattentiveness and hyperactivity. The responses were entered into a factor model (identified through feature extraction) to extract a continuous symptom score for inattentiveness and hyperactivity. We denote the appointment number by the indexing variable a so that a = 1, 2, . . . , A<sup>s</sup> . Let the independent "input" variables ma,1, ma,2, ma,3, ma,4 be the four types of medications, respectively, IR-MPH, XR-MPH, DEX, and ATOM for subject s at appointment a. Using datasheets for the medicines used, the dosages of DEX and ATOM were normalized to an equivalent daily dosage (EDD) of IR-MPH. For all a and s, this results in input and output matrices of the form:

$$\text{Input:} \mathbf{M}\_{\mathcal{S}} = \begin{bmatrix} m\_{1,1} & m\_{1,2} & m\_{1,3} & m\_{1,4} \\ m\_{2,1} & m\_{2,2} & m\_{2,3} & m\_{2,4} \\ \vdots & \vdots & \vdots & \vdots \\ m\_{a,1} & m\_{a,2} & m\_{a,3} & m\_{a,4} \\ \vdots & \vdots & \vdots & \vdots \\ m\_{A\_{\mathcal{S},1}} & m\_{A\_{\mathcal{S},2}} & m\_{A\_{\mathcal{S},3}} & m\_{A\_{\mathcal{S},4}} \end{bmatrix} \tag{1}$$
 
$$\text{Output:} \mathbf{y}\_{\mathcal{S}} = \begin{bmatrix} r\_1 & r\_2 & r\_3 & \dots & r\_{A\_{\mathcal{S}}} \end{bmatrix}^{\mathsf{T}}, \tag{2}$$

where r is the symptom severity measure and can be either the inattentiveness factor score or the hyperactivity factor score.

The combined EDDs of medications (for the 4 types) prescribed over the appointment number for all patients are plotted as a boxplot in **Figure 3**. One can observe that as forced titration progressed over the appointments, the overall dosage level increased.

**Figure 4** shows the distribution of inattentiveness and hyperactivity symptom factor scores for the patients for each appointment. The lower the factor scores, the less severe the symptoms are. In terms of a general trend, one can clearly see an effective and quick reduction in symptom levels over the first 5 appointments, as stimulant medication prescription ramps up during forced titration. The symptom scores cease to improve for appointments 6–8, after which a slight increase can be observed. This hints at adherence or persistence issues, but the available data do not allow further investigation—as such issues are not consistently reported by the parent/guardian or recorded in the clinical notes. While the model has no mechanism for modeling such effects, the adaptive learning nature of the Bayesian algorithm is able to self-correct and compensate for small deviations, for example, by "learning" to weight down the dose-response parameter for a given medication when the patient has a low adherence.

#### 3.2. Treatment Response Model Formulation

The treatment outcome is modeled as a linear combination of the baseline variables and the medication dosage,

$$\mathbf{y}\_s = \mathbf{X}\_s \boldsymbol{\alpha}\_s + \boldsymbol{\epsilon} \tag{3}$$

where

$$\mathbf{X}\_{s} = \begin{bmatrix} \mathbf{b}\_{s} & & 1 \\ \vdots & \mathbf{M}\_{s} & \vdots \\ \mathbf{b}\_{s} & & 1 \end{bmatrix},\tag{4}$$

ǫ ∼ N 0, σ<sup>ǫ</sup> ,s 2 ∈ RAs×<sup>1</sup> is an error term and ω<sup>s</sup> ∈ RP×<sup>1</sup> is the subject-specific parameter vector moderating the effect of the baseline variables **b**<sup>s</sup> on the treatment response, i.e., it accounts

boxes: interquartile range; whiskers: 95% confidence intervals; red crosses: outliers.

for how large an effect each of the various baseline variables or medication types has on treatment outcome. "Subject-specific" means that the parameter vector was allowed to be different for each subject so that patients with similar baseline characteristics can still have a different prediction outcome. The number of free parameters required is P = L + 4 + 1 = 19 for L = 14 (see **Figure 2**).

The baseline variables remain unchanged over different appointments while the medication dosage may vary according to the titration regime specified by the clinician. Hence, every row of the matrix **X**<sup>s</sup> contains the same baseline characteristic vector for an individual patient, combined with the medication dosage vector. The row number in **X**<sup>s</sup> corresponds to the appointment number.

Because the number of visits A<sup>s</sup> of the subjects was usually fewer than the dimension of the parameter space P, the problem is mathematically underdetermined. Hence, classical least squares regression methods would fail without an appropriate regularization (Goodfellow et al., 2016). To this end, we employ a Bayesian formulation. In particular, the Bayesian linear regression was used to model the temporal evolution of the dose-response relationship for each patient.

In essence, a Bayesian approach allows prior or expert knowledge to be encoded into the problem formulation and this enables a probabilistic solution to be found despite the limited data available.

#### 3.2.1. Prior Distributions and Knowledge

The joint prior probability density function Pr ω, σ 2 is given in Equation (A3, Supplementary Material). In this exercise, the causal treatment response model based on the literature was used to constrain the prior of ω and its covariance matrix 3 −1 0 .

When eliciting the prior, quantitative information from the literature was not used, e.g., setting the mean of the prior distribution of the parameter vector omega to a specific numerical value. This is because the demography, sample sizes and effect sizes across literature vary and there is no correct way to normalize them. Instead, only the sign (direction) of the effect was encoded. For example, there is evidence that methylphenidate improves treatment response in the literature (a positive dose results in lower symptom score), therefore a negative value of −1 was specified for the columns of τ <sup>0</sup> corresponding to ma,1 . . . ma,4 in Equation (1). For positive associations with symptom scores, +1 was used instead. The same magnitude is used in other factors (i.e., either +1 or −1).

Each cohort study or clinical trial from the rapid review was appraised, respectively, using the STROBE and the CONSORT checklists by counting the number of pass and fail items out of the total. Evidence from the literature was marked as good quality when both the checklist score was similar to other studies on the same topic (within 20% from the best) and effect sizes were statistically significant as reported by the authors for the sample size used. The diagonals of the covariance matrix 3 −1 0 were assigned an initial value of one; for every contradicting evidence (the effect sizes are opposite in direction) satisfying these criteria, 0.5 was added to the corresponding variance in the covariance matrix. A higher value may be specified if necessary, to ensure that the prior distribution of parameter omega spans both positive and negative sides sufficiently—within one standard deviation of τ <sup>0</sup>. On the other hand, if the effect sizes are positive the variance was reduced by 0.1 for each supporting studies, at the same time ensuring the variance does not go below 0.5. The (lack of) proposed existence of causal links between the variables in the causal model (see **Figure 1**) ensures the sparsity of the covariance matrix 3 −1 0 .

While these numbers may not be completely objective, the amount of data available means that the sensitivity of the results to the prior is low—sensitivity analysis shows that the effect of scaling the prior covariance between 50 and 150% of its original values changes the errors by about 5% of the training root mean squared (rms) error, and 3% for the validation rms error.

#### 3.2.2. Posterior Distributions

The posterior distribution is given in Equation (A4, Supplementary Material), where the parameters of the distributions are obtained through Equation (A5) in Supplementary Material. In the training set, Bayesian learning uses data from all of the appointments that a subject had, in which case n = A<sup>s</sup> where, as before, A<sup>s</sup> is the total number of visits or appointments a subject has and data are available for. We introduce the simplified notations after Bayesian update has been applied to the training set using Equation (A5) in Supplementary Material, so that

$$
\hat{\mathbf{A}}\_{A\_{\boldsymbol{s}}}^{-1} \to \hat{\mathbf{A}}\_{\boldsymbol{s}}^{-1}, \quad \mathbf{r}\_{A\_{\boldsymbol{s}}} \to \hat{\mathbf{r}}\_{\boldsymbol{s}}, \quad \boldsymbol{\alpha}\_{A\_{\boldsymbol{s}}} \to \hat{\boldsymbol{\alpha}}\_{\boldsymbol{s}} \text{ and } \boldsymbol{\beta}\_{A\_{\boldsymbol{s}}} \to \hat{\boldsymbol{\beta}}\_{\boldsymbol{s}}. \tag{5}
$$

This notation will be used in later sections. The reader should be reminded that the parameters are derived from each subject and hence are different across subjects. Notice that in a prediction exercise (instead of retrospective regression formulated here), the learning can be applied incrementally for each future observation with each update using just the new observation.

# 3.3. Virtual Patient Profile

When a new patient (denoted by s = ∗) is received, one can measure their baseline variables **b**∗, but not their model parameter space ω∗. The goal is to estimate a virtual patient profile that is believed to best describe the new patient using only the available baseline measurements. To do this one derives the mathematical mapping functions from the baseline characteristics of a patient to their posterior parameters **b**<sup>∗</sup> 7→ Pr(ω∗) and Pr σ 2 ∗ , such that a prediction can be made from the baseline variables. These functions are forged using machine learning on the existing pool of training data. Since Pr(ωs) is parameterized by τ s , 3 −1 s and likewise Pr σ 2 s by αs , β<sup>s</sup> , one has to learn the mappings from the baseline variables to the parameters. The learnt mathematical mapping functions can then be used to obtain estimates of (τˆ <sup>∗</sup>, 3ˆ −1 ∗ ) and (αˆ∗, βˆ ∗), which represent the virtual patent profile for the new patient in the model space.

Due to the conjugate nature of the priors, one does not need to derive the hyperparameters αˆ<sup>∗</sup> and βˆ ∗ from Equation (5) for the purpose of having point estimates for the treatment response prediction. However, these hyperparameters are necessary in order to derive the posterior distribution of the predicted value—commonly referred to as the posterior predictive distribution. Knowing the distribution allows us to approximate the uncertainties of the estimates, e.g., 95% confidence intervals. Two methods were proposed for learning the mappings from the baseline variables to the virtual patient profile and they are introduced in the following subsections.

#### 3.3.1. Method 1: Generalized Linear Regression

To determine the mappings, one finds the functions: a) f τ (**b**∗) ≈ τˆ <sup>∗</sup> and b) f**<sup>u</sup>** (**b**∗) ≈ 3ˆ −1 <sup>∗</sup> where **u**<sup>s</sup> is a row vector containing non-zero elements of the upper (or lower) triangular part of 3ˆ −1 s . Since 3ˆ −1 is a covariance matrix (hence symmetric), knowledge of the lower/upper half of the off-diagonal elements plus the diagonal elements is sufficient to fully recreate the matrix.

The mappings are learnt from the training data, in which the posterior distributions of τˆ <sup>s</sup> and 3ˆ −1 s are already available through Equation (A5) in Supplementary Material. Linear regression models of the form **Y** = **PB**ˆ were used to model the two mappings, where the matrix **B** has rows of **b**<sup>s</sup> vectors—one for each subject in the training set—and similarly **Y** is composed of rows of a) τ <sup>s</sup> for determining f τ or b) **u**<sup>s</sup> for determining f**u** . The least squares solutions for the models are given by the Moore-Penrose pseudo-inverse,

$$
\hat{\mathbf{Q}} = \left(\mathbf{B}^{\mathsf{T}} \mathbf{B}\right)^{-1} \mathbf{B}^{\mathsf{T}} \mathbf{Y}.\tag{6}
$$

For prediction, f τ and f**<sup>u</sup>** can both be formulated as f(**b**∗) = **b**∗**Q**ˆ .

The posterior estimates for the hyperparameters for a new patient are taken as the averaged values of αˆ<sup>s</sup> and βˆ <sup>s</sup> across all subjects in the training set, resulting in αˆ<sup>∗</sup> = 5.5 and βˆ <sup>∗</sup> = 1.7.

#### 3.3.2. Method 2: Gaussian Kernel Weighted Averaging

An alternative method is to find τˆ <sup>∗</sup> and 3ˆ −1 ∗ using a weighted average of τˆ <sup>s</sup> ′ and 3ˆ −1 s ′ , with s ′ ∈ S<sup>∗</sup> being a subset of subjects in the existing training pool whose baseline variables (**b**<sup>s</sup> ′) were "similar" to those of the new patient (**b**∗). Highly similar subjects will have a higher influence on the value of τˆ <sup>∗</sup> and 3ˆ −1 ∗ . The "(dis)similarity" d<sup>s</sup> is measured using the pairwise euclidean distance between **b**<sup>s</sup> and **b**∗, such that

$$d\_s = (\mathbf{b}\_s - \mathbf{b}\_s) \left(\mathbf{b}\_s - \mathbf{b}\_s\right)^\mathsf{T}.\tag{7}$$

This is then sorted and the 17.5% of subjects in S<sup>∗</sup> with the smallest "dissimilarity" values are kept; this percentage value was chosen as it resulted in the lowest validation error. The weighting w<sup>s</sup> was taken as the normalized Gaussian kernel

$$w\_s = \frac{\exp\left(-\lambda \cdot d\_s\right)}{\sum\_{\forall s' \in \mathbb{S}\_\*} \exp\left(-\lambda \cdot d\_{s'}\right)}\tag{8}$$

where the parameter value λ = 1.15 was chosen as it again resulted in the lowest validation error. Using Equations (7) and (8), one can estimate τˆ <sup>∗</sup> and 3ˆ −1 ∗ as

$$
\hat{\mathfrak{r}}\_{\*} = \sum\_{\forall s \in \mathbb{S}\_{\*}} w\_{s} \mathfrak{r}\_{s}, \tag{9a}
$$

$$\hat{\mathbf{A}}\_{\*}^{-1} = \sum\_{\forall s \in \mathbb{S}\_{\*}}^{\cdot} \boldsymbol{w}\_{s} \mathbf{A}\_{s}^{-1},\tag{9b}$$

and similarly the estimates of the hyperparameters are calculated using

$$
\hat{\alpha}\_\* = \sum\_{\forall s \in \mathbb{S}\_+} w\_s \alpha\_s,\tag{9c}
$$

$$
\hat{\boldsymbol{\beta}}\_{\*} = \sum\_{\forall s \in \mathbb{S}\_{\boldsymbol{\theta}}} \boldsymbol{w}\_{s} \boldsymbol{\theta}\_{s}. \tag{9d}
$$

# 3.4. Prediction Using the Posterior Predictive Distribution

When a new subject visits the clinician, their **b**∗ vector may be measured and used to approximate τˆ <sup>∗</sup>, 3ˆ −1 ∗ , αˆ<sup>∗</sup> and βˆ ∗ using either of the methods in the previous subsections. Given a hypothetical medication input **x**∗, the treatment response for the new subject can then be predicted through the posterior predictive distribution

$$\Pr\left(\mathcal{y}\_{\*}\right) = \mathsf{t}\_{\vee}\left(\mathbf{x}\_{\*}\hat{\mathsf{r}}\_{\*}, \frac{\hat{\beta}\_{\*}}{\hat{\alpha}\_{\*}}\left(\mathbf{I} + \mathbf{x}\_{\*}\hat{\mathsf{A}}\_{\*}\mathbf{x}\_{\*}^{\mathsf{T}}\right)\right) \tag{10}$$

where the number of degrees of freedom for the Student's t-distribution is given by ν = 2αˆ∗.

Equation (10) may be used to predict the treatment response for this new patient over their course of the treatment directly without learning; that is to treat each appointment as independent and the parameters are not updated. On the other hand, it is possible to perform incremental Bayesian learning over the course of treatment, by using Equation (A5) in Supplementary Material to update the parameters given the treatment outcome measured for each new visit and the associated inputs. Through incremental learning, the model corrects for discrepancies between the true profile and the virtual patient profile of the new patient. As such, one would expect the prediction to improve as data from more visits to the clinic become available. The implementation of these methods is discussed in more detail in Section 4.4.

At some point, the profile of the new patient in terms of the treatment outcome, inputs, and baseline characteristics can be added to the pool of existing patient profiles (training set) to improve the model's generalizability for future patients.

#### 3.5. Training and Validation

As discussed in Section 3.1, 157 patients with longitudinal data were randomized and 10-fold cross-validation partitions were constructed resulting in 10-folds of training-validation data partitions.

First, for each fold, the framework detailed in Section 3.2 was followed and Bayesian linear regression was performed to fit patient-specific parameters to each patient in the training dataset. Second, either of the methods specified in Section 3.3 was used in order to construct virtual patient profiles for each patient in the validation set, using the patient-specific parameters. Finally, the procedure outlined in Section 3.4 was followed in order to obtain a prediction for patients in the validation set; effectively treating each patient as new.

# 3.6. Dichotomous Remission Prediction

Although the model was initially formulated to predict a continuous scale of symptom scores, one can explore dichotomizing the outcome into patients who have shown reduced symptoms and those who have not. The justification is that clinicians and doctors are less likely to be interested in a predicted SNAP-IV score or symptom severity scale as opposed to a simple "yes/no" answer as to whether the patient will be in remission for a given medication. A simple way to adapt the current model to do this is to apply a threshold to the continuous symptom score prediction, below which the patient is predicted to be in remission.

Some of the literature loosely defines remission in ADHD as having a large majority of SNAP-IV responses rated in category 0 (not at all) or 1 (a little) (Hechtman, 2005; Chou et al., 2012). Therefore in this paper, the thresholds were chosen such that the approximate continuous symptom score corresponds to the raw responses from the 18-item SNAP-IV questionnaire all lying in category 1. The resulting symptom score thresholds are only slightly different for inattentiveness and hyperactivity (−0.97 vs. −0.92, see Section 2.3).

Using these thresholds, it was found that the proportion of visits when measurements were taken indicates that remission was relatively rare, 160 out of a total of 1, 147 (13.95%) for the inattentiveness score and 139 (12.1%) for the hyperactivity score. This is expected, as forced titration initially starts with a low medication dosage and one would not expect an effective reduction in symptom ratings to remission levels before the dosage was ramped up in later appointments; in addition, because of medical persistence issues patients can drop out before the clinicians are able to find an effective dose.

Note that all the methods in this paper were first used to predict the continuous symptom scores by regressing the baseline variables, medication prescribed to the treatment response at following appointments. Dichotomized remission prediction only occurs at a later stage. Right censoring (where patients prematurely drop out of dose optimization stage without achieving remission) is therefore not an issue; repression methods can utilize the remaining appointment information to model treatment response regardless of whether remission was achieved or not.

# 4. PERFORMANCE METRICS

To facilitate a comparison between the performance of the different approaches, several performance metrics were used. For the regression tasks, one is interested in the deviation in the predicted symptom scores against the true symptom scores; whereas for the remission classification tasks, one is interested in the performance of the classifiers with regard to the probabilities or ratios of true positive, false positive, true negative and false negative cases.

### 4.1. Regression Task

The root mean squared (rms) error measure is defined as the square of the averaged squared error across the 10-folds, across subjects and across all appointments for each individual, i.e.,

$$\sqrt{\text{rms} = \frac{1}{10|\mathbb{S}|} \sum\_{k=1}^{10} \sum\_{\forall s \in \mathbb{S}} \sum\_{a=1}^{A\_s} A\_s^{-1} \left| \boldsymbol{\nu}\_s - \boldsymbol{\hat{\boldsymbol{\nu}}}\_s \right|^2} \tag{11}$$

where S is the set of all subjects considered (e.g., those in the validation set), |S| denotes the number of subjects in S; y<sup>s</sup> , yˆs , and A<sup>s</sup> are, respectively, the true outcome symptom score, the fitted or predicted outcome symptom score, and the total number of appointments for the individual subject s.

# 4.2. Classification Task

#### 4.2.1. Sensitivity and Specificity

The sensitivity (SEN, also known as the true positive rate or recall) is defined as NTP/N<sup>P</sup> where NTP is the number of true positives—appointments where measurements indicated remission and were correctly predicted as such; and N<sup>P</sup> is the actual number of positive cases, i.e., the number of appointments where the corresponding subjects were indeed in remission. This is reported in addition to the specificity (SPC, also known as the true negative rate or fall-out), defined as NTN/NN, where NTN is the number of true negatives—those not in remission and correctly predicted as such; and N<sup>N</sup> is the actual number of negative cases (Fletcher and Fletcher, 2005). Note that if one lets NFP and NFN be the number of false positives and false negatives respectively, then N<sup>P</sup> = NTP + NFN and N<sup>N</sup> = NTN + NFP (Fletcher and Fletcher, 2005).

Sensitivity characterizes the ability of a classifier to rule out false negative predictions (type-II errors) given that a condition is true. On the other hand, specificity measures the ability of a classifier to rule out false positive predictions (type-I errors) given that a condition is false. In this exercise, the sensitivity measure is more important; due to the rarity of remission, and the goal is to try to predict what level of medication is required to achieve remission, the ability of a classifier to recall remission cases (ruling out type-II errors) is more important than ruling out type-I errors.

#### 4.2.2. PPV and NPV

The positive predictive value (PPV, also known as the precision) is the proportion of true positives in the predicted positive cases and is the probability of remission given a positive prediction by the algorithm. As such, the PPV is a measure of the "quality" of a given positive prediction. PPV is given by NTP/(NTP + NFP). Conversely, the negative predictive value (NPV) is the proportion of true negatives in the predicted negative cases, and is the probability of non-remission given a negative prediction. NPV is given by NTN/(NTN + NFN) (Fletcher and Fletcher, 2005).

By the argument outlined above, the PPV is more important for this exercise than the NPV.

#### 4.2.3. Balanced Accuracy

The overall accuracy of a dichotomous predictor is defined by

$$\text{Accuracy} = (N\_{\text{TP}} + N\_{\text{TN}}) / N,$$

where N = 1, 147 is the total number of appointments across all subjects.

However, the overall accuracy measure is known to be problematic when the prevalence of success/failure is low (Alberg et al., 2004), i.e., the data are imbalanced (see also the end of Section 3.6). Due to this, some of the literature uses the balance accuracy (BAC) measure, defined as the average of sensitivity and specificity (Brodersen et al., 2010). This is the accuracy measure used throughout this paper. Note that, numerically, the BAC is closely related to the Youden's J-statistic (Youden, 1950), also known as "informedness" or "DeltaP′ " (Powers, 2011), since it is equal to sensitivity plus specificity minus one.

#### 4.2.4. ROC and AUC

The receiver operating characteristic (ROC) curve is commonly used in the medical and the machine learning community to evaluate the performance of binary classifiers Fawcett (2006). It plots the true positive rate (sensitivity) against the false positive rate (one minus specificity) for a given classifier. A curve is obtained when its classification performance can be tuned through setting a threshold or changing a parameter, trading off the true positive rate against the false positive rate. Binary classifiers that can achieve good compromise between sensitivity and specificity have a large area-under-the-curve (AUC), and this single metric may be used to compare the performance between the different classifiers Bradley (1997).

# 4.3. Trading Off Sensitivity and Specificity

From Section 3.6, the proportion of appointments without remission is (100 − 13.95)% = 86.05%. Therefore, given this statistic, one would expect that a null model guessing the result randomly would have a sensitivity of 13.95% and a specificity of 86.05%. Simply using point estimates of the continuous symptom score from the learning in the model space approach and thresholding them to give dichotomous predictions of remission results in classifiers with low sensitivity values between 22 and 28% and high specificity values of 94–97%. Due to the low number of remission cases compared to the non-remission cases, the classification is biased against predicting the remission cases, leading to low sensitivity (but high specificity). A classifier can be tuned to improve its sensitivity performance by trading off specificity to a certain degree. A good compromise would be maximizing both sensitivity and specificity equally, which is in essence maximizing the BAC or the Youden's J-statistic in Section 4.2.3.

The thresholds for remission are defined by the SNAP-IV symptom factor scores as in Section 2.3, and this defines the ground truth of whether a patient is in remission or not. However, one can take advantage of the fact that the predicted continuous symptom scores from the learning in the model space approach form full posterior distributions with uncertainties associated, and the levels of uncertainty are known (e.g., see the error bars in **Figures 5**, **6**). One may define a critical value as the lower bound of the prediction, above which the probability of the prediction being correct is x%. Instead of the remission thresholds comparing against the point estimates, they may be compared against the point estimates minus a critical value. The larger the critical value, the higher the prediction score has to be in order to be classified as not in remission. This in effect is equivalent to raising the threshold, classifying more and more cases into remission, which increases the sensitivity and lowers the specificity. The range of "thresholds" or classifier parameter settings that makes this trade-off can be used to generate a ROC plot (Section 4.2.4). A similar trade-off can be made with classical machine learning algorithms and will be discussed in Section5.

The training data are used to find an optimal classifier setting in order to achieve the highest BAC, and the same classifier setting is then used to classify the validation data. This ensures that the validation data are not used to minimize the validation error.

# 4.4. Benchmarking and Implementation

The Bayesian learning in the model space approach relies on prior knowledge (Section 3.2.1) and virtual patient profiles in the model space (Section 3.3), as well as iterative learning (Bayesian update) in order to function. To assess whether these components contribute to the prediction capability of the model, several implementation strategies are investigated, namely:


The Bayesian approach was implemented ad hoc in MATLAB software with custom routines. The performance of the prediction was compared across the three different implementations above, using the validation data.

### 5. COMPARISON WITH CONVENTIONAL METHODS

To provide context to the results achieved using the learning in the model space approach, the performance of conventional linear regression methods and machine learning methods was also investigated.

FIGURE 6 | Examples of continuous hyperactivity symptom score prediction with the validation set. (A) Subject #6. (B) Subject #148.

### 5.1. Mixed Effects Models

Linear mixed effects models (MEM) are widely used in many fields; for instance, biology (Rico et al., 2007), ecology (Stevens et al., 2007), linguistics (Nooteboom and Quené, 2008) and social sciences (Kliegl et al., 2009). They extend upon classical linear regression techniques to support data that have some form of grouping. For example, in this paper, each patient had one or more clinical appointments, and the data from each subject form a group. For each patient s with a number of appointments (from 1 to As), the severity symptom score vector **y**<sup>s</sup> (as in Equation 2) is, for simplicity, assumed to have a linear relationship with the baseline and treatment effect via the following formulation:

$$\mathbf{y}\_s = q\_s + b\_0 + \hat{\mathbf{X}}\_s \boldsymbol{\omega}\_s + \boldsymbol{\epsilon},\tag{12}$$

where **X**ˆ s is similar in structure to **X** defined in Equation (4) but without the last column of ones; ω<sup>s</sup> ∈ RP×<sup>1</sup> is the subjectspecific parameter vector for the fixed effects, q<sup>s</sup> ∼ N(0, σ 2 q ) is the random effect affecting only the intercept b0, and ǫ ∼ N(0, σ 2 ǫ,s ) ∈ RAs×<sup>1</sup> is an error term assumed to have a normal distribution. Observe that in this model, the scalar intercept b<sup>0</sup> is a fixed component for all of the patients in the population and the random effect a<sup>s</sup> is a subject-specific scalar and is not grouped under any other parameters.

A linear mixed effects model was constructed within the R software (R Core Team, 2013), using the package "lme4" (Bates et al., 2015). Severity score predictions were produced by performing out-of-sample forecasts, i.e., on the validation data for each of the folds, using the "predict" function in the R software. To generate a prediction, the random effects are assumed to be zero and the population intercept was used. The continuous symptom score regression results for the MEM are prefixed MER.

For a dichotomized clinical remission classification, the symptom score thresholds 0.92 and 0.97 from Section 2.3 for hyperactivity and inattentiveness were used to generate the ground truths. Following the rationale in Section 4.3, the symptom scores predicted by the MEM are given thresholds at different levels to produce a set of classifiers trading off sensitivity against specificity. These threshold-adjusted classifiers are labeled taMEC. The best (in terms of Youden's statistic) threshold settings found using the training data were used for the validation data; the thresholds were 0.05 and -0.20, respectively, for the inattentiveness and hyperactivity symptom scores. In addition, the "melogit" function in the Stata software (StataCorp, 2015) was used to directly estimate a mixed effects logistic regression model—a MEM with a logistic link function that predicts the probability of the binary remission outcome. In this case, the threshold procedure was applied to the probabilities rather than the raw symptom scores. The resulting classifier is labeled lrMEC.

# 5.2. Support Vector Machines and Gaussian Processes

In addition to MEM, machine learning classification approaches using support vector machines (SVM) and Gaussian processes (GP) were benchmarked. Both the SVM and GP learning methods are kernel machines and were implemented using linear (dot product kernel: k(**x**<sup>i</sup> , **x**j) = (**x**<sup>i</sup> .**x**j)) and nonlinear kernels (the Gaussian kernel: k(**x**<sup>i</sup> , **x**j) = exp - −γ(k**x**<sup>i</sup> − **x**jk) 2 ). Readers are invited to refer to Burges (1998) for a detailed description of support vector machines and to Rasmussen and Nickisch (2010) for a detailed description of GP learning. Compared to MEM and the learning in the model space approach, the SVM and GP are non-parametric methods—there are no subject specific parameters to identify; the models map the subject-specific inputs, such as baseline characteristics and the medication dosage, to the output symptom scores.

For the SVM, nested cross-validation was employed to optimize the parameters in the model. A broad log range spanning [10−<sup>3</sup> : 10<sup>2</sup> ] was arbitrarily chosen as the search range for the regularization parameter C. Similarly, the gamma parameter of the Gaussian kernel was optimized in the log range spanning [10−<sup>4</sup> : 10<sup>1</sup> ]. For the GP, the model parameters were optimized using conjugate gradient descent, avoiding the need for nested cross-validation. SVM and GP learning approaches were employed as regression models (support vector regression SVR and Gaussian process regression GPR) for the linear and nonlinear kernels to predict the clinical scores. Dichotomous remission predictions were obtained by thresholding the distance from the hyperplane for the SVM, and for thresholding the probabilistic predictions of class membership for GP. These binary classifiers are respectively labeled as SVC and GPC.

From Section 3.6, the number of remission cases outweighed non-remission cases by a ratio of roughly 1:7. For a classification task, this imbalance of data is problematic for many classification algorithms (He and Garcia, 2009). To help alleviate this, a downsampling approach was implemented for both linear and nonlinear kernels of the SVM and GP classifiers dsSVC and dsGPC. During the training phase, the non-responder class was downsampled randomly to match the number of training instances in the remission class. By repeating this downsampling procedure, an ensemble of 1,000 classifiers was trained. A classification prediction was generated by majority voting of the ensemble. Additionally, for the SVMs, an alternative is to learn the regularization parameters C on a per-class basis. The rationale is that a higher penalty for errors can be placed on the more abundant class (Osuna et al., 1997); this method is referred to as the weighted SVM (rwSVC). The per-class C parameters C <sup>+</sup> and C <sup>−</sup> were optimized using the ranges - 10−<sup>3</sup> : 10<sup>1</sup> and - 10−<sup>2</sup> : 10<sup>2</sup> , respectively. Finally, for the Gaussian process classifier, it is possible to calibrate the probabilistic predictions in order to help account for imbalanced data (Bishop, 2006); this approach is referred to as a re-calibrated GP (rcGPC).

Using MATLAB software, the SVM was implemented using the libsvm toolbox (Chang and Lin, 2011), the Gaussian process learning was implemented using the GPML toolbox (Rasmussen and Nickisch, 2010).

# 6. RESULTS AND DISCUSSIONS

# 6.1. Continuous Symptom Score Prediction

The rms errors across all models are reported in **Table 1**.

#### TABLE 1 | Rms errors for predicting symptom scores for inattentiveness (INA) and hyperactivity (HYP) using the (A) learning in model space and (B) conventional approaches.



\*AIR: appointment-independent Bayesian linear prediction.

†BRR: retrospective Bayesian linear regression.

‡BUR: incremental Bayesian learning/update linear regression.

#### (1B) Conventional approaches


\*SVR: support vector machine regression.

†GPR: Gaussian processes regression.

‡MER: mixed effects regression.

#### 6.1.1. Learning in the Model Space Approach

Looking at the learning in the model space approach, one can observe that the virtual patient profile construction method, labeled Method 2, resulted in lower errors overall compared to Method 1. In Method 1, the mappings were learnt using simple linear regression from baseline variables to the parameter space. In addition to this simple linear regression, low degree polynomial (quadratic to quartic) basis functions were tried; whilst degrees up to a cubic resulted in a slightly lower training error, there was worse generalizability (i.e., higher validation error). For Method 2, the incremental Bayesian learning (BUR) approach performed the best overall; its performance advantage over the appointment-independent prediction (AIR) approach is expected given that it allows the model to adapt to a new patient as the treatment continues. The performance advantage over the retrospective Bayesian linear regression (BRR) approach can be attributed to the fact that the virtual patient profile (Section 3.3) had utilized the prior whilst the Bayesian linear regression only uses the elicited prior (see Section 3.2.1). This supports the fact that the training population was able to add valuable information to the prediction task.

We recall that the BUR constructs virtual patient profiles while the BRR only uses the prior knowledge. It is interesting to note that the BRR outperforms the BUR using Method 1, suggesting that Method 1 was not an effective method for incorporating information from existing patient models.

**Figure 7** shows the rms values averaged across all subjects during the validation phase and sorted by the clinical appointment (visit) number. Data above 15 visits are not shown as only a single patient had more than 15 visits. There is a slight downward trend visible with the BUR; suggesting that incremental Bayesian learning approach is able to reduce the prediction error as more data are known about a new patient through repeated appointments. The BRR also shows a downward trend, but the error is slightly higher than the BUR. This is because the BRR starts with only the elicited prior and performs learning (fitting) when more data are available, unlike the BUR which starts off with information from the training set in the form of a constructed/estimated virtual patient profile.

**Figure 5** illustrates some examples of Bayesian linear regression performed during the training phase and their associated fitting rms errors. It can be seen that Bayesian linear regression fits similarly well for both the inattentiveness and hyperactivity symptom scores. Switching to a new type of medicine is usually associated with larger uncertainty (error bars). Looking at subject #74 (**Figure 5B**) in particular, it can be seen that, despite having the same input dosage from appointments 6–8, there were variations in the severity of the ADHD symptoms. It is not possible to know the exact reason for the variation for this subject during this particular period, without further information—perhaps this was due to adherence issues (the patient not taking their medication as prescribed), physiological factors, measurement "noise," or perhaps something else entirely. By design, Bayesian linear regression can only fit the same outcome given the same input. This does highlight the fact that the current model may not have enough information in the form of covariates to account for some of these factors.

A subset of results for the validation phase is plotted in **Figure 6**. For brevity, only prediction outcomes for hyperactivity

symptom scores are shown and retrospective Bayesian linear regression results (BRR) are omitted. The lack of solid lines connecting the predictions in the topmost subplot serves as a reminder that the model does not incorporate temporal aspects for the case of appointment-independent (AIR) prediction, which treats each appointment as the first (new) appointment for a new patient. This is also why the 95% confidence intervals for AIR are larger (more uncertain) than those for the BUR. Also note that, by design, the prediction results for the first appointment are identical for both approaches.

The figures illustrate that, during prognosis, incremental learning does not always improve the prediction error compared to simply predicting at every appointment without updating the model using new information. However, based on the rms errors in **Table 1A**, one expects incremental learning to perform better overall across subjects, especially for subjects with a prediction offset, such as over- or under-estimates. This is illustrated by the results for subject #148 given in **Figure 6B**, where the virtual patient profile for this patient consistently underestimates the actual hyperactivity score. Here, the incremental Bayesian learning was able to adapt the parameter ω and shifted the prediction upwards, resulting in lower prediction errors over the subsequent appointments.

#### 6.1.2. Conventional Machine Learning Approaches

Looking at the results in **Table 1B**, the conventional approaches yield similar performance, with linear SVR and GPR methods performing better than their nonlinear counterparts. The mixed effects model has slightly worse results. Errors of linear SCR and linear GPR are similar to each other, and to those for the learning in the model space approach BUR with Method 2. We conclude that for the task of predicting continuous symptom scores with the dataset investigated, the learning in the model space approach performs comparably with conventional approaches.

# 6.2. Dichotomous Remission Prediction 6.2.1. Learning in the Model Space Approach

For the learning in the model space approach, the ROC curves for the dichotomous predictor are plotted in **Figure 8** for both of the virtual patient profile (Section 3.3) construction methods. As the results for inattentiveness and hyperactivity scores were similar, only the ROC curves for inattentiveness are shown. The AUC values are given in the legend. The crosses on the lines mark the resulting classifier performance if one uses point estimates for the continuous symptom score from the model and simply applies the clinical remission thresholds. The squares mark the classifiers that have critical values based on maximizing the Youden's J-statistic (or the BAC, see Section 4.2.3) for the training set—this is equivalent to the sensitivity and specificity measures being maximized equally as a function of the critical values. Lastly, the circles mark the best classifier for the validation set in terms of the Youden's J-statistic. The closer the squares are to the circles, the better optimized the classifier is assuming no knowledge of the validation dataset. Those optimized classifiers marked by squares in the graph are used to generate various binary classifier performance metrics (see Section 4.2) in **Table 2**.

Looking at **Table 2**. The confusion matrices (CFM) show the number of true positives and false negatives in the first column, and false positives and true negatives in the second column. These may be used to calculate any classification performance metrics not included in this paper, such as the F-measure.

Similar to the continuous symptom prediction task, the BRC outperforms the AIC showing that posterior information is utilized effectively. As in the continuous task, the virtual patient profile construction method labeled Method 2 is better overall than Method 1, but the difference is much smaller in the classification task and the advantage is not universal across all metrics, especially for the AIC. Note that the virtual patient profile construction method has little effect on the BRC as it does not use it. The BRC achieves higher sensitivity values but a lower PPV compared to the BUC, meaning that the BRC is better at recalling remission cases, but the remission predictions by the BUC are more reliable. The SPC achieved by the BUC is notably higher, being better at ruling out false positives.

#### 6.2.2. Conventional Machine Learning Approaches

**Table 3** shows the binary classifier performance metrics for the conventional machine learning approaches. Apart from the AUC, all of the other metrics in the table were derived from classifier settings (set-points) that had optimized the balanced accuracy (BAC) during the training stage. Apart from rcGPC, the BAC values across the different approaches are similar. The MEC classifiers perform well compared with GPC and SVC, with consistently high AUC values for both inattentiveness and hyperactivity. However, the set-points of the MEC classifiers achieve lower sensitivity (but higher specificity) than the SVC. As mentioned in Section 4.2, a higher sensitivity is more important for this exercise. PPV is the other measure of interest; the lrMEC, in particular, achieved the highest PPV amongst all the conventional approaches—partially helped by its low sensitivity.

The ROC plot for the MEC is shown in **Figure 9**. The lrMEC variant fitted the training set better but both the lrMEC and the taMEC achieve similar validation performance. Tracing the ROC values, the lrMEC seems more suitable for high specificity settings while taMEC appears to be more suitable for high sensitivity classification.

For the machine learning approaches GPC and SVC, linear models work better. This was similarly observed in the continuous symptom score prediction task. Comparing methods in tackling data imbalance, the weighted SVM classifier rwSVC method performed better than the downsampled dsSVC method, while the downsampled Gaussian process classifier dsGPC method performed better than the re-calibrated rcGPC. Looking at both the BAC and AUC metrics, rwSVC and dsGPC perform similarly, with the former slightly better at classifying remission of hyperactivity, whereas the latter is slightly better for inattentiveness.

Overall for the conventional methods, the rwSVC achieves the best compromise between SEN and PPV, meaning that it can identify remission cases more readily and at the same time the remission predictions are more reliable. Comparing **Tables 2**, **3** it can be seen that the learning in the model space approach is superior overall. With respect to the BAC and AUC measures, the best performing BUC approach has an advantage of about 6–7%. This is interesting given the similar performance in the continuous symptom score prediction task amongst all approaches. The rms error measure in the symptom score prediction task was based on point estimate calculations, and thus used no information on the shape of the posterior distribution. The posterior predictive distribution (Section 3.4) for the BUC has a Student's t-distribution specific to each patient. The distributions were used to construct a probabilistic threshold in trading off specificity and specificity. This subject-specific nonlinear thresholding procedure may have contributed to its performance advantage over other approaches.

#### 6.2.3. Comparison with Literature

As far as the authors are aware, Kim et al. (2015) is the only published literature on treatment response prediction of ADHD patients using machine learning techniques. Their best attempt achieved an AUC value of 0.84 and 86.4% classification accuracy (that is, the percentage of correct predictions, different from the BAC measure used in this paper) using a wide range of information types including demographical, clinical, genetic, environmental, neuropsychological and neuroimaging measures. In comparison, this paper includes only the more readily obtainable demographical and clinical information and is able to achieve best-case AUCs of 0.82–0.84. Restricting to demographical and clinical information, the highest performing method using SVMs in Kim et al. (2015) had an AUC of 0.69. Granted, the comparison is imprecise because the quality, quantity and sources of demographical and clinical information are different between this paper and Kim et al. (2015). Judging from the AUC values achieved by SVMs in this paper of about 0.71 (see **Table 3**), the results appears to be very close to those in Kim et al. (2015). Due to this similarity, the previous comparisons should be valid.

# 7. CLINICAL UTILITY AND FURTHER WORK

The proposed learning in the model space approach is capable of predicting, for an individual, the minimum dosage of a particular medication required to have a user-defined chance of achieving symptomatic remission. It is highly flexible and potentially can be extended to any disease or disorder where medication is used


TABLE 2 | Sensitivity, specificity, accuracy, and AUC of the remission classifier with critical values adjusted with respect to uncertainties in the predicted symptom scores.

<sup>a</sup>CFM: confusion matrix.

<sup>b</sup>SEN: sensitivity.

<sup>c</sup>SPC: specificity.

<sup>d</sup>BAC: balanced accuracy.

<sup>e</sup>PPV: positive predictive value.

<sup>f</sup> NPV: negative predictive value. <sup>g</sup>AUC: area under ROC curve.

\*AIC: appointment-independent classifier.

†BRC: retrospective Bayesian linear regression classifier.

‡BUC: incremental Bayesian learning/update classifier.

Shaded values represent best performance amongst the compared methods.

in the course of treatment, speeding up and reducing the cost of the dose optimization/forced titration process, and potentially improving the quality of life for patients by ending the treatment sooner.

The current model, however, does not take into account adverse drug reactions (ADRs), minimization of which is another goal of a dose optimization titration process. To improve clinical utility, it is essential that ADRs are modeled. While data on this are available from the clinical notes accompaning the ADDUCE trial, a different modeling approach is required to incorporate the many different types of ADRs, with prevalence ranging from infrequent to very rare.

While the proposed approach achieves excellent performance in terms of treatment response classification, there is room for improvement. One obvious way to achieve this is to incorporate more data, especially covariates that are functions of time. In this exercise for example, the body mass index and age variables measured at baseline (first appointment) of the patients contribute to the latent factors, which in turn form the baseline variables. As such, they do not vary over time. It may be worth investigating whether the addition of temporal covariates, such as blood pressure, would improve the model.

Another venue for potential improvement is to extend the linear model to a nonlinear model—there is no guarantee that all


TABLE 3 | Sensitivity, specificity, accuracy, and AUC of the remission classifier with critical values adjusted with respect to uncertainties in the predicted symptom scores.

<sup>a</sup>dsSVC: down-sampled support vector machine classifier; <sup>b</sup>dsGPC: down-sampled Gaussian processes classifier; <sup>c</sup>rwSVC: regularization-weighted support vector machine classifier; <sup>d</sup>rwGPC: regularization-weighted support Gaussian processes classifier; <sup>e</sup>taMEC: threshold-adjusted mixed effects classifier; <sup>f</sup>lrMEC: logitic regression mixed effects classifier. Shaded values represent best performance amongst the compared methods.

the covariates have a linear relationship with treatment response. Identifying the level and nature of nonlinear relationships is the first challenge. In the current Bayesian framework, the introduction of nonlinearities increases computational complexity for Bayesian inference, requiring the use of techniques such as Gibbs sampling.

There are other areas of interest. For example, what is the optimum strategy, in terms of timing and requirements, for incorporating semi-new patient data to the model space to improve the generalizability of the model for other new patients? How can medical adherence/concordance be modeled? Does gender of the patient play a role in their treatment response?

# 8. CONCLUSION

A learning in the model space framework has been utilized to develop a personalized medicine approach to treatment response prediction. First of all, factor analysis was performed to extract latent factors from a large clinical dataset, collected from a UK sample of 157 patients suffering from attention-deficit hyperactivity disorder. The resulting reduced-order patient information was then encoded in a model parameter space resulting in a cloud of personalized models. Then, the patientspecific model space parameters were used to train a Bayesian linear regression model. New patients are then matched to existing patients most similar to themselves to obtain a virtual patient profile, which in turn forms a prior parameter set for the Bayesian linear regression model. Through a Bayesian update algorithm, new data are continuously integrated to improve the prediction performance for a given patient. In addition, the parameters of the "new" patients can be added to the model parameter space (once sufficient data are available) to improve the generalizability of the model for future patients.

Comparisons were made between the learning in the model space approach with conventional data-driven machine learning and regression approaches. In terms of the prediction of the continuous symptom factor scores, the performance of the learning in model space framework was on a par with conventional approaches. However, the new approach is shown to outperform support vector machines, Gaussian processes and linear mixed effects classifiers in the prediction of symptomatic remission. The effective gain in classification performance of the new model can potentially speed up and reduce the cost of a forced titration or dose optimization titration process, which is normally manually performed by the clinician to assess the effective dosage of medication. Further work includes incorporating the prediction of adverse drug reactions, which is also an important element in the dose optimization titration process.

# ETHICS STATEMENT

This is secondary data analysis of the ADDUCE ADHD study. ADDUCE has received a favorable ethical opinion from the relevant UK National Health Service (NHS) Research Ethics Committee (REC reference 11/ES/0016). All subjects gave written informed consent in accordance with the Declaration of Helsinki. The protocol was approved by the REC. Consent to information access and secondary data analysis has been granted. All data were handled as per relevant guidelines and rules for such information, including the Data Protection Act 1998 and as per the initial ethical approval.

# AUTHOR CONTRIBUTIONS

HW was responsible for performing feature extraction of the clinical data, literature search, causal factor model simplification, the software implementation of the learning in model space framework, and determined benchmarking protocol. PAT provided the clinical context of ADHD, constructed the prior knowledge used by the Bayesian framework, aided development on the causal factor model, and designed the literature search protocol. PW aided HW on the literature search and written the background on ADHD. OD implemented the SVM and GP machine learning algorithms. BL constructed the linear mixed effects models. YS helped with the technical implementation of the Bayesian linear regression algorithm. PT provided the mathematical formulation of the learning in model space approach and guidance on software implementation. MC and TN suggested candidate modeling approaches compatible with the framework, suggested a number of conventional approaches to be evaluated, appraised the output of the Bayesian approach, and provided recommendations for the improvement of the implementation and the methodology to ensure fair comparisons. SI coordinated access to the clinical data, facilitated digitization of paper-based clinical records, and maintained the database. DC provided the clinical context of ADHD, suggested causal mechanisms for treatment response, and defined the scope of clinical data collection.

# FUNDING

We gratefully acknowledge the support from the UK Engineering and Physical Sciences Research Council (EPSRC), grant number EP/L000296/1. The ADDUCE project from which this piece of research borrowed clinical data is funded by the European Union's Seventh Framework Programme for research, technological development and demonstration under grant agreement number 260576.

#### ACKNOWLEDGMENTS

The authors would like to thank the staff of Ninewells Hospital and Medical School, the University of Dundee, and NHS Tayside; in particular trial database manager Emma McKenzie for coordinating the data exchange and clinical research nurse

#### REFERENCES


Jacqueline Paton for her meticulous data digitization efforts. The authors also would like to acknowledge the invaluable feedback by fellow POEMS (Predictive modeling for hEalthcare through MathS) network members and members in the ADDUCE consortium.

#### SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: http://journal.frontiersin.org/article/10.3389/fphys. 2017.00199/full#supplementary-material

deficit/hyperactivity disorder symptoms: Psychometric properties and normative ratings from a school district sample. Assessment 15, 317–328. doi: 10.1177/1073191107313888


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2017 Wong, Tiffin, Chappell, Nichols, Welsh, Doyle, Lopez-Kolkovska, Inglis, Coghill, Shen and Tiño. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Systematic Characterization of Dynamic Parameters of Intracellular Calcium Signals

Laurent Mackay 1 † , Nicholas Mikolajewicz 2, 3 †, Svetlana V. Komarova2, 3 and Anmar Khadra<sup>1</sup> \*

<sup>1</sup> Department of Physiology, McGill University, Montreal, QC, Canada, <sup>2</sup> Faculty of Dentistry, McGill University, Montreal, QC, Canada, <sup>3</sup> Shriners Hospital for Children-Canada, Montreal, QC, Canada

#### Edited by:

Krasimira Tsaneva-Atanasova, University of Exeter, UK

#### Reviewed by:

Silvina Ponce Dawson, University of Buenos Aires, Argentina James Sneyd, University of Auckland, New Zealand Martin Falcke, Max Delbrück Center for Molecular Medicine (HZ), Germany

\*Correspondence:

Anmar Khadra anmar.khadra@mcgill.ca

† These authors have contributed equally to this work.

#### Specialty section:

This article was submitted to Computational Physiology and Medicine, a section of the journal Frontiers in Physiology

Received: 31 August 2016 Accepted: 24 October 2016 Published: 10 November 2016

#### Citation:

Mackay L, Mikolajewicz N, Komarova SV and Khadra A (2016) Systematic Characterization of Dynamic Parameters of Intracellular Calcium Signals. Front. Physiol. 7:525. doi: 10.3389/fphys.2016.00525 Dynamic processes, such as intracellular calcium signaling, are hallmark of cellular biology. As real-time imaging modalities become widespread, a need for analytical tools to reliably characterize time-series data without prior knowledge of the nature of the recordings becomes more pressing. The goal of this study is to develop a signal-processing algorithm for MATLAB that autonomously computes the parameters characterizing prominent single transient responses (TR) and/or multi-peaks responses (MPR). The algorithm corrects for signal contamination and decomposes experimental recordings into contributions from drift, TRs, and MPRs. It subsequently provides numerical estimates for the following parameters: time of onset after stimulus application, activation time (time for signal to increase from 10 to 90% of peak), and amplitude of response. It also provides characterization of the (i) TRs by quantifying their area under the curve (AUC), response duration (time between ½ amplitude on ascent and descent of the transient), and decay constant of the exponential decay region of the deactivation phase of the response, and (ii) MPRs by quantifying the number of peaks, mean peak magnitude, mean periodicity, standard deviation of periodicity, oscillatory persistence (time between first and last discernable peak), and duty cycle (fraction of period during which system is active) for all the peaks in the signal, as well as coherent oscillations (i.e., deterministic spikes). We demonstrate that the signal detection performance of this algorithm is in agreement with user-mediated detection and that parameter estimates obtained manually and algorithmically are correlated. We then apply this algorithm to study how metabolic acidosis affects purinergic (P2) receptor-mediated calcium signaling in osteoclast precursor cells. Our results reveal that acidosis significantly attenuates the amplitude and AUC calcium responses at high ATP concentrations. Collectively, our data validated this algorithm as a general framework for comprehensively analyzing dynamic time-series.

Keywords: algorithm, calcium imaging, kinetics, osteoclast pathophysiology, parameter characterization, purinergic/P2 receptors, real-time imaging

**46**

# INTRODUCTION

Cellular biology is vastly populated with dynamic processes, which can be altered dramatically or subtly by pathological causes. Calcium signals, characterized by fast and transient increases in cytosolic free calcium concentration ([Ca2+]i), which vary in amplitude and duration and can exhibit oscillatory dynamics with frequency-dependent downstream effects (Clapham, 2007), represent a prominent example of such dynamic processes (**Figure 1A**). To fully understand the data of such dynamic complexity, a robust methodology to analyse, and characterize these responses is necessary. Numerous studies have investigated [Ca2+]<sup>i</sup> dynamics, but the analysis have in many cases been limited to qualitative assessments (Cao et al., 1997; Frame and de Feijter, 1997; Jorgensen et al., 1997; Jørgensen et al., 2003; Isakson et al., 2001; Romanello and D'Andrea, 2001). Studies that have pursued quantitative analysis of calcium time-series reported a number of different, often non-overlapping characteristics of the response (**Table 1**). In cases where experiments were conducted on a smaller-scale, manual analysis was achievable. However, to achieve larger-scale analyses for experiments with hundreds of individual recordings, open-source signal-processing algorithms are required and becoming increasingly relied on to overcome these bottlenecks in productivity. None of 11 published algorithms we examined provided a comprehensive analysis of the entire response observed within a recording (**Table 2**). As a direct consequence of the lack of a standardized methodology to quantify such data-sets, findings from various studies are challenging to compare, relate, and generalize. Hence, the motivation of this study was to achieve faster analysis while standardizing the methodology involved, thereby minimizing user-bias, and ensuring consistency in the analysis of complex physiological signals. While such a tool may or may not change the conclusions of individual studies, it would improve comparability between different studies, and potentially enable meta-analysis of different experiments. We have therefore developed an algorithm that addresses these concerns and focused on the dynamic signals generated by purinergic (P2) receptors to demonstrate its utility.

Purinergic receptors that evoke intracellular responses upon extracellular stimulation with nucleotides, such as ATP and ADP, are known to induce complex [Ca2+]<sup>i</sup> signals. P2 receptors are subdivided into two families, P2X and P2Y receptors, which are omnipresent in virtually all mammalian tissue (Burnstock and Verkhratsky, 2009). The mammalian P2X receptor family, consisting of seven subtypes (P2X1–7), are ionotropic ligand-gated cation channels that can permit the influx of extracellular calcium upon stimulation (Kaczmarek-Hajek et al., 2012). The mammalian P2Y receptor family, consisting of eight subtypes (P2Y1–2,4,6,11–14), are metabotropic G-protein coupled receptors that can indirectly modulate the release of calcium from intracellular calcium stores through inositol triphosphate (von Kugelgen and Hoffmann, 2016). P2 receptors have been demonstrated to play an important role on bone physiology (Lenertz et al., 2015). Since individual bone cells commonly express multiple active P2 receptors (Gallagher and Buckley, 2002), responses to purinergic stimulation result in complex, concentration-dependent [Ca2+]<sup>i</sup> transients (Xing et al., 2016). While it remains difficult to experimentally isolate the contribution of individual receptors, a number of studies have demonstrated that various P2 receptor subtypes have distinct calcium response kinetics and signatures. For instance, various P2X receptors desensitize at distinct rates under sustained agonist stimulation (Koshimizu et al., 1999). P2X7-mediated responses in particular are biphasic (Yan et al., 2010) and characterized by sustained [Ca2+]<sup>i</sup> elevation (Nobile et al., 2003). It is becoming increasingly clear, however, that P2 receptors cannot be studied and manipulated as individual components, but rather must be regarded as building blocks of a far more "dynamic architecture" that permits diverse functionality and flexibility (Volonte et al., 2006).

The goal of this study is to develop a universal signalprocessing algorithm for MATLAB (MathWorks, Natick, MA) that would facilitate and standardize the parameter characterization of time series calcium imaging recordings containing prominent single transient responses (TR) and/or multi-peaked responses (MPR). All signals, no matter their complexity, can be reduced to a set of defined characteristics that describe the magnitude and kinetics of a given response. Based on our expertise and literature review (**Tables 1, 2**), we have selected the following parameters: time of onset after stimulus application (tonset), activation time (time for signal to increases from 10 to 90% of peak; t10–90%), and amplitude of response. Additionally, TRs are specifically described by their area under the curve (AUC), response duration (time between ½ amplitude on ascent and descent of the transient; FWHM), and decay constant of the exponential decay region of the deactivation phase of the response (τdecay, **Figure 1B**) while MPRs are described by their number of peaks (Nosc), mean peak magnitude (E), mean periodicity (T), standard deviation of periodicity (σT), oscillatory persistence (time between first and last discernable peak; losc), and duty cycle (fraction of period during which system is active; ξ peak/T, where ξ peak is the width of the oscillatory peaks, **Figure 1C**). Since MPRs can be either stochastic or deterministic (Skupin et al., 2008; Dupont and Combettes, 2009; Dupont et al., 2011), the algorithm reports two sets of MPR parameters. The first describes MPR parameters for all the peaks present, while the second set reports the MPR parameters describing the subset of coherent oscillations, to omit the influence of stochastic processes, and to focus on the deterministic properties of the signal.

Live cell recordings will inevitably contain signal contaminations arising from experimental conditions and instrumentation, including (a) photochemical effects induced by the measurement process and (b) unrelated biological processes. While these imperfections are inherent to the experimental

**Abbreviations:** ADP, Adenosine diphosphate; ATP, Adenosine triphosphate; AUC, Area under curve; [Ca2+]<sup>i</sup> , Cytosolic free calcium concentration; D(t), Global drift model; E, Mean peak magnitude of MPR; FWHM, Full-width half-max; F(t), Measured fluorescent signal; losc, Oscillatory persistence; MPR, Multi-peak response; Nosc, Number of peaks in MPR; ξ peak, Width of oscillatory peak; P2, Purinergic; σT, Standard deviation of periodicity; T, Period of oscillation; t10–90%, Activation time; τdecay, Decay constant; tonset, Time of response onset; TR, Transient response; TV, Total-variational.

FIGURE 1 | Characterization of dynamic calcium signals. (A) Examples of heterogeneity of [Ca2+] i responses observed in various live-cell recordings; note many signature forms that TRs and MPRs can exhibit. (B) Analysis of parameters for single-peak TRs. Inset: Representative single-peak TRs. (C) Analysis of parameters for MPRs. Inset: Representative MPRs. Time of onset, tonset; area under curve, AUC; full-width half-max, FWHM; activation time, t10–90%; decay constant, τdecay; periodicity, T; number of peaks, Nosc; oscillatory peak magnitude, E; width of oscillatory peaks, ξ peak; point of inflection in deactivation phase of TR, F ′′(ρ). Arrows/text in red illustrate how parameters of interest are obtained.



process, dynamic processes of interest can still be extracted from these recordings. This process in itself can be complicated and highly subjective depending on the extent to which the raw data are corrupted by noise and drift. Therefore, to reliably evaluate the magnitude and kinetics of these signals, we have developed a systematic way of first identifying unwanted


TABLE 2 | Published signal-processing algorithms.

\*Classifier: grouping of time-series by characteristic signature of response.

signal contaminations, and then removing their effects when determining the parameters of interest. Following algorithm validation, we have also investigated the effect of acidosis on ATP-mediated [Ca2+]<sup>i</sup> responses in bone-marrow derived osteoclast precursors to demonstrate the efficacy of this algorithm in characterizing real-time cellular dynamics.

To implement the algorithm in MATLAB, the user is required to store the set of discrete sample points of the measured signal F (t), { F<sup>i</sup> }, i = 1, 2, · · · , N, along with the corresponding discrete time points {t<sup>i</sup> } in an Excel file (filename.xlsx). The algorithm can then be run using the MATLAB command

#### ≫characterizeDocument("filename.xlsx")

The MATLAB code required to execute this command along with examples of data and scripts are provided in Supplmentary Materials (Data Sheet 1).

# MATERIALS AND METHODS

#### Cell Culture

All procedures were approved by McGill University's Animal Care Committee and complied with the ethical guidelines of the Canadian Council on Animal Care. Bone marrow precursor cells were isolated from the femur and tibia of 6 week old FVB mice (Charles River), plated at a density of 7.5 × 10<sup>3</sup> on 48-well glass-bottom plates (No. 1.5 Coverslip, 6 mm glass diameter, uncoated, MatTek Corp.) and cultured for 3 days in αMEM (12,000-022, GIBCO) supplemented with 10% FBS (080152, Wisent), 1% sodium pyruvate (600-110-UL, Wisent), 1% penicillin streptomycin (450-201-EL, Wisent), 50 ng/mL MCSF (300-25, Peprotech), and 50 ng/mL RANKL according to the protocol previously described (Boraschi-Diaz and Komarova, 2016).

#### Intracellular Calcium Measurements

After 3 days of culture, osteoclast precursors were loaded with fura2-AM, a ratiometric fluorescent calcium dye (F1221, Invitrogen), incubated at room temperature for 30 min and washed twice with physiological solution (130 mM NaCl; 5 mM KCl; 1 mM MgCl2; 1 mM CaCl2; 10 mM glucose; 20 mM HEPES, pH 7.6). The final volume of 270 µL of physiological solution at pH 7.6 or pH 7.0 was added and cells were acclimatized for 10 min to reduce the effects of mechanical agitation that resulted from fura2-AM loading and washing. 10X ATP (Sigma) solutions were prepared in physiological solution at corresponding pH and 30 µL was added after 10 s of baseline [Ca2+]<sup>i</sup> recording to obtain a 1X dilution (i.e., final concentrations ranging from 1 µM to 10 mM ATP). [Ca2+]<sup>i</sup> was imaged for an additional 110 s at a sampling rate of 2 images per second using a fluorescent inverted microscope (T2000, Nikon). The excitation wavelength was alternated between 340 and 380 nm using an ultra-highspeed wavelength switching illumination system (Lambda DG-4, Quorum Technologies). Regions of interest (ROI) were manually defined and the ratio of the fluorescent emission at 510 nm, following 340 and 380 nm excitation (f340/f380), was calculated and exported using the imaging software (Velocity, Improvision). All data were imported into an excel spreadsheet for subsequent analysis.

# Validation and Statistical Analysis

Algorithm performance was evaluated using the algorithm generated figures for 450 individual signal fitting that enabled retrospective visual examination of both response-detection and quality of parameter fitting. Manual and automated estimates were compared using a correlation plot and Bland Altman analysis (Bland and Altman, 1986) to assess the degree of correlation and agreement, respectively. For correlation analysis, the line of exact linear correlation (i.e., y = x) is plotted as a reference to assess deviation of the linear regression curve from the desired 1:1 relationship between the manual and automated estimates. For the Bland Altman analysis, we compared the automated (a) and manual (m) parameter estimates of the ith recording to obtain a Z-score, given by

$$\mathcal{Z}\_{\mathrm{i}} = \frac{\mathfrak{x}\_{\mathrm{i}} - \bar{\mathfrak{x}}}{\sigma\_{\mathfrak{x}}}$$

where x<sup>i</sup> = (m<sup>i</sup> + ai)/2 is the average value of the estimated parameter, x¯ is the mean value of x averaged over all recordings, and σ<sup>x</sup> is the standard deviation of x overall recordings. Furthermore, the percent difference, 4<sup>i</sup> , of the ith recording is defined by

$$
\Delta\_{\rm i} = \frac{\text{error}}{\text{average value}} \times 100\% = \frac{a\_{\rm i} - m\_{\rm i}}{x\_{\rm i}} \times 100\%.
$$

4<sup>i</sup> vs. Z<sup>i</sup> were plotted to illustrate systematic biases. Negative values of 4<sup>i</sup> were interpreted as manual estimates being greater than automated estimates, and vice versa. A quantitative estimate of the interval of agreement, within which 95% of differences lie, is defined by

95% interval of agreement <sup>=</sup> <sup>4</sup>¯ <sup>±</sup> 1.96σ4¯

where 4¯ is the mean percent difference over all recordings and σ4¯ is the standard deviation of 4¯ .

Experimental data were expressed as means ± S.E.M. Effect of ATP treatment under control conditions was evaluated using one-way ANOVA followed with Bonferroni post hoc test. The effect of acidosis was evaluated using two-way ANOVA with Bonferroni post hoc test. Results were accepted as significant at p < 0.05. Statistical analysis was performed in MATLAB.

#### RESULTS AND DISCUSSION

Although the notation used throughout the text implies that the recorded signals are fluorescence, the methodology remains the same for any other type of signals. For fluorescent recordings, the measured fluorescence F may consist of multiple parts: the drift, TR including the activation and deactivation phases, and the superimposed MPR. It can be expressed as the sum of the actual signal, Ftrue, and the normally distributed noise with a standard deviation σ.

$$F(t) = F\_{true}(t) + \mathcal{N}(0, \sigma) \,. \tag{1}$$

To characterize parameters that reliably reflect Ftrue(t), F(t) is first preprocessed to remove the effects of noise (Section Noise Characterization) and to estimate the contributions of drift to Ftrue(t) (Section Baseline Drift). Next, the activation phase of TR is fit while simultaneously refining the estimated contribution of the drift. This approach allows us to determine if the recording is consistent with the expected model of a TR superimposed on a drifting baseline (i.e., whether activation phase is followed by a deactivation phase, Section Activation Fitting). If a TR is detected, we proceed by fitting the full set of TR model parameters simultaneously with the drift parameters [Section Transient Response (TR) Model]. In the case where there remain multiple significant deviations in the data from the TR model, we investigate and characterize the presence of oscillatory MPRs (Section Multi-Peaked Responses). The fitting of the TR is refined to remove the effects of the multiple peaks on the initial fit (Section Identifying Coherent Oscillations), in order to provide the best estimate of the baseline around which the MPRs oscillate. The deviations resulting from this secondary fitting of the TR are then analyzed to determine those resulting from coherent oscillatory processes (Section Characterizing Oscillatory Parameters). At each step throughout the fitting procedure, an updated estimate of the optimal set of parameters (e.g., the drift parameters) is obtained. These parameters are then used in a feed-forward manner, where the optimal set of parameters of the preceding fit is used as an initial guess for the subsequent step. This ensures that the algorithm produces high-fidelity fittings. Finally, the algorithm performance and utility is demonstrated with a new data set describing the effect of acidosis on ATP-induced calcium signaling in osteoclast precursors (Section Application to Pathophysiology).

#### Noise Characterization

The first step in the processing of data is to evaluate four values that will be used for the remainder of the text: the derivative (uˆ) of the noisy signal, the standard deviation of noise (σ−), indices at which this noise is not prevalent (j), and the noise-to-signal ratio (φ). uˆ will be used in Section Drift Delimitation to separate the TR from the underlying drift. Data points excluded by j (i.e., noise) will be omitted. The methodology detailed throughout the following section is an iterative procedure. In the instances where those quantities are used, we are referring to the value determined by the final iteration of the procedure.

#### Euler-Lagrange Formalism

The presence of noise in a recording renders naïve methods of derivative estimation inadequate (Chartrand, 2011). This is particularly exacerbated by the intermittent presence of large amplitude noise (spikes) related to the use of high gain settings on instrumentation. To reliably delimit (i.e., define the boundaries of) the drift in a recording of a noisy transient signal (Section Drift Delimitation), we have adapted the total-variational (TV) technique commonly used to estimate the first derivative of a signal contaminated with various types of noise (Chartrand and Staneva, 2008; Chartrand and Wohlberg, 2010; Chartrand, 2011). This technique performs better than the low-pass filter in distinguishing the drift from the transient response, as it does not indiscriminately remove high frequency components that affect the overall trend of the signal.

Our TV-based methodology seeks a function, uˆ (t), which represents the derivative of Ftrue(t), that solves the optimization problem.

$$\min\_{u} \alpha \int\_{0}^{L} \left| \frac{du}{d\mathbf{x}} \right| d\mathbf{x} + \frac{1}{2} \int\_{0}^{L} \left| (Au\left(\mathbf{x}\right) - F\left(\mathbf{x}\right)) \right|^{2} d\mathbf{x} \tag{2}$$

The first term in Equation (2) is a regularization term which penalizes sudden changes in the derivative (to make the fitting smooth), the second term is an L <sup>2</sup> data fidelity term, where A is the anti-differentiation operator (Au ≈ Ftrue), and α is a parameter dictating the balance between the two terms. In order to solve this minimization problem, we have to find the stationary solution to the following equation

$$u\_t\left(\mathbf{x}\right) = \alpha \frac{d}{d\mathbf{x}} \frac{u'\left(\mathbf{x}\right)}{|u'\left(\mathbf{x}\right)|} - A^T \left(Au\left(\mathbf{x}\right) - F\right) \tag{3}$$

derived from the Euler-Lagrange equation associated with Equation (2), where A T v (x) = L r x vdx is the L 2 -adjoint of A. Within the context of ratiometric fluorescent dyes (such as Fura2 AM used for [Ca2+]<sup>i</sup> recordings in this study) the recorded signal is the ratio of two Poisson random variables. The variance and the mean of such a signal follow a complex, and seemingly nonlinear, function of the photon count rates at each wavelength. Since these rates are assumed to be unknown a posteriori in a recording, we cannot accurately determine how the noise is distributed. However, we will assume that instrumentation and experimental settings contribute to a noise distribution that is approximately Gaussian. Moreover, in the specific case of ratiometric dyes (Section Intracellular Calcium Measurements), we have found that true noise distribution is a complex function of time, but can be represented reliably using a time-dependent Gaussian noise model. Therefore, to reduce data-fidelity and conversely increase regularity in the regions of highest amplitude noise while accurately reproducing data in regions of lowest noise, we find instead a stationary solution to the following equation

$$u\_t(\mathbf{x}) = \alpha \frac{d}{d\mathbf{x}} \frac{u'(\mathbf{x})}{|u'(\mathbf{x}) + \varepsilon|} - \frac{A^T(Au(\mathbf{x}) - F)}{\psi\_-(\mathbf{x}, u) + \eta},\tag{4}$$

where ψ (x, u) is an iteratively determined weighting function (as described below), and ε and η, are parameters introduced to avoid dividing by zero.

At the n th iteration of the algorithm, we solve for u <sup>n</sup> <sup>+</sup> <sup>1</sup> by setting the left-hand-side of Equation (4) to zero, and linearizing the problem through substituting every u appearing in the denominators by the value of u <sup>n</sup> obtained from the previous iterate. For a more detailed descritption of the means used to solve this type of problem (see Vogel, 2002; Chartrand and Staneva, 2008; Chartrand and Wohlberg, 2010). It is known that an appropriate choice of the denominator offsets, ε and η, is necessary to produce acceptable minimizations (Chartrand, 2007), yet this choice is rarely considered beyond their status as parameters that must be tweaked to obtain acceptable results (Li et al., 2007; Chambolle et al., 2009; Oh et al., 2013). In what follows, we detail a methodology on how to determine the parameters α, ε, η, and the function ψ (t), based on the data F (t), and the derivative estimate u (t).

#### Dynamic Determination of Total-Variational Parameters

Given a set of fluorescence recordings { F<sup>i</sup> } of length N, at the n th iterate of the regularization algorithm, we identify the set of indices j = {i = 1, 2, · · · , N |ψ n (ti) 6= ∞ } (as explained in Section Removal of Noise Spikes) whose data are not likely dominated by noise and thus should contribute to the fidelity term of Equation (4). Letting 1<sup>i</sup> = F<sup>i</sup> − F<sup>i</sup> <sup>−</sup> <sup>1</sup>, we can define the weighting sequence g<sup>i</sup> = 1 − |1<sup>i</sup> |/max i |1<sup>i</sup> | in order to provide an upper bound on the noise of the signal, given by

$$
\sigma\_+^n = \frac{\sum\_j |\Delta\_j| \lg\_j}{\sum\_j \mathcal{g}\_j}.
$$

The weights g<sup>i</sup> will tend to zero as 1<sup>i</sup> approach their maximum, and converge to a positive number (< 1) as |1<sup>i</sup> | approach their minimum. We can thus conclude that the weighted average of 1j  will identify the smallest differences as being the most informative of the magnitude of noise. Large discrete differenecs, whether they result from transient increases in the noise level or from the fact that the signal is non-stationary, contribute only modestly to the estimate σ n <sup>+</sup>. On the other hand, we can also estimate a lower bound on the noise using

$$
\sigma\_-^n = mean \left( \left| \zeta\_j^n \right| \right),
\tag{5}
$$

where ζ n <sup>i</sup> = F<sup>i</sup> −(Au<sup>n</sup> )i , i.e., by taking the difference between the data and the cumulative integral of u. Because we use the discrete differences 1<sup>i</sup> as our initial solution: u 0 <sup>i</sup> = 1<sup>i</sup> , the integral of u n (n > 0) will likely diverge away from the data with each succesive iteration of the algortithm due to the action of the regularization term in Equation (4). This tends to result in σ n <sup>−</sup> being smaller than σ n <sup>+</sup>, although this is not always strictly true. The use of the two different estimators for the noise allows for a more robust performance of the methodology, as both estimators are prone to becoming inaccurate in different scenarios. With an estimate of the noise, we can also estimate the noise-to-signal ratio by

$$
\phi^n = \frac{\sigma\_+^n}{\max\left(A u^n\right)}.
$$

The value of φ is a critical parameter in our algorithm as it discreminates between small and large values of various quantities. For example, it is employed to calculate an appropriate value of ε, defined in Equation (4), based on the scale of variations of small values of u ′ , as follows

$$\varepsilon^n = \text{std}\left(\left\{|u'\_r| : |u'\_r| \le \left(\phi^n\right)^2 \max\_i \left(|u'\_i|\right), r \in i\right\}\right).$$

The calculation of η requires defining another weighting sequence h<sup>i</sup> = u ′ i /max i u ′ i , which tends to zero when u is the most regular, as well as an estimate for the upper bound on the total error, χ, between the integral of u and the data in the least regular regions of the solution, given by

$$\chi^n = \frac{\sum\_{\mathbf{i}} h\_{\mathbf{i}} \left| \zeta\_{\mathbf{i}}^n \right|}{\sigma\_{-}^n}$$

.

When χ is small, η must be made large enough to improve the smoothness of the fitting (at the expense of data-fidelity). This can be acheived by making η a decreasing function of χ. However, if a recording does not contain any rapid jumps or noise spikes (but is nonetheless noisy), such a relation between χ and η will not be sufficient to infer a proper choice of η given χ. Thus, we must include another term independent of χ which will produce modest data-fidelity for signals dominated by drift. We therefore define η to be

$$\begin{split} \eta^{n} = \frac{\exp}{\sigma\_{+}^{n}} \Bigg[ \begin{split} & -\left(\frac{\chi^{n}+1}{2\sigma\_{-}^{n}} \max\left(\left\{ \left|F\_{i} - F\_{i-1}\right| - std\left(\left\{ \left|F\_{r} - F\_{r-1}\right| \right.\right.\right. \\ & \left.:= 2, \dots, N\right\} \right) \left| \begin{array}{c} \frac{\sqrt{\chi^{n}}}{2} \\\\ \frac{\chi^{n}}{2} \end{array} \right. \\ & + \operatorname{mean}\left(\arg\max\left(\left|d\right|\_{\ell^{1}} : \operatorname{mean}(d) \ge 3\operatorname{std}(d)\right) \right) . \end{split} \end{split}$$

which includes the smallest non-zero scale of the weighted differences δ n <sup>i</sup> = 1 − h n i 1<sup>n</sup> i .

TV methods tend to smooth the fit when there are large amplitude jumps in the data, or where u is large. If a single large jump dominates the derivative, this can lead to excessive local smoothing, which can be resolved by having enhanced datafidelity at that point. On the other hand, in the presence of large jumps in the data, small noise-driven fluctuations may be under-regularized. In this case, data-fidelity at these points must be reduced. The function ψ allows for local enhancement or reduction of data-fidelity. Unlike ε, η does not depend on φ, but data-fidelity must through the weighting function ψ (by making ψ proportional to φ). Therefore, we define ψ to be

$$
\psi\_{\mathbf{i}}^n = \left(\phi^{\sqrt{\chi^n + 2}}\right)^{f\_1} + \phi^{f\_2}, \tag{6}
$$

where

$$\begin{array}{rcl} J\_1 &=& \overline{\left[\frac{1}{\left(\sigma\_+^n\right)^{1+\sqrt{\phi^n}}} \sqrt{\frac{\left|u\_{i+1}^{n-1}\right|}{\min\left(\left\{\left|u\_r^{n-1}\right| : u\_r^{n-1} \neq 0\right\}\right)}}\right]}, \\ J\_2 &=& \overline{\left[\sqrt{\frac{1}{\left(\frac{1}{\tau\_{\text{max}}} + \frac{1}{3}\right) \sigma\_+^n}} + \frac{\text{mean}\left(\left|u^{n-1}\right|\right)}{\left|\max\left(\left|\nu^n\right|\right) - \left|\upsilon\_i^n\right| + \phi^n\right|}\right]}, \end{array}$$

τ n max = mean  ζ n j <sup>+</sup> std  ζ n j /σ<sup>n</sup> <sup>+</sup> is a iterative error scale parameter, υ n i = u <sup>n</sup> <sup>−</sup> <sup>1</sup> <sup>i</sup> <sup>+</sup> <sup>1</sup> + 1 − h<sup>i</sup> <sup>+</sup> <sup>1</sup> 2 u n − 1 i − 1 is an estimator of u<sup>i</sup> based on the adjacent values of u, and ω<sup>i</sup> is a three-point moving average of δ<sup>i</sup> . The exponent J<sup>1</sup> emphasizes data-fidelity (regularity) when the derivative of the following point is large (small), whereas the exponent J<sup>2</sup> emphasizes data-fidelity when 1<sup>i</sup> are small or when the derivative is near its maximum. When 1i is large and u<sup>i</sup> <sup>−</sup> <sup>1</sup>, u<sup>i</sup> are regular, on the other hand, J<sup>2</sup> is small allowing regularity to propagate forward into regions of signal possessing large ampltidue noise (i.e., where the data is not informative). The balance between the two effects of J<sup>2</sup> along with the χ -dependent terms of Equation (6) produce an acceptable compromise between the regularity of the fit and data-fidelity for recordings across a wide range of signal-to-noise ratio and extremely varied dynamics. Finally, once the maximum relative change between two iterations of the procedure becomes less than √ φ, we consider the solution to have achieved quasi-stationarity and terminate the procedure.

#### Removal of Noise Spikes

When a recording exhibits intermittent periods of high amplitude noise (noise spikes), the data contaminated by these noise spikes is minimally informative. Detecting them allows for determining the indices j (**Figure 2A**). Within our regularization algorithm, this is done by (a) setting ψ to infinity at those time points in such a way that Equation (4) only penalizes irregularity at these time points, and (b) determining the fit at these points based on the surrounding (reliable) data. After each iteration, n, of the regularization algorithm, a smoother fit, Au<sup>n</sup> , of the data is obtained. We also obtain a criterion that determines whether or not each point represents a noise spike based on a comparison between the residual differences, ζ n , and a chosen threshold value. This threshold is specified using the parameter τ n rm, given by

$$
\mathfrak{r}\_{m}^{n} = (1 - \xi) \, + \, \xi \, \mathfrak{r}\_{\text{max}}^{n},
$$

where ξ = exp − σ n <sup>+</sup> − σ n − /max σ n <sup>+</sup>, σ n − − ( χ n ) 2 /N is a convergence parameter for the noise rejection method. Positive ζ n i are rejected if they are greater than τ n rmσ n <sup>+</sup>, while negative ζ n i are rejected if they are less than τ n rm<sup>2</sup> σ n <sup>+</sup>. The use of two thresholds is due to the asymmetry of the Poisson statistics underlying data collection using photodetectors. Rejection is achieved by setting ψ<sup>i</sup> = ∞, which serves as the basis for defining the set of indices j (i.e., j = {i = 1, 2, · · · , N | ψ n (ti) 6= ∞ } as stated before).

#### Baseline Drift

When fitting data to specific functional forms, it is important to take into account temporal drifting of the baseline in a signal. The specific nature of the processes underlying this drift are not of particular interest here. Rather, we consider them as nuisance trends and aim to remove their effects from the data.

#### Drift Model

Some drifts are quite slow compared to the timeframe of the experimental recording, and thus can be fairly well represented by linear functions, whereas others are fast and better represented by an exponential decay function. Since, in a given recording, a number of processes will result in the observed drift, we postulate that, the drift throughout the signal can be well fit to a combination of linear and exponential functions

$$d\_t\left(t; a\_t, \tau\_t, m\_t\right) = a\_t \left[1 - \exp\left(\,^{-t}\angle\_t\right)\right] + m\_t t \tag{7}$$

where, aι , τ<sup>ι</sup> , and mι are the exponential amplitude, exponential time constant and the slope of the linear component of the ι th drift within the signal, respectively (examples of fitted drifts shown in **Figures 2B,E,F**). With an appropriate choice of data, these are determined using the least squares fitting as discussed in Section Drift Fitting. We have found numerous cases where either the linear or exponential components were not justifed. However, this knowledge is unavailable to us prior to conducting manual or automated analysis of the data, and we cannot assume a priori a less general form than Equation (7). Thus, we have

Drift Fitting.

the MPR baseline. (F) Complex drift; Change in the baseline drift, tshift, prior to onset of TR is ignored in favor of d1 after the reweighting procedure outlined in Section

to rely on the fitting to optimize the contribution of the two compoenets in a data-dependent manner.

It is possible for the signal to exhibit (multiple) drifts with different trends separated perhaps by a TR. In order to capture this effect in a signal, we use a global drift model that combines the intial and secondary drifts (**Figure 2B**) in a semi-piecewise manner in which the initial drift, d1, continues to contribute to the overall observed drift and succesfully captures the global behavior. This can be written as

$$D\left(t\right) = \begin{cases} d\_1\left(t\right) + z & \text{if } t < t\_2\\ d\_1\left(t\right) + d\_2\left(t - t\_2\right) + z & \text{if } t \ge t\_2 \end{cases},\tag{8}$$

where t<sup>2</sup> is the time at which the secondary drift, d2, begins and z is the offset at t = 0. Rather than assuming that the drift is similar for all recordings and attempting to construct a standard curve, we assume that a few points in each recording are highly informative of the drift in the baseline.

#### Drift Delimitation

To fit the drift model to the corresponding portion of the recorded signal, it is necessary to delimit the boundaries of TR by identifying the start of activation and end of deactivation. Firstly, we will aim to estimate the point in time at which the TR of a recording begins. Experimental TRs rarely activate abruptly and simultaneously for a field of imaged cells. Many factors play a role in the heterogeneity of responses observed such as variable diffusion fronts of applied agonist or heterogenous receptor expression among cells. These effects can manifest as very gradual rises or additional small amplitudes prior to a certain activation threshold being surpassed and a rapid activation phase being observed (**Figures 2C,D**). Therefore, analyzing the activation times of all components of the biological unit manually can be subjective. While a simple threshold value in the signal can be effective in detecting activation, the choice of the threshold requires some knowledge of the amplitude of the noise σ and

is complicated by the presence of drift in the signal. Instead, we determine the end points of the time intervals dominated by the drift through statistical analysis of an estimate of the first derivative of the signal uˆ (t). In other words, to distinguish the drift from TR, the derivative of the latter must change in a way that is more statisically significant than that of the former. The methodology used to obtain the estimate for the derivative is detailed in Section Noise Characterization.

Using the estimated time derivative, we aim to determine (i) the earliest possible time of activation t i 0 act (defined as the last time point exhibitng a significant increase in the first derivative before it reaches its maximum value), (ii) the time at which the activation reaches its maximum value timax , and (iii) the time at which the deactivation ends and the signal is once again dominated by the drift t i 0 end .

i Assuming that the estimated first derivative during activation reaches a local maximum, we can find the most significant local maxima of uˆ at the location

$$\mathbf{i}\_{\text{local}} = \left\{ \begin{aligned} \mathbf{i} : \hat{\boldsymbol{u}}\_{\text{i}} \succ & \text{std}\left(\hat{\boldsymbol{u}}\right), \hat{\boldsymbol{u}}\_{\text{i}} \succeq \hat{\boldsymbol{u}}\_{\text{i}} \dots \operatorname{and} \hat{\boldsymbol{u}}\_{\text{i}} \succeq \hat{\boldsymbol{u}}\_{\text{i}+1} \right\}, \end{aligned}$$

where we denote the first significant local maximum by i 0 local = min {ilocal}. By focusing on the portion of the signal containing the first drift and the activation phase of the TR, we restrict our attention to the time interval in which the first derivative uˆ (t) is non-negative to disentangle the effects of activation and drift in the data. In other words, we restrict our analysis to h t i 0 + , t i 0 locali , where the derivative is non-negative and

$$\mathfrak{i}\_+^0 = \min\_{i \in \mathcal{0}, \dots, \mathcal{i}\_{\text{local}}^0} \left\{ i : \hat{\mathfrak{u}}\_r \succeq 0 \quad \forall r = i, \dots, \mathcal{i}\_{\text{local}}^0 \right\} \dots$$

Without prior knowledge about the sign of the derivative of the baseline drift, we cannot conclude that i 0 <sup>+</sup> corresponds to the beginning of activation. To resolve this issue, we employ a statistical test to determine the likely time at which the activation occurs, located at the index

$$\hat{u}\_{\text{act}}^0 = \max\_{\mathbf{i} \in \mathbf{i}\_{+}^0, \dots, \mathbf{i}\_{\text{local}}^0} \left\{ \hat{\imath} : \hat{\mu}\_{\text{i}} < \text{std} \left( \left\{ \hat{\mu}\_{\text{i}\_{+}^0}, \dots, \hat{\mu}\_{\text{i}\_{\text{local}}^0} \right\} \right) \right\} \ .$$

This methodology, based on the properties of the derivative around its first significant local maximum, generally picks out the first visually unambiguous activation (**Figure 2C**). As a result, it may be necessary to trim recordings where there are (large amplitude) artifacts prior to the activation of interest.

ii. TRs may be produced by the action of multiple active units (e.g., different receptor species) within the biological system under consideration, each having distinct properties and activation times. This leads to multiple delayed activations taking place over a broad range of time (**Figure 2D**). Due to the superposition of the drift in the baseline with these responses, it is entirely possible for recordings to be contaminated by strongly decreasing drift and for the expected maximum TR to not coincide with the actual maximum of the data. Thus, we have developed a method to search for the visually most likely point at which the TR reaches its maximum in the presence of a drift. For each one of the Nlocal significant local maxima of the first derivative along the activation phase, we find the location of the previous local minimum of the TV estimate of the data using

$$i\_{local}^{min} = \left\{ \max\_{n \le r} \{ n : \hat{u} \le 0 \} : r \in \mathbf{i}\_{local} \right\}$$

as well as the location of the next local maximum at

$$i\_{local}^{max} = \left\{ \min\_{n \ge \, : \, r} \left\{ n : \hat{\mu} \le 0 \right\} : r \in \mathbf{i}\_{local} \right\}.$$

From the positions of the local extrema of the data, we can estimate the value of the baseline drift from each local minimum to the next local maximum using the linear extrapolation

$$\nu\_{-}=\left\{ \begin{array}{c} \left( \boldsymbol{A}\hat{\boldsymbol{u}}\right)\_{q} + \left( \boldsymbol{t}\_{r} - \boldsymbol{t}\_{q} \right) \frac{\hat{\boldsymbol{u}}\_{r-1} + \hat{\boldsymbol{u}}\_{r}}{2} \\ \boldsymbol{r} = \left( \boldsymbol{i}\_{local}^{\max} \right)\_{s}, s \in \{1, \dots, N\_{local}\} \end{array} \right\} \tag{9}$$

where A is the operator of antidifferentiation with Au ≈ Ftrue (see Equation 1). Based on this, we then estimate the average rate of activation for each significant local maximum of the derivative according to

$$\mu = \left\{ \begin{array}{ll} \frac{(A\hat{u})\_r - (\upsilon\_-)\_s}{t\_r - t\_q} & : q = (i\_{local}^{min})\_s, r = (i\_{local}^{max})\_s, \\ & s \in \{1, \ldots, N\_{local}\} \end{array} \right\}$$

and select the first local maximum at the location

$$\mu\_{\text{max}}^0 = \min \left\{ \begin{array}{l} r : \mu\_r \succeq mean \left( \mu \right) - \mathfrak{d} \cdot \text{std}(\mu) \text{ and } \upsilon\_r \\ > mean \left( \upsilon \right) - \text{std} \left( \upsilon \right) \end{array} \right\}.$$

which has an average rate and magnitude within a statistically acceptable range (that excludes small outliers). t i 0 max is the time point at which the derivative reaches a local maximum. It may differ from the one that corresponds to the local maximum immediately following the first activation time point t i 0 act (**Figure 2D**). This is because i 0 act depends solely on the derivative around its first local maximum, while i 0 max takes into account an approximation to the average rate of change around all local maxima; the local maximum after i 0 act should only differ from i 0 max when there is a succession of activations and the first does not have the largest rate of activation.

Starting from this local maximum of the derivative, we seek the location of the first point in time t i end max where the change in the signal drops below the estimated noise level σ− (see Section Dynamic Determination of Total-Variational Parameters)

$$\mathbf{f}\_{\text{max}}^{\text{end}} = \min\_{\mathbf{i} \in \mathbf{i}\_{\text{max}}^0, \dots, \mathbf{N}} \left\{ \begin{array}{l} i : \hat{u}\_{\text{i}} \cdot (t\_{\text{i}} - t\_{\text{i}} - 1) < \sigma\_{-} \right\}, \tag{10}$$

i

Mackay et al. Characterization of Dynamic Signals

and thus isolate the time interval in which the most significant activation occurs. Having isolated the most significant activation, we finally arrive at the location of the first estimate of the time where the response reaches its maximum

$$\mathbf{i}\_{\max} = \min \left\{ \underset{\mathbf{i} \in \mathbf{i}^{0}\_{\max}, \dots, \mathbf{i}^{\text{end}}\_{\max}}{\arg \max} \left( A \hat{\boldsymbol{u}} \right)\_{i} \right\}.$$

iii. Having identified the time at which activation is likely to begin t i 0 act , we can now assume that the signal prior to this point is the drift. If the recording is of a sufficiently long duration, the response will return to baseline and the end of the recording should once again be dominated by the (secondary) drift (**Figure 2A**). To account for this drift, we need to estimate the time duration of deactivation. This is done in a manner nearly identical to how we determined the first time of activation t i 0 act . However, due to the possibility of having MPR after the initiation of TR (**Figure 2E**), we cannot restrict ourselves to time intervals in which the first derivative is non-positive. To solve this issue, we define a set of time points after the presumed maximum of TR, located at idecay = {imax, ... , N}, and construct a measure

$$n\_{end} = \max\_{n \in \{3, 2, 1, 0\}} \left\{ \begin{array}{l} n: \exists \left| \hat{u}\_{\mathrm{i}} \right| - \operatorname{mean} \left( \left| \hat{u}\_{\mathrm{i}\_{\mathrm{decay}}} \right| \right) \\ \qquad > n \cdot \operatorname{std} \left( \left| \hat{u}\_{\mathrm{i}\_{\mathrm{decay}}} \right| \right), i \in \mathrm{i}\_{\mathrm{decay}} \right\} \right\}$$

to quantify how far the derivative deviates from its mean during the decay. This measure allows us to robustly detect the location of the time point where TR is negligible

$$\hat{\mu}\_{end}^{0} = \min\_{\mathbf{i} \in \mathbf{i}\_{\text{decay}}} \left\{ \mathbf{i} : \left| \hat{\mathbf{u}}\_{\mathbf{i}} \right| - \text{mean} \left( \left| \hat{\boldsymbol{u}}\_{\mathbf{i}\_{\text{decay}}} \right| \right) > n\_{\text{end}} \cdot \text{std} \left( \left| \hat{\boldsymbol{u}}\_{\mathbf{i}\_{\text{decay}}} \right| \right) \right\}.$$

The time point t i 0 end is then used to determine the start of the new drift.

#### Drift Fitting

To isolate the TR from the drift, it is necessary to generate an accurate fit for the drift. This is achieved by employing a succession of least square fits that progressively incorporates more data and models that account for additional components of the signal (including activation and deactivation phases of TR). The first step in this successive least-square-fitting method is to obtain preliminary estimates of the parameters θdrift = [a1, τ1, m1, a2, τ2, m2] of the drift model. The MATLAB implementation of the non-linear least squares method (Marquardt, 1963; Moré, 1978) is used. More specifically, we initially minimize the error function between the drift model and the data

$$S\_{drift}^{0} \left( \theta\_{drift} \right) = \sum\_{k} \left( D \left( t\_k; \theta\_{drift} \right) - F\_k \right)^2,\tag{11}$$

where the set of indices k is defined by k = 1, ... , i 0 act S i 0 end, ... , N T j (i 0 act and i 0 end are as defined in Section Drift Delimitation while j is defined in Section Removal of Noise Spikes). We denote the first estimate of parameters obtained from the minimizaiton of Equation (11) by θ 0 drift. There is no guarantee, however, that the drift model described by Equation (8) can accurately represent the actual drift in the baseline. The emergence of drifting trends which are not related to the onset of TR (**Figure 2F**) may require the inclusion of more than two functions of the type described in Equation (7), yet the decision to include more drift terms, or alternatively to truncate the signal, would require manual intervention. To circumvent this limitation, we perform a second fit where the individual terms in Equation (11) are weighed according to a weight function w, and the first derivative of the data is taken into account. Although the set of parameters θ 0 drift will be able to produce the "general" trends in the data before and after TR, the presence of multiple drifting trends necessitates the use of the derivative estimate, uˆ, to identify the segment(s) of the signal that actually follow the trend described in Equation (8) and remove the effects of others. The inclusion of uˆ, however, in the sum of squared errors can lead to erroneous fittings when the drift is not well represented by Equation (8). The weight function w alleviates this problem by preventing the linear and exponential trends in Equation (7) from growing unjustifiably large.

To determine w, we require that it approaches zero when the match between uˆ and D˙ t; θ 0 drift is minimal. To achieve this, we choose w to depend on y = <sup>u</sup><sup>ˆ</sup> <sup>−</sup> <sup>D</sup>˙ t; θ 0 drift  as follows

$$w\_k = \exp\left(\frac{-\nu\_k^2}{mean\ (\nu\_k^2)}\right).$$

We then apply the non-linear least squares fitting procedure to minimize the error function

$$\mathcal{S}\_{drift}\left(\theta\_{drift}\right) = \sum\_{k} \boldsymbol{\nu}\_{k} \left[ \begin{aligned} &\left(\boldsymbol{D}\left(\boldsymbol{t}\_{k}; \theta\_{drift}\right) - \boldsymbol{F}\_{k}\right)^{2} \\ &+\left(\hat{\boldsymbol{u}}\_{k} - \dot{\boldsymbol{D}}\left(\boldsymbol{t}\_{k}; \theta\_{drift}\right)\right)^{2} \end{aligned} \right], \end{aligned}$$

and use θ 0 drift as an initial condition for the fitting procedure.

#### Activation Fitting

For the activation phase of the response, we use the model

$$\begin{split} g\_{\rm act}(t; A\_{\rm act}, \beta, n\_{\rm act}, m\_{\rm act}) &= A\_{\rm act} \frac{t^{n\_{\rm act}}}{t^{n\_{\rm act}} + \beta^{n\_{\rm act}}} \\ &+ m\_{\rm act} \int\_{0}^{t} \frac{\varkappa^{n\_{\rm act}}}{\varkappa^{n\_{\rm act}} + \beta^{n\_{\rm act}}} d\chi. \end{split}$$

where Aact is the maximum of the Hill function, nact is the Hill coefficient, β is the time at which the the Hill function reaches its half maximum, and mact is the slope of the quasilinear function that accounts for the trend which dominates at the end of activation. The Hill function allows for a rapid rise and switch in convexity due to many biological units being activated at once, whereas the linear trend allows for a delayed and slower rise induced by more units being progressively recruited into the generation of the signal. The values of Aact and mact determine the magnitude of these two trends, whereas nact and β affect primarily the timescales of switching between the two. The time at which the activation phase begins is denoted as ton, and its estimate is confined to the time interval h t i 0 act , <sup>t</sup>imax <sup>i</sup> . In order to obtain a preliminary estimate of the values of these parameters and those of the drift function, θact = [ton, Aact, β, nact, mact, θdrift], we minimize the sum of square errors between the activation data and the model along with their derivatives, given by the function

$$\mathcal{S}\_{\text{act}}\left(\theta\_{\text{act}}\right) = \sum\_{j} \left[ \frac{\phi^2 \left( \mathcal{F}\_j - \mathcal{G}\_{\text{act}}\left(t\_j; \theta\_{\text{act}}\right) \right)^2}{+\phi \left( \hat{u}\_j - \dot{\mathcal{G}}\_{\text{act}}\left(t\_j; \theta\_{\text{act}}\right) \right)^2} \right] + \mathcal{S}\_{drift}\left(\theta\_{drift}\right) \tag{12}$$

where φ is the noise-to-signal ratio as defined in Section Dynamic Determination of Total-Variational Parameters, and

$$\begin{aligned} &= \begin{cases} G\_{\text{act}}\left(t; \theta\_{\text{act}}\right) \\ \qquad \qquad \qquad \qquad \begin{cases} D\left(t; \theta\_{\text{act}}\right) + \text{g}\_{\text{act}}\left(t - t\_{\text{on}}; \theta\_{\text{act}}\right) & \text{if } t\_{\text{on}} \le t < t\_{\text{max}} \\ D\left(t; \theta\_{\text{act}}\right) & \text{otherwise} \end{cases} \end{aligned} $$

is our activation data model that takes into account the effect of the drift D (defined in Equation 8) on the perceived activation. To abrogate the influence of noise spikes on measured parameters, the sum of squares in Equation (12) is evaluated at the set of indices j (defined in Section Dynamic Determination of Total-Variational Parameters) which are not dominated by noise. Moreover, we use θˆact drift to denote the set of drift parameters obtained from the minimization of Equation (12).

#### Signal Detection

In order for the algorithm to resolve whether there is a discernable TR present in Ftrue(t), three conditions must be satisfied: (i) ilocal must be a non-empty set, (ii) the initial drift estimate

$$D\left(t; \operatorname\*{arg\,min}\_{\boldsymbol{\theta} \in \left\{\hat{\theta}\_{d\text{right}}, \hat{\theta}\_{d\text{right}}^{\text{act}}\right\}} \operatorname{S}\_{d\text{right}}(\boldsymbol{\theta})\right)$$

must be below the TV data estimate by a detection threshold of 4σ− for at least six data points, and (iii) the difference between the TV data estimate and the initial drift estimate must not be a strictly increasing function of time after t 0 max. These three criteria allow for the detection of the TR and further analysis of its characteristics [Section Transient Response (TR) Model].

To evaluate the signal detection performance of the algorithm, 450 individual traces of ATP-induced calcium responses were used as a validation set. Manual results were then compared to automated detection of TRs to assess extent of agreement between the two methods. Classical signal detection nomenclature (i.e., true positive or negative and false positive or negative) was intentionally avoided due to lack of certainty in determining the true presence of TRs in more ambiguous cases. We found that the automated and manual methods agreed in detecting a TR in 88.1% of cases, and they disagreed in 11.9% of cases (**Figure 3A**). Further dissection of these results showed that 64.7% of disagreement arose from the algorithm reporting an absence of TR while visual evaluation suggested otherwise (**Figure 3A**), indicating that the algorithm has a tendency to be more conservative than user-mediated assessments.

To determine whether there were particular types of recordings that contributed to these disagreements, time-series traces were qualitatively divided into two groups: Clean signals with clearly defined responses were classified as "pronounced" (**Figure 3B**, top), and signals containing ambiguous signals with low signal to noise ratio or strong drift were classified as "obscure" (**Figure 3B**, bottom). The total frequency of disagreement was 3.6 times greater for obscure signals compared to those classified as pronounced (17.4 vs. 4.8%; **Figure 3C**). Regardless of the group, the algorithm signal detection remained more conservative compared to the manual method.

#### Transient Response (TR) Model

Transient cellular responses are generally complex with multiple time scales and amplitudes. They may, in fact, exhibit prolonged MPRs superimposed on a more acute response (see **Figure 2E**). For these reasons, a complete characterization of all possible TRs is unlikely to be attainable. In order to remain as general as possible, we propose modeling TR as a continuously differentiable piece-wise defined function that first increases during the activation phase and subsequently decreases during the deactivation phase. Due to the large number of parameters required for such a description and the automated nature of our fitting procedure, we decompose the fitting of the whole TR into a sequential fitting of the activation phase alone followed by a fit of both phases simultaeously. This yields significantly more reliable results with faster convergence rates over a wide gamut of input data, because it allows for information obtained during preliminary simple fits to be used in a progressively more complex manner. In order to capture the complex fluorescence response generated by the spatially separated units in a live cell, we use a combination of Hill functions and quasi-linear functions generated by the integral of the corresponding Hill functions.

#### Response Fitting

Following the least squares fitting of the activation data, we now seek to fit the entire recording (with the drift and TR) using a continuously differentiable function. A decreasing Hill Function is used to describe deactivation phase of the signal in a manner similar to Equation (11), as follows

$$g\_{de}\left(t; A\_{de}, \boldsymbol{\chi}, \boldsymbol{n\_{de}}\right) = A\_{de}\frac{\boldsymbol{\chi}^{n\_{de}}}{t^{n\_{de}} + \boldsymbol{\chi}^{n\_{de}}} + m\_{de}\int\_{0}^{t} \frac{\boldsymbol{\chi}^{n\_{de}}}{\boldsymbol{\mathscr{X}}^{n\_{de}} + \boldsymbol{\chi}^{n\_{de}}}d\boldsymbol{\mathscr{X}}\_{t}$$

where Ade is the amplitude of the Hill function, nde is the Hill coefficient, γ is the time at which the Hill function reaches its half maximum, and mde is the slope of the quasi-linear function that accounts for the trend dominating at the beginning

FIGURE 3 | Signal detection performance of algorithm. (A) Signal detection analysis demonstrates relative selectivity and sensitivity of the algorithm as compared to user-mediated manual detection. "Agreement" refers to instances where the detection of a response was consistent between algorithm and manual methods. "Disagreement" refers to the contrary. "Present" indicates cases where a response was detected using the indicated method, while "absent" refers to cases where a response was not detected. (B) Representative time-series for pronounced (top) and obscure (bottom) signals. (C) Analysis of disagreement cases for pronounced and obscure signals. Automated detection, auto; manual detection, manu; sample size, n.

of deactivation. The time when the response switches to the deactivation function is denoted by tde. The parameter mde is chosen such that the response function returns to zero by the end of the recording and is given by

$$m\_{de} = \frac{g\_{act}\left(\ell\_{act}; A\_{act}, \beta, n\_{act}, m\_{act}\right) - A\_{de}\frac{\mathbf{y}^{n\_{de}}}{(t\_N - t\_{de})^{n\_{de}} + \mathbf{y}^{n\_{de}}}}{\int\_0^{\tau} \frac{\mathbf{y}^{n\_{de}}}{\mathbf{x}^{n\_{de}} + \mathbf{y}^{n\_{de}}}d\mathbf{x}}$$

where ℓact = tde − ton is the time duration of the activation phase of the response. If differentiability is not enforced at the point tde, where the two functions gact and gde meet, then the fitting may contain sharp edges indicative of unconverged solution. To solve this issue, the continuity of the first derivative of these two functions, particularly at tde, is achieved through a third-order Hermite spline (Traub, 1964) on the time interval [tde − ςact, tde + ςde], where ςact and ςde are two parameters that must satisfy ςact < ℓact and ςde < 2γ (see Data Sheet 2 in Supplementary Materials). The overall response model is thus given by

$$\begin{aligned} S\_{\text{resp}}\left(\theta\_{\text{resp}}\right) &= \phi \sum\_{j} \begin{bmatrix} \phi\left(\mathbf{f}\_{j} - \mathbf{G}\_{\text{resp}}\left(\mathbf{t}\_{j}; \theta\_{\text{resp}}\right)\right)^{2} \\ &+ \left(\hat{\mathbf{u}}\_{j} - \hat{\mathbf{G}}\_{\text{resp}}\left(\mathbf{t}\_{j}; \theta\_{\text{resp}}\right)\right)^{2} \\ &+ \kappa \phi\left(D\_{2}\left(\mathbf{t}\_{k} - \mathbf{t}\_{2}; \mathbf{a}\_{2}, \mathbf{r}\_{2}, m\_{2}\right)\right)^{2} \end{bmatrix} + \frac{\lambda}{\phi^{2}} S\_{drift}\left(\theta\_{drift}\right), \text{(14)} \end{aligned}$$

to obtain the fitting, where κ is a parameter quantifying the apparent coherence between the drift and reponse models (D and gresp), given by

$$\kappa = \left| \frac{(A\hat{\boldsymbol{\mu}})\_N - \operatorname{g\_{act}}(t\_{\max} - t\_{\text{on}}; A\_{\text{act}}, \boldsymbol{\beta}, \boldsymbol{n}\_{\text{act}}, \boldsymbol{m}\_{\text{act}})}{(A\hat{\boldsymbol{\mu}})\_N - \min\_{\mathbf{i} \in \mathbf{i}\_{\text{act}}^0, \dots, \boldsymbol{i}\_{\text{max}}} (A\hat{\boldsymbol{\mu}})\_{\mathbf{i}}} \right|, \ldots$$

λ is a parameter defined by

$$\lambda = \frac{\max\_{\mathbf{i} > \mathbf{i}\_{\text{end}}^{0}} |\hat{u}\_{\mathbf{i}}|}{\max\_{\mathbf{i} \in \mathbf{i}\_{\text{act}}^{0}, \dots, \mathbf{i}\_{\text{max}}}},$$

$$g\_{\rm rep}\left(t;\theta\_{\rm rep}\right) = \begin{cases} g\_{\rm act}\left(t - t\_{\rm on};\theta\_{\rm rep}\right) & \text{if } t\_{\rm on} \le t < t\_{\rm de} - \xi\_{\rm act} \\ p\_{\rm Hermite}\left(t - \left(t\_{\rm de} - \rho\_{\rm act}\right); \theta\_{\rm rep}\right) & \text{if } t\_{\rm de} - \xi\_{\rm act} \le t < t\_{\rm de} + \xi\_{\rm de} \\ g\_{\rm de}\left(t - t\_{\rm de}; \theta\_{\rm rep}\right) & \text{if } t\_{\rm de} + \xi\_{\rm de} \le t \end{cases}$$

where θresp = [θact, tde, ςact, ςde, Ade, γ , ndeθdrift]. Given the response model, we define the global data model as

$$G\_{\rm rep}\left(\mathbf{t};\theta\_{\rm rep},\theta\_{\rm drift}\right) = \begin{cases} D\left(\mathbf{t};\theta\_{\rm rep}\right) + \mathbf{g}\_{\rm rep}\left(\mathbf{t}\_{\rm i};\theta\_{\rm rep}\right) & \text{if } t\_{\rm on} \le \mathbf{t} \\\ \qquad D\left(\mathbf{t};\theta\_{\rm rep}\right) & \text{otherwise} \end{cases} \tag{13}$$

and minimize the error function

and the values for the parameters Aact, nact, β, mact, and ton are taken from the activation fitting. The first two terms in the sum of squares in Equation (14) are analgous to those in Equation (12), whereas the third term minimizes the AUC for the second drift function D2. By including the coefficient κ in this third term, however, allows D<sup>2</sup> to become more significant when there is a large mismatch in the value of the baseline between t i 0 act and t<sup>N</sup> (that cannot be explained by d1).

#### Activation Parameter Validation: tonset, t10–90%, and Amplitude

The three parameters, tonset, t10–90%, and amplitude, are considered together because they describe what happens at the activation phase of the TR, with no regard for the deactivation phase or MPR. The time at which a TR is discernable from baseline is defined as tonset and is estimated directly as the parameter ton of the response function gresp. There was a strong linear agreement between the manual and automated-estimates, with a linear slope of 1.06 and a correlation coefficient (r 2 ) of 0.94 (**Figure 4A**, left). On average, automated estimates of tonset were 4.6% greater than manual estimates, with limits of agreement ranging between −10 and 19% difference (**Figure 4A**, right). The t10–90% is also estimated numerically from the response function gresp. Using the response function allows to overcome issues arising from the subsampling of rapid dynamics by numerically evaluating on a time grid 10 times finer than the input times. Manual estimation of this parameter is contingent upon accurate estimation of the baseline and peak occurrences, both of which present potential sources of error, particularly for noisy signals with drift. The relationship between manual and automated estimates had a linear slope of 0.77 and a correlation coefficient of 0.77 (**Figure 4B**, left). The higher degree of scatter away from the line of exact linear correlation is reflected by the wider Bland–Altman interval of agreement, ranging from −89 to 54% difference between manual and automated estimates. Overall, there was a −17% difference between all paired estimates of t10–90%, revealing that t10–90% was manually overestimated compared to the automated estimates (**Figure 4B**, right). Amplitude estimates obtained by the manual and automated methods had a strong linear relationship with a slope of 1.02 and a correlation coefficient of 0.84 (**Figure 4C**, left). The limits of agreement, ranging from −26 to 25%, were narrow with a mean percent difference of −0.3% between all paired estimates of amplitude (**Figure 4C**, right).

#### TR Parameter Validation: AUC, FWHM, and τdecay

Due to the inherent differences between TRs and MPRs and the approach taken with this algorithm, the AUC, FWHM, and tdecay are limited to describing TRs. Nevertheless, these parameters will be also reported in the presence of MPRs where they should be interpreted with the following considerations. (i) if a MPR is superimposed on a TR, the reported parameters describe the underlying TR, not the superimposed MPR. (ii) if TR presence is not detectable and MPR demonstrates a purely oscillatory response, the reported parameters characterize the first peak only. With these considerations in mind, manual evaluation of TR parameters was performed with a variety of signals, including MPRs. AUC estimates are manually determined using a geometric estimation of the area of a triangle whose vertices are at the start, peak, and end of the TR. Algorithmically, AUC was evaluated from the area under gresp, using the trapezoidal rule (Rice, 1973) implemented in MATLAB using "trapz()." Comparing the manual and automated estimates demonstrated a linear relationship with a slope of 0.85 and a correlation

coefficient of 0.79 (**Figure 5A**, left). On average, automated estimates were 4.6% larger than manual estimates with an interval of agreement ranging between −38 and 47% difference (**Figure 5A**, right). Considering the geometric-approach used for manual-estimation of AUC values, it is reasonable to assume that the error arose from manual limitations.

Due to the difficulty in determining the precise time at which the signal returns to its former baseline, it would be challenging to manually describe the duration and decay characteristics of the

response. Incomplete recordings and background drift are largely responsible for generating such behavior. The full-width halfmax (FWHM) is defined as the time elapsed between the two halfmax coordinates of a peak. Our analysis of FWHM revealed that the linear relationship between manual and automated-estimates was strong, with a correlation coefficient of 0.87 and slope of 0.95 (**Figure 5B**, left). The Bland Altman analysis demonstrated that the agreement interval ranged from −20 to 22% difference with a mean percent difference of 0.85% between all paired estimates (**Figure 5B**, right).

Finally, to manually estimate the decay constant, the time of the inflection point (ρ) is visually estimated and the general trend of the data following ρ is represented by a mono-exponential decay. The decay constant is then determined by the time it takes for the signal to reduce to approximately 37% of its initial value (1/e). Algorithmically, the time ρ is determined by solving for the inflection point in the deactivation function, given by

$$
\rho = t\_{de} + \nu \left( \frac{n\_{de} - 1}{n\_{de} + 1} \right)^{1/n\_{de}}.
$$

The data following ρ is then fit to a mono-exponential decay function using least squares, to determine the time constant of decay. The slope of the linear agreement between manual and automated-estimates was 0.81 and the correlation coefficient was 0.75 (**Figure 5C**, left). The Bland Altman analysis revealed a strong systematic bias of −14% difference, with an agreement range of −63 to 34% difference, signifying that manual efforts to estimate the decay constant consistently overshot the values reported by the algorithm (**Figure 5C**, right).

#### Multi-Peaked Responses

To isolate the characteristic parameters of MPRs that are frequently superimposed on TRs (**Figure 2D**), the TR model gresp must be refined to serve as a non-stationary baseline around which the MPR will oscillate. This refinement is necessary because it is often the case that the TR model gresp will produce sub-optimal fittings where the data deviates significantly from the TR fitting. Therefore, to accurately characterize the TR and quantify the properties of truly oscillatory MPRs, it is first necessary to adapt the least squares fitting procedure of Section Response Fitting to remove the effects of data points not well represented by Gresp. This is done by first identifying large deviations representing MPRs from the Gresp -fitting (obtained by minimizing Equation 14), and then reweighing the sum of squares in Equation (14) to remove the influence of those deviations from the fits. We subsequently perform a secondary fitting of Gresp to determine more accurately the baseline, delineating the TR, where the MPR-associated deviations originate from. Finally, we analyze the MPRs by determining whether they represent oscillations and, if so, quantify their properties.

To identify these MPR-associated deviations from the newly defined baseline, we first employ the MATLAB "findpeaks()" function. This finds the peaks and troughs of the significant deviations in the TV data estimate, Auˆ (t), from the TR estimate, <sup>G</sup>resp t; θˆ resp , truncated at its half-maximums, where θˆ resp is the optimal parameter set obtained from minimizing Equation (14). This truncation permits for the possibility that the onset of TR coincides with the first peak of the MPR-associated deviations. The implementation of peak-finding algorithm, on the other hand, allows for the specification of minimum heights and timing between peaks, set to be 6σ− and 5 s, respectively. The algorithm yields the heights (E peak , E trough), the FWHM (ξ peak , ξ trough), and the times (t˜ peak , t˜ trough) of significant peaks and troughs of the deviations, respectively (see **Figure 2D**). In total, there are N˜ = Npeak + Ntrough of these deviations, including Npeak peaks Mackay et al. Characterization of Dynamic Signals

and Ntrough troughs. If N˜ ≤ 2, then the only deviation in the signal is the TR and the algorithm can terminate. Without prior knowledge of the nature of the MPR-associated deviations, it is very difficult to determine whether they result from trends which are above, below, or symmetric to the TR. To resolve this issue, we assume that the estimated baseline from Section Response Fitting underlies the signal in the absence of deviations. To incorporate this assumption algorithmically, we define two bias parameters based on the relative heights of the first peak and trough, as follows

$$\begin{array}{rcl} o\_{\text{peak}} & = & \exp\left[ -\left( \frac{E\_1^{\text{peak}}}{E\_1^{\text{ trough}}} \right)^4 \right] \quad \text{and} \\ o\_{\text{twoglh}} & = & \exp\left[ -\left( \frac{E\_1^{\text{rough}}}{E\_1^{\text{peak}}} \right)^4 \right] . \end{array}$$

These quantities are then used to calculate weighting functions for the data based on the properties of peaks

$$\begin{split} \pi\_{\mathrm{i}}^{\mathrm{peak}} &= \sum\_{r=1}^{N\_{\mathrm{peak}}} o\_{\mathrm{peak}} \exp \left( -\phi \left( \frac{o\_{\mathrm{peak}} \left( \mathbf{t}\_{\mathrm{i}} - \widetilde{\mathbf{t}}\_{r}^{\mathrm{peak}} \right)}{\xi\_{r}^{\mathrm{peak}}} \right)^{2} \right) \\ &+ \left( 1 - o\_{\mathrm{peak}} \right) \left( 1 - \exp \left( -\phi \left( \frac{o\_{\mathrm{peak}} \left( \mathbf{t}\_{\mathrm{i}} - \widetilde{\mathbf{t}}\_{r}^{\mathrm{peak}} \right)}{\xi\_{r}^{\mathrm{peak}}} \right)^{2} \right) \right) \end{split}$$

and troughs

$$\begin{split} & \pi\_{\mathrm{i}}^{\mathrm{traughter}} \\ &= \sum\_{r=1}^{N\_{\mathrm{traughter}}} o\_{\mathrm{traughter}} \exp\left(-\phi \left(\frac{o\_{\mathrm{traought}}\left(t\_{\mathrm{i}} - \widetilde{t}\_{r}^{\mathrm{traought}}\right)}{\xi\_{r}^{\mathrm{traughter}}}\right)^{2}\right) \\ &+ \left(1 - o\_{\mathrm{traought}}\right) \left(1 - \exp\left(-\phi \left(\frac{o\_{\mathrm{traought}}\left(t\_{\mathrm{i}} - \widetilde{t}\_{r}^{\mathrm{traought}}\right)}{\xi\_{r}^{\mathrm{traughter}}}\right)^{2}\right)\right). \end{split}$$

The weighting functions π peak i , π trough i quantify the relative reliability of the data around each deviation based on how close it is to the fitting function Gresp and on its duration. We can also assess the reliability of each time point of the recording (including the TR, the drift, and any MPR-associated deviations present) by how well its derivative matches G˙ resp t; θˆ resp . This is done using another weighting function, defined by

$$\Gamma\_{\mathbf{i}} = \exp\left(-\left(\frac{\hat{u}\_{\mathbf{i}} - \dot{G}\_{\mathrm{resp}}\left(\mathbf{t}; \hat{\theta}\_{\mathrm{resp}}\right)}{2\,\mathrm{std}\left(\hat{u}\_{\mathbf{i}} - \dot{G}\_{\mathrm{resp}}\left(\mathbf{t}; \hat{\theta}\_{\mathrm{resp}}\right)\right)}\right)^{2}\right).$$

We combine these weighting functions using the criterion that for a data point to be reliable, it must have either a large value of π peak or π trough, and a large value of Ŵ. It is implemented in the weighting function , as follows

$$\Omega\_{\mathbf{i}} = \Gamma\_{\mathbf{i}} \left( \pi\_{\mathbf{i}}^{\text{peak}} + \pi\_{\mathbf{i}}^{\text{ trough}} - \min\_{\mathbf{i}} \left( \pi\_{\mathbf{i}}^{\text{peak}} \right) - \min\_{\mathbf{i}} \left( \pi\_{\mathbf{i}}^{\text{traugh}} \right) \right).$$

With <sup>i</sup> , we can fit the TR reliably in the presence of significant deviations from the model of Equation (13). This is done by minimizing the error function

$$\begin{split} S\_{\mathrm{resp}}^{\Omega} \left( \theta\_{\mathrm{resp}} \right) &= \phi \sum\_{j} \left[ \begin{array}{c} \phi \,\Omega\_{j}^{2} \left( \mathrm{F}\_{j} - \mathrm{G}\_{\mathrm{resp}} \left( t\_{j}; \,\theta\_{\mathrm{resp}} \right) \right)^{2} \\ + \Omega\_{j}^{2} \left( \hat{u}\_{j} - \mathrm{G}\_{\mathrm{resp}} \left( t\_{j}; \,\theta\_{\mathrm{resp}} \right) \right)^{2} \\ + \kappa \phi (D\_{2} \left( t\_{k} - t\_{2}; \,a\_{2}, \tau\_{2}, m\_{2} \right))^{2} \end{array} \right] \\ &+ \frac{\lambda}{\phi^{2}} S\_{d\mathrm{rff}} \left( \theta\_{d\mathrm{rff}} \right) . \end{split} \tag{15}$$

#### Identifying Coherent Oscillations

Not all MPRs correspond to periodic oscillations (Thurley et al., 2014). To address this, the algorithm reports two sets of MPR parameters, the first to describe all the peaks detected within a MPR, and the second to describe the subset of coherent peaks present within the same MPR. This section focuses on how the subset of coherent oscillatory peaks is identified. We use a clustering algorithm which is an unsupervised learning technique that enables for the identification of natural groupings or patterns with a defined data set. By minimizing Equation (15), we obtain the most reliable estimate of the TR (specified by the model Gresp and its optimal parameter set θˆ resp), which we take to be the baseline of the MPR-associated deviations. Given the estimate <sup>G</sup>resp t; θ resp , we repeat the peak finding steps detailed in Section Multi-Peaked Responses. To determine whether or not the detected deviations represent oscillations, we use a Gaussian mixture model clustering technique. It groups together (in clusters) peaks and troughs with comparable periods, T (determined by the difference between two consecutive peak or trough times; i.e., t˜ peak or t˜ trough) and FWHM, ξ . Two adjacent deviations are deemed to be coherent oscillations if they are grouped in the same cluster. In situations where the period or FWHM are modulated throughout time, Gaussian clustering technique may not be able to cluster all coherent oscillations adequately. We therefore process clusters by defining periodand FWHM-trends for all coherent oscillations. If this trend can accurately predict the period and FWHM of the first deviation of an adjacent cluster, then both clusters are deemed to form a set of coherent oscillations. This is repeated for all pairs of adjacent clusters, progressively updating the set of coherent oscillations with those previously deemed incoherent at prior steps.

This procedure use the Expectation Maximization (EM) clustering algorithm (McLachlan and Peel, 2005) to cluster the period and FWHM of a potential oscillatory MPRs, and to determine the optimal number of clusters using gap statistic (Tibshirani et al., 2001). It allows for a reliable separation of oscillatory data from recording artifacts or non-oscillatory MPRs with visually different properties. It also yields a set of Nˆ peak peaks (Nˆ trough troughs) occurring at times tˆ peak (tˆ trough), which together represent coherent oscillations. Having determined the properties of the individual features making up the oscillations, they can be used to quantify the properties of the oscillations.

#### Characterizing Oscillatory Parameters

In order for the algorithm to report the oscillatory properties of a signal, the MPR-associated deviations must satisfy N˜ > 2, for the set of MPR parameters describing all detected peaks. If these deviations form a coherent set of oscillations, a second set of MPR parameters characterizing this coherent oscillatory behavior is also reported. In both instances, the following parameters will be reported: the number of oscillations (Nosc), the average magnitude of the oscillations (defined as E peak + E trough), the average period of oscillation (T), the standard deviation of the periodicity (σT), the total time for which the oscillations persist (defined as ℓosc = tˆ peak Nˆ peak − tˆ peak 1 ), and the mean duty cycle parameter (given by ξ peak/T ) (Smedler and Uhlén, 2014).

MPR Parameter Validation: Nosc, E, T, losc, and ξ peak/T Manual estimates of Nosc is determined by counting the number of discernable peaks within the signal. The slope of correlation was 0.85 with an r 2 -score of 0.78 (**Figure 6A**, left). The mean difference between manual and automated estimates was 15% with an interval of agreement ranging from −39 to 69% (**Figure 6A**, right). The peak magnitude of the oscillations, E, is manually estimated by the mean change between peak maxima and their subsequent trough minima, after correcting for a nonstationary baseline that is often a consequence of a concurrent TR. For most signals the non-stationary baseline can be manually estimated to be linear. However, there are a few cases where an estimate of an exponential baseline is required. The correlation between manual and automated estimates was relatively strong, with an r 2 -value of 0.92 and a slope of 1.08 (**Figure 6B**, left). The agreement analysis on the other hand revealed relatively no bias, with a mean difference of −2% and limit of agreement ranging from −48 to 44% difference (**Figure 6B**, right). The periodicity is manually estimated by the average time between adjacent peaks. The linear relationship was slightly weaker with a r 2 -value of 0.55 and slope of 0.65 (**Figure 6C**, left). The mean difference between manual and automated estimates was negligible, at only 0.3%, indicating an absence systematic bias, and the limits of agreement spanned from −54 to 54% difference (**Figure 6C**, right). The standard deviation of periodicity was obtained from the same set of periods used to estimate the mean period. The linear slope was 0.77 and the r 2 -value was 0.77 (**Figure 6D**, left). Similar to periodicity, the mean difference for the standard deviation of periodicity was a negligible −0.4%, with limits of agreement ranging from −61 to 62% difference (**Figure 6D**, right). Oscillatory persistence is chosen to describe how long oscillations are sustained within a given recording, and is estimated as the elapsed time between the first and last discernable peaks in the MPR. The correlation between manual and automated estimates was supported by a r 2 -value of 0.79 and slope of 0.85 (**Figure 6E**, left). The mean difference between paired estimates was only −5.3% with a limit of agreement between −47 and 37% difference (**Figure 6E**, right). Finally, the duty cycle is manually estimated by the ratio between ξ peak and T. ξ peak is manually determined by the mean FWHM of individual oscillatory peaks and the same T value obtained above is used to calculate xpeak /T. The linear relationship between manual and automated estimated of ξ peak /T was decidedly weak with a slope of 0.51 and r <sup>2</sup> of 0.26 (**Figure 6F**, left). The Bland Altman analysis, however, suggests that there was a systematic bias that could explain the poorer correlation results. The mean difference between manual and automated estimated was −19% with a limit of agreement between −69 and 31% (**Figure 6F**, right).

The MPR parameter validation described above focuses on the set of parameter estimates describing all the peaks in the MPR, rather than the subset of coherent oscillations. This is because manual detection of each peak in the MPR is less subjective than detecting only the coherent peaks in the MPR. Since the algorithm sub-selects the coherent oscillatory peaks from the initial set of identified deviations, the performance reported for the characterization of all peaks extends to the subset of coherent oscillations. Furthermore, as expected, the standard deviation of the periodicity is consistently lower for coherent oscillations when compare to the σ<sup>T</sup> reported for all peaks in the same MPR (i.e., more regular periodicities result in lower standard deviations).

To ensure confidence in the reported MPR parameters, users of this algorithm are urged to visually verify the quality of the signal fittings to determine whether the algorithm is characterizing their peaks of interest, as these may not always coincide with the most prevalent oscillatory component of the signal (Thurley et al., 2014). Furthermore, Nosc reported for all peaks and coherent peaks can be compared to be aware of how many peaks were omitted during the clustering step. Collectively, the information reported for MPRs is sufficient for the informed analysis of a diverse selection of MPRs, including those that exhibit coherent oscillations and those that do not.

# Application to Pathophysiology

In the context of bone physiology, the deleterious consequences of disrupting extracellular nucleotide-mediated cross talk have been highlighted by the emergence of P2 receptor knockout mouse models (Lenertz et al., 2015). P2 receptors are particularly sensitive to changes in the extracellular milieu. Consequently, P2 receptor pathophysiology is often coupled to events that influence the extracellular composition, thereby compromising processes regulated by the P2 receptor network. In particular, changes in extracellular pH alter P2 receptor function (King et al., 1997; Gerevich et al., 2007; Wildman, 2009; Langfelder et al., 2015). Such conditions arise from pathological acidosis that is commonly caused by systemic acid-base disturbances, such as metabolic or respiratory acidosis (Krieger et al., 2004; Miller, 2012; Berend et al., 2014). More localized acidifications can also be associated with tumors (Martin and Jain, 1994; Kato et al., 2013). Since the skeleton is a common metastatic site for cancer, and participates in systemic buffering of protons, the effect of acidosis on the skeletal system is of particular interest. On the cellular level, acidosis promotes the activation of osteoclasts, resulting in elevated bone resorption which manifests itself in osteoporotic phenotype (Bushinsky and Frick, 2000; Krieger et al., 2003; Ahn et al., 2012; Gasser et al., 2014). However, it

Period, σT. (E) Oscillatory persistence, losc. (F) Duty cycle, ξ peak/T. Arbitrary Units, A.U. Insets: Visual representations of parameters measured.

remains unclear whether the P2 receptor network plays a direct role in this cascade of events. The most immediate influence of acidosis on the P2-receptor network can be studied at the level of the [Ca2+]<sup>i</sup> response evoked immediately upon application of a purinergic agonist.

We investigated the effect of acidosis on ATP-mediated [Ca2+]<sup>i</sup> responses in bone-marrow derived osteoclast precursors to demonstrate the applicability of the developed algorithm. The application of ATP (100 nM to 10 mM) to the fura2-loaded osteoclast precursors evoked a [Ca2+]<sup>i</sup> TR in a dose-dependent manner in control and acidosis conditions (**Figures 7A,B**). The response amplitudes under acidic conditions were virtually indistinguishable from the control for ATP concentrations up to 10 µM. However, above this threshold concentration, the amplitude of the control responses continued to increase with rising concentrations of ATP, while [Ca2+]<sup>i</sup> responses under acidic conditions plateaued at 10 µM (**Figure 7C**). With respect to the AUC of the [Ca2+]<sup>i</sup> responses, the observed differences between the two conditions were more gradual with a diverging trend beginning as low as 1µM ATP and becoming more prominent at high ATP (**Figure 7D**). Finally, acidosis was found to have no significant effect on the periodicity of the oscillatory responses (**Figure 7E**).

These findings support that acidosis, while having no effect on ATP-mediated [Ca2+]<sup>i</sup> responses at lower ATP concentration, significantly attenuates the magnitude of [Ca2+]<sup>i</sup> transients responding to higher ATP concentrations (>10 µM ATP). Within the limited scope of this study that is focused on the development of a data analysis algorithm, we can only hypothesize on the mechanism by which these differences arise. One possibility is that the rise in extracellular [H+] has a significant influence on the electro-chemical gradient across the cellular membrane, which may consequently alter the extent of calcium flux across certain ionotropic P2X receptors. Since the oscillations are commonly driven by inositol triphosphate-mediated release of calcium from internal calcium stores (i.e., isolated from extracellular [H+]), it may explain why the oscillatory behavior is not affected by acidosis. Alternatively, there may exist a subset of P2 receptors that are sensitive to fluctuations in extracellular [H+], while P2-receptors involved in oscillatory behavior and/or responses to lower ATP concentrations (≤10 µM) are resilient to such changes. Regardless of the underlying mechanism, these results highlight that the P2 receptor network can be differentially modulated by extracellular pH.

# CONCLUSIONS

This paper presents an autonomous signal-processing algorithm capable of robustly removing signal-contaminating noise and delineating the various components seen in a calcium response, including non-stationary drift, TRs, and MPRs (possibly caused by flickers, puffs, and sparks) sampled with at least twice the Nyquist frequency. By fitting piece-wise defined model functions to data, the algorithm also extracts estimates for the parameters that are relevant to the characterization of cellular transient dynamics. Any time-series recordings can be used as an input for the algorithm, provided that they resemble a single or multi-peak transient response. As demonstrated in the validation process, manual estimation of certain parameters has an inherent degree of subjectivity and measurement error associated with

FIGURE 7 | Algorithm application in characterization of pathological states. ATP (100 nM–10 mM) was applied to Fura-2 loaded osteoclast precursors, under control (pH 7.6) and acidosis (pH 7.0) conditions, and [Ca2+] i responses were recorded. Algorithm was used to obtain estimates for amplitude, AUC and periodicity. (A) Representative [Ca2+] i response traces for 100µM and 1 mM ATP under control conditions. (B) Representative [Ca2+] i response traces for 100µM and 1 mM ATP under acidosis conditions. (C) Amplitude dose-response curves. (D) AUC dose-response curves. (E) Period dose-response curves. For (C–E), data are mean ± S.E.M. The effect of ATP under control conditions was examined using one-way ANOVA. The effect of acidosis on ATP-mediated responses was examined using two-way ANOVA. The Bonferroni test was used for post hoc multi-comparison analysis; \*\*p < 0.01; \*\*\*p < 0.001 indicate significant difference compared to the response to the lowest ATP concentration; #p < 0.05 indicates significant difference between responses to the same [ATP] in control and acidosis conditions.

it. In particular, the manual evaluations of AUC values, decay constant and the time of onset, as well as most of the MPRparameter values, were found to rely on subjective estimates and thus lacked true accuracy and consistency. Because of such limitation in the validation method, manual-estimates are to be recognized as representative estimates, rather than accurate values for these parameters. Consequently, validation method applied here should be considered as a comparison against imperfect estimates.

Nevertheless, our analysis of the automated method has verified that the algorithm performs within acceptable margins of agreement when compared to manual analysis. Regarding the response detection capabilities, the algorithm behaves conservatively compared to manual assessments, especially when presented with low-magnitude TRs or ambiguous response signals. Most importantly, our algorithm has been validated against experimental [Ca2+]<sup>i</sup> recording data, rather than simulated data, ensuring that the method is capable of handling variations in drift and noise that realistically reflect signal contaminations of experimental data acquisition. We have demonstrated that this automated methodology is effective in analyzing empirical data, providing quantitative insights about them and identifying differences between them.

A particularly unique feature of this algorithm is its capacity to characterize the magnitude and temporal characteristics of MPRs exhibiting stochastic and deterministic behavior. It is well established that a diverse amount of biochemical processes can be amplitude- and/or frequency-modulated (Adachi et al., 1999; Micali et al., 2015). To analyse such oscillatory data, the fast Fourier transform (FFT) is commonly used, which allows for the conversion of a signal from its time domain, into the frequency domain. Unfortunately, the variance in the frequency domain is proportional to the number of repetitive components in the timedomain. Therefore, if the oscillatory signals present few repetitive components then reliable resolution of the true periodicity of the signal is unachievable. To circumvent the limitations inherent to FFT, we apply the MATLAB "findpeaks()" function to identify peaks of interest. To isolate underlying coherent oscillations that are often present, we applied a clustering method. This is based on the principle of clustering deviations from baseline according to their temporal offset and respective FWHM. The advantage of this approach is that it allows for the reliable detection of periodic peaks, even in the presence of stochastic discharges, as is often the case in experimental recordings. Secondly, comparison of the set of MPR parameters for all peaks and subset of coherent peaks allows users to quantify the extent of stochastic activity within MPRs. Alternatively, the relationship between mean and standard deviation of periodicity in a MPR has been previously used to reveal the contribution of stochastic processes to the periodicity (Thurley et al., 2014). We anticipate this methodology will contribute to the comprehensive analysis of diverse MPRs.

Calcium signaling is by no means unique to the P2-receptor network, but rather represents the most ubiquitous and versatile messenger found in biological systems. All kinds of extracellular signals exploit calcium as a secondary messenger, including P2 agonists (i.e., ATP, ADP, UTP, and UDP), endothelin-1, oxotremorine-M, norephinephrine, thrombin, PDGF, bombensin (Balla et al., 1991; Palmer, 1994; Burnstock, 2004). The universal involvement of calcium ranges from basic physiological processes such as muscle contraction, neuronal discharge and pancreatic secretion, to early development events including mammalian egg fertilization and embryonic pattern formation (Berridge et al., 2000). Calcium signaling is also known to be impaired in various pathological states, as suggested for metabolic acidosis in this study, chronic renal failure (Massry et al., 1995), Alzheimer's (Brawek et al., 2014), Diabetes (Chen et al., 2015), and zinc deficiency (O'Dell and Browning, 2013). However, despite all that we know about calcium's role in biological processes, there remains ongoing debate on how calcium signals robustly encode information while still exhibiting a large degree of heterogeneity within and between various cellular populations. Many theories have been proposed to establish how information can be encoded. Some of these involve encoding information on the basis of calcium binding cooperativity (Larsen et al., 2004), amplitude and frequency modulation (De Pitta et al., 2009), changes in spike time variation (Thurley et al., 2014), and signal integration (Hannanta-anan and Chow, 2016). In order to reconcile these theories and establish a universal syntax for calcium-encoded information, tools such as this algorithm will aid in the largescale analysis of experimental data sets required for the validation of mathematical models.

The consideration of signaling nuances that are specifically found in physiological signals, but may or may not be present in non-biological signals, was a critical step in the development of this algorithm. As demonstrated in this study, physiological signals were decomposed into their elementary components and mathematically generalized to enable for the computational reconstruction of a diverse range of signature forms. In doing so, we were able to provide a foundation for further modeling of the nonlinear multiparametric physiological signals. This study demonstrates that the accurate description of complex physiological signals is non-trivial, but rather an extensive mathematical undertaking. Therefore, we believe that, beyond serving the purpose of a signal-processing tool, this algorithm will also contribute to future efforts to modeling physiological signals.

In summary, we have detailed an open-source MATLAB algorithm intended to facilitate the analysis of time-series recordings. With minimal user-input required, this tool dramatically decreases analysis time and ensures consistency in parameter characterization of complex physiological signals. This algorithm is capable of handling noise and drift and robustly characterizes the magnitude and kinetics of dynamic processes, outputting the amplitude, time of onset (tonset), activation time (t10–90%), full-width half-max (FWHM), AUC, and decay constant (τdecay). In the presence of MPR, six additional parameters are characterized which include number of oscillations (Nosc), magnitude of oscillatory peaks (E), periodicity (T), standard deviation of periodicity (σT), oscillatory persistence (losc), and the duty cycle (ξ peak/T). This algorithm is not limited to any specific data-type, but [Ca2+]<sup>i</sup>

recordings represent an obvious application. In addition to calcium imaging, other imaging modalities such as adapted fluorescence resonance energy transfer (FRET) biosensors, realtime bioluminescence and voltage and current measurements can generate time-series data for which characterization of signal magnitude and kinetics can provide valuable information. As data acquisition becomes more efficient and data sets become increasingly complex, automated analysis will serve as an essential tool for conducting basic research and clinical screening.

# AUTHOR CONTRIBUTIONS

Study conception and design: LM, NM, SK, AK. Algorithm development: LM, AK. Acquisition of data: NM. Analysis and interpretation of data: LM, NM. Drafting of Manuscript: LM,

#### REFERENCES


NM, SK, AK. All authors contributed to the critical revision of manuscript and approved the final version to be published.

#### ACKNOWLEDGMENTS

This research work was supported by the Natural Sciences and Engineering Research Council (NSERC) grants to AK and SK. The authors would like to acknowledge the partial financial support provided to LM by Dr. Michael Mackey through his NSERC fund.

#### SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: http://journal.frontiersin.org/article/10.3389/fphys. 2016.00525/full#supplementary-material


rat liver epithelial cells. Exp. Cell Res. 230, 197–207. doi: 10.1006/excr.19 96.3409


or by the clock? Biophys. J. 94, 2404–2411. doi: 10.1529/biophysj.107. 119495


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2016 Mackay, Mikolajewicz, Komarova and Khadra. This is an openaccess article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Cancer Markers Selection Using Network-Based Cox Regression: A Methodological and Computational Practice

Antonella Iuliano<sup>1</sup> \* † , Annalisa Occhipinti <sup>2</sup> \* † , Claudia Angelini <sup>1</sup> , Italia De Feis <sup>1</sup> and Pietro Lió<sup>2</sup>

1 Istituto per le Applicazioni del Calcolo "Mauro Picone," Consiglio Nazionale delle Ricerche, Naples, Italy, <sup>2</sup> Computer Laboratory, University of Cambridge, Cambridge, UK

#### Edited by:

Krasimira Tsaneva-Atanasova, University of Exeter, UK

#### Reviewed by:

Guanglong Jiang, Indiana University School of Medicine, USA Anelia Horvath, George Washington University, USA

#### \*Correspondence:

Antonella Iuliano a.iuliano@na.iac.cnr.it; Annalisa Occhipinti ao356@cam.ac.uk † Joint first authors.

#### Specialty section:

This article was submitted to Computational Physiology and Medicine, a section of the journal Frontiers in Physiology

> Received: 04 April 2016 Accepted: 22 May 2016 Published: 17 June 2016

#### Citation:

Iuliano A, Occhipinti A, Angelini C, De Feis I and Lió P (2016) Cancer Markers Selection Using Network-Based Cox Regression: A Methodological and Computational Practice. Front. Physiol. 7:208. doi: 10.3389/fphys.2016.00208 International initiatives such as the Cancer Genome Atlas (TCGA) and the International Cancer Genome Consortium (ICGC) are collecting multiple datasets at different genome-scales with the aim of identifying novel cancer biomarkers and predicting survival of patients. To analyze such data, several statistical methods have been applied, among them Cox regression models. Although these models provide a good statistical framework to analyze omic data, there is still a lack of studies that illustrate advantages and drawbacks in integrating biological information and selecting groups of biomarkers. In fact, classical Cox regression algorithms focus on the selection of a single biomarker, without taking into account the strong correlation between genes. Even though network-based Cox regression algorithms overcome such drawbacks, such network-based approaches are less widely used within the life science community. In this article, we aim to provide a clear methodological framework on the use of such approaches in order to turn cancer research results into clinical applications. Therefore, we first discuss the rationale and the practical usage of three recently proposed network-based Cox regression algorithms (i.e., Net-Cox, AdaLnet, and fastcox). Then, we show how to combine existing biological knowledge and available data with such algorithms to identify networks of cancer biomarkers and to estimate survival of patients. Finally, we describe in detail a new permutation-based approach to better validate the significance of the selection in terms of cancer gene signatures and pathway/networks identification. We illustrate the proposed methodology by means of both simulations and real case studies. Overall, the aim of our work is two-fold. Firstly, to show how network-based Cox regression models can be used to integrate biological knowledge (e.g., multi-omics data) for the analysis of survival data. Secondly, to provide a clear methodological and computational approach for investigating cancers regulatory networks.

Keywords: cancer, Cox model, high-dimensionality, gene expression, network, regularization, survival

# INTRODUCTION

Recent developments in high-throughput technology have produced a huge amount of multiple and diverse genomescale data to deal with biological and clinical questions in cancer. For example, genomics, transcriptomics, and epigenomics information is nowadays publicly available for tens of different cancer cell lines from thousands of patients in The Cancer Genome Atlas (TCGA, http://cancergenome. nih.gov/). Mutations data over one million tumor samples are also reported in Cosmic (http://cancer.sanger.ac.uk/cosmic), the world's largest and most comprehensive resource for exploring the impact of somatic mutations. Other valuable databases include The Gene Expression Omnibus (GEO, http://www.ncbi. nlm.nih.gov/gds) among others. Such amount of data is likely to revolutionize genetics and biomedical cancer research, but a thorough integration of all these different types of information is necessary. Indeed, cancer is a "multi-factorial" disease caused by a combination of genetic, environmental, and lifestyle factors. Such factors play an important role in discovering prognostic and diagnostic cancer gene signatures opening a new way toward the so called "personalized medicine." The term refers to a new type of therapy that is essentially based on the features of each patient. For instance, the anticancer drug Cetuximab (Karapetis et al., 2008) inhibits cells proliferation by binding to the EGF receptor and, consequently, preventing activation of the downstream signaling pathway. However, it has been found that Cetuximab can work only if the K-RAS gene is not mutated. Another example is the anti-cancer drug Trastuzumab (Hudis, 2007), which is effective only in patients that highly express the human epidermal growth factor (HER2) at the cell surface, to which the antibody binds. These examples highlight the need of identifying stable and interpretable biomarkers able to predict patient survival and characterize a patient-personalized therapy. In addition, the knowledge of complex cancer processes and networks is important to optimize the use of technology within health care (Raghupathi and Raghupathi, 2014). By discovering associations within the data, big data analytics has the potential to improve care, save lives, and lower costs.

As a consequence, in the last years, there has been a growing interest in developing methods that integrate different genomescale data into regression models for survival data to create a comprehensive view of human biology and disease (Wang et al., 2014). A popular used approach for the integration of genomic and clinical information is the Cox proportional hazard model (Cox, 1972). The main goal of such method is investigating the connection between gene expression data and survival information to predict cancer survival, assess cancer outcomes, and identify new gene markers. However, since gene expression data are usually characterized by a number of covariates p much larger than the sample size n, the traditional Cox model cannot be applied. Hence, several penalized Cox regression methods have been developed to identify core pathways and biomarkers involved in cancer progression, e.g., the Cox model based on Lasso penalty (Tibshirani, 1996, 1997; Gui and Li, 2005). Alternative penalized Cox regression models based on variable selection include the SCAD (Fan and Li, 2001), the adaptive Lasso (Zou, 2006), the elastic net model (Zou and Hastie, 2005; Simon et al., 2011a; Wu, 2012), and the Dantzig selector (Candes and Tao, 2007) among others. These methods are able to cope with the high-dimensionality of gene expression data, thus solving the "p ≫ n" issue (Engler and Li, 2009). All these penalized models are statistically efficient in high-dimensional regression, but they perform poorly on data with high collinearity. Moreover, no biological knowledge is taken into account. Indeed, they are simply based on statistical frameworks completely ignoring biological regulatory network, protein–protein interaction (PPI), signaling pathways, and well-known relationships among genes. In such models, the lack of biological information produces instability in predictors reducing the predictive ability of the models. Hence, in order to provide more reliable and biologically meaningful results, the inclusion of a-priori biological knowledge into the models is mandatory. To address this issue, new penalized Cox methods based on the integration of genomic information have been recently proposed (Zhang et al., 2013; Gong et al., 2014; Sun et al., 2014). In such models, the genomic information is encoded by a network whose graph structure identifies a given relation (edges) between genes (nodes). The resulting Laplacian matrix is then integrated as penalty in the Cox regression models. In particular, the network can represent the correlation between genes (Zhang et al., 2013), KEGG pathways identification (Sun et al., 2014), functional interaction network (Huttenhower et al., 2009), or PPI. These Cox models based on a-priori biological network are called "network-based Cox regression."

The network-based Cox regression methods provide an efficient tool to perform Cox regression on high-dimensional data incorporating genes network information. In literature, there are some recent approaches that analyze different Cox methods. For instance, an accurate review of eight different methods that integrate network information into multi-variable Cox models is presented to study the risk prediction in breast cancer and the integrated Brier score is used as a performance measure (Fröhlich, 2014). However, the study performed enrichment analysis on the signatures genes selected by the compared models without showing any survival prediction analysis in terms of Kaplan–Meier curves. A network-based Cox regression model that explores gene-to-gene connections in multiple cancer datasets is also performed for maximizing the overall association of the sub-network with clinical outcomes (Martinez-Ledesma et al., 2015). A potential limitation of these conventional networks is that the edges only reflect the information of within-features or within-relations, and do not consider the association between features and outcomes, which may be useful in improving the predictive power. Therefore, an alternative network construction method for the outcomeguided gene-interaction network has to be introduced in order to improve the performance of survival analysis in network-based Cox regression (Jeong et al., 2015).

In this work, we present a methodological framework for the analysis of molecular and survival data through a crossvalidated approach of network-based Cox regression algorithms (Net-Cox, Adalnet, and fastcox, see Section Methods). The method starts from the analysis of raw data and, through a cross-validated penalty approach, it guides the reader to the interpretation of the final results. As shown in **Figure 1**, the general steps of our approach are the following: (i) defining the biological question and the experimental design using microarray data, then integrating a-priori biological information using functional map of the human genome such as HEFalMp (Huttenhower et al., 2009) and KEGG; (ii) performing biological screening of the data for selecting relevant features through crossvalidated penalization (Simon et al., 2011b); (iii) implementing network-based Cox regression models for the analysis of cancerrelated genes; (iv) evaluating survival models to predict cancer patient prognosis and exploring cancer associated pathways. The presented approach provides a new methodological framework for the study and the interpretation of regression methods through gene-network and pathways analyses and it can be easily adapted to incorporate other network-based Cox regression algorithms.

A preliminary study for the comparison of penalized Cox models was presented in Iuliano et al. (2014), where the analysis was limited to cancer survival prediction using top ranked genes. No simulation studies, extensive pathways analysis or validation of the data were performed in that study. On the contrary, this article presents a more accurate and complete analysis based on a cross-validated approach (Simon et al., 2011b), the overall workflow (see **Figure 2**) that includes both simulation studies and novel real cancer datasets (see Section Data Analysis). Simulated data have been used to perform a statistical comparison of the methods in terms of sensitivity, specificity, number of selected genes, false positive rates, and Matthews correlation coefficient in two simulation settings with different genetic effects. On the other hand, real datasets analysis was performed to assess the relevance of the selected genes in the training dataset and to test the survival prediction accuracy of each model. Cross-validated Kaplan–Meier curves for survival analysis and pathway analysis were also computed (see Section Results). The novelty of the current study consists in the integration of a cross-validated approach (Simon et al., 2011b) to obtain an accurate survival prediction even when the number of cases is relatively small for an effective sample splitting (see **Figure 2**). Cross-validation methods have been largely applied in Cox regression models to estimate prediction errors and for model parameters tuning (Vasselli et al., 2003; Molinaro et al., 2005; Simon et al., 2011b). Some of the most relevant crossvalidation approaches include leave-one-out cross-validation (LOOCV; Kearns and Ron, 1999), k-fold (Refaeilzadeh et al., 2009), and bootstrap algorithms (Kohavi, 1995). However, all these methods do not provide a good estimation if the data available are limited for an effective division in training and test sets. On the contrary, the cross-validation method used in our analysis (Simon et al., 2011b) is based on a re-sampling algorithm that allows an accurate prediction of the survival risk model regardless the data size. Therefore, in this work, we first present a novel statistical approach to infer pathway interaction networks from gene expression data that relies on a new mathematical

order to select IN variables to use in the analysis; (3) implement network-based Cox regression models with the integration of a re-sampling method based on a cross-validated approach; (4) apply survival analysis to predict cancer patients and pathway analysis to explore groups of genes associated to the disease.

#### FIGURE 2 | Continued

cancer-related genes, pathway information, and overall survival (OS) times. Step (2) illustrates the novelty of the work based on a k-fold cross-validation Kaplan–Meier procedure by integrating network-regularized Cox models for selecting significant genes and pathways structures. The Prognostic Index (PI) has been used to divide the patients in high-risk and low-risk groups. Then, the union of these two groups is done to plot single cross-validated Kaplan–Meier curves and to calculate the p-value permutation test. Step (3) shows the survival prediction to test how well the models generalize across independent cancer datasets.

concept (based on the biological screening and network-based Cox regression methods) for understanding pathways' activity and relationships. Second, we provide a methodological strategy to researchers for the use of network-based Cox regression models in order to turn cancer research results into clinical applications.

#### METHODS

#### Network-Regularized Cox Regression Models

The Cox Proportional hazards model (Cox, 1972) is the most widely used model to describe the relationship between survival times and predictor covariates.

Given a sample of n subjects, let T<sup>i</sup> and C<sup>i</sup> be the survival time and the censoring time, respectively, for subject i = 1, . . . , n. Let t<sup>i</sup> = min {Ti, Ci} be the observed survival time and δ<sup>i</sup> = I(T<sup>i</sup> ≤ Ci) the censoring indicator, where I(·) is the indicator function (i.e., δ<sup>i</sup> = 1 if the survival time is observed and δ<sup>i</sup> = 0 if the survival time is censored). We denote by **X**<sup>i</sup> = (Xi1, . . . , Xip) ′ the regression vector of p-variables for the ith subject (i.e., the gene expression profile of the ith patient over p genes). The survival time T<sup>i</sup> and the censoring time C<sup>i</sup> are assumed to be conditionally independent given **X**<sup>i</sup> . Furthermore, the censoring mechanism is assumed to be non-informative. The observed data can be represented by the triplets {(ti, δi, **X**i), i = 1, ..., n}. The Cox regression method assumes that the hazard function h(t|**X**i), which is the risk of death at time t for the ith patient with gene expression profile **X**<sup>i</sup> , can be written as

$$h(t|\mathbf{X}\_i) = h\_0(t) \exp\left(\mathbf{X}\_i^\prime \mathcal{B}\right)$$

where h0(t) is the baseline hazard and β = (β1, . . . , βp) ′ is the column vector of the regression parameters.

In the classical setting, the regression coefficients are estimated by maximizing the Cox's log-partial likelihood

$$pl(\mathcal{B}) = \sum\_{i=1}^{n} \delta\_i \left\{ \mathbf{X}\_i^\prime \boldsymbol{\mathcal{B}} - \log \left[ \sum\_{j \in R(t\_i)} \exp(\mathbf{X}\_j^\prime \boldsymbol{\mathcal{B}}) \right] \right\},\tag{1}$$

where t<sup>i</sup> is the survival time (observed or censored) for the ith patient, R(ti) is the risk set at time t<sup>i</sup> (i.e., the set of all patients who still survived prior to time ti).

However, in the analysis of gene expression data, the number of genes p is usually larger than the sample size n and the standard Cox-model cannot be directly applied. To cope with the curse of dimensionality (p ≫ n), a variety of penalization approaches have been proposed for achieving good prediction performance and easy interpretation of the data. Although these regularization methods induce sparsity into the solution by shrinking some estimates to zero, the biological relationship of gene expression profiles is not taken into account. Hence, in order to integrate information from molecular interactions between genes, network-based constrained methods for highdimensional Cox regression have been introduced.

In this context, the regression coefficients are estimated by maximizing the penalized Cox's log-partial likelihood function

$$p l\_{pen}(\mathcal{B}) = \sum\_{i=1}^{n} \delta\_i \left\{ X\_i^\prime \mathcal{B} - \log \left[ \sum\_{j \in R(t\_l)} \exp(X\_j^\prime \mathcal{B}) \right] \right\} - P\_\lambda(\mathcal{B}), \tag{2}$$

where Pλ(β) is a network-constrained penalty function on the coefficients β.

Such penalty function describes the existing relationships among the covariates (genes) specified by a network G = (V, E, W) (weighted and undirected graph), where V = 1, . . . , p is the set of vertices (genes/covariates), an element (i, j) in the edge set E ⊂ V × V indicates a link between vertices i and j and W = (wij), (i, j) ∈ E is the set of weights associated with the edges. These weights are usually used to represent the relations between genes in terms of gene–gene interaction, KEGG pathway analysis or PPI. Hence, the network structure plays an important role since it incorporates prior gene regulatory information often ignored.

The three regularized network-based Cox regression models used in our study are presented below and differ in the form of the penalty function Pλ(β).

#### Net-Cox method

Net-Cox regression (Zhang et al., 2013) is an extension of the L2-Cox model and uses the following penalty function

$$P\_{\lambda,\alpha}(\mathcal{B}) = \lambda \left[\alpha \|\mathcal{B}\|\_2^2 + (1-\alpha)\Phi(\mathcal{B})\right],\tag{3}$$

where λ > 0 and α ∈ (0, 1] are two regularization parameters in the network constraint. and

$$\Phi(\boldsymbol{\beta}) = \sum\_{(i,j)\in E} \boldsymbol{w}\_{ij} (\beta\_i - \beta\_j)^2. \tag{4}$$

The penalty (3) consists of two terms: the first one is an L2-norm of β that regularizes the uncertainty in the network constraint; the second term is a network Laplacian penalty 8(β) that encourages smoothness among correlated gene in the network and encode prior knowledge from a network.

Given a normalized graph weight matrix W, we assume that co-expressed (related) genes are assigned similar coefficients by defining the cost term 8(β) as reported in Equation (4). 8(β) can be also written as 8(β) = β ′ (**I** − **W**)β = β ′ <sup>L</sup>¯<sup>β</sup> where <sup>L</sup>¯ is a positive semi-definite matrix derived from network information (weight matrix **W**) and **I** is an identity matrix. Hence, the objective function will result in a significant cost in the network if any pair of genes is connected by an high weight edge and the difference between their coefficients is large.

Note that to identify the signature genes classified by Net-Cox, which is a ridge regression based method, we create a consensus ranking of the relevant cancer genes.

#### AdaLnet Method

Adaptive Laplacian net (Sun et al., 2014) is a modified version of a network-constrained regularization procedure for fitting linear models and for variable selection (Li and Li, 2008, 2010) where the predictors are genomic data with graphical structures. AdaLnet is based on prior gene regulatory network information, represented by an undirected graph for the analysis of gene expression data and survival outcomes.

Denoting with d<sup>i</sup> = P <sup>i</sup>:(i,j)∈<sup>E</sup> wij the degree of vertex i, AdaLnet defines the normalized Laplacian matrix **L** = (lij) of the graph G by

$$l\_{i,j} = \begin{cases} -1, & \text{if } i = j \text{ and } d\_i \neq 0, \\ -\omega\_{ij} / \sqrt{d\_i d\_j}, & \text{if} (i, j) \in E, \\ 0, & \text{otherwise}. \end{cases} \tag{5}$$

Note that **L** is positive semi definite. The network-constrained penalty in Equation (2) is given by

$$P\_{\lambda,\alpha}(\mathcal{J}) = \lambda \left[ \alpha \parallel \mathcal{J} \parallel + (1 - \alpha) \,\Psi(\mathcal{J}) \right],\tag{6}$$

with

$$\Psi(\boldsymbol{\beta}) = \sum\_{(i,j)\in E} w\_{ij} \left( \text{sign}(\tilde{\beta}\_i) \beta\_i / \sqrt{d\_i} - \text{sign}(\tilde{\beta}\_j) \beta\_j / \sqrt{d\_j} \right)^2 . \tag{7}$$

Equation (6) is composed by two penalty terms. The first one is an L1-penalty that induces a sparse solution, the second one is a quadratic Laplacian penalty 9(β) = β ′ L˜β that imposes smoothness of the parameters β between neighboring vertices in the network. Note that L˜ = S ′LS with S = diag(sign(β˜ <sup>1</sup>), . . . ,sign(β˜ <sup>p</sup>)) and β˜ = (β˜ <sup>1</sup>, . . . , β˜ <sup>p</sup>) is obtained from a preliminary regression analysis. The scaling of the coefficients β respect to the degree allows the genes with more connections (i.e., the hub genes) to have larger coefficients. Hence, small changes of expression levels of these genes can lead to large changes in the response.

An advantage of using penalty (6) consists in representing the case when two neighboring variables have opposite regression coefficient signs, which is reasonable in network-based analysis of gene expression data. Indeed, when a transcription factor (TF) positively regulate gene i and negatively regulate gene j in a certain pathway, the corresponding coefficients will result with opposite sign.

Note that in Net-Cox and AdaLnet, λ is the parameter controlling the weight between the likelihood and the network constraint and α ∈ (0, 1] is the parameter weighting the network constraint.

#### Fastcox Method

The penalty function of fastcox (Yang and Zou, 2012) computes the solution paths of the elastic net penalized Cox's proportional hazards model (Wu, 2012). In this method the penalty function in Equation (2) is given by

$$P\_{\lambda,\alpha}(\mathcal{J}) = \lambda \left[ \alpha \mathcal{W} \| \mathcal{J} \|\_{1} + \frac{1}{2} (1 - \alpha) \| \mathcal{J} \|\_{2}^{2} \right],$$

where the non-negative weights w allow a more flexible estimation. In particular, setting w<sup>j</sup> = 0 implies no shrinkage and the variable j will be always included in the final model. Default is 1 for all variables. α ∈ (0, 1] is the elastic net trade off. This regularization technique is a combination of the lasso and ridge penalty that produce a sparse model (given by the L1-penalty) with good prediction accuracy, while encouraging a grouping effect. It is worthy to note that this method does not include any gene network information. It has been used in our study to obtain pathways investigation and survival prediction from a relevant method that is simply based on statistical framework.

# Tuning Parameters by Five-Fold Cross-Validation

For all the methods, we estimated the regularization parameters using cross-validation. Four-folds of data are used to build a model for validation on the fifth fold, cycling through each of the five-folds in turn. Then, the (λ,α) pair that minimizes the cross-validation log-partial likelihood (CVPL) are chosen as the optimal parameters. CVPL is defined as

$$\text{CVPL}(\lambda,\alpha) = -\frac{1}{n} \sum\_{k=1}^{K} \{\ell(\hat{\boldsymbol{\mathcal{B}}}^{(-k)}(\lambda,\alpha)) - \ell^{(-k)}(\hat{\boldsymbol{\mathcal{B}}}^{(-k)}(\lambda,\alpha))\},\tag{8}$$

where βˆ (−k) (·) is the estimate obtained from excluding the kth part of the data with a given pair of (λ, α), ℓ(·) is the Cox logpartial likelihood on all the sample and ℓ (−k) (·) is the log-partial likelihood when the kth fold is left out (van Houwelingen et al., 2006).

# General Algorithm: A Re-Sampling Method for Survival Prediction

The prediction capabilities of a given method are usually evaluated using a training set to select the markers and a testing set to measure the goodness of the prediction. In several cases training and test sets are obtained splitting a given dataset in two parts. However, findings could be over optimistic depending on the specific split. To further understand the role of the network information in cross-validation and to overcome the drawbacks of investigating only one split, each network-based model was validated with the re-sampling procedure suggested by Simon et al. (2011b). This method is based on a cross-validated estimate of the survival distribution of the risk groups and provide a more efficient use of data than fixed sample splitting (see **Figure 2**). The steps of the re-sampling algorithm for survival prediction are presented below.

**Procedure 1**: k-fold Cross-validated Kaplan–Meier survival method

1. The full dataset D is partitioned into K approximately equal parts D1, . . . , DK.

For each k = 1, . . . , K


$$PI\_{i\_k}^{D\_k} = \alpha'\_{i\_k} \hat{\beta}\_{T\_k},$$

where xi<sup>k</sup> is the vector of gene expression value associated to the i<sup>k</sup> -th patient into the k-fold. Each patient i<sup>k</sup> in D<sup>k</sup> is assigned into the high/low-risk group if its prognostic index PID<sup>k</sup> ik is above (or below) a fixed threshold PI∗,T<sup>k</sup> defined adaptively on T<sup>k</sup> .

	- (i) from the m-th permutation data (m = 1, . . . , M), compute the log-rank χ 2 b statistic using the crossvalidation procedure (1–6),
	- (ii) compute the permutation p-value, pˆ, as

$$\hat{p} = M^{-1} \sum\_{i=1}^{M} I(P\_m \ge P\_0).$$

For our analysis, the estimate βˆ Tk in step 4 was computed by using five-fold cross-validation (i.e., K = 5) to select the optimal tuning parameter values (λˆ Tk , αˆT<sup>k</sup> ), that we used to fit the corresponding penalized function Pλ<sup>ˆ</sup> Tk ,αˆTk (βˆ Tk ) on T<sup>k</sup> . In particular, we first set α to a sufficiently fine grid of values on [0, 1]. For each fixed α, λ was chosen from {10−<sup>5</sup> , 10−<sup>4</sup> , 10−<sup>3</sup> , 10−<sup>2</sup> , 10−<sup>1</sup> , 1} for Net-Cox, while it was set λ to a decreasing sequence of values λmax to λmin automatically chosen for AdaLnet and fastcox.

In step 5, we selected PI∗,T<sup>k</sup> as the optimal cut-off in terms of PID<sup>k</sup> . By using the PIT<sup>k</sup> ik , it was possible to split the patients in two subgroups, i.e., high-risk and low-risk prognosis groups. Thus, the patient i<sup>k</sup> in T<sup>k</sup> was assigned to the high-risk (or low-risk) group if his prognostic index PIT<sup>k</sup> ik was above (or below) the quantile selected on a grid of given values that spans from 30 to 70%. The cut-off PI∗,T<sup>k</sup> was chosen in correspondence to the lowest p-value in a log rank test on this grid.

In step 7, we set M equal to 500.

#### Survival Analysis

Network-based Cox regression model was used to discover significant variables, i.e., genes, correlated with death risk. Overall survival (OS) curves were estimated using the Cross Kaplan–Meier estimator and compared using the two-sided log-rank test as implemented in the R package survival. The statistical significance of the log-rank statistic related to the cross-validated Kaplan–Meier curves was obtained through a permutation distribution (Simon et al., 2011b) as described in the previous section. Permutation test was used to test the association between high-risk or low-risk groups and p < 0.05 were considered statistically significant. A simple scheme of the applied procedure for OS estimation is reported in **Figure 2**.

Furthermore, we also validated the predictive performance of the three methods using independent dataset for training and testing. In this context, we used the largest dataset as training set to identify the gene expression signatures (see **Figure 2**, step 2). Then, the second independent dataset was considered as test set in order to analyze the survival prediction of the models. We used Kaplan–Meier survival curves and log-rank test to perform the analysis (see **Figure 2**, step 3).

#### Pathway Analysis

We performed pathway analysis based on KEGG database and on the Human Experimental/Functional Mapper (Huttenhower et al., 2009). In particular, we focused on a gene–gene interaction analysis developing gene-networks that describe the relations between genes in terms of KEGG pathways. Each node in the network represents a gene and an edge between two nodes means that the two genes belongs to the same pathway. Different colors are used for different pathways. The color of each node indicates how strong is the relationship between the gene and the disease under analysis (ovarian and breast cancer; Huttenhower et al., 2009). The p-value chosen within the interval [0, 0.1] represents the node color intensity. Red color, that is p = 0, means that there is a high significant gene-disease relation, while green color, that is p = 0.1, means that not exist a relevant gene-disease relation.

Gene networks have been computed by considering only the not isolated genes in the intersection between KEGG pathways and the set of genes selected by each method. Given a set of genes G and the set of all the KEGG pathways K, we defined a gene g as not isolated if G ∩ K ! {g}. Namely, g is not isolated if there is at least another gene g ′ ∈ G belonging to the same pathways of g.

#### Software

The methodological approach presented in **Figure 2** has been implemented as an integrative R script that allows to run the different algorithms under the same R environment. Net-Cox, which is a Matlab toolbox (http://compbio.cs.umn.edu/Net-Cox/), AdaLnet, available as an R code and sent us upon request and fastcox, which is an R package (http://code.google.com/p/ fastcox/) were merged together by using R.matlab, https://cran. r-project.org/web/packages/R.matlab/index.html. The script also includes the implementation of the re-sampling permutation approach (Simon et al., 2011b) and the cross-validation method for parameters estimation. Both simulated and real data can be used to run the script which can be easily adapted for the integration of new Cox models.

For real data analysis, the microarray data were preprocessed using R packages available in Bioconductor. First, we selected from the initial dataset the genes that were more likely to be involved in cancer by using a functional map summarizing the most relevant interactions in the cancer area of interest (Huttenhower et al., 2009). Then, we used HEFaIMp tool (Huttenhower et al., 2009) to build the genes network and identify the weight of the edges between the selected genes. Finally, Net-Cox, AdaLnet, and fastcox were implemented integrating a cross-validation method for selecting the optimal tuning parameters λ and α and a re-sampling based procedure (Simon et al., 2011b), see Procedure 1.

The scripts are available upon request from the first two authors.

#### DATA ANALYSIS

#### Simulation Scheme

We used the three methods in two different simulation settings (Wu and Wang, 2013; Sun et al., 2014) in order to investigate the performances and the properties of the three models and to facilitate the interpretation of results. We considered two scenarios that are likely to be encountered in genomic studies and we simulated gene expression data as network constrained. Both the two settings consist of 100 regulatory networks. Each regulatory network is composed by one transcription factor (TF) that regulates 10 genes resulting in a total of 1100 genes. Detailed settings are given below.

#### Scenario 1: Not-Overlapped Networks

The first setting simulates a scenario with not-overlapped networks, which means that the 100 regulatory networks are disjoint each other and each gene is linked to only one TF. Under this assumptions, the degree d<sup>i</sup> of each TF = 10 and d<sup>i</sup> = 1 for the regulated genes. The edges' weight wij = 1 between the TFs and their regulated genes, wij = 0 otherwise. The expression value of each TF was generated from a normal standard distribution. The expression values of the ten regulated genes were generated from a conditional normal distribution with positive correlation (ρ = 0.7) between the expression of five genes and the corresponding TF, and negative correlation (ρ = −0.7) for the remaining five genes. This simulates the activation or repression of each gene under the effect of the corresponding TF. The failure times were generated from the Cox model

$$
\lambda(t|X) = \lambda\_0(t) \exp(\sum\_{j=1}^{\text{ss}} \beta\_j X\_j),
$$

which includes only s = 88 relevant genes (i.e., eight regulatory networks). The baseline hazard function λ0(t) was specified by a Weibull distribution with shape parameter 5 and scale parameter 2. Censoring times were generated from U(2, 15) with a censoring rate of about 30%. The sample size was fixed at n = 200 and the simulation were replicated 100 times. In this setting of not-overlapped genes, the coefficients β<sup>j</sup> , j=1, . . . , 44 were generated from the uniform distribution U(0.1, 1), while β<sup>j</sup> , j=45, . . . , 88 were generated from U(−1.5, −0.1).

For each of the settings above, we quantified the noise as the mean between the variance of each transcription factor (TF) and the variance of the 10 corresponding regulated genes.

#### Scenario 2: Overlapped Networks

The second setting simulates a scenario with overlapped networks, where four regulatory networks (i.e., 44 genes) are connected to the other four networks. This mimics the fact that some genes can belong to different pathways regulating different biological processes, as often observed in cancer. For the sake of simplicity, we assume that all the genes (including the TF) in the networks P3, P4, P5, and P<sup>6</sup> are connected to the genes in the remaining four network P1, P2, P7, and P<sup>8</sup> which are maintained disjointed and independent each other. The expression values of the TFs and the regulated genes were generated from a multivariate normal distribution with cov(Xi, Xj) = 0.5|i−j<sup>|</sup> . The coefficients β<sup>j</sup> , j = 1, . . . , 22, corresponding to P1 and P2, were generated from the uniform distribution U(0.1, 0.5), the coefficients corresponding to the 44 common genes β<sup>j</sup> , j = 23, . . . , 66 were generated from U(−0.1, 0.1) and the coefficients βj , j = 67, . . . , 88, corresponding to P<sup>7</sup> and P8, were generated from the uniform distribution U(−1, −0.5). Survival times were generated as reported in the first setting with the same censoring rate.

#### Statistical Measures

The performance of each method is summarized by four measures: sensitivity, specificity, number of genes selected, and the Matthews correlation coefficient (MCC). The sensitivity or true positive rate (TPR) and specificity or true negative rate (TNR) are given by

$$TPR = \frac{TP}{TP + FN}, \quad TNR = \frac{TN}{TN + FP},$$

where TP, TN, FP, and FN denote the numbers of true positives, true negatives, false positives, and false negatives, respectively. A test with high sensitivity (few false negative) has a low type II error rate, while a test with a high specificity (few false positive) has a low type I error rate. The number of genes selected refers to the genes identified as relevant by each method in the training set. The analysis of these genes gives information on prediction accuracy.

The Matthews correlation coefficient (MCC) is defined as

$$\text{MCC} = \frac{\text{TP} \times \text{TN} - \text{FP} \times \text{FN}}{\sqrt{(\text{TP} + \text{FP})(\text{TP} + \text{FN})(\text{TN} + \text{FP})(\text{TN} + \text{FN})}}.$$

The MCC measure is an global measure of accuracy, and a larger MCC indicates a better performance.

#### Real Data Applications

We applied the three network methods on different real datasets containing large-scale microarray gene expression measurements from ovarian and breast cancer including survival information (see **Table 1**) in order to facilitate the detection of molecular biomarker and pathway analysis with clinical utility.



#### Ovarian Datasets

We downloaded the first ovarian dataset from NCBI Gene Expression Omnibus as raw .CEL files (Bonome: GSE26712). The data contain gene expression profiling for extensive set of 185 primary ovarian tumors untreated late-stage (III–IV) highgrade (2,3) patients hospitalized at the Memorial Sloan-Kettering Cancer Center between 1990 and 2003. The Affymetrix human U133A microarray platform was used. The second ovarian dataset, the ovarian TCGA, was downloaded from The Cancer Genome Atlas data portal (The Cancer Genome Atlas Research Network, 2011). It was obtained at the gene level (level 3) using the Affymetrix human U133A microarray from 578 samples. All patients were diagnosed with high-grade serous carcinoma and were in an advanced stage. We noted that such datasets are very similar in terms of type of patients, platforms, and cancer disease. Therefore, they can be also used for validation.

#### Breast Datasets

The breast cancer microarray datasets were downloaded from NCBI GEO database as raw .CEL files (Kao: GSE20685 and Desmedt: GSE7390). Gene expression profiling of the first dataset was conducted on fresh frozen breast cancer tissue collected from 327 patients diagnosed and treated between 1991 and 2004 at the Koo Foundation Sun-Yat-Sen Cancer Center. Hybridization targets were prepared from total RNA according to the Affymetrix U133 plus 2.0 platform. The second breast cancer dataset was chosen on gene expression profiling of frozen samples from 198 N—systemically untreated patients at the Bordet Institute. It was based on the Affymetrix U133 platform.

#### Preprocessing

All the raw files were processed and normalized by RMA package available in Bioconductor (Gentleman et al., 2004). Between arrays normalization was carried out by using the preprocessCore package available in Bioconductor (Gentleman et al., 2004). Survival data (OS, i.e., overall survival), censoring indicator and time to death, for each patients in every dataset were also given (**Figure 2**, step 1).

#### Cancer Genes and Related Functional Networks

Following our previous study (Iuliano et al., 2014), in order to better analyze real datasets, we first applied a biologically inspired size reduction of the dataset, then we built an a-priori network information for the type of cancer under investigation (see **Figure 2**, step 1). For a better focus on genes that are more likely to be relevant in cancer, we selected the high-risk cancer genes using the Human Experimental/Functional Mapper

#### TABLE 2 | Significant genes number selected using HEFaIMp tool.


(Huttenhower et al., 2009), which is based on a regularized Bayesian integration system. This mapper provides a p-value for each gene describing the significance of the relation between the gene and the disease of interest (breast and ovarian cancer, respectively). In our analysis, we selected only the genes with p < 0.05. A summary of the final number of the genes selected from each dataset is reported in **Table 2**. The network matrices used to test the network-based Cox models in our analysis were also derived from the Human Experimental/Functional Mapper which provides maps describing the genes functional activity and interaction networks in over 200 areas of human cellular biology with information from 30,000 genome-scale experiments. This functional network summarizes information from a variety of biologically informative perspectives: prediction of protein function and functional modules, cross-talk among biological processes, and association of novel genes and pathways with known genetic disorders (Huttenhower et al., 2009). The edges of the network are weighted between [0, 1] and express the functional relation between two genes. Note that the functional linkage network includes more information than Human PPI, frequently used as the network prior knowledge. It is clear that taking into account such biological knowledge helps in identifying significant genes that are functionally related in order to obtain important results biologically interpretable.

In order to adapt the gene network to the different methods, the final weight matrix was slightly different from method to method. In particular, since AdaLnet requires a weight matrix consisting of 0 and 1, each matrix element was set equal to 0 (or 1) if the weight value was below (or above) a fixed threshold equals to 0.5. On the other hand, Net-Cox uses the original weight matrix as obtained in the original paper (Huttenhower et al., 2009).

#### RESULTS

In our study, we analyzed three network-based Cox regression methods described in Section Methods both on simulated


TABLE 3 | Simulation results for Not-Overlapped settings. Sensitivity, specificity, number of selected genes, false positive rates, and MCC were averaged over the 100 replications.

The table reports three consensus rankings for Net-Cox obtained selecting 44, 88, and 176 genes. For AdaLnet and fastcox, we show the results related to the general setting, and the statistical measures obtained when the number of selected genes is higher (or lower) of a fixed threshold (threshold was set equal to 100 for AdaLnet and equal to 10 for fastcox). Standard deviation is reported in brackets.


Sensitivity, specificity, number of selected genes, false positive rates and MCC were averaged over the 100 replications. The table reports three consensus rankings for Net-Cox obtained selecting 44, 88, and 176 genes. For AdaLnet and fastcox, we show the results related to the general setting, and the statistical measures obtained when the number of selected genes is higher (or lower) that a fixed threshold (threshold was set equal to 100 for AdaLnet and equal to 10 for fastcox). Standard deviation is reported in brackets.Standard deviation is reported in brackets.

and real data. Here, the major interest is the association of genomic features with clinical outcomes under specific scenarios. Simulation studies were based on two different biological scenarios and were introduced to show the performance of the selected network methods. While, real data analysis was performed in order to provide a better understanding of the outcomes in terms of predictive/prognostic biomarkers and to demonstrate their validity and clinical utility. In particular, we first investigated the three methods in terms of survival prediction performances and then, a pathway analysis was carried out focusing on the relevance in cancer of the selected genes.

It is important to note that the goal of this study is not to provide a rank list of the analyzed methods, but to present a accurate study for the identification of new cancer related genes and core pathways in order to make available such information to biomedical community in the form of a comprehensive methodological procedure (see **Figure 1**).

#### Simulation Studies

We analyze the performance of the three analyzed methods in two simulation settings where the number of relevant genes is fixed a-priori to 88 genes. The first setting simulates a scenario with not overlapped pathways, which means that each gene in the network belongs to only one pathway (notoverlapped pathways). The second setting represents a more realistic scenario with a set of genes shared among different pathways (overlapped pathways). In both cases, a five-fold cross validation was conducted on the full dataset in order to select the tuning parameters (λ, α) and to obtain the coefficient estimates


TABLE 5 | Optimal α cross-validated value calculated on the k training sets.

by using the three methods. The details of the simulation data are reported in Section Methods.

The performance of each method is summarized by several statistical measures: sensitivity, specificity, number of selected genes, false positive rates, and Matthews correlation coefficient (MCC). Simulation results for both the models are reported in **Tables 3**, **4**, respectively (standard deviation is reported in brackets). To analyze the signature genes identified by Net-Cox, which is a method based on ridge regression, we considered three different consensus rankings where the number of significant genes selected by the method was fixed to 44, 88, and 176 genes, respectively. The selected genes were classified in descending order according to the absolute value of the regression coefficients. On the other hand, to better highlight the variable selection performance of AdaLnet and fastcox, we split the 100 iterations in two groups based on the number of genes selected at each iteration. We fixed 100 genes as threshold for AdaLnet and 10 genes for fastcox, then we computed again the statistical measures based on the two groups.

In the not-overlapped setting, Net-Cox performed better than the other two methods as showed by the MCC, which provides an overall measure of accuracy. In particular, when considering 44 and 88 genes, the false positive rate in Net-Cox was 22.910 and 44.940, respectively, with MCC equals to 0.300 and 0.445. Sensitivity and specificity were, respectively, 0.240 and 0.977 in the first case, 0.489 and 0.956 in the second case study. When the number of selected genes was increased to 176, even if the false positive rate increased resulting in a lower specificity (0.890), the sensitivity reached its highest values producing the highest MCC (0.464).

Since the majority of the selected genes were irrelevant and both AdaLnet and fastcox resulted in sparse models, specificity was much higher than sensitivity and was comparable between the two variable selection methods. In particular, in the notoverlapped setting, AdaLnet selected in average 249.360 genes with a false positive rate equals to 210.330. Sensitivity and specificity were equal to 0.444 and 0.792 resulting in a MCC of 0.190. On the other hand, fastcox selected in average 42.62 genes with a false positive rate of 30.19. MCC was equal to 0.160 with sensitivity 0.141 and specificity 0.970.

AdaLnet had the best performance when the number of selected genes was below 100, while fastcox exhibit the best performance when the number of genes was above 10. This means that in the other cases the methods fail in the execution of the cross-validation (see Supplementary Image 1).

In the overlapped-pathways setting, Net-Cox obtained the highest MCC overall when considering 88 genes (MCC equals to 0.227) with a false positive rate equals to 62.620, sensitivity 0.288 and specificity 0.938. However, even if the specificity levels of the three consensus rankings were almost equal to the previous setting (specificity for 44, 88, and 176 genes equals to 0.970, 0938, and 0.860, respectively), in this setting Net-Cox sensitivity decreased resulting in lower MCC compared to the not-overlapped case (MCC for 44, 88, and 176 genes equals to 0.175, 0.227, and 0.182, respectively). AdaLnet and fastcox also reported lower MCCs compared to the not-overlapped setting (MCC equals to 0.166 in AdaLnet and 0.134 in fastcox). In particular, both AdaLnet and fastcox showed an higher specificity than before (0.879 and 0.974, respectively) but a lower sensitivity (0.262 and 0.098). Further analysis showed that AdaLnet had the highest MCC when the number of selected genes was below 100 (MCC 0.196), while fastcox had the highest MCC (0.158) when the number of selected genes was above 10, in accordance with the previous results (see Supplementary Image 2).

#### Real Data Analysis

In order to evaluate the performance of the three Cox models in terms of survival analysis, we used cross-validated Kaplan–Meier curves (Simon et al., 2011b) for overall survival (OS) both on ovarian and breast microarray studies (see **Figure 2**, step 2). Note that p-value was estimated within the same dataset but the crossvalidation approach is used to correct over optimistic conclusions due to the lack of independence between samples.

Moreover, since the ovarian datasets are comparable in terms of types of patients, platforms and cancer disease, Kaplan–Meier curves and two-side log-rank test were used to estimate the survival time and stratify the low-risk and high-risk groups on the independent test set (see **Figure 2**, step 3).

**Table 5** reports the number of genes selected by the three Cox regression methods for each OS and the optimal tuning parameter α. Interestingly, the optimal α was often equal to 0.5, indicating that there was a good balance between statistical constraints and network information. These results confirm that the network carries important information useful for improving survival analysis. Moreover, since Net-Cox is a method based on ridge regression, the genes are only shrunk and it is necessary to fix a threshold for selecting the most relevant cancer genes. Hence, within each fold, we ordered the genes according to the absolute value of the corresponding regression coefficients, then we considered the union of the top 50 genes selected in each fold.

In the following, we present the main results obtained.

#### Results on the Ovarian Datasets

**Figures 3**, **4** show the cross-validated Kaplan–Meier curves for high-and-low risk groups patients selected in the ovarian datasets (Benome: GSE26712 and OV TCGA datasets, respectively).

**Figure 3** shows that in the Bonome dataset the gap between the survival curves of the two risk groups in Net-Cox (**Figure 3A**) and fastcox (**Figure 3C**) is wider compared to AdaLnet (**Figure 3B**). In particular, in predicting survival probabilities, fastcox (permuted p < 0.05) seem to discriminate the risk groups better than Net-Cox and AdaLnet where the permuted p > 0.05. These

findings confirm the results previously obtained in Iuliano et al. (2014), in relation to the survival curves for each method. This was mainly due to the cross-validation approach used in this analysis to overcome the sample splitting problem with too small dataset.

On the other hand, in the OV TCGA dataset (**Figure 4**), the survival curves for high-and-low risk patients are not significantly separated. In particular, fastcox is the only method with a

significant difference (permuted p < 0.05) in the OS between the high-and-low-risk groups.

Finally, to test the survival prediction across independent datasets, we used the ovarian OV TCGA dataset as training set, and the Benome dataset as the test set to predict the risk scores

of the patients (see **Figure 2**, step 3). **Figure 5** shows the Kaplan– Meier curves for the two risk groups (high-and-low risk groups) in the Benome dataset obtained by Net-Cox (**Figure 5A**), AdaLnet (**Figure 5B**), and fastcox (**Figure 5C**). All the three methods gave a significant p-value at the 5% significance level (log-rank test, p < 0.05).

#### Results on Breast Datasets

**Figures 6**, **7** show the cross-validated Kaplan–Meier curves for high-and-low risk groups patients selected in the breast datasets (Kao: GSE20685 and Desmedt: GSE7390, respectively). In the Kao dataset, the permuted p-value related to **Figure 6A** (Net-Cox) and **Figure 6C** (fastcox) was smaller than 0.05, which means the high-risk and low-risk groups were significantly separated and the selected pathways and genes were related to survival times. In **Figure 6B** (AdaLnet), a patient of the high-risk group fell in the low-risk group and the permuted p-value is not significant.

We performed the same analysis for high-and-low risk patients in the Desmedt dataset. Also in this case, there was a significant difference in OS between the two risk groups as shown in **Figure 7A** (Net-Cox) and **Figure 7C** (fastcox) where the permuted p-value is smaller than 0.05. In **Figure 7B** (AdaLnet) the permuted p-value is not significant.

#### Identified Pathways

In this section, we present the results of the analysis in terms of KEGG pathways analysis based only on not-isolated genes (see section Methods for details). We report here only the networks related to AdaLnet and Net-Cox since all the networks related to fastcox have more than 100 node and 2000 edges and a clear visualization would not be possible. However, the lists of the genes selected by fastcox and the related pathways are reported in Supplementary Table 1 (ovarian datasets) and Supplementary Table 2 (breast datasets).

**Figures 8**, **9** show the gene-networks obtained for the Bonome dataset (GSE26712) built on the genes identified by Net-Cox and Adalnet, respectively. From the color of the nodes, we can infer that all the selected genes have a significant relation with ovarian cancer. Indeed, almost all the genes are close to red except for AKT3 which has a p-value correlation equal to 0.039. Indeed, AKT3 is usually involved in prostate and breast cancer (Nakatani et al., 1999). However, since it was selected both by Net-Cox and fastcox, a possible significant relation between AKT3 and ovarian cancer could be inferred as indeed confirmed by literature (Liby et al., 2012). In particular, AKT3 has a specific role in the genesis of ovarian cancer through modulation of G2-M phase transition (Cristiano et al., 2006). As showed in **Figure 8**, AKT3 is also involved in many cancer pathways, such as KEGG basal cell carcinoma, KEGG prostate cancer, and KEGG melangiogenesis. It is worthy to note that this gene was also selected in our previous study (Iuliano et al., 2014) by all the analyzed methods and it was also involved in the same cancer related pathways. These findings confirm the importance of AKT3 in ovarian cancer as confirmed indeed by literature (Cristiano et al., 2006).

In the Bonome dataset (GSE26712), Adalnet selected only two not-isolated genes (RB1 and BRCA2) involved in two different cancer pathways (**Figure 9**). Both the genes have been frequently observed in epithelial ovarian cancer (Flesken-Nikitin et al., 2003; Dinulescu et al., 2005; Naora and Montell, 2005) and several studies report their stable correlation (Flesken-Nikitin et al., 2003; The Cancer Genome Atlas Research Network, 2011). Moreover, the strong interaction between RB1 and the tumor protein TP53 (Dong et al., 1997; Schuijer and Berns, 2003) has been identified by Net-Cox and fastcox (**Figure 8**).

the p-value of the interaction between the gene and ovarian cancer (Huttenhower et al., 2009). Genes with p > 0.10 are represented in green.

**Figures 10**, **11** show the gene-networks obtained for the OV TCGA ovarian dataset built on the genes identified by Net-Cox and Adalnet, respectively. As already observed in the Bonome dataset analysis, all the selected genes in the OV TCGA dataset resulted strongly correlated with ovarian cancer. Indeed, almost all the genes are close to red. The only gene with a slightly different color is FZD3 which has a p-value of 0.049 and was selected by all the three methods. Hence, even if this gene has been mainly classified as gastric-cancer-related (Katoh, 2005), our results prove that it also has a relevant effect in ovarian cancer as confirmed by literature (Tapper et al., 2001). It is also important to note that other genes have been selected by all the three methods (i.e., GMPR, ENPP1, and APC). Such genes have been already classified as ovarian-related in cancer literature (Gayther et al., 1997; Kikuchi et al., 2007; Rikova et al., 2007), but, in our analysis, the pathways involved in such relation are also investigated. For example, while GMPR and ENPP1 interact simply through the KEGG purine metabolism pathway, the APC-FZD3 interaction involves three different pathways: KEGG basal carcinoma, KEGG pathways in cancer, and KEGG wnt signaling pathway.

It is worthy to note that some of the genes selected by the three methods (e.g., NPY, COL5A1, EGFR, and FBL1) have been already reported in literature (Zhang et al., 2013) where an analysis of subnetwork signatures in ovarian cancer based on Cox model is presented. Moreover, our approach selected new genes, such as AKT3 and RB1, which are also related to ovarian cancer (Flesken-Nikitin et al., 2003; Cristiano et al., 2006). These results show that our findings are consistent with the previous ones including, at the same time, other gene signatures.

**Figures 12**, **13** report the gene-networks selected in the Kao dataset (GSE20685) by Net-Cox and Adalnet, respectively. FGFR2 and BCL2 were again selected in this dataset confirming the strong relevance of the two genes in breast cancer. Moreover,

BRCA2 (Wooster et al., 1995) was selected by Net-Cox and fastcox confirming the accuracy of our analysis. It is also worthy to note that in all the breast cancer gene-networks the KEGG prostate cancer is always recurrent. This is mainly due to the common biomarkers between the two diseases (Yang et al., 1998; Mattie et al., 2006) and through our analysis new common biomarkers can be identified.

In the Desmedt dataset (GSE7390), all the genes selected by Adalnet were isolated and no network was built in this case. A list of the genes selected is reported in **Table 6**. **Figure 14**

two nodes means that the two genes belongs to the same pathway. Different colors are used for different pathways. The color of each node represents the p-value of the interaction between the gene and breast cancer (Huttenhower et al., 2009). Genes with p > 0.10 are represented in green.

reports the gene-network related to the genes selected by Net-Cox. All the selected genes show a strong relation with the disease, such as FGFR2 and BCL2, which were selected by both Net-Cox and fastcox and are involved in KEGG prostate cancer and in KEGG pathways in cancer. Both the genes are largely known as independent prognostic marker in breast cancer (Hunter et al., 2007; Thomadaki et al., 2007; Callagy et al., 2008). Both Net-Cox and fastcox selected UGT2B15, which has a breast-cancer-correlation p = 0.049. This gene has been usually involved in prostate cancer (Gsur et al., 2002), but recent works highlight its role also in breast cancer (Wegman et al., 2007).

In the analysis of the breast datasets, there was no overlap with our previous study (Iuliano et al., 2014). This was mainly due to the different datasets analyzed here potentially (different cancer subtype and different types of conditions) and to the more sophisticated procedures followed in this analysis. Indeed, in our previous work, we split the dataset in training and test set only

TABLE 6 | List of genes selected by Adalnet in the breast dataset GSE7390.


The second column reports the breast-cancer correlation p-value of each gene accordingly (Huttenhower et al., 2009). All the selected genes resulted isolated and no network was built in this case.

once, while here we used a cross-validation procedure that is expected more robust results.

# DISCUSSION AND CONCLUSIONS

A key issue in cancer survival analysis is uncovering the relation between gene expression profiles and cancer patients survival in order to identify biomarkers for disease diagnosis and treatment. In the last years, there has been a growing interest in methods that incorporate network information into classification algorithms for genes signature discovery. The main aims are to identify molecular biomarkers that reliably predict patient's response to therapy and to avoid ineffective treatment for reducing drug side-effects and associated costs. For this purpose, prognostic and diagnostic biomarker signatures need to be derived from omics data for various disease entities in order to offer useful methodological and practical strategy in research and clinical settings.

Here, we presented an extended methodological strategy for the analysis of gene signatures and survival prediction (see **Figure 1**). We integrated a new cross-validation method (Simon et al., 2011b) with the most recent network penalized Cox models (Yang and Zou, 2012; Zhang et al., 2013; Sun et al., 2014) to obtain an effective multi-splitting of the data and achieve an accurate survival prediction (see **Figure 2**). The analysis of the models was based both on simulated and real datasets in order to provide an accurate analysis in terms of statistical and biological investigation. Indeed, we showed that, given a number of variables not extremely high, all the analyzed methods were able to select the altered genes under different simulation settings. On the other hand, the analysis on real cancer datasets showed that through the integration of network information into Cox regression methods it is possible to identify cancer gene signatures with an accurate prognostic performance. Therefore, the contribution of this study is two-fold. Firstly, to obtain an integrative analysis of cancer genes networks and survival prediction. Secondly, to provide a computational and methodological framework for better investigating cancers regulatory networks and facilitating the management of patients in terms of prognosis, diagnosis and treatment.

The findings of this study have a number of important implications for future practice. Firstly, a practically appealing study based on a fast screening procedure (Fan and Lv, 2008; Fan et al., 2010) could be introduced in order to reduce the size of the feature space to a moderate scale. In fact, several types of screening procedures could be combined to integrate biological information into statistical screening analysis and provide more definitive understanding of the gene-regulatory networks. Secondly, the integration of clinical information and data from different omics (e.g., epigenomics or metabolomics) into the screening procedure could also provide a more accurate investigation and prevent the drawbacks of the current methods. Moreover, a more accurate biomarkers investigation could be performed using a number of high-quality binary PPIs available in literature (Rolland et al., 2014) where a proteome-scale map of the human binary interactome is compared to alternative network maps in order to give a deeper insight into genotype-phenotype relationships. Finally, it will be necessary to develop an user-friendly interface to turn this methodological framework into a practical tool.

### AUTHOR CONTRIBUTIONS

AI and AO are joint first authors and both authors contributed equally. AI and AO prepared the computational codes and carried out all of the statistical analysis. CA, ID, and PL initiated and coordinated the work, guided the study design, supervised all data curation and analysis, and finalized all study conclusion. CA, ID, and PL are equal contributors. All the authors wrote, reviewed and approved the final manuscript.

#### REFERENCES


#### ACKNOWLEDGMENTS

This research was partially supported by BioforIU Project and by EPIGEN Project. AO acknowledges the Qualcomm Research Scholarship and Cambridge Home and European Scholarship Scheme (CHESS). AI and AO would like to thank Prof. Pietro Lió for the hospitality at the Computer Laboratory in Cambridge and CA and ID for the hospitality at the IAC in Naples. We would like to thank also Prof. Hokeun Sun for sharing AdaLnet code and the reviewers for their insightful comments. PL received funding from the EU FP7-Health-F5-2012 under grant no. 305280 (MIMOmics).

# SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: http://journal.frontiersin.org/article/10.3389/fphys. 2016.00208


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2016 Iuliano, Occhipinti, Angelini, De Feis and Lió. This is an openaccess article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Parameter Identifiability of Fundamental Pharmacodynamic Models

David L. I. Janzén1, 2, 3 \*, Linnéa Bergenholm1, 2 \*, Mats Jirstrand<sup>3</sup> , Joanna Parkinson<sup>4</sup> , James Yates <sup>5</sup> , Neil D. Evans <sup>1</sup> and Michael J. Chappell <sup>1</sup>

<sup>1</sup> Biomedical and Biological Systems Laboratory, School of Engineering, University of Warwick, Coventry, UK, <sup>2</sup> Drug Metabolism and Pharmacokinetics, Cardiovascular and Metabolic Diseases, iMED, AstraZeneca, Gothenburg, Sweden, <sup>3</sup> Fraunhofer-Chalmers Centre, Chalmers Science Park, Gothenburg, Sweden, <sup>4</sup> Early Clinical Development, Quantitative Clinical Pharmacology, iMED, AstraZeneca, Gothenburg, Sweden, <sup>5</sup> Oncology, iMED, AstraZeneca, Cambridge, UK

Issues of parameter identifiability of routinely used pharmacodynamics models are

#### Edited by:

Krasimira Tsaneva-Atanasova, University of Exeter, UK

#### Reviewed by:

Yasunori Aoki, National Institute of Informatics, Japan Xin Lai, Universitätsklinikum Erlangen, Germany

#### \*Correspondence:

David L. I. Janzén D.L.I.Janzen@warwick.ac.uk Linnéa Bergenholm E.L.Bergenholm@warwick.ac.uk

#### Specialty section:

This article was submitted to Computational Physiology and Medicine, a section of the journal Frontiers in Physiology

Received: 31 August 2016 Accepted: 14 November 2016 Published: 05 December 2016

#### Citation:

Janzén DLI, Bergenholm L, Jirstrand M, Parkinson J, Yates J, Evans ND and Chappell MJ (2016) Parameter Identifiability of Fundamental Pharmacodynamic Models. Front. Physiol. 7:590. doi: 10.3389/fphys.2016.00590 considered in this paper. The structural identifiability of 16 commonly applied pharmacodynamic model structures was analyzed analytically, using the input-output approach. Both fixed-effects versions (non-population, no between-subject variability) and mixed-effects versions (population, including between-subject variability) of each model structure were analyzed. All models were found to be structurally globally identifiable under conditions of fixing either one of two particular parameters. Furthermore, an example was constructed to illustrate the importance of sufficient data quality and show that structural identifiability is a prerequisite, but not a guarantee, for successful parameter estimation and practical parameter identifiability. This analysis was performed by generating artificial data of varying quality to a structurally identifiable model with known true parameter values, followed by re-estimation of the parameter values. In addition, to show the benefit of including structural identifiability as part of model development, a case study was performed applying an unidentifiable model to real experimental data. This case study shows how performing such an analysis prior to parameter estimation can improve the parameter estimation process and model performance. Finally, an unidentifiable model was fitted to simulated data using multiple initial parameter values, resulting in highly different estimated uncertainties. This example shows that although the standard errors of the parameter estimates often indicate a structural identifiability issue, reasonably "good" standard errors may sometimes mask unidentifiability issues.

Keywords: structural identifiability, practical parameter identifiability, mixed effects models, pharmacodynamic models, fixed effects models

# INTRODUCTION

Pharmacodynamic (PD) models quantify processes involved in drug action such as distribution to the effect site, receptor binding and signal transduction. PD models are valuable in making predictions of drug effects in un-tested scenarios such as outcomes across different populations or with new dosing schedules. Such predictions may not always be valid: In particular, there may be issues related to parameter identifiability. Within the concept of parameter identifiability, there are

**88**

two distinct types: structural identifiability (Bellman and Åström, 1970) and practical identifiability (Raue et al., 2009).

As suggested by the name, structural identifiability concerns the inherent identifiability of the parameters in a model given its structure and observed outputs (Bellman and Åström, 1970). If a model is structurally unidentifiable, this means that at least one parameter can have any value without changing the model output (albeit with possible readjustment of remaining parameters). A well-known structurally unidentifiable problem is the linear model commonly used for estimating bioavailability F and volume of distribution V from plasma concentrations measured after oral drug administration, the most simple case being the one compartment PK model with first order absorption, where the plasma drug concentration C following a single dose is defined according to

$$\mathcal{C}(t) = \frac{F \cdot DOSE \cdot k\_d}{V \left(k\_d - k\_e\right)} \left(e^{-k\_{\mathcal{C}} \cdot t} - e^{-k\_{\mathcal{C}} \cdot t}\right) \tag{1}$$

where F is the bioavailability of the drug, DOSE is the orally administered dose, V is the volume of distribution, k<sup>a</sup> is the rate of absorption and k<sup>e</sup> is the rate of elimination. It has been shown that only the fraction <sup>F</sup> V can be identified, and any estimate of F will therefore inversely correlate to V and both values will be biologically meaningless (Cheung et al., 2013). Importantly, predictions of C(t) are still valid as these depend on the identifiable fraction <sup>F</sup> V . While structural identifiability is a property of the postulated model structure given a set of outputs, practical identifiability is related to the experimental data. In particular, it is a measure of the amount of information contained in the experimental data and how this information is translated to parameter uncertainty and subsequent prediction uncertainty.

Parameter identifiability is unfortunately often only investigated and considered at the level of practical identifiability using more simple measurements such as standard errors or correlation matrices rather than more sophisticated approaches such as the profile likelihood approach (Raue et al., 2009). This is problematic for several reasons. The primary reason is that it cannot be guaranteed that the estimated parameter values are uniquely determined by just looking at the estimation results. In addition, if the structural identifiability of a model is unknown, it means that the source of uncertainty in the parameter estimates may be either due to the experimental data, the model structure, or both (**Figure 1**). Thus, increasing the quality of the data may or may not improve the precision of the parameter estimates. However, if structural identifiability analysis has concluded that the model is identifiable, the uncertainty in the model parameters is directly linked to the quality of the data and how well the model can describe them. In this scenario, the uncertainty of the model parameters can be improved by increasing the quality of the data. However, there will always be uncertainties in the parameter estimates even if the model is structurally identifiable and the quality and quantity of the experimental data are relatively high. An approach to further strengthen the plausibility of the model predictions under such conditions is to divide the experimental data into two parts: data used for parameter estimation and data used for model validation, i.e., by estimating the unknown parameters using a subset of the experimental data and using the resulting estimates to predict the validation data.

To further exemplify the importance of structural identifiability, consider the two following biological examples. In Evans et al. (2004), a model which aims to describe the activity of an anti-cancer agent named topotecan and its delivery to nuclear target DNA is presented. Prior to parameter estimation it was found that a subset of the model parameters was unidentifiable but if additional experimental measurements were made, in this case determining volume ratios, then the model would become structurally identifiable. In Evans et al. (2001), a parentmetabolite model for ivabradine is considered. The model was shown to be structurally unidentifiable with either intravenous, oral or combined intravenous and oral administration. It was also shown that by either fixing the volume parameter for the central compartment, or with a particular simplification of the model structure, then the model becomes identifiable for the given observations. If a formal structural identifiability analysis had not been performed then these two research projects would have most likely continued without these insights with the potential risk of missleading outcomes.

While PD models are highly diverse, many basic processes involved in drug action are similar across drugs and systems, such as distribution from the plasma to the target tissues, interaction with a target such as receptor binding or altered rates of production or loss of a target. These general processes have been described using semi-mechanistic models. For example, the effect compartment model (Sheiner et al., 1979) has been used to describe short delays in drug action due to distributional delays using a hypothetical "effect compartment." Similarly, receptor binding models (Danhof et al., 2007; Gabrielsson et al., 2011), turnover models (Gabrielsson et al., 2011) and the operational model (Black and Leff, 1983; Danhof et al., 2007) have been used to describe the processes of drug binding and signaling. However, despite frequent use, relatively few PD models have been analyzed from a structural identifiability perspective. An example of a published structural identifiability analysis is for an approximation of the receptor binding model. Receptor binding often occurs over very fast timescales relative to the PK, and sometimes also with respect to the effects elicited by the receptor once bound. In such cases, the receptor binding model may be approximated by a quasi- or pseudo-steady state approximation. When using such an approximation, it has been shown that the individual on and off rates of drug binding to the receptor cannot be uniquely identified (Chappell, 1996). Another example is the target-mediated drug disposition model (Mager and Jusko, 2001) applicable to the modeling of biologics, which has been shown to be structurally identifiable (Eudy et al., 2015). However, the identifiability of the effect compartment model and the operational model have, to our knowledge, not previously been analyzed. Furthermore, mixed effects ("population") models are often used to account for and quantify known sources of variability in data sets, such as between-subject variability (BSV). Such models are combined structural and statistical models, with additional statistical parameters describing the variance of a postulated distribution of the model parameter values across e.g., subjects. The structural identifiability of

mixed effects models describing BSV has not previously been analyzed.

The primary goal of this paper is to illustrate the concept and importance of parameter identifiability, both from a structural and practical perspective. Structural identifiability analysis is performed on a family of 16 commonly used PD models to serve as a database for modelers in the pharmaceutical domain. Both fixed-effects models and the corresponding mixed-effects (population) models are analyzed. Pharmacodynamic models describing combinations of none to three different mechanisms of delays in drug action are analyzed: (i) delays in drug distribution to the site of action applying the effect compartment model, (ii) delays in signal transduction, build-up or loss of effect applying turnover models and (iii) delays due to slow dissociation to the target applying receptor binding models. These and similar models are extensively used within mechanism-based PD models in pharmaceutical research (Ploeger et al., 2009; Peletier and Gabrielsson, 2012). In addition, the problem of structural identifiability and its relation to practical identifiability will be illustrated through a set of examples using both simulated data and real experimental data.

# METHODS

Structural identifiability analysis has been performed on all models written in state-space form. A fixed-effects state-space model is written on the following form

$$\dot{\mathfrak{x}}(t) = f(\mathfrak{x}(t), \mathfrak{u}(t), \theta), \quad \mathfrak{x}(t\_0) = \mathfrak{x}\_0 \tag{2}$$

$$\mathbf{y}(t) = h(\mathbf{x}(t), \boldsymbol{\mu}(t), \boldsymbol{\theta}) \tag{3}$$

where **x**(t) ∈ R n is the state (e.g., plasma concentration of the drug, bound and unbound receptors etc.) **u**(t) ∈ R q is the input (IV bolus, IV infusion etc.), θ ∈ R p is the vector of model parameters (e.g., clearance rate, maximum saturation, etc.), **y**(t) ∈ R <sup>m</sup> is the output (measurement of plasma concentration, drug effects) and **f** and **h** are smooth functions as C<sup>∞</sup> with respect to the functional arguments.

A mixed-effects model is written on one of the forms

$$\dot{\mathbf{x}}\_i(t) = f(\mathbf{x}\_i(t), \boldsymbol{\mu}\_i(t), \boldsymbol{\phi}\_i) \quad \mathbf{x}\_i(t\_0) = \mathbf{x}\_0(\boldsymbol{\phi}\_i) \tag{4}$$

$$\mathcal{Y}\_i(t) = h(\mathfrak{x}\_i(t), \mathfrak{u}\_i(t), \mathfrak{\phi}\_i) \tag{5}$$

where φ<sup>i</sup> = g(θ, η<sup>i</sup> , **C**i) are the parameters for the i:th subject, η<sup>i</sup> ∼ N(**0**, ) are the random effects variables where is the variance-covariance matrix of the random effects η<sup>i</sup> , θ are the population parameters and **C**<sup>i</sup> are the covariates for the different subjects in the population.

#### Structural Identifiability: Definition

As mentioned in the introduction, structural identifiability is a theoretical concept with direct practical relevance. This is because if a model is structurally unidentifiable, some of the model parameters may take on arbitrary numerical values while the model may still describe the experimental data equally well. In a numerical structural identifiability analysis different numerical values are sought that will result in identical model responses. In an analytical structural identifiability analysis, more general conclusions can often be drawn since in such as analysis symbolic relationsships between the model parameters can be derived allowing for suggestions of reparametrization and/or additional measurements required to render an unidentifable model to become identifiable. Since different values of the unidentifiable parameters result in identical responses or predictions any subsequent biological interpretations of the estimates of those unidentifiable parameters (e.g., clearance, IC50) are effectively meaningless in a biological context. It is because of this that structural identifiability is often referred to as a prerequisite to successful parameter estimation. In other words, if a structural identifiability analysis (in which perfect experimental conditions e.g., noise-free and continuous measurements, are assumed) has shown that some of the model parameters can not be determined, it follows directly that these parameters can never be determined in the less ideal case, i.e., under real experimental conditions for discrete measurements with noise present.

To define exactly what is meant by structural identifiability there now follows a more rigorous mathematical definition of the concept in the context of fixed-effects models.

Let the generic parameter vector θ belong to a feasible parameter space 2, i.e., θ ∈ 2. Let y(t, θ) be the output function from the state-space model. Further, consider a parameter vector ¯θ where **y**(t, θ) = **y**(t, ¯θ) for all t. If this equality, in a neighborhood **N** ⊂ 2 of θ, implies that θ = ¯θ then the model is structurally locally identifiable. If **N** = 2 then the model is structurally globally identifiable. If a model is structurally unidentifiable, then every neighborhood of θ contains a ¯θ 6= θ such that y(t, θ) = y(t, ¯θ) for all t.

Since the mixed-effects models to be considered in this paper are also analyzed from a structural identifiability perspective it must first be defined what is meant by the identifiability of such models. Since mixed-effects models yield individual predictions, in contrast to single predictions in the fixed-effects case, the previous definition is not immediately applicable to mixedeffects models. Instead, a generalized version of the definition of structural identifiability is used. In this new definition, first presented in Janzén et al. (2016), a model is defined to be structurally identifiable if the distribution of the output from the model determines both the structural and statistical parameters, i.e., the parameters in the vector θ and the variance parameters in denoting the variance of the random effects η respectively. Now follows a more rigorous definition of structural identifiable for mixed-effects models.

Let p(**y**{θ,} , t) denote the distribution of the output signals **y** at time t. Let the generic parameter vector and matrix {θ, } belong to a feasible parameter space {θ, } ∈ 2, and consider the following two sets of parameters {θ, } and { ¯θ, ¯ }. If p(**y**{θ,} , t) = p(**y**{ ¯θ,¯ } , t) for all t implies that {θ, } = {¯θ, ¯ } in a neighborhood N ⊂ 2 then the model is structurally locally identifiable, and if N = 2 the model is structurally globally identifiable. For a structurally unidentifiable parameter, θ<sup>i</sup> , or ω<sup>i</sup> ∈ , every neighborhood N around θ<sup>i</sup> , or ω<sup>i</sup> , has a parameter vector/matrix ¯θ, or ¯ , where θ<sup>i</sup> 6= θ¯ i , or ω<sup>i</sup> 6= ¯ω<sup>i</sup> , give rise to the same distribution of identical input-output relations.

# Investigated Model Structures

The model structures investigated to determine structural identifiability were all combinations of sub-models representing receptor binding, a hypothetical effect compartment and direct or indirect transduction (see **Figure 2**). In total, 16 different model structures were investigated (**Table 1**). Both fixed-effects and mixed-effects versions of each model were analyzed from a structural identifiability perspective.

#### Structural Identifiability: Example

To exemplify the structural identifiability analysis, a summary of the analysis of the structural identifiability of Model 13 (**Table 1**) is provided. This model is a dynamic receptor binding model with an effect compartment and linear transduction. The details of the structural identifiability analysis for this model is available in the Supplementary Materials. The mathematical model has the following structure

$$\begin{aligned} \dot{C}\_e &= k\_{e0} (C\_p - C\_\varepsilon) \\ \dot{RC} &= k\_{on} (R\_{tot} - RC) C\_\varepsilon - k\_{off} RC \\ E &= k\_\varepsilon RC \end{aligned} \tag{6}$$

with the unknown parameter vector θ = (ke0, ke, kon, koff ) and where C<sup>p</sup> is the concentration in the blood plasma and is in this case a known input signal, C<sup>e</sup> is a state representing the concentration in the hypothetical effect compartment, RC is the receptor complex, E is the observed effect and Rtot representing the percentage of total number of receptors, which is fixed at 100%.

The approach chosen to study structural identifiability here is the input-output approach, for which details can be found in Bearup et al. (2013). A general outline of the method is given here followed by an example of how a structural identifiability analysis is performed.

The input-output approach used in this paper was chosen for three reasons. The first reason was because the input-output approach can be used to show whether a model is globally or locally identifiable, or unidentifiable. Some of the other methods that are available for performing a structural identifiability analysis can only be used to show whether a model is at least locally identifiable or unidentifiable. The second reason was that there is a direct extension from non-population (fixed-effects) models to population (mixed-effects) models when it comes to structural identifiability analysis using the input-output approach as will be explained further below. The third reason is because the method is applicable to both linear and nonlinear models.

The main idea behind the input-output form approach is to transform the model to a form from which the identifiability problem can more easily be studied. This is performed by iteratively computing higher order time derivatives of the output function and using subsequent substitution to eliminate all state variables in order to express the system as a monomial solely in terms of the output functions(s) and its (their) derivatives. As the assumption of perfect experimental conditions is made, it follows that the output function and its higher order derivatives are assumed to be known. In other words, a model rewritten on an input-output form is a single equation with the output

function and its higher order derivatives being known and the model parameters (that enters as the monomial coefficients) being unknown. Determining whether a model is structurally identifiable or otherwise is then a case of showing whether the resultant input-output equation has a single, finite, or an infinite number of solutions for the parameters in the coefficient expressions.

By iteratively differentiating the output signal and eliminating the state variables the model can be rewritten in the following input-output form

$$-R\_{\rm tot}\,^2C\_pk\_e^2k\_{e0}k\_{on} - 2\,R\_{\rm tot}\,^C\_pk\_ek\_{e0}k\_{on}E + R\_{\rm tot}\,k\_ek\_{e0}k\_{off}E - $$

$$-C\_pk\_{e0}k\_{on}E^2 + R\_{\rm tot}\,k\_ek\_{e0}\dot{E} + R\_{\rm tot}\,k\_ek\_{off}\dot{E} + k\_{e0}k\_{off}E^2 + $$

$$R\_{\rm tot}\,k\_e\dot{E} + k\_{e0}E\dot{E} + E\dot{E} - \dot{E}^2 = 0. \tag{7}$$

The structural identifiability of a model can then be studied by considering the coefficients in the input-output form of the model. Introducing an alternative parameter vector ¯<sup>θ</sup> and collecting the coefficients in the input-output form as

$$\sum\_{k=1}^{l} c\_k(\theta, \bar{\theta}) \phi\_k(E(t, \theta), \dot{E}(t, \theta), \ddot{E}(t, \theta), \dots) = 0 \tag{8}$$

permits determination of whether the model is structurally identifiable or otherwise, given that the φ<sup>k</sup> (·) are linearly independent. The analysis shows that <sup>θ</sup> <sup>=</sup> ¯θ, meaning that model 13 is structurally globally identifiable (details are given in the Supplementary Materials).

Similarly, a mixed-effects version of the model can be studied by using the coefficients in the input-output relation. As outlined and discussed in detail in Janzén et al. (Under review), since individual estimates are obtained in a mixed-effects model a distribution, assuming an infinite number of subjects (i.e., ideal experimental conditions in a mixed-effects context), of ck (θ) is in turn obtained. This distribution is directly linked to the distribution of the output functions. By introducing the random effects on the coefficients from the input-output form, according to the statistical sub-model, functions of random variables are derived. By studying whether the distributions of the generated functions of random variables determine both the fixed effects and the random effects related parameters, conclusions regarding whether the mixed-effects model is structurally identifiable or otherwise can be made. The mixedeffects version of model 13 with lognormally distributed random effects on all model parameters with a diagonal covariance matrix is also structurally globally identifiable. This follows from the fact that the structural model has been shown to be structurally globally identifiable (detail in the Supplementary Material) and the statistical parameters are uniquely determined by the lognormal distribution.

It is worth mentioning that structural identifiability analysis using analytical techniques such as the input-output approach may encounter certain limitations in terms of model size and complexity. In general, the more complex a model is in terms of the state-space dimensions and number of unknown parameters, then the more computationally demanding the subsequent analysis may become. If an analytical approach is not possible due to symbolic computational intractability then a hybrid symbolic/numerical analysis approach is an alternative, see the profile likelihood approach (Raue et al., 2009) or the Exact Arithmetic Rank approach (Karlsson et al., 2012). For an extensive comparison between the profile likelihood approach, the Exact arithmetic Rank approach and a differential algebra approach implemented in a software called DAISY, see Raue et al. (2014).

#### Practical Identifiability

Once the structural identifiability of the postulated model has been determined, parameter estimation can be performed. As with the structural identifiability example, Model 13 (**Table 1**) was selected for the simulation study to investigate the influence of varying data quality on the practical identifiability of the parameters. This model includes two different sources of delay, one from distribution to the effect site, where the rate is controlled by the parameter ke0, and also through slow receptor dynamics, where the off-rate is controlled by the parameter koff . The possibility to distinguish the two different delays in practice



N, Model number; I/O, Model inputs/outputs; ICs, Initial conditions. <sup>a</sup>Rtot was fixed at 100 when analysing the mixed effects models. <sup>b</sup>Each mixed-effects model was assumed to have a diagonal covariance matrix with lognormally distributed random effects.

under varying data quality was investigated in a simulation study. Rtot was fixed to 1 following the results of the structural identifiability analysis to ensure the structural identifiability of the model. True parameter values were assigned to each model parameter: k<sup>e</sup> = 1, ke<sup>0</sup> = 0.2, koff = 0.02 and kon = 0.05 amounts per minute. All parameters were assumed to vary between subjects following a log-normal distribution as this ensures positive rates for all subjects, with standard deviation σ = 0.3 amounts per minute to represent differences in a population. The model is summarized in **Figure 3A**.

The simulation study was performed in MATLAB 2013b (The MathWorks, Inc., 2016) and Monolix 4.3.2 (Lixoft, 2012) as outlined in **Figure 3B**. (1) PK data were simulated without variability or noise, applying an intravenous bolus dose of 20 mg/kg to a hypothetical typical individual with volume of distribution 1 and rate of elimination 0.2 mg/kg. (2) Model 13 with the selected "true" parameter values was used to simulate data sets of varying size and quality. Three factors were changed that influence the information available in the data: (i) different sampling densities 1t = 1, 2, 5, 10, 15, 20, 25, 30 min. (ii) different additive noise levels σ = 0.05, 0.15, 0.5 response units and (iii) different numbers of subjects n = 100, 40, 12. (3) Parameters were estimated using each simulated data set, with the following initial guess selected for the optimization algorithm: k<sup>e</sup> = 1, ke<sup>0</sup> = 0.1, koff = 0.01 and kon = 0.01 units per minute for the structural parameters and 0.3 units per minute for the

parameter estimates for the combined effect compartment/dynamic receptor model fitted to simulated data. The accuracy of each parameter (y-axis: estimated/true parameter value, line of unity marked by black line) and its uncertainty (normalized standard error, filled lighter area) is given at each data resolution level (x-axis: time between samples increasing from 1, 2, 5, 10, 15, 20, 25 to 30 min) for simulated data for 100, 40 and 12 subjects (row 1, 2, 3) adding additive noise with standard deviations 0.05, 0.15, and 0.5 response units (r.u, column a, b, c) respectively.

standard deviations. (4) The ratio between the final parameter estimates and the true parameter values were calculated and compared for the typical parameters to investigate the effects of varying sampling frequency, noise levels and number of subjects on parameter accuracy.

### RESULTS

#### Structural Identifiability Analysis

The results of the structural identifiability analysis applying the input-output approach are summarized in **Table 2**, including the structural identifiability results and the reparameterization solutions to achieve structurally identifiable models.

All fixed effects versions of the models were in their original parameterization shown to be structurally unidentifiable. For all of the models, the source of the unidentifiability problem was the parameters Rtot and either RC<sup>50</sup> (Models 2, 6, 10, 14) or k<sup>e</sup> (remaining models) (see **Table 2**). The analysis showed that these parameters are unidentifiable and therefore any numerical estimates of them are effectively meaningless from a biological perspective. Furthermore, it was shown that even though Rtot and k<sup>e</sup> or RC<sup>50</sup> are unidentifiable, the product Rtotk<sup>e</sup> and fraction Rtot/RC<sup>50</sup> are globally identifiable. The remaining parameters in the analyzed models were all shown to be globally identifiable. Therefore, three methods may be applied to ensure structurally globally identifiable models: (1) A new parameter may be defined as Rtotke, representing the effect when all targets are bound, and Rtot/RC50, representing the transducer ratio, to replace the unidentifiable parameters. (2) Rtot or (3) k<sup>e</sup> and RC<sup>50</sup> may be fixed to known or assumed numerical values. However, this affects the units and interpretation of the non-fixed parameter. For example, Rtot may be fixed at 100%, resulting in changed units for k<sup>e</sup> to units per percent bound receptor.

As discussed in Janzén et al. (Under review), if the structural model is structurally globally identifiable, and if the statistical sub-model is structurally globally identifiable, it follows that the mixed-effects model is also structurally globally identifiable. The statistical sub-model for the random effects considered in this paper takes the form of the structurally globally identifiable lognormal distribution. Therefore, the mixed-effects versions of the models in **Table 1** are structurally globally identifiable following the reparameterization or fixing of Rtot or ke.

#### Simulation Study of Practical Identifiability

In the simulation study, increasing noise, reducing sampling frequency and reducing the number of subjects all led to worse parameter estimation results (**Figure 3**). At the lowest noise level (column a), the model parameters were well estimated up to a sampling density of 1t = 10, while increasing the sampling interval above this level led to over- and underestimation of k<sup>e</sup> and kon respectively. At the intermediate noise level (column b), similar results were obtained, although problems occurred at smaller sampling intervals. At the highest noise level (column c), the parameter estimation was unsuccessful for all estimation runs except for 100 subjects and 1 min sampling interval. The simulation study shows a trend of decreasing accuracy to estimate the true parameters when the amount and quality of the data decreases. Some of the model parameters vary more than others when the data become worse in terms of noise levels, the number of measurements and the number of subjects. For instance, koff was estimated reasonably well, except for the very worst case 3c, while the estimates for k<sup>e</sup> and kon are poor in 1a. It can also be seen that the uncertainty in the parameter estimates (standard errors) generally widens with either increased noise, reduced sampling density or reduced number of number subjects. Interestingly, high precision (small


TABLE 2 | Results of the structural identifiability analysis of the mixed-effects models 1–16 in Table 1.

SU, Structurally unidentifiable; SI, Structurally identifiable. <sup>a</sup>Rtot was fixed at 100 when analysing the mixed effects models.

Structurally identifiable and unidentifiable parameters and a suggested reparameterization are provided for the corresponding fixed effects models. Random effects were evaluated for the reparameterized models.

standard errors) is in many optimizations acquired despite low accuracy in the parameter estimates.

### Case Study: Analysing Cardiac (Side) Effects

A case study was conducted in order to exemplify the process of model development, including structural identifiability analysis. Side effects of potential new drugs on the heart must be evaluated by monitoring changes in the duration of specific intervals monitored in the electrocardiogram (ECG), such as the QT interval (defined by the Q and T peaks in the ECG) which corresponds to the duration of the ventricular action potential. The main part of the QT interval constitutes the ventricular repolarization phase, corresponding to the JT interval (defined by the J point and T peak in the ECG), and prolongations are strongly linked to inhibition of the cardiac ion channel hERG (Pollard et al., 2010). In this example, model 10 (**Table 1**) was applied to link inhibition of the hERG ion channel in vitro to prolongation of the JT interval following treatment with the anti-arrhythmic compound and mixed ion channel blocker AZD1305, a proprietary AstraZeneca compound. Model 10 was selected since an identifiable version of this model has been used previously to fit this type of data (Jonker et al., 2005) and following evaluation of additional structures, for example model 2 (without the effect compartment).

#### Methods

Clinical study and PK and QT interval data are described in Parkinson et al. (2013). This phase I study was performed in accordance with the ethical principles of the Declaration of Helsinki and is consistent with the International Conference on Harmonisation (ICH)/Good Clinical Practice. JT intervals were calculated by subtracting QRS from QT. In vitro data were acquired from the original data collected by Carlsson et al. (2009). Methods for PKPD model development are detailed in Bergenholm et al. (2016). Baseline variability of JT intervals was minimized applying a circadian rhythm and RR correction models (Chain et al., 2011; Bergenholm et al., 2016). The PK and PD were modeled sequentially, and Model 10 (**Table 1**) was selected to describe the drug effect. K<sup>d</sup> was estimated prior to the PKPD modeling using the Imax model, where the inhibition in % is calculated according to

$$I(\text{C}) = 100 \ast \text{C} / (IC\_{50} + \text{C}) \tag{9}$$

where IC<sup>50</sup> corresponds to the drug concentration resulting in 50% inhibition, substituting K<sup>d</sup> in Model 10. Parameter estimations were performed using the stochastic approximation expectation maximization (SAEM) algorithm as implemented in Monolix 4.3.2 (Lixoft, 2012).

#### Results

The estimated IC<sup>50</sup> of hERG was 0.37 ± 0.04 µM with between cell variability of 0.19 ± 0.09 µM. Fitting all parameters of the operational model led to high uncertainty and correlation between Rtot and RC<sup>50</sup> (**Table 3**). Structural identifiability analysis of this model showed that only the fraction Rtot/RC<sup>50</sup> is identifiable (see **Table 2**) and the model was therefore reparameterized with τ = Rtot/RC50, resulting in a structurally identifiable model. Estimation of the reduced model resulted in



SE, Standard error; BSV, Between-subject variability; −2LL, −2 LogLikelihood.

similar parameter values for all of the identifiable parameters, similar goodness of fit values and residuals and good precision in the population estimate of τ (**Table 3**). The fits to the generated data can be seen in **Figure 4**.

#### Discussion

Both the full and reparametererized versions of model 10 described the data well. However, standard errors and correlations of Rtot and RC<sup>50</sup> correctly indicated identifiability issues with the former. The estimated parameters were converted to the traditional Emax and EC<sup>50</sup> parameters, which describe the maximal effect and the drug concentration at half-maximum effect respectively. Emax and EC<sup>50</sup> were calculated according to

$$E\_{max} = \frac{E\_m \pi^n}{1 + \pi^n} \tag{10}$$

$$EC\_{50} = \frac{IC\_{50}}{(2 + \pi^n)^{1/n} - 1} \tag{11}$$

and resulted in an estimated Emax of 117 ms and 116 ms and EC<sup>50</sup> of 0.36 and 0.35 µM respectively for the full and reparameterized models. This highlights that identifiable parts of a structurally unidentifiable model are still informative. The estimated Emax is similar to that in previous hERG-QT modeling of dofetilide (Jonker et al., 2005), while the estimated hERG block at 10 ms JT prolongation was slightly higher (18 vs. 9%). This may be caused by AZD1305-induced calcium block (Carlsson et al., 2009), as the calcium current depolarizes the cardiac cells (Amin et al., 2010), counter-acting the repolarization by hERG. The structural identifiability analysis showed that two model parameters could not be estimated. This led to model reduction. Performing this analysis prior to parameter estimation ensures the theoretical possibility of estimating all parameters in the model. Estimating the parameters of the unidentifiable model could have been avoided, reducing the number of iterations in the optimization. Also, ensuring structural identifiability improves confidence in the biological interpretation of the estimated parameter values.

#### DISCUSSION

Unidentifiability issues can cause many different types of problems if not mitigated when models are used to quantify, predict and understand the effects of potential drugs. Most importantly, the biological/physiological interpretations of structurally or practically unidentifiable parameters are not valid. This may lead to wrong conclusions, for example when unknowingly comparing unidentifiable parameters to rate candidate drugs or for comparison with competitors. Also, any predictions based on the profiles of unmeasured states of the system may be meaningless if the parameters directly or indirectly related to those states are unidentifiable. For example, if the effect of interest in a toxicity or efficacy study depends on the concentration in a compartment for which the profile is linked to structurally unidentifiable parameters, it may be impossible to separate the distribution to this compartment and the drug effect. Unidentifiability issues may also cause technical problems, as the parameter estimation step may take a very long time, or fail (crash), if a structurally unidentifiable model is used (depending on what form of optimization routine is used).

We have investigated the structural identifiability of 16 fundamental pharmacodynamic models and identified parameterizations that are structurally identifiable both for fixed effects- and mixed-effects- versions of the models, as summarized in **Table 2**. For all of the investigated models, the total amount of receptor in the system was fixed (to e.g., 1 or 100%) in order to achieve structural identifiability. This implies that some parameters for the "signal transduction" are relative. For example, the units of a proportional signal transduction are effect units per fraction bound/inhibited receptor if Rtot is fixed to 1. This analysis shows that given sufficient data quality, it is, in theory, possible to distinguish between different sources of delay from the data. Thus, it is possible to differentiate delays that are compound-specific (e.g., distribution, drug-receptor binding kinetics) from delays that are system-specific (e.g., turnover of receptors) to compare compounds and simulate untested systems. The investigated models have been used successfully and repeatedly in practice (Ploeger et al., 2009; Peletier and Gabrielsson, 2012), and our results confirm the general assumption of structural identifiability. This provides confidence in the theoretical soundness of using these models.

Next, we estimated parameters of the unidentifiable and reparameterized versions of Model 5 (**Tables 1**, **2**) in to investigate the possible consequences of estimating unidentifiable models (**Figure 5**). Three separate runs of parameter estimation were performed. Parameters in the unidentifiable version of the model were estimated in two different runs using different initial estimates. For the third parameter estimation run, the model was reparameterized following insights from the structural identifiability analysis.

Investigating the estimated parameters shows that standard errors of unidentifiable parameters differ significantly between the two estimation runs, and are larger than the standard error of the product of the parameters. For one of the estimation runs the magnitude of the standard errors (37.6 and 33.7%) did not clearly indicate a structural identifiability problem. In

FIGURE 5 | (A) Results of the structural identifiability analysis of Model 5. (B) Optimization results following estimation of unidentifiable and identifiable versions of Model 5 using example data.

the second estimation run the standard errors (163 and 154%) did indicate a structural identifiability problem. However, for both estimation runs the estimated correlations between Rtot

and k<sup>e</sup> were −0.9 and −0.99 respectively, indicating a potential structural identifiability problem in both cases. Alternatively, analysing the models using the profile likelihood approach (Raue et al., 2009) would also potentially indicate a problem with structural and practical identifiability. Although estimation of an unidentifiable model in theory should lead to infinitely large uncertainty for the structurally unidentifiable parameters due to a flat likelihood function in the directions representing those parameters, this did not happen in practice. The reason why this did not happen can be explained by measurement and numerical noise. In real-world problems, the likelihood function is never completely flat which introduces false local minima where the optimization routine may become "stuck" depending on the initial guesses used for the model parameters and the optimization algorithm itself. This example shows the potential danger of using practical identifiability analysis as a tool to deduce structural identifiability. For the first set of initial guesses for the parameters, the reported RSE-values are unreasonably high indicating a structural identifiability issue. However, the RSE-values reported using a different set of initial guesses for the model parameters do not indicate that there is any structural identifiability problem. The results of these estimations were used to draw some general conclusions. These are as follows:


Similar findings regarding masking of structural unidentifiability, i.e., estimation of seemingly reasonable RSE-values of structurally unidentifiable parameters, has been reported in the conference contribution (Aoki et al., 2015) and in the follow-up paper (Aoki et al., 2014). These findings were reported using NONMEM, rather than Monolix, which indicates that estimation of misleading RSE-values under structural unidentifiability conditions is not a software specific issue but instead a general numerical computational instability issue. In these two publications, a numerical approach called preconditioning is suggested. In short, this approach involves reparametrization of the model in such a way so that the subsequent numerical computations of the RSE-values reportedly becomes more stable and thus more reliable under structurally unidentifiable conditions.

It is important to remember that having a structurally identifiable model is only a prerequisite for successful parameter estimation. In other words, that parameters are identifiable with ideal data (continuous, noise-free data from an infinite number of subjects in the mixed effects model case) does not guarantee that they will be practically identifiable with a finite number of noisy data points from a finite number of subjects.

The effects of practical identifiability were investigated in a simulation study, where the quality of the data was varied from good to worse, but the structural model was known to be identifiable (Model 13). Conclusions from this example are that:


When the data do not contain information on the time scale of the rate parameters in the system, the model should be reduced to only account for effects over the relevant time scales. This applies even when all parameters are structurally identifiable.

# CONCLUSIONS

Parameter identifiability should be investigated to ensure both structural and practical identifiability. Our work confirms the structural identifiability of a set of fundamental pharmacodynamic models, and provides examples of estimation results with unidentifiable models. The investigated models have been proven to have a sound theoretical basis in terms of structural identifiability and thus are reliable in this respect. This in turn increases the reliability of using such models in clinical pharmacology and therapeutics.

# AUTHOR CONTRIBUTIONS

All authors participated in the design of the research study. DJ performed the structural identifiability analyses. LB and DJ performed the practical identifiability analyses. LB conducted the analyses for the case study. All authors participated in analysing the results. All authors participated in writing the manuscript.

# ACKNOWLEDGMENTS

The case study was conducted specifically for this publication, using previously published clinical data (Parkinson et al., 2013). This work is funded through the Marie Curie FP7 People ITN European Industrial Doctorate (EID) project No.316736, IMPACT (Innovative Modeling for Pharmacological Advances through Collaborative Training).

# SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: http://journal.frontiersin.org/article/10.3389/fphys. 2016.00590/full#supplementary-material

# REFERENCES


medicinal chemistry. Curr. Top. Med. Chem. 11, 404–418. doi: 10.2174/ 156802611794480864


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2016 Janzén, Bergenholm, Jirstrand, Parkinson, Yates, Evans and Chappell. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Spatiotemporal Dynamics of Insulitis in Human Type 1 Diabetes

Kyle C. A. Wedgwood<sup>1</sup> \*, Sarah J. Richardson<sup>2</sup> , Noel G. Morgan<sup>2</sup> and Krasimira Tsaneva-Atanasova3, 4

*<sup>1</sup> Centre for Biomedical Modelling and Analysis, University of Exeter, Exeter, UK, <sup>2</sup> University of Exeter Medical School, University of Exeter, Exeter, UK, <sup>3</sup> College for Engineering, Mathematics and Physical Sciences, University of Exeter, Exeter, UK, <sup>4</sup> Engineering and Physical Sciences Research Council Centre for Predictive Modelling in Healthcare, University of Exeter, Exeter, UK*

Type 1 diabetes (T1D) is an auto-immune disease characterized by the selective destruction of the insulin secreting beta cells in the pancreas during an inflammatory phase known as insulitis. Patients with T1D are typically dependent on the administration of externally provided insulin in order to manage blood glucose levels. Whilst technological developments have significantly improved both the life expectancy and quality of life of these patients, an understanding of the mechanisms of the disease remains elusive. Animal models, such as the NOD mouse model, have been widely used to probe the process of insulitis, but there exist very few data from humans studied at disease onset. In this manuscript, we employ data from human pancreases collected close to the onset of T1D and propose a spatio-temporal computational model for the progression of insulitis in human T1D, with particular focus on the mechanisms underlying the development of insulitis in pancreatic islets. This framework allows us to investigate how the time-course of insulitis progression is affected by altering key parameters, such as the number of the CD20+ B cells present in the inflammatory infiltrate, which has recently been proposed to influence the aggressiveness of the disease. Through the analysis of repeated simulations of our stochastic model, which track the number of beta cells within an islet, we find that increased numbers of B cells in the peri-islet space lead to faster destruction of the beta cells. We also find that the balance between the degradation and repair of the basement membrane surrounding the islet is a critical component in governing the overall destruction rate of the beta cells and their remaining number. Our model provides a framework for continued and improved spatio-temporal modeling of human T1D.

Keywords: type 1 diabetes, insulitis, agent-based modeling, spatio-temporal dynamics, peri-islet basement membrane

# 1. INTRODUCTION

Type 1 diabetes (T1D) is an auto-immune disease characterized by the selective destruction of pancreatic beta cells in the islets of Langerhans by the immune system (Eisenbarth, 1986; Atkinson, 2012; Boitard, 2012; La Torre and Lernmark, 2012; Pugliese, 2014; Richardson et al., 2014; Roep and Tree, 2014). This destruction takes place during an inflammatory phase, known as insulitis, in which various immune cells infiltrate the islets (Lecompte, 1958; Gepts, 1965; Willcox et al., 2009; Morgan et al., 2014). As the beta cell mass decreases over the course of the disease, the ability of

#### Edited by:

*Joseph L. Greenstein, Johns Hopkins University, USA*

#### Reviewed by:

*Josep Bassaganya-Riera, Virginia Tech, USA Anmar Khadra, McGill University, Canada*

\*Correspondence: *Kyle C. A. Wedgwood k.c.a.wedgwood@exeter.ac.uk*

#### Specialty section:

*This article was submitted to Computational Physiology and Medicine, a section of the journal Frontiers in Physiology*

Received: *16 September 2016* Accepted: *05 December 2016* Published: *27 December 2016*

#### Citation:

*Wedgwood KCA, Richardson SJ, Morgan NG and Tsaneva-Atanasova K (2016) Spatiotemporal Dynamics of Insulitis in Human Type 1 Diabetes. Front. Physiol. 7:633. doi: 10.3389/fphys.2016.00633*

**100**

the islets to secrete sufficient quantities of insulin to properly regulate blood glucose levels becomes compromised. As a result, patients with type 1 diabetes ultimately become reliant on the lifelong administration of external insulin.

Well-defined genetic components have been identified which predispose individuals to T1D; in particular the HLA-genotype (Itoh et al., 1993; Somoza et al., 1994). However, this alone is not sufficient to predict which individuals will develop the disease with precision. Since monozygotic twin studies have shown limited pairwise concordance for T1D (Barnett et al., 1981; Lo et al., 1991; Redondo et al., 2001), it is clear that environmental factors, such as viral infection, vitamin D status and childhood nutrition may also contribute significantly to the development of T1D (Knip et al., 2005).

Among the different subtypes of immune cells which infiltrate islets, CD8+ T cells are considered as the likely mediators of beta cell destruction (Bottazzo et al., 1985; Itoh et al., 1993; Somoza et al., 1994). It is widely believed that these promote beta cell apoptosis by both direct and indirect mechanisms and that macrophages then clear dying and dead beta cells very quickly. Other T lymphocytes, such as those expressing CD4, are also thought to play a role, though their precise functions are less clear (Willcox et al., 2009; Richardson et al., 2011). In addition to the T cells, B lymphocytes (CD20+) are also present in significant numbers during certain stages of insulitis in some patients. Indeed, recent evidence has suggested that the number of B cells and/or the ratio of B-cells to CD4+ cells present in the infiltrate can be used to classify the disease into two distinct phenotypes. These have been defined as "hyper-immune," characterized by elevated numbers of CD20+ cells and a rapid loss of beta cell mass whereas, by contrast, the "pauci-immune" phenotype, is associated with a lower proportion of B cells and a much slower destruction of beta cells (Morgan et al., 2014; Leete et al., 2016). The mechanisms by which the B cells affect the rate of disease progression are unknown, but it is possible that they collaborate with CD8+ T cells to drive beta cell loss (Huppa and Davis, 2003).

Progress in understanding the cellular and molecular mechanisms underlying insulitis in humans has been hindered by the paucity of available samples from patients who died at, or close to, disease onset. Fewer than 200 such samples are available worldwide (Gepts, 1965; Foulis and Stewart, 1984; Klöppel et al., 1985; Dotta et al., 2007; Walker et al., 2011; Campbell-Thompson et al., 2012; Pugliese, 2014), and inferring the time course of a disease process from histological samples is fraught with difficulty since a range of assumptions and extrapolations about the likely progression are inevitably required to achieve this.

To offset this problem, much work has been performed using animal models such as the non-obese diabetic (NOD) mouse as a proxy for the human condition (Kachapati et al., 2012). The advantages of this are clear—experiments can be performed over relatively short periods of time and analysis of circulating and pancreatic lymphyocyte populations is achieved more readily during the course of disease. Such studies have, therefore, been extremely informative as a means to identify important cellular and molecular factors involved (Lally and Bone, 2003), although considerable effort needs to be made to verify that results are translatable to humans. While a large number of potential therapies have been identified in rodents, a means to prevent the human disease remains elusive (Brehm et al., 2012; In't Veld, 2014; Pugliese et al., 2014; Reed and Herold, 2015).

An alternative approach, which may shed light on human insulitis is offered by mathematical modeling. By constructing environments that mimic the pancreas and immune system, experiments can be performed in silico, and the results compared to empirical data arising from studies of relevant human tissue. Through model construction and analysis, candidate mechanisms giving rise to T1D can be interrogated in a systematic way. To this end, a number of authors have developed models describing: beta cell function (Bertram and Sherman, 2004), defective macrophage clearing (Marée et al., 2006), immune cell populations (Mahaffy and Edelstein-Keshet, 2007), multi-clonal populations of immune cells (Khadra et al., 2009, 2011; Jaberi-Douraki et al., 2015), immune cell cycles (Jaberi-Douraki et al., 2014b), and apoptotic stress generated by the loss of beta cell mass (Jaberi-Douraki et al., 2014c).

A common theme among the models outlined above is that they can be regarded as "lumped models," in that they deal with averaged quantities taken over the whole body. For certain applications, this seems appropriate, as it is commensurate with current clinical practice. The biomarkers clinicians have available are typically derived from blood samples, in particular, measures of HbA1c and fasting glucose levels (American Diabetes Association, 2006; Inzucchi, 2012). Other biomarkers, such as measures of C-peptide (NIC, 2015) and islet cell and anti-insulin antibodies (Taplin and Barker, 2008) can be used to identify individuals who may be susceptible to diabetes and, though it is not currently routine clinical practice, they can also be used to aid classification of diabetes (Jones and Hattersley, 2013). Biomarkers obtained from blood samples are, by their nature, whole body measures. However, it is known from histological samples that insulitis displays a pronounced spatial dependence (Willcox et al., 2009; Morgan et al., 2014). This is due in part to the distribution of the islets within the pancreas, but is also a consequence of the presence of a basement membrane around the islets (Korpos et al., 2013), establishing a barrier to immune cell infiltration. Interactions and communication between immune cells also play a role and are likely to contribute to this spatial dependence.

Insulitis is a spatially heterogeneous process, both within an individual islet and across the pancreas as a whole. Islets that are heavily infiltrated can be located near to islets that are free from inflammation. Moreover, within an infiltrated islet, the destruction of beta cells appears not to follow a "wave-like" profile, as might be expected, but seems more random (Willcox et al., 2009; Morgan et al., 2014). These observations lend credence to the notion that immune cell communication is a critical component of insulitis.

To investigate insulitis and its relevance to the progression of T1D, these spatial aspects must thus be taken into account. Moreover, since the number of immune cells infiltrating any given islet is low (Willcox et al., 2009; Morgan et al., 2014), density based approaches, such as ordinary differential equation (ODE) modeling, cannot be applied with precision and alternatives are required.

In this manuscript, we construct an agent-based model of the insulitis process within a single islet. The model is posed on a regular domain that represents the space surrounding the islet. The principal agents in our model are the immune cells and we describe how they locate and target beta cells and how they interact with one another. By using an agent-based approach, we avoid the problems associated with density-based (averaged) descriptions of cell mass. We begin by introducing the relevant biology for our problem. Following this, we describe the development of, and results from, the agent-based model simulations. Finally, we conclude with a discussion of the model and its potential extensions.

#### 2. RELEVANT BIOLOGY

In this section, we highlight the biology corresponding to the components of our model.

#### 2.1. Cell Types

Our model consists of three distinct cell types—two types of lymphocyte and the beta cells. We consider in particular CD8+ T lymphocytes and CD20+ B lymphocytes. Of the various immune cells that have been shown to be involved in human insulitis, these are the ones that are believed to be most relevant to the questions under investigation in our study. The CD8+ cells are implicated in the destruction of the beta cells, while the CD20+ cells are deduced to play an important role, since their absence is associated with a weakly aggressive phenotype (Morgan et al., 2014). In our model, the B cells will essentially act as antigen presenting cells to the T cells (Rodríguez-Pinto, 2005). We assume that macrophages efficiently clear apoptotic beta cells. We do not account for mitosis in any of the cell types; whilst division and selection of relevant lymphocytes has been demonstrated in the lymphatic system, evidence for it in the peri-islet space is minimal (Willcox et al., 2010). Immune cells are assumed to have a finite lifespan and beta cells die following interactions with T cells (Cnop et al., 2010). We do not consider other cell types, such as α and δ cells, that are also found within the islets of Langerhans (Kim et al., 2009).

#### 2.2. Basement Membrane

Individual islets are contained within an encapsulating membrane. This membrane is comprised of various laminins and collagen (Korpos et al., 2013) and acts as barrier to invading immune cells. In mice with T1D, these compounds are lost, suggesting that the basement membrane has been degraded. In our model, we assume that the T cells are responsible for degrading the membrane, but in principle, other cells could be also responsible for this. We also make the assumption that the membrane can be repaired over time.

#### 2.3. Cell Movement

We assume that beta cells have a fixed location, but that immune cells are free to move around. The direction of this movement is mediated by a chemokine gradient, which will be described below (Stein and Nombela-Arrieta, 2005). In addition, immune cells of different types are attracted to one another, instigated by different chemical signaling pathways (Janeway et al., 1985). Cells are not allowed to pass through the basement membrane, nor through each other, but are allowed to remain in contact with one another.

# 2.4. Chemokine

The beta cells are considered to secrete a chemokine signal that attracts immune cells toward the islet (Christen and Von Herrath, 2004). In our study, we assume that the chemokine molecules are bound to the membrane of the beta cells and may also be cleaved off so that they diffuse freely.

#### 2.5. Cell Interactions

In their role as antigen presenting cells, the B cells can cause the T cells to enter an activated state, in which both their sensitivity to the chemokine gradient and their killing efficiency are enhanced (Friedl et al., 2005). We assume that activated T cells have a shorter lifespan than their non-activated counterparts (Green et al., 2003).

When a T cell encounters a beta cell, it triggers a pro-apoptotic pathway within the beta cell, ultimately resulting in the death of the beta cell (Cnop et al., 2005).

The behavior of the cells and membrane are summarized below.

#### 2.6. Summary of Basic Behavior

Below, we will summarize the behavior of each of the cells types included in our modeling.

#### **T cells**


When activated by B cells, T cells become more sensitive to the chemokine gradient and become more effective at killing beta cells.

#### **B cells**


#### **Beta cells**


We do not include in our model the possibility of beta cell replication. Studies have suggested that beta cell proliferation is increased, particularly in the early stages of insulitis (Willcox et al., 2010). Here, we disregard this effect.

#### **Basement membrane**


#### 3. METHODS

#### 3.1. Building a Cellular Automaton

ODE-based modeling approaches average the dynamics associated with insulitis over space essentially representing the entire pancreas. Such approaches have provided numerous insights into the potential roles of T and B cell involvement in T1D and we refer the reader to Jaberi-Douraki et al. (2014a) for a recent and comprehensive review of this literature. However, we know that insulitis is heterogeneous and that the number of immune cells directly involved in the destruction of the beta cells in humans is low - far lower than would be necessary for a density approximation to be justified (Willcox et al., 2009; Morgan et al., 2014). Instead, we employ a different, agent-based, modeling approach (An et al., 2009; Holcombe et al., 2012). Our agents, the immune and beta cells, occupy finite area within a two dimensional space. In a real islet, cells are free to move in three dimensional space, but for ease of modeling (as a starting point of first approximation) and for a more direct comparison with data, we shall here restrict ourselves to the planar case. The agents are given specific rules to locomote, interact with one another and with the basement membrane. Simulations will then be performed to track emergent behavior arising from the basic rules. In this way, the model will evolve over a spatially extended domain in which cell-cell interactions can be explicitly defined.

Note that the rules that govern our agents' dynamics are not intended to be exhaustive lists for all potential behaviors exhibited by the cells. Instead, they are intended to represent a minimal set of interactions that can explain the observed phenomena. Additionally, the model can be iteratively developed to account for new interactions as required.

#### 3.2. Cell Movement

In this modeling framework, the cells are treated as being discrete circular particles (Levine et al., 2000; Palsson and Othmer, 2000; Tijskens et al., 2003; Maini and Baker, 2011; Bruna and Chapman, 2012; Plank and Simpson, 2012). Cell movement is simulated by modeling the forces that act upon cells from different sources and then by resolving these forces using Newton's second law of motion. In the absence of external forces, intrinsic forces acting upon the immune cells cause them to diffuse randomly, obeying a random walk. In the presence of a chemokine gradient, these same cells will seek to move up the gradient. Cells may be attracted to one another and may be in contact with one another for extended periods of time, that is, we allow cells to overlap, but they may not pass through each other.

#### 3.3. Immune Cell Forces

Before discussing the governing equations, it is useful to define indexing sets for each of the cell types. In the following, we define the sets T , B, and β as sets containing the indices for T, B, and beta cells respectively. From Newton's second law of motion, we have:

$$m\_i \frac{\mathbf{d}^2 \mathbf{x}\_i}{\mathbf{d}t^2} = -\eta\_i \frac{\mathbf{d} \mathbf{x}\_i}{\mathbf{d}t} + F\_i + \sigma \xi\_i(t), \quad i \in \mathcal{T} \cup \mathcal{B}, \tag{1}$$

for each cell where x<sup>i</sup> ∈ R 2 is the cell's location, m<sup>i</sup> ∈ R≥<sup>0</sup> is the mass of the cell, η<sup>i</sup> ∈ R≥<sup>0</sup> is the cell's viscosity and F<sup>i</sup> ∈ R 2 is the force acting on the cell. The final term in Equation (1) represents a Gaussian white noise process: hξ (t)i = 0, hξ (t)ξ (t + s)i = δ(t – s), with strength σ such that the immune cells perform random walks in the absence of any other forces (Palsson and Othmer, 2000; Middleton et al., 2014). In biological systems, we assume that the cells have low Reynold's numbers such that inertial forces are small compared to viscous ones. Under this approximation, we can replace (Equation 1) with:

$$\frac{\mathrm{d}x\_i}{\mathrm{d}t} = \frac{1}{\eta\_i} F\_i + \sigma \,\xi\_i(t), \quad i \in \mathcal{T} \cup \mathcal{B}\_i$$

For simplicity, we set η<sup>i</sup> = 1 for all cells. Note that, in a general mathematical framework, specific values for η<sup>i</sup> can be absorbed into the definition of F<sup>i</sup> . For our model definition, we include heterogeneity between immune cells through variations in their sensitivity to chemical gradients, as we shall discuss in Section 3.11. We will also initially assume that the cells within the subgroups are homogeneous with respect to their geometry, that is, they share a common radius rimm.

The force F can be broken up into its constituent parts:

$$F\_i = F\_i^{\text{chemo}} + F\_i^{\text{cell-cell}} + F\_i^{\text{islet}} + F\_i^{\beta}. \tag{2}$$

In the above, F chemo i represents chemotactic force, F cell-cell i represents cell attraction and repulsion, whilst F islet i represents the interaction with the basement membrane and F β i represents interaction with the beta cells.

#### 3.4. Chemokine Signaling

Since the chemokine is a chemical signal, we establish a gradient using a reaction-diffusion equation:

$$\frac{\partial \mathcal{C}}{\partial t} = D \nabla^2 \mathcal{C} + f(\mathcal{C}). \tag{3}$$

We assume that the chemokine evolves on a much faster timescale compared to the movement of the cells, so we assume the chemokine to be at steady state by setting the LHS of Equation (3) to zero. The domain on which we simulate our model represents only a small space around an individual islet. We note that this region is small compared to the entire pancreas, and further that the chemokine is free to diffuse out of our prescribed region. Typically, when solving PDEs, boundary conditions are applied at the edge of the domain over which the equation is being solved. However, as we are treating our region as only a small part of a much larger one, we feel it is more appropriate to not apply boundary conditions, leaving them open. The reaction term is given by local decay with point sources given by the locations of the beta cells. The chemokine equation now reads:

$$D\nabla^2 C = \lambda C - \sum\_{j \in \beta} \nu\_j(t)\delta(x - x\_j),\tag{4}$$

where λ is the degradation rate of the chemokine and x<sup>j</sup> are the centroids of the beta cells. To reflect the fact that dead beta cells will not secrete chemokine, we assume that the production of chemokine is dependent on the current viability of the cell νj(t). For a viable beta cell, ν = 1, whilst dead beta cells have ν = 0. This equation can itself be recast as:

$$QC = \sum\_{j \in \beta} \nu\_j(t)\delta(\mathbf{x} - \mathbf{x}\_j), \quad Q = (\lambda - D\nabla^2). \tag{5}$$

The Green's function for the operator Q at a given time is a Gaussian function and so we lump parameters together and choose a form for the chemokine signal given by:

$$C(\mathbf{x}, t) = \sum\_{j \in \beta} \alpha \upsilon\_j(t) \exp\left(-(\mathbf{x} - \mathbf{x}\_j)^2 / (2\sigma\_C^2)\right),\tag{6}$$

where α now represents the strength of the chemokine signal and σ<sup>C</sup> represents the range over which it decays. In order to represent our assumption that the chemokine exists in forms in which it is bound to the membrane and forms in which it freely diffuses, we replace Equation (6) by:

$$C(\mathbf{x}, t) = \sum\_{k=1}^{2} \sum\_{j \in \beta} \alpha \upsilon\_j(t) C\_0^k \exp\left(-(\mathbf{x} - \mathbf{x}\_j)^2 / (2\sigma\_{C,k}^2)\right), \tag{7}$$

where σC,1 is small and reflects the membrane bound chemokine, whilst C k 0 , k = 1, 2 control the proportion of chemokine that is membrane bound vs. that which is freely diffusing. The immune cells respond to this gradient via:

$$F\_i^{\text{chemo}} = \mathcal{S}\_i \nabla C\_i$$

where S<sup>i</sup> is the sensitivity of the i'th cell to the chemokine gradient. We note that this represents the notion that immune cells tend to move up a potential gradient in which the locations of the beta cells are given by local peaks of this potential. The use of such potentials to model forces is an integral part of this modeling framework (Middleton et al., 2014).

#### 3.5. Cell-Cell Interactions

We assume that T and B cells are attracted to one another. For simplicity's sake, we assume that this can be modeled via another Gaussian function. It is worth noting that this could represent attraction acting at a distance, or could represent a local effect acting to keep cells in contact with one another (or some combination of the two). We do not allow cells to pass through one another. We can achieve both of these through the use of another potential. Common choices for these potentials include the Lennard–Jones potential and the Morse potential (Middleton et al., 2014). We shall use the following potential:

$$U\_{i}^{\text{cell-cell}} = \sum\_{j \neq i, j \in T \cup \mathcal{B}} A\_{\text{a}}^{ij} \exp(-|\mathbf{x}\_{j} - \mathbf{x}\_{i}|^{2}/d\_{\text{a}}^{2})$$

$$-A\_{\text{r}} (|\mathbf{x}\_{j} - \mathbf{x}\_{i}| - 2r\_{\text{irmm}})^{6} H (|\mathbf{x}\_{j} - \mathbf{x}\_{i}| - 2r\_{\text{irmm}}). \tag{8}$$

The first of these terms is the attraction between immune cells of different types—B cells seek out T cells to activate and T cells are also attracted to B cells. This is achieved by setting:

$$A^{ij}\_{\mathbf{a}} = \begin{cases} A^0\_{\mathbf{a}} & \chi\_i \neq \chi\_j \\ 0 & \chi\_i = \chi\_j \end{cases},$$

where χ<sup>i</sup> is an indicator function taking value 1 if cell i is a T cell and 0 if it is B cell. The spatial scale of this attraction is set by da.

The second term accounts for the fact that cells cannot pass through one another. If the distance between cells is greater than the sum of their radii, that being 2rimm, they exert no repulsive force on one another, achieved through the inclusion of a Heaviside function H. However, if this distance falls below this sum, a large repulsive force is exerted. This is known as a hard-core potential (Levine et al., 2000). Note that some authors choose this potential to be infinite at the point of contact, so that cells cannot occupy or share any part of physical space (Bruna and Chapman, 2012). However, this necessitates choosing a smaller time step and, since cells can remain in contact with one another for significant periods of time, we will not make such a choice. Note also that the parameter A<sup>r</sup> is shared amongst all immune cells. Once again, the forces acting on the i'th cell can then be expressed as:

$$F\_i^{\text{cell-cell}} = -\nabla U\_i^{\text{cell-cell}}.\tag{9}$$

Note that, since immune cells move around the domain, the potential "landscape," together with its peaks and troughs, is constantly evolving.

#### 3.6. Islet Interactions

For simplicity, and to remain consistent with our chosen geometry of the cells, the basement membrane encapsulating the islet is chosen to be a circle centred at the origin with radius R. Cells cannot pass through the membrane, so when it is present, immune cells remain on the inside or outside of the islet. This can be reflected through the use of another hard-core potential. Whilst we have not yet defined the dynamics for the membrane, we shall assume that there is a integrity threshold, h, below which the membrane does not prevent cells from passing through. Since we do not expect the membrane to have the same integrity across its whole length, we also need to account for the location at which the cell interacts with the islet. The relevant hard-core potential is then given by:

$$U\_i^{\text{islet}} = A\_{\text{islet}} H(m(\theta) - h) H(r\_{\text{imm}} - |d\_i - R|)$$

$$\begin{split} (r\_{\text{imm}} - |d\_i - R|)^6, \\ d\_i = |\varkappa\_i - \varkappa\_{\text{islet}}|, \end{split} \tag{10}$$

where m(θ) ∈ [0, 1] is the membrane integrity at angle θ ∈ S 1 around the membrane, xislet ∈ R 2 is the location of the center of the islet and Aislet is a constant indicating the strength of repulsion. Once more, the force acting on the cells is then given by:

$$F\_i^{\text{islet}} = -\nabla U\_i^{\text{islet}}.\tag{11}$$

Finally, we must define the forces acting upon the immune cells exerted by the beta cells. This is functionally the same as the repulsive force between cells (Equation 8) and is given by:

$$U\_i^{\beta} = A\_{\mathbf{r}} \sum\_{j \in \beta} \upsilon\_j(t) H(|\mathbf{x}\_j - \mathbf{x}\_i| - (r\_{\beta} + r\_{\text{irmm}})) $$

$$(|\mathbf{x}\_j - \mathbf{x}\_i| - (r\_{\beta} + r\_{\text{irmm}}))^6,\tag{12}$$

where r<sup>β</sup> is the radius of the beta cells. The time-dependence of ν<sup>j</sup> here reflects that the beta cells may die, after which we no longer need to consider repulsive effects generated by them (assuming the dead cell bodies are cleared by macrophages), and so ν<sup>j</sup> is set to 0 for that cell. The final force in Equation (2) is defined through

$$F\_i^{\beta} = -\nabla U\_i^{\beta}.\tag{13}$$

#### 3.7. Immune Cell Lifespan

The timescale over which insulitis takes place is long compared to the average lifespan of an immune cell. Thus, we need to incorporate immune cell death into our model. We assume that T and B cells have a lifespan of LT, and L<sup>B</sup> respectively. Each cell then has a counter χ<sup>i</sup> , which is incremented by setting χ<sup>i</sup> 7→ χ<sup>i</sup> + 1 at each time step. Activated T cells are expected to have shorter lifespans than unactivated T cells (Green et al., 2003), so for those cells, we instead update the counter via χ<sup>i</sup> 7→ χ<sup>i</sup> + 1χ where 1χ > 1.

When χ<sup>i</sup> exceeds L<sup>T</sup> (or LB) for a given T (B) cell, it is considered to be dead and is removed. We keep the number of T and B cells constant throughout the simulation by assuming that each dead cell is replaced by a newly arriving one. We assume that the vasculature is sufficiently dense that immune cells can enter at any point in the extra-islet space. As such, the location of the new cell is drawn randomly from a uniform distribution over the extra-islet space, achieved through the use of polar coordinates. The cell's counter is reset to 0 and, if the cell is a T cell, it is chosen to be in the unactivated state.

#### 3.8. Membrane Dynamics

In our simplified geometry, the encapsulating membrane is a represented by a circle, which can be parameterized by a single variable θ ∈ S 1 . The equation governing the evolution of the membrane viability, m, is given by:

$$\begin{split} \frac{d m(\theta)}{dt} &= \alpha\_m (1 - m(\theta)) \\ &- \lambda\_m \sum\_{i \in T} \exp \left( - (|\boldsymbol{\chi}\_i - \boldsymbol{\chi}\_m(\theta)| - r\_{\text{imm}}) / d\_m \right), \end{split} \tag{14}$$

where α<sup>m</sup> and λ<sup>m</sup> are respectively the repair and degradation rates of the membrane and xm(θ) is the location in real space of the membrane at position θ. The term in the sum represents the fact that we expect T cells to break down the membrane and this can only occur when T cells are close enough. The range over which the T cells can degrade the membrane is set by dm.

We note that as beta cells are destroyed, the morphology of the affected islets may change. In particular, some islets in T1D present with small size and irregular outline (Gepts, 1965). The membrane in our study, where it exists, simply follows a circle with fixed radius, and thus does not account for these changes. However, we note that not all degranulated islets have small size and irregular outline (Gepts, 1965). Moreover, the morphology of islets, irrespective of insulitis, is highly variable. It is thus difficult to infer how the morphology of a given islet varies over the course of the insulitic process. Nevertheless, it seems likely that changes to islet morphology will occur, and we discuss approaches to incorporate this into our model in Section 5.1.1.

#### 3.9. Activation and Apoptosis

When T cells and B cells are in contact with one another, we assume that B cells can activate non-activated T cells and that this process has a characteristic time course. We thus describe the activation level of the i'th T cell, a<sup>i</sup> , via:

$$\frac{\mathrm{d}a\_{i}}{\mathrm{d}t} = \begin{cases} \sum\_{j \in \mathcal{B}} H(|\mathbf{x}\_{j} - \mathbf{x}\_{i} - 2r\_{\mathrm{imm}} + 0.1|) & a\_{i} < 1 \\\ -\lambda\_{d} a\_{i} & a\_{i} \in \mathcal{T}, \\\ 0 & a\_{i} \ge 1, \end{cases}, \quad i \in \mathcal{T}, \tag{15}$$

where λ<sup>a</sup> is the decay rate of the activation signal. The constant 0.1 is included inside the Heaviside function to account for the fact that we are not explicitly modeling cell-cell contacts, and the use of hard-core potentials tends to make cells move apart quickly when they are close. Our specific choice for this constant is selected such that cells are allowed to remain in contact with one another. We note that if this value is selected to be too large, cells can essentially occupy the same location, whilst values that are too small will cause cells to "bounce" off one another. We wish to avoid both of these behaviors, and over a range of choices, we found that a value of 0.1 satisfied these criteria. Once the activation signal reaches 1, the cell is activated and remains so until that cell dies, (or until the end of the simulation if that occurs first), reflected by setting the RHS to zero upon the activation reaching 1. Activated cells have an increased sensitivity to the chemokine signal and enhanced killing rate. This is captured in the model by increasing the sensitivity and killing parameters:

$$\mathcal{S}\_{\bar{i}} \mapsto \mathcal{S}\_{\bar{i}} + \Delta \mathcal{S}, \quad \kappa\_{\bar{i}} \mapsto \kappa\_{\bar{i}} + \Delta \kappa, \quad \bar{i}$$

where 1S and 1κ are positive constants.

The dynamics for the apoptosis of beta cells b<sup>i</sup> follows a similar prescription:

$$\frac{\mathbf{d}b\_i}{\mathbf{d}t} = \begin{cases} \sum\_{j \in T} \kappa\_j H(|\mathbf{x}\_j - \mathbf{x}\_i - 2r\_{\text{irmm}} + 0.1|) & b\_i < 1 \\ & -\lambda\_b b\_i \\ 0 & b\_i \ge 1, \end{cases}, i \in \beta, 1$$

where λ<sup>b</sup> is the decay rate of the apoptotic signal, κ<sup>j</sup> is the killing rate of the j'th T cell, and we include the constant, 0.1, for the same reasons as for (Equation 15). If b<sup>i</sup> exceeds 1 for a given cell i, that cell is assumed to have been killed and is removed from the simulation. This is achieved by setting its viability, ν<sup>i</sup> , to be zero.

Since the system we are solving is a Langevin equation, we must use an appropriate numerical method to deal with the stochasticity. For computational efficiency, we shall use the forward Euler–Maruyama scheme. This method only has strong order dt1/<sup>2</sup> but requires the fewest function evaluations of any of the solvers for stochastic differential equations (SDEs). The one exception to this is the equation governing the membrane dynamics where the explicit Euler scheme is unstable and causes solutions to blow up. For this equation, we use an implicit, backward Euler scheme. This keeps the total order accuracy of solutions consistent with the other equations, but provides the required stability without incurring additional computational cost.

Note that the membrane (Equation 14) treats distinct points along the membrane as being distinct from one another. This is the only system that is grid-based – all of the other equations are grid-free. In order to solve this system, we first discretize the membrane into N<sup>θ</sup> points, θ<sup>i</sup> = −π + 2π(i − 1)/N<sup>θ</sup> , i = 1, N, and subsequently solve (Equation 14) at each of these points. To find the value of the m at a point not on the discrete grid, we use band-limited interpolation for periodic signals as described in Schanze (1995).

#### 3.11. Initial Conditions and Parameters

The immune cells are initially located at random, nonoverlapping positions within annular domain with inner radius R and outer radius 1mm. The islet is positioned at the center of the domain, which for simplicity, we set as our origin. To distribute the beta cells, Nβ cells are arranged in concentric circles within the islet. Next, for T = 1000 timesteps, they are allowed to freely evolve governed by:

$$\frac{d\mathbf{x}\_i}{dt} = F\_i^{\text{islet}} + F\_i^{\beta - \beta} + \sigma \xi\_i(t), \quad i \in \beta,\tag{16}$$

where x<sup>i</sup> is the center of the i'th beta cell, F islet i is the same as Equation (13) operating on the beta cells rather than the immune cells with:

$$F\_i^{\beta-\beta} = -\nabla U\_i^{\beta-\beta},\tag{17}$$

where

$$U\_i^{\beta-\beta} = \sum\_{j \neq i, j \in \beta} H(|\mathbf{x}\_j - \mathbf{x}\_i| - 2r\_\beta)(|\mathbf{x}\_j - \mathbf{x}\_i| - 2r\_\beta)^6,\tag{18}$$

where x<sup>i</sup> ∈ R <sup>2</sup> now represents the location of the beta cell. After the n steps, the beta cells are fixed in location for the remainder of the simulation. Note that during this process, the immune cells are fixed in position. We distribute cells in this manner as it (a) generates a more realistic islet geometry compared with spacing them evenly within the islet and (b) it overcomes the known computational problems with randomly distributing nonoverlapping disks in a confined region (Song et al., 2008). In **Figure 1**, we show the initial and final configuration of the beta cells following this approach. All simulations and analyses were performed in Matlab.

Other initial conditions are given by m = 1 for all θ, a<sup>i</sup> = 0 for all i ∈ T , and b<sup>i</sup> = 0, ν<sup>i</sup> = 1, ∀i ∈ β.

The sensitivities of the T and B cells to the chemokine gradient are drawn from distinct Gaussian distribution with means S<sup>T</sup> and S<sup>B</sup> and variances σ<sup>T</sup> and σ<sup>B</sup> respectively. Parameters governing the geometry of our domain, such as immune cell and beta cell sizes were chosen to be matched to available data. In particular, beta cell diameters were taken from Saisho et al. (2013), immune cell diameters were based on values found in Wang et al. (2012), numbers of cells in each immune cell population were taken from Willcox et al. (2009) and islet composition (in terms of beta cells) were based on those found for humans in Kim et al. (2009). Data regarding the in vivo lifespan of specific effector lymphocytes in humans are rare and these may also be heterogeneous even within the same population. However, effector lymphocytes are unlikely to have lifespans ranging beyond several weeks (Sprent, 1993). To address the uncertainty in lifespans, we performed simulations across a range of values, from 3 to 7 weeks and compared results.

Other parameters in our model were tuned to provide the required behaviour, though we comment that much of the general behavior observed in our simulations is robust to parameter variations, suggesting that the model outcomes are robust. We note that certain processes, such as the degradation and repair of the peri-islet basement membrane, are phenomenological in nature, and as such, it is difficult to ascribe to them meaningful parameter values. Whilst the islet membrane is a physical component of the real system, we represents its viability by a scalar in the range 0–1, and so parameter values should be interpreted with respect to this scaling. Similar comments hold for the chemokine signal. Though a chemoattractant gradient is thought to exist in the peri-islet space, it is not clear what its composition, and subsequent properties, might be. As such, we have chosen a prototypical form to represent our chemokine signal, and investigated how changes to the strength of this signal impacts upon the resulting dynamics. All parameter values are summarized in **Table 1** and referenced in text where they are altered for specific numerical experiments.

#### 4. RESULTS

In **Figure 2**, we show the initial state of a prototypical simulation, and the state of the system at T = 4000 days. Here, we



*Where indicated, references show the studies upon which specific parameter values were based. Parameters are altered as discussed in the text for specific numerical experiments.*

can see that the islet exhibits significant beta cell loss. We also see that immune cells congregate at the islet membrane and that a small number infiltrate the islet itself. Across all parameter sets used in our experiments, this behavior is preserved. In the Supplementary Material, we provide videos showing the full time-course of typical simulations with base parameters as indicated in **Table 1**, lymphocyte lifespan of 21 days and other parameters selected to reflect a variety of conditions, as discussed in the forthcoming sections. These simulations may be compared with the averaged results shown in **Figures 3**–**7**.

represents a chemokine signal. Parameters are as in Table 1.

For all parameter sets, we simulate 100 realizations and average at specific time points to develop a time-course. During the simulations, we keep track of the number of immune cells within the islet and the number of viable beta cells. We take the total area encompassing the viable beta cells as being reflective of the beta cell mass. We shall now report both the time-courses of the remaining beta cell mass as a percentage of the initial mass (calculated as the ratio of the total mass of viable beta cells at time t and the initial mass of beta cells, averaged over the 100 realizations), and the number of immune cells in the islet for a number of conditions, along with the standard error of the mean (SEM) at each time point. In each case, we shall use the base set of parameters as indicated in **Table 1**, changing specific values where indicated. In all cases, to assess how the lifespan of the immune cells affects the resulting dynamics, we vary this between 21, 28, and 56 days as indicated in the figures.

### 4.1. Varying Number of B Cells

We begin by examining a key condition in this study, namely how the number of B cells in the peri-islet space impacts the rate of destruction of beta cells. To address this, we compare a condition with few B cells, setting N<sup>B</sup> = 5, and one with a higher number of B cells, with N<sup>B</sup> = 30. These results are summarized in **Figure 3**.

We observe that when the immune cell lifespan is short, neither the low nor the high B cell condition exhibit significant beta cell destruction. As the lifespan is increased, we observe a pronounced increase in this rate. For an immune cell lifespan of 56 days, we see that the high B cell condition reaches an equilibrium value at around 3000 days. Note that the equilibrium value is below the critical threshold of ∼20% of remaining beta cell mass (Cnop et al., 2005) (however, it should be noted that this threshold is a whole-body threshold and may not be valid for an individual islet).

# 4.2. Varying Repair Rate of Membrane

We have assumed in our model that the peri-islet basement membrane is capable of being repaired at a rate αm. If we decrease αm, the rate of degradation of the membrane may be such that

holes caused by infiltrating T cells are not repaired (at least over the timescale of our simulation). In the extreme case, the membrane is incapable of repair, in which case we set α<sup>m</sup> = 0. In **Figure 4**, we display the time-course of a simulation in such a regime.

It is clear from these graphs that removing the repair mechanism of the membrane dramatically increase the susceptibility of the beta cells to cytotoxicity. In the most extreme case, in the situation with few B cells and a short immune cell lifespan, we now see a huge difference in both the rate of beta cell destruction and the final cell mass at the end of our simulation. In the case with no membrane repair, the final beta cell mass of almost all simulations is below ∼20% of the initial mass. These results highlight the importance of the peri-islet basement membrane to the progression of insulitis, and also suggest that ongoing repair of this membrane is critical to slowing the infiltration of the immune cells.

We also observe, in contrast to the results in **Figure 3**, that the immune cell lifespan now has a more muted affect on the resulting dynamics. Whilst there is an increase in the rate of beta cell destruction with increasing lifespan, the more significant contribution appears to be to lower the equilibrium value of remaining beta cell mass. This likely arises due to the fact that, as the beta cell population decreases, so too does the chemokine signal to which the immune cells respond. This means that the cells (on average) move into the islet at a slower rate, since their paths becomes more dominated by noise. If the cell lifespan is too short, the immune cells may die before entering the islet if there are insufficient beta cells generating a chemokine gradient.

### 4.3. Increasing the Membrane Degradation Rate

Instead of reducing the basement membrane repair rate, we now increase the efficacy of the immune cells of degrading the membrane, setting λ<sup>m</sup> = 0.2 and display the results in **Figure 5**. For comparison with the condition in which the encapsulating membrane is not repaired, we additionally plot results from this case in the same figure, as indicated.

As expected, increasing the rate of membrane degradation increases the rate of beta cell destruction and subsequently decreases the final remaining beta cell mass. Compared with the condition with no membrane repair, we see that the initial rate of beta cell destruction is greater in the case with high membrane degradation. However, as the insulitis process continues, this trend becomes reversed. Remarkably, across all cases varying the number of B cell and the immune cell lifespan, the time at which this occurs appears to be approximately the same. We also see that the final beta cell mass is lower for the condition with no repair than for that with high membrane degradation.

At the beginning of simulations, the ability of the T cells to degrade the basement membrane at a faster rate allows them to infiltrate the islet and subsequently kill the beta cells more quickly. Whilst the initial degradation of the membrane in the no repair condition is slower, the fact that "holes" in the

in Videos S3, S4.

encapsulating membrane caused by T cells are not repaired means that, at later times, immune cells can simply pass through these holes.

When the membrane can be repaired, the rate of immune cell infiltration is governed by the balance between how quickly it can be degraded and how quickly it can be repaired. As the chemokine signal across the peri-islet space becomes weaker when beta cells die, the average rate that T cells reach the basement membrane falls, since the strength of the signal is a contributory factor in determinining the transit time for a T cell toward the islet. As there are fewer T cells aggregrating at the islet membrane at any given time, the degradation of the membrane becomes slower, and hence the beta cells are destroyed at a slower rate as time progresses. At later times the arrival rate of T cells may be so slow that the membrane is repaired at the same rate as it is degraded, on average, and so islet infiltration by the immune cells will be halted.

These observations account for the slower rate of beta cell destruction and higher remaining beta cell mass for the high degradation condition compared to that with no membrane repair. This further highlights the importance of continual basement membrane repair to the slowing of insulitis.

#### 4.4. Increasing the Killing Efficiency of the T Cells

We now increase the killing rate of the T cells. To reflect the fact that we assume that activated T cells are responsible for most of the beta cell destruction, we achieve this by doubling the killing rate boost, setting 1κ = 1.8. The results for these experiments are displayed in **Figure 6**.

We observe that increasing the killing rate of the immune cells does not significantly affect the death rate of beta cells, either quantitatively or qualitatively. In all cases, we observe an increase

are observed across all immune cell lifespans and number of B cells. Note, however, that whilst the initial rate of beta cell destruction for the condition with high membrane degradation is higher than that for the case with no membrane repair, the final beta cell mass is higher for the former compared with the latter. We refer the reader to the text in Section 4.3 for further interpretation of this observation. Parameters are as in Table 1 with λ*m* = 0.2 for the high degradation condition and α*m* = 0 for the no membrane repair condition. Videos of typical simulations for the case with immune cell lifespans of 21 days may be found in Videos S5, S6.

in the beta cell mass destruction rate, but this is fairly minimal. The limited impact of increased killing rate can be understood by noting that, whilst the killing rate is an important factor in the overall insulitis process, this is dominated by transit time to and within the islet, as well as activation by B cells. Once inside the islet, T cells can interact with and kill beta cells but the total number of beta cells an individual T cell can destroy will be influenced more strongly by its remaining lifespan than on its killing rate. This highlights the need to incorporate the spatial aspects of the immune response and that high T cell killing rates do not necessarily significantly affect the progression of insulitis.

We comment that these observations at first seem at odds with previous results from the NOD mouse where the avidity of a T cell population is well-correlated with its pathogenic potential, and the progression of T1D occurs through avidity maturation (Amrani et al., 2000). Theoretical studies have supported the viewpoint that higher avidity CD8+ T cell populations lead to increased rates of beta cell destruction (Khadra et al., 2011) and that low avidity populations may have a protective effect against T1D development (Khadra et al., 2009). However, recent results in the NOD model have suggested that the highest avidity CD8+ cells do not escape thymocyte negative selection, and accordingly, the cells responsible for beta cell destruction in the murine pancreas may be regarded as those with "intermediate avidity" (Han et al., 2016). Furthermore, the paucity of data on the specificity of lymphocytes involved in human insulitis makes it difficult to propose quantitative statements about the effective killing potential of individual T cells. This is particularly difficult since the majority of the results regarding the avidity of

human lymphocytes come from peripheral blood measurements and are not necessarily indicative of avidities within the pancreas (Reijonen et al., 2004; Standifer et al., 2009). Finally, the lifespan of a T cell that has successfully infiltrated an islet should still have a significant impact on the number of beta cells it can destroy, assuming its avidity is sufficiently high.

# 4.5. Increasing the Rate of Chemokine Production

To reflect increases in the rate of chemokine production, we now double α 1 C and α 2 C simultaneously, setting α 1 <sup>C</sup> = 1.0 and α 2 <sup>C</sup> = 20.0. Note that, since we are assuming a quasi-steady state approximation for the chemokine, this simply adjusts the profile of the chemokine signal across our domain. The results for these simulations are depicted in **Figure 7**.

As for the situation with increased killing rates, we observe similar qualitative behavior to the case with lower chemokine production rates. However, in contrast to the former case, the quantitative differences between the beta cell death rate and final cell mass are much greater. This further highlights the contribution of cell movement in the overall inflammatory response.

# 4.6. Combining High Chemokine Signals and Reduced Membrane Repair

In our final numerical experiment, whose results are shown in **Figure 8**, we simultaneously increase the strength of the chemokine signal, setting α 1 <sup>C</sup> = 1.0, α 2 <sup>C</sup> = 20.0, as in the previous subsection, and remove the membrane repair, setting α<sup>m</sup> = 0. As expected, with these combined alterations, we now observe significantly high rates of beta cell destruction for all cases considered. Moreover, the final beta cell mass at the end of our simulations is below 20% of the original cell mass. For the cases with the longest lived immune cells, those having a lifespan of 56 days, we now see that essentially all of the beta cells have been destroyed within 1000 days.

# 4.7. Immune Cell Invasion Profiles

We now consider how the invasion of immune cells is dependent on the remaining beta cell mass and report the dependence of the number of immune cells within the islet (i.e., within the basement membrane) on the remaining beta cell mass. These results are shown in **Figure 9** for the case with 5 B cells and in **Figure 10** for the case with 30 B cells. The results for the case with high chemokine signals with no basement membrane repair, and for the case with increased membrane degradation are omitted, but are qualitatively similar to the case with no repair alone. In both figures, we only show results when the immune cell lifespan is 56 days.

For situations with 5 B cells, we observe two qualitative behaviors amongst the range of cases considered. Using the default parameters and those with high killing rates, we observe no infiltration of the islet by the B cells. The immune cell population undergoes an initial increase to a maximum value, followed by an approximately linear decay with the remaining beta cell mass. The behavior with high chemokine signals or no membrane repair does show islet invasion by the B cells. In these cases, the T cell population now both increases and decreases more gradually as a function of the beta cell mass, and attains a higher maximum value than in either of the two previous cases.

If we now increase the number of B cells to 30, we observe, in all cases, invasion of the islets by B cells and a qualitatively similar profile of T cell infiltration. With the default parameters, or those with high killing rates, the maximum value of invading T cells occurs when approximately 50% of the beta cell mass remains. In contrast to the T cell population, the number of invading B cells varies little with the beta cell mass. In the case with no membrane repair, this is no longer true, and we now find that the B cell profile follows the T cell profile and furthermore, that the

maximum value of immune cell infiltration occurs at low values of the remaining beta cell mass.

# 4.8. Trial-to-Trial Variability and the Role of Space

For each of the scenarios listed, we report the standard error of the mean for all recorded values at each time point. In general, we observe that variability across our simulations for a given scenario is low, suggesting that the model behavior is robust and that noise does not significantly affect the intrinsic model dynamics. In particular, scenarios with either very high rates or very low rates of beta cell destruction exhibit very little variability. Variation is highest in scenarios that lead to intermediate rates of beta cell death, suggesting that the role of noise in such situations is much more significant in these conditions.

The results reported in **Figures 3**–**10** reflect spatial averages that are taken over the peri-islet space. As such, they do not reflect the spatial component of the dynamics produced by the agent-based model. These effects are, in general, quite subtle. In particular, since we assume that activation of CD8+ cells by CD20+ cells requires them to be co-localized, the spatial arrangement of the immune cells will play an important role in determining the overall rate of beta cell destruction. The geometry in this initial version of the model has been chosen to be simplistic in nature. In future versions, the geometry will be adjusted to better match that of real islets as discussed in Section 5.1.1. One could imagine scenarios in which the routes taken by immune cells when migrating toward the islet could influence the average number of interactions with one another, and further, that these paths would be influenced by prominent vasculature near the islet.

The spatial configuration of the immune cells also plays a significant role in determining how quickly the basement membrane is degraded. Where many CD8+ cells accumulate at specific points along the periphery of the islet, we would expect high rates of membrane degradation. Conversely, if the cells

FIGURE 9 | Profiles of immune cell invasion dependent on the remaining beta cell mass in cases with 5 B cells. Each graph shows the mean ± SEM for 100 simulations. We report the number of T and B cells within the islet in a number of conditions. The top-left panel shows results using the parameters in Table 1, the bottom-left uses these values with 1κ = 1.8, in the top-right, we set α 1 *C* = 1.0 and α 2 *C* = 20.0, and the bottom-right has α*m* = 0.

100 simulations. We report the number of T and B cells in the same conditions as displayed in Figure 9.

are evenly distributed along the membrane, average degradation rates are likely to be low. Our results suggest that the dynamical processes governing the repair and destruction of the basement membrane play a key role in determining the overall rate of beta cell destruction and so it is clear that the spatial distribution of immune cells along this membrane will also be critical.

These spatial considerations are the key determinant for the degree of variability observed between trials, as reported in each of the figures. In the Supplementary Material, we provide animations of typical simulations in a subset of the scenarios listed to demonstrate the subtle role that these effects have on the insulitis process.

# 5. DISCUSSION

In this manuscript, we have constructed a spatial, agent-based model of immune cell invasion of a prototypical islet of Langerhans. The agents in the model are immune and beta cells, and each obeys rules that broadly match the general behavior of these cell types. The model attempts to mimic a real islet, allowing for in silico experiments to be performed through model simulation. The agent-based framework allows both the spatial aspects of insulitis to be investigated, and avoids issues of applying ODE-based approaches to a system that has low cell numbers. It is intended that this will lay down the framework for more extensive spatio-temporal modeling of insulitis, with an ultimate aim to understand its core mechanisms and to devise strategies to slow or halt its progress.

Overall, the observed phenomena in the model are well matched to real data, which have been collected and analyzed from human patient samples (Willcox et al., 2009; Morgan et al., 2014), and are reported in **Figure 11**. We note that data for the pauci-immune phenotype have recently appeared in Leete et al. (2016), but since our model is primarily focussed on the hyper-immune phenotype, we highlight these results here. Results from the agent-based model have the same qualitative behavior as displayed by these data. This in turn provides evidence that the mechanisms included within our model are sufficient to capture qualitatively the key features of the inflammatory response. In particular, we note that the case with no membrane repair appears to provide the best qualitative match to the existing data, in which we see a rise in the number of infiltrating immune cells during disease progression, with a peak at around 10–20% of remaining beta cell mass. We note that scenarios with low overall rates of beta cell death are associated with low levels of B cell infiltration. This further supports the viewpoint that the CD20+ cells play an important role in determining the rate of beta cell destruction. By adding and removing mechanisms in a systematic way, we can probe what effect their inclusion or exclusion will have on the resulting dynamics and subsequent prognosis. It is important to note that, since we investigate the disease at the level of the islets, we can make predictions about how modifying certain cell properties will affect the inflammation of the islets, rather than considering whole body responses.

Accordingly, in the present study, we have performed several parameter studies to examine how the relative contributions of core processes affects insulitis. We have demonstrated that, under the assumptions used in constructing the model, an increased number of B cells in the peri-islet space gives rise to a faster destruction of the beta cells. This effect can be amplified by factoring in manipulations of other processes, such as the killing rate of the beta cells by T lymphocytes and the repair rate of the islet's basement membrane. Simply increasing the killing rate of the T cells did not have a significant impact of the rate of the beta cell destruction. However, increasing the strength of the chemokine signal to which the immune cells respond did show a pronounced increase in the death rate. These facts highlight the importance of the chemotactic process by which the immune cells enter the islet, find and kill the beta cells.

In scenarios in which the islet basement membrane is unable to repair itself, or is degraded more rapidly, we also see significant increases in the beta cell destruction. Moreover, we also find that the final beta cell mass at the end of our simulations is substantially depleted. This highlights the importance of this membrane as a barrier to the infiltrating immune cells. In particular, when chemokine signals are sufficiently high, if the membrane is incapable of being repaired, all of the beta cells may ultimately be killed, whereas in other situations, we find residual beta cells even after long simulation times (not shown). The importance of the basement membrane during insulitis in our model is in line with a recent study of this membrane in samples from human patients with T1D (Bogdani, 2016).

Upon examining the dependence of the immune cell invasion on the remaining beta cell mass, we find in most cases the profile of B cell infiltration is relatively flat. In cases with low B cell numbers, low chemokine signals and with a membrane that is repaired, there is essentially no B cell invasion. In situations with higher numbers of B cells, the beta cell mass at which the T cell invasion is maximal is shifted to lower values. In the case with no membrane repair, this maximal point is shifted to very low values of the remaining cell mass. Moreover, in this instance, we also see that the B cell invasion profile follows that of the T cells. This condition is most consistent with the human data reported in Willcox et al. (2009), suggesting that, for our model, low membrane repair rates are an important factor in the insulitis process.

The rise and subsequent decrease in T cell populations observed in our simulations is in accordance with experimental results of T1D specific autoimmune responses in the NOD model (Trudeau et al., 2003). These dynamics have been described as "waves" and have been explained mathematically by transitions through Hopf and homoclinic bifurcations in Mahaffy and Edelstein-Keshet (2007) and by transient bistability of the autoimmune state in Jaberi-Douraki et al. (2014b). Interestingly, in the latter of these two theoretical studies, inclusion of multiple clones of T cells with different avidities can give rise to multiple waves in the T cell dynamics, which can significantly impact the rate of overall beta cell destruction. This highlights potential avenues for intervention through the promotion of lower avidity T cell populations.

It should be noted that data on the specificities of T cells within human insulitic islets are rare, but that cells with varying antigen specificities may be present (Coppieters et al., 2012; Babon et al., 2016). Thus, a potential development of our model would be to include multiple clonal populations to explore these dynamics in the spatial context, particularly given the potential insights that may be applicable from the theoretical results in Mahaffy and Edelstein-Keshet (2007) and Jaberi-Douraki et al. (2014b).

Agent-based and lattice-based approaches are becoming increasingly popular in biological modeling. ODE based approaches, which have long been a linchpin of the mathematical biology community become ill fit for purpose when wishing to describe processes in which the number of agents is small. Considering stochastic variants of ODEs and partial differential equations (PDEs) models is one potential approach to account for the variability that arises from low cell numbers, but this too is only a good approximation when the number of cells is sufficiently high. Moreover, these approaches do not typically account for trial-to-trial variability that may be important for describing individual results. Note that, at various levels of description, systems of differential equations can still form part of agent-based models.

One of the attractive properties of ODE and PDE modeling when compared with agent-based simulations is the computational cost associated with the latter, whereas density (continuum) based methods typically are low dimensional in nature. Indeed, a significant barrier to the widespread use of agent- and lattice-based models in the past has been the requirement of significant computational power. In recent years, the technological advances in computing have started to erode this barrier. In addition, domain decomposition methods (Chen et al., 2007; Tapia and D'Souza, 2011) and the ability to parallelise computations on large scales on commercially available graphics processing units (Chen et al., 2007; Tapia and D'Souza, 2011; Harvey et al., 2015) has made agent-based simulations achievable universally.

Here, we have focussed on a specific type of agent-based model. This is based on a discrete particle formulation, where each cell is parameterized by a finite area and location. Broadly, agent-based models can be divided into lattice-based and latticefree methods (Plank and Simpson, 2012). The discrete particle approach is lattice-free, meaning that cells are free to move anywhere in the domain. One limitation of this approach is that the cells are restricted to be described as circles; that is, their shape cannot be deformed. Immune cell deformation may play an important role when considering how cells pass through degraded parts of the basement membrane. In the current model formulation, cells must "wait" until the gaps in the membrane are sufficiently large before they can pass through.

An alternative to the discrete particle formulation is the lattice-based cellular Potts model. This modeling framework has been growing in popularity and has been applied to cell migration in a number of contexts, including vasculogenesis (Merks et al., 2006; Daub and Merks, 2013), morphogenesis (Chen et al., 2007; Marée et al., 2007) and wound healing (Scianna, 2015). Although the cellular Potts model is lattice-based, the cells within the framework can occupy many lattice sites and thus their shape, as well as their location, evolves over time. On a fine enough lattice, such models can look remarkably similar to the system they are approximating. However, with the increase in spatial resolution of the lattice comes an associated computational cost. It should be noted that there are several actively maintained packages for agent-based modeling, such as CHASTE (Mirams et al., 2013) and CompuCell3D (Swat et al., 2012).

One approach that combines the flexibility of lattice-free approaches together whilst allowing for cells to change shape is to treat points on the cell membrane as particles and resolve forces acting upon them, under the assumption that the cell remains bounded by its membrane (Elliott et al., 2012). However, the mathematical machinery and computing power required to embed such a description into our domain are prohibitive for our purposes. A "halfway" house could, in which the cells are treating as deformable ellipsoids, is another possible way of relaxing our assumption that cells are perfectly circular (Palsson and Othmer, 2000).

# 5.1. Future Directions

The modeling framework described within this manuscript is not intended to be a fully comprehensive endpoint for the study of the spatio-temporal dynamics associated with insulitis. Instead, it is intended to be the first step to building a general virtual environment in which to simulate the invasion of islets during T1D. We will now highlight some potential areas for model development.

#### 5.1.1. Geometry

We have treated perhaps the simplest possible geometry that is reflective of the islet. The model is planar in nature, whereas real islets and immune cells are three dimensional structures. We have ignored the mechanical effects on the basement membrane: as beta cells are removed from the islet, the morphology of the islet itself may change (Brereton et al., 2014). There is currently no structure imposed upon the extra-islet space. Accurate modeling, for example, in which the vasculature is taken into account may constrain locations at which immune cells are allowed to enter the domain. Through the use of imaging techniques, it should be possible to build a more faithful representation of both the intraand extra-islet space and include these in the model.

#### 5.1.2. Different Cell Types

The model could be developed to include more cell types, such as the macrophages that clear apoptotic beta cells, or different endocrine and exocrine cells which are not targeted by immune cells, but may slow their movement. There may be other cell-cell interactions that may be important during insulitis, for example the role of CD4+ cells in the activation of CD8+ cells (Castiglioni et al., 2005). Even restricting the model to the two immune cell types described in this manuscript, we could consider different clonal populations of the T cells (Khadra et al., 2011). One hypothesized strategy to slow the progression of insulitis is to promote the replication of a low avidity clone of T cells, that will out compete the higher avidity clones for space and resources, leaving only relatively passive T cells (Amrani et al., 2000; Khadra et al., 2009; Bluestone et al., 2015). Such a mechanism could be tested in our modeling framework.

#### 5.1.3. Cell Proliferation

Currently, none of the cell types included in the model undergo cell division. In general, post-natal beta cell replication in humans is thought to occur at low rates, and we have disregarded it here. Similarly, immune cell replication in and around the islet is also assumed to be negligible. However, there is some evidence to suggest both that beta cell proliferation is enhanced during insulitis, at least during its early phases (Willcox et al., 2010; Dirice et al., 2014) and that immune cell replication also takes place within the islet itself (Graham et al., 2012) in the NOD model, though these findings have not been corroborated in human tissue. In particular, data collected from ex vivo human samples suggested that very few lymphocytes in the peri-islet space were positive for proliferative markers (Willcox et al., 2010). In order for the hypothesis of promotion of low avidity clones to be tested, the model would have to include cell division, since the action of long-lived memory cells is an integral part of that theory, however this may require coupling of the agent-based model with one representing the lymphatic system.

#### 5.1.4. Multiple Islets

One of the striking features of insulitis is its heterogeneity, even within an individual patient. Islets near to one another can display different inflammatory profiles—insulitis in one islet thus does not imply insulitis in nearby islets. By extending our domain to include multiple islets, we can use the modeling framework to investigate under what scenarios this can occur. In particular, we can hope to identify what cellular interactions, on a broad scale, must occur for such heterogeneity in disease progression to be observed. One simple way to incorporate the influence of other islets in the current framework would be to include additional point sources of chemokine outside of the working domain. This would have subtle effects on the dynamics of cell movement, particularly when the number of beta cells becomes low.

#### 5.1.5. Membrane Degradation

In the current model prescription, the membrane integrity is assumed to be compromised in a simple way by the action of nearby immune cells. This process is solely dependent on the proximity of the immune cells to the membrane. A more sophisticated approach might be to model the cells secreting proteases which diffuse freely and erode the membrane when they meet it. Whilst this is perhaps better fitted to the true processes that degrade the membrane, it is still not clear which cells are responsible for mediating this process. One potential route would thus be to consider the effects of having different immune cells responsible for this process within the modeling framework.

#### 5.1.6. Waxing and Waning

One of the phenomena not accounted for in our description is the honeymoon period (Akirav et al., 2008). This is a phase lasting, in some cases, up to and beyond a year, typically immediately following diagnosis, in which the requirement for exogenous insulin is diminished. Though understanding of the mechanisms giving rise to this transient asymptomatic period is poor, one notable theoretical study demonstrated that this can occur naturally (in rare cases) when taking into account the role of endoplasmic reticulum stress and subsequent beta cell apoptosis (Jaberi-Douraki et al., 2014c). This study highlighted the possibility that elevations in the maximal unfolded protein response due to metabolic therapies could benefit patients by helping to meet metabolic demand, but despite this, they could not ultimately maintain a sufficient population of healthy beta cells to regulate blood glucose levels.

It is not clear whether the honeymoon period is reflective of events occurring at the level of individual islets, whether it is only present at the systemic level and whether it arises as a result of some innate periodicity present in either the immune system or in the beta cells. Modeling studies involving multiple islets may help to address some of these questions.

#### 5.1.7. Therapeutic Intervention

Once a realistic geometry, motivated by real data, has been constructed, and the model sufficiently developed, an ultimate goal would be to test therapeutic interventions that target the pancreas specifically, such as the transplantation/implantation of encapsulated islets or populations of beta cells (Rickels et al., 2005; Robertson, 2010). We have discussed one hypothesis involving the promotion of low avidity T cell clones (Amrani et al., 2000; Khadra et al., 2009). One potential method for doing this is through the intravenous injection of ex vivo-selected and ex vivo-expanded autologous regulatory T cells, such as CD4+ cells (Bluestone et al., 2015), another is to incite the same expansion of regulatory T cells in vivo through the intravenous injection of nanoparticles coated with pMHC (Sugarman et al., 2013). Our modeling framework could easily be extended to include the effect of infusion of either the cells themselves or the nanoparticles to test the spatial aspects of these interventions.

#### 5.1.8. Cell-Cell Coupling

It is evident from the multiple sources of data that communication between immune cells is an important aspect of insulitis, though the exact processes by which these cells exchange information is unknown. One potential avenue of investigation is to assess specific cell interactions based on known immunological mechanisms (Huppa and Davis, 2003; Friedl et al., 2005). These can be incorporated into an agentbased model to see what the overall effects of these specific types of interaction are, with a view to identifying which signaling pathways are most likely to result in the behavior observed in patient data (McLennan et al., 2012).

#### 5.1.9. Comparison with Animal Models

Animal models, and in particular, the NOD mouse model have been extensively used to unravel potential causes and treatment options for human T1D. That true parallels between the disease in rodents and man may be fewer than is ideal, and the fact that treatments that work in mice are not as efficacious in humans (In't Veld, 2014) has led to questions about the suitability of animal models for human disease. From a modeling perspective, this opens up interesting questions. In particular, is it that the underlying mechanisms of insulitis truly are different between different species, or are some processes universal, albeit with potential different time courses and parameters? The former suggests that in order to find a cure for human T1D, only focus on the human condition is likely to yield fruitful results. However, the latter option suggests that, if relations between mice and men for those preserved pathways can be found, that the models can still provide vital insights. An in silico modeling framework seems like a natural place to address such a question as it allows quantitative probing of such relationships.

#### 5.1.10. Experimental Design

One of the critical factors hindering progress in understanding T1D is the lack of human pancreatic samples from which to extract data. Moreover, these provide data only at one specific timepoint, so that very little is known about the time course of insulitis in humans.

One role for mathematical modeling is to identify avenues to explore to aid our understanding of disease mechanisms. The present study has suggested that the balance between repair and degradation of the peri-islet basement membrane surrounding the islets is a critical factor which determines the overall rate of beta cell destruction. Presently, little is known about the composition of this membrane in humans, though recent studies have identified its components in both healthy and infiltrated islets in the NOD mouse (Korpos et al., 2013). We thus feel that gaining an improved understanding of the structure of the human peri-islet basement membrane will significantly our improve understanding of insulitis.

Results from the agent-based model also suggest that the chemoattractant produced by the beta cells, and the lymphocytes' response to it is also important for governing disease progression. As such, in vitro experiments to assess and quantify the rate of immune cell migration to candidate chemokines would be useful to explore how immune cells initially reach the islet.

Finally, little is understood about the interaction between CD20+ and CD8+ cells during insulitis. The co-localization of these immune cell types in pancreatic samples suggest an interaction between the two (Huppa and Davis, 2003), and as such, we speculate that this could give rise to the activation of T cells. However, experiments have yet to confirm this for the immune cells involved in human T1D. Moreover, the specific dynamics of this process need to be better understood before the true role of the CD20+ cells in human insulitis can be revealed.

Overall, we believe that the proposed spatio-temporal framework has great potential as a tool to investigate insulitis in humans, and by focussing attention on the pancreas, it maximizes the use of the currently available human tissue data. We have identified a number of potential avenues in which the framework could be developed and a number of questions that could be addressed in doing so and we hope to continue in this line of research.

# AUTHOR CONTRIBUTIONS

All authors contributed to the initial design and subsequent development of the research project and model. KW developed code to run simulations and analyzed results from them. All authors drafted, edited and approved the submitted manuscript.

# FUNDING

This work was generously supported by the Wellcome Trust Institutional Strategic Support Award (WT105618MA). KT gratefully acknowledges the financial support of the EPSRC via grant EP/N014391/1. We are also pleased to acknowledge financial support from the European Unions Seventh Framework Programme PEVNET [FP7/2007-2013] under grant agreement number 261441 to NM. The participants of the PEVNET consortium are described at http://www.uta.fi/med/pevnet/ publications.html. Additional support was from a JDRF Career Development Award (5-CDA-2014-221-A-N) to SR and project grant 15/0005156 from Diabetes UK (to NM and SR).

# SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: http://journal.frontiersin.org/article/10.3389/fphys. 2016.00633/full#supplementary-material

Video S1 | Typical example of agent-based model simulation using parameters in Table 1, with immune cell lifespans of 28 days and B = 5.

Video S2 | Typical example of agent-based model simulation using parameters in Table 1, with immune cell lifespans of 28 days and B = 30.

Video S3 | Typical example of agent-based model simulation with no peri-islet membrane repair, as indicated in Section 4.2 with immune cell lifespans of 28 days, and B = 5.

Video S4 | Typical example of agent-based model simulation with no peri-islet membrane repair, as indicated in Section 4.2 with immune cell lifespans of 28 days, and B = 30.

Video S5 | Typical example of agent-based model simulation with high rates of peri-islet membrane degradation, as indicated in Section 4.3 with immune cell lifespans of 28 days, and B = 5.

Video S6 | Typical example of agent-based model simulation with high rates of peri-islet membrane degradation, as indicated in Section 4.3 with immune cell lifespans of 28 days, and B = 30.

#### REFERENCES


Video S7 | Typical example of agent-based model simulation using in the high chemokine regime, as indicated in Section 4.5 with immune cell lifespans of 28 days, and B = 5.

Video S8 | Typical example of agent-based model simulation using in the high chemokine regime, as indicated in Section 4.5 with immune cell lifespans of 28 days, and B = 30.

of cell-based morphogenesis. Comput. Phys. Commun. 176, 670–681. doi: 10.1016/j.cpc.2007.03.007


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2016 Wedgwood, Richardson, Morgan and Tsaneva-Atanasova. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# A Pointwise Method for Identifying Biomechanical Heterogeneity of the Human Gallbladder

Wenguang Li <sup>1</sup> \*, Nigel C. Bird<sup>2</sup> and Xiaoyu Luo<sup>3</sup>

*<sup>1</sup> School of Engineering, University of Glasgow, Glasgow, UK, <sup>2</sup> Academic Surgical Unit, Royal Hallamshire Hospital, Sheffield, UK, <sup>3</sup> School of Mathematics and Statistics, University of Glasgow, Glasgow, UK*

Identifying the heterogeneous biomechanical property of human gallbladder (GB) walls from non-invasive measurements can have clinical significance in patient-specific modeling and acalculous biliary pain diagnosis. In this article, a pointwise method was proposed to measure the heterogeneity of ten samples of human GB during refilling. Three different points, two on the equator of GB body 90◦ apart and one on the apex of GB fundus, were chosen to represent the typical regions of interest. The stretches at these points were estimated from ultrasound images of the GB during the bile emptying phase based on an analytical model. The model was validated against the experimental data of a lamb GB. The material parameters at the different points were determined inversely by making use of a structure-based anisotropic constitutive model. This anisotropic model yielded much better accuracy when compared to a number of phenomenologically-based constitutive laws, as demonstrated by its significantly reduced least-square errors in stress curve fitting. The results confirmed that the human GB wall material was heterogeneous, particularly toward the apex region. Our study also suggested that non-uniform wall thickness of the GB was important in determining the material parameters, in particular, on the parameters associated with the properties of the matrix and the longitudinal fibers—the difference could be as large as 20–30% compared to that of the uniform thickness model.

Keywords: gallbladder, strain energy function, heterogeneity, anisotropic property, constitutive law, optimization, inverse problem

# INTRODUCTION

Human gallbladder (GB) is a small pear-shaped organ that is attached to the underside of the right lobe of the liver. Its function is to store and concentrate bile produced continuously by the liver. Induced by cholecystokinin (CCK), bile can be expelled from the GB to the gut to aid the digestion of fat. Cholecystitis, often due to blockage of the cystic duct by gallstones, and acalculous biliary pain are common GB diseases that affect both women and men (Cozzolino et al., 1963; Williamson, 1988). The symptoms in acalculous biliary pain disease vary widely from discomfort to severe pain, which usually follows food intake. However, the painful symptoms remain in nearly 50% patients following gallbladder removal (Cholecystectomy) (Smythe et al., 1998, 2004). This is in part due to the lack of understanding of the underlying mechanism for acalculous biliary pain.

#### Edited by:

*Vanessa Diaz, University College London, UK*

#### Reviewed by:

*Haoxiang Luo, Vanderbilt University, USA Andrew James Narracott, University of Sheffield, UK*

\*Correspondence: *Wenguang Li wenguang.li@glasgow.ac.uk*

#### Specialty section:

*This article was submitted to Computational Physiology and Medicine, a section of the journal Frontiers in Physiology*

Received: *10 November 2016* Accepted: *07 March 2017* Published: *31 March 2017*

#### Citation:

*Li W, Bird NC and Luo X (2017) A Pointwise Method for Identifying Biomechanical Heterogeneity of the Human Gallbladder. Front. Physiol. 8:176. doi: 10.3389/fphys.2017.00176*

Interestingly, many human tissues, such as artery, breast, liver, and pancreas, can develop local disease, examples include vulnerable plaque (Baldewsing et al., 2004a; Trivedi et al., 2007), atheroma in coronary and femoral arteries (Chandran et al., 2003; Baldewsing et al., 2004b; Hamilton et al., 2005), arterial stenosis (Franquet et al., 2011), and cerebral aneurysms (Zhao et al., 2011a,b). Biomechanical properties of the diseased soft tissue are different to those of the healthy ones and are often heterogeneous. Inverse methods have been developed to identify isotropic biomechanical properties (Chandran et al., 2003; Baldewsing et al., 2004a,b; Hamilton et al., 2005; Trivedi et al., 2007; Franquet et al., 2011) in terms of the Young's modulus. In studies by Zhao, Raghavan, and Lu, pointwise inverse approaches were used to reveal the anisotropic heterogeneous biomechanical properties of cerebral aneurysms (Zhao et al., 2011a,b), ascending thoracic aneurysms (Davis et al., 2015) and murine aortas (Bersi et al., 2016) on a membrane mechanic model.

Healthy human GB wall is commonly regarded as a homogenous anisotropic non-linear material in passive state, i.e., bile refilling phase (Li et al., 2012, 2013; Xiong et al., 2013). However, recent work based on in vitro test of a healthy lamb GB suggested that this might not be true (Genovese et al., 2014). In addition, human acalculous biliary disease can lead to increased material heterogeneity in the GB wall. In this paper, we addressed this issue by extending the homogenous anisotropic non-linear biomechanical model for human GB wall proposed in Li et al. (2013) to a heterogeneous anisotropic case. We used an inverse pointwise method to identify the heterogeneous anisotropic property at three different points on the GB wall. The method was based on an ellipsoid membrane model and an in-house developed program using MATLAB.

#### COMPUTATIONAL MODELS

### Geometrical Model and Stresses under Internal Pressure

A series of ultrasonic images of acalculous human GB had been scanned in 10 min interval for 60 min during the emptying phase at the Sheffield Hallamshire Teaching Hospital. A typical example is illustrated in **Figure 1**, marked by the three axes D1, D2, and D<sup>3</sup> (D<sup>1</sup> ≤ D<sup>2</sup> ≤ D3). From these images we generated the corresponding ellipsoid models, as shown in **Figure 2A**, which are used to estimate the GB volume.

The passive biomechanical property of GB wall exhibits in the refilling phase only, hence we will focus this process. The refilling phase is the reverse process of the emptying phase. This means that for the same volume, the refilling and the emptying phases share the same ellipsoid (Li et al., 2013). The heterogeneous anisotropic biomechanical property of human GB wall in the refilling phase will be determined inversely at points 1, 2, and 3 in a GB wall. Point 1 is an intersected point of two ellipses, one is along the equator and the other is in the longitudinal direction in a meridian plane, point 2 is also on the equator but 90◦ apart from point 1, and point 3 is at the apex as shown in **Figure 2A**. In the spherical coordinate system (r, φ, θ), the coordinates of points 1, 2, and 3 are (D1/2, 0, π/2), (D2/2, π/2, π/2), and (D3/2, 0, π), respectively.

Observing that the volume of the GB model was reduced by 50% by the end of emptying (Li et al., 2013), we chose the end of the emptying configuration of the GB as the reference configuration.

We interpolated the GB model with fifteen time moments throughout the refilling phase based on GB images using the geometrical similarity (Li et al., 2011). The GB wall circumferential and longitudinal in-plane stretches at point 1 at a time instant t<sup>j</sup> is calculated with (Ragab and Bayoumi, 1998).

$$\begin{cases} \lambda\_{1j}^{\phi} = 1 + \frac{u\_r}{D\_{11}/2} + \frac{D\_{11}}{2} \frac{\partial u\_\phi}{\partial \phi} \\ \lambda\_{1j}^{\theta} = 1 + \frac{u\_r}{D\_{11}/2} + \frac{D\_{11}}{2} \frac{\partial u\_\theta}{\partial \theta} \end{cases} \tag{1}$$

where the radial displacement u<sup>r</sup> = 0.5 D1<sup>j</sup> − D11 , and j = 1,2,3...,N, here N = 15, D<sup>11</sup> is the length of the principal axis D<sup>1</sup> at time t1, D1<sup>j</sup> is the length of the principal axis D<sup>1</sup> at time t<sup>j</sup> . Since point 1 is on the axis of the ellipsoid, symmetry requires that ∂uφ/∂φ = ∂u<sup>θ</sup> /∂θ = 0. Hence, Equation (1) can be simplified to:

$$\begin{cases} \lambda\_{1j}^{\phi} = 1 + \frac{0.5(D\_{1j} - D\_{11})}{0.5D\_{11}} = D\_{1j}/D\_{11} \\ \lambda\_{1j}^{\theta} = 1 + \frac{0.5(D\_{1j} - D\_{11})}{0.5D\_{11}} = D\_{1j}/D\_{11} \end{cases} \tag{2}$$

The incompressibility of the GB wall means that the stretch component of GB thickness must satisfy:

$$
\lambda\_{1j}^h = 1/\left(\lambda\_{1j}^\phi \lambda\_{1j}^\theta\right). \tag{3}
$$

Similarly, the stretch components are for point 2:

$$\begin{cases} \lambda\_{2j}^{\phi} = \lambda\_{2j}^{\theta} = D\_{2j}/D\_{21} \\ \lambda\_{2j}^{h} = 1/\left(\lambda\_{2j}^{\phi}\lambda\_{2j}^{\theta}\right), \end{cases} \tag{4}$$

and for point 3:

$$\begin{cases} \lambda\_{3j}^{\phi} = \lambda\_{3j}^{\theta} = D\_{3j}/D\_{31} \\ \lambda\_{3j}^{h} = 1/\left(\lambda\_{3j}^{\phi}\lambda\_{3j}^{\theta}\right). \end{cases} \tag{5}$$

where D<sup>21</sup> and D2<sup>j</sup> are the length of principal axis D<sup>2</sup> at time t<sup>1</sup> and t<sup>j</sup> , while D<sup>31</sup> and D3<sup>j</sup> are the length of principal axis D<sup>3</sup> at time t<sup>1</sup> and t<sup>j</sup> .

These stretch components at t<sup>j</sup> (j = 1,...,N) and point i (i = 1,2,3) can be presented simply:

$$\begin{cases} \lambda\_{ij}^{\phi} = \lambda\_{ij}^{\theta} = D\_{ij}/D\_{i1} \\ \lambda\_{ij}^{h} = 1/\left(\lambda\_{ij}^{\phi}\lambda\_{ij}^{\theta}\right). \end{cases} \tag{6}$$

The stretches during the refilling phase are plotted against the GB volume in **Figure 2B** at points 1, 2, and 3 for a typical GB sample. The GB volume changed with time exponentially based on an earlier model in Li et al. (2011): V = Ge Ht + M, where

G, H, and M are parameters determined analytically using the measured GB volume and pressure at the moments t<sup>1</sup> and tN.

The expressions of in-plane stress components in the GB wall during the refilling phase were the same as these in the emptying phase (Li et al., 2011) since we assumed the GB material is an elastic membrane:

$$\begin{cases} \sigma\_{ij}^{\theta} = p\_{j} F\_{\theta} F\_{n} \\ \sigma\_{ij}^{\phi} = p\_{j} \frac{F\_{\phi}}{F\_{n}}, \end{cases} \tag{7}$$

where p<sup>j</sup> is the refilling pressure at time t<sup>j</sup> , and F<sup>θ</sup> , Fφ, and F<sup>n</sup> are the functions describing the geometry of the GB:

$$\begin{cases} F\_{\theta} = \frac{D\_{\overline{3}}K\_{\overline{1}\overline{}}K\_{\overline{2}\overline{}}}{4h\_{\overline{\imath}\overline{}}} \left(1 - \frac{K\_{\overline{1}\overline{\jmath}}^{2} - K\_{\overline{2}\overline{\jmath}}^{2}}{K\_{\overline{1}\overline{\jmath}}^{2}K\_{\overline{2}\overline{\jmath}}^{2}} \cos 2\phi\_{i}\right) \\\\ F\_{\theta} = \frac{D\_{\overline{3}}}{4K\_{\overline{1}\overline{}}K\_{\overline{2}\overline{\jmath}}h\_{\overline{\imath}}} \left[\frac{K\_{\overline{1}\overline{\jmath}}^{2}K\_{\overline{2}\overline{\jmath}}^{2} + \left(K\_{\overline{1}\overline{\jmath}}^{2} + K\_{\overline{2}\overline{\jmath}}^{2} - 2K\_{\overline{1}\overline{\jmath}}^{2}K\_{\overline{2}\overline{\jmath}}^{2}\right)\sin^{2}\theta\_{i}}{\left(K\_{\overline{1}\overline{\jmath}}^{2} - K\_{\overline{2}\overline{\jmath}}^{2}\right)\cos^{2}\theta\_{i}\cos 2\phi\_{i}}\right] \\\\ F\_{n} = \frac{\sqrt{K\_{\overline{1}\overline{\wp}}^{2}\cos^{2}\theta\_{i}\cos^{2}\phi\_{i} + K\_{\overline{2}\overline{\jmath}}^{2}\cos^{2}\theta\_{i}\sin^{2}\phi\_{i} + \sin^{2}\theta\_{i}}{\sqrt{K\_{\overline{1}\overline{\wp}}^{2}\sin^{2}\phi\_{i} + K\_{\overline{2}\overline{\wp}}^{2}\cos^{2}\phi\_{i}}}\end{cases} \tag{8}$$

where K1<sup>j</sup> = D1j/D3<sup>j</sup> , K2<sup>j</sup> = D2j/D3<sup>j</sup> , hij is the GB wall thickness at point i, and time t<sup>j</sup> , and Dij = λ θ ijDi1, hij = λ h ijhi1. The internal pressure p<sup>j</sup> is given by Li et al. (2013):


TABLE 1 | Parameters of ten human GB samples, these parameters are for one dataset rather than an average of the whole set.

*A uniform thickness, i.e., h<sup>11</sup>* = *h<sup>21</sup>* = *h<sup>31</sup>* = *2.5 mm, is assumed.*

$$p\_j = p\_1 \left(\frac{p\_N}{p\_1}\right)^{t\_j/t\_N}, t\_j \in [0, t\_N] \tag{9}$$

where p<sup>N</sup> is the mean final bile pressure in a GB after the refilling phase chosen to be 1,466.5 Pa (11 mmHg), p<sup>1</sup> is the bile pressure when the refilling starts, p<sup>1</sup> = 466.6 Pa (3 mmHg), and t<sup>N</sup> is the total time of the refilling phase. These values are estimated from in vivo measurements (Li et al., 2013).

# The Constitutive Model

To determine the heterogeneous material parameters of human GB wall, the structure-based anisotropic constitutive model used in Li et al. (2013) was extended so that the material parameters are location dependent. At each point, the GB wall is assumed to be composed of homogeneous matrix and two families of fibers along the circumferential and longitudinal directions, respectively, as shown in **Figure 3**. The strain energy functions are:

$$\begin{split} \psi\_{i} &= \boldsymbol{\varepsilon}^{i} \left( \boldsymbol{I}\_{1} - \mathbf{3} \right) + \frac{\kappa\_{1}^{i}}{2\kappa\_{2}^{i}} \Bigg[ \boldsymbol{\varepsilon}^{\kappa\_{2}^{i} \left( \left( \boldsymbol{\lambda}\_{i}^{\phi} \right)^{2} - 1 \right)^{2}} - 1 \Bigg] \\ &+ \frac{\kappa\_{3}^{i}}{2\kappa\_{4}^{i}} \Bigg[ \boldsymbol{\varepsilon}^{\kappa\_{4}^{i} \left( \left( \boldsymbol{\lambda}\_{i}^{\theta} \right)^{2} - 1 \right)^{2}} - 1 \Bigg] . \end{split} \tag{10}$$

where the parameters c i , κ i <sup>m</sup> (i = 1,2,3, m = 1–4) are location dependent. The total number of material property constants in Equation (10) for points 1–3 should be 15 in general. However, at point 3, there are no circumferential fibers, so the second term in Equation (10) disappears, i.e., κ 3 1 and κ 3 2 vanish. Hence, there are a total of 13 parameters to be determined.

The in-plane Cauchy stress components at t<sup>j</sup> are:

$$\begin{cases} \sigma\_{\dot{\boldsymbol{y}}}^{\prime \phi} = 2\boldsymbol{\epsilon}^{l} \left(\lambda\_{\dot{\boldsymbol{y}}}^{\phi 2} - \lambda\_{\dot{\boldsymbol{y}}}^{h2}\right) + 2\lambda\_{\dot{\boldsymbol{y}}}^{\phi 2} \kappa\_{1}^{l} \left(\lambda\_{\dot{\boldsymbol{y}}}^{\phi 2} - 1\right) \exp\left(\kappa\_{2}^{l} \left(\lambda\_{\dot{\boldsymbol{y}}}^{\phi 2} - 1\right)^{2}\right) \\ + \sigma\_{l1}^{\phi} \\ \sigma\_{\dot{\boldsymbol{y}}}^{\prime \theta} = 2\boldsymbol{\epsilon}^{l} \left(\lambda\_{\dot{\boldsymbol{y}}}^{\theta 2} - \lambda\_{\dot{\boldsymbol{y}}}^{h2}\right) + 2\lambda\_{\dot{\boldsymbol{y}}}^{\theta 2} \kappa\_{3}^{l} \left(\lambda\_{\dot{\boldsymbol{y}}}^{\theta 2} - 1\right) \exp\left(\kappa\_{4}^{l} \left(\lambda\_{\dot{\boldsymbol{y}}}^{\theta 2} - 1\right)^{2}\right) \\ + \sigma\_{l1}^{\theta} \end{cases} \tag{11}$$

fibers, the thirteen material parameters are location dependent, changing from points 1–3.

where σ φ i1 and σ θ i1 (i = 1,2,3) are interpreted as the initial stresses imbedded in the GB wall, which are estimated using Equations (7–9) with the internal pressure p1.

### Comparison with Other Constitutive Models

Several phenomenological anisotropic strain energy functions have been proposed for soft tissues. Here we did not intend to be exhaustive but will choose three commonly used strain energy functions for comparisons. These include the Fung strain energy function (Ferruzzi et al., 2011):

$$\psi = \frac{c}{2} \left\{ e^{\frac{1}{4} \left[ a\_1 \left( \lambda^{\phi 2} - 1 \right)^2 + a\_2 \left( \lambda^{\theta 2} \right) \right]^2} \right.$$

$$+ 2a\_3 (\lambda^{\phi 2} - 1)(\lambda^{\theta 2} - 1) \right\} - 1 \right\},\tag{12}$$

The Choi-Vito strain energy function (Ferruzzi et al., 2011):

$$\psi = c \begin{cases} e^{\frac{1}{4}a\_1 \left(\lambda^{\phi 2} - 1\right)^2} + e^{\frac{1}{4}a\_2 \left(\lambda^{\theta 2} - 1\right)^2} \\ + e^{\frac{1}{4}a\_3 \left(\lambda^{\phi 2} - 1\right) \left(\lambda^{\theta 2} - 1\right)} - 3 \end{cases} \tag{13}$$

and the Zhou-Fung strain energy function (Zhou and Fung, 1997):

$$\begin{split} \psi &= \frac{c}{2} \begin{Bmatrix} e^{\frac{1}{4} \left[ a\_1 \left( \lambda^{\phi 2} - 1 \right)^2 + a\_2 \left( \lambda^{\phi 2} - 1 \right)^2 + 2a\_3 \left( \lambda^{\phi 2} - 1 \right) \left( \lambda^{\phi 2} - 1 \right) \right] \\ - \frac{1}{4} \left[ a\_1 \left( \lambda^{\phi 2} - 1 \right)^2 + a\_2 \left( \lambda^{\phi 2} - 1 \right)^2 \\ + 2a\_3 \left( \lambda^{\phi 2} - 1 \right) \left( \lambda^{\phi 2} - 1 \right) \right] - 1 \\ &+ \frac{1}{8} \left[ b\_1 \left( \lambda^{\phi 2} - 1 \right)^2 + b\_2 \left( \lambda^{\phi 2} - 1 \right)^2 \right. \\ &+ 2b\_3 \left( \lambda^{\phi 2} - 1 \right) \left( \lambda^{\phi 2} - 1 \right) \end{Bmatrix} \end{split} \tag{14}$$

### Inverse Estimate of the Material Parameters

The material parameters in Equations (10) or (12) or (13) or (14) are selected to minimize the objective function:

$$f = \sum\_{i=1}^{3} \sum\_{j=1}^{N} \left[ \left( \sigma\_{ij}^{\phi} - \sigma\_{ij}^{\prime \phi} \right)^{2} + \left( \sigma\_{ij}^{\theta} - \sigma\_{\vec{\eta}}^{\prime \theta} \right)^{2} \right]. \tag{15}$$

The minimization was performed using the Trust-Region-Reflective algorithm in MATLAB (More and Sorensen, 1983) which terminated when the objective function value is less than 10−<sup>5</sup> . In addition, the following RMS error (RMSE) is calculated to assess the curve-fitting quality:

$$\varepsilon = \frac{\sqrt{\frac{1}{6N} \sum\_{i=1}^{3} \sum\_{j=1}^{N} \left[ \left( \sigma\_{ij}^{\phi} - \sigma\_{ij}^{\prime \phi} \right)^{2} + \left( \sigma\_{ij}^{\phi} - \sigma\_{ij}^{\prime \phi} \right)^{2} \right]}}{\frac{1}{6N} \sum\_{i=1}^{3} \sum\_{j=1}^{N} \left[ \sigma\_{ij}^{\phi} + \sigma\_{ij}^{\phi} \right]} \times 100\%. \tag{16}$$

It should be pointed out that the optimization process was conducted at points 1, 2, and 3 simultaneously rather than separately at each point. To secure a global minimum, the initial guesses of the parameters were chosen randomly within a suitable range, such as [0.01, 10] for c 1 , κ 1 1 , κ 1 2 , κ 1 4 , c 2 , κ 2 1 , κ 2 2 , κ 2 4 , c 3 , and κ 3 4 , but [0.01, 3] for κ 1 3 , κ 2 3 , and κ 3 3 . In those ranges, the optimized material parameters did not occur at the boundaries, and the curve fitting error was in the minimum. 80 initial guesses were generated randomly, followed by 80 optimization processes. The mean property constants and curve fitting errors were chosen to be the results. The detail of initial guesses on property constants optimization is given in Section Effects of Material Heterogeneity.

# The Variable Wall Thickness

with permission.

The GB wall thickness is related to the stress magnitude determined from the experimental images, as shown in Equations (7) and (8). This means that even when the pressure is the same, stresses can be different due to a varied thickness. This leads to different material parameters in the strain energy function in Equation (10). We now address the issue of the variable wall thickness of GB. A three-dimensional in vivo measurement of wall thickness of the GB was not yet available (Engel et al., 1980; Sanders, 1980; Prasad et al., 2008; Mohammed et al., 2010; Ugwu and Agwu, 2010); however, varying thickness of GB wall was measured in vitro with a digital slide caliper (Su, 2005; Khan et al., 2012). A contour of GB wall thickness is illustrated in **Figure 4** (Su, 2005). It is observed that the thickness of the GB apex in the fundus increases to around 5 mm maximum and the wall of the neck is as thin as 2 mm. The ratio of the maximum thickness over the thickness of the body is 1.2.

In Khan et al. (2012), 62 GB samples were divided into three age groups; (10–20) years, (21–40) years, and (41–70) years. The thicknesses of these GBs were measured manually at the fundus, body and neck. It was identified that the maximum thickness was found on the neck, and the thinnest wall is located at the fundus. For the (41–70) years group, which coincides with the patient's age group for GB surgery in the paper, the ratio of the thickness at the fundus over the thickness of the body is 0.9. This is contrary to the finding in Su (2005). These ratios are used to examine the effect of a varying thickness.

#### GB Samples

The input data for our model were the geometrical parameters based on ultrasound images, and internal pressures at the start and end of emptying/refilling phase of ten GB samples from a previous study (Li et al., 2013). These geometrical parameters and internal pressures are shown in **Table 1**. Additionally, the geometrical parameters and pressure profile at 15 or more moments between the start and the end of refilling phase were interpolated according to the method in Li et al. (2011) and Equation (9). A uniform wall thickness was assumed at the start of refilling phase, i.e., h<sup>11</sup> = h<sup>21</sup> = h<sup>31</sup> = 2.5 mm (Li et al., 2011,

2012, 2013). The stretch components of GB No. 1 over time are shown in **Figure 2B**. Note that the stretch-volume profiles are patient-specific.

In **Table 1**, the ejection fraction (EF) of a GB is defined as the ratio of the difference between the initial and emptied GB volumes at 30 min after venous injection of stimulator-CCK. Ethical approval for the use of data in **Figure 4** and (Li et al., 2013) were approved by the ethical committees in the hospital where the studies were conducted, and the subjects gave informed consent to these studies.

#### The Solution of Inverse Problem

From **Table 1**, we had the ellipsoid model geometrical parameters at the beginning and the end of the refilling phase, which were the same as the end (30 min after CCK) and beginning of the emptying phase from the routine ultrasound images taken in hospital, as shown in **Figure 1**. As the least squares method required more scattered points than the number of parameters to be estimated, the ellipsoid model (Li et al., 2011) was interpolated over 15 or more time points for the emptying phase. The internal bile pressure is then given by Equation (9). These data were used to obtain the initial guess for the optimization process. The stretch and stress components were then computed, and the objective function and the RMSE were evaluated and compared to a given criterion of 10−<sup>6</sup> . If the criterion was not satisfied, a new guess based on the Trust-Region-Reflective algorithm would be generated, and the procedure repeated until the convergent result is reached.

# RESULTS

# Effects of the Initial Guesses on Material Property Constants

GB No.1 shown in **Table 1** was randomly chosen to identify effects of initial guesses on the repeatability of inversely determined GB wall biomechanical property constants at points 1, 2, and 3. The initial guesses of the constants were generated randomly in the ranges for search of property constants mentioned in Section Inverse Estimate of the Material Parameters by normal distribution function in MATLAB and the numbers of initial guesses were specified 10, 20, 30, ..., 130, respectively. The means of the determined property constants and RMSE as well as their standard error at 95% confidence level are illustrated in **Table 2**. The true value of these property constants should be equal to the mean ± its standard error at 95% confidence level.

Note that the parameters at points 1–3 determined by the least squares method based on the Trust-Region-Reflective algorithm could not be repeated from one initial guess to another due to the complexity of the inverse problem. Considering the property constants determined from statistics point of view, however, the material parameters and their standard error at 95% confidence level inversely determined remained unchanged basically, especially when the number of initial guesses was 80 or more. This suggests that the biomechanical property constant values are repeatable in a statistical sense and are globally optimum. In the following sections, the property constants are extracted with 80 initial guesses at a computational cost of around



TABLE 3 | Heterogeneous material parameters of ten GB samples determined inversely with model Equation (10) and compared with homogeneous model (Li et al., 2013) in uniform thickness <sup>h</sup><sup>11</sup> <sup>=</sup> <sup>h</sup><sup>21</sup> <sup>=</sup> <sup>h</sup><sup>31</sup> <sup>=</sup> 2.5 mm.

30 min. This time consumption is much less than 3.5–7.0 h based on the approach of ABAQUS 3D FEA plus MATLAB optimization solver in Li et al. (2013).

#### Effects of Material Heterogeneity

The material parameters of heterogeneity inversely determined are listed in **Table 3** and compared with those from the corresponding homogenous model in Li et al. (2013). The error ε in the homogenous model reflects the error in GB volume between image observation and homogenous model prediction.

For all the GBs, the material parameter associated with the matrix in the heterogeneous model is around 10 times that of the homogenous model. For GB 3, 4, 17, 29, 37, 39, and 43, the mean values of fibers-related material parameters at points 1 and 2, κ 2 1 and κ 2 3 , basically agree with κ<sup>1</sup> and κ<sup>3</sup> in the homogenous model, i.e., κ 2 <sup>1</sup> ≈ (1–2) κ<sup>1</sup> and κ 2 <sup>3</sup> ≈ (1–2) κ3. For GB 1, 19, and 21, κ 2 1 and TABLE 4 | The first principal stresses in 10 GB samples wall estimated by using the homogenous model in Li et al. (2013) and the heterogeneous model Equation (10) in the present paper.


TABLE 5 | Material parameters of Fung strain energy function Equation (14) inversely determined with uniform thickness <sup>h</sup><sup>11</sup> <sup>=</sup> <sup>h</sup><sup>21</sup> <sup>=</sup> <sup>h</sup><sup>31</sup> <sup>=</sup> 2.5 mm.


κ 2 3 are different from κ<sup>1</sup> and κ<sup>3</sup> in the homogenous model. On one hand, the material parameters in the heterogeneous model at point 1 and point 2 are similar, implying the heterogeneity of the GB wall along the circumference is small. This is especially true for GB 37 which has D<sup>1</sup> ≈ D2; the points 1 and 2 share the same parameters. On the other hand, the parameters at point 3 differ substantially from the other two, suggesting a strong heterogeneity from GB body to fundus.

The first principal stresses of all the GB samples are compared in **Table 4** with those predicted by the homogenous model in


TABLE 6 | Material parameters of Choi-Vito strain energy function Equation (13) inversely determined with uniform thickness <sup>h</sup><sup>11</sup> <sup>=</sup> <sup>h</sup><sup>21</sup> <sup>=</sup> <sup>h</sup><sup>31</sup> <sup>=</sup> 2.5 mm.

Li et al. (2013). These stresses are extracted at point 1 since the length of an ellipsoid major axis is the shortest through that point, resulting in the highest stress level there based on Equations (7) and (8) in the φ direction. It is shown that the homogenous model underestimate the stresses in the wall of GB1, 3, 4, 29, 39, and 43, and overestimates them for the remaining GBs. As a result, the relative error in the first principal stresses varies in a range of −11.4%∼ +10.8% in comparison with the stresses in the homogenous model.

# Comparison with Other Constitutive Models

The inversely estimated parameters are shown in **Tables 5**–**7** for the Fung, Choi-Vito, and Zhou-Fung strain energy functions, respectively. The parameters in the Zhou-Fung model were as many as 17 in total at the three points; thus the number of time instants was increased to 30 in the optimization procedure.

Our results show that even though the model parameters using these strain energy functions can also be inversely determined, the errors in stress are quite large. For instance, the mean errors are 14.8, 14.6, and 14.0% for the Fung, Choi-Vito and Zhou-Fung strain energy functions, respectively, while the structure-based model Equation (10) yields a mean error of 5.0% only (**Figure 5**).

#### Variation of the GB Wall Thickness

We notice from **Table 3** that there are some large errors ranged between 5.2 and 7.8% for the parameters estimated for five GB samples: 1, 17, 21, 37, and 39. To identify the cause of the errors, the stress-volume curves of GB 3 and 39 at points 1, 2, and 3


are shown in **Figure 6**. The predicted stress agrees well with the observations at points 1 and 2, but not so well at point 3. In the following, we show that this is due to the uniform wall thickness assumption used in the model.

In Section Comparison with Other Constitutive Models, the GB wall heterogeneity mainly occurs in the apex region, resulting in poor agreement in the stress, as shown in **Figure 6**. Therefore, we altered the GB wall thickness at the apex to examine the effect of varying thickness. First, the apex thickness was changed to 3.0 mm, based on the ratio of 1.2 found in Su (2005), and kept at 2.5 mm at points 1 and 2. The extracted pointwise mechanical properties in Equation (10) are shown in **Table 8**.

The relative changes in these 13 parameters are tabulated in **Table 9**. The increased thickness at the apex by 20% has a considerable effect on the material parameters, with changes up to 30%, in particular, on κ 1 4 , c 2 , κ 2 4 , c 3 , κ 3 3 , and κ 3 4 , which are associated with the properties of the matrix and the longitudinal fibers. This is very different to the membrane theory, in which the Young's modulus is independent of the membrane thickness (Timoshenko and Woinowsky-Krieger, 1959).

We also found that an increased h<sup>31</sup> could lower the error in the stress between the model production and the observation. If we increase h<sup>31</sup> to 5.0 mm, the error reduces by 2.5%. Further increase in thickness does not decrease the error much, see **Figure 7**. Interestingly, 5.0 mm apex thickness seems to agree with measurement in Su (2005). Finally, if h<sup>31</sup> is reduced to be 10% thinner than h<sup>11</sup> and h21, 2.25 m, according to Khan et al. (2012), then the errors in the stresses are greater, as shown in **Figure 7**. Thus, the observation that apex thickness was thinner than the GB body in Khan et al. (2012) did not agree with the results from the cohort of GB samples used here.

Note that in Khan et al. (2012) post-mortem samples from "unclaimed bodies" were used and so would not have been fresh. When left in situ the bile will start to break down the gallbladder wall–a process known as autolysis. Therefore, the results for wall thickness might not be reliable. Samples in Su (2005), on the other hand were obtained fresh from the operating theater and washed immediately. So we would have more faith in the results in Su (2005).

#### Comparison with Animal Test

Our ellipsoid model is different from the patient specific GB geometries. One may ask if such a simple model is of any practical use. To answer this question, we compared our model prediction with the in vitro measurements of a lamb GB (Genovese et al., 2014). In Genovese et al. (2014), a lamb GB was harvested and inflated at a pressure up to 50 mmHg, then a series points on the GB outside surface were tracked optically, and the strain fields were estimated from fitting curves of these points. The tension/stress fields were then calculated by using the elastic membrane model and solved numerically. In our model, we only used the diameters from Genovese et al. (2014) as the two minor axis lengths D<sup>1</sup> and D2, respectively. The stress field, hence tension, are obtained analytically from Equation (7). The results are shown in **Figure 8**, where the comparisons of the second Piola-Kirchhoff surface tensions are plotted for pressure p = 20 and 50 mmHg (we couldn't compare the results at p = 3.5 mmHg, as the tension profile in Genovese et al. (2014) seems to

FIGURE 6 | Comparison of the modeled (lines) and estimated (symbols) circumferential and longitudinal stresses with the image-based ellipsoid membrane mechanic model at points 1, 2, 3, for GB 3 (A–C), and GB 39 (D–F).



be unrealistically large, which is possibly due to a typo in the color map scale).

The overall agreement is encouraging; in both our model and experiments, the highest tension is found near the GB equator, and the minimum tension occurs at the apex. The values of the predicted surface tension are also in good agreement with the experimental data except some isolated tension spots due to the rapid change in the wall curvature of the lamb GB. The predicted surface tension magnitude near the GB body/equator is in a range of 0.023–0.027 N/mm, compared with 0.03–0.04 N/mm in the experiments at 20 mmHg pressure. Likewise, the predicted tension is in a range of 0.063–0.073 N/mm, compared to the range of 0.06–0.08 N/mm in the experiments at 50 mmHg.

# DISCUSSION

Ultrasonography is a common method for monitoring GB volume variations in daily diagnosis (Dodds et al., 1985; Portincasa et al., 2003; Ugwu and Agwu, 2010). Although a detailed 3D model is more accurate, simplified geometry models are fast and therefore frequently used in clinical assessment. When an ellipsoidal model is used to estimate GB volume based on the images scanned during emptying phase, the error of the model in GB volume is about 0.8 ± 0.1 ml. This compares better to the error of 2.1 ± 0.2 ml if using sum-of-cylinder method (Dodds et al., 1985). To assess if the simplified model could predict the stress distribution of a realistic GB sample, we also


TABLE 9 | Relative changes in the parameters due to varied wall thickness.

∆*c<sup>i</sup> ,* ∆κ*<sup>i</sup> 1 ,* ∆κ*<sup>i</sup> 2 ,* ∆κ*<sup>i</sup> 3 ,* ∆κ*<sup>i</sup> 4 and* ∆ε *are the differences of these parameters and error between the case of h<sup>31</sup>* = *3.0 mm thick apex wall and the case of 2.5 mm uniform GB wall, c<sup>i</sup> ,* κ *i 1 ,* κ *i 2 ,* κ *i 3* κ *i 4 are the parameters for the 2.5 mm uniform GB wall.*

compared our model prediction with the surface tension data for a lamb GB, and the overall agreement was surprisingly good.

We comment that the segmentation error of estimating the GB diameter from a GB image is usually around 4.31–7.21% (Bocchi and Rogai, 2011). To address the effect of this error on the GB wall material parameters, we introduce a random error (or noise) of 4.31–7.21% in the major axis lengths, i.e.:

$$
\varepsilon\_{\text{seg}} = 0.0431 + rand \times (0.0721 - 0.0431) \tag{17}
$$

where rand is the inner random function in MATLAB to generate a random number in value 0–1. Then we run the inverse heterogeneous problem code with these noisy data for a number of GB sample, say No. 1, 3, 17, and 21. The parameters estimated are compared in **Table 10** against those without the noise.

When noise is considered, the error in the curve fitting increased mostly by 3.1–6.7%, some can go as high as 12.3%, in comparison with the case without the noise. The influence of segmentation error on the parameters varies from one GB to another, however, the parameters at points 1, 2 are mostly likely affected by the segmentation error, particularly, changes in c 1 , κ 1 2 , κ 1 3 , κ 1 4 , c 2 , κ 2 3 , are κ 2 4 can be large. Hence, the segmentation error should be reduced as much as possible to

improve the accuracy of the inverse estimation. In future, using an automatic segmentation method with small segmentation error as introduced in Bocchi and Rogai (2011) may be the way forward.

In our previous work in Li et al. (2013), the human GB wall was considered to be a non-linear composite material of matrix and two orthogonal families of fibers in the circumferential and longitudinal directions, respectively, and the material parameters were assumed to be constant. These parameters were determined inversely in Li et al. (2013) by using the FEM software-ABAQUS with a user subroutine and a MATLAB code. However, such an inverse approach is extremely timeconsuming (∼7 h) and unsuitable for clinical applications. In this work, we have developed a simpler approach using analytical or simpler forward solvers, which makes it possible for clinical assessment of in GB human wall disease in real time.

In addition, we extended the previous model from homogenous membrane model in Li et al. (2013) to heterogeneous model, which has significantly improved the fitting accuracy. The heterogeneity of the GB has been confirmed in the experimental work on lamb GB (Genovese et al., 2014). The inverse estimation of the heterogeneous property constants had an error less than 7% for the ten human GB samples, and the computational time was reduced by 20 times (∼30 min). Further, allowing the wall thickness variation following experiments (Su, 2005), reduced the error to be less than 4%.

One potential clinical use of the model is to assessing the GB pain. In **Table 4**, we compare the first principal stress with the pain score associated with the CCK venous injection. It is clear that there is a strong correlation between the magnitude of the stress and the pain score. Although given the limited sample size, the homogeneous and heterogeneous model seem to do equally well in terms of pain prediction.

The limitations of our study should also be mentioned. In the study, the stretches at the points 1–3 were determined analytically during GB emptying phase. The analytical method was based on the GB volume change from images. To our best knowledge, no speckle tracking echocardiography on GB has been reported to validate our model, unlike extensive measured on human left ventricle (Edvardsen et al., 2002; Marwick, 2006; Crosby et al., 2009; Maffessanti et al., 2009; Marwick et al., 2009; Tanaka et al., 2010; Hoit, 2011; Kleijn et al., 2011). In addition, there were also no in vitro passive tensile tests on the specimens harvested from the body and fundus of human GB. In future, we may be able to utilize the measured strain/stretch to validate our analytical method for stretch extraction, this will make our regional GB biomechanical property identification more accurate.

Further, we only used one in vitro observation to determine the reference configuration of human GBs. In reality, the size of a reference configuration may not be exactly 50% of the size of totally refilled GB. It is possible that the GB reference configuration can be estimated using GB ejection fraction (EF) in cholecystokinin-cholescintigraphy (CCK-CS) (Ozden and DiBaise, 2003) or fatty meal Cholescintigraphy (FM-CS) (Al-Muqbel et al., 2010) exanimation for GB patients' in vivo clinical diagnosis.

Although we have investigated the thickness variation in our model, the values we used were applicable only for a healthy GB. When human GBs suffer from diseases such as acute cholecystitis, acalculous cholecystitis and ascites (Sanders, 1980; Runner et al., 2014), the GB thickness can increase significantly. Indeed, diseased GB body wall thickness was ranged in 3–5 mm (Sanders, 1980; Mohammed et al., 2010; Runner et al., 2014). How to estimate the GB wall thickness change in disease will be an important challenge for future studies.

# CONCLUSIONS

The heterogeneity of ten samples of human GB is investigated theoretically in refilling phase using a structure-based constitutive model, ellipsoidal GB and membrane in-plane mechanic model. Three different points, two on the equator of GB body with 90◦ apart and one on the apex of GB fundus, are chosen to evaluate the variation of the material properties. The stretches at these points are tracked over time from the routine ultrasonic images scanned at the Sheffield Hallamshire Hospital during the emptying phase. The material parameters at the three different points are determined inversely using a MATLAB code. The human GBs are found to exhibit heterogeneity, especially from GB body to its apex region. It is found that using a homogeneous model underestimate the peak stresses in the GB wall, and that a strong heterogeneity occurs from GB body to fundus. Finally, our model results indicate that the GB wall is much thicker at the apex, which clarify the contrary findings reported in the literature.


TABLE 10 | Heterogeneous material parameters of four GB samples inversely determined with Equation (10) and uniform thickness <sup>h</sup><sup>11</sup> <sup>=</sup> <sup>h</sup><sup>21</sup> <sup>=</sup> <sup>h</sup><sup>31</sup> <sup>=</sup> 2.5 mm when the error in segmentation is considered.

#### AUTHOR CONTRIBUTIONS

WL developed math model, designed numerical method, composed and validated code, carried out case studies, analyzed the results and drafted the manuscript. NB checked medical issues in human gallbladder, revised the draft. XL checked the math model,

#### REFERENCES

Al-Muqbel, K. M., Bani Hani, M, N., Elheis, M. A., and Al-Omari, M. H. (2010). Reproducibility of gallbladder ejection fraction measured by fatty meal cholescintigraphy. Nucl. Med. Mol. Imaging 44, 246–251. doi: 10.1007/s13139-010-0046-8

method, result explanation and conducted English text editing.

#### ACKNOWLEDGMENTS

The project was supported by EPSRC (Grant no. EP/G015651 EP/G028257, and EP/N014642/1).


elastography of human atherosclerotic coronary arteries. Ultrasound Med. Biol. 30, 803–813. doi: 10.1016/j.ultrasmedbio.2004.04.005


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2017 Li, Bird and Luo. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# **Input Estimation for Extended-Release Formulations Exemplified with Exenatide**

*Magnus Trägårdh1,2 \*, Michael J. Chappell <sup>1</sup> , Johan E. Palm<sup>3</sup> , Neil D. Evans <sup>1</sup> , David L. I. Janzén1,2,4 and Peter Gennemark <sup>2</sup>*

*<sup>1</sup> School of Engineering, University of Warwick, Coventry, UK, <sup>2</sup> Cardiovascular and Metabolic Diseases, Innovative Medicines and Early Development Biotech Unit, AstraZeneca, Mölndal, Sweden, <sup>3</sup> Global Product Development, Pharmaceutical Technology and Development, AstraZeneca, Mölndal, Sweden, <sup>4</sup> Department of Systems and Data Analysis, Fraunhofer-Chalmers Centre, Gothenburg, Sweden*

*Edited by:*

*Krasimira Tsaneva-Atanasova, University of Exeter, UK*

#### *Reviewed by:*

*Laurent Simon, New Jersey Institute of Technology, USA Ivo Siekmann, University of Göttingen, Germany*

> *\*Correspondence: Magnus Trägårdh m.a.tragardh@warwick.ac.uk*

#### *Specialty section:*

*This article was submitted to Computational Physiology and Medicine, a section of the journal Frontiers in Bioengineering and Biotechnology*

> *Received: 05 August 2016 Accepted: 28 March 2017 Published: 19 April 2017*

#### *Citation:*

*Trägårdh M, Chappell MJ, Palm JE, Evans ND, Janzén DLI and Gennemark P (2017) Input Estimation for Extended-Release Formulations Exemplified with Exenatide. Front. Bioeng. Biotechnol. 5:24. doi: 10.3389/fbioe.2017.00024* Estimating the *in vivo* absorption profile of a drug is essential when developing extendedrelease medications. Such estimates can be obtained by measuring plasma concentrations over time and inferring the absorption from a model of the drug's pharmacokinetics. Of particular interest is to predict the bioavailability—the fraction of the drug that is absorbed and enters the systemic circulation. This paper presents a framework for addressing this class of estimation problems and gives advice on the choice of method. In parametric methods, a model is constructed for the absorption process, which can be difficult when the absorption has a complicated profile. Here, we place emphasis on non-parametric methods that avoid making strong assumptions about the absorption. A modern estimation method that can address very general input-estimation problems has previously been presented. In this method, the absorption profile is modeled as a stochastic process, which is estimated using Markov chain Monte Carlo techniques. The applicability of this method for extended-release formulation development is evaluated by analyzing a dataset of Bydureon, an injectable extended-release suspension formulation of exenatide, a GLP-1 receptor agonist for treating diabetes. This drug is known to have non-linear pharmacokinetics. Its plasma concentration profile exhibits multiple peaks, something that can make parametric modeling challenging, but poses no major difficulties for non-parametric methods. The method is also validated on synthetic data, exploring the effects of sampling and noise on the accuracy of the estimates.

**Keywords: input estimation, deconvolution, Markov chain Monte Carlo, exenatide, extended release**

# **1. INTRODUCTION**

Extended-Release (ER) drug formulations are commonly used to improve the properties of drugs. They can allow for less frequent dosing schedules, improving compliance and quality for the patient. They can also improve safety by lowering the peak plasma concentration and enable the development and use of drugs whose pharmacokinetic (PK) properties would otherwise be unacceptable. For ER medications, the formulation design is specifically intended to provide a targeted release or input rate that optimizes the compound PK. ER medications are typically administered orally (tablets and capsules) or injected as intramuscular/subcutaneous depot formulations.

Most types of oral ER technologies today are based on polymeric systems (Yang and Pierstorff, 2012; Arafat, 2015). The oral formulations can be categorized into matrix, reservoir (or membrane controlled), and osmotic systems (Ratnaparkhi and Gupta Jyoti, 2013; Khalane et al., 2016). The drug release mechanisms involve drug diffusion, system swelling, or erosion and dissolution, or osmotic pressure-induced release (Siepmann and Göpferich, 2001; Arifin et al., 2006). Parenteral depot injections are used to achieve extended drug release over a week or longer. They include formulation types such as oil-based solutions, drug suspensions, polymer-based microspheres and polymer-based or lipid liquid crystal *in situ* formings (Rhee et al., 2010; Gulati and Gupta, 2011; Schwendeman et al., 2014). Biodegradable microsphere systems (e.g., made of PLGA copolymer) have proved to be a successful approach to deliver macromolecular drugs (Mitragotri et al., 2014).

In any ER-formulation development process, it is fundamental to determine the *in vivo* drug release/absorption profile of each candidate formulation. This is done routinely in drug discovery and development. Measuring the absorption profile *in vivo* is generally difficult and expensive. Typically, the data that are available are plasma concentration profiles following extravascular administration. If a model of the PK is available, it is possible to infer the absorption profile from plasma concentration data. The total amount of drug absorbed, and therefore the bioavailability, can be computed by integrating the absorption profile. Standard methods exist for the case where the PK is linear (Verotta, 1996). However, methods that are applicable to the non-linear case are not widely available.

When predicted *in vivo* input profiles are available, it may be possible to validate or invalidate the translatability of the *in vitro* system. Given data for several candidate formulations, an *in vitro in vivo correlation* (ivivc) can be established, relating the *in vitro* drug dissolution or release to the *in vivo* drug absorption or release (Lu et al., 2011; Cardot and Davit, 2012). Ideally, one can then predict the *in vivo* performance based on the *in vitro* release profile and optimize the formulation by *in vitro* testing at low cost. In addition, knowledge of the absorption profile in an animal model can help in predicting, and hence optimizing, the human PK profile. To achieve this, a human intravenous PK model is required, either from real data or predicted from cellular or animal data. The absorption profile obtained from animal data is fed to the human model, resulting in human PK trajectories. This type of human predictions is always desired in drug discovery to assess feasibility. Naturally, prediction reliability increases with the amount and quality of data.

One way to estimate the absorption profile is to build a parametric model of the drug release and absorption processes. For the drug release process, various models have been proposed, ranging from simple empirical models to detailed mechanistic models that account for various processes such as degradation and erosion (Siepmann and Peppas, 2001; Versypt et al., 2013). However, if the release profile is complicated, it may be difficult to create a model that is able to capture the observed plasma concentration (Shen and Burgess, 2015). One example is long acting biodegradable particles for subcutaneous injection. The model may also need to be tailored to the particular type of drug and formulation used. For sparse data, such models may also have practical identifiability issues. An alternative is to use *non-parametric* methods. In these methods, the release/absorption profile is allowed to take any functional form as long as it matches the data and does not exhibit any unrealistic behavior, such as taking negative values. Predictions from such non-parametric methods are often sufficient for compound/formulation selection in drug discovery.

This paper considers such non-parametric methods for estimating the release/absorption profile and bioavailability of extended-release formulations and gives advice on the choice of methods, given the data and system knowledge that are available. The choice of method depends on the characteristics of the PK model:


$$\mathbf{C}(t) = I(t) \* \boldsymbol{\mu}(t),\tag{1}$$

where *I*(*t*) is the impulse response of the system and *∗* is the *convolution operator*. Estimating *u*(*t*) from *C*(*t*) is consequently referred to as *deconvolution* (Verotta, 1996). The impulse response can be derived from a model, if one is available, or may be determined empirically, e.g., from intravenous data.

*•* The most general case is when the dynamics of the PK model are non-linear. Here, the relationship between *u*(*t*) and *C*(*t*) cannot be expressed by a convolution operation. Estimating the absorption profile is still possible if a (non-linear) PK model is available. In this case, the dynamics are represented by a system of ordinary differential equations, which is integrated numerically as part of the estimation procedure. Since this operation is not related to convolution, we prefer the more general term *input estimation*.

A decision tree summarizing these aspects is given in **Figure 1**. In its most general form, *input estimation* is the technique of estimating the input to a dynamical system, given measurements of the system's state. In the present case, the input is the release/absorption profile of the drug, *u*(*t*), the dynamical system is the PK model, and the measurements are of plasma concentrations *C*(*t*). A methodology for performing such analyses has been presented in Trägårdh et al. (2016). These methods do not make any assumptions about stationarity or linearity and are therefore applicable to the most general case presented above. Previously, the methods have been applied to estimating the

absorption profile of an immediate-release (IR) formulation of the drug eflornithine as well as for estimating the energy intake in bodyweight models. The purpose of this paper is to evaluate the applicability of the method of Trägårdh et al. (2016) to ERformulation problems and to investigate what adaptations, if any, are necessary in order to ensure good performance on this kind of problem. Additionally, the accuracy of the method is evaluated on simulated data for which the true input function is known. Estimation of ER release/absorption profiles differs from estimation of IR profiles in the following respects:


Model dynamics that are fast compared to the timescales of interest can cause stiffness issues. In general, an estimation method that has previously been shown to perform well will not necessarily perform well when applied to a problem with substantially different data and model parameters. For this reason, it is essential to evaluate the methodology in Trägårdh et al. (2016) on a realistic ER estimation problem.

In this paper, these methods are applied to Bydureon (Buse et al., 2010, 2013), an extended-release microsphere formulation of the GLP-1 receptor agonist exenatide (Buse et al., 2004; DeFronzo et al., 2005). The Bydureon formulation consists of exenatide encapsulated within poly-(,-lactide-co-glycolide) (PLG) microspheres that are designed to release exenatide over an extended period of time which allows once-weekly patientadministered subcutaneous injections (European Medicines Agency, 2011). Typical *in vitro* release curves for Bydureon are given in Figure 3 in DeYoung et al. (2011). Such curves can be used, together with predicted input profiles from *in vivo* data, to establish an ivivc. In humans, Bydureon exhibits a multiphasic concentration–time profile over approximately 10 weeks consistent with the proposed mechanism of release from PLG microspheres. This is characterized by a limited initial rapid release of loosely bound surface exenatide (*<*1% released in the first few hours) followed by two additional phases corresponding to diffusion and erosion release with peak plasma concentrations at around week 2 and week 7 (DeYoung et al., 2011).

The reason for choosing Bydureon as an example is that it is a drug that is already on the market, and data (Fineman et al., 2011; Li et al., 2015) as well as PK models (Gao and Jusko, 2012) are available in the literature. The complicated absorption profile of exenatide (**Figure 2**) cannot be easily captured by a simple parametric model. A compartmental model of the release and absorption processes has been proposed (Li et al., 2015), where the ER process is modeled by a cascade of transition compartments, and the initial amount of several compartments is nonzero. However, this model was designed to fit data from multipledosing experiments, where the multiple absorption peaks are not as noticeable.

The outline of the paper is as follows:


#### **2. MATERIALS AND METHODS**

#### **2.1. Model**

The PK model of Gao and Jusko (2012) was used for the system dynamics. The model includes non-linear target-mediated drug disposition (TMDD) and is given by

$$\frac{\mathrm{dC}(t)}{\mathrm{d}t} = \frac{u(t)}{V\_{\varepsilon}} - (k\_{\mathrm{el}} + k\_{\mathrm{pt}}) \cdot \mathrm{C}(t) + k\_{\mathrm{tp}} \cdot \frac{A\_T(t)}{V\_{\varepsilon}}$$

$$-k\_{\mathrm{on}} \cdot (\mathrm{R}\_{\mathrm{tot}} - \mathrm{RC}(t)) \cdot \mathrm{C}(t) + k\_{\mathrm{off}} \cdot \mathrm{RC}(t), \tag{2}$$

$$\frac{\mathrm{d}A\_T(t)}{\mathrm{d}t} = k\_{\mathrm{pt}} \cdot \mathrm{C}(t) \cdot V\_c - k\_{\mathrm{tp}} \cdot A\_T(t), \tag{3}$$

$$\frac{d\mathcal{R}C(t)}{dt} = k\_{on} \cdot (R\_{tot} - \mathcal{R}C(t)) \cdot C(t) - (k\_{off} + k\_{int}) \cdot \mathcal{R}C(t),\tag{4}$$

**TABLE 1 | Pharmacokinetic parameters, from Gao and Jusko (2012)**.


where *C*(*t*) is the drug concentration in the central compartment, *AT*(*t*) is the drug amount in a peripheral compartment, *RC*(*t*) is the concentration of the drug–receptor complex, and *u*(*t*) is the unknown input. The parameter values reported in Gao and Jusko (2012) were used (**Table 1**). The model structure is similar to the PK model for eflornithine by Johansson et al. (2013), which was used for evaluating the estimation method on IR formulations in Trägårdh et al. (2016). The main difference between these model structures is that the model used here has an additional elimination mechanism in the form of drugreceptor complex internalization, represented by *kint* in equation (4). Additionally, the parameter values are substantially different.

In addition to the model presented in Gao and Jusko (2012), similar models have been developed by Li et al. (2015) and Chen et al. (2013). These models differ from that of Gao and Jusko in that the total receptor concentration is described by a turnover model, instead of a fixed amount. Additionally, the Li et al. model was estimated using data from an ER formulation, using a linear 5-compartment model to represent the combined release and absorption process, while the Chen et al. model was estimated using data from an IR formulation and applying a Michaelis–Menten absorption function. As the inputestimation methods considered here provide non-parametric estimates of the release/absorption profiles, no parametric model was used in this paper for the release or absorption process. Instead, only the part of the PK model that describes the system dynamics following absorption was used, as detailed in equations (2)–(4).

To ensure the validity of the reported parameter values of these PK models, the structural identifiability was analyzed using the Exact Arithmetic Rank approach (Karlsson et al., 2012). The analysis showed that the PK models in Li et al. (2015) and Gao and Jusko (2012) are structurally identifiable. This result holds for intravenous (IV) administration, either bolus or continuous, as well as for subcutaneous (SC) administration. Since the model in Chen et al. (2013) shares the model structure of the PK after absorption with Li et al. (2015), it follows that this too is identifiable for IV administration. To summarize, any of these models could have been used in the subsequent analysis.

Once the PK parameters were shown to be identifiable, the next step was to ensure that the absorption profile can be estimated from plasma concentration data, given known PK parameters. An analysis of the identifiability of the input signal in the PK model in Gao and Jusko (2012) was performed using a Taylor series approach (Pohjanpalo, 1978). The analysis found that the input function is identifiable, given that the PK parameters are known. Details of the structural identifiability analysis of the PK models and the input function can be found in Section 1 in the Supplementary Material.

# **2.2. Estimation Method**

The analysis in this paper uses the methods described in Trägårdh et al. (2016). In order to ensure that the methods were suitable for the ER estimation problem at hand, the following adaptations were performed:


As described in Trägårdh et al. (2016), these input-estimation methods model the absorption profile as a stochastic process, which is equipped with a prior whose role is to discourage solutions that have unrealistically large oscillations (Verotta, 1996; De Nicolao et al., 1997). In any given estimation problem, a choice has to be made for


Here, the *regularization parameter*, which determines the trade-off between the data fit and the smoothness conditions, was treated as a parameter to be estimated, being assigned a Gamma prior distribution with parameters *α* = *β* = 10*<sup>−</sup>*<sup>3</sup> . MCMC Samples were drawn by alternately updating the basis function coefficients using the Simplified Manifold Metropolisadjusted Langevin algorithm (SMMALA) (Girolami and Calderhead, 2011), and updating the regularization parameter using Gibbs sampling (Geman and Geman, 1984). The Raftery–Lewis method (Raftery and Lewis, 1992) was used to assess the number of samples required, estimating the quantiles *q* = [0.025 0.25 0.5 0.75 0.975] with precision *r* = [0.02 0.05 0.06 0.05 0.02] and probability *s* = 0.95, as defined in Raftery and Lewis (1992).

It can be noted that on timescales of weeks, the system can be considered to be in a steady state. This means that the plasma concentration depends only on the current absorption rate, and the dynamical ODE model can be simplified to an algebraic model. This is done by setting all derivatives to zero, and solving for the measured quantity. This gives

$$\begin{split} C(t) &= \frac{1}{2V\_{\epsilon}k\_{\rm el}k\_{\rm on}} \Big( -R\_{\rm tot}V\_{\epsilon}k\_{\rm int}k\_{\rm on} - V\_{\epsilon}k\_{\rm el}k\_{\rm int} - V\_{\epsilon}k\_{\rm el}k\_{\rm off} \\ &+ \iota(t)k\_{\rm on} + \left( R\_{\rm tot}^2 V\_{\epsilon}^2 k\_{\rm int}^2 k\_{\rm on}^2 + 2R\_{\rm tot}V\_{\epsilon}^2 k\_{\rm el}k\_{\rm int}^2 k\_{\rm on} \right. \\ &+ 2R\_{\rm tot}V\_{\epsilon}^2 k\_{\rm el}k\_{\rm int} k\_{\rm off} k\_{\rm on} - 2R\_{\rm tot}V\_{\epsilon}\iota(t)k\_{\rm in}k\_{\rm on}^2 \\ &+ V\_{\epsilon}^2 k\_{\rm el}^2 k\_{\rm int}^2 + 2V\_{\epsilon}^2 k\_{\rm el}^2 k\_{\rm int} k\_{\rm off} + V\_{\epsilon}^2 k\_{\rm el}^2 k\_{\rm off}^2 \\ &+ 2V\_{\epsilon}\iota(t)k\_{\rm el}k\_{\rm int} k\_{\rm on} + 2V\_{\epsilon}\iota(t)k\_{\rm el}k\_{\rm off} k\_{\rm on} + \iota(t)^2 k\_{\rm on}^2 \Big) \frac{1}{\!} \Big). \end{split} \tag{5}$$

Large computational savings can be achieved by utilizing this result instead of integrating the system of ODEs. This is especially valuable when using MCMC methods, which need to perform these computations a large number of times, potentially thousands. **Figure 3** shows that for long timescales, the predictions from the dynamic and algebraic models are virtually identical, with relative differences of the order of 1%.

# **2.3. Simulated Data**

A suitable synthetic input function was chosen based on the following criteria:


It can be noted that a compartmental model for the release/absorption process has been reported (Li et al., 2015). However, this model was not able to capture the peaks present from a single dose. Instead, it was found that datasets similar to

**FIGURE 3 | Comparison of algebraic and dynamic models for short timescales (top) and long timescales (bottom)**. For a given release/absorption profile (left), the plasma concentration predicted by the algebraic and dynamic models is shown on the right. On short timescales, the predictions differ substantially. In this case, the steady-state approximation is not valid, and the "Input estimation" options in **Figure 1** should be used. On longer timescales, the predictions are identical, suggesting that the computationally cheaper algebraic model can be used. This is the "Analytical or numerical steady-state solution" option in **Figure 1**.

the real data by Fineman et al. (Section 2.4) could be created by the following function:

*•* Each of the two peaks at longer timescales could be modeled as an Erlang distribution:

$$u\_i(t) = a\_i \frac{k\_{tr\_l}^{n\_l} \cdot t^{n\_l - 1} \cdot e^{-k\_{tr\_l}t}}{(n\_l - 1)!}, \quad i = \{1, 2\}, \tag{6}$$

where *ktr<sup>i</sup>* is a rate constant that controls the rise and fall time of the peak, while *n<sup>i</sup>* largely controls the time delay.

*•* The initial absorption rate over short timescales was modeled as a bi-exponential function:

$$
\mu\_3(t) = r\_1 e^{-k\_1 t} + r\_2 e^{-k\_2 t}.\tag{7}
$$

*•* The final input function *u*(*t*) was given by:

$$
\mu(t) = \mu\_1(t) + \mu\_2(t) + \mu\_3(t). \tag{8}
$$

This function can be interpreted as the output of a compartmental model, as commonly used in PK modeling. This is explained in greater detail in Section 2 of the Supplementary Material. This methodology of evaluating a non-parametric method using parametric test functions is similar to that presented by Madden et al. (1996), the main difference being that the test function here was specifically designed to mimic the Bydureon profile, with initial fast dynamics followed by multiple peaks at longer timescales.

#### **2.4. Real Data**

The data for this analysis are from a study by Fineman et al. (2011). In the study, 54 subjects in 5 dose groups were given a single dose of exenatide. The plasma concentration was measured at 12 timepoints during the first 48 h, and subsequently once per week for a total duration of 12 weeks.

**TABLE 2 | Parameter values for the generated test data**.


#### **3. RESULTS**

# **3.1. Method Validation Using Simulated Data**

To test the input-estimation method on simulated data, the test input function described in Section 2.3 was used, with the parameter values shown in **Table 2**. The parameters were selected to generate plasma concentrations similar to those observed in real data for the 10 mg dose.

Test data were generated by applying this input function to the dynamical system, and extracting the plasma concentration values at a set of timepoints. For each of the long and short timescales, two sampling schedules were used: one very dense, with 100 equally spaced points between *t* = 0 and the last timepoint (48 h for short timescales, 12 weeks for long), and one sparse using the same timepoints as in the real datasets. Additionally, two noise models were used: no noise and 10% proportional Student's *t*distributed noise with four degrees of freedom. The noise model was chosen to be equal to that assumed for the real datasets (see Section 3.2). This resulted in eight combinations of timescale, sampling schedule and noise levels. The number of MCMC samples was determined by the Raftery–Lewis diagnostics, which showed that all parameters for all datasets could be determined using 10,000 samples.

**Figure 4** shows the plots for the long timescale response. From the figure, it can be seen that the method performs accurately on dense noise-free data. The exception is at the very first timepoint, which has a large contribution from the initial fast peak from *u*3(*t*). The performance was assessed by computing the root mean square error (RMSE) of the input function (**Table 3**). One RMSE value was computed for each trajectory sampled by the MCMC sampler, and the results were averaged. This way, the performance criterion accounts for the variance of the estimated input functions—an estimate where most of the posterior density

#### **TABLE 3 | RMSE (root mean square error) for the test datasets**.


increases the uncertainty of the estimate.

is concentrated close to the true function will yield a lower mean RMSE than an estimate with a large variance.

# **3.2. Input Estimation on Real Data**

We now turn to the analysis of real Bydureon data from Fineman et al. (2011). From now on, the input will be shown as the fraction absorbed rather than the absorption rate, as this is the most common way to present such results. In contrast, when validating against test data, the absorption rate is more useful, since it shows features in the estimated absorption profile more clearly. **Figure 5** shows the data and estimates for the initial 48 h. On this timescale, a dynamic model is necessary. **Figure 6** shows the data and estimates over longer timescales. All plots are produced using a Student's *t*-distribution with four degrees of freedom for the residual model. To assess the sensitivity to the number of degrees of freedom, inference was also performed using six degrees of freedom, which resulted in only marginal differences. The Raftery–Lewis diagnostics showed that 10,000 MCMC samples were enough for all datasets except for the 7 mg dose, which required 25,000 samples for short timescales and 40,000 for long timescales.

Using the estimated profiles, the total absorbed amount of the drug, and hence the bioavailability *F*, was estimated. The amount absorbed during the first 48 h was determined to be insignificant compared to the total amount. Therefore, only the longer timescale was used. **Figure 7** shows kernel density estimates for *F* for each dose group, where clearly *F* appears to be dose dependent. In all cases, the results are lower than the previously reported values of 22–25% (European Medicines Agency, 2011).

# **4. DISCUSSION**

The validation results confirm that, given a known PK model and dense, noise-free data, the absorption profile can be accurately

**FIGURE 6 | Estimation results for the whole 12-week period of the real exenatide data**. Solid lines are mean predictions, while shaded areas are 95% credible intervals. At these timescales, the system is essentially at steady state. Note that the very large plasma concentration value in the first week for the 7.0 mg dose is treated as an outlier by the estimation method. This is a consequence of using an error model based on the Student's *t*-distribution.

estimated (**Figure 4**). The exception is the failure to capture the large initial peak in the long timescale data, which is only visible at the first data point (**Figure 4**, upper right). The method assumes that the data are noisy and considers that data point an outlier. Given that the function does not otherwise show any sharp peaks, this decision seems reasonable. In principle, this peak could be captured by even denser sampling, or by assuming a lower noise level. While such dense data are never obtained from real experiments, it is important to make sure that the method can handle this case before testing it on more realistic datasets.

When noise is added to the measurements, or more realistic sampling schedules are introduced, estimation accuracy decreases. Notably, in this problem, sparsity of sampling has a larger impact than noise for long timescales, while this is not the case for shorter timescales. This may partially be explained by the fact that at short timescales, the plasma concentration at any timepoint depends on the complete absorption rate profile up to that time. In that sense, the measurements provide information about the absorption rate between measurements. In contrast, over long timescales the plasma concentration essentially only provides information about the current absorption rate. Any inference on the absorption between measurements relies solely on the assumption that the function is smooth.

Input estimation of synthetic data could also be useful when designing future experiments. Using such simulations, it is possible to determine how the sampling affects the estimation accuracy, and one can then design a suitable sampling schedule accordingly.

The bioavailabilities obtained are substantially lower than previously reported values. However, the previous values refer to the bioavailability relative to a subcutaneous administration of an immediate-release formulation, making a direct comparison difficult to make. The differences may also partly be explained by uncertainty in the PK model. At least three PK models of exenatide have been published (Gao and Jusko, 2012; Chen et al., 2013; Li et al., 2015). These models are structurally similar but have substantially different parameter values. These parameters clearly influence estimated bioavailability—a drug with higher clearance requires higher absorption rates in order to maintain a specific plasma concentration. Also, in the Gao and Jusko model, the bioavailability of subcutaneous administration was fixed to 1, while the true bioavailability might be lower. In that case, the computed bioavailability would have to be rescaled to obtain the bioavailability relative to subcutaneous administration, in order to compare to previously reported values.

The estimation results demonstrate the extended and complex combined release/absorption kinetics of exenatide from PLG microspheres after subcutaneous injection. This is most clearly seen when the profiles are shown as absorption rate over time (Figures S2 and S3 in Supplementary Material) rather than as fraction absorbed over time (**Figures 5** and **6**). Immediately after injection, during the first couple of hours, a limited amount of exenatide is rapidly absorbed, corresponding to the release of freely available drug. It should be noted that the individual microspheres hydrate after injection and thereby tend to agglomerate to form an amalgam which will affect the release properties (DeYoung et al., 2011). The initial release is followed by an extended-release period of approximately 10 weeks where the polymer matrix of the PLG microspheres is slowly hydrolyzed to smaller fragments. The drug release rate is controlled by the diffusional transport of the drug through the polymer matrix and the erosion of the PLG depot system. The absorption rate over time profiles show two distinct peaks in the absorption rate, one at approximately 2 weeks and another at about 6–7 weeks after which the absorption rate declines until the PLG polymer is fully hydrolyzed and all the drug is released.

The choice of measurement noise model can have a large impact on the resulting estimates. An obvious default choice is to use a Gaussian noise model. However, as this model is log-quadratic, it will assign low probabilities to any candidate input function that disagrees significantly with even a single data point, making this model sensitive to outliers. Forcing the function to agree closely with every data point can drive the estimate of the regularization parameter to very low values. As a result, the method will assign high probabilities even to unrealistic, oscillatory functions, causing the reported uncertainty to be very high. Robust noise models using the Student's *t*-distribution with a small number of degrees of freedom can decrease the sensitivity to outliers.

The methods that have been presented here are very general in that no mechanistic assumptions need to be made about the dissolution or absorption process. Still, some assumptions always have to be made when inferring a continuous-time function from a sparsely sampled and noisy dataset. In the input-estimation approach, these assumptions are encoded in the prior distribution of the stochastic process representing the input function. The prior chosen here, based on the norm of the second derivative, is only one of many possible choices. Previously, penalization of the first derivative (Verotta, 1996) and the use of entropic priors (Hattersley et al., 2008) have been suggested. The form of penalization of derivatives described here can also be viewed as a special case of the application of Gaussian processes, a rich class of probabilistic models for stochastic processes (Rasmussen and Williams, 2006). Depending on the application, other Gaussian processes may be more appropriate.

The number of MCMC samples required can be highly dependent on the data, even when the model remains unchanged. For the 7 mg dose, a much larger number of samples was required than for other doses, for both timescales. It can be noted that the 7 mg dataset contains possible outliers, suggesting that the presence of outliers could be an important factor in determining sampling efficiency. For most doses, a relatively modest number of MCMC samples was sufficient to obtain a high-quality estimate. This increases confidence in the applicability of these kinds of methods to ER-formulation research.

A strength of the methods presented here is that they are applicable even when the PK model is non-linear, but this obviously assumes that a model is available or can be constructed. In contrast, linear systems are completely characterized by their impulse response. The impulse response can be derived from a PK model, but it can also be estimated empirically from data. In that regard, non-linear systems require stronger assumptions to be made.

In **Figure 1**, all non-parametric modeling options can be viewed as special cases of input estimation. In that regard, any method that can handle the input-estimation case should be able to handle the other cases too. It may still be advantageous to use special-purpose methods for other cases. As an example, making a steady-state approximation removes the need to perform expensive numerical integration for each MCMC sample.

Input-estimation methods provide an attractive alternative to building a model of the release and absorption processes.

Building such a model is non-trivial, and highly dependent on factors such as the type of delivery system and its geometry. In contrast, input-estimation methods strive to make minimal assumptions about these processes, requiring only that an intravenous PK model is available, which is independent of the formulation. In addition, these methods typically allow for more rapid analysis compared to building a mechanistic model. In this way, input estimation is a useful complement to model-based approaches.

In summary, this paper presents a framework for addressing input-estimation problems for drug-formulation development. It first gives an overview of what methods are available in various situations (**Figure 1**) and then puts emphasis on the most complicated case—non-parametric methods applied to dynamical systems with non-linear PK. The method presented in Trägårdh et al. (2016) is demonstrated to work robustly for a challenging ER test case with multiple peaks on various time scales, from hours to weeks, subject to the modifications required to cater for ER-formulation scenarios. The method provides estimates of the uncertainty, given the assumptions used in the statistical model. This has not previously been available. This helps increase confidence in the prediction of release and absorption rates. The predicted profiles make it possible to rank candidate formulations, predict the human PK, and establish an ivivc in order to minimize the need for *in vivo* studies. Additionally, we believe this approach has great potential from a practical perspective in supporting dose scheduling and

# **REFERENCES**


regimens to yield optimal responses at the required times. Another possible application is to infer the release profile from data when a parametric model for the absorption of an immediaterelease formulation is available, which could allow for a direct *in vitro* to *in vivo* comparison. This will be the topic of future work.

# **AUTHOR CONTRIBUTIONS**

MT, MC, NE, JP, and PG initiated and planned the work. MT implemented and carried out the simulations, estimation, and analysis; drafted the manuscript, with input from the other authors. DJ performed the structural identifiability analysis. All the authors reviewed and approved the final manuscript.

# **FUNDING**

This work is funded through the Marie Curie FP7 People ITN European Industrial Doctorate (EID) project, IMPACT (Innovative Modelling for Pharmacological Advances through Collaborative Training). Project Number: 316736.

# **SUPPLEMENTARY MATERIAL**

The Supplementary Material for this article can be found online at http://journal.frontiersin.org/article/10.3389/fbioe. 2017.00024/full#supplementary-material.


injectables, including commercially marketed products. *Pharm. Technol.* 2010(Suppl).


**Conflict of Interest Statement:** JP, DJ, and PG are employees of AstraZeneca R&D. MT is working together with Cardiovascular and Metabolic Diseases, Innovative Medicines and Early Development Biotech Unit, AstraZeneca but does not receive any funding from AstraZeneca.

*Copyright © 2017 Trägårdh, Chappell, Palm, Evans, Janzén and Gennemark. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Modeling Disease Progression: Angiotensin II Indirectly Inhibits Nitric Oxide Production via ADMA Accumulation in Spontaneously Hypertensive Rats

Haidong Wang, Hao Jiang, Haochen Liu, Xue Zhang, Guimei Ran, Hua He\* and Xiaoquan Liu\*

*Center of Drug Metabolism and Pharmacokinetics, China Pharmaceutical University, Nanjing, China*

#### Edited by:

*Krasimira Tsaneva-Atanasova, University of Exeter, UK*

#### Reviewed by:

*Radu Iliescu, Grigore T. Popa University of Medicine and Pharmacy, Romania Maarten Koeners, University of Bristol, UK*

#### \*Correspondence:

*Hua He huahe827@163.com Xiaoquan Liu lxq@cpu.edu.cn*

#### Specialty section:

*This article was submitted to Computational Physiology and Medicine, a section of the journal Frontiers in Physiology*

Received: *17 May 2016* Accepted: *03 November 2016* Published: *17 November 2016*

#### Citation:

*Wang H, Jiang H, Liu H, Zhang X, Ran G, He H and Liu X (2016) Modeling Disease Progression: Angiotensin II Indirectly Inhibits Nitric Oxide Production via ADMA Accumulation in Spontaneously Hypertensive Rats. Front. Physiol. 7:555. doi: 10.3389/fphys.2016.00555* Nitric oxide (NO) production impairment is involved in the onset and development of hypertension. Although NO production impairment in spontaneously hypertensive rat (SHR) has been reported in a variety of researches, the time course of this progressive procedure, as well as its relationship with asymmetric dimethylarginine (ADMA) and angiotensin II (Ang II), has not been quantified. The aim of this research is to establish a mechanism-based disease progression model to assess Ang II and ADMA's inhibition of NO production in SHR's disease progression with/without ramipril's intervention. SHR were randomly divided into three groups: one disease group (*n* = 8) and two treatment groups (*n* = 8 for each group): standard treatment group (receiving ramipril 2 mg/kg∗day) and intensive treatment group (receiving ramipril 10 mg/kg∗day). ADMA, Ang II, NO, and SBP were determined weekly. Intensive treatment with ramipril was found to have no further attenuation of plasma NO and ADMA than standard treatment beyond its significantly stronger antihypertensive effects. Four linked turnover models were developed to characterize the profiles of ADMA, Ang II, NO, and SBP during hypertensive disease progression with/without ramipril intervention. Our model described Ang II and ADMA's contribution to NO production impairment and their responses to ramipril treatment throughout the disease progression in SHR. Model simulations suggested that Ang II affected NO production mainly through inhibiting ADMA elimination rather than affecting nitric oxide synthase (NOS) directly.

Keywords: angiotensin II, asymmetric dimethylarginine, disease progression modeling, hypertension, intensive blood-pressure control, nitric oxide, spontaneously hypertensive rat

# INTRODUCTION

Hypertension is a serious chronic disease that causes mortality and morbidity worldwide. A variety of pathophysiological mechanisms are involved in the genesis and development of hypertension (e.g., the activation of renin angiotensin system (RAS), impairment of nitric oxide synthase (NOS), oxidative stress, etc.; Hamza and Dyck, 2014). In hypertensive patients (Schulz et al., 2011) and rats (Landmesser et al., 2002; Mollnau et al., 2002), nitric oxide (NO) molecules are easily diminished

**153**

by angiotensin II (Ang II) mediated over production of reactive oxygen species (ROS), leading to a reduction of NO bioavailability and endothelial dysfunction. Besides, enhanced ROS could also reduce the activity of NOS through oxidizing tetrahydrobiopterin (BH4), the cofactor of NOS, causing NOS impairment and convert NOS to superoxide generators, thus creating a vicious cycle (Baylis, 2012; Roe and Ren, 2012; Su, 2015). Although the end products of NO, nitrite and nitrate, could not reflect NO bioavailability, plasma nitrite, and nitrate together (NOx) has been widely used as an index of NO formation and break down, reflecting NOS activity indirectly (Zeballos et al., 1995; Jungersten et al., 1996). In addition, it is also suggested that urinary NO<sup>x</sup> could not be used as a truly quantitative indicator of NO production (Baylis and Vallance, 1998), since NO might also be excreted through expired air or as other end products. Therefore, plasma NO<sup>x</sup> was selected to be the indicator of NO production in this work. On the other hand, asymmetric dimethyl arginine (ADMA) plays an important role in bridging Ang II and NO. Elevated plasma ADMA level has been widely reported in hypertensive patients (Surdacki et al., 1999) and in SHR (Ghiadoni et al., 2007; Tain et al., 2011). According to previous studies, the major removal of ADMA is provided with dimethylarginine dimethylaminohydrolase (DDAH; Baylis, 2012). Hence, the activity of DDAH would affect the level of ADMA predominantly. As shown in **Figure 1**, the activity of both isoforms of DDAH suffers from an intensive inhibition by Ang II-mediated ROS generation (Palm et al., 2007; Baylis, 2012). Since DDAH provides the majority of ADMA removal (Baylis, 2012), inhibition of DDAH activity would lead to accumulation of ADMA in vivo, which subsequently affects the activity of NOS. Since these markers are closely related to each other and contribute greatly to hypertension, assessing the longitudinal time course of these markers might provide a better understanding of NO production impairment in hypertension disease progression of SHR.

The idea of modeling disease progression has been widely used in chronic diseases, for example diabetes (Cao et al., 2011; Gao et al., 2011), Parkinson's disease (Vu et al., 2012), Alzheimer's disease (Zhou et al., 2013), and hypertension (Zhou et al., 2012). Describing disease progression with responses to treatment in a quantitative way makes predicting clinical outcome events possible, which is especially essential in long-term progressive diseases with poor prognosis (Holford, 2015). On the other hand, model-based evaluation of disease progression provides insight into the mechanism as well as evaluation of drug effect on disease progression (Mould et al., 2007). In a previous study carried out with Zhou et al., a disease progression model was established for capturing the counter-balance relationship between Ang II and Ang-(1-7) in SHR (Zhou et al., 2012). The model satisfyingly described the two peptides' counter regulatory effects on blood pressure. In this paper, we aimed to offer a

better understanding of Ang II and ADMA's contributions to NO production impairment in disease progression of SHR, with our proposed model.

Ramipril, an ACE (angiotensin converting enzyme) inhibitor, has been demonstrated to show cardiovascular protection in SHR beyond antihypertensive action (Linz et al., 1995; Gohlke et al., 1996). In addition, ramipril was also reported to ameliorate endothelial dysfunction, restore NOS impairment, and improve oxidative stress (Linz et al., 2003; Yilmaz et al., 2007). Therefore, ramipril was selected as a tool for modeling and validation, in order to gain more information about the relationship between elevated plasma Ang II level and NO production impairment. Two different doses were set up to investigate whether intensive treatment with ramipril could have more attenuation of NO production impairment dependently of intensified blood pressure control.

#### MATERIALS AND METHODS

#### Animals

Twenty-four 4-week-old male spontaneously hypertensive rats were purchased from Vital River Laboratory Animal Technology Co., Ltd. (Beijing, China). All rats were raised in 12-h light/12 h dark cycle environment and had free access to water and

**Abbreviations:** ADMA, asymmetric dimethylarginine; Ang II, angiotensin II; DDAH, dimethylarginine dimethylaminohydrolase; GK rat, Goto-Kakizaki rat; NO, nitric oxide; NOS, nitric oxide synthase; RAS, renin-angiotensin system; ROS, reactive oxygen species; SBP, systolic blood pressure; SHR, spontaneously hypertensive rat.

food. This study was approved by Ethics Committee for Animal Experimentation of China Pharmaceutical University. All efforts were made to minimize animal suffering.

## Materials

Ramipril was supplied by Kunshan Rotam Reddy Pharmaceutical Co., Ltd. (Kunshan, China). Systolic blood pressure was measured with ALC-NIBP (tail-cuff method) from ALCBIO (Shanghai, China). Iodine [125I] Angiotensin II Radioimmunoassay Kit was obtained from Beijing North Institute of Biological Technology (Beijing, China).

# Experimental Design

All rats were acclimatized for 1 week. From the age of 5 week, 24 SHR were randomly assigned to three groups: one disease group and two ramipril treatment groups: standard treatment group (receiving ramipril 2 mg/kg/day) and intensive treatment group (receiving ramipril 10 mg/kg/day). Rats in two treatment groups were given ramipril by gavage at 9:00 AM every day from 18 to 21-week-age. SBP were measured weekly. Six hundred microliter of blood sample was collected via tail vein once a week with collection time fixed at 2:00 PM, blood samples were anticoagulated with EDTA and centrifuged at 4000 g for 15 min immediately. Plasma samples were aliquoted and stored at −80◦C until analysis. At the age of 21 week, all rats were sacrificed by cervical dislocation.

# Blood Pressure Measurement

The protocol for blood pressure measurement was designed based on the method introduced in the works of Whitesall et al. (2004) and Kubota et al. (2006). During the first week, all rats were acclimated to restraint, tail-cuff inflation, and heating. Rats were placed in plastic restrainers with heating pad remaining at 33∼34◦C. The instrument (ALC-NIBP, ALCBIO; Shanghai, China) automatically takes ten 30-s measurements. The values of systolic blood pressure were recorded when more than five consecutive stable readings were available. The highest and lowest readings were discarded, and the remaining readings were averaged for one data point.

# ADMA, NO, and Ang II Assays

Plasma asymmetric dimethylarginine (ADMA) was measured using an HPLC-MS/MS method introduced by He (He et al., 2013). Plasma nitric oxide (NO) is determined by measuring the stable end products, nitrite and nitrate, which is described in the work of Moshage (Moshage et al., 1995). Plasma angiotensin II (Ang II) was measured using radioimmunoassay with commercial kit obtained from Beijing North Institute of Biological Technology (Beijing, China).

# Disease Progression Model

The general structure of the disease progression model is shown in **Figure 2**. Basically, the model was composed with three components: one defined the natural disease progression in SHR without treatment as disease model; two of which described ameliorated hypertensive disease progression with two different doses of ramipril intervention as treatment model. Four turnovers were applied for describing the dynamics of

plasma Ang II, plasma ADMA, plasma NO, and SBP, which are represented by the following four equations:

$$\frac{dC\_{ANG}}{dt} = k\_{in\\_ANG} \left( 1/(EI\_{RAM} \cdot \exp(DOSE)) \right)$$

$$-k\_{out\\_ANG} \cdot C\_{ANG} \cdot \left( 1 + ES\_{ANG} \cdot C\_{ANG} \right) \tag{1}$$

$$\begin{aligned} \frac{dC\_{ADMA}}{dt} &= k\_{in\\_ADMA} - k\_{out\\_ADMA} \\ &\quad \cdot C\_{ADMA} \left(1 - IA\_{ANG} \left[m\right] \cdot C\_{ANG}\right) \\ dC\_{NO} &\quad \cdot \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \qquad \cdot \end{aligned}$$

$$\frac{u\_{\rm NO}}{dt} = k\_{\rm in\\_NO} \left(1 - \text{INADMA} \left[n\right] \cdot \text{C}\_{\rm ADMA} - \text{INANG} \left[o\right]\right)$$

$$\cdot \text{C}\_{\rm ANG} + \text{ES}\_{\rm SBP} \cdot \text{C}\_{\rm SBP} \left(-k\_{\rm out\\_NO} \cdot \text{C}\_{\rm NO}\right) \tag{3}$$

$$\frac{d\text{SBP}}{dt} = k\_{in\\_SBP} \left(1 + E\text{S}\_{ANG} \left[p\right] \cdot \text{C}\_{ANG}\right) - k\_{out\\_SBP}$$

$$\cdot \left(1 + E\text{S}\_{NO} \left[q\right]\right) \cdot \text{SBP} \tag{4}$$

In which, CANG, CADMA, CNO, and SBP indicate plasma Ang II, ADMA, NO concentration, and SBP level, respectively. In this model, plasma Ang II was the marker that triggers the cascade of downstream reactions. Plasma Ang II is assumed to be formed at a zero-order constant rate (kin\_ANG) and degraded by the first-order constant rate (kout\_ANG). The degradation of Ang II is stimulated through a negative regulatory feedback loop (ESANG; Zhou et al., 2012), which is characterized with a linear model represented with ESANG. In two treatment groups, plasma Ang II level is subjected to an inhibitory effect from ramipril (EIRAMI), EIRAMI is a drug-specific parameter. The dose of ramipril was indicated by parameter DOSE. DOSE was fixed at 0 when model was fitted in disease group, while at 2 or 10 in two treatment groups, respectively. ADMA is synthesized by Protein Arginine Methyltransferase (PRMT) and eliminated mainly through dimethylarginine dimethylaminohydrolase (DDAH) hydrolysis. Prior to occurrence of disease, plasma ADMA level

effects.

#### TABLE 1 | Estimates and definition of parameters of the progression model in three groups.


remains at a relatively stable situation, which is represented by a zero-order constant rate (kin\_ADMA) for production and a first-order constant rate (kout\_ADMA) for elimination. According to the previously mentioned mechanism, elimination of ADMA is restrained with Ang II (IAANG[m]). IAANG[m] represents the inhibitory effect that Ang II exerts on DDAH, where m indicates the number of transit compartments that is applied to simulate the inhibition of ADMA elimination by Ang II. A zeroorder constant rate kin\_NO and a first-order constant rate kout\_NO are used to describe the generation and degradation of plasma NO, respectively. In this system, activity of NOS is stimulated by elevated blood pressure according to previous research (Vaziri et al., 1998), which is simulated with a linear effect represented by parameter ESSBP. The suppression of NOS activity by plasma ADMA and plasma Ang II are described with two transduction procedures INADMA[n] and INANG[o], respectively. n and o indicate the number of the transit compartments that is required for describing the effects. In this model, systolic blood pressure is assumed to be input and output in zero-order rate (kin\_SBP) and first-order rate (kout\_SBP), respectively, where Ang II contributes to the climb of systolic blood pressure by causing vasoconstriction, while NO eases systolic blood pressure through vasodilation effect, which are represented with two separate series of transit compartments ESANG[p] and ESNO[q].

#### Ang II Dynamics

At the beginning of the disease progression, plasma Ang II level is described with the equation below:

$$C\_{ANG}\left(0\right) = \frac{k\_{in\\_ANG}}{k\_{out\\_ANG}}\tag{5}$$

In this model, Ang II accumulation is assumed to be spontaneous and no other markers in this system would affect the procedure of accumulation.

#### ADMA Dynamics

Plasma ADMA was assumed to be maintained at a steady level before the initiation of the disease development, which is reflected with the equation below:

$$C\_{ADMA}\ (0) = \frac{k\_{in\\_ADMA}}{k\_{out\\_ADMA}}\tag{6}$$

The turnover of plasma ADMA was mainly affected by the activity of DDAH, which was inhibited by Ang II-induced

ROS due to its high sensitivity to oxidative environment. This inhibitory effect was simulated by a series of transit compartments, which are represented by the following equations:

$$\frac{dIA\_{ANG}(0)}{dt} = OX\_-DDAH - kt\_1 \cdot IA\_{ANG}(0)$$

$$\frac{dIA\_{ANG}(1)}{dt} = kt\_1 \cdot IA\_{ANG}(0) - kt\_1 \cdot IA\_{ANG}(1)$$

$$\dots$$

$$\frac{dIA\_{ANG}(m)}{dt} = kt\_1 \cdot IA\_{ANG}(m-1) - kt\_1 \cdot IA\_{ANG}(m) \tag{7}$$

This transduction effect was assumed to be initiated by a disease factor OX\_DDAH, where m indicates the number of transit compartments that were applied to describe the inhibition of Ang II-mediated ROS on the elimination of ADMA; each transit compartment was connected by a turnover rate constant kt1. Different transit compartment numbers were evaluated to find a number that sufficiently captured the stimulation.

#### NO Dynamics

The initial plasma NO level is represented with the following equation:

$$\mathcal{C}\_{NO}(0) = \frac{k\_{\text{in\\_NO}}}{k\_{\text{out\\_NO}}} \tag{8}$$

During disease progression, the turnover of NO was mediated by other three markers in this system. ADMA inhibits the synthesis of NO directly by competitive binding to NOS. A series of transit compartments was applied to describe this effect:

$$\frac{dIN\_{ADMA}(0)}{dt} = IN\_-NOS - kt\_2 \cdot IN\_{ADMA}(0)$$

$$\frac{dIN\_{ADMA}(1)}{dt} = kt\_2 \cdot IN\_{ADMA}(0) - kt\_2 \cdot IN\_{ADMA}(1)$$

$$\dots$$

$$\frac{dIN\_{ADMA}(n)}{dt} = kt\_2 \cdot IN\_{ADMA}(n-1) - kt\_2 \cdot IN\_{ADMA}(n) \tag{9}$$

This transduction effect was assumed to be initiated by a disease factor IN\_NOS, where n indicates the number of transit compartments that were applied to simulate the inhibitory effect of ADMA on the generation of NO; each transit compartment was connected by a turnover rate constant kt2. Different transit compartment numbers were evaluated to find a number that sufficiently captured the inhibitory effect from ADMA on NOS.

Ang II-induced generation of ROS would sharply cut down the activity of NOS. To mimic this effect, a series of transit

(open circles). The solid lines are the predicted values generated based on the original dataset.

compartments were utilized:

$$\frac{d\text{IN}\_{\text{ANG}}(\mathbf{0})}{dt} = \text{OX\\_NOS} - kt\_3 \cdot \text{IN}\_{\text{ANG}}(\mathbf{0})$$

$$\frac{d\text{IN}\_{\text{ANG}}(\mathbf{1})}{dt} = kt\_3 \cdot \text{IN}\_{\text{ANG}}(\mathbf{0}) - kt\_3 \cdot \text{IN}\_{\text{ANG}}(\mathbf{1})$$

$$\dots$$

$$\frac{d\text{IN}\_{\text{ANG}}(o)}{dt} = kt\_3 \cdot \text{IN}\_{\text{ANG}}(o-1) - kt\_3 \cdot \text{IN}\_{\text{ANG}}(o) \quad \text{(10)}$$

This transduction effect was assumed to be initiated by a disease factor OX\_NOS, where INANG (o) indicates the number of transit compartments that were applied to describe the inhibition from Ang II-induced ROS on the generation of NO; each transit compartment was connected by a turnover rate constant kt3. Different transit compartment numbers were evaluated to find a number that sufficiently captured the stimulation.

#### SBP Dynamics

SBP continues to climb during the growth of SHR till the age of 16–17 weeks. At the age of 5 weeks, the SBP of SHR is described by the equation below:

$$C\_{SBP}(0) = \frac{k\_{in\\_SBP}}{k\_{out\\_SBP}}\tag{11}$$

In the system we investigated, SBP was adjusted by Ang II and NO in a manner of counterbalance. Ang II exerted its pressor effect by causing vasoconstriction through binding to AT1 receptors. On the contrary, the pressor effect was being counterbalanced by vasodilator NO. This effect could be described through two series of transit compartments, which are represented by the following equations, respectively.

The first group of equations is proposed to represent vasopressor effect:

$$\frac{dES\_{ANG}(0)}{dt} = \,^\prime \text{CON} - kt\_4 \cdot ES\_{ANG}(0)$$

$$\frac{dES\_{ANG}(1)}{dt} = kt\_4 \cdot ES\_{ANG}(0) - kt\_4 \cdot ES\_{ANG}(1)$$

$$\dots$$

$$\frac{dES\_{ANG}(p)}{dt} = kt\_4 \cdot ES\_{ANG}(p-1) - kt\_4 \cdot ES\_{ANG}(p) \tag{12}$$

VCON was assumed to be the initiative factor of vasoconstriction, p indicates the number of transit compartments that were applied to describe the vasoconstriction effect from Ang II; each transit compartment was connected by a turnover rate constant kt4. Different transit compartment numbers were evaluated to find a number that sufficiently captured the stimulation.

The vasodilation effect of NO was represented with the equations below:

$$\begin{aligned} \frac{dES\_{NO}(0)}{dt} &= VDL - kt\_5 \cdot ES\_{NO}(0) \\ \frac{dES\_{NO}(1)}{dt} &= kt\_5 \cdot ES\_{NO}(0) - kt\_5 \cdot ES\_{NO}(1) \\ \dots \\ \frac{dES\_{NO}(q)}{dt} &= kt\_5 \cdot ES\_{NO}(q-1) - kt\_5 \cdot ES\_{NO}(q) \end{aligned} \tag{13}$$

The initiation of the vasodilation from NO was assumed to be initiated by VDIL, q indicates the number of transit compartments that were applied to describe the vasodilation effect of NO; each transit compartment was connected by a turnover rate constant kt5. Different transit compartment numbers were evaluated to find a number that sufficiently captured the stimulation.

#### Modeling and Simulation

The hypertensive disease progression combined with ramipril's effect was modeled using Phoenix 6.4 (CERTARA). The data from total 24 rats in the disease group and treatment groups were pooled together for baseline analysis in the initial 13 weeks. In the following 4 weeks, 8 rats in disease group were contributed continually for baseline modeling, and 8 rats in standard treatment group were used to estimate the drug effect parameter EIRAMI. Eight rats in intensive treatment group were used to validate the drug effect modeling. The validation was performed by visual predictive check (VPC). Estimates of parameters from the standard treatment group were used for the performance of VPC. The baseline parameters were accordingly fixed in drug effect estimation, assuming the baseline in treatment groups did not change evidently from the disease group. The dose of ramipril was represented with the parameter DOSE. DOSE was fixed at 0 in the disease group, while 2 and 10 in two treatment groups, respectively. Model evaluation was performed using nonparametric bootstrap analysis, introduced in previous research

(Chen et al., 2009). Random draws of individual data from the original dataset was repeated 100 times. The stability of the final model was evaluated by comparing the model parameter estimates from the average values of new datasets with that obtained from the fit of the average values of original dataset.

Model simulations were conducted using mean estimates obtained from the model to observe the effects on plasma NO turnover throughout the disease progression from the three disease factors: OX\_DDAH, IN\_NOS, and OX\_NOS. OX\_DDAH, IN\_NOS, or OX\_NOS was fixed at zero in each simulation respectively, which simulates the blockade of such disease procedure.

# RESULTS

# Dynamics of SBP, Plasma Ang II, ADMA, and NO

The time course of plasma angiotensin II (Ang II), asymmetric dimethylarginine (ADMA), nitric oxide (NO) and systolic blood pressure (SBP) variations during 5 to 21-week-age are shown in **Figure 3**. In disease group, the change of SBP could be divided into three stages. At the first stage, SBP climbed quickly from under 140–180 mmHg during the first 4–5 weeks. SBP then grew to around 200 mmHg at the age of 17 weeks in a more slowly pace and remained at this level afterwards. Ramipril showed significant antihypertensive effect from the data of two treatment groups with a good dose-effect relationship. After 4 weeks' therapy, SBP reached 146.8 ± 13.44 mmHg in standard treatment group and 127.4 ± 11.9 mmHg in intensive treatment group, respectively. Time course of plasma Ang II level in all three groups of SHR went accordingly with the variation of SBP. Higher dose of ramipril also showed a stronger effect on plasma Ang II. Plasma ADMA, and NO did not vary sharply compared to SBP and plasma Ang II. Nevertheless, the accumulation of ADMA was observed in disease group. Plasma NO was maintained at a relatively higher level before a drop at the age of 12 weeks in all groups. Ramipril increased plasma NO during the 4-week therapy. It is worth mentioning that administration of 2 mg/kg ramipril successfully dragged SBP to 146.8 ± 13.44 mmHg, reversed the accumulation of plasma ADMA and increased plasma NO level, compared to disease group. However, a higher dose (10 mg/kg) of rampril with better antihypertensive effect (dragging SBP to 127.4 ± 11.9 mmHg) failed to exert further attenuation of plasma ADMA and NO.

# Disease Progression Model Analysis

The profiles of SBP, plasma Ang II, ADMA, and NO throughout disease progression were reasonably fitted with our proposed

model. The predicted values of SBP, plasma Ang II, ADMA, and NO in disease group (**Figure 4**) and standard treatment group (**Figure 5**) were fitted with the mean values from the original data set. Parameter estimates and the optimized transit compartment numbers are listed in **Table 1**. Parameter estimates obtained from the fit of original mean data were within the mean ± SD estimates of the bootstrap replicates. The result of VPC for intensive treatment group has been demonstrated in **Figure 6**. The observed values are well within the range between 5 and 95 percentiles of 1000 simulated values. The conditional weighted residuals (CWRES) were randomly and homogenously distributed around 0 (**Figures 7**, **8**). The results suggested a reasonable precision in the parameter estimates for the final model.

### Model Simulation

Based on mean parameter estimates of the model, simulations were performed to predict the fraction of three disease factors contributed to the turnover of plasma NO (**Figure 9**). Each of the three disease factors (OX\_DDAH, OX\_NOS, IN\_NOS) was fixed at 0 for each simulation, assuming the blockade of such disease procedure initiated by the corresponding disease factor. The locations of simulated blockades are shown in **Figure 1**. NO production was affected by three disease factors with different extensions (**Figure 9**). The blockade of IN\_NOS showed the most significant improvement in NO production (**Figure 9**, blue line). While blocking Ang II-mediated inhibition of dimethylarginine dimethylaminohydrolase (DDAH) activity (OX\_DDAH) showed much milder effects on NO production

(**Figure 9**, purple line). Compared with them, blocking the direct effect of Ang II-mediated oxidative stress on NOS (OX\_NOS) has the minimum improvement on NO production (**Figure 9**, red line). Simulated data indicate that Ang II inhibited NO production mainly through affecting ADMA hydrolysis rather than suppressing NOS activity directly.

# DISCUSSION

Nitric oxide (NO) is regarded as the controller of vascular tone together with vasoconstriction factors, controlling blood pressure. Besides, the abnormality of NO production will result in endothelial dysfunction, leading to various cardiovascular pathologies, like hypertension and atherosclerosis (Bryan, 2006; Rochette et al., 2013). Therefore, modeling and simulating the progression of NO production impairment could provide better understanding of the disease progression in hypertension. On the other hand, oxidative stress has been considered to cause endothelial dysfunction in hypertensive subjects (Schulz et al., 2011). Over produced ROS induced by angiotensin II (Ang II) would suppress the activity of NO synthase (NOS) through oxidizing tetrahydrobiopterin (BH4). Besides, dimethylarginine dimethylaminohydrolase (DDAH), the metabolic enzyme of asymmetric dimethylarginine (ADMA), is also sensitive to free radicals (Palm et al., 2007). An enhancement of oxidative stress in SHR has been demonstrated in a variety of researches. In the work of Simao et al, renal H2O2, NADPH oxidase expression as well as urinary thiobarbituric acid reactive substances (TBARS) was found to increase in SHR (Simao et al., 2011). Increased kidney TBARS was also found in SHR, indicating enhanced oxidative stress (Chandran et al., 2014). Therefore, we incorporated two series of transit compartments in our model,

simulating Ang II-mediated oxidative effects on DDAH and NOS activities, triggered by two disease factors (OX\_DDAH and OX\_NOS), respectively.

In the work of ND Vaziri et al. (1998), elevated plasma and urinary NO<sup>x</sup> was observed in 12-week-age SHR, as well as aorta NOS activity. However, in pre-hypertensive SHR (3 week-old), plasma NO<sup>x</sup> was not significantly elevated compared with Wistar Kyoto (WKY) rats. This might indicate that the elevation of NO production in young SHR is progressive. It is also mentioned that increased NO production during the early stage of hypertension in SHR could not be maintained till the advanced phase of the disease. With progressive endothelial dysfunction, NO production may fall, leading to true NO deficiency in animals with advanced hypertension. In other researches, impaired NO production was also reported preceding the onset of hypertension in SHR (Mokuno et al., 2001). Our results supported the finding of ND Vaziri. In our work, NO production remained at a higher level before the age of 12 weeks in SHR (**Figure 3C**), indicating elevated NOS activity. Besides, the drop of NO production initiated at the 12th week (**Figure 3C**) might indicate the start of NOS impairment. Our model captured this procedure (**Figure 4C**). Model simulations also revealed that three disease factors contribute differently in NO production during disease progression (**Figure 9**). Firstly, blocking IN\_NOS showed the most significant improvement in NO production, indicating disease factor IN\_NOS (inhibition of NOS activity by ADMA) was the major contributor to the inhibition of NO production in disease progression (**Figure 9**, blue line). Secondly, blocking Ang II-mediated inhibition of DDAH activity (OX\_DDAH) showed much milder effects on NO production (**Figure 9**, purple line). According to the mechanism (**Figure 1**), blockade of OX\_DDAH could reduce the accumulation of ADMA, decreasing ADMA level. While blocking IN\_NOS could directly intercept ADMA's inhibition on NO production. Since direct interception of ADMA can obviously provide better improvement on NO production than reducing ADMA accumulation, the results of simulation is reasonable. Finally, blocking the direct effect of Ang IImediated oxidative stress on NOS (OX\_NOS) has the minimum improvement on NO production (**Figure 9**, red line). Compared with the simulated values of blocking OX\_DDAH, Ang IImediated oxidative stress might affect NO production mainly through decreasing ADMA elimination instead of inhibiting NOS activity directly. Blocking RAS was proved to protect renal and vascular NOS, increasing NO production (Vaziri et al., 2002; Zhou et al., 2008). In our work, ramipril increased the level of plasma NO in rats from both treatment groups. However, this effect was not enhanced with the increase of dose, which was also reported by Pechánová (2007) and Christian Delles (Delles et al., 2002). This might be explained that blocking RAS could only suppress Ang II-mediated oxidative stress, but other sources of ROS (e.g., xanthine oxidase, mitochondria and cyclooxygenase) might not be attenuated by this action. As a result, the activities of DDAH and NOS might not be better protected from ROS despite a stronger antihypertensive action by a higher dose of ramipril.

# CONCLUSION

Our work revealed that intensive blood pressure control with ramipril did not bring more benefits to attenuating plasma ADMA and NO in SHR. The proposed model assessed Ang II and ADMA's contribution to NO production impairment in SHR's disease progression. The simulations suggested that Ang II inhibited NO production mainly through affecting ADMA elimination rather than directly affecting NOS activity in hypertension progression of SHR.

# AUTHOR CONTRIBUTIONS

Research design: HW, HH, and XL. Experiment conduction: HW, HJ, XZ, and GR. Modeling and data analysis: HW, HL, and XL. Wrote or contributed to the writing of the manuscript: HW, HH, and XL.

# ACKNOWLEDGMENTS

We thank Kunshan Rotam Reddy Pharmaceutical Co., Ltd (Kunshan, China) for providing us with ramipril API. We are also grateful to medical clinical laboratory of Zhongda Hospital Southeast University for providing us with instrument and laboratory for the conduction of radioimmunoassay. This work was funded by the National Natural Science Foundation of China (No. 81273588 and No. 81473274).

Frontiers in Physiology | www.frontiersin.org

#### REFERENCES


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2016 Wang, Jiang, Liu, Zhang, Ran, He and Liu. This is an openaccess article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Development of a Patient-Specific Multi-Scale Model to Understand Atherosclerosis and Calcification Locations: Comparison with In vivo Data in an Aortic Dissection

Mona Alimohammadi <sup>1</sup> \* † , Cesar Pichardo-Almarza1 †, Obiekezie Agu<sup>2</sup> and Vanessa Díaz-Zuccarini <sup>1</sup>

<sup>1</sup> Mechanical Engineering, University College London, London UK, <sup>2</sup> Vascular Unit, University College Hospital, London, UK

#### Edited by:

Zbigniew R. Struzik, The University of Tokyo, Japan

#### Reviewed by:

Umberto Morbiducci, Politecnico di Torino, Italy Miguel Angel Martinez, University of Zaragoza, Spain

#### \*Correspondence:

Mona Alimohammadi mona.alimohammadi.10@ucl.ac.uk

† These authors have contributed equally to this work.

#### Specialty section:

This article was submitted to Computational Physiology and Medicine, a section of the journal Frontiers in Physiology

> Received: 01 April 2016 Accepted: 03 June 2016 Published: 21 June 2016

#### Citation:

Alimohammadi M, Pichardo-Almarza C, Agu O and Díaz-Zuccarini V (2016) Development of a Patient-Specific Multi-Scale Model to Understand Atherosclerosis and Calcification Locations: Comparison with In vivo Data in an Aortic Dissection. Front. Physiol. 7:238. doi: 10.3389/fphys.2016.00238 Vascular calcification results in stiffening of the aorta and is associated with hypertension and atherosclerosis. Atherogenesis is a complex, multifactorial, and systemic process; the result of a number of factors, each operating simultaneously at several spatial and temporal scales. The ability to predict sites of atherogenesis would be of great use to clinicians in order to improve diagnostic and treatment planning. In this paper, we present a mathematical model as a tool to understand why atherosclerotic plaque and calcifications occur in specific locations. This model is then used to analyze vascular calcification and atherosclerotic areas in an aortic dissection patient using a mechanistic, multi-scale modeling approach, coupling patient-specific, fluid-structure interaction simulations with a model of endothelial mechanotransduction. A number of hemodynamic factors based on state-of-the-art literature are used as inputs to the endothelial permeability model, in order to investigate plaque and calcification distributions, which are compared with clinical imaging data. A significantly improved correlation between elevated hydraulic conductivity or volume flux and the presence of calcification and plaques was achieved by using a shear index comprising both mean and oscillatory shear components (HOLMES) and a non-Newtonian viscosity model as inputs, as compared to widely used hemodynamic indicators. The proposed approach shows promise as a predictive tool. The improvements obtained using the combined biomechanical/biochemical modeling approach highlight the benefits of mechanistic modeling as a powerful tool to understand complex phenomena and provides insight into the relative importance of key hemodynamic parameters.

Keywords: mathematical modeling, multiscale, atherosclerosis, patient-specific, aortic dissection, in vivo data

# INTRODUCTION

Atherogenesis is a complex, multifactorial and systemic process; the result of a number of factors, each operating simultaneously at several spatial and temporal scales. The bewildering molecular and cellular complexity is well-described in Lusis' classical review more than a decade ago (Lusis, 2000), which highlights a plethora of biological mechanisms and gene associations,

**166**

revealing an incredible etiological complexity. In addition to the biological components of the disease, atherosclerosis is also known to be related to mechanical stimuli on the vessel wall and hemodynamic parameters (Suo et al., 2006). Experimental evidence indicates that hemodynamic stimuli influence mechanotransduction and affect permeability (Davies, 1995). Increased permeability can lead to penetration and accumulation of lipoproteins (e.g., Low Density Lipoproteins– LDL) in the arterial wall and thus initiation of atherosclerosis. In recent years, much research has been conducted in order to draw correlations between hemodynamics and the atherogenic process (Peiffer et al., 2013a; Alimohammadi et al., 2015a). Certain hemodynamic parameters have been identified as key; these include flow distribution, pressure and wall shear stress (WSS) indices. Nevertheless, given the incredible complexity of the atherogenesis process, these hemodynamic analyses, on their own, have been inconclusive (Peiffer et al., 2013a) and a clear metric for plaque location remains elusive. Strong correlations between atherosclerotic disease and vascular calcification have been well-documented in the literature, including large cohort studies (Sangiorgi et al., 1998). Although the underlying molecular cause of calcification is unknown (Lanzer et al., 2014), the severity and extent of mineralization in calcification reflect atherosclerotic plaque burden (Demer and Tintut, 2008). Given the central role of inflammation in atherogenesis, an interesting possibility is that vascular mineral itself may initiate, promote, or perpetuate atherosclerosis by inducing inflammatory cytokines in monocytes that encounter and ingest hydroxyapatite crystals (Nadra, 2005).

Modeling and simulation have been used in a large number of studies in order to improve understanding of the role of hemodynamic variables in plaque formation. Although this work is mathematically elegant and can provide detailed insight into hemodynamics, it is disconnected from molecular research. Similarly, biomedical researchers often reduce the complexity of investigations of cardiovascular disease into manageable parts, for example, working on cell-lines or employing large-scale genome wide association studies (GWAS) to identify SNPs related to CVD (Tegner et al., 2006). However, such statistical genetic models have no mechanistic basis, and it is significant that Lusis highlighted the revival of functional studies in a relatively recent review (Lusis, 2012).

In this paper, a clear application of mathematics for healthcare will be made by unifying multi-mechanistic factors in the prediction of atherosclerosis location. The location of atherosclerotic plaque and vascular calcification will be investigated using a patient-specific biomechanical model of an aortic dissection (AD). From a physiological point of view, AD is a life threatening condition in which a tear forms in the wall of the aortic wall and blood splits the media layer, forming two lumina: the true lumen (TL) and false lumen (FL; Braverman, 2010). Blood flows from the TL into the FL via a primary tear and, in communicating dissections, returns to the TL via one or multiple tears downstream. The section of intima and media that separates the two lumina, called the intimal flap (IF), often stiffens over time due to fibrosis (Criado, 2011). Common comorbidities in patients with AD are atherosclerosis (Coady et al., 1999; Tsai et al., 2006) and inner wall calcification (de Jong et al., 2014). Additionally, patients suffering from AD usually have an elevated pulse pressure, which would likely be further increased in the presence of calcification, due to the reduction of vessel elastance (Demer and Tintut, 2008). The patient data used for this research showed an AD with significant atherosclerosis and calcification regions, consistent with the condition. Detailed hemodynamic characterization was achieved by using patient-specific dynamic boundary conditions representing the downstream vasculature, based on in vivo measurements collected for the same patient, treated in University College Hospital (UCH; Alimohammadi et al., 2014). Although complex flow simulations for this patient have been published previously, the work presented here, including the quantification and analysis of plaque and calcification areas as well as the multi-scale framework used in this context, is completely new.

The postulate of this research is that multiscale modeling and simulation can pave the way to study multi-mechanistic factors to explain disease in a cohesive modeling framework, which can integrate key markers at different biological scales and can provide insight into endothelial mechanotransduction, as well as potential predictive power in patient-specific analyses, compared to purely hemodynamic, biomechanical or biochemical approaches. We will use a virtual "followup" approach, combining a fluid-structure interaction (FSI) simulation model of a dissected aorta with a model of plaque formation, using a number of somewhat disparate indicators available in the literature, which will be described below. The patient-specific simulation results are then coupled to an endothelial permeability model following the three-pore approach (Olgac et al., 2008a), also recently used by Kim and Giddens (2015).

This paper is organized as follows: Section Methods presents the methods and simulation details. Results are presented in Section Results, including a clear, interpretable metric for atherogenic potential, which shows a better performance when compared to others based on "established" descriptors in the current literature. Simulation results of the model will be presented in and compared to atherosclerotic plaques and calcifications indicated in the original CT scans. The discussion, limitations and conclusions of this work will be presented in Section Discussion.

# METHODS

# Details of the Simulation

The specifics of the FSI simulation used for the present study are presented briefly in this section and are described in detail in Alimohammadi et al. (2015b). The patient gave informed oral consent and ethical approval was given by the National Research Ethics Service, UK, REC reference: 13/EM/0143. The fluid domain was reconstructed from CT scans of a female patient, starting at the ascending arch and extending down to the thoracic aorta, upstream of the iliac bifurcation. The supraaortic branches were included in the model, but the visceral branches were not clearly resolvable from the CT data and so were omitted. The vessel wall was modeled by extruding the outer wall of the fluid geometry uniformly by 2.5 mm, based on reports of thickened aortic walls in hypertension (Malayeri et al., 2008), which occurs in ∼70% of AD patients (Hagan et al., 2000; Khan and Nair, 2002). The intimal flap (IF), separating the TL and FL, was created by filling the gap between the two lumina and was 2.45 ± 0.34 mm (median ± median absolute deviation). Fluid and solid meshes were generated in ANSYS ICEM-CFD, and contained ∼230,000 (with 7 prismatic layers at the wall) and 50,000 elements respectively.

Simulations were carried out using ANSYS mechanical and CFX. The vessel wall was modeled using the isotropic hyperelastic model of Raghavan and Vorp (2000). This model is comparable to a linear elastic model with a Young's modulus of 1 MPa, but displays a small amount of strain-stiffening. An external pressure of 52.5 mmHg (diastolic pressure in the descending aorta) was applied and at each of the outlets of the 3D domain, the solid geometry was restricted to planar motion about the center-point of the lumen, enabling expansion of the vessels.

At the inlet, a flow wave from another study of a patient with type-B aortic dissection was used (Karmonik et al., 2008), as such data was not available for the present patient. At the fluid outlets, three element Windkessel models were used, with parameters tuned to invasive pressure measurements on the patient, using an iterative technique described in a previous study (Alimohammadi et al., 2014). Blood was modeled as an incompressible fluid of density 1056 kg/m<sup>3</sup> . Turbulence was modeled using the hybrid k-ε, k-ω shear stress transport (SST) turbulence model, with a 1% turbulence level at the inlet. In order to account for the shear-thinning properties of blood, the Carreau-Yasuda (CY) viscosity model was utilized with the parameters reported by Gijsen et al. (1999). The CY viscosity model includes infinite and zero shear viscosities, a characteristic time constant and two exponents without direct physical meaning to describe the viscosity-shear response of blood. The parameters of Gijsen et al. (1999) were fitted to a blood analog, which showed similar characteristics to blood samples. The viscosity predicted by this model is ∼3.5 mPa s (a commonly used Newtonian viscosity) at 100/s, and decreases for higher shear rates to around 2.5 mPa s at 1000/s. At shear rates below 100/s, the viscosity increases, exceeding 10 mPa s for shear rates below 2/s.

Simulations were run with a time step of 5 ms, a periodic solution was achieved after two cycles and the third cycle was extracted for further analysis. In a previous study (Alimohammadi et al., 2015b), it was shown that the low shear rates in the slow flow regions of the distal and proximal FL led to significantly increased viscosity.

#### Shear Stress Indices (SSI)

In the permeability calculations described in Section Endothelial Permeability Model, one of the inputs is an index indicative of the shear stress condition that leads to increased permeability. The time-average wall shear stress (TAWSS) and oscillatory shear index (OSI; Ku et al., 1985) are two commonly used indices that are considered important for plaque formation. TAWSS describes the average magnitude of the shear stress and the OSI gives an indication of the directionality of the shear stress, yielding 0 for uniaxial flows and 0.5 when there is no preferential direction. Typically, in permeability models for atherogenesis, TAWSS is used as the shear index related to permeability.

A number of studies (Malek et al., 1999; Xiang et al., 2010; Chiu and Chien, 2011; Meng et al., 2014) have shown that regions with low average shear stress combined with highly oscillatory shear stress have increased endothelial permeability along with other pathological responses. More recently, Sáez et al. (2015) presented the 3D remodeling of endothelial cells as the combined effect of OSI and TAWSS in a computational framework, which fitted experimental works presented before in in vitro studies.

In light of the indications for increased permeability in low, oscillatory regions, we propose an index, HOLMES (Highly Oscillatory, Low MagnitudE Shear), given by:

$$\text{HOLMES} = \text{TAWSS}(0.5-\text{OSI})\tag{1}$$

This parameter is equivalent to half the reciprocal of relative residence time (RRT) which was previously identified as a potential index for combining these two characteristics (Himburg, 2004). The HOLMES indicator can be understood as a modified TAWSS, with the (0.5 − OSI) term further reducing the index in regions where the wall shear stress is both low in magnitude and oscillatory in nature. Additionally, HOLMES provides a conceptually alternative explanation, offering a linear (rather than reciprocal) index, proportional to WSS that intuitively corresponds to the observed effects of shear characteristics on endothelial permeability.

In the present study, we compare the efficacy of using the most-widely used hemodynamic/shear stress index (SSI) for atheroprone regions, i.e., TAWSS, with HOLMES, separate and in a mechanistic model, as shown in the sections below.

# Endothelial Permeability Model

The endothelial permeability model proposed here is based on previous work describing the early stages of atherosclerosis, using a transport model of low density lipoprotein (LDL) from the artery lumen into the arterial wall, taking into account the effects of mechanical stimuli exerted by the blood flow on the endothelial cell layer and its pathways of volume and solute flux; see Díaz-Zuccarini et al. (2014) for more details. An excellent and recent analysis and use of this model along the same lines using time-average wall shear stress (TAWSS) has been recently published by Kim and Giddens (2015).

The endothelial layer is described with a three-pore modeling approach considering the contributions of the vesicular pathway, normal junctions, and leaky junctions. The fraction of leaky junctions is calculated as a function of the mechanical stimuli and is used in conjunction with the pore theory to determine the transport properties of this pathway.

The LDL transport equations are decomposed using three main penetration pathways: leaky junctions, normal junctions and vesicular pathways; so the bulk of volume flux (Jv) through the endothelial membrane is given by:

$$J\_{\nu} = J\_{\nu, \text{lj}} + J\_{\nu, n\text{j}} \tag{2}$$

where Jv,lj is the flux through leaky junctions and Jv,nj is the flux through normal junctions.

The volumetric flux through leaky junctions (Jv,lj) is calculated using a modified version of the Kedem-Ketchalsky equations for membrane transport:

$$J\_{\nu,lj} = L\_{p,lj} \left(\Delta p\_{end} - \sigma\_d \Delta \Pi\right) \tag{3}$$

where Lp,lj is the hydraulic conductivity, ∆pend is the pressure difference through the endothelium, σ<sup>d</sup> is the osmotic reflection coefficient and ∆Π is the osmotic pressure. The value of ∆pend is estimated by subtracting the externally applied pressure from either the average pressure throughout the domain (uniform pressure gradient) or the spatially varying time average pressure at each wall location (time-averaged pressure gradient).

According to the three pores theory, solute flux (LDL flux in this case) only occurs through endothelial leaky cell junctions and vesicles:

$$J\_s = J\_{s, \text{lj}} + J\_{s, \nu} \tag{4}$$

Assuming that the solute flux through the vesicular pathway (Js,v) is 10% of the solute flux through the leaky junction pathway (Js,lj) (Olgac et al., 2008b).

Finally, in the interest of brevity, Equation (5) shows a general form of the function used to calculate Js,lj which is proportional to the magnitude of Jv,lj (more details about the calculation of Js,lj are shown elsewhere (Díaz-Zuccarini et al., 2014; Alimohammadi et al., 2015a):

$$J\_{s,lj} = \phi\left(J\_{v,lj}, \text{Pi}, \text{Pe}, \sigma, \ c\_{lum}, \ c\_{w,end}, \ c\_{ave}\right) \tag{5}$$

where Pi is the diffusive permeability, Pe the modified Peclet number, clum and cw,end the LDL concentration in the lumen and the sub-endothelial layer respectively,cave is the mean endothelial concentration and σ the solvent drag coefficient.

Endothelial cell shape will affect the amount of leaky junctions. Experimental findings have shown that in areas of altered hemodynamics, endothelial cells do not have a typical cobblestone shape, but rather exhibit a more circular shape as well as increased permeability. Previous models used a relationship between endothelial permeability and local WSS based on the Endothelial Cell Shape Index (ECSI; Levesque et al., 1986). ECSI is related to the cellular shape and takes values from zero to one, i.e., a circle has an ECSI of one whilst a straight line has an ECSI of zero.

Some approaches based on ECSI calculated this variable as a function of WSS (for instance Olgac et al., 2008b used WSS values in steady state simulations). However, as previously mentioned, recent work shows that ECSI can be affected by other indices as well, such as OSI. Based on the seminal paper by Levesque et al. (1986), excellent work from Sáez et al. (2015), presents how different OSI and TAWSS modify the endothelial cell shape. This combined relationship will be key for the modeling work presented here.

In the present study, we consider the role of the two different shear stress indices (SSI), i.e., TAWSS and HOLMES, described in Section Shear Stress Indices (SSI). For each SSI, ECSI is defined according to

$$ECSI = \, 0.380e^{-0.79SSI} + 0.225e^{-0.043SSI} \tag{6}$$

**Figure 1** shows a comparison of normalized ECSI using Saez et al. results (Sáez et al., 2015), which are dependent on TAWSS and OSI, with the normalized ECSI using Equation (6) and HOLMES. Very similar behavior can be seen in both surfaces, showing how areas of low TAWSS and high OSI will increase the values of ECSI. Following this, we will use HOLMES as a combined index able to capture key haemodynamic features in the regions of interest for atherosclerotic plaque/calcification locations.

Leaky cells have high permeability to LDL, which can be linked to the magnitude of the SSI. Areas with high ECSI will be related

to a higher number of mitotic cells (MC) which are calculated as follows:

$$MC = \; 0.003739e^{14.75ECSI} \tag{7}$$

Assuming that within the endothelium the quantity of leaky mitotic cells is ∼80.5%, which represents ∼45.3% of the total number of leaky cells (LC) in that area (Tedgui and Lever, 1984a), the number of LC is calculated as:

$$LC = \begin{array}{c} 0.307 \ + \ 0.805MC \end{array} \tag{8}$$

The ratio of endothelium (φ) covered by LCs is calculated using

$$\phi = \frac{LC \,\pi \, R\_{cell}^2}{unit \, area} \tag{9}$$

where Rcell is the radius of a single cell. Finally, the total hydraulic conductivity of the endothelial leaky junctions (Lp,lj) is defined as:

$$L\_{\mathbb{P},l\circ} = \phi \cdot L\_{\mathbb{P},sl\circ} \tag{10}$$

where Lp,slj is the hydraulic conductivity of a single leaky junction calculated as follows:

$$L\_{p,sl\dot{j}} = \frac{\omega^2}{3\mu l\_{\dot{l}\dot{j}}} \tag{11}$$

with w and llj being the half-width (20 nm) and the length (2µm) of the leaky junctions and µ the viscosity term used for the estimation of the LDL penetration.

# Estimating Plaque Location and Metrics Evaluation

Previous work (Alimohammadi et al., 2015a) has shown that calculation of the hydraulic conductivity in leaky junctions (Lp,lj) can be used to estimate the magnitude of LDL fluxes across the wall along the artery and identify atheroprone regions. However, as mentioned previously, other variables such as the volume flux Jv,lj [which is closely related to the solute (LDL) flux, Js,lj], may be a better metric to identify atheroprone areas. As shown in Equation (3), Jv,lj is proportional to both Lp,lj and the pressure drop across the endothelium ∆pend and thus, Jv,lj may be more sensitive in the estimation of plaque locations. For a given metric (Jv,lj or Lp,lj), the aorta was divided into subsections and the mapping of this variable was compared with the plaque/calcification observed in the same subsections from CT scans, as shown in **Figure 2**. Plaque areas are calculated as a percentage (%) of the total area covered for the selected section of the artery, with plaque defined as HU>220 in the clinical image (Isgum et al., 2004) and Lp,lj >1.2 × 10−<sup>11</sup> m<sup>2</sup> s/kg (Tedgui and Lever, 1984b). The ratio of the area coved in the model compared to the clinical image is reported as a percentage match.

FIGURE 3 | The right anterior and left posterior views of the aorta. (A) 3D CT data visualized in Aquarius and (B) 3D LUT volume viewer in FIJI (Image J).

Frontiers in Physiology | www.frontiersin.org

Comparison of the calcification/plaque area observed in the CT scans and those calculated by the chosen metric following the methodology shown here, was used to evaluate the efficacy of the different metrics and the role of some of the selected hemodynamic (viscosity and SSI) variables.

# RESULTS

**Figure 3A** shows an image of the 3D CT image reconstruction in Aquarius (TeraRecon, USA), in which the highest intensity regions indicate plaques (white) or vessel wall calcification. **Figure 3B** shows an image of the CT scans reconstructed in the LUT volume viewer in Fiji (Schindelin et al., 2012), with intensities in the range 0–255. The colorbar shows Hounslow units (HU) from the CT scan. Although there is not a single threshold that can be considered to be a calcification, a value of 220 HU has been used previously as an indicator of calcified regions (Isgum et al., 2004). The plaques appear red and the calcifications yellow (it should be noted that the exact rendering of both panels in **Figure 2** is dependent on the distance from the first slide as there is no easy way to ensure the same depth in both pieces of software, and so the views may differ slightly).

As seen in both panels, a number of completely formed plaques can be observed in the lower edge of the aortic arch, between the two tears in FL and around the iliac bifurcation. Calcifications can be seen along the aortic arch proximal to the supraaortic branches, and in the proximal and distal FL. The existence and locations of the plaques and calcifications were confirmed and approved by the vascular surgeon who managed this patient.

# Wall Shear Stress Indices

**Figure 4** shows the distributions of the three SSI used in the present study. The TAWSS (**Figure 4A**) is moderate in the ascending aorta and in the FL between the two tears. Elevated regions of TAWSS can be observed in the visceral branches, at the coarctation and in the distal TL. The OSI (**Figure 4B**) shows scattered elevated regions throughout the domain, with particularly high values in the distal and proximal FL (which would not be captured using a rigid wall model; Alimohammadi et al., 2015b). The HOLMES index (**Figure 4C**) is similar to TAWSS, but lower in magnitude overall, and particularly in regions where the OSI is high, such as between the two tears and in the aortic arch.

In the following sub-sections, we systematically compare the indices and parameters used in the literature to indicate or predict locations of calcification and/or atherogenesis with our integrated model, the use of HOLMES and spatially varying viscosity.

# Analysis of the Total Hydraulic Conductivity

As the volume flux is dependent on both hydraulic conductivity and pressure gradient across the endothelium, we first consider L<sup>p</sup> alone.

**Figures 4A,B** show the total hydraulic conductivity of leaky junctions (Lp,lj) values calculated assuming a constant viscosity at the wall, equal to the plasma viscosity and TAWSS and HOLMES,

respectively, as wall shear stress indices. **Figures 4C**, **5D** show the same wall shear stress indices, but with the viscosity at the wall predicted based on the continuum assumption using the Carreau-Yasuda viscosity model. Note that all panels use the same color scale. Tedgui and Lever (1984b) reported Lp,lj values of 1.2 × 10−<sup>11</sup> m<sup>2</sup> s/kg, corresponding to green values in the figure. Higher values indicate pathologically elevated Lp,lj. For both of the plasma viscosity models (**Figures 4A,B**), the estimated Lp,lj values are extremely high, and no qualitative correlation can be observed when comparing these images to **Figure 3B**. Using the non-Newtonian viscosity model at the vessel wall and TAWSS to calculate Lp,lj (**Figure 5C**), reduced the magnitude of the calculated Lp,lj and resulted in a reasonable indication of the calcified/atheroprone locations in both the distal and proximal FL, although the extent of the calcified regions is shorter than those observed in **Figure 2** and the calcified region between the two tears is not identified. Additionally, this model did not identify any of the plaque locations. Comparing the Lp,lj distribution predicted using HOLMES and non-Newtonian viscosity (**Figure 5D**) to the clinical images (**Figure 3B**), it can be seen that there is a good qualitative correlation between the two figures. **Figure 5** shows this Lp,lj distribution in greater detail in both right anterior and left posterior views.

Under the assumption that plaque formation and calcification occur as a result of elevated Lp,lj, the correlation between **Figures 6**, **3B** implies that this combination of variables is able to identify regions of calcification and plaque formation, and thus could be potentially used to predict further development of such pathologies.

The region of calcification with scattered small plaques observed in the FL between the two tears is captured in the Lp,lj prediction in **Figure 6**. Similarly, the calcification of the proximal FL (both views) and the distal FL (left posterior view), are predicted by elevated Lp,lj in these regions. The low Lp,lj in the distal TL and ascending arch in the left posterior view also correlate with minimal calcification in **Figure 3B**. The plaques at the lower edge of the aortic arch, best observed in **Figure 3A** in the left posterior view, correspond to regions of elevated Lp,lj in **Figure 6**.

Some regions of the image do not correlate, such as the low Lp,lj in the right anterior view of the ascending arch, where there is calcification in the CT image.

#### Analysis of the Volume Flux

**Figure 6** provides images of the predicted volume flux across the vessel wall, Jv,lj, using various combinations of wall shear indices and viscosity assumptions, along with either a uniform pressure gradient or a time-averaged pressure gradient. These figures can be compared with **Figure 3B** to analyze the hypothesis that calcification and plaque formation are better correlated with volume flux, Jv,lj, rather than Lp,lj alone. Meyer et al. (1996) reported J<sup>v</sup> values in the range ∼2–3 ×10−<sup>8</sup> m/s, corresponding to green regions in the figure. Higher values indicate elevated solute flux.

None of the constant plasma models (**Figures 7A,B,E,F** produced Jv,lj distributions that compared well with **Figure 3B**. **Figures 7C,D** using the uniform pressure gradient are qualitatively identical to **Figures 4C,D** as the uniform pressure distribution does not affect the relative values of Jv,lj as compared to Lp,lj. When using the time averaged pressure gradient (**Figures 7G,H**), the distributions do not significantly change in qualitative terms, indicating that the prediction is more sensitive to Lp,lj, and thus appropriate models for wall shear stress and blood viscosity at the wall, than to the pressure gradient. **Figure 8**, showing the distribution of Jv,lj using HOLMES, the non-Newtonian model and time averaged pressure distribution, is therefore qualitatively similar to **Figure 6**, although there are some differences in the magnitudes.

#### DISCUSSION

The results shown in Section Results are promising and thus warrant further investigation. They also open up a number of questions. Is the volume flux (which determines the magnitude of the solute flux) a metric for plaque location? If so, then what is required in order to estimate it correctly? As mentioned in the introduction, the question of plaque location remains elusive. There are presently no available tools to predict where this might occur, or indeed to sufficiently explain why. This may be because, if we take volume flux (Jv,lj) (or even hydraulic conductivity) as

the metric for plaque location, its generation is multifactorial and complex and thus simplified models lack some of the key features that seem to impact plaque location (see Section Results). In the present study, strong correlations were only produced through the use of a multiscale model combining multiple hypotheses at the cellular and physiological level, as evidenced by the comparison of **Figures 5**, **7** with **Figure 3**. This complex approach involves the detailed formulation, extraction and combination of more than 11 different variables including the calculation of variable pressure differences along the arterial wall across the endothelium, the use of a non-linear model for blood viscosity and use of the HOLMES index to isolate regions of low, oscillatory shear. This formulation, making use of mechanistic models, also provides these metrics (Jv,lj and Lp,lj) with strong

interpretability and physical meaning. As previously explained, the origin and interpretation of HOLMES as a combined index is strongly rooted in the most recent, relevant literature (Sáez et al., 2015). The work presented in this paper exemplifies the power of mechanistic models as "hypothesis tester" and a key example of the use of mathematics and mechanistic models and formulations to understand biological processes. In fact, it can be can argued that a posteriori hypotheses, as a result of abandoning part of a priori thinking in the light of new observations, can pave the way for future studies (Erren, 2007). Here, an unambiguous formulation could enable others to follow up on the findings and modified conjectures to advance knowledge in this field. Despite some limitations (please see below), it remains a fact that the work presented here is strongly anchored in the state of the art and current knowledge in this area.

In addition to the clear qualitative improvement in the correlation between the CT scans and Lp,lj and Jv,lj distributions, we utilized a simple measure based on plaque areas, as shown in **Figure 9** for the descending aorta, a key region in the case of AD (as explained in Section Estimating Plaque Location and Metrics Evaluation).

When comparing Lp,lj values in specific regions of the aorta, using the results shown in **Figure 5**, the calcified regions predicted by the model overlapped 80% of the calcified regions observed in the CT scans for the HOLMES model. No calcification was predicted using TAWSS. The next step was to analyze the results of Jv,lj (given its stronger relationship with the LDL flux).

**Figures 10A,C** show Jv,lj in this region calculated with TAWSS and HOLMES, respectively and **Figures 10B,D** show the calcified region only. **Figures 10E,F** show the equivalent figures for the CT scans. Comparing the proportional area of the calcified region between the two cases, 26 and 93% of the regions overlap for TAWSS and HOLMES, respectively. When calculated using HOLMES in the selected region (Region 3 in **Figure 2**), 45% of the visible arterial wall is predicted to be prone to calcification, compared to 48% of the visible arterial wall observed in the CT scan image. Applying the same analysis using HOLMES to the whole aorta predicts between 80 and 95% of overlapping. This analysis supports the potential predictive power of the proposed model.

From a hemodynamics point of view, the heterogeneous distribution of calcification/plaque formation will be influenced by disturbed flow. Wall shear indices such as TAWSS and OSI capture only partial aspects of the complex flow fields. The wall shear stress vector has both direction and magnitude at each moment throughout the cardiac cycle. TAWSS captures the average magnitude, but contains no information about directionality. OSI characterizes the variability in the directionality. However, a number of indices ("emerging" multidirectional predictors, as defined by Gallo et al., 2016) have been proposed and some of these might have potential as predictors of atherogenesis. Peiffer et al. (2013b) proposed the transverse wall shear stress, which is the temporal average of the component of the wall shear stress vector that is perpendicular to the time average wall shear stress vector. Morbiducci et al. (2015) defined the preferential direction as being relative to the direction of the vessel centerline and similar considered the temporal average, producing an additional index based on the temporal average of the ratio of these two components. Arzani and Shadden (2016) also introduced backward wall shear stress, considering the average of the negative instances of wall shear stress in the direction opposite to the centerline. In future studies, it will be important to explore the efficacy of these new indices.

In this paper, we have exclusively focused on the "established" predictors, for which the links between endothelial cell behavior and haemodynamics have been shown in the relevant literature as described above.

In the work presented here, the use of patient-specific data (including invasive haemodynamic measurements) and BCs, inclusion of wall motion and use of a non-Newtonian blood viscosity model are likely to improve the estimates of these hemodynamic variables and thereby increase the likelihood of an improved correlation. Importantly, as evidenced by **Figures 4**, **7**, the interpretation of the role of these variables individually was unable to predict distributions of atherogenesis/calcification. It was only through a combination of HOLMES, a compound shear index, a non-Newtonian fluid viscosity and a sophisticated simulation model that this was achieved. In general, when extracting information related to values calculated at the wall, motion is extremely important. In this particular case and as

shown previously (Alimohammadi et al., 2015b), simulating wall motion is key, since the dynamic interactions between the intraluminal pressure-gradient, the vessel wall elasticity and the intimal flap motion play a critical role in accurately predicting haemodynamics in the false lumen. Moreover, in healthy aortae, rigid wall simulations have been reported to produce somewhat comparable distributions of WSS, albeit with overestimated magnitude (Brown et al., 2012; Reymond et al., 2013). In dissected aortae, the differences between the results of rigid wall (CFD only) and FSI simulations are significant (Alimohammadi et al., 2015b), which has a clear impact in the interpretation of the effect of mechanical stimuli on endothelial behavior in this case. These differences are enhanced by the complex, intertwining lumina, in which even small motion variations have a decisive effect. Hence, the use of FSI, while more time-consuming and technically challenging, is important.

#### Limitations

One limitation of the fluid dynamics simulation is the absence of the visceral arteries, which would alter the flow in the descending aorta. As previously stated, it was not possible to resolve these vessels from the CT scans, so they were omitted. Nonetheless, even in their absence, we believe the usefulness of the model was demonstrated.

A patient-specific inflow waveform was not available for the present study, and as such, a waveform from the literature was selected from a patient suffering from a similar type-B AD. The absence of a patient-specific inflow reduces the specificity to the patient of the present results however, given that the boundary conditions were tuned to patient-specific values using the same inflow, the hemodynamic environment predicted by the simulations is expected to be an appropriate representation of a patient with type-B AD. The flow waveform at the inlet was

applied as a uniform velocity, rather than mapping to parabolic or Womersley profiles. Whilst accurate axial velocity profile, as extracted using pcMRI can provide improved predictions of characteristics such as helical flow (Morbiducci et al., 2013), it is not clear that parabolic or Womersley profiles offer any improvement over a uniform velocity (Marzo et al., 2009; Campbell et al., 2012; Morbiducci et al., 2013).

The isotropic hyperelastic model of Raghavan and Vorp (2000) is a simplification of the true aortic wall properties, which are known to be anisotropic (Gasser et al., 2006). This model has been shown to provide improved predictions of stress distributions compared with a linear elastic model, but underestimates peak stresses relative to anisotropic models (Roy et al., 2014). However, it is important to note that wall stresses are not explicitly evaluated in the present study; additionally, fibrosis in the vessel wall alters wall properties over time in AD, and is thus highly patient-specific. Furthermore, wall motion was not captured as part of the clinical data collection. More detailed imaging as well as experimental data on vessel wall properties in AD is necessary for better, patient-specific simulations. Given these constraints, although we acknowledge that more complex models of vessel wall properties have been applied in AD and that this might result in local variations, in this case these are unlikely to yield improved accuracy in what is our ultimate goal, the quantification of their effect on hemodynamic parameters for individual patients.

Another limitation of this approach, which is fairly consistent throughout the related literature is the use of "visual maps," in order to establish qualitative correlations between the in vivo data and the simulation results. Recent studies address the issue of quantification of these type of results by using statistical methods to establish quantitative correlations and statistical significance of the variables analyzed (simulated) with respect to plaque location, obtained from in vivo data (Morbiducci et al., 2015; Gallo et al., 2016). This is clearly a welcome development that we will be keen to use for future, multi-patient studies. As it has been presented here, the focus of this paper is on the use of multi-scale modeling and simulation tools and the development of interpretable, physiologically-based metrics to understand plaque location.

The results presented in this paper when using the multi-criteria, patient-specific, multi-scale complex framework described here, compare well, qualitatively, to in vivo data and although there is no perfect match, this is to be expected, when considering the number of assumptions, simplifications and limitations as described (please see above). Moreover, as for patient-specific data, only anatomical information was fed into the model and hemodynamic variables were calculated based on invasive pressure measurements in order to properly characterize the flow, as described in Alimohammadi et al. (2014). Apart from the hemodynamic calculations previously described, it is important to mention that the authors have been extremely careful to inform the model with data and values taken straight from the literature, so, there is no unique estimation of parameters or "fitting" apart from appropriate characterization of flow variables, for the simulation presented here. It remains a striking feature of this approach that even taking this into account; the correspondence between observed and simulated calcified/atherosclerotic regions is high, can be explained and appears to be a marked improvement upon other standard methods reported in previous studies. It also offers a coherent, mechanistic explanation that is able to shine some light on combined mechanisms responsible for the location of atherosclerotic/calcification areas and to interpret them together and simultaneously. Although the focus of this paper is on the use of multi-criteria and multi-scale mathematical modeling to understand atherosclerosis and results are shown for one patient only, it is important to say that preliminary and very encouraging results from a different anatomical site with a simpler model, but also following a mechanistic approach have been published in Alimohammadi et al. (2015a). Equally, results from other simulations currently performed by our group (not shown here) using the approach developed in this work for different arteries, show similar effectiveness than the one discussed here, in terms of identification of calcification/atheroprone areas. We acknowledge nevertheless the need to test this approach in a small cohort of patients, taking into account the limitations and new developments described above. This is work that is already under way.

# CONCLUSIONS

In the present study, we have presented a mechanistic, mathematical model of endothelial mechanotransduction to understand plaque location/calcification. The model is tested on a patient-specific case for which in vivo measurements were obtained at University College Hospital and a patient-specific biomechanical model produced (Alimohammadi et al., 2015a) and attempts to provide a clear, multi-factorial metric for plaque location, with strong physiological meaning and interpretation. The results from this model compared favorably with in vivo data and outperformed other well-established indices currently used in the literature. The model used an advanced FSI simulation, comprising patient specific dynamic outlet boundary conditions and non-Newtonian blood viscosity. This was coupled with an established model for atherogenesis in order to investigate the roles of various hemodynamic parameters on the development of calcified regions in the aortic wall.

As an input to the endothelial permeability model relating to shear stress, we hypothesized that regions of oscillatory, low magnitude shear stress would be susceptible to calcification, due to the known connection to increased permeability. We therefore, proposed the compound HOLMES shear index, which includes both magnitude and oscillatory characteristics and will thus emphasize oscillatory, low magnitude shear stress and found that it considerably improved the predictive power of the model over TAWSS-based analysis.

The role of the complex characteristics of the hemodynamics near the wall was investigated by hypothesizing that elevated blood viscosity near the wall would limit the convection of plasma into the vessel wall, and vice versa. It was found that when the spatially varying blood viscosity at the wall as estimated using an empirical non-Newtonian viscosity model, was used in the endothelial permeability model, the predicted Lp,lj and Jv,lj distributions bore increased resemblance to the observed regions of calcification.

This work provides a good example of the use of multiscale mathematical modeling to understand physiology. The promising results obtained from this approach warrant further investigation. Next steps will include studies of a larger number of patients to enable comparisons amongst patients and potentially statistical analyses, in order to investigate in detail the predictive power of the model.

# AUTHOR CONTRIBUTIONS

MA and CP conceived the study and carried out simulations and wrote the manuscript. OA acquired the clinical data and VD conceived the study and wrote the manuscript.

# FUNDING

EPSRC grant "Personalised Medicine Through Learning in the Model Space" (grant number EP/L000296/1). Leverhulme Trust Senior Research Fellowship "Exploring the Unknowable Using Simulation: Structural Uncertainty in Multiscale Models" (Fellowship number RF-446 2015-482).

# REFERENCES


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2016 Alimohammadi, Pichardo-Almarza, Agu and Díaz-Zuccarini. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Signal Reconstruction of Pulmonary Vein Recordings Using a Phenomenological Mathematical Model: Application to Pulmonary Vein Isolation Therapy

Harry D. Green1, 2, 3, Glyn Thomas <sup>4</sup> and John R. Terry 1, 2, 3, 5 \*

*<sup>1</sup> College of Engineering, Mathematics and Physical Sciences, University of Exeter, Exeter, United Kingdom, <sup>2</sup> Wellcome Trust Centre for Biomedical Modelling and Analysis, University of Exeter, Exeter, United Kingdom, <sup>3</sup> Living Systems Institute, University of Exeter, Exeter, United Kingdom, <sup>4</sup> Bristol Heart Institute, Bristol, United Kingdom, <sup>5</sup> EPSRC Centre for Predictive Modelling in Healthcare, University of Exeter, Exeter, United Kingdom*

#### Edited by:

*Joseph L. Greenstein, Johns Hopkins University, United States*

#### Reviewed by:

*Natalia A. Trayanova, Johns Hopkins University, United States Mohsin Saleet Jafri, George Mason University, United States Thomas Hund, The Ohio State University Columbus, United States*

> \*Correspondence: *John R. Terry J.Terry@exeter.ac.uk*

#### Specialty section:

*This article was submitted to Computational Physiology and Medicine, a section of the journal Frontiers in Physiology*

Received: *31 August 2016* Accepted: *28 June 2017* Published: *17 July 2017*

#### Citation:

*Green HD, Thomas G and Terry JR (2017) Signal Reconstruction of Pulmonary Vein Recordings Using a Phenomenological Mathematical Model: Application to Pulmonary Vein Isolation Therapy. Front. Physiol. 8:496. doi: 10.3389/fphys.2017.00496* Atrial fibrillation (AF), the most prevalent cardiac arrhythmia, is commonly initiated by ectopic beats originating from a small myocardial sleeve extending over the pulmonary veins. Pulmonary vein isolation therapy attempts to isolate the pulmonary veins from the left atrium by ablating tissue, commonly by using radiofrequency ablation. During this procedure, the cardiologist records electrical activity using a lasso catheter, and the activation pattern recorded is used as a guide toward which regions to ablate. However, poor contact between electrode and tissue can lead to important regions of electrical activity not being recorded in clinic. We reproduce these signals through the use of a phenomenological model of the cardiac action potential on a cylinder, which we fit to post-AF atrial cells, and model the bipolar electrodes of the lasso catheter by an approximation of the surface potential. The resulting activation pattern is validated by direct comparison with those of clinical recordings. A potential application of the model is to reconstruct the missing electrical activity, minimizing the impact of the information loss on the clinical procedure, and we present results to demonstrate this.

Keywords: atrial fibrillation, radiofrequency ablation, pulmonary vein isolation, mathematical model, pulmonary vein recording, signal reconstruction, minimal cardiac models, cardiology

# INTRODUCTION

Cardiac disease is the most common cause of death among the adult population worldwide (Murray and Lopez, 1997). Of the main contributors to cardiac disease, atrial fibrillation (AF) is the most common arrhythmia (Kannel et al., 1998), with a lifetime incidence of one in four at age 40 (Lloyd-Jones et al., 2004) and prevalence aged 80+ of approximately 9%. AF is associated with a near doubling of mortality (Benjamin et al., 1998) due primarily to a three-fold increase in the likelihood of congestive heart failure and a five-fold increase in the likelihood of stroke (Camm et al., 2012). Consequently, AF is a significant burden on public health. For example, in the UK the cost of treating cases of AF and complications thereof are estimated at £2 billion annually (The Office of Health Economics, 2009), whilst in the USA AF is predicted to double in prevalence from 2010 to 2030 (Colilla et al., 2013). AF is characterized by a rapid, irregular, atrial rate due to spiralling

**181**

wavefronts (Jalife, 2003; Nattel et al., 2008; Calvo et al., 2014), and is most commonly initiated from a small section of the left atrial myocardium that extends over the base of the pulmonary veins [responsible for an estimated 88% (Chen et al., 1999) to 94% (Haissaguerre et al., 1998) of cases].

Herein we focus on Circumferential Pulmonary Vein Isolation (CPVI), a minimally invasive surgical technique for treatment of AF, in which a circular lesion is formed surrounding the pulmonary vein via the application of radiofrequency energy, electrically isolating the left atrium from the pulmonary vein and so preventing the propagation of an action potential (AP) in or out of the myocardial sleeve. Whilst the initial success rate of pulmonary vein isolation is approximately 85% (Bänsch et al., 2013), recurrence rates 5 months after ablation therapy can be as high as 30% in paroxysmal AF patients or 78% in permanent AF patients (Oral et al., 2002). It is desirable to ensure that the ablation process is completed as quickly as possible, as the duration of the procedure is known to strongly correlate with the rate of recurrence (Shim et al., 2013). Additionally, ablation of the pulmonary veins carries a risk of pulmonary vein stenosis (Robbins et al., 1998) and if complete electrical isolation is not achieved, the lesions can become pro-arrhythmic through the creation of conduction obstacles that facilitate the initiation of re-entrant waves.

It is common for the initial circular lesion made during CPVI to be incomplete and small conduction gaps remain. These are most commonly due to poor depth penetration of the lesion and the ablation catheter not maintaining a continuous contact with the heart tissue. To provide a guide to the surgeon as to the location of the conduction gaps, bipolar recordings of electrical activity around the pulmonary vein are taken using a lasso catheter typically consisting of 10 or 20 electrodes (see **Figure 1** for an exemplar time-trace). The conduction gap is assumed to correspond to the location of the electrode(s) where the first spikes are observed and these sites are targeted for further ablation (Haissaguerre et al., 1998; Haïssaguerre et al., 2000).

FIGURE 1 | Clinical pulmonary vein recording. The pulmonary vein recording of a patient with atrial fibrillation during pulmonary vein isolation therapy. Spiking indicates electrical activity as the action potential propagates through the recording catheter. Since the pulmonary vein is not a perfect cylinder, not all electrodes make a good contact. For example, Channels 3-4, 17-18, and 19-20 show no spiking activity for this reason. These are referred to as missing channels throughout the paper.

However, as the pulmonary vein is not a perfect cylinder it is common for some electrodes to make poor contact with the tissue. **Figure 1** is an example of this happening in clinic, and in this case it is difficult to infer the activation pattern across PV 17- 18 and 19-20. If these missing channels correspond to the region of first activation, this information loss could potentially lead to ablating the wrong region, or concluding the process has been successful.

In this paper, we focus on developing a mathematical representation of the phenomenology of the electrical signal recorded from the lasso catheter and to use this to reconstruct missing electrical signals. This is in contrast to typical approaches to modeling the cardiac AP or the body surface ECG where physiological detailed models are typically used (see, for example, Clayton et al., 2011; Noble et al., 2012 for comprehensive reviews). Developed appropriately, phenomenological models can be used to produce patient-specific simulations of the electrophysiology during treatment and could therefore form a part of a therapeutic decision support system to minimize the impact of information loss in clinic. This approach is motivated by our experience in neurology, where mathematical models of the phenomenology of electrical recordings from scalp electroencephalography have demonstrable potential in providing decision support for the diagnosis of epilepsy, without recourse to detailed models of the underlying neurophysiology (Schmidt et al., 2016).

The use of physiologically detailed mathematical models has enabled personalized 3D modeling of the atria, largely involving detailed biophysical models to investigate mechanisms behind the sustenance of AF (McDowell et al., 2013; Zahid et al., 2016). Additionally, fibrosis patterns have attracted significant recent attention (McDowell et al., 2012), and results obtained from the detailed models have elucidated the role of so called "islands of fibrosis" in the atria (Chrispin et al., 2016). Further, techniques are in place for the simulation of "virtual ablation" and bipolar electrograms (Dang et al., 2005; Reumann et al., 2008; Tobon et al., 2010; Yun et al., 2014). In a 2014 study (Hwang et al., 2014) a variety of ablation strategies were simulated and compared in a computational study, finding that CPVI with two additional linear lesions (along the roof and posterior wall) showed the highest AF termination rate.

However, such studies typically make the following assumptions:


Both assumptions are likely to be invalidated in the clinical setting, where significant information loss due to poorly connected electrodes is commonplace and conduction gaps create discontinuous lesions. These were highlighted in 2011 by Miyamoto et al. (2011) who proposed a method to infer a pulmonary vein activation map via gentle movement of the catheter. In conclusion they raised concerns that signals were unreliable due to some electrodes touching the endocardium whilst others did not. A further issue is that bipolar electrodes located symmetrically to a conduction gap will record a zero signal despite a wavefront passing through.

To address these challenges, we introduce a phenomenological reaction-diffusion model of the cardiac AP [the so-called Bueno-Orovio, Cherry and Fenton (BOCF) model Bueno-Orovio et al., 2008] on a cylinder with regions of zero conduction representing ablated tissue to build simulated representations of the bipolar signals recorded by the lasso catheter. Our focus on a simplified model of the phenomenology of the electrical signal, rather than a detailed model of the underlying electrophysiology, is two-fold. First, a cardiologist uses information from the macroscopic electrical recordings to identify appropriate site(s) to ablate, without recourse to any detailed understanding of the underlying electrophysiology. Second, the time available for the surgical procedure is of the order 1 h meaning that the model must efficiently reproduce a signal to be of use as a decision support tool during the procedure. The BOCF model provides a pragmatic balance between the quality of the simulated signal and the computational time required to produce the output. For example, many detailed biophysical cardiac models, such as Courtemanche et al. (1998); Nygren et al. (1998); Priebe and Beuckelmann (1998); Iyer et al. (2004); ten Tusscher (2004) require significant time (of order hours) to compute appropriate APs, rendering them inappropriate in the clinical setting. In contrast, the BOCF model can be run multiple times for parameter estimation and sensitivity analysis over much shorter timescales (of order seconds to minutes). There exist models, verified either with data or by their to the output of detailed biophysical models that satisfy these conditions (Mitchell and Schaeffer, 2003; Bueno-Orovio et al., 2008; Fenton and Cherry, 2008).

We demonstrate that this simple model can reproduce the activation pattern across electrodes recorded in clinic. Furthermore, we test the potential of the model to reconstruct recordings that have been lost to poor contact. We verify the accuracy of the simulated recording using clinical data and minimizing the root mean squared error between the activation patterns in the model and those in the data. To test the accuracy of the reconstruction, we use recordings for which all channels are spiking cleanly, and remove a subset, so that the original signal can be used for error calculation. Further, we present results showing cases in which the reconstruction of signals via the model would lead to reducing the number of RF pulses. Reducing the number of RF pulses would both minimize unnecessary damage to the heart and shorten the duration of the procedure. This is significant due to the correlation between the duration of the procedure and the rate of recurrence (Shim et al., 2013). Finally, as we are motivated by the ultimate potential for clinical applicability, we also demonstrate a small trial which shows (a) that the loss of information affects the decision of the cardiologist, and (b) that the magnitude of this effect is reduced when the reconstructed signals are provided to the cardiologist.

# METHODS

In this section we introduce the mathematical model used to generate the underlying AP which is in turn used to generate a traveling wave of intracellular potential within the pulmonary vein. We describe the methods used to simulate the models and how their parameters may be calibrated (either from synthetic data or clinical recordings). We further describe how the model can be used to reconstruct missing channels from data collected clinically from a lasso catheter. A schematic of how the overall process might be used to provide clinical decision support is illustrated in **Figure 2**.

# Mathematical Model of the Underlying AP

In the current paper we model the pulmonary vein AP using an extension of the 1998 Fenton-Karma model (Fenton and Cherry, 2008): the four variable Bueno-Orovio Cherry Fenton (BOCF) model. This is a monodomain

FIGURE 2 | Schematic and application of model. Figure showing the intended application of the model. (A) A cardiologist records Pulmonary Vein Recordings using a lasso catheter. (B) An example of a pulmonary vein recording with some channels not correctly spiking (missing channels). (C) A model simulation demonstrating the propagation through a conduction gap, showing high intracellular potential in red and low in blue. White stars are plotted on the electrode locations. (D) The result of applying the model to the signal in B to recover the lost channels.

phenomenological model of the human ventricular AP first introduced in Bueno-Orovio et al. (2008):

$$\begin{split} \dot{u} &= \nabla \cdot \left( D\_{\text{BOCF}} \nabla u \right) - \left( f\_{\text{fi}} + f\_{\text{so}} + f\_{\text{si}} \right) \\ \dot{\nu} &= \left[ 1 - H(u - \theta\_{\text{v}}) \right] \left( \nu\_{\infty} - \nu \right) / \tau\_{\text{v}}^{-} - H(u - \theta\_{\text{v}}) \nu / \tau\_{\text{v}} \\ \dot{\nu} &= \left[ 1 - H(u - \theta\_{\text{w}}) \right] \left( \nu\_{\infty} - \nu \right) / \tau\_{\text{w}}^{-} - H(u - \theta\_{\text{w}}) \nu / \tau\_{\text{w}} \\ \dot{s} &= \left( \left[ 1 + \tanh[k\_{\text{s}}(u - \mu\_{\text{s}}) \right] \right) / 2 - s \right) / \tau\_{\text{s}}. \end{split} \tag{1}$$

Here u represents the transmembrane voltage, Jfi, Jso and Jsi are phenomenological summations of the fast inward, slow outward, and slow inward currents respectively. Jfi is effectively gated by the gating variable v, Jso is voltage gated, and Jsi is effectively gated by the product of the gating variables w and s. DBOCF is either a spatially dependant diffusion constant (under the assumption of isotropic diffusion), or a diffusion tensor (under the assumption of anisotropic diffusion). We always take initial conditions at the resting state, where [u(0), v(0),w(0),s(0)] = [0, 1, 1, 0]. A full description of this model can be found in Bueno-Orovio et al. (2008).

#### Calibrating BOCF Model Parameters

Given that the shape of the emergent electrical activity recorded on the lasso catheter may be constrained by the underlying structure and function of the AP, a propagating AP was simulated using the detailed biophysical Courtemanche model for the human atrium (Courtemanche et al., 1998; Imaniastuti et al., 2014; Labarthe et al., 2014) as a proxy for clinical AP data. A generic AP from the Courtemanche model was modified to account for the electrical remodeling associated with AF (Courtemanche et al., 1999) and used as the initial stimulus for the BOCF model with parameters as defined in the sample fitting code in the appendix of Bueno-Orovio et al. (2008). These parameter choices were then evolved using the Nelder-Mead Simplex Algorithm (Nelder and Mead, 1965) (implemented by MATLAB's FMINSEARCH), by minimizing the root mean squared error between subsequent APs (see **Figure 3**). With a spatial resolution 1x = 0.2 mm, a diffusion constant of DCourt = 2.615 was necessary for the simulated wavefront to match the conduction velocity of 48 cm/s observed clinically (Labarthe, 2013). To eliminate any effects from boundary conditions or transients from the stimulus, the fit was performed at the point x = 10 mm on a tissue cable 20 mm long. A cycle length of 600 ms was used to match the clinical data. This process resulted in the parameter choices defined in **Table 1**.

**Figure 3** shows the shape of the propagating APs under the above conditions using the BOCF model with parameters as in **Table 1**, alongside the Courtemanche AF model as described in Courtemanche et al. (1999). The important qualities reproduced were conduction velocity (indicated by the simultaneous spike), upstroke velocity, and AP duration.

For the case of anisotropic diffusion an asymmetric finite difference method was used to simulate the BOCF model (see van Es et al., 2014 for full details). Since in general the degree of anisotropy for an individual patient is unknown, we included the principal axes and eigenvalues of the diffusion tensor as additional parameters to be optimized by our fitting algorithm.

models using the parameters in Table 1, of an AP at a point 10 mm from the stimulus with a spatial resolution of 0.2 mm and a time step of 0.01 ms. Model calibrated by minimizing the root mean squared error using the Nelder-Mead method. The Courtemanche model is solved using the parameters in Courtemanche et al. (1999).

TABLE 1 | Parameter values of Bueno-Orovio Cherry Fenton model.


*The original parameters of the Bueno-Orovio Cherry Fenton model (Bueno-Orovio et al., 2008) alongside the parameters obtained from our fitting algorithm. Diffusion coefficient DBOCF was not given in the original model.*

Physiological studies place the anisotropy ratio between 2 and 10, (Koura et al., 2002; Xie and Zemlin, 2016), which were used as bounds in our algorithms. The initial principal axes were placed at 45 degrees to the x and y axes, maximizing the effect on the propagation pattern.

#### Simulating Pulmonary Vein Recordings

2D simulations of the pulmonary vein were performed by numerical integration of Equation (1) by a finite difference method over a discretized cylindrical domain to represent the excitable myocardial sleeve extending over the base of the pulmonary vein. Dimensions vary from vein to vein, with the right inferior typically the largest and the left inferior the smallest (Stojanovska and Cronin, 2008). We assume dimensions within the range of observed measurements: a length of 15 mm (Cronin et al., 2007) and a diameter of 12.5 mm (Cabrera et al., 2002; Kim et al., 2005). A spatial resolution of 1x = 0.2 mm was used to discretize this cylinder into a rectangular domain of 200 × 75 grid points. Periodic boundary conditions were used along the lines x = 1 and x = 200, whilst Neumann boundary conditions were used along the lines y = 1 and y = 75 (where x and y represent nodes on the grid). We set the conductivity to 0 to model the effect of lesions due to ablated tissue at the relevant points, following the approach introduced in Dang et al. (2005); Reumann et al. (2008); Tobon et al. (2010). As we are only concerned with the effect on the AP propagation from the ablation process, we do not require a model of the thermodynamic processes of the catheter itself (Berjano, 2006; Suárez et al., 2010).

A visual representation of this structure is shown in **Figure 2**, which shows the propagating intracellular potential with the lines y = 1, y = ha, and y = h<sup>r</sup> annotated (**Figure 2C**). An ectopic is initiated from a stimulus along the line y = 1; the edge of the myocardial sleeve furthest from the atrial junction. Virtual ablation is performed by introducing a line of lesions on the circle y = h<sup>a</sup> such that D(x, ha) = 0. Conduction gaps are modeled such that D(x, ha) = DBOCF (for conductive tissue on small segments of the circle y = ha). Consequently, semi-circular wavefront(s) will form on the other side of the lesions. Although loosely based on the underlying mechanisms, the values of the obtained parameters are phenomenological, and fit to the available data to ensure an accurate simulation on the lasso catheter electrodes, not to provide an estimation of the real location of the conduction gap.

We simulate pulmonary vein recordings from the lasso catheter across n electrodes (where n is typically 10 or 20), on y = h<sup>r</sup> , where h<sup>r</sup> > ha. The electrodes are assumed to be equally spaced d = 200/n apart, such that for an n electrode catheter c = (a, hr) where a = {d, 2d, ..., nd}). At each point c = (x ′ , y ′ ), an approximation for the surface potential 8 described originally in Gima and Rudy (2002) is given by:

$$\Phi(\mathbf{x}', \mathbf{y}') = aD(\mathbf{x}', \mathbf{y}') \int \int \left(-\nabla \mu\right) \cdot \left[\nabla \frac{1}{r}\right] d\mathbf{x} d\mathbf{y},\tag{2}$$

where

$$r = \sqrt{(\mathbf{x'} - \mathbf{x})^2 + (\mathbf{y'} - \mathbf{y})^2}. \tag{3}$$

Bipolar recordings between electrodes i and j (denoted PV i-j clinically) are simulated by:

$$\text{PV i} - \text{j} = \Phi(a\_{\text{i}}, h\_{\text{r}}) - \Phi(a\_{\text{j}}, h\_{\text{r}}).\tag{4}$$

Throughout this paper, we divide the pulmonary vein into three equal sections, with the ablation line positioned at h<sup>a</sup> = 25 and the recording catheter positioned at h<sup>r</sup> = 50. This is a practical consideration, as quantifying these measurements during the procedure would be difficult given information collected as standard in clinical practice.

#### Relative Activation Time Curves

The important characteristics of both the simulated and recorded data are the activation times (from maximal absolute value of dV/dt) of each signal compared to the others, as this gives a representation of the wavefront shape termed the relative activation time curve. It is necessary to use the absolute value as the recordings are bipolar. The relative activation time curve can be visualized by plotting the catheter along the x axis and its activation time on the y axis, giving a curve of the activation times of each signal relative to the others.

To understand the relationship between the relative activation time curve and parameters of the overall pulmonary vein model, the quantity, size and locations of conduction gaps are used as input parameters, since these have the most profound effect on the emerging wavefront shape. The root mean squared error between relative activation times obtained from simulated and clinical recordings are minimized, again using the Nelder-Mead Simplex Search method (implemented by MATLAB's FMINSEARCH) to establish the location of conduction gaps which result in the most accurate activation time curve. Here it is important to note we do not claim to have found the location of the conduction gap(s) via this fit, only that we have calibrated model parameters that most closely recreate the phenomenology of the waveforms from the recording catheter.

#### Reconstruction of Missing Electrodes

In the cases for which there is poor contact between recording catheter and tissue, the signal is typically flat or white noise. This is evident, for example, in channels PV 3-4 17-18, and 19- 20 in **Figure 1**. To reconstruct missing electrode recordings, a partial relative activation time curve was obtained from the active channels. Model parameters of the overall pulmonary vein model were calibrated from the active channels, using the Nelder-Mead Simplex Search (implemented by MATLAB's FMINSEARCH).

#### Clinical Data

Pulmonary vein recordings used in this paper were obtained from adult male and female subjects undergoing pulmonary vein isolation therapy at Bristol Heart Institute. Bipolar recordings were obtained from a deflectable, circular, 20-pole Lasso catheter (Biosense Webster Ltd). Patients with both paroxysmal and persistent AF were included but all cases were paced into normal sinus rhythm by pacing at 600 ms intervals, as per standard clinical practice. All data were appropriately anonymized prior to their use in this study. Under United Kingdom law, patient data collected during normal clinical routine and anonymized before research use may be used for research without additional consent.

#### RESULTS

#### Simulated Pulmonary Vein Recordings

First we consider how well the model can reproduce the phenomenology of the pulmonary vein recordings when all 10 channels are active. To consider this, we use an exemplar set of clinical pulmonary vein recordings collected during pulmonary vein isolation therapy (as described in the methods). The goodness of fit between clinical recordings and model simulations is determined by minimizing the root mean squared error between the relative activation times of the model and the

FIGURE 4 | Parameter fitting and comparison with clinical recordings. (A) Presenting the relative activation time curves for the clinical data (presented in full in B) (black), the model assuming isotropic diffusion (blue) and the model assuming anisotropic diffusion (red). Parameters of the model were calibrated by minimizing the root mean squared error between clinical recording and the model using the Nelder-Mead method. (B) Presenting an exemplar of clinical pulmonary vein recordings collected during pulmonary vein isolation therapy at the Bristol Heart Institute (black). Overlaid are the model simulations, with parameters calibrated as described in (A), under the assumption of isotropic diffusion (blue) and isotropic diffusion (red) respectively.

data. This is achieved by varying the positions of conduction gaps in the model. The average of ten recorded events in the data is used to form the target relative activation time curve. This ensures some robustness to variation in the data and enables us to estimate the conduction gap location and width as parameters, which should be constant until ablated.

For the chosen clinical data, and for parameter choices of the underlying BOCF model as in **Table 1**, we find that the root mean squared error between the relative activation time curve of the clinical data and that of the model (assuming isotropic diffusion) is minimized by placing conduction gaps centered on points x = 65.5 and x = 167, with widths 11 and 6 respectively. Both the number of minima and their locations are used to optimize position and width of the conduction gaps. This is important since both the number of minima and their locations within the relative activation time curve emerge as a result of the conduction gaps generating the signal. In current clinical practice, the earliest activation time(s) (e.g., the minima of the relative activation curve), are the most important, as these are assumed to be closest to the conduction gap and therefore the optimal ablation site. This is illustrated in **Figure 4A**, where we also present a model fit under the assumption of anisotropic diffusion. In this case the conduction gaps are centered on points x = 59.5 and x = 164.5, with width 11 and 13 respectively.

In **Figure 4B**, we present a comparison between the original choice of clinical pulmonary vein recordings and simulations for the two classes of model. Time units of the model are rescaled such that the relative activation time-scale of the model is equivalent to that of the clinical recordings, which permits a clearer visual comparison. Note that both classes of model result in visually similar simulated pulmonary vein recordings. We perform a more rigorous analysis of differences between anisotropic and isotropic diffusion later, when considering the ability of the model to reconstruct missing channels in the clinical data.

ectopic event *N* increases away from the initial ten ectopic events used to calibrate model parameters. The line of best fit is displayed in blue. The periodicity in the error is likely due to rhythmic movements of the patient, such as breathing.

Next, we tested the capacity of the model to predict future ectopic events, given an average over an initial ten events. For the identified choice of model parameters from the initial ten events, we simulated a series of additional ectopic events and for each event we calculated the root mean squared error between relative activation time curves obtained from either simulated or clinical ectopic events. We define t<sup>0</sup> as the time of the last event in the training set, and t<sup>N</sup> as the time of the N th subsequent ectopic event. **Figure 5** shows how this error scales as the number N of the ectopic event moves further away from the training set. The apparent periodicity in the error is most likely due to rhythmic movements of the patient, such as breathing.

original clinical recording. For each case 20 simulations for up to 20 random choices of channels to be reconstructed were performed. (+) symbols denote outliers in the 1% tail of the error distributions. The case of 0 reconstructed channels enables a comparison of the limit of goodness of fit between models and the clinical data. We see that for 3 or fewer channels being reconstructed, uncertainty is predominantly due to model choice rather than the number of reconstructed channels, as we see no significant change in the mean error between model and clinical data. We use an unequal variances *t*-test to determine whether the errors came from a distribution with equal mean and find that with the exception of 3 channels reconstructed there is no significant difference in the errors between models.

#### Reconstructing Missing Channels

We now focus on the capacity of the model to reconstruct missing channels, exemplars of which were shown in **Figure 1**. This is a key result of this paper, and the one with most relevance to a potential clinical decision support system. To test the accuracy of the model, we start with a clinical recording for which all channels are active. We then eliminated a subset n (n = 0 to 5) of the channels replacing them with a 0 time trace. Five was chosen as the upper limit, since clinically a recording with less than half the channels active would not be relied on for determining the site of ablation. We then estimated model parameters using the same approach as in the previous section, but only data from those channels that were active. Using these parameters we then contrasted the error between the relative activation time curves obtained from the simulated next ectopic event and the subsequent ectopic event from the original clinical recording (including all channels). This enables us to assess how well the model can reproduce clinically relevant information (since the relative activation time curve is used for determining the site of ablation).

**Figure 6** shows a box plot for each value of n. Each box in the box plot represents the root mean squared error between the relative activation time curves obtained from the average across 20 model simulations (with anisotropic diffusion and without) and that obtained from a clinically recorded ectopic event. The case n = 0 enables us to consider the limit of the goodness of fit between the model and the clinical data. This is effectively the intrinsic error attributable to the choice of model. For subsequent plots, n random channels were removed from the training set (simulating the effect of lost information due to poor contact). Different time intervals and different signals were used for each calculation so that the error distribution presented is as close as possible to the errors that we might expect to observe in clinic. This is important as it minimizes the likelihood of observations simply being due to an artifact of the ectopic event chosen for the fit.

For up to 3 channels reconstructed, the median and maximum errors do not significantly increase over that of the control whether or not anisotropy is considered. This is an important result as it demonstrates reconstructing up to three missing channels is not a significant source of additional error and therefore the model as presented may ultimately have clinical use under these conditions. Removing more than 3 channels leads to information loss resulting in outliers with statistically significant errors (see the cases for 4 and 5 channels removed). This demonstrates the limit of the number of missing channels that the considered models can reliably reconstruct.

To consider the whether the assumption of anisotropic diffusion is significant, we performed an unequal variances t-test (so-called Welch's t-test) to test whether the errors from each model could have come from a distribution with the same mean. This test consistently showed no significant difference (p > 0.05), except for the case of 3 signals reconstructed (p = 0.0414). This suggests that whilst anisotropy is clearly important in terms of the underlying physiology, it does not significantly affect the quality of model fit to the phenomenology of the recorded signals. This is an important consideration as calibration of model parameters is more efficient under the assumption of isotropic diffusion.

# Potential Clinical Application

To test the potential of this technique to aid the clinical procedure, we presented a cardiologist specializing in pulmonary vein isolation therapy, with three variations of clinical recordings collected from three patients:

data to the cardiologist who made a decision about the tissue underlying the channels they would ablate in each case. This process was repeated for patients 2 and 3. In all three cases reconstruction of the missing signals resulted in an improved clinical decision in contrast to only using the recordings with missing signals.


These scenarios are illustrated in **Figure 7**. The cardiologist was unaware of the origin of each recording, and to avoid bias, the recordings were supplied in a random order. The following TABLE 2 | Effect of signal reconstruction on clinical decision.


*The channel corresponding to the area the cardiologist would ablate given the recording with channels missing, the original data, and the data with the missing channels reconstructed via the model. The decision made with the reconstructed signals is closer to the original than the decision made using the missing recordings, demonstrating that the model reconstruction has minimized the impact of the information loss on the clinical outcome. Recordings from Patient 1 are given in* Figure 7*.*

results were obtained (summarized in **Table 2** for convenience). For patient 1, given the original data, the first point of ablation would have been around PV 5-6, with PV 15-16 noted as a second choice. With channels 15-16, 17-18, and 19-20 removed, only PV 5-6 was identified as the only appropriate ablation zone. When these channels were reconstructed by the model, PV 17-18 was identified as the second choice of ablation target. For this patient, the model has helped to identify a second relevant ablation target that was not identified when channels were missing. If initial ablation is not successful, the cardiologist will ablate in the area surrounding the target area, hence an initial estimate closer to the optimal location will result in successful isolation using fewer radiofrequency pulses. This will result in a smaller region of tissue being ablated and a shorter procedure.

In the second patient presented, a clear earliest spike time was present on PV 5-6. The removal of PV 5-6 and its neighbors led to PV 9-10 being identified as an ablation target. In this case, the reconstruction led to the same zone being targeted as the original signal, while the estimation with the recordings missing was two channels away. For this patient, reconstruction of the missing recordings led the cardiologist directly to the optimal decision.

In the final patient, the earliest spike, on PV 7-8, was removed, along with PV 5-6 and 9-10. As previously, these missing channels shifted the chosen ablation target by 2 channels. The reconstructed signal led to a target selected which was closer to the target chosen with all information present. As with patient 1, we infer this result as satisfactory, as starting closer to the optimal target will lead to quicker isolation of the pulmonary vein.

In all three cases, the missing channels influenced the decision made by the cardiologist, demonstrating the potential impact of information loss in clinic. However, when the cardiologist used the recordings combined with the signals reconstructed by the mathematical model to make a decision, the decision made was closer to the decision that would have been made had all information been present. Whilst these results provide only limited proof of concept at this stage, assuming the original data and clinical decision to represent the "ground truth," then we believe there is significant potential for our approach to minimize the effect of this lost information.

#### DISCUSSION

In this study we have demonstrated that key features of pulmonary vein recordings can be generated by a phenomenological model, in this case the BOCF model. Calibrating parameters of the BOCF model using the postfibrillation AP of atrial myocytes, simulated using the biophysical Courtemanche model, provides a method for rapid simulation of atrial cells afflicted by AF-induced electrical remodeling. This is in contrast to more detailed biophysical models (Courtemanche et al., 1998; Nygren et al., 1998; Priebe and Beuckelmann, 1998; Iyer et al., 2004; ten Tusscher, 2004) which may take several hours to produce an output. Given that pulmonary vein isolation therapy typically lasts at most 2 h, having a mathematical model that can run in close to real time, is a critical advantage when assessing suitability as a potential clinical decision support system.

Toward this aim, a primary result of this study was to model the phenomenology of recordings from the lasso catheter used during the pulmonary vein isolation therapy of AF. We found that the resulting model simulations accurately reproduce the relative activation time curve seen in recordings from patients undergoing this procedure. The pulmonary vein recordings made in this process are not always complete; there is often the complete loss of some of the recording channels. This is most commonly due to poor contact made between electrodes on the catheter and the pulmonary vein itself. This loss of information can result in non-optimal clinical decision making during the isolation therapy procedure. To address this issue we have demonstrated that a mathematical model fitted to the available channels of the data can be used to reconstruct those missing channels and we presented evidence in support of the accuracy of these reconstructions through comparison to clinical data. Of note, we find that up to three channels can be reconstructed without significantly increases the inherent error due to the use of a model. The results show that, in principle, these ideas could be adapted as part of a clinical decision support system, which could be run in the operating theater and provide information to the cardiologist during the procedure.

A potential limitation of this study is the loss of biological detail arising from our use of a phenomenological model over a biophysical one. However, it is important to note that the appropriateness of any mathematical model is dependent on the challenge it is designed to address. Here, we focus on the case of pulmonary vein isolation therapy, where a cardiologist is using recordings of the emergent electrical signal from the heart to make rapid decisions about regions of the heart to ablate. Consequently a model that can capture the phenomenology of these recordings (which ultimately are what the cardiologist is using to guide their decision making) is a valid approach and does not require a detailed analysis of the contribution of ionic channels and other physical quantities involved in AP propagation.

In our current model a number of assumptions have been made, most importantly regarding the conductance and the geometry. Whilst we account for anisotropic diffusion by considering additional parameters, the diffusion tensor used is still homogeneous across conductive tissue, and the wave approaching the conduction gap is planar. This is primarily since detailed fiber direction information would not be accessible to the cardiologist during the clinical procedure. A further key assumption is the approximation of the pulmonary vein sleeve as a cylinder. In the clinical procedure, the relevant region of cardiac tissue is not only the pulmonary vein sleeve, but also the atrial tissue surrounding the ostium. However, while tissue expansion and asymmetry of an anatomically accurate domain may affect the results for a given set of parameters, the signal reconstruction technique incorporates the fitting of the parameters to the available signals, which will account for the impact of these assumptions. Further geometrical assumptions, such as the angles between the incoming wave, the ablation line and the recording catheter, can not be quantified using standard clinical equipment and so we do not consider them in the current study. We also assume that all cells are free atrial wall myocytes, rather than pulmonary vein myocytes which have a shorter AP duration and amplitude in addition to a lower upstroke velocity in comparison to the left atrium (Mahida et al., 2015). However, under current clinical practice, it is not possible to identify which areas of the pulmonary vein ostium is populated by pulmonary vein myocytes as opposed to those of the atrial wall.

While the model developed in this paper has been developed with clinical applicability in mind, future work will be necessary to establish the ultimate validity of and optimize this approach in a clinical context. In particular it is important to establish the optimal level of detail of model required to reconstruct missing signals, and whether additional detail can improve the accuracy of the methods, given the constraints of time and recording protocols in standard clinical practice. Further, clinical metadata regarding the locations and times at which ablation was performed on the patient is typically not collected during the ablation procedure, so it is difficult to infer the optimal ablation zone from patient data. The availability of such data would open up many new lines of research, including the use of either phenomenological or biophysically detailed patientspecific models to estimate the optimal ablation site directly.

# AUTHOR CONTRIBUTIONS

HG and JT provided concept; designed and executed the study; analyzed and interpreted the data. GT provided data; performed analysis of clinical and reconstructed signals. All authors wrote and approved the final manuscript.

# FUNDING

JT acknowledges the generous support of the Wellcome Trust via Institutional Strategic Support Award (WT105618MA). JT further acknowledges financial support of the EPSRC via grant EP/N014391/1. HG acknowledges the financial support of the University of Exeter.

# ACKNOWLEDGMENTS

We would like to acknowledge Rémi Dubois and Yves Coudière for valuable discussion and insight in the development of the model.

#### Green et al. Mathematical Reconstruction of PV Recordings

# REFERENCES


simulation study," in Engineering in Medicine and Biology Society (EMBC), 2010 Annual International Conference of the IEEE (IEEE) (Buenos Aires).


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

The reviewer NT and handling Editor declared their shared affiliation, and the handling Editor states that the process met the standards of a fair and objective review.

Copyright © 2017 Green, Thomas and Terry. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Analysis of a Compartmental Model of Endogenous Immunoglobulin G Metabolism with Application to Multiple Myeloma

Felicity Kendrick <sup>1</sup> , Neil D. Evans <sup>1</sup> , Bertrand Arnulf <sup>2</sup> , Hervé Avet-Loiseau<sup>3</sup> , Olivier Decaux <sup>4</sup> , Thomas Dejoie<sup>5</sup> , Guillemette Fouquet <sup>6</sup> , Stéphanie Guidez <sup>7</sup> , Stéphanie Harel <sup>2</sup> , Benjamin Hebraud<sup>8</sup> , Vincent Javaugue<sup>7</sup> , Valentine Richez <sup>9</sup> , Susanna Schraen<sup>6</sup> , Cyrille Touzeau<sup>5</sup> , Philippe Moreau<sup>5</sup> , Xavier Leleu<sup>7</sup> , Stephen Harding<sup>10</sup> and Michael J. Chappell <sup>1</sup> \*

<sup>1</sup> School of Engineering, University of Warwick, Coventry, UK, <sup>2</sup> Hôpital Saint-Louis, Paris, France, <sup>3</sup> Unité de Génomique du Myélome, Institut Universitaire du Cancer de Toulouse Oncopole, Toulouse, France, <sup>4</sup> Centre Hospitalier Universitaire de Rennes, Rennes, France, <sup>5</sup> Centre Hospitalier Universitaire de Nantes, Nantes, France, <sup>6</sup> Centre Hospitalier Régional Universitaire de Lille, Lille, France, <sup>7</sup> Centre Hospitalier Universitaire de Poitiers, Poitiers, France, <sup>8</sup> Centre Hospitalier Universitaire de Toulouse, Toulouse, France, <sup>9</sup> Centre Hospitalier Universitaire de Nice, Nice, France, <sup>10</sup> Department of Research and Development, The Binding Site Group Limited, Birmingham, UK

#### Edited by:

Krasimira Tsaneva-Atanasova, University of Exeter, UK

#### Reviewed by:

Marc Thilo Figge, Leibniz-Institute for Natural Product Research and Infection Biology, Hans-Knoell-Institute, Germany Stefano Severi, University of Bologna, Italy

#### \*Correspondence:

Michael J. Chappell m.j.chappell@warwick.ac.uk

#### Specialty section:

This article was submitted to Computational Physiology and Medicine, a section of the journal Frontiers in Physiology

Received: 30 November 2016 Accepted: 24 February 2017 Published: 17 March 2017

#### Citation:

Kendrick F, Evans ND, Arnulf B, Avet-Loiseau H, Decaux O, Dejoie T, Fouquet G, Guidez S, Harel S, Hebraud B, Javaugue V, Richez V, Schraen S, Touzeau C, Moreau P, Leleu X, Harding S and Chappell MJ (2017) Analysis of a Compartmental Model of Endogenous Immunoglobulin G Metabolism with Application to Multiple Myeloma. Front. Physiol. 8:149. doi: 10.3389/fphys.2017.00149 Immunoglobulin G (IgG) metabolism has received much attention in the literature for two reasons: (i) IgG homeostasis is regulated by the neonatal Fc receptor (FcRn), by a pH-dependent and saturable recycling process, which presents an interesting biological system; (ii) the IgG-FcRn interaction may be exploitable as a means for extending the plasma half-life of therapeutic monoclonal antibodies, which are primarily IgG-based. A less-studied problem is the importance of endogenous IgG metabolism in IgG multiple myeloma. In multiple myeloma, quantification of serum monoclonal immunoglobulin plays an important role in diagnosis, monitoring and response assessment. In order to investigate the dynamics of IgG in this setting, a mathematical model characterizing the metabolism of endogenous IgG in humans is required. A number of authors have proposed a two-compartment nonlinear model of IgG metabolism in which saturable recycling is described using Michaelis–Menten kinetics; however it may be difficult to estimate the model parameters from the limited experimental data that are available. The purpose of this study is to analyse the model alongside the available data from experiments in humans and estimate the model parameters. In order to achieve this aim we linearize the model and use several methods of model and parameter validation: stability analysis, structural identifiability analysis, and sensitivity analysis based on traditional sensitivity functions and generalized sensitivity functions. We find that all model parameters are identifiable, structurally and taking into account parameter correlations, when several types of model output are used for parameter estimation. Based on these analyses we estimate parameter values from the limited available data and compare them with previously published parameter values. Finally we show how the model can be applied in future studies of treatment effectiveness in IgG multiple myeloma with simulations of serum monoclonal IgG responses during treatment.

Keywords: biomedical systems, lumped-parameter systems, identifiability, parameter identification, sensitivity analysis, immunoglobulin G, metabolism, multiple myeloma

**192**

# 1. INTRODUCTION

Immunoglobulin G (IgG) is protected from degradation by the neonatal Fc receptor (FcRn), resulting in an unusually long metabolic half-life at normal concentrations (∼23 days; Rosenthal and Tan, 2010) and a high serum concentration in healthy adults (10–16 g l−<sup>1</sup> ; Hall and Yates, 2010). The half-life of IgG is not constant, but varies with its serum concentration, due to saturation of recycling receptors. Elevated IgG concentrations saturate receptors such that a greater proportion of circulating IgG is degraded; conversely at low concentrations a greater proportion of IgG is recycled and the half-life is extended. Circulating IgG is internalized into intracellular endosomes in order to be degraded. FcRn expressed within the cells binds IgG inside the acidic environment of endosomes with a pH-dependent affinity. FcRn then sequesters the bound IgG away from the degradation pathway and back to the cell membrane, releasing it once again into the circulation. Those IgG molecules that are not bound to FcRn continue to follow the pathway to be degraded in lysosomes (Junghans and Anderson, 1996).

In multiple myeloma, clonal plasma cells in the bone marrow secrete a unique, monoclonal immunoglobulin (Ig). Half of patients have IgG-producing clones and are said to have IgG myeloma (Anderson, 2003). The monoclonal Ig produced by the cancer offers a convenient opportunity for clinicians to monitor the response of the tumor to therapy via the secreted protein, which is readily quantified in a blood sample. The cancer itself is only accessible by bone marrow biopsy or aspirate, both of which are unpleasant, invasive procedures. The concentration of monoclonal Ig in the blood is therefore the preferred measure by which the tumor is monitored; patient monitoring in clinical trials and the non-trial setting alike is heavily reliant on measurements of monoclonal Ig concentration in the blood (Kumar et al., 2016).

IgG myeloma patients typically present with an elevated concentration of serum monoclonal IgG. During treatment, the malignant plasma cells are killed and the production rate of monoclonal IgG correspondingly decreases, resulting in a fall in serum monoclonal IgG concentration. In this way, the serum monoclonal IgG response is used as a surrogate for the tumor response to treatment. The possible effects of the metabolism of IgG on its application as a cancer marker in multiple myeloma have been little studied, but are acknowledged in the literature. Sullivan and Salmon (1972) first brought the issue of IgG metabolism to the attention of the multiple myeloma community. Serum monoclonal IgG concentration, plasma volume, and IgG synthesis rate per cell were measured in 11 patients with IgG myeloma. Calculating the fractional catabolic rate of IgG using the equation provided by Waldmann and Strober (1969), Sullivan and Salmon (1972) estimated the tumor burden at a number of time points during treatment for each patient, concluding that increases and decreases in the tumor burden were underestimated by increases and decreases in monoclonal IgG. More recently, Bradwell et al. (2013), Koulieris et al. (2012), and Durie et al. (2006) have cited the concentration-dependent metabolism of IgG as a possible explanation for why monoclonal IgG may be seen as an unreliable response marker in multiple myeloma.

In order to investigate the dynamics of IgG in multiple myeloma, a mathematical model characterizing the metabolism of endogenous IgG in humans is required. Many mathematical models of IgG metabolism have been published in the literature (more than 20 at the time of writing), usually with the aim of describing the pharmacokinetics of therapeutic monoclonal antibodies that are similarly regulated by FcRn. Many of the models are therefore pharmacokinetic in nature: their parameter values are obtained from animal experiments and they may be physiologically based, with up to ten organs and the lymphatic system explicitly represented in the model (Hansen and Balthasar, 2003; Ferl et al., 2005; Garg and Balthasar, 2007; Fang and Sun, 2008; Urva et al., 2010; Chen and Balthasar, 2012; Deng et al., 2012; Xiao, 2012; Yan et al., 2012; Fronton et al., 2014; Ng et al., 2014). Physiologically based pharmacokinetic (PBPK) models may be unnecessarily complex for investigating serum IgG dynamics in multiple myeloma, particularly considering the limited human-derived data that are available for parameter estimation. More suitably, several authors have proposed a comparatively simple two-compartment model of IgG metabolism in which saturable recycling by FcRn is described using Michaelis–Menten kinetics (Waldmann and Strober, 1969; Kim et al., 2007; Hattersley et al., 2013). They also provide certain parameter values for humans.

In order to investigate serum IgG responses in IgG multiple myeloma, the parameter values used are highly important in order to have confidence in model-based predictions. Parameter estimation using limited data is an important problem in the mathematical modeling of physiological systems. Methods for parameter identification including structural identifiability analysis and sensitivity analysis should be used in the early stages of the model validation process; specifically, these analyses address whether parameters can be estimated from the available measurements and, where further experiments are possible, inform experiment design. In this paper we analyse the nonlinear two-compartment model of IgG metabolism (Waldmann and Strober, 1969; Kim et al., 2007; Hattersley et al., 2013) and the available measurements in humans for structural identifiability and sensitivity, in order to make optimal use of the limited data available in the literature. Having considered the identifiability problem, we estimate parameter values from the data, with the intention that the model can be used in the future to make generalized predictions for patients.

# 2. METHODS

# 2.1. Experimental Data

Data for parameter estimation were obtained from the literature. Studies of protein metabolism involve intravenously injecting a subject with radioisotopically labeled protein, known as a tracer, and then monitoring the proportion of tracer remaining in the blood over a period of time following administration. Radioactive tracers allow for distinction between the injected dose and the endogenously produced protein, enabling direct visualization of the distribution and elimination processes of a protein despite it

being homeostatic. The radioactive label (usually iodine) remains bound to the protein until the protein is degraded, at which point the label is released and rapidly excreted in urine. Several tracer studies were performed for IgG in humans in the middle of the last century and the results collated by Waldmann and Strober (1969).

#### 2.1.1. Individual Timecourse Data

The data from a single subject consist of the timecourse of the proportion of an administered dose of radiolabeled IgG remaining in plasma and the proportion remaining in the whole body, calculated by subtracting the radioactivity in urine from the administered dose. The data collected from an individual are shown in **Figure 1A**. The data have been extracted from a plot by Solomon et al. (1963) using OriginPro<sup>1</sup> . Seven plots of this type have been found by the authors in the literature. The data in these plots are assumed to arise from seven individuals to whom we refer as subjects A–G. The data for subjects A–D are taken from Solomon et al. (1963), for subjects E and F from Waldmann and Terry (1990) and for subject G from Waldmann and Strober (1969). Subjects A and C have IgG myeloma, subject D has macroglobulinemia and subject E has familial hypercatabolic hypoproteinemia. These conditions do not preclude the subjects from this study but there may be a correlation between these conditions and individuals' parameter values of IgG metabolism; this is discussed in Section 3.2.

#### 2.1.2. Fractional Catabolic Rate and Half-Life

In compartmental analysis, parameters are often considered as either micro constants or macro (hybrid) constants. The micro constants are dependent upon the assumed structure of the compartmental model, whereas the macro constants can be determined directly from the profile of concentration or radioactivity over time, such as the exponents of a multiexponential profile, and do not assume a particular model structure (Riviere, 2011).

Waldmann and Strober (1969) have plotted two macro parameters, the fractional catabolic rate (FCR) and the terminal half-life (T1/2), that can be calculated directly from an individual subject's timecourse of radioactivity. The FCR is defined as the fraction of the administered IgG in plasma that is catabolized per day and is calculated by dividing the rate at which the administered dose leaves the body at any time t > 0 by the amount of the dose remaining in plasma at that time. The rate at which the dose leaves the body is given by the slope of the timecourse of the dose remaining in the whole body. The T1/<sup>2</sup> is defined as the time taken for half of the administered IgG to be eliminated, after completion of the distribution phase. This is obtained from the terminal slope of the timecourse observations plotted on a logarithmic scale.

The plots of FCR and T1/<sup>2</sup> provided by Waldmann and Strober (1969) are reproduced in **Figures 1B,C**. Each point in these plots was obtained from the timecourse data of a single subject, an example of which is shown in **Figure 1A**. The parameters have been taken from a large number of subjects (FCR − 41 subjects; T1/<sup>2</sup> − 44 subjects) with a wide range of plasma concentrations of IgG, in order to capture the concentration-dependent behavior of IgG metabolism. Macro parameters are functions of the micro parameters of the assumed compartmental structure therefore in this paper the FCR and T1/<sup>2</sup> data are used in the estimation of the parameters of the underlying compartmental model.

# 2.2. Model of Endogenous IgG Metabolism

The nonlinear two-compartment model of endogenous IgG metabolism, with Michaelis–Menten kinetics describing the rate of recycling by FcRn receptors (Waldmann and Strober, 1969; Kim et al., 2007; Hattersley et al., 2013), is given by:

$$\begin{aligned} \dot{x}\_1(t) &= -\left(k\_{21} + k\_{31} - \frac{V\_{\text{max}}}{K\_{\text{M}} + \varkappa\_1(t)}\right) x\_1(t) + k\_{12} x\_2(t) + I(t) \\ \dot{x}\_2(t) &= k\_{21} x\_1(t) - k\_{12} x\_2(t) \end{aligned} \tag{1}$$

where x1(t) and x2(t) represent the quantities in µmol of IgG in plasma and in a peripheral compartment, respectively. I(t) represents the synthesis of IgG into plasma in µmol day−<sup>1</sup> . Rate constants kij represent material flow from compartment j to compartment i. The rate constant of the removal of IgG from the plasma compartment into intracellular endosomes for degradation is given by k31, with the indices denoting the transfer from plasma to a third compartment representing intracellular endosomes, which is omitted from the model. The rate of FcRnmediated recycling, as a fraction of the quantity of IgG in plasma, is given by Vmax/(K<sup>M</sup> + x1(t)). The parameters Vmax and K<sup>M</sup> are the maximum absolute rate of FcRn-mediated recycling in µmol day−<sup>1</sup> and the Michaelis constant, representing the quantity of IgG in plasma in µmol at which the absolute recycling rate is half Vmax. Those IgG molecules which are removed from the plasma compartment into intracellular endosomes and which do not get recycled by FcRn are degraded in lysosomes. The amino acid products of lysosomal degradation are reused in the synthesis of new proteins (Appelqvist et al., 2013). A schematic of the system model is shown in **Figure 2**. **Table 1** summarizes the model states and parameters.

All states and parameters can only take non-negative values. The rate at which IgG is recycled cannot exceed the rate at which it leaves the plasma compartment to be degraded in intracellular endosomes; equivalently, the net elimination rate must be positive for all states and input rates: k<sup>31</sup> − Vmax KM > 0.

When the production rate of IgG is assumed constant, I(t) = I0, in order to determine the model's steady states, solving x˙1(t) = 0 and x˙2(t) = 0 simultaneously gives the equilibrium point:

$$
\hat{\mathbf{x}}\_1 = \frac{-k\_{31}K\_{\mathbf{M}} + I\_0 + V\_{\text{max}} + \sqrt{4k\_{31}K\_{\mathbf{M}}I\_0 + (-k\_{31}K\_{\mathbf{M}} + I\_0 + V\_{\text{max}})^2}}{2k\_{31}}
$$

$$
\hat{\mathbf{x}}\_2 = \frac{k\_{21}}{k\_{12}}\hat{\mathbf{x}}\_1. \tag{2}
$$

The stability of this equilibrium point for all parameter values is demonstrated in Section 2.2.1.

<sup>1</sup>Originpro (2016). Windows. Northampton: OriginLab Corporation.

FIGURE 1 | (A) Proportion of administered IgG remaining in plasma (blue circles) and the body (red triangles) in a typical normal subject; data from Solomon et al. (1963). Plasma concentration dependence of (B) fractional catabolic rate (FCR) and (C) half-life (T1/<sup>2</sup> ) of IgG; redrawn from Waldmann and Strober (1969) with permission from S. Karger AG, Basel.

#### 2.2.1. Stability of Steady States

Linearizing the system described by Equation (1) about the equilibrium point gives:

$$
\begin{pmatrix}
\dot{\boldsymbol{x}}\_1(t) \\
\dot{\boldsymbol{x}}\_2(t)
\end{pmatrix} = \begin{pmatrix}
k\_{21} & -k\_{12}
\end{pmatrix} \begin{pmatrix}
\boldsymbol{x}\_1(t) \\
\boldsymbol{x}\_2(t)
\end{pmatrix}. \tag{3}
$$

According to the Routh–Hurwitz stability criterion, the two-state system is stable provided the coefficients of the characteristic polynomial of the linearized system are positive (Routh, 1877). The coefficients of the characteristic polynomial are given by:

$$\begin{aligned} a\_2 &= 1\\ k\_{31}K\_{\mathcal{M}}^2 - K\_{\mathcal{M}}V\_{\text{max}} + 2k\_{31}K\_{\mathcal{M}}\hat{\boldsymbol{x}}\_1 + k\_{31}\hat{\boldsymbol{x}}\_1^2 + k\_{12}(K\_{\mathcal{M}} + \hat{\boldsymbol{x}}\_1)^2\\ a\_1 &= \frac{(K\_{\mathcal{M}} + \hat{\boldsymbol{x}}\_1)^2}{(K\_{\mathcal{M}} + \hat{\boldsymbol{x}}\_1)^2}\\ a\_0 &= \frac{k\_{12}\left(k\_{31}\left(K\_{\mathcal{M}} + \hat{\boldsymbol{x}}\_1\right)^2 - K\_{\mathcal{M}}V\_{\text{max}}\right)}{\left(K\_{\mathcal{M}} + \hat{\boldsymbol{x}}\_1\right)^2}. \end{aligned} \tag{4}$$

The denominators in the expressions for a<sup>0</sup> and a<sup>1</sup> are always positive. All parameters and the steady state xˆ<sup>1</sup> are positive. The sign of a<sup>0</sup> is thus given by the sign of k<sup>31</sup> K<sup>M</sup> + ˆx<sup>1</sup> <sup>2</sup> <sup>−</sup> <sup>K</sup>MVmax . For stability of the equilibrium point it is necessary that k<sup>31</sup> K<sup>M</sup> + ˆx<sup>1</sup> <sup>2</sup> <sup>−</sup> <sup>K</sup>MVmax > 0. This condition is met when k31K 2 <sup>M</sup> − KMVmax > 0, or equivalently

TABLE 1 | States and parameters of IgG metabolism model.


k31− Vmax KM > 0. The sign of a<sup>1</sup> is given by the sign of its numerator, k31K 2 <sup>M</sup> − KMVmax + 2k31KMxˆ<sup>1</sup> + k31xˆ 2 <sup>1</sup> + k12(K<sup>M</sup> + ˆx1) <sup>2</sup> + k21(K<sup>M</sup> + ˆx1) 2 . Once again, the sign of a<sup>1</sup> is positive provided that k<sup>31</sup> − Vmax KM > 0.

Both of the coefficients a<sup>0</sup> and a<sup>1</sup> are positive provided that all parameter values are positive and k<sup>31</sup> − Vmax KM > 0. Referring back to Equation (1), that is the condition which ensures a positive IgG elimination rate for all x<sup>1</sup> > 0. A negative elimination rate does not make sense physiologically and as such parameter values are not permitted which violate this condition. The equilibrium point is thus stable for all permitted parameter values.

#### 2.3. Model of Observed Measurements

In this section we consider how the observable measurements (timecourse of radioactivity, FCR and T1/2) relate to the system model. Tracer experiments are designed specifically so that the tracer-labeled protein observes linear kinetics, despite the mode of metabolism being in fact nonlinear (Anderson, 1983). A linear model describing the timecourse observations is derived here.

#### 2.3.1. Timecourse Observations

Assuming that the radiolabeled IgG dose and unlabeled endogenous IgG are indistinguishable by the system, both are described by the model in Equation (1). The injected and endogenous IgG can be explicitly represented by letting xi(t) = xi,T(t) + xi,E(t) for i = 1, 2, with "T" denoting tracer and "E" denoting endogenous IgG. Then, from Equation (1), the dynamics of labeled and unlabeled IgG are given by:

$$\begin{aligned} \dot{\mathbf{x}}\_{\mathrm{I},\mathrm{T}}(t) &= -\left(k\_{21} + k\_{31} - \frac{V\_{\mathrm{max}}}{K\_{\mathrm{M}} + \mathbf{x}\_{\mathrm{1},\mathrm{E}}(t) + \mathbf{x}\_{\mathrm{1},\mathrm{T}}(t)}\right) \mathbf{x}\_{\mathrm{1},\mathrm{T}}(t) \\ &+ k\_{12} \mathbf{x}\_{\mathrm{2},\mathrm{T}}(t) \\ \dot{\mathbf{x}}\_{\mathrm{2},\mathrm{T}}(t) &= k\_{21} \mathbf{x}\_{\mathrm{1},\mathrm{T}}(t) - k\_{12} \mathbf{x}\_{\mathrm{2},\mathrm{T}}(t) \\ \dot{\mathbf{x}}\_{\mathrm{1},\mathrm{E}}(t) &= -\left(k\_{21} + k\_{31} - \frac{V\_{\mathrm{max}}}{K\_{\mathrm{M}} + \mathbf{x}\_{\mathrm{1},\mathrm{E}}(t) + \mathbf{x}\_{\mathrm{1},\mathrm{T}}(t)}\right) \mathbf{x}\_{\mathrm{1},\mathrm{E}}(t) \\ &+ k\_{12} \mathbf{x}\_{\mathrm{2},\mathrm{E}}(t) + I\_{\mathrm{E}} \\ \dot{\mathbf{x}}\_{\mathrm{2},\mathrm{E}}(t) &= k\_{21} \mathbf{x}\_{\mathrm{1},\mathrm{E}}(t) - k\_{12} \mathbf{x}\_{\mathrm{2},\mathrm{E}}(t) \end{aligned} \tag{5}$$

where xi,T(t) and xi,E(t) represent the quantities in µmol of radiolabeled and endogenous IgG in compartment i, respectively.

The intravenous bolus injection of tracer can be treated as a non-zero initial condition for x1,T(t); thus the initial conditions of the tracer are given by:

$$\begin{aligned} \mathbf{x}\_{1,\Gamma}(0) &= D\\ \mathbf{x}\_{2,\Gamma}(0) &= 0 \end{aligned} \tag{6}$$

where D is the dose of tracer in µmol. The production rate of endogenous IgG, I<sup>E</sup> µmol day−<sup>1</sup> , is assumed constant. The initial conditions of the endogenous IgG are given by the equilibrium point in Section 2.2, with I<sup>0</sup> = IE. The experimenter measures the proportion of the initially injected radioactivity in plasma and in the whole body. The observation functions are thus given by:

$$\begin{aligned} \chi\_1(t) &= \chi\_{1,\Gamma}(t)/D \\ \chi\_2(t) &= \left(\chi\_{1,\Gamma}(t) + \chi\_{2,\Gamma}(t)\right)/D. \end{aligned} \tag{7}$$

A sufficiently small quantity of radiolabeled IgG, typically 0.5– 1 mg (3.33 × 10−3–6.67 × 10−<sup>3</sup> µmol) (Solomon et al., 1963), is administered into plasma so as not to perturb the steady state of the endogenous protein. Thus x1,E and x2,E can be assumed constant. Then the equations describing the tracer dynamics are no longer coupled with those describing the endogenous IgG dynamics. A second assumption is required in order to derive a linear model: the quantity of tracer, x1,T(t), is assumed to be much smaller than the quantity of the subject's endogenous IgG, x1,E. Thus the term <sup>V</sup>max K<sup>M</sup> + x1,E + x1,T(t) can be approximated by <sup>V</sup>max K<sup>M</sup> + x1,E . In this way, the elimination rate of the tracer is determined by the quantity of the subject's endogenous plasma IgG only. A further simplification can be made by noticing that for a linear model, the initial conditions and observation gain cancel out (see Equations 6, 7). The equations describing the tracer kinetics are thus given by:

$$\begin{aligned} \dot{x}\_{1, \text{P}}(t) &= -\left(k\_{21} + k\_{31} - \frac{V\_{\text{max}}}{K\_{\text{M}} + \varkappa\_{1, \text{E}}}\right) \varkappa\_{1, \text{P}}(t) + k\_{12} \varkappa\_{2, \text{P}}(t) \\ \dot{x}\_{2, \text{P}}(t) &= k\_{21} \varkappa\_{1, \text{P}}(t) - k\_{12} \varkappa\_{2, \text{P}}(t) \end{aligned} \tag{8}$$

where x1,P(t) and x2,P(t) represent the proportion of the radiolabeled IgG dose D in the central and peripheral compartments, respectively, at time t. x1,E represents the quantity of the subject's endogenous IgG in the central compartment, which is assumed to remain in steady state. All other parameters are defined as in Section 2.2.

The initial conditions of the model are now given by:

$$\begin{aligned} \varkappa\_{1,\mathcal{P}}(0) &= 1 \\ \varkappa\_{2,\mathcal{P}}(0) &= 0. \end{aligned} \tag{9}$$

The corresponding observation functions are given by:

$$\begin{aligned} \chi\_1(t) &= \chi\_{1,\mathcal{P}}(t) \\ \chi\_2(t) &= \chi\_{1,\mathcal{P}}(t) + \chi\_{2,\mathcal{P}}(t) \end{aligned} \tag{10}$$

The linearized model represented by Equations (8–10) is a valid approximation of the nonlinear model (Equations 5–7) when

the administered dose of radiolabeled IgG, D, is sufficiently smaller than the quantity of endogenous IgG in plasma at t = 0, x1,E(0). In **Figure 3** simulations of the nonlinear model and the linearized model are compared. The parameter values used are those estimated in this paper and summarized in **Table 6**. The production rate of endogenous IgG is set to I<sup>E</sup> = 0.0727 µmol day−<sup>1</sup> to give x1,E(0) = 5 µmol for the nonlinear model and x1,E = 5 µmol for the linearized model, representing the lower limit of the quantities of endogenous IgG in plasma seen in the data. In **Figure 3A** the tracer dose D is 0.01 µmol, representing the upper limit of administered tracer doses (Solomon et al., 1963). The nonlinear and linearized model responses are indistinguishable, illustrating that for typical tracer doses the linearized model is a valid approximation of the nonlinear model. In **Figure 3B** the tracer dose D is 10 µmol, 1,000 times larger; at this point the assumptions weaken and there is a noticeable difference between the responses of the two models.

#### 2.3.2. Fractional Catabolic Rate and Half-Life

The FCR is defined as the proportion of the radiolabeled IgG in plasma that is catabolized per day. From Equation (8) this is given by:

$$\text{FCR} = k\_{\text{31}} - \frac{V\_{\text{max}}}{K\_{\text{M}} + \varkappa\_{\text{1,E}}}.\tag{11}$$

The terminal half-life, T1/<sup>2</sup> , is related to the elimination phase of the kinetics, after the distribution phase is complete. The model described by Equation (8) is a linear two-compartment model with the solutions for x1,P(t) and x2,P(t) given by the bi-exponential functions:

$$\begin{aligned} \varkappa\_{1, \mathcal{P}}(t) &= A\_{11} \exp(\lambda\_1 t) + A\_{12} \exp(\lambda\_2 t) \\ \varkappa\_{2, \mathcal{P}}(t) &= A\_{21} \exp(\lambda\_1 t) + A\_{22} \exp(\lambda\_2 t) \end{aligned} \tag{12}$$

where Aij and λ<sup>j</sup> are macro constants, with |λ1| > |λ2|. By definition, T1/<sup>2</sup> is given by:

$$T\_{1\ne} = -\frac{\log 2}{\lambda\_2}.\tag{13}$$

Solving Equation (8) for λ<sup>2</sup> and substituting into Equation (13) gives the following expression for T1/<sup>2</sup> in terms of the micro parameters of the model:

#### 3. RESULTS

# 3.1. Structural Identifiability of Model Parameters

Structural identifiability addresses the question of whether model parameters can be uniquely identified from available observations, under the assumption of the availability of ideal (i.e., noise-free) and continuous observational data. Structural identifiability of parameters does not imply that they are identifiable in practice, from observations that are inevitably measured with noise; therefore in this paper structural identifiability analysis is used alongside sensitivity analysis.

Here we determine which of the model parameters are structurally uniquely identifiable from the following measurements: an individual subject's timecourse, FCR vs. the quantity of endogenous IgG in plasma, and T1/<sup>2</sup> vs. the quantity of endogenous IgG in plasma.

#### 3.1.1. Individual Timecourse

Here the transfer function method is used (Bellman and Åström, 1970). To apply this approach the system described by Equations (8–10) is re-written in vector-matrix notation as

$$\begin{aligned} \dot{\mathbf{x}}(t,p) &= A(p)\mathbf{x}(t,p) + B(p)u(t) \\ \mathbf{x}(0,p) &= 0 \\ \mathbf{y}(t,p) &= \mathbf{C}(p)\mathbf{x}(t,p), \end{aligned} \tag{16}$$

where x(t, p) = x1,P(t), x2,P(t) , and y(t, p) = y1(t), y2(t) are column vectors representing the state and the observation, respectively. u(t) represents the single input to the system, an impulse at time t = 0, given by u(t) = δ(t). A(p) and C(p) are 2 × 2 matrices and B(p) is a column vector. A(p), B(p), and C(p) are given by:

$$A(\boldsymbol{\rho}) = \begin{pmatrix} -\left(k\_{21} + k\_{31} - \frac{V\_{\text{max}}}{K\_{\text{M}} + \mathbf{x}\_{1,\text{E}}}\right) & k\_{12} \\ k\_{21} & -k\_{12} \end{pmatrix},$$

$$B(\boldsymbol{\rho}) = \begin{pmatrix} 1 \\ 0 \end{pmatrix}, \mathbf{C}(\boldsymbol{\rho}) = \begin{pmatrix} 1 & 0 \\ 1 & 1 \end{pmatrix}. \tag{17}$$

Note that the administration of a bolus dose is now represented as an impulse at time t = 0, rather than a non-zero initial condition, such that x(0, p) = 0.

$$T\_{1\natural} = 2\log 2 \int \left( k\_{12} + k\_{21} + k\_{31} - \frac{V\_{\text{max}}}{K\_{\text{M}} + \varkappa\_{1,\text{E}}} \right) \left( k\_{12} + k\_{21} - \frac{V\_{\text{max}}}{K\_{\text{M}} + \varkappa\_{1,\text{E}}} \right) \tag{14}$$

$$= \sqrt{-4k\_{12}\left( k\_{31} - \frac{V\_{\text{max}}}{K\_{\text{M}} + \varkappa\_{1,\text{E}}} \right) + \left( k\_{12} + k\_{21} + k\_{31} - \frac{V\_{\text{max}}}{K\_{\text{M}} + \varkappa\_{1,\text{E}}} \right)^{2}} \right). \tag{14}$$

From Equations (11, 14), we find that the relationship between T1/<sup>2</sup> and FCR is given by:

$$T\_{1\text{\textquotedblleft}1} = \frac{2\log 2}{k\_{12} + k\_{21} + \text{FCR} - \sqrt{-4k\_{12}\text{FCR} + \left(k\_{12} + k\_{21} + \text{FCR}\right)^2}}.\tag{15}$$

Taking Laplace transforms of Equation (16), the inputoutput relation is described by Y(s) = G(s)U(s), where G(s) is the transfer function matrix, given by G(s) = C(p)(sI − A(p))−1B(p), where I is the 2 × 2 identity matrix. G(s) has two elements, corresponding to the two measured outputs, which are given by:

$$G\_{1}(s) = \frac{s + k\_{12}}{s^{2} + \left(k\_{31} - \frac{V\_{\text{max}}}{K\_{\text{M}} + \mathbf{x}\_{1,\text{E}}} + k\_{12} + k\_{21}\right)}$$

$$s + \left(k\_{31} - \frac{V\_{\text{max}}}{K\_{\text{M}} + \mathbf{x}\_{1,\text{E}}}\right)k\_{12}$$

$$G\_{2}(s) = \frac{s + k\_{12} + k\_{21}}{s^{2} + \left(k\_{31} - \frac{V\_{\text{max}}}{K\_{\text{M}} + \mathbf{x}\_{1,\text{E}}} + k\_{12} + k\_{21}\right)}$$

$$s + \left(k\_{31} - \frac{V\_{\text{max}}}{K\_{\text{M}} + \mathbf{x}\_{1,\text{E}}}\right)k\_{12}.\tag{18}$$

Let 8(p) = (φ1(p), ..., φ4(p)), where p = k12, k21, k31,Vmax, KM, x1,E , denote the (distinct) coefficients of s in Equation (18). The coefficients, 8(p), are uniquely determinable from the input-output relationship of the system and are given by:

$$\begin{aligned} \phi\_1(p) &= k\_{12} \\ \phi\_2(p) &= k\_{12} + k\_{21} \\ \phi\_3(p) &= k\_{12} \left( k\_{31} - \frac{V\_{\text{max}}}{K\_{\text{M}} + \chi\_{1,\text{E}}} \right) \\ \phi\_4(p) &= k\_{31} - \frac{V\_{\text{max}}}{K\_{\text{M}} + \chi\_{1,\text{E}}} + k\_{12} + k\_{21} \end{aligned} \tag{19}$$

Introducing an alternative parameter vector p¯ = ¯k12, ¯k21, ¯k31,V¯ max, <sup>K</sup>¯ <sup>M</sup>, <sup>x</sup>¯1,E and equating 8(p) = 8(p¯), it can readily be seen from φ1(p) and φ2(p) that the parameters k<sup>12</sup> and k<sup>21</sup> are uniquely determined (i.e., k<sup>12</sup> = ¯k<sup>12</sup> and k<sup>21</sup> = ¯k21) and therefore structurally globally identifiable from the timecourse of radioactivity remaining in plasma and the body. The parameters k31, Vmax, K<sup>M</sup> and x1,E are not uniquely identifiable; however the FCR (given by Equation 11) is uniquely identifiable.

#### 3.1.2. Fractional Catabolic Rate

The relationship between the FCR and x1,E is given by Equation (11). The SolveAlways function was used in Mathematica<sup>2</sup> to find out whether the parameters k31, Vmax, and K<sup>M</sup> are uniquely determinable by the relationship in Equation (11). Introducing an alternative parameter vector ¯k31,V¯ max, K¯ <sup>M</sup> and solving the equation:

$$k\_{31} - \frac{V\_{\text{max}}}{K\_{\text{M}} + \varkappa\_{1,\text{E}}} = \bar{k}\_{31} - \frac{\bar{V}\_{\text{max}}}{\bar{K}\_{\text{M}} + \varkappa\_{1,\text{E}}},\tag{20}$$

over all values of <sup>x</sup>1,E, gives ¯k31,V¯ max, K¯ <sup>M</sup> = k31,Vmax, K<sup>M</sup> as the only solution for the unknown parameters. Therefore, the parameters k31, Vmax, and K<sup>M</sup> are uniquely determinable from the relationship between the FCR and x1,E.

#### 3.1.3. Terminal Half-Life

The relationship between T1/<sup>2</sup> and x1,E is given by Equation (14). We now wish to know whether the parameter vector p = k12, k21, k31,Vmax, K<sup>M</sup> is uniquely determinable from the relationship in Equation (14). From Equation (13), this is equivalent to asking whether p is uniquely determinable from the relationship between λ<sup>2</sup> and x1,E, given by:

$$\begin{split} \lambda\_{2} &= \frac{1}{2} \left( -k\_{12} - k\_{21} - k\_{31} + \frac{V\_{\text{max}}}{K\_{\text{M}} + \mathbf{x}\_{1,\text{E}}} \right. \\ &+ \sqrt{-4k\_{12} \left( k\_{31} - \frac{V\_{\text{max}}}{K\_{\text{M}} + \mathbf{x}\_{1,\text{E}}} \right) + \left( k\_{12} + k\_{21} + k\_{31} - \frac{V\_{\text{max}}}{K\_{\text{M}} + \mathbf{x}\_{1,\text{E}}} \right)^{2}} \right) . \end{split} \tag{21}$$

The structural identifiability problem amounts to determining whether there exists an alternative parameter vector p¯ such that λ2(x1,E, p) = λ2(x1,E, p¯) with p 6= p¯. λ<sup>2</sup> is one of the roots of the characteristic polynomial equation, given by:

$$\lambda^2 + \left(k\_{12} + k\_{21} + k\_{31} - \frac{V\_{\text{max}}}{K\_{\text{M}} + \varkappa\_{1,\text{E}}}\right)\lambda$$

$$\quad + k\_{12}\left(k\_{31} - \frac{V\_{\text{max}}}{K\_{\text{M}} + \varkappa\_{1,\text{E}}}\right) = 0.\tag{22}$$

We wish to know whether there exists an alternative parameter vector p¯ 6= p, such that:

$$\begin{split} \lambda^2 + \left(k\_{12} + k\_{21} + k\_{31} - \frac{V\_{\text{max}}}{K\_{\text{M}} + \mathbf{x}\_{1,\text{E}}}\right) \lambda + k\_{12} \left(k\_{31} - \frac{V\_{\text{max}}}{K\_{\text{M}} + \mathbf{x}\_{1,\text{E}}}\right) \\ = \lambda^2 + \left(\bar{k}\_{12} + \bar{k}\_{21} + \bar{k}\_{31} - \frac{\bar{V}\_{\text{max}}}{\bar{K}\_{\text{M}} + \mathbf{x}\_{1,\text{E}}}\right) \lambda \\ + \bar{k}\_{12} \left(\bar{k}\_{31} - \frac{\bar{V}\_{\text{max}}}{\bar{K}\_{\text{M}} + \mathbf{x}\_{1,\text{E}}}\right). \end{split} \tag{23}$$

The coefficients of the quadratic are unique, therefore

$$k\_{12} + k\_{21} + k\_{31} - \frac{V\_{\text{max}}}{K\_{\text{M}} + \varkappa\_{1,\text{E}}} = \bar{k}\_{12} + \bar{k}\_{21} + \bar{k}\_{31} - \frac{\bar{V}\_{\text{max}}}{\bar{K}\_{\text{M}} + \varkappa\_{1,\text{E}}},$$

$$k\_{12} \left(k\_{31} - \frac{V\_{\text{max}}}{K\_{\text{M}} + \varkappa\_{1,\text{E}}}\right) = \bar{k}\_{12} \left(\bar{k}\_{31} - \frac{\bar{V}\_{\text{max}}}{\bar{K}\_{\text{M}} + \varkappa\_{1,\text{E}}}\right). \tag{24}$$

The only solution to Equation (24), over all values of x1,E and for positive parameter values only, is p = p¯. Therefore the parameters k12, k21, k31,Vmax, and K<sup>M</sup> are uniquely determinable from the relationship between T1/<sup>2</sup> and x1,E.

#### 3.1.4. Summary of Structural Identifiable Parameters

From an individual's timecourse of radioactivity remaining in plasma and the body, described by Equations (8–10), the parameters k<sup>12</sup> and k<sup>21</sup> are structurally uniquely identifiable. The parameters k31, Vmax, KM, and x1,E are not uniquely identifiable; however the FCR is uniquely identifiable. The parameter x1,E may be measured independently; however the parameters k31, Vmax, and K<sup>M</sup> remain unidentifiable even when x1,E is known. This result is intuitive, as the unidentifiable parameters describe the nonlinear behavior which is not demonstrated when the quantity of endogenous IgG in plasma, x1,E, is constant.

<sup>2</sup>Mathematica 10.4 (2016). Windows. Champaign: Wolfram Research Inc.

In order to show the nonlinear behavior of the model, observations need to be made over a range of steady state quantities of endogenous IgG in plasma, x1,E. Given a set of timecourses, each described by Equations (8–10), for a range of different values of x1,E, it is possible to measure the FCR, T1/<sup>2</sup> and x1,E for each timecourse. From the relationship between the FCR and x1,E, the parameters k31, Vmax and K<sup>M</sup> are structurally globally identifiable. From the relationship between T1/<sup>2</sup> and x1,E, the parameters k12, k21, k31,Vmax, and K<sup>M</sup> are structurally globally identifiable. The structurally identifiable parameters are summarized in **Table 2**.

# 3.2. Estimation of Parameters from Individual Timecourse Data

The model parameters k<sup>12</sup> and k<sup>21</sup> along with the FCR are structurally globally identifiable from the individual timecourse measurements y1(t) and y2(t), as described by the linearized model in Equations (8–10). These three parameters were estimated from timecourse data from seven individuals whom we refer to as subjects A–G. The data are described in Section 2.1.

Parameter values were estimated for each subject by analytically solving the linear ODE system and minimizing the sum of squared residual errors between the model output and the data, using the function NonlinearModelFit in Mathematica<sup>2</sup> . For each subject, the model outputs y1(t) and y2(t) were simultaneously fitted to the plasma and whole body timecourse data, respectively. Three examples of the fits (subjects A–C) are shown in **Figure 4**. The corresponding plots for all subjects are provided in the Supplementary Material.

The parameter estimates and their standard errors are given in **Table 3**. For all three parameters across all subjects, the


standard errors are small relative to their respective parameter estimates. The distribution of parameter estimates among the seven subjects is illustrated in **Figure 5**. The mean and median of each parameter are summarized in **Table 3**. The root mean squared error (RMSE), as a measure of the goodness-of-fit, for each fitted timecourse is also provided in **Table 3**.

As stated in Section 2.1, several subjects have IgG myeloma, macroglobulinemia or familial hypercatabolic hypoproteinemia. Patients with familial hypercatabolic hypoproteinemia do not express FcRn, explaining the large value of the FCR (0.247 day−<sup>1</sup> ) for subject E. The parameter Vmax (not estimated here) for subject E should be equal to zero, reflecting the absence of recycling receptors. Subjects A and C have IgG myeloma and subject D has macroglobulinemia. The high or low values of the FCR in these patients should be explained by the concentrationdependent catabolism of IgG, as described by Equation (11), with abnormally high or low plasma IgG concentrations likely occurring as symptoms of the respective disease.

#### 3.2.1. Sensitivity to Model Parameters

Along with structural identifiability, parameter identification requires sensitivity of the model output to the parameters. There are two types of sensitivity function: traditional sensitivity functions (TSFs) and generalized sensitivity functions (GSFs) (Thomaseth and Cobelli, 1999). Used together, TSFs and GSFs can provide insight in terms of the information about individual parameters provided by a measured output over the time duration of the experiment.

In order to estimate a model parameter from measurements of a model output it is necessary that the measured output is sensitive to the parameter over the time interval of the experiment (Banks et al., 2007). The TSF of a measured model output with respect to one component of the model parameter vector is given by the partial derivative of the output with respect to the parameter, for example:

$$s\_{\text{TSF},\chi\_1,k\_{12}}(t) = \frac{\partial \mathcal{y}\_1}{\partial k\_{12}}(t) \tag{25}$$

is the TSF of the model output y1(t) with respect to the parameter k12. The TSF is locally defined for the "true" parameter vector of the system. For the inverse problem in this section the true model

subjects A, B, and C.


TABLE 3 | Parameter estimates and their standard errors (SE) and the root mean squared error (RMSE) for each fitted timecourse.

and parameter vector for each subject is unknown; therefore the TSFs are calculated for the estimated parameter vectors for each subject to investigate parameter sensitivity over a range of parameter vectors that are likely to be seen in individuals.

The TSFs for the timecourse outputs y1(t) and y2(t) are plotted in **Figure 6**, evaluated at the parameter estimates for subjects A, B, and C, respectively, provided in **Table 3**. The TSFs were calculated in Mathematica<sup>2</sup> . The TSFs show that the model outputs are sensitive to all three parameters over the time interval of observation for each of the parameter vectors. The corresponding plots for all subjects are provided in the Supplementary Material. A similar pattern is observed for the remaining subjects D–G.

A shortcoming of the TSF is that it does not account for correlation between parameters. An alternative function, the GSF, takes account of parameter correlations and quantifies the information content of a model output on an individual parameter over the time duration of observation. The GSFs for a general model output function y(t) = f(t, θ0) with parameter vector θ<sup>0</sup> are defined as:

$$s\_{\text{GSF},\text{y},\theta}(t\_l) = \sum\_{i=1}^{l} \frac{1}{\sigma^2(t\_i)} [F^{-1} \times \nabla\_{\theta} f(t\_i, \theta\_0)] \bullet \nabla\_{\theta} f(t\_i, \theta\_0), \tag{26}$$

where

$$F = \sum\_{j=1}^{n} \frac{1}{\sigma^2(t\_j)} \nabla\_{\boldsymbol{\theta}} f(\boldsymbol{t}\_j, \boldsymbol{\theta}\_0) \nabla\_{\boldsymbol{\theta}} f(\boldsymbol{t}\_j, \boldsymbol{\theta}\_0)^T \tag{27}$$

is the Fisher information matrix and the model output y(t) = f(t, θ0) is observed with error at discrete times t<sup>l</sup> , l = 1, ..., n. The measured values of the output are given by Y<sup>l</sup> = f(t<sup>l</sup> , θ0)+ǫ<sup>l</sup> with σ 2 (tl ) the variance of the error on the observation at time t<sup>l</sup> , ǫ<sup>l</sup> (Thomaseth and Cobelli, 1999). In the definition of the GSF the true parameter vector θ<sup>0</sup> is assumed known. Here the GSFs are calculated for the estimated parameter vectors for each subject, in order to investigate the inverse problem for the different dynamics seen in individuals.

The GSFs for the timecourse outputs y1(t) and y2(t) are plotted in **Figure 6**, for the parameter estimates of subjects A, B, and C, respectively, provided in **Table 3**. The GSFs were calculated in Mathematica<sup>2</sup> . Unlike the TSF, the GSF is defined only at discrete measurement times. In order to obtain an approximation of the smooth function for continuous measurement data, as in Thomaseth and Cobelli (1999) and Banks et al. (2007), we assume a high rate of sampling, calculating the GSF as though a measurement is taken every 0.1 days for each subject. We assume that the variances of the errors are equal at all measurement times, such that the variance terms cancel out in the definition of the GSF (see Equations 26, 27).

According to the interpretation of GSFs given by Thomaseth and Cobelli (1999), a steep increase in the GSF of a particular parameter between 0 and 1 indicates the interval on which the information on that parameter, provided by the measured output, is concentrated. **Figures 6H,I** show the ideal pattern of three distinct intervals of steep increase between 0 and 1 for

the three parameters, respectively, with the information on k<sup>21</sup> concentrated at the beginning of the experiment, followed by k<sup>12</sup> and then the FCR. Intuitively this makes sense, because the dose is administered to the first compartment, from where it transfers to the second compartment in the initial days of the experiment. The pattern shown by the GSFs of y1(t) for subject A, depicted in **Figure 6G**, appears slightly different to that shown by subjects B and C; however, if we extend the experiment time to 50 days for subject A, the GSFs then appear extremely similar to those of subjects B and C. This is due to the slower dynamics exhibited by this subject—see **Table 3** and **Figure 4**. The pattern exhibited by the GSFs of y2(t) for each patient indicates a high correlation between the parameters, producing large oscillations in the GSFs. The corresponding plots for all subjects are provided in the Supplementary Material.

The GSF is clearly a useful tool for understanding the behavior of estimators of correlated parameters; however, it is still important to use the TSF alongside the GSF when analysing parameter sensitivity. As mentioned, the GSFs of y1(t) for subject A appear more similar to those for subjects B and C when the duration of the experiment is extended. The trajectory of the GSF is dependent on the times at which data are collected, and is forced to equal one at the final measurement time. This can result in misleading GSFs when the observation interval is defined over a period of low sensitivity (as defined by the TSF) of the output to the parameters. For this reason we use the TSF and GSF alongside one another. This is discussed in more detail by Banks et al. (2007).

### 3.3. Estimation of Parameters from Fractional Catabolic Rate and Half-Life

As shown in Section 3.1, the parameters k31, Vmax, and K<sup>M</sup> are structurally unidentifiable from an individual subject's timecourse data, assuming the linearized model given by Equations (8–10). It is therefore necessary to make use of the relationships between FCR and T1/<sup>2</sup> , respectively, and the quantity of endogenous IgG in plasma, x1,E. Unfortunately, these relationships are not known for an individual subject; obtaining them would require performing the tracer experiment over a range of different plasma concentrations of endogenous IgG within an individual subject, which is not practically feasible. We therefore estimate parameters from the FCR and T1/<sup>2</sup> measurements taken from a sample of patients with a wide range of endogenous IgG plasma concentrations, as though the data arose from an individual subject, in what may be described as a naive pooled approach (Wright, 1998). It is therefore not possible to gain a sense of the distribution of the parameters k31, Vmax, and K<sup>M</sup> within the population as for those parameters estimated from the individual timecourse data, k<sup>12</sup> and k21, as illustrated in **Figure 5**.

The parameters k12, k21, k31, Vmax, and K<sup>M</sup> were estimated from FCR vs. x1,E and T1/<sup>2</sup> vs. x1,E data, simultaneously. Waldmann and Strober (1969) provide plots of FCR and T1/<sup>2</sup> vs. plasma endogenous IgG concentration in g l−<sup>1</sup> . The data were extracted using the Digitizer tool in OriginPro<sup>1</sup> . The plasma IgG concentration in g l−<sup>1</sup> was converted to µmol l−<sup>1</sup> by dividing by

TABLE 4 | Parameter estimates and standard errors estimated from FCR and T½ data.


the molar mass of IgG, 0.15 g µmol−<sup>1</sup> . The concentration was then multiplied by an average plasma volume of 3 l (Solomon et al., 1963) to obtain the quantity x1,E in µmol.

The FCR and T1/<sup>2</sup> data were fitted simultaneously by the model outputs described in Equations (11, 14). The parameter values were estimated by minimizing the sum of squared residual errors between the model outputs and the measured values, using the function NonlinearModelFit in Mathematica<sup>2</sup> . Due to the different scales of the parameters (0.02 < FCR < 0.17 day−<sup>1</sup> ; 10 < T1/<sup>2</sup> < 72 days) the T1/<sup>2</sup> data points were assigned different weights to the FCR data points. It was assumed that the standard deviation of the residual errors is of the order of the size of the measured values, for both FCR and T1/<sup>2</sup> , respectively. Therefore, the weights given to the T1/<sup>2</sup> data points were equal to the squared mean of the measured T1/<sup>2</sup> values divided by the squared mean of the measured FCR values, and the weights given to the FCR data points were set to 1. Using this approach, the variance of the T1/<sup>2</sup> residuals was assumed to be 1.08 × 10<sup>5</sup> times the variance of the FCR residuals.

The data and model fits are shown in **Figure 7**. The parameter estimates and their standard errors are given in **Table 4**. The standard errors are almost as large as, or larger than, the estimates themselves for the parameters k<sup>12</sup> and k21, indicating that these parameters cannot be estimated with a reasonable level of precision. This may be because the measured output T1/<sup>2</sup> is insensitive to variations in the parameters k<sup>12</sup> and k<sup>21</sup> or due to correlations between the parameters.

#### 3.3.1. Sensitivity to Model Parameters and Parameter Correlations

The sensitivity of the outputs FCR and T1/<sup>2</sup> to the model parameters is illustrated by the TSFs shown in **Figure 8**. In **Figure 8** the TSFs are evaluated for the parameter estimates given in **Table 4**. Due to the wide range in parameter estimates (0.158– 272), each TSF is multiplied by the value of the corresponding parameter estimate; thus the normalized TSF can be seen as representing the sensitivity of the output to variation in a parameter proportional to its value (for the particular parameter values used here). The GSFs of the FCR with respect to the parameters k31, Vmax, and K<sup>M</sup> were calculated for the parameter estimates in **Table 4**. We assume measurements taken in intervals of 10 µmol between 0 µmol and 2000 µmol plasma endogenous IgG, x1,E, in order to get an approximately smooth function. Again we assume that the variances of the errors are equal over

FIGURE 7 | Expressions for (A) FCR (Equation 11) and (B) T1/<sup>2</sup> (Equation 14) fitted to data from Waldmann and Strober (1969).

all concentrations, such that the variance terms cancel out in the definition of GSF (see Equations 26, 27).

**Figure 8A** shows the TSFs of the FCR with respect to the parameters k31, Vmax, and KM, evaluated for the parameter estimates in **Table 4**. The plot shows that, for the parameter vector used, the FCR is sensitive to all three parameters over the range of plasma endogenous IgG concentrations measured, with greater sensitivity to K<sup>M</sup> and Vmax at smaller concentrations. The similarity between the TSFs of the FCR with respect to K<sup>M</sup> and Vmax, respectively, may cause high correlation between these two parameters.

**Figure 8C** shows the GSFs of the FCR with respect to k31, Vmax, and KM. The GSFs show a similar pattern to those of the timecourse observations shown in **Figure 6**, however the larger magnitude of oscillation indicates a strong correlation between the parameters. The GSFs indicate that, for the parameter vector used, measurements at very small quantities of plasma endogenous IgG, below around 100 µmol, have the greatest influence on the estimate of KM, then the region between around 100 and 1.000 µmol has the greatest influence over the estimate of Vmax, with, finally, the information on k<sup>31</sup> available at the remaining larger quantities. This interpretation is consistent with the TSFs given in **Figure 8A**: the sensitivity to K<sup>M</sup> decreases rapidly at small quantities, followed by the sensitivity to Vmax, with the sensitivity to k<sup>31</sup> constant, that is insensitive to the quantity x1,E itself.

**Figure 8B** shows the TSFs of T1/<sup>2</sup> with respect to the parameters k12, k21, k31, Vmax, and KM. The plot shows that T1/<sup>2</sup> is much more sensitive to all of the parameters at smaller quantities of endogenous IgG. The similarity between the trajectories of the TSFs with respect to all parameters suggests that they are highly correlated. It was not possible to calculate the GSFs of T1/<sup>2</sup> due to the Fisher information matrix being ill-conditioned, such that the inverse could not be computed. If we attempt to estimate all five parameters from the T1/<sup>2</sup> data alone, we obtain standard errors of the order of 10 × 10<sup>6</sup> and higher, and correlation coefficients of 1 or −1 between the parameters.

When the parameters are estimated from FCR vs. x1,E and T1/<sup>2</sup> vs. x1,E simultaneously, we find high correlation coefficients between the parameters, with the correlation matrix given by:


Parameters k31, Vmax, and K<sup>M</sup> are highly correlated pairwise, with the strongest correlation between K<sup>M</sup> and Vmax. k<sup>12</sup> and k<sup>21</sup> are highly correlated with one another (correlation coefficient of 0.996) and not with the other parameters, explaining why they cannot be estimated with a reasonable level of precision from these data.

#### 3.4. Simulations of IgG Responses in IgG Multiple Myeloma

In order to simulate monoclonal IgG responses in IgG multiple myeloma, the model of endogenous IgG metabolism given by Equation (1) needs to explicitly account for monoclonal IgG produced by the malignant plasma cells, and polyclonal IgG produced by healthy plasma cells, since both types of IgG undergo the same processes of recycling and elimination and therefore one is influenced by the other. The dynamics of monoclonal and polyclonal IgG in an IgG myeloma patient may be described by:

$$\begin{aligned} \dot{\mathbf{x}}\_{\rm 1,m}(t) &= -\left(k\_{21} + k\_{31} - \frac{V\_{\rm max}}{K\_{\rm M} + \mathbf{x}\_{1,\rm m}(t) + \mathbf{x}\_{1,\rm p}(t)}\right) \mathbf{x}\_{1,\rm m}(t) \\ &+ k\_{12} \mathbf{x}\_{2,\rm m}(t) + I\_{\rm m}(t) \\ \dot{\mathbf{x}}\_{2,\rm m}(t) &= k\_{21} \mathbf{x}\_{1,\rm m}(t) - k\_{12} \mathbf{x}\_{2,\rm m}(t) \\ \dot{\mathbf{x}}\_{1,\rm p}(t) &= -\left(k\_{21} + k\_{31} - \frac{V\_{\rm max}}{K\_{\rm M} + \mathbf{x}\_{1,\rm m}(t) + \mathbf{x}\_{1,\rm p}(t)}\right) \mathbf{x}\_{1,\rm p}(t) \\ &+ k\_{12} \mathbf{x}\_{2,\rm p}(t) + I\_{\rm p}(t) \\ \dot{\mathbf{x}}\_{2,\rm p}(t) &= k\_{21} \mathbf{x}\_{1,\rm p}(t) - k\_{12} \mathbf{x}\_{2,\rm p}(t), \end{aligned} \tag{28}$$

where x1,m(t) and x2,m(t) are the quantities of monoclonal IgG in plasma and in the peripheral space, respectively, x1,p(t) and x2,p(t) are the quantities of polyclonal IgG in plasma and in the extravascular space, respectively, Im(t) is the production rate of monoclonal IgG in µmol day−<sup>1</sup> , Ip(t) is the production rate of polyclonal IgG in µmol day−<sup>1</sup> , and all other parameters are as previously defined.

It is assumed that the production rate of monoclonal IgG, Im(t), is determined by the number of myeloma cells, or tumor burden. Modeling the response of the myeloma cell population under therapy is in itself a significant problem. Here we assume a highly simplified, phenomenological model which nevertheless shows good qualitative agreement with responses seen in real patients. In the following simulations the production rate of monoclonal IgG, Im(t), is given by:

$$I\_{\rm m}(t) = \left(I\_{\rm m,0} - I\_{\rm m,\infty}\right) \exp\left(-k\_{\rm kill}t\right) + I\_{\rm m,\infty},\tag{29}$$

where Im(0) = Im,0 µmol day−<sup>1</sup> , Im(t) tends to Im,<sup>∞</sup> µmol day−<sup>1</sup> for large t, and kkill day−<sup>1</sup> is the rate constant of tumor kill.

In **Figure 9** we present simulations of monoclonal IgG responses in IgG myeloma patients during treatment. Simulations of the plasma concentration of monoclonal IgG are shown alongside plasma IgG concentrations from six IgG myeloma patients, taken from the Intergroupe Francophone du Myélome (IFM) 2009-02 clinical trial (Leleu et al., 2013). The plasma concentration of monoclonal IgG was measured by serum protein electrophoresis at regular intervals during treatment. The simulated quantities of monoclonal and polyclonal IgG are defined by Equation (28), with the monoclonal IgG production rate Im(t) given by Equation (29). The polyclonal IgG production rate Ip(t) is assumed to remain constant at Ip,0 = 15 µmol day−<sup>1</sup> , as in normal individuals (Waldmann and Strober, 1969). At time t = 0 the system is assumed to be in steady state, such that the initial conditions of monoclonal and polyclonal IgG are given by:

$$\begin{aligned} \chi\_{1,\text{m}}(0) &= \frac{-k\_{31}K\_{\text{M}} + I\_0 + V\_{\text{max}}}{I\_0} \\ \chi\_{2,\text{m}}(0) &= \frac{I\_{\text{m},0}}{I\_0} \frac{+\sqrt{4k\_{31}K\_{\text{M}}I\_0 + (-k\_{31}K\_{\text{M}} + I\_0 + V\_{\text{max}})^2}}{2k\_{31}} \\ \chi\_{2,\text{m}}(0) &= \frac{k\_{21}}{k\_{12}}\chi\_{1,\text{m}}(0) \end{aligned} \tag{30}$$

$$\begin{aligned} \text{(2.1.2.2)}\\ \text{(2.1.2)}\\ \text{(2.1.3.0)} &= \frac{I\_{\text{p,0}}}{I\_0} \frac{+\sqrt{4k\_{31}K\_{\text{M}}I\_0 + (-k\_{31}K\_{\text{M}} + I\_0 + V\_{\text{max}})^2}}{2k\_{31}} \\ \text{(2.1.2)} &= \frac{k\_{21}}{k\_{12}}x\_{1,\text{p}} \text{(0)} \end{aligned}$$

where I<sup>0</sup> = Im,0 + Ip,0.

In order to convert the quantity of plasma monoclonal IgG in µmol into concentration in g l−<sup>1</sup> the simulated quantity was multiplied by the molecular weight of IgG, 0.15 g µmol−<sup>1</sup> , and divided by an average plasma volume of 3 l. For the parameters k<sup>12</sup> and k<sup>21</sup> the mean values of the respective parameter estimates from the individual fits in Section 3.2 are used; for the parameters k31, Vmax, and K<sup>M</sup> the values estimated from FCR and T1/<sup>2</sup> vs. x1,E in Section 3.3 are used. The parameters of the model describing the monoclonal IgG production rate, given by Equation (29), namely Im,0, Im,<sup>∞</sup> and kkill, were manually adjusted in order to produce simulations that approximately replicate the responses seen in patients. The parameter values used are provided in **Table 5**.

The purpose of these simulations is to demonstrate how the model analyzed in this paper may be used in the future to investigate monoclonal IgG responses in IgG myeloma. Patient data are presented alongside the simulations to show that they provide good qualitative agreement with in vivo responses, supporting the suitability of the model for future investigations. In these simulations, the initial and final monoclonal IgG production rates, Im,0 and Im,∞, and the rate at which the monoclonal IgG production rate falls during treatment, kkill, have been varied, whilst the parameters of the metabolic model have been fixed. In reality there will be inter-patient variability in both the parameters of the tumor response model and the metabolic model. Here we have not explicitly modeled the response in

the myeloma cell population itself—only the production rate of monoclonal IgG. Additional assumptions are required in order to investigate the relationship between the tumor response and the serum monoclonal IgG response. The simplest approach is to assume that the rate of synthesis per myeloma cell is constant and as such the total body synthesis rate Im(t) is directly proportional to the total myeloma cell population. It is possible that the cellular synthesis rate may vary over the course of treatment; however studies of in vitro IgG synthesis in plasma cells taken from IgG myeloma patients have shown that, whilst the cellular synthesis rate varies between patients, for an individual patient it remains constant over a period of months (Salmon and Smith, 1970). In the present study we have also assumed constant polyclonal IgG production, however it is known that normal plasma cells in the bone marrow are frequently suppressed by the clonal cell presence. If we assume that the cellular rate of IgG synthesis remains constant in normal plasma cells then suppression of these cells likely contributes to a decrease in overall polyclonal IgG synthesis. There are several complex mechanisms involved, but fundamentally the suppression of polyclonal cells is believed to be due to competition between monoclonal and polyclonal cells for survival niches in the bone marrow microenvironment (Paiva et al., 2011). Mathematically modeling normal plasma cell suppression could be the subject of future research. It has also been assumed that the system is in steady state at the commencement of treatment. Future studies will be required to validate this assumption or employ alternative models in which the tumor is growing initially, and to assess the impact of the chosen assumptions on any conclusions drawn from the simulations.

# 4. DISCUSSION

The aim of this study was to analyse a previously published model of endogenous IgG metabolism and available measurements in humans with respect to parameter identifiability. The model was linearized to replicate experimental conditions in which small doses of administered IgG exhibit linear timecourse responses. The linearized model was then analyzed in terms of parameter structural identifiability and parameter sensitivity. The analyses show that certain important parameters are not structurally identifiable from an individual's timecourse response; however they are structurally identifiable using the relationships between the FCR and T1/<sup>2</sup> , respectively, and the quantity of endogenous IgG in plasma, which is assumed to remain in steady state.

A limitation of the linear model of an individual's timecourse response, given by Equations (8–10), is that the parameters k31, Vmax, and K<sup>M</sup> are structurally unidentifiable. It is not known whether these parameters are structurally identifiable in the nonlinear model of coupled radiolabeled and endogenous IgG responses given by Equations (5–7). There are two reasons this approach was not pursued here: firstly, structural identifiability analysis of a four-state nonlinear model with rational terms presents a more challenging problem; secondly, if the nonlinear model were found to be structurally identifiable, the responses available nonetheless do not demonstrate nonlinear behavior due to the small doses of radiolabeled IgG administered (see **Figure 3**)—therefore the parameters representing nonlinear behavior may not be practically identifiable by fitting the nonlinear model, and there is an increased risk of fitting the noise in the data with the increased model complexity.

TABLE 5 | Parameter values used to produce the simulations in Figure 9.


Structural identifiability analysis alone does not imply that parameters can necessarily be estimated in practice. In this paper two sensitivity functions were investigated: the traditional sensitivity function (TSF) and generalized sensitivity function (GSF). TSFs and GSFs were computed and plotted for the timecourse outputs y1(t) and y2(t), using the parameter values estimated from the timecourse data of seven individuals. The TSFs show that the measured timecourse outputs are sensitive to the model parameters over the duration of the experiment, which is different for each individual. In addition, the GSFs show the influence of the duration of observation taking into account correlation between the parameters; for example, subject A is observed over a relatively short time period compared to subjects B and C, considering its slower dynamics. The GSF curves of y1(t) for subject A show a larger magnitude of oscillation than those of subjects B and C; however when the duration of observation is increased for subject A, the GSFs of y1(t) are remarkably similar for the three subjects. This would suggest that the estimation of the parameters for subject A would benefit from a longer duration of observation. If the tracer experiments were to be repeated, the insights obtained from the GSFs in particular could be used to inform the sampling grid of measurements. With the data that are currently available, k12, k21, and the FCR are estimated with a good level of precision for all subjects.

TSFs and GSFs were also calculated for the relationship between the FCR and x1,E. The TSFs indicate that the FCR is most sensitive to the parameters Vmax and K<sup>M</sup> at small concentrations of plasma endogenous IgG; this is also indicated in the GSFs, which show that the information on K<sup>M</sup> is concentrated very close to zero, followed by Vmax and finally k31. The TSFs for the T1/<sup>2</sup> imply high correlation between all the parameters. From the simultaneous estimation of the parameters from both FCR and T1/<sup>2</sup> data, the parameters k31, Vmax, and K<sup>M</sup> are estimated with a reasonable level of precision. We notice that the estimated values of k<sup>12</sup> and k<sup>21</sup> are quite different from the averages of the estimates of the same parameters obtained from fitting individual timecourses. From the FCR and T1/<sup>2</sup> data, the 95% confidence interval estimates of k<sup>12</sup> and k<sup>21</sup> are given by (−0.150, 0.467) and (−0.273, 0.647), respectively. These large intervals containing zero imply that the parameters k<sup>12</sup> and k<sup>21</sup> are not well estimated from these data. The estimates of k<sup>12</sup> and k<sup>21</sup> from the FCR and T1/<sup>2</sup> data are very highly correlated with one another, but not with the remaining three parameters; this offers an explanation for why they are not well estimated from these data. Nevertheless, the 95% confidence intervals for k<sup>12</sup> and k<sup>21</sup> contain the averages of the individual estimates from timecourse data, 0.38 and 0.42, respectively. If we fix k<sup>12</sup> to 0.38 and k<sup>21</sup> to 0.42 in the estimation from the FCR and T1/<sup>2</sup> data, the newly obtained parameter estimates of k31, Vmax, and K<sup>M</sup> are 0.161, 45.6, and 307, respectively. These values fall well within the confidence intervals of the previous estimation where k<sup>12</sup> and k<sup>21</sup> are unconstrained.

**Table 6** compares previously published parameter values alongside the parameter values estimated in this paper. For the parameters k<sup>12</sup> and k<sup>21</sup> the mean value of the parameter among the seven subjects is chosen to represent the average; for the parameters k31, Vmax, and K<sup>M</sup> the values estimated from FCR and T1/<sup>2</sup> vs. x1,E data from many subjects are taken to represent the population average. At first glance the newly estimated parameter values are not wildly dissimilar to the previously published values. Waldmann and Strober (1969) give the values of k<sup>31</sup> = 0.18 day−<sup>1</sup> and Vmax/w = 147 mg day−<sup>1</sup> kg−<sup>1</sup> , where w is body weight in kg. Assuming a 70 kg human, this is equivalent to Vmax = 68.6 µmol day−<sup>1</sup> . The value of k<sup>31</sup> was obtained 'from the asymptotic value of the plot of the IgG fractional catabolic rate versus its concentration'. These are the same data that were utilized in this paper as described in Section 2.1. The authors subtracted the value of the FCR for each individual from 0.18 to find the fractional recycling rate. The fractional recycling rate (FRR) is thus given by:

$$\text{FRR} = k\_{31} - \text{FCR} = k\_{31} - \left(k\_{31} - \frac{V\_{\text{max}}}{K\_{\text{M}} + \varkappa\_{1,\text{E}}}\right) = \frac{V\_{\text{max}}}{K\_{\text{M}} + \varkappa\_{1,\text{E}}} \tag{31}$$

They then multiplied the plasma concentration of endogenous IgG by the plasma volume per kg of body weight for each individual to get the quantity of endogenous IgG in plasma per kg of body weight, x1,E/w. Finally the authors multiplied the FRR by x1,E/w to obtain the absolute recycling rate per kg of body weight, which we will call ARR. They then plotted the reciprocal of the ARR against the reciprocal of x1,E/w to obtain a straight line relationship, given by:

$$\frac{1}{\text{ARR}} = \frac{K\_{\text{M}}}{V\_{\text{max}}} \frac{\text{w}}{\text{x}\_{\text{1,E}}} + \frac{\text{w}}{V\_{\text{max}}}.\tag{32}$$

From the intercept the authors obtained the value of Vmax/w = 147 mg kg−<sup>1</sup> day−<sup>1</sup> .

Kim et al. (2007) provide values for all model parameters. Again using the FCR vs. endogenous IgG concentration data first published by Waldmann and Strober (1969) and described here in Section 2.1, Kim et al. (2007) estimate KM/w using a least-squares fitting. The equation fitted to the data is

$$\text{FCR} = k\_{31} - \frac{V\_{\text{max}}/\text{w}}{\frac{\nu\_1}{\text{w}} \left(\frac{K\_{\text{M}}}{\nu\_1} + \frac{\varkappa\_{1,\text{E}}}{\nu\_1}\right)},\tag{33}$$

where v<sup>1</sup> is the plasma volume. The authors fit Equation (33) to FCR vs. x1,E/v<sup>1</sup> to obtain KM/v<sup>1</sup> whilst fixing the remaining


#### TABLE 6 | Comparison with published parameter values.

\*Assuming 70 kg human. \*\*Assuming 3 l plasma volume. †Taken from Waldmann and Strober (1969).

parameters as follows: k<sup>31</sup> = 0.18 day−<sup>1</sup> (Waldmann and Strober, 1969), Vmax/w = 0.98 µmol kg−<sup>1</sup> (Waldmann and Strober, 1969), and v1/w = 0.042 l kg−<sup>1</sup> (Waldmann and Terry, 1990). By this approach the authors obtain a value of KM/v<sup>1</sup> = 140 µmol l−<sup>1</sup> . Assuming a plasma volume of 3 l this is equates to K<sup>M</sup> = 420 µmol. The authors also estimate the parameters k<sup>12</sup> and k<sup>21</sup> by curve fitting to tracer experiment data from Waldmann and Terry (1990).

The parameter values provided by Hattersley et al. (2013) were obtained by methods described in Hattersley (2009). Hattersley (2009) estimates parameters k<sup>12</sup> and k<sup>21</sup> by fitting the model to tracer experiment data in Waldmann and Strober (1969). For the remaining model parameters, k31, Vmax, and KM, the author takes a completely different approach, fitting the model directly to serum IgG data from an IgG myeloma patient, assuming a delayed logistic function to describe the production of tumorproduced IgG. For this approach, the parameters k12, k21, and v<sup>1</sup> were fixed.

In this paper data from a number of subjects have been used for parameter estimation: parameters k<sup>12</sup> and k<sup>21</sup> were estimated individually for seven subjects and parameters k31, Vmax, and K<sup>M</sup> were estimated from the pooled data of around 40 subjects. In order to make predictions of IgG responses in IgG multiple myeloma that can be generalized across patients, a model which characterizes average, or expected, behavior is advantageous. One of the limitations of this study is that a full population approach has not been taken. Fitting the timecourse data individually and summarizing the parameter estimates by the sample mean or median can be seen as a two-stage approach, and fitting the FCR and T1/<sup>2</sup> data as though they arose from an individual is essentially a naive pooled approach; these approaches have been shown to be inferior to a full population approach (Wright, 1998). A population approach has not been taken here due to the limited data that are available in the literature. In future studies it may be possible with further experiments to apply a full population approach and gain information on the distribution of all parameters within the population. Furthermore, the work presented here could be enhanced with a simulation study in which parameters are estimated from synthetic data, in order to provide additional understanding of the identification problem and inform the design of future experiments.

In Section 3.4 we have shown how the model can be extended to simulate monoclonal IgG responses in IgG myeloma. The assumptions behind these simulations are discussed in detail in that section. In IgG myeloma patients the serum monoclonal IgG response is used as a surrogate for the tumor response to treatment; however the relationship between the tumor response and the monoclonal IgG response is inevitably influenced by the natural elimination of IgG from the body, which is known to have a non-constant relationship with serum IgG concentration. It is our intention that the model analyzed in this paper can be used in the future as the basis of more detailed investigations into the dynamics of IgG responses in IgG multiple myeloma; for example, is the prediction of long-term outcomes by monoclonal IgG responses influenced by the concentration-dependent and comparatively long half-life of the protein? Such future studies could impact upon how responses to treatment are assessed in patients.

In addition, the concentration-dependent elimination of IgG may be implicated in the pharmacokinetics of monoclonal antibody (mAb) agents for multiple myeloma, for example daratumumab, which is currently undergoing evaluation in patients with relapsed or refractory disease (Costello, 2017). Due to FcRn-mediated recycling, the longevity of daratumumab is predicted to be shorter in patients with high monoclonal IgG loads whereas low monoclonal IgG concentrations may favorably affect the pharmacokinetic profile of the agent, such that doses could be administered at less frequent intervals. The pharmacokinetics of various mAbs have been well studied; however the use of mAbs in multiple myeloma is recent and the dynamic response of the tumor-produced IgG in IgG myeloma patients adds an additional level of complexity. With the appropriate data it would be highly interesting from a pharmacological point of view to couple mathematical models of the mAb disposition and the tumor-produced IgG response, which is in turn directly affected by the efficacy of the agent.

IgG metabolism is implicated in other medical applications beyond patient monitoring in multiple myeloma, including antibody mediated rejection of transplants, infection and intravenous IgG (IVIG) therapy. In medical applications, mathematical models can be used to investigate biomedical systems in silico, allowing many scenarios and interventions to be simulated whilst avoiding the costs associated with human and animal experimentation. For biomedical applications, the parameter values used are of high importance as they can greatly affect conclusions drawn from simulations. In this paper structural identifiability analysis and sensitivity analysis are presented as a first stage in the model validation process, showing which model parameters are identifiable from which measured outputs. Together structural identifiability analysis and sensitivity analysis can be used to inform parameter estimation and improve confidence in the methodology used. In order for the model to applied in other biomedical scenarios, validation against patient data would be necessary. A limitation of the model analyzed here is that it may not contain a sufficient level of detail for all applications. Future work may involve comparing this simple model with more complex models that are based closely on the biological mechanisms, such as the model presented by Ferl et al. (2005).

# 5. CONCLUSIONS

In this research a previously published model of endogenous IgG metabolism in humans has been analyzed and parameter values estimated using limited data from the literature. The analyses herein provide an understanding of how parameter estimates have been obtained and sources of uncertainty; for future applications in which the parameter values themselves are of key importance, an understanding of how they were obtained and why is crucial. The parameterized model can have a wideranging impact in studies of endogenous IgG metabolism in biomedical applications, not limited to investigations of IgG

#### REFERENCES


as a response marker in IgG multiple myeloma, supporting therapeutic interventions and patient monitoring.

# AUTHOR CONTRIBUTIONS

FK performed model analyses and parameter estimation. FK, MC, and NE wrote the manuscript. MC, NE, and SHarding initiated, coordinated, and supervised the work. XL provided discussion on the clinical application of the work. XL, BA, HA, OD, TD, GF, SG, SHarel, BH, VJ, PM, VR, SS, and CT provided data from the Intergroupe Francophone du Myélome (IFM) 2009-02 clinical trial. All authors reviewed and approved the final manuscript.

# FUNDING

This research was supported by a Biotechnology and Biological Sciences Research Council (BBSRC) studentship, through the Midlands Integrative Biosciences Training Partnership (MIBTP), and an Engineering and Physical Sciences Research Council (EPSRC) Impact Acceleration Account (IAA) award.

#### SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: http://journal.frontiersin.org/article/10.3389/fphys. 2017.00149/full#supplementary-material


HevyliteTM assay: clinical and prognostic impact in multiple myeloma. Exp. Hematol. Oncol. 1:9. doi: 10.1186/2162-3619-1-9


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2017 Kendrick, Evans, Arnulf, Avet-Loiseau, Decaux, Dejoie, Fouquet, Guidez, Harel, Hebraud, Javaugue, Richez, Schraen, Touzeau, Moreau, Leleu, Harding and Chappell. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Mathematical Modeling of *Streptococcus pneumoniae* Colonization, Invasive Infection and Treatment

#### Elisa Domínguez-Hüttinger 1, 2 \*, Neville J. Boon<sup>1</sup> , Thomas B. Clarke<sup>3</sup> \* and Reiko J. Tanaka<sup>1</sup> \*

<sup>1</sup> Department of Bioengineering, Imperial College London, London, UK, <sup>2</sup> Instituto de Ecología, Universidad Nacional Autónoma de México, Mexico City, Mexico, <sup>3</sup> Department of Medicine, Imperial College London, London, UK

#### *Edited by:*

Krasimira Tsaneva-Atanasova, University of Exeter, UK

#### *Reviewed by:*

Samantha Jane King, The Research Institute at Nationwide Children's Hospital, USA Jason H. Yang, Massachusetts Institute of Technology, USA

#### *\*Correspondence:*

Elisa Domínguez-Hüttinger elisa.dominguez@mail.ecologia. unam.mx Thomas B. Clarke thomas.clarke@imperial.ac.uk Reiko J. Tanaka r.tanaka@imperial.ac.uk

#### *Specialty section:*

This article was submitted to Computational Physiology and Medicine, a section of the journal Frontiers in Physiology

*Received:* 05 December 2016 *Accepted:* 13 February 2017 *Published:* 02 March 2017

#### *Citation:*

Domínguez-Hüttinger E, Boon NJ, Clarke TB and Tanaka RJ (2017) Mathematical Modeling of Streptococcus pneumoniae Colonization, Invasive Infection and Treatment. Front. Physiol. 8:115. doi: 10.3389/fphys.2017.00115 Streptococcus pneumoniae (Sp) is a commensal bacterium that normally resides on the upper airway epithelium without causing infection. However, factors such as co-infection with influenza virus can impair the complex Sp-host interactions and the subsequent development of many life-threatening infectious and inflammatory diseases, including pneumonia, meningitis or even sepsis. With the increased threat of Sp infection due to the emergence of new antibiotic resistant Sp strains, there is an urgent need for better treatment strategies that effectively prevent progression of disease triggered by Sp infection, minimizing the use of antibiotics. The complexity of the host-pathogen interactions has left the full understanding of underlying mechanisms of Sp-triggered pathogenesis as a challenge, despite its critical importance in the identification of effective treatments. To achieve a systems-level and quantitative understanding of the complex and dynamically-changing host-Sp interactions, here we developed a mechanistic mathematical model describing dynamic interplays between Sp, immune cells, and epithelial tissues, where the host-pathogen interactions initiate. The model serves as a mathematical framework that coherently explains various in vitro and in vitro studies, to which the model parameters were fitted. Our model simulations reproduced the robust homeostatic Sp-host interaction, as well as three qualitatively different pathogenic behaviors: immunological scarring, invasive infection and their combination. Parameter sensitivity and bifurcation analyses of the model identified the processes that are responsible for qualitative transitions from healthy to such pathological behaviors. Our model also predicted that the onset of invasive infection occurs within less than 2 days from transient Sp challenges. This prediction provides arguments in favor of the use of vaccinations, since adaptive immune responses cannot be developed de novo in such a short time. We further designed optimal treatment strategies, with minimal strengths and minimal durations of antibiotics, for each of the three pathogenic behaviors distinguished by our model. The proposed mathematical framework will help to design better disease management strategies and new diagnostic markers that can be used to inform the most appropriate patient-specific treatment options.

Keywords: *Streptococcus pneumoniae*, upper airway epithelium, data integration, hybrid systems, commensal bacteria, systems biology, antibiotics resistance

# 1. INTRODUCTION

Streptococcus pneumoniae (Sp) is a commensal bacterium that is part of the upper airway microbiota. While it normally resides on the upper airway epithelium without causing serious infection or tissue damaging inflammation (World Health Organization, 2012), factors such as co-infection with the influenza virus often result in the development of life-threatening infectious and inflammatory diseases, including pneumonia, meningitis or even sepsis (World Health Organization, 2012; McCullers, 2014), since these factors can cause a weakened immune response to Sp or tissue damage that may disrupt the normal interactions between Sp and host. The threat of Sp infection has been increasing despite interventions by widely available antibiotics, due to the increasing presence of multiple antibiotic-resistant Sp strains (Nuorti et al., 1998; McCullers et al., 2000). Reduced susceptibility to penicillin was detected in all WHO regions (World Health Organization, 2014) and the pneumococcus remains a major cause of morbidity and mortality, not solely from the lower lung infection Siegel and Weiser (2015). There is an urgent need to devise better intervention strategies that can effectively halt the onset or persistence of Sp-mediated pathology at its early stages using a minimal amount of antibiotics for a short duration, in order to avoid the emergence of further antibiotic-resistant Sp strains (Schrag et al., 2001; Prina et al., 2015b).

Identification and design of effective intervention strategies require systems-level and quantitative understanding of the complex and dynamically-changing host-pathogen interactions that can lead to either healthy Sp colonization or pathological conditions, such as infection or inflammation. This paper proposes a mathematical model of the host-pathogen interactions between Sp and the upper airway epithelium, the initial site of interaction between Sp and the host which is the first step in all disease tiggered by this bacterium (Siegel and Weiser, 2015). We analyse the model to systematically and quantitatively investigate the mechanisms by which the homeostatic interactions are disrupted, for example by a weakened barrier function (McCullers, 2014) or immune suppression (Didierlaurent et al., 2008), and cause the onset of infectious processes.

Previously proposed mathematical models (Smith et al., 2011; Shrestha et al., 2013; Smith et al., 2013; Mochan et al., 2014) considered Sp infections in the lung which is a normally sterile site of the airway epithelium. In this paper, we develop a mechanistic model of homeostatic interactions between the host's upper airway and Sp as a commensal bacterium, based on a variety of experimental data from in vivo and in vitro studies. Given that the tissue-damaging effects of neutrophil transmigration are responsible for part of the pathology of infection (Chin and Parkos, 2007; Zemans et al., 2009), we specifically model how impaired host-pathogen interactions lead to loss of epithelial homeostasis and serious infection. Our mechanistic model describes dynamic interplays between Sp, immune cells, and epithelial tissues by a hybrid system of ordinary differential equations (ODEs), and elucidates the mechanisms by which commensal bacteria cause infection.

Our model demonstrates a robust behavior of healthy clearance of asymptomatic pneumococcal colonization without overt disease. Perturbation of the model parameters, corresponding to virtual patient cohorts, demonstrates three clinically observed pathological behaviors (disease phenotypes) triggered by disrupted Sp-host interactions. Using this mathematical model of pneumococcal colonization, we further suggest optimal treatment regimens that minimize use of antibiotics to intervene the pathogenic processes for each patient cohort. As colonization is a prerequisite for all pneumococcal disease (Siegel and Weiser, 2015), studying and the modeling of colonization by Sp to understand its interaction with the host could be important for not only looking at invasive infections but also other types of interaction/infection of the pneumococcus and host.

# 2. RESULTS

# 2.1. Mathematical Model of *Sp* Colonization in the Upper Airway Epithelium

Our proposed mathematical model of Sp colonization (**Figures 1A,B**) is a system-level representation of the prominent interactions between Sp, the airway epithelium, and immune cells (a–j) in **Figures 1A,B**, that were identified based on the empirical evidence from numerous experimental in vivo and in vitro studies as detailed below.

Under homeostatic conditions, a population of commensal bacteria, Sp, resides in the lumen on the apical side of the airway epithelium, where they are contained by a competent epithelial barrier integrity (Beisswenger et al., 2007) (**Figures 1Aa,Ba**) and immune responses mediated by neutrophils and macrophages (Dick et al., 2008; Standish and Weiser, 2009) (**Figures 1Ab,Bb**). Through disrupted barrier, apically located Sp can translocate to reach the blood vessel (Beisswenger et al., 2007) (**Figures 1Ac,Bc**), where they are either killed by resident immune cells that circulate in the blood (Li et al., 2002; Li, 2004), or grow uncontrollably and result in invasive infection if the immune cells cannot contain the translocated Sp (Silverstein and Rabadan, 2012). The amount of the translocated Sp in the blood vessel is therefore a determinant of whether disrupted Sp-host interactions cause serious infection such as sepsis.

Translocation of Sp occurs through the airway epithelial barrier, whose integrity is regulated by the apically located bacteria load. The bacteria bind to Pattern-Recognition immune receptors, specifically Toll-like receptors (TLR2s) that are preferentially expressed on the apical side of the airway epithelial cells (Melkamu et al., 2009), and activate the TLR signaling cascade (**Figures 1Ad,Bd**). The activation of the TLR cascade in epithelial cells decrease the barrier integrity of the airway epithelium (**Figures 1Ae,Be**) by TLRmediated activation of proteases that damage the epithelial cells (Oggioni et al., 2004; Schmeck et al., 2004; Attali et al., 2008; Tieu et al., 2009) and by reduction of the barrier recovery rate due to the increased expression of the transcriptional repressor SNAIL1, which inhibits the expression

of claudin, a component of the tight junctions (Clarke et al., 2011).

Active TLR signaling also induces recruitment of neutrophils from the neutrophil pool in the blood vessel, via the release of IL-17 (Zhang et al., 2009) that activates neutrophil-attracting interleukins IL-8 (Lindén, 2001) (**Figures 1Af,Bf**). The recruited neutrophils trigger transmigration of macrophages to the site of infection (Zhang et al., 2009), further potentiating the immune responses to the apically located pathogens (**Figures 1Ag,Bg**), whereas macrophages on the lumen restrict neutrophil transmigration (Zhang et al., 2009) by releasing neutrophil-repellent anti-inflammatory cytokines (Knapp et al., 2003) (**Figures 1Ah,Bh**). Transmigrating neutrophils release barrier degrading proteases (Chin et al., 2008) to reduce the barrier integrity (Nash et al., 1987; Nusrat et al., 1997; Zemans et al., 2009) (**Figures 1Ai,Bi**). The reduced barrier integrity in turn allows more transmigration of both neutrophils and macrophages from the blood vessel to the site of infection (Nash et al., 1987) (**Figures 1Aj,Bj**).

The model elucidates the main control structure of the system that maintains homeostatic interactions between commensal bacteria, Sp, and the host, via a dynamic interplay between environmental stressors, epithelial barrier integrity and immune responses (**Figure 1B**). At the apical side of the airway epithelium, Sp load is regulated via activation of TLRs, which induce immune responses that decrease the bacterial load but also reduce the epithelial barrier integrity. While the reduced barrier integrity enables transmigration of immune cells from the blood vessel for effective killing of Sp at the apical side of the epithelium barrier, it also allows transmigration of Sp from the apical side of the epithelium to the blood vessel, potentially causing systemic infection (sepsis). The dynamic interplay between the immune responses and the epithelial barrier integrity are further modulated by their mutual inhibition.

Our model further assumes two switches, an R-switch for TLR activation and an Sv-switch for the growth of the transmigrated bacteria in the blood vessel, based on the experimental evidence described below. The R-switch for Sp-mediated activation of TLRs reflects the observations that low concentrations of Sp do not cause activation of TLR signaling, while high concentrations lead to a sharp increase in TLR activity with hysteresis (He et al., 2009; Shalek et al., 2013; Sung et al., 2014). We model the R-switch by a perfect switch, which is a phenomenological representation of the bistable switch (Sung et al., 2014), and is described by the off- and on-states (R = Roff and Ron) with the activation (S <sup>+</sup>) and inactivation (S <sup>−</sup>) thresholds for the critical concentrations of apically located Sp that abruptly and sharply turn on-or-off TLR activity (**Figure 1C** and Equation 2). The Sv-switch reflects the observations that transmigrated bacteria in the blood vessel (Sv) either overgrow (Benton et al., 1997) or are contained by resident immune cells depending on the bacterial concentration (Supplementary Figure 3B). We model the Sv-switch with a switching threshold of S ∗ v , above which the infiltrated bacteria in the blood vessel grow exponentially (**Figure 1D**).

The resulting model is described by a hybrid system of five ODEs (Equation 1). The nominal values of the 24 model parameters (**Table 1**) were derived by fitting the model outcome to datasets from 11 independent studies, namely three in vivo studies (Benton et al., 1997; Zhang et al., 2009 and our own experiment) and eight in vitro studies (Nash et al., 1987; Coyne et al., 2002; Lagrou et al., 2003; Attali et al., 2008; Chin et al., 2008; Komori et al., 2011; Hathaway et al., 2012; Kwok et al., 2012), as detailed in the Supplementary Material. Our model therefore provides a

#### TABLE 1 | Nominal parameters of the model.


coherent mathematical framework to explain both in vivo and in vitro data.

### 2.2. Healthy Clearance of Asymptomatic *Sp* Colonization is Robustly Observed

One of the dataset used for the parameter estimation was obtained from in vivo studies in Zhang et al. (2009), where the mice were challenged with 10<sup>7</sup> CFU of Sp and recovered their healthy state, which is characterized by nonzero apical commensal bacterial load that does not trigger host responses. Our model was fitted to reproduce the experimental measurement in Zhang et al. (2009) for the apical bacterial load (Sa) and the concentrations of neutrophils (N) and macrophages (M) (**Figure 2A**). Both the experimental data and our model simulation demonstrate that the transient Sp challenge (increase of Sa) triggers a transient increase in N and a subsequent increase in M. These immune responses can bring S<sup>a</sup> down to a homeostatic level, when the saturation limit for S<sup>a</sup> is not high enough, which enabled the recovery of the mice from the bacterial challenge within 7 days without demonstrating invasive infection.

The simulation of our model with the data-calibrated nominal parameters further predicts the dynamics of three variables that were not measured in this experiment, TLR activity (R), barrier integrity (B) and infiltrated Sp in the blood vessel (Sv), thereby explaining the underlying mechanism of the healthy recovery from a bacterial challenge. Upon a Sp challenge, the apically located bacterial load becomes high enough (Sa(0) > S <sup>+</sup>) to activate TLRs (R = Ron), which trigger recruitment of immune cells to the site of infection. These immune responses bring the initially high S<sup>a</sup> down to below S <sup>−</sup>, where the R-switch turns off (R = Roff) and stays off as S<sup>a</sup> remains below S <sup>+</sup>, as suggested by the focal point analysis (see Methods). The epithelial barrier integrity (B) continuously decreases while the R-switch is on (R = Ron), allowing bacteria to invade the blood vessel, as demonstrated by a rise in Sv. However, a healthy clearance of S<sup>v</sup> is achieved without causing sepsis, since the peak of S<sup>v</sup> remains below the threshold, S ∗ v , of the Sv-switch. Note that both the experiments and our model simulation demonstrate that M stays high while B is kept high after N goes to zero, suggesting the importance of M as an immune mediator that does not compromise the barrier integrity.

The healthy recovery behavior described above is characterized in our model by convergence to the off-state of TLR activity (R = Roff) accompanied by the containment of S<sup>v</sup> (S<sup>v</sup> < S ∗ v ), and is robustly observed under perturbations to the parameter values. Among 10,000 simulations conducted by randomly sampling parameter values from an uniform distribution over two orders of magnitude around the nominal values, 83% of the simulations demonstrated a healthy recovery

from a transient Sp challenge (**Figures 3A,B**). In 98% of the healthy recovery cases computationally observed, S<sup>v</sup> reached its peak while R = Ron (**Figure 3E**), suggesting that the appropriate host responses via TLR activation are responsible for containing Sv. The appropriate level of the barrier damage by active TLRs enables effective recruitment of immune cells that can reduce Sa, but prevents excessive transmigration of S<sup>a</sup> to Sv, keeping the S<sup>v</sup> lower than the threshold, S ∗ v . The robust appearance of the healthy recovery in our model simulations confirms that that our model can coherently explain the mechanism behind the healthy recovery of the host from pneumococcal colonization, which can be effectively cleared by the natural host responses without any treatments.

## 2.3. Four Phenotypes Classified by the Double Switch

The remaining 17% of the simulations with parameters perturbed from their nominal values demonstrated systems dynamics that correspond to serious infection or inflammation. They are classified into three disease phenotypes, depending on the states of the Sv- and R-switches (**Figure 3A**). The state of the Sv-switch determines whether sepsis occurs due to invasive infection of Sv(> S ∗ v ), and that of the R-switch determines whether immunological scarring occurs due to persistent host responses caused by R = Ron. Immunological scarring refers to the cumulative and long-term effects of immune response to pathogens, including tissue remodeling and altered immune responses to new pathogenic challenges, that persist after the pathogenic organism has been cleared (Fonseca et al., 2015). In our simulations, 13% demonstrated sepsis without immunological scarring (S<sup>v</sup> > S ∗ v and R = Roff) and the other two disease phenotypes, immunological scarring (R = Ron) with and without sepsis, were observed 2% each (**Figure 3A**).

Immunological scarring is characterized by a persistent onstate of the R-switch due to S<sup>a</sup> staying above S <sup>−</sup> (**Figure 3C**). The R-switch triggers persistent host responses leading to sustained immune responses which are not strong enough to decrease S<sup>a</sup> below S <sup>−</sup> but cause persistent barrier damage. Note that the peak of N is much lower in the immunological scarring phenotype than that in the healthy recovery and sepsis phenotypes with R = Roff (**Figures 3B,D**), resulting in weak immune responses that are not sufficient to decrease Sa. As a result, the host becomes vulnerable to a second bacterial attack due to the damaged barrier and the sub-threshold concentration of S<sup>v</sup> (< S ∗ v ), which stay as silent remainders of the first pathogenic challenge.

Sepsis is characterized by outgrowth of S<sup>v</sup> once it surpasses the threshold S ∗ v (**Figure 3C**). In 99.7% of the sepsis phenotypes simulated by our model, the onset of sepsis occurs (when S<sup>v</sup> = S ∗ v is achieved) while R is on (**Figure 3G**), suggesting that whether sepsis occurs or not is determined by the dynamics of S<sup>v</sup> while R is on. It is similar with the healthy recovery case, where S<sup>v</sup> reaches its peak below the threshold S ∗ v , while R is on. Moreover, the duration of R = Ron is much longer for the sepsis phenotype compared to the healthy recovery phenotype (**Figure 3H**), suggesting that persistent host response may allow excessive transmigration of Sp into the blood vessel above S ∗ v .

When both the Sv- and R-switches are on, sepsis is accompanied by immunological scarring (**Figure 3E**), where the barrier is severely damaged and S<sup>v</sup> continues increasing above S ∗ v , while S<sup>a</sup> remains above S −.

The four phenotypes, including a healthy phenotype and three disease phenotypes, correspond to different patient cohorts observed in the clinic. Healthy recovery from colonization is the most common outcome of host-pneumococcal interactions (Austrian, 1986) and corresponds to patients who can clear their symptoms from transient infection without any antibiotics treatment. Sepsis corresponds to patients who would develop systemic infection as a consequence of dysregulated transepithelial crossing of bacteria if no treatment is applied (Clarke et al., 2011; Siegel and Weiser, 2015). Immunological scarring corresponds to tissue-damaging inflammation that prevails even after clearance of the pathogens (Periselneris et al., 2015). The long-term deleterious consequences of such sterile inflammation and the associated tissue restructuring/damage are considered to underlie many diseases, including pulmonary fibrosis associated to previous Sp infections (Knippenberg et al., 2015), chronic obstructive pulmonary disease (Garcha et al., 2012) and cancer (Elinav et al., 2013; Pradere et al., 2014). A sustained activation TLR is recognized to be an important molecular player responsible for this tissue damage (Pradere et al., 2014), as in our model. Sepsis with immunological scarring corresponds to patient cohorts who would develop a severe infection with long-term deleterious effects in absence of treatment (Leibovici, 2013).

#### 2.4. Risk Factors for Disease Phenotypes

the R-switch ON time for healthy recovery and sepsis cases.

To identify the model parameters that affect the states of the R- and Sv-switches thereby determine the four phenotypes, we conducted the global parameter sensitivity analysis of our model with respect to R and Sv, respectively, using both Sobol and eFAST sensitivity indices (Marino et al., 2008; Cannavó, 2012).

The analysis identified the three most sensitive parameters for the propensity to turn on both the R-switch (to develop immunological scarring) and the Sv-switch (to develop sepsis)

ON time vs. Time for sepsis onset for the sepsis phenotype. (H) Boxplots representing the minimum, first quartile, median, third quartile, maximum and outlayers of

(**Figure 4**): the rate of bacterial transmigration through the barrier (θS), the bacterial carrying capacity (µS), and the killing rate of bacteria by macrophages (φMS) further confirming the importance of macrophages. Simulations with systematic variations of these three parameters further suggest that they affect the occurrence of sepsis and of immunological scarring, as well as how quickly these occur after the Sp challenge (**Figure 5**).

These three sensitive parameters have a direct correspondence with risk factors for disease triggered by Sp that have been reported in the experimental literature. For example, increase in θ<sup>S</sup> can be caused by co-infection, which damages the barrier directly or by having triggered previous immune responses (McCullers, 2014). Increase in µ<sup>S</sup> is caused by previous infections, for example by influenza virus, that damage the tissue, increase nutrient contents (Siegel et al., 2014), or shift the microbiome composition affecting the dynamics of the different bacterial populations (McCullers, 2014). φMS can be affected for example by severe asthma (Liang et al., 2014).

Other parameters that are also affected by co-infection were not identified to be very sensitive for the propensity to develop sepsis (increase in Sv) or unresolved host responses (increase in R) (**Figure 4**). These parameters include S <sup>+</sup> which can increase as a consequence of TLR2 desensitization caused by a previous influenza virus infection (Didierlaurent et al., 2008), M that may increase as a consequence of previous infectious events (La Gruta et al., 2007; Yin et al., 2013), and the size of the neutrophil pool (Nv) which may decrease by chemotherapy or severe infections (Dick et al., 2008). The unsensitivity to the initial conditions can be partially explained by the existence of a unique stable steady state corresponding to the healthy recovery.

# 2.5. A Rapid Onset of Sepsis Triggered by a Transient *Sp* Challenge

In the septic behavior observed in 15% of the simulations (**Figures 3C,D**), the sepsis occurred (S<sup>v</sup> increases above S ∗ v ) within 2 days post Sp challenge in 79% of the cases (**Figure 6**). When sepsis is accompanied with immunological scarring (R stays on and S<sup>v</sup> increases above S ∗ v , **Figure 3D**), the time to sepsis is longer than when it is not (**Figure 6**). The computationally predicted rapid onset of the sepsis is consistent with experimental observations in Andonegui et al. (2009) that the mice either survived or died within 36 h upon Sp challenge applied directly into the lumen of the lungs. The results suggest that rapid treatments within 36 h are crucial to prevent the onset of sepsis.

Increasing the immune activity, for example by activation of adaptive immune responses, could be an effective way to decrease the risk of sepsis onset, as it elevates the switching threshold, S ∗ v , which depends on the strength of the resident immune cells. While the adaptive immunity could be activated naturally, the time for activation of the adaptive immune responses (which involves the de novo differentiation of naive T cells into mature T cells) by infiltrated pathogens was experimentally evaluated to

be more than 2 days in mice (Zheng and Flavell, 1997). Such slow activation of the adaptive immunity therefore cannot prevent the onset of sepsis within 36 h.

These results suggest that prophylactic activation of the adaptive immune responses, for example by vaccinations, could be an effective strategy to prevent the incidence of sepsis, as demonstrated by the protective effects of Sp vaccination in mice (Cao et al., 2013). It is also consistent with the clinical suggestions to use vaccines as a preventive measurement against transient bacterial challenges in the high-risk patients (World Health Organization, 2012).

# 2.6. Optimal Antibiotics Treatment Regimens for Each of the Three Patient Cohorts

Using the proposed model, we investigate optimal treatment regimens and determine the minimal strength and duration of antibiotics treatment that are required to prevent or revert the pathological consequences of a transient Sp challenge. The minimal use of antibiotics is important for tackling the problem of antibiotics resistance (Nuorti et al., 1998), since the emergence of antibiotic-resistant Sp strains has been associated to the excessive use of antibiotics (Schrag et al., 2001; Prina et al., 2015b). We consider two different types of bactricidal antibiotics treatments in our modeling framework: apical application of antibiotics in the luminal side of the mucosa that decreases S<sup>a</sup> and can thereby turn off the R-switch and stop the immunological scarring, and systemic application of antibiotics in the blood vessel that directly decreases S<sup>v</sup> to prevent the onset of invasive infection (described in the Methods Section 4.3).

When the patients have immunological scarring without sepsis (**Figure 3C**), the treatment by apical application of antibiotics should aim to reduce S<sup>a</sup> down below S <sup>−</sup> to turn off the R-switch (**Figure 7A**). Once the R-switch is turned off, further use of antibiotics is no longer needed, as the healthy steady state with R = Roff is locally attractive (S<sup>a</sup> < S <sup>+</sup>) for all the parameter combinations tested (over 10, 000 simulations). The minimal treatment potency of apically applied antibiotics (minimal strength × minimal duration) to bring S<sup>a</sup> down below S <sup>−</sup> depends on the severity of the phenotype measured by the deviation of the high focal point from S <sup>−</sup> (R <sup>2</sup> = 0.46804).

When the patients are susceptible for sepsis (**Figure 3D**), the treatment by systemic application of antibiotics should aim to reduce S<sup>v</sup> to avoid reaching S ∗ v and thereby causing invasive infection (**Figure 7B**). The minimal strength of systemically applied antibiotics allows the maximum of S<sup>v</sup> to reach just below S ∗ v , and the minimal duration of the treatment with the minimal strength corresponds to the time required for R(t) to naturally turn off by S<sup>a</sup> reaching S <sup>−</sup>. The minimal treatment potency of systemic antibiotics to prevent sepsis depends on the severity of the phenotype measured by the time to reach S ∗ v in the absence of treatments.

When the patients are susceptible to the combination of sepsis and immunological scarring (**Figure 3E**), they require antibiotics that are strong enough to be apically applied until S<sup>a</sup> decreases below S <sup>−</sup> to turn off the R-switch (**Figure 7C**). Our model simulations predicted that the apical treatment is enough to prevent invasive infection for 48% of these cases, since the reduction of S<sup>a</sup> also reduces Sv, but the remaining 52% of the cases require additional application of comparatively small amounts of antibiotics directly in the blood vessel.

The distributions of the minimal treatment strengths and durations that we computationally predicted can be used as a guide to design safe and effective treatment options for the three patient cohorts. For example, our results suggest that antibiotics treatment for 20 days can prevent or revert most of immune scarring (**Figure 7A**) or invasive infection (**Figure 7B**), but that a much longer antibiotics treatment is needed for patients with a propensity for both sepsis and immune scarring (**Figure 7C**).

# 3. DISCUSSION

In this paper, we have proposed the first mathematical model of Sp colonization of the upper airway epithelium, and demonstrated that it robustly reproduces the healthy co-existence between this bacterium and the host. Our mathematical model is a hybrid system of ODEs, describing the interactions between the bacteria, immune cells and epithelial barrier function in a mechanistic, dynamical, quantitative and integrative way.

A key element of our model to determine the healthy and pathological phenotypes is a "double switch motif " (Domínguez-Hüttinger et al., in press). The first switch describes activation of the TLR2 signaling pathway by apically located bacteria (**Figure 1C**). It can reflect both the resting, homeostatic Spupper airway interactions (when R = Roff) characteristic of Sp as a commensal bacterium, and the transient host response to a Sp challenge (when R = Ron). Failure to inactivate this R-switch due to impaired host-pathogen interactions, for example by weakened immune responses (**Figures 4B**, **5B**), can have long term consequences such as immunological scarring (**Figure 3C**) that require treatment to resolve it. The second switch distinguishes a transient growth of S<sup>v</sup> that can be

contained without treatments (when S<sup>v</sup> ≤ S ∗ v ) and an invasive infection (when S<sup>v</sup> > S ∗ v ) which would require a large dose of antibiotics treatments (**Figure 1D**). Our model simulations predict that invasive infection is developed within 36 h, in consistent with experimental observations (Andonegui et al., 2009) that were not used for development of our model. Our model analysis identified the most likely risk factors for an increased susceptibility to develop invasive infection, in response to transient Sp challenges (**Figures 4**, **5**). Based on the state of this double-switch motif, we characterized four different phenotypes (**Figure 3**), and identified those susceptible cohorts that require specific antibiotics treatment to prevent or revert the adverse effects of a Sp challenge. We further used our mathematical model to calculate the minimal strengths and durations of antibiotics application to effectively treat each of these disease phenotypes (**Figure 7**). These results suggest that the proposed quantitative and systems-level framework of Sp infection can be used to design optimal and personalized treatment strategies, as it can predict the minimal application times that are required to achieve prevention or remission for individual patient cohort.

While our mathematical model was constructed based on murine and in vitro experiments, future calibration of the model with human data could make the proposed mathematical modeling framework directly translatable to the clinic, to help stratification of patients and identification of patient-specific optimal treatment strategies. For example, our model analysis suggested that the efficacy of bacterial killing by immune cells (**Figures 4**, **5**) could be used as a marker to distinguish vulnerable patient cohorts who would require preventive treatments before the onset of sepsis. This efficacy could be determined ex vivo, from serum or broncheoalveolar lavage fluid extracted from patients, to predict patient-specific responses characteristic of those vulnerable patient cohorts. A similar approach to stratify patients based on measurements of isolated components of a more complex physiological system has been shown to be effective for other complex diseases (Fey et al., 2015). The computational method demonstrated in this paper could then allows us to predict the minimal strength and duration of antibiotics application for individual patient cohort and for a specific antibiotics, given the experimentally determined information on the efficacy of the antibiotics (Mandell et al., 2007; Prina et al., 2015b) and the growth rate of the pneumococcal strain in the patients' serum. Our model will also enable us to investigate and design preventive strategies by early vaccines against invasive infection in patient cohorts who are identified to be high-risk. Extension of our modeling framework to human disease will also require systematic investigation of the dosedependent outcome of Sp-airway interactions (Yershov et al., 2005). Another interesting future research direction includes the assessment of the long-term effects of immunological scarring on subsequent Sp challenges with different amplitudes and frequencies to identify the mechanisms behind the increased risk of developing serious infections after a first bacterial challenge (Habibzay et al., 2013). Finally, extending our model to incorporate the local spread of Sp from the upper airway epithelium to the lung and other sites on the respiratory epithelium that are normally sterile, for example by combining our model to the model of Smith et al. (2013), could allow us to investigate the association between a dysregulated colonization of the upper airway epithelium and the development of pneumococcal pneumionia.

The results of our mathematical model of commensal bacteria infection at the upper airway epithelium shed light on the mechanism behind a loss of homeostasis caused by dysregulation of the complex interactions between epithelial surfaces and microorganisms. A key element in this control structure is a "double-switch motif," which has been shown to govern other complex epithelial diseases, such as Atopic dermatitis (Domínguez-Hüttinger et al., in press) and cancer (Tian et al., 2013). Analysis of complex disease with a mechanistic, quantitative and systems-level framework as proposed here will help to reveal further general mechanisms underlying epithelium function in health and disease.

# 4. METHODS

#### 4.1. Model Description

The proposed model for commensal bacterial infection describes the dynamics of bacterial load on the surface of the airway epithelium barrier, Sa(t)[CFU/ml], infiltrated bacterial load, Sv(t)[CFU/ml], concentrations of neutrophils and macrophages on the surface of the mucosal barrier, N(t) and M(t) [cells/ml], and the strength of barrier integrity, B(t) relative to the maximum strength, by

$$\begin{split} \frac{d\mathbf{S\_{\hat{a}}(t)}}{dt} &= \frac{\kappa\_{\rm S}}{\mu\_{\rm S}} \mathbf{S\_{\hat{a}}(t)} (1 - \mathbf{S\_{\hat{a}}(t)}) - \frac{\theta\_{\rm S}}{1 + \epsilon\_{\rm S\&} B(t)} \mathbf{S\_{\hat{a}}(t)} \\ &- \phi\_{\rm NS} N(t) \mathbf{S\_{\hat{a}}(t)} - \phi\_{\rm MS} M(t) \mathbf{S\_{\hat{a}}(t)}, \end{split} \tag{1a}$$

$$\frac{d\mathcal{S}\_{\rm V}(t)}{dt} = \kappa\_{\rm S}\mathcal{S}\_{\rm V}(t) + \frac{\theta\_{\rm S}}{1 + \epsilon\_{\rm SB}B(t)}\mathcal{S}\_{\rm a}(t) - \frac{\delta\_{\rm S}}{K + \mathcal{S}\_{\rm V}(t)}\mathcal{S}\_{\rm V}(t), \tag{1b}$$

$$\frac{dN(t)}{dt} = \alpha \frac{R(\mathbf{S}\_{\mathbf{a}}(t))}{(1 + \epsilon\_{\rm NB}B(t))(1 + \epsilon\_{\rm NM}M(t))} N\_{\rm V} - \delta\_{\rm N}N(t), \tag{1c}$$

$$\frac{dM(t)}{dt} = \beta \frac{N(t)}{1 + \epsilon\_{\rm MB} B(t)} M\_{\rm V} - \delta\_{\rm M} M(t), \tag{1d}$$

$$\frac{dB(t)}{dt} = \frac{\kappa\_{\rm B}}{1 + \epsilon\_{\rm BS} R(\rm S\_a(t))} B(t)(\vec{B} - B(t))$$

$$-\phi\_{\rm SB} R(\rm S\_a(t))B(t) - \phi\_{\rm NB} N(t)B(t). \tag{1e}$$

The variable R(Sa(t)) denotes the Sa-dependent TLR activation level described by a perfect switch,

$$R(\mathcal{S}\_{\mathbf{a}}(t)) = \begin{cases} R\_{\text{off}} & \text{if } \mathcal{S}\_{\mathbf{a}}(t) < \mathcal{S}^{-} \text{ or } \{\mathcal{S}^{-} \le \mathcal{S}\_{\mathbf{a}}(t) < \mathcal{S}^{+} \text{ and} \\ & R(\mathcal{S}\_{\mathbf{a}}(t^{-})) = R\_{\text{off}}\}, \\ R\_{\text{on}} & \text{if } \mathcal{S}\_{\mathbf{a}}(t) \ge \mathcal{S}^{+} \text{ or } \{\mathcal{S}^{-} \le \mathcal{S}\_{\mathbf{a}}(t) < \mathcal{S}^{+} \text{ and} \\ & R(\mathcal{S}\_{\mathbf{a}}(t^{-})) = R\_{\text{on}}\}, \end{cases} \tag{2}$$

where t <sup>−</sup> is a time slightly before the time t. The dynamics of the TLR activity stabilizes within hours (Filewod et al., 2009; Witt et al., 2009; Hoffman et al., 2015).

The growth of the bacterial load on the apical side of the barrier, Sa, is modeled by a logistic equation (Smith et al., 2011), where the growth rate is limited by a carrying capacity (saturation term µS) that reflects the limited availability of nutrients in the epithelial lumen (Burnaugh et al., 2008). S<sup>a</sup> is eradicated by immune cells, N and M, and transmigrates to the basal side of the epithelial barrier. The transmigrated bacteria, Sv, is assumed to grow exponentially in the blood vessel with abundant nutrients, but is killed by resident immune cells. The capacity to contain S<sup>v</sup> is described by the saturated degradation of Sv, leading to the complete decay of S<sup>v</sup> if it is below the threshold S ∗ v , which corresponds to the unstable steady state of the ODE for S<sup>v</sup> when S<sup>a</sup> = S −.

Recruitment of neutrophils (N) and macrophages (M) to the site of infection from their respective pool in the blood vessel (Nv) and the airway tissues (Mv) is inhibited by the epithelium barrier integrity (B), and is enhanced by the TLR activation (R) and the recruited neutrophils, respectively. Recruitment of N is further inhibited by M. N and M decay with a respective constant decay rate, as de novo production of the immune cells does not occur outside the bone marrow (Tak et al., 2013) and they do not divide in the epithelial tissue.

The self-recovery of the mucosal barrier to its homeostatic level (Nusrat et al., 1997; Coyne et al., 2002; Heijink et al., 2012) is modeled in a phenomenological manner, with the recovery rate being compromised by a decreased gene expression of epithelial cell differentiation markers (Clarke et al., 2011) induced by TLR activation. The barrier is directly damaged by transmigrating neutrophils and by proteases that are activated via TLR signaling (Chun and Prince, 2009). The switch-like activation of the TLR signaling is triggered by apically located bacteria, and is modeled by a phenomenological representation (Mochan et al., 2014; Domínguez-Hüttinger et al., in press). For simplicity, the inhibition by x is modeled phenomenologically by <sup>1</sup> 1 + x .

### 4.2. Numerical Integration of the Hybrid Model

All the numerical model analysis was conducted using MATLAB version R2014a (The MathWorks, Inc., Natick, MA, USA). Numerical integration was conducted by ode15s from the initial conditions corresponding to a transient Sp challenges with Sa(0) = 10<sup>7</sup> CFU/ml, Sv(0) = 0 CFU/ml, N(0) = 0 cells/ml, M(0) = 10 cells/ml, and B(0) = 1, with R(0) = 1, as in the experiments in Zhang et al. (2009). The switch-dependent governing equation was chosen by the event-location function.

### 4.3. Modeling Antibiotics—Calculation of Minimal Strength and Minimal Duration of Antibiotics Treatment

We model the effects of bactericidal antibiotics, such as penicillin, ceftriaxone and amoxilin, which are commonly prescribed to treat pneumococcal infection (Mandell et al., 2007; Prina et al., 2015b) by dS<sup>v</sup> dt = −VSv(t) (systemic application of antibiotics) and dS<sup>a</sup> dt = −ASa(t) (apical application of antibiotics), where V and A represent a constant strength of antibiotics that kill Sv(t) and Sa(t), respectively. The strength of the antibiotics (with a unit of 1/h) is described by V = EVD<sup>V</sup> or A = EADA, where E<sup>V</sup> and E<sup>A</sup> is the antibiotics killing efficacy and D<sup>V</sup> and D<sup>A</sup> is the amount applied, and can be chosen in the clinic by either selecting an antibiotic with a particular killing efficacy and/or adjusting the dose administrated. The killing efficacy of an antibiotic over a specific bacterial strain is commonly evaluated by the Minimal Inhibitory Concentration (MIC), the lowest concentration of antibiotics that will inhibit the visible growth of a bacterium after overnight incubation in a kinetic growth assay. From such experimental information, the antibiotics efficacy in our model, E<sup>A</sup> (and EV) can be calculated as E<sup>A</sup> = κS µS (1 − SMIC(t)) × 1 D MIC A , where SMIC(t) is the concentration of Sp exposed to an antibiotics dose of D MIC <sup>A</sup> = MIC, and hence does not increase further. This expression is obtained from the steady state equation dSMIC(t) dt = κSµS(1 − SMIC(t)) − EAD MIC A SMIC(t) = 0, which holds for the value of SMIC(1day) = S<sup>a</sup> in these experiments.

Minimal strength of antibiotics treatment to achieve remission was determined by checking whether Sv(t) < S ∗ v or Sa(t) < S <sup>−</sup> is achieved while gradually increasing V or A, respectively, by an increment of 0.01. The minimal duration of treatments corresponds to the time required to achieve S<sup>a</sup> = S −. The minimal antibiotics treatment regimens to revert immune scarring without invasive infection (**Figure 7A**) were calculated under the assumptions that the treatment starts when S<sup>a</sup> reached its steady state, after a transient S<sup>a</sup> challenge that was modeled with initial conditions of Sa(0) = 10<sup>7</sup> CFU/ml, Sv(0) = 0 CFU/ml, N(0) = 0 cells/ml, M(0) = 10 cells/ml, and B(0) = 1, with R(0) = 1. For the regimens to prevent invasive infection (**Figure 7B**), as well as to reverse immunological scarring and to prevent sepsis (**Figure 7C**), we assumed that the treatment starts at the time of the S<sup>a</sup> challenge.

### 4.4. Robustness Analysis and Parameter Sensitivity Analysis

We varied all the model parameters over one order of magnitude (0.1 - 10 times) around the nominal values, and the initial conditions N(0), M(0) and B(0) within the intervals [0 1,000], [0 100] and [0 1], respectively. Robustness of the healthy behavior was tested by simultaneously varying all the parameters which were sampled from uniform distributions for 10, 000 iterations. The global parameter sensitivity was evaluated by 10, 000 iterations using the Global Sensitivity Analysis Toolbox for Matlab (Cannavó, 2012), with respect to the final concentrations of S<sup>v</sup> and R at 7 days post Sp challenge with Sa(0) = 10<sup>7</sup> CFU/ml.

### 4.5. Focal Point Analysis

We conducted a focal point analysis to determine the long-term behavior of the model in absence of a Sp challenge. Following the methodology in Oyarzún et al. (2012), we considered two subsystemsfor Equation (1), defined by fixing R to either R = Roff or to R = Ron, and evaluated the local stability of their steady states.

#### ETHICS STATEMENT

Blood used in pneumococcal growth assays was obtained from healthy volunteers who had given written consent. Ethical approval for this work was obtained from the Tissue Management Committee of the ICHTB (Project: R14053, ICHTB HTA license: 12275, REC Wales approval: 12/WA/0196). Tissue samples were provided by the Imperial College Healthcare NHS Trust Tissue Bank. Other investigators may have received samples from these same tissues. The research was supported by the National Institute for Health research (NIHR) Biomedical Research Centre based at Imperial College Healthcare NHS Trust and Imperial College London. The views expressed are those of the author(s) and not necessarily those of the NHS, NIHR, or the Department of Health.

# AUTHOR CONTRIBUTIONS

ED, TC, NB, and RT designed the research, developed the mathematical model and analyzed data. ED performed the computational experiments. TC performed the Sp growth experiments. ED, TC, and RT wrote the paper.

#### FUNDING

ED acknowledges funding from the Mexican Council for Science and Technology (CONACyT, Ph.D. scholarship 212800) and from the National Autonomous University of Mexico (UNAM, postdoctoral scholarship). TC is a Sir Henry Dale Fellow jointly funded by the Wellcome Trust and Royal Society (grant No. 107660/Z/15/Z). RT

## REFERENCES


acknowledges EPSRC Career Acceleration Fellowship (EP/G007446/1).

#### ACKNOWLEDGMENTS

We acknowledge George Buckle for his inputs on the parameter derivation of the model. Work with human blood was conducted in collaboration with Dr. Andrew Edwards (Imperial College London).

#### SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: http://journal.frontiersin.org/article/10.3389/fphys. 2017.00115/full#supplementary-material

junctions by proinflammatory cytokines. Mol. Biol. Cell 13, 3218–3234. doi: 10.1091/mbc.E02-03-0134


stratum corneum natural moisturizing factor. Skin Res. Technol. 21, 366–372. doi: 10.1111/srt.12201


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2017 Domínguez-Hüttinger, Boon, Clarke and Tanaka. This is an openaccess article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Quantifying the Frictional Forces between Skin and Nonwoven Fabrics

#### Kavinda Jayawardana<sup>1</sup> , Nicholas C. Ovenden<sup>1</sup> \* and Alan Cottenden<sup>2</sup>

*<sup>1</sup> Department of Mathematics, University College London (UCL), London, UK, <sup>2</sup> Department of Bioengineering and Medical Physics, University College London (UCL), London, UK*

When a compliant sheet of material is dragged over a curved surface of a body, the frictional forces generated can be many times greater than they would be for a planar interface. This phenomenon is known to contribute to the abrasion damage to skin often suffered by wearers of incontinence pads and bed/chairbound people susceptible to pressure sores. Experiments that attempt to quantify these forces often use a simple capstan-type equation to obtain a characteristic coefficient of friction. In general, the capstan approach assumes the ratio of applied tensions depends only on the arc of contact and the coefficient of friction, and ignores other geometric and physical considerations; this approach makes it straightforward to obtain explicitly a coefficient of friction from the tensions measured. In this paper, two mathematical models are presented that compute the material displacements and surface forces generated by, firstly, a membrane under tension in moving contact with a rigid obstacle and, secondly, a shell-membrane under tension in contact with a deformable substrate. The results show that, while the use of a capstan equation remains fairly robust in some cases, effects such as the curvature and flaccidness of the underlying body, and the mass density of the fabric can lead to significant variations in stresses generated in the contact region. Thus, the coefficient of friction determined by a capstan model may not be an accurate reflection of the true frictional behavior of the contact region.

#### Edited by:

*Krasimira Tsaneva-Atanasova, University of Exeter, UK*

#### Reviewed by:

*Adam Jan Gadomski, University of Science and Technology, Poland Luis Vilhena, University of Coimbra, Portugal*

\*Correspondence:

*Nicholas C. Ovenden n.ovenden@ucl.ac.uk*

#### Specialty section:

*This article was submitted to Computational Physiology and Medicine, a section of the journal Frontiers in Physiology*

Received: *30 November 2016* Accepted: *09 February 2017* Published: *06 March 2017*

#### Citation:

*Jayawardana K, Ovenden NC and Cottenden A (2017) Quantifying the Frictional Forces between Skin and Nonwoven Fabrics. Front. Physiol. 8:107. doi: 10.3389/fphys.2017.00107* Keywords: friction, nonwoven fabrics, shell theory, capstan equation, abrasion dermatitis, incontinence, free boundary

# 1. INTRODUCTION

Pressure ulcers are an area of localized cutaneous damage typically associated with pressure from bony protuberances on aged skin. They can develop when a large amount of pressure is applied to an area of skin over a short period of time or occur when less pressure is applied over a prolong period of time. When pressure is applied to soft tissue, it may result in completely or partially obstructed blood flow to the soft tissue, starving the tissue of oxygen and nutrients, which eventually leads to necrosis in the affected area, and thus an ulcer. Shear (i.e., constant and prolonged static friction) is also a cause, as it can pull on blood vessels that feed the skin, consequently restricting the blood flow. Pressure ulcers often occur in very sedentary individuals, such as those with impaired mobility (Maklebust and Sieggreen, 2001).

It is assumed that friction contributes to skin damage via stripping of the epidermal layer of the skin, creating an environment conducive to further skin damage due to friction. An alteration in the coefficient of friction increases the skin's adherence to the outside surface, which can eventually lead to wounds and infections. A publication by Murray et al. (2001) highlights many preventive

**224**

measures for pressure ulcers. For example, to eliminate shear and friction, it is recommended that the exposed skin is covered by protective dressings, padding or sheepskin. For those who are bedridden, elevating the foot of bed to 20◦ is advised when sitting to prevent sliding as well as maintaining the head of the bed at the lowest possible elevation, consistent with the individual's medical condition and comfort.

Athletes, due to excessive movement, are also subject to repeated mechanical trauma to the skin, which is often painful (Bergfeld and Taylor, 1985). Conditions such as fissure of the nipple (Powell, 1983; Conklin, 1990) and friction blisters (Herring and Richie, 1990) are well documented traumas to the skin caused by friction. It is also established that friction plays a role in the development of dermatitis in other settings. In many cases, it is observed that friction damages the stratum corneum and the stratum basale to varying degrees (Wilkinson, 1985). Friction is also believed to play a significant role in incontinence-associated dermatitis with published literature on the coefficient of friction (measuring techniques and actual measurements) between human skin and fabrics in incontinence related publications (Berg, 1987; Cottenden et al., 2008a,b; Cottenden and Cottenden, 2009). The NHS estimates that between 3 and 6 million adults in the UK have some degree of urinary incontinence (Irwin et al., 2006), and the prevalence is set to increase due to an aging population. It is documented that the wearing of incontinence pads over prolong periods of time is a major cause of incontinence associated dermatitis. Although the pads absorb moisture, they can also act as a barrier that prevents water from escaping. This leads to over-hydration of the stratum corneum in the epidermis (i.e., the upper most part of the skin). Scheuplein and Blank (1971) found that an increase in skin hydration leads to an increase in the thickness of the stratum corneum, resulting in a weakening of the cell structure. Moreover, tests conducted on adults and on infants showed that overhydration of the stratum corneum is responsible for a threefold increase in the coefficient of friction.

It is estimated that over 400, 000 individuals develop a new pressure ulcer annually in the UK (mainly the elderly) and approximately 51, 000 of them will be admitted to hospital (Farage et al., 2007). A study conducted in 1993 showed that the cost to the NHS of treating pressure ulcers was around £180 − £321 million, approximately 0.4–0.8% of total health spending (Touche, 1993). However, more recently this figure was considered to be a substantial underestimate, even allowing for inflation (Bennett et al., 2004). Bennett et al. (2004) found that the cost of treating pressure ulcers in UK (excluding methicillinresistant Staphylococcus aureus (MRSA), surgical interventions and litigation costs) ranges between £1.4 to £2.1 billion annually, which is over 4% of gross NHS expenditure. The costs were deduced by estimating the daily cost of the resources required to deliver protocols of care reflecting good clinical practice. In the USA, the Omnibus Budget Reconciliation Act in 1987 made it easier for claimants to prove that a provider had been negligent following the development of pressure ulcers. Between 1992 and 1996, the median settlement value following successful litigation for negligence regarding a pressure ulcer was \$279,000 (Thomson and Brooks, 1999).

The above evidence strongly indicates that preventive measures are immensely important. Lowering the prevalence of pressure ulcers should lead to fewer resources being spent their treatment, thus reducing overall spending in the NHS. To design new products and protocols that reduce skin damage, there has been a great deal of experimental work aiming to accurately quantify the mechanical forces generated by fabrics and other materials in contact with human skin and its response (Dowson, 1996; Sivamani et al., 2003; Silver et al., 2003; Cottenden et al., 2008b). In these experiments it is vital to quantify the relationship between normal and tangential forces across the contact region and this is often achieved by determining a coefficient of friction (Gwosdow et al., 1986; Zhang and Mak, 1999; Cottenden et al., 2008b).

In recent years, the UCL Continence & Skin Technology group has developed and validated a novel method for measuring friction between fabrics and skin (Cottenden et al., 2008a,b), as well as developing mathematical models (Cottenden and Cottenden, 2009) to determine coefficients of friction. One aim of this research is to predict the magnitude of potentially damaging frictional forces generated between skin in contact with certain fabrics, such as the nonwoven materials used in incontinence pads. **Figure 1** shows the basic setup of the experiments and a typical force trace obtained by the tensometer measurements. The tensometer is used to measure the force needed to drag a strip of nonwoven fabric (30 mm wide in the work referenced here) over the volar forearm of a volunteer while a weight (an applied mass, m) is attached to the other end of the strip. The arm is supported such that its upper surface, in the vicinity of the fabric strip, is held horizontal and at the same height as the tensometer grips, while the other end of the strip hangs vertically in a plane perpendicular to the pull direction of the tensometer. The angle of the arc of contact between the arm and strip is typically π/2 although, in the group's work using arm phantoms (Cottenden et al., 2008a), angles other than π/2 were achieved by adjusting the height of the tensometer grips relative to the arm phantoms.

One remarkable aspect of the experimental results obtained is that their rather simple capstan-type model (Rao et al.,

2003; Jung et al., 2008), based only on the ratio of tensions at either end of the fabric strip and the arc of contact, appears able to produce reliable values for the coefficient of friction from experiments with various applied weights across different human subjects. This is despite the fact that during experiments significant skin deformation, rucking and wrinkling is often observed (**Figure 2**).

The capstan equation is the most perfect example of a beltfriction model, which describes behavior of a belt-like object moving over a rigid obstacle subjected to friction (Rao et al., 2003). Consider a membrane (i.e., a two-dimensional elastic body) with zero-Poisson's ratio or a string (i.e., a one-dimensional elastic body with arbitrary Poisson's ratio) over a rough rigid cylinder subject to appropriate boundary conditions such that the body in question is at limiting equilibrium, in other words it is at the point of slipping. This is a simple belt-friction problem and its properties are well-described by the capstan equation. However, we ask: what if our rigid contact body is no longer a cylinder, but some arbitrary geometry? Should a simple capstan equation apply to these geometries? Would a capstan-type equation still apply in the case where the underlying body is not rigid but deformable. Are the coefficients of friction obtained by a capstan equation reliable in such cases? These questions are the main focus of this paper and we examine the aspects of geometry and deformability of the underlying body separately in Sections 2 and 3 respectively.

In Section 2, we look at a thin membrane pulled dynamically at a constant speed over a rigid body. The section begins by introducing the capstan equation and highlighting some experimental data on arm phantoms that does not fit the predictions of the capstan model. We then extend an established model for Coulomb's law of static friction (Kikuchi and Oden, 1988) to curvilinear coordinates and use it in conjunction with a numerical model of a thin membrane pulled dynamically over a rigid body to investigate how the calculated coefficient of friction varies with (i) membrane parameters such as Poisson's ratio, Young's modulus and mass density; (ii) the speed and the applied tensions at the membrane edges; and (iii) the underlying geometry of the body, specifically, the Gaussian curvature. In Section 3, we examine the behavior of shells supported by elastic foundations when subjected to a friction condition. Faced with a free-boundary problem at the contact region, we use the same model for Coulomb's law of static friction (Kikuchi and Oden, 1988) to derive a more computationally tractable displacementbased static friction condition. We then take the overlying shell theory and use the displacement-based friction condition to transform the model into a constrained elastic two-body contact problem and explicitly derive the governing equations and the boundary conditions for the static friction problem of a thin shell-membrane on an elastic foundation. Finally, we present some numerical results to examine how the thickness

and elasticity of the foundation affect the displacement and shear stress across the contact region. Discussion and conclusions follow in Section 4.

# 2. MODELING A NON-WOVEN FABRIC AS A MEMBRANE SUPPORTED BY A RIGID FOUNDATION WITH FRICTION

# 2.1. Capstan Equation and Applications in Friction Modeling

The capstan equation or, as otherwise known, Euler's equation of tension transmission, is the relationship governing the maximum applied tension Tmax with respect to the minimum applied tension T<sup>0</sup> of an elastic string wound around a rough cylinder. The governing equation is given by

$$T\_{\text{max}} = T\_0 \exp(\mu\_F \,\theta) \,, \tag{1}$$

where θ is the contact angle and µ<sup>F</sup> is the coefficient of friction. By string we mean a one-dimensional elastic body and rough is an engineering term implying that the contact area exhibits friction. Note that the coefficient of friction is the physical ratio of the shear force and the normal force between two contacting bodies. In engineering, the capstan equation describes a body under a load, in equilibrium, involving friction between rope and a wheel-like circular object, and thus it is widely used to analyse the tension transmission behavior of cable-like bodies in contact with circular profiled surfaces (Jung et al., 2008; Baser and Konukseven, 2010) as well as in the field of robotics (Behzadipour and Khajepour, 2006).

As an example of the application of a capstan-type equation to experimental results, consider the results of Cottenden et al. (2008a) where the coefficient of friction is determined from experiments as that shown in **Figure 1** of a non-woven fabric strip in contact with various arm phantoms made from Plaster of Paris and covered in Neoprene. Note, in particular, Figure 11 of Cottenden et al. (2008a) which shows the coefficients of friction obtained with different geometries, applied weights, and contact angles. While the capstan equation proves quite successful in obtaining coefficients of friction, the authors observe a steady increase in the mean coefficient of friction as the applied weight increases. Such dependence contradicts the assumptions of Equation (1). There is also variation in the coefficients of friction measured across different geometries and contact angles which seems to get wider as the applied mass is reduced. The authors acknowledge the apparent dependence of the coefficient of friction on the applied weight in Cottenden et al. (2008a) but highlight that the mean variation is small compared to the scatter of the data. The authors suggest that the departure from the capstan equation is likely to be caused by an interaction between the Neoprene on the underlying body and the moving nonwoven fabric at large tension.

The raw data of the case of a fabric strip in contact with an arm phantom of cylindrical cross section with a <sup>127</sup> <sup>360</sup>π contact angle can be found in Karavokiros' masters thesis (see table 2a of Karavokiros, 2007); for convenience, this raw data is reproduced here in **Table 1**. The table shows five repeated measurements involving dragging the fabric strip over the arm phantom with different applied masses (m in **Figure 1**). **Figure 3** is a plot using these data of the measured tension ratio δτ = Tmax/T0, vs. the applied mass, where the mean ratio obtained for each applied mass is also shown. Note that Equation (1) implies that the tension ratio is constant for all applied masses, i.e., δτ = exp(µdθ0), where µ<sup>d</sup> and θ<sup>0</sup> are constants. However, **Figure 3** suggests as the applied mass increases, the tension ratio increases also. Of course, such an effect could be attributed to a number of factors, including experimental errors, but one possibility is that the standard capstan equation is simply not valid in these cases. To test this, we now develop two, more sophisticated, threedimensional numerical models of a thin compliant sheet over an underlying body and vary the geometry and material properties of both the thin sheet and underlying body, as well as the applied tensions.

TABLE 1 | Tensometer readings: Plaster of Paris cylinder covered in Neoprene with <sup>127</sup> <sup>360</sup> π contact angle, where g is the acceleration due to gravity.


*Applied mass m and tension F refer to the experimental schematic in* Figure 1*. Data courtesy of Karavokiros (2007).*

#### 2.2. Belt-Friction Model

In this section, we derive a pure-traction belt-friction model to describe the behavior of dynamic membranes supported by static rigid foundations.

Let ω ⊂ R <sup>2</sup> be a simply connected open bounded domain with the sufficiently smooth boundary ∂ω and let σ ∈ C 2 (ω; **E** 3 ) be an injective immersion. Now assume that an isotropic elastic membrane is in contact with a rigid surface with positive mean curvature initially such that, in the stress-free configuration of the membrane, the contact area can be parameterized by the immersion σ(x 1 , x 2 ) in curvilinear coordinates. Now we assert that the membrane is dynamic but the contact area remains static and constant as ω. Invoking a similar terminology to that of Cottenden and Cottenden (2009), the governing Cauchy momentum equations for the membrane stress τ αβ (α, β = 1, 2) in tangential and normal directions can be derived thus

$$\nabla\_{\alpha} \tau^{\alpha \beta}(\mathfrak{u}) + f\_r^{\beta}(\mathfrak{u}) + \mathfrak{g}\_r^{\beta} = \varrho \left( \partial\_{tt} \mathfrak{u}^{\beta} + \Gamma\_{\alpha \gamma}^{\beta} \partial\_t \mathfrak{u}^{\alpha} \partial\_t \mathfrak{u}^{\gamma} \right), \tag{2}$$

$$F\_{\text{[II]}\alpha\underline{\gamma}}\pi^{\alpha\underline{\gamma}}(\mathfrak{u}) + f\_r^3(\mathfrak{u}) + \mathfrak{g}\_r^3 = \varrho F\_{\text{[II]}\alpha\underline{\gamma}}\partial\_t u^\alpha \partial\_t u^\gamma \,. \tag{3}$$

with the boundary conditions

$$n\_{\alpha} \pi^{\alpha \beta}(\mathfrak{u})|\_{\partial \alpha} = \mathfrak{r}\_0^{\beta}|\_{\partial \alpha},\tag{4}$$

where **u** ∈ C 2 (ω; R 2 ) is the displacement field and ̺ is the mass density of the membrane, **g**<sup>r</sup> ∈ C 0 (ω; R 3 ) is an external loading field, f β <sup>r</sup> (**u**) are the shear force densities, f 3 r (**u**) is the normal reaction density, **n** ∈ C 0 (ω; R 2 ) is the unit outward normal to the boundary and τ <sup>0</sup> ∈ C 0 (ω; R 2 ) is the traction field applied to the boundaries of the membrane. The covariant second fundamental form tensor of σ with respect to the curvilinear coordinates is defined as

$$F\_{[\Pi]a\beta} = N\_i \partial\_{a\beta} \sigma^{\dot{i}} \text{, } \forall \; \alpha, \beta \in \{1, 2\} \text{ .}$$

where **N** = ||∂1σ × ∂2σ||−<sup>1</sup> (∂1σ × ∂2σ) is the unit normal to the surface σ, × is the Euclidean cross product and || · || is the Euclidean norm.

Notice that this is a pure-traction problem. This implies that the boundary traction field τ <sup>0</sup> cannot be arbitrary chosen. To proceed, we assume that the velocity, the acceleration and the force density fields are known and fixed prior to the problem. Typically here, in line with the tensometer experiments, we assume that the membrane moves at a given constant speed. The time-dependent translational part of the displacement is thus easily subtracted out, leaving the residual displacement u β . We now invoke the compatibility condition for pure-traction problems (see Section 1.3.4 of Necas et al., 2011 or Section 1.8 of Ciarlet, 2000) to find

$$\begin{split} & \int\_{\partial\omega} \mathbf{r}\_0^{\beta} \, \boldsymbol{w}\_{\beta} \, d\langle \partial\omega \rangle \\ & \qquad + \int\_{\omega} \left( f\_r^{\beta}(\boldsymbol{\omega}) + \boldsymbol{g}\_r^{\beta} - \varrho \left( \partial\_{tt} \boldsymbol{u}^{\beta} + \Gamma\_{\alpha\gamma}^{\beta} \partial\_t \boldsymbol{u}^{\alpha} \partial\_t \boldsymbol{u}^{\gamma} \right) \right) \boldsymbol{w}\_{\beta} \, d\boldsymbol{\omega} = \mathbf{0}, \end{split} \tag{5}$$
 
$$\forall \; \mathbf{w} \in \left\{ \mathbf{v} \in H^{1}(\omega) \mid \boldsymbol{\epsilon}(\mathbf{v}) = \mathbf{0} \right\}, \tag{5}$$

where ǫ(·) is the strain tensor of a true-membrane. The compatibility condition implies that the internal forces are balanced by all the applied external forces.

Suppose now that the contact area is rough and the friction law governing this region is given by the model in chapter 10 of Kikuchi and Oden (1988) for Coulomb's law of static friction for the slip case. For a flat contact surface with normal (compressive) and tangential stress fields given by σ<sup>n</sup> and σ <sup>T</sup> respectively, Coulomb's law of static friction can be expressed by the following relationship between these surface stresses and the tangential displacement field **u**T:

$$\begin{aligned} |\sigma\_T| &< \nu\_F |\sigma\_\hbar| \Rightarrow \mathfrak{u}\_T = 0, \\ |\sigma \, r| &= \nu\_F |\sigma\_\hbar| \Rightarrow \mathfrak{u}\_T = -\lambda \sigma \, r \quad \text{for some } \lambda \ge 0. \end{aligned}$$

Here, ν<sup>F</sup> is the coefficient of friction in respect to Coulomb's law of friction. Kikuchi and Oden point out that it is not possible to mathematically analyse a variational problem for Coulomb's law using conventional mathematical methods. As a result, the question of existence of solutions to such friction problems remains an open one. To circumvent this difficulty, Kikuchi and Oden propose the following regularized version of the static friction law involving a small parameter ε:

$$-\sigma\_T = \upsilon\_F |\sigma\_n| \frac{\mathfrak{u}\_T}{|\mathfrak{u}\_T|} \quad \text{if } |\mathfrak{u}\_T| \gg \varepsilon,\tag{6}$$

$$-\sigma\_T = \nu\_F |\sigma\_n| \frac{\mathfrak{u}\_T}{\mathfrak{s}} \quad \text{if } |\mathfrak{u}\_T| \prec \mathfrak{s}.\tag{7}$$

In the limit ε → 0, Coulomb's original law of static friction is recovered. For the slip case, we assume the first condition holds for large tangential displacements which, on converting this to a membrane in contact with a rigid surface in curvilinear coordinates, yields

$$f\_r^\beta(\mathfrak{u}) + \upsilon\_F \left(\mu\_\alpha \mathfrak{u}^\alpha\right)^{-\frac{1}{2}} u^\beta f\_r^\Im(\mathfrak{u}) = 0 \,. \tag{8}$$

We now rearrange the compatibility condition Equation (5) and use Coulomb's law of friction Equation (8) to find a relationship between the coefficient of friction and the external loadings, given by

$$\begin{split} \mathbb{V}\_{\mathbb{V}} & \int\_{\boldsymbol{\alpha}} \frac{\left(\mathbb{F}\_{\left[\Pi\right] \geqslant \boldsymbol{\rho}} \tau^{\beta\gamma} \left(\boldsymbol{\mu}\right) + \mathcal{g}\_{r}^{3} - \varrho \mathbb{F}\_{\left[\Pi\right] \geqslant \boldsymbol{\rho}} \partial\_{t} \boldsymbol{u}^{\beta} \,\partial\_{t} \boldsymbol{u}^{\beta} \right)}{\left(\boldsymbol{u}\_{\delta} \boldsymbol{u}^{\delta}\right)^{\frac{1}{2}}} \boldsymbol{u}^{\alpha} \,\boldsymbol{w}\_{\alpha} \,d\boldsymbol{w} \\ & + \int\_{\left[\partial\_{t}\right]} \tau^{\alpha}\_{0} \,\boldsymbol{w}\_{\alpha} \,d\left(\partial \boldsymbol{w}\right) \\ & + \int\_{\boldsymbol{\alpha}} \boldsymbol{g}^{\alpha}\_{r} \boldsymbol{w}\_{\alpha} - \varrho \left(\partial\_{tt} \boldsymbol{u}^{\alpha} + \Gamma^{\alpha}\_{\boldsymbol{\gamma}\delta} \partial\_{t} \boldsymbol{u}^{\beta} \,\partial\_{t} \boldsymbol{u}^{\delta}\right) \boldsymbol{w}\_{\alpha} \,d\boldsymbol{w} = \boldsymbol{0} \,, \quad \text{(9)} \\ & \forall \,\boldsymbol{w} \in \left\{ \boldsymbol{\nu} \in H^{1}(\boldsymbol{\omega}) \mid \boldsymbol{\epsilon}(\boldsymbol{\nu}) = \mathbf{0} \right\}. \end{split}$$

The residual displacement u β and f j <sup>r</sup>(**u**) give us five unknowns and Equations (2), (3), and (8) provide us with five equations. Thus, the system is fully determined with boundary conditions Equation (4). Furthermore, Equation (9) provides us with a stronger system as if the coefficient of friction is unknown then a known traction can close the system and vice versa.

While it is a closed system, it is not straightforward to prove that a solution exists for this model. The problem of proving the existence of solutions arises from the function f β <sup>r</sup> (**u**), as it is a function of both **u** and ∇u. If f β <sup>r</sup> (**u**) was purely a function of **u**, then the existence of solutions may be proved by variational methods for semi-linear elliptic equations (see Badiale and Serra, 2010) but with the ∇u dependence, our model is not even a variational problem. Nevertheless, a numerical finite-difference solution is pursued in the next section.

#### 2.3. Numerical Analysis

To conduct numerical experiments, assume that we are dealing with a surface of revolution case where both the contact surface and the unstressed membrane are parameterized by the same immersion. Let this immersion be σ(x 1 , x 2 ) = (x 1 , ϕ(x 1 ) sin(x 2 ), ϕ(x 1 ) cos(x 2 ))E, where x <sup>1</sup> ∈ (0, l) and x <sup>2</sup> ∈ (− 1 2 π, 0). To permit changes in lateral curvature we assert that ϕ(x 1 ) = r<sup>0</sup> − 16c(l −1 x <sup>1</sup> − 1 2 ) 4 , where, initially, to keep the contact area as a surface of positive mean curvature, c is a positive parameter with c < r0. Note that l,r<sup>0</sup> are some positive real constants that are specified later. With some calculations, we find the first fundamental form tensor to be **F**[I] = diag((ψ1) 2 , (ψ2) 2 ), where ψ<sup>1</sup> = (1 + (ϕ ′ (x 1 ))2 ) 1 <sup>2</sup> and ψ<sup>2</sup> = ϕ(x 1 ). With a few more calculations one can find that

$$\begin{aligned} \Gamma\_{11}^1 &= (\psi\_1)^{-1} \partial\_1 \psi\_1 \,, \quad F\_{\text{[II]}1}^1 = (\psi\_1)^{-1} \varphi''(\mathbf{x}^1) \left(1 + (\varphi'(\mathbf{x}^1))^2\right)^{-1} \,, \\\Gamma\_{21}^2 &= (\psi\_2)^{-1} \partial\_1 \psi\_2 \,, \quad F\_{\text{[II]}2}^2 = -(\psi\_2)^{-1} \left(1 + (\varphi'(\mathbf{x}^1))^2\right)^{-\frac{1}{2}} \,, \end{aligned}$$

where Ŵ γ αβ are the Christoffel symbols of the second kind and **F**[II] is the second fundamental form tensor. Now given that **u** = (u 1 (x 1 , x 2 ), u 2 (x 1 , x 2 )) is the displacement field, one can derive the following:

$$\begin{aligned} \nabla\_1 u^1 &= \partial\_1 u^1 + \Gamma^1\_{11} u^1, \\ \nabla\_1 u^2 &= \partial\_1 u^2 + \Gamma^2\_{21} u^2, \\ \nabla\_2 u^1 &= \partial\_2 u^1 - (\psi\_1)^{-2} (\psi\_2)^2 \Gamma^2\_{21} u^2, \\ \nabla\_2 u^2 &= \partial\_2 u^2 + \Gamma^2\_{22} u^2. \end{aligned}$$

Now assume that our membrane is subjected to the acceleration of gravity i.e., subject to (0, 0, −g)<sup>E</sup> in Cartesian coordinates. With coordinate transforms, from Euclidean to curvilinear, one may re-express acceleration due to gravity in the curvilinear coordinates as g**J**, where

$$J = \left( -\varphi'(\mathbf{x}^1)(\psi\_1)^{-2}\cos(\mathbf{x}^2), \varphi^{-1}(\mathbf{x}^1)\sin(\mathbf{x}^2), -(\psi\_1)^{-1}\cos(\mathbf{x}^2) \right), \dots$$

and x 2 is the angle that the vector (ψ1, ψ2, 0) makes with the vector (0, 0, 1)E.

Now given that ̺ is the mass density, (0, 0, 0) is the acceleration field and (0, (ψ2) <sup>−</sup>1V, 0) is the velocity field of the membrane, then we can express the governing equations of the membrane as

$$\begin{aligned} &\left(\Lambda+\mu\right)\partial^1\left(\nabla\_{\alpha}u^{\alpha}\right)+\mu\Delta u^1+\varrho g l^1+f^1\_r(\mathfrak{u})=-\varrho\left(\psi\_1\right)^{-2}\Gamma^2\_{21}V^2, \\ &\left(\Lambda+\mu\right)\partial^2\left(\nabla\_{\alpha}u^{\alpha}\right)+\mu\Delta u^2+\varrho g l^2+f^2\_r(\mathfrak{u})=0 \,,\end{aligned}$$

and

$$\begin{aligned} \left( \left( \Lambda + 2\mu \right) \left( \partial\_1 u^1 + \Gamma^1\_{11} u^1 \right) + \Lambda \left( \partial\_2 u^2 + \Gamma^2\_{21} u^1 \right) \right) F^{1}\_{[\Pi][1]} \\ + \left( \Lambda \left( \partial\_1 u^1 + \Gamma^1\_{11} u^1 \right) + (\Lambda + 2\mu) \left( \partial\_2 u^2 + \Gamma^2\_{21} u^1 \right) \right) \\ F^{2}\_{[\Pi]2} + \varrho g l^3 + f^3\_r(\mathfrak{u}) = \varrho F^{2}\_{[\Pi]2} V^2, \end{aligned}$$

where 3 = 2λµ(λ + 2µ) −1 and λ and µ are the first and second Lamé parameters respectively. Assuming that our contact area is rough, the final governing equation required is the friction law Equation (8) where the coefficient of friction ν<sup>F</sup> is considered to be an unknown.

Now divide the boundary into sub-boundaries, so that

$$\begin{aligned} \partial \omega\_{\!\!\!f} &= \left\{ \{0\} \times \left( -\frac{1}{2}\pi, 0 \right) \right\} \cup \left\{ \{l\} \times \left( -\frac{1}{2}\pi, 0 \right) \right\}, \\ \partial \omega\_{T\_0} &= \left\{ [0, l] \times \left\{ -\frac{1}{2}\pi \right\} \right\}, \\ \partial \omega\_{T\_{\text{max}}} &= \left\{ \{[0, l] \times \{0\} \} \right\}, \end{aligned}$$

and we assert that the boundary conditions are

$$\begin{aligned} \left(\psi\_{1}\right)^{2}\partial\_{2}u^{1} + \left(\psi\_{2}\right)^{2}\partial\_{1}u^{2}|\_{\partial\alpha} &= 0 \text{ (zero-Robin)},\\ \left(\Lambda + 2\mu\right)\left(\partial\_{1}u^{1} + \Gamma\_{11}^{1}u^{1}\right) + \Lambda \left(\partial\_{2}u^{2} + \Gamma\_{21}^{2}u^{1}\right)|\_{\partial\alpha\_{\mathcal{I}}} \\ &= 0 \text{ (zero-Robin)},\\ \Lambda\left(\partial\_{1}u^{1} + \Gamma\_{11}^{1}u^{1}\right) + \left(\Lambda + 2\mu\right)\left(\partial\_{2}u^{2} + \Gamma\_{21}^{2}u^{1}\right)|\_{\partial\alpha\mathcal{I}\_{0}} \\ &= \tau\_{0} \text{ (traction)},\\ \Lambda\left(\partial\_{1}u^{1} + \Gamma\_{11}^{1}u^{1}\right) + \left(\Lambda + 2\mu\right)\left(\partial\_{2}u^{2} + \Gamma\_{21}^{2}u^{1}\right)|\_{\partial\alpha\mathcal{I}\_{\text{max}}} \\ &= \tau\_{\text{max}}\text{ (traction)} \end{aligned}$$

where τmax > τ<sup>0</sup> are positive real constants.

Finally, we take Equation (9) and modify it ever so slightly to make it easier to numerically model, thus obtaining the following relation,

$$\begin{split} & \left( \mathfrak{r}\_{\text{max}} - \mathfrak{r}\_{0} \right) \int\_{0}^{l} \psi\_{1} \psi\_{2} \, d\mathfrak{x}^{1} \\ & \quad + \varrho \int\_{\alpha^{0}} \left( \mathcal{g} f^{\alpha} \psi\_{\alpha} + (\psi\_{1})^{-1} \Gamma\_{21}^{2} V^{2} \right) \psi\_{1} \psi\_{2} \, d\mathfrak{x}^{1} \, d\mathfrak{x}^{2} \\ & \quad - \sqrt{2} \, v\_{F} \int\_{\alpha^{0}} f\_{r}^{3}(\mathfrak{u}) \psi\_{1} \psi\_{2} \, d\mathfrak{x}^{1} \, d\mathfrak{x}^{2} = \mathbf{0} \,. \end{split} \tag{10}$$

Now we are ready to conduct some numerical experiments. Our goal in this section is to investigate how variables such as the Gaussian curvature, Young's modulus, Poisson's ratio, speed and the mass density of the membrane and the traction may affect the value of the coefficient of friction obtained. Note that for our experiments, we keep the values τ<sup>0</sup> = 1, l = 1, r<sup>0</sup> = 1 and g = 9.81 fixed.

To conduct numerical experiments, we employ a secondorder accurate finite-difference method in conjunction with Newton's method for nonlinear systems. On discretising the domain, as we are dealing with curvilinear coordinates, we find that 1x <sup>2</sup> ≤ ψ01x 1 , for all ψ<sup>0</sup> ∈ {ψ1/ψ<sup>2</sup> | x <sup>1</sup> ∈ [0, l]} where 1x β is a small increment in the x <sup>β</sup> direction. For our purposes we use 1x <sup>2</sup> = 1 N−1 and ψ<sup>0</sup> = ψ1/ψ2| x 1= 1 2 l , where N = 250. We also choose to terminate our iterating process once |1 − νFm+1/νFm| falls below a certain specified tolerance, where νFm is the mth iterative solution for the coefficient of friction. Note that to numerically model Equation (10), we use the prismoidal formula (Meserve and Pingry, 1952). Unfortunately, as this is a puretraction problem iterative schemes can be highly unstable and so, to ensure convergence, the condition u 2 |∂ωT<sup>0</sup> ≤ 0 is strictly enforced.

We initially ran the numerical code using the following values: τmax = 3 2 , c = 0, E = 10<sup>3</sup> , ν = 1 4 , V = 0.01, ̺ = 0.01 and with a grid of 160 × 250 points. The grid size N was varied to confirm accuracy of the converged solution. The coefficient of friction calculated in this case is ν<sup>F</sup> = 0.195 to three significant figures. We now proceed to investigate how the variation in certain parameters may change this value and thus see whether this differs from the value predicted by the classical capstan model Equation (1).

We begin by varying the tension applied to the membrane. **Figure 4** is calculated with the values of τmax ∈ {1.25, 1.30, 1.35, ... 2.00}, c = 0, E = 10<sup>3</sup> , ν = 1 4 , V = 0.01 and ̺ = 0.01. It shows that as the tension ratio increases, the coefficient of friction also increases. This is intuitive because, as the maximum applied tension increases, the coefficient of friction must increase to maintain a constant speed. Equation (1) predicts a similar trend and so agreement with the classical capstan model seems quite good although the figure shows that the numerical code produces consistently lower values. The principal reason for the discrepancy is the non-zero mass density of the fabric which is investigated below.

Varying Poisson's ratio ν, the Young's modulus E and the speed of the membrane V do not lead to any significant changes in the coefficient of friction. But when we examine varying the mass density of the fabric we find significant alterations to the coefficient of friction determined by the model. **Figure 5** is calculated for three different applied tensions: τmax ∈ {1.50, 1.75, 2.00}, with values c = 0, E = 10<sup>3</sup> , ν = 1 4 , V = 0.01 and ̺ ∈ {0, 0.003, 0.006, ... , 0.03}. This shows as the mass density (with respect to the volume) of the membrane increases, the coefficient of friction decreases markedly in all cases. As ρ → 0 the coefficient of friction obtained from Equation (1), e.g., (2/π) ln(3/2) = 0.258 ... , is attained, but even a small mass density, ρ = O(10−<sup>3</sup> ) for instance, can lead to the accuracy being significantly reduced to only one significant figure. This feature is ignored in typical capstan equation calculations.

Finally, let us examine what effect varying the Gaussian curvature K has on the coefficient of friction. Note that, for our given geometry, K is a function of x 1 , and thus it varies across the contact surface. As our curvature parameter c changes the Gaussian curvature changes across the contact surface. For c = 0, K = 0 everywhere. For c > 0 the contact surface is a barrel-shape with non-negative (i.e., positive or zero) Gaussian curvature whereas for c < 0 the contact surface is a saddle-shape with non-positive (i.e., negative or zero) Gaussian curvature across the contact area. This implies that there exists a positive correlation between c and K, i.e., as c increases so does K and so we investigate how c relates to νF. **Figure 6** is calculated for non-negative Gaussian curvatures with the values of τmax ∈ {1.50, 1.75, 2.00}, and c ∈ {0, <sup>1</sup> 40 , 2 40 , 3 40 , 4 40 , 5 40 , 6 40 , 7 40 , 8 40 , 9 <sup>40</sup> }, E = 10<sup>3</sup> , ν = 1 4 , V = 0.01, and ̺ = 0.01. The three plots in the figure for different tension ratios demonstrate that as c increases (i.e., Gaussian curvature increases) the coefficient of friction decreases and there exists an optimum value of c, where the coefficient of friction is at a minimum, given that all other variables are constant. For our simulations, this value is observed around c = 1 8 . The initial decrease in ν<sup>F</sup> as c increases is intuitive. To illustrate this, consider a membrane pulled over a surface with high curvature. The higher curvature would leads to higher normal reaction force which, in turn, results in a higher friction force and finally a higher maximum applied tension. Also inclusion of a nonzero lateral curvature (note that we are considering a case with two positive principal curvatures) means that a higher maximum applied tension is required to support the strains in the lateral direction. Thus, for even a relatively small coefficient of friction, a higher maximum applied tension can be observed. Hence, if one kept every variable fixed, except for the Gaussian curvature and the coefficient of friction, then one would expect to see a low coefficient of friction for a high Gaussian curvature. However, the existence of a minimum ν<sup>F</sup> for positive c is a surprising outcome suggesting a possible optimum Gaussian curvature that can lead to a minimum coefficient of friction. Conversely, it is possible numerically to extend our range of c values to include {− <sup>1</sup> <sup>40</sup> , − 2 <sup>40</sup> , − 3 <sup>40</sup> , − 4 <sup>40</sup> } that represent saddle-type contact regions. These simulations suggest that such saddle-type contact regions lead to significantly higher values of ν<sup>F</sup> than for the zero-curvature case (**Figure 7**). Extending that range yet further to c = −0.02 (not shown) suggests that no maximum coefficient of friction is attained but that the coefficient of friction continues to increase as c becomes more negative. Such intriguing variations in ν<sup>F</sup> captured by our model here cannot be simulated by the classical capstan model Equation (1).

#### 3. SHELL-MEMBRANES SUPPORTED BY ELASTIC FOUNDATIONS WITH STATIC FRICTION

In this section, we examine the effect that the deformation of the underlying substrate has on the frictional forces generated in the region of contact. To do this, we derive a shellmembrane model to describe the behavior of an overlying compliant sheet on an elastic foundation subjected to static friction. However, such a computation requires our frictional law (e.g., Coulomb's law of static friction) to be imposed on a free boundary as the displacement across the contact region is unknown.

# 3.1. Deriving a Displacement-Based Frictional Law

Recall Kikuchi and Oden's model (chapter 10 of Kikuchi and Oden, 1988) for Coulomb's law of friction for slip that we extended to curvilinear coordinates Equation (8). For the static case, by eliminating the regularization parameter ε from their original Equations, (6) and (7), the friction law can be expressed in the following form

$$T\_3^{\beta}(\mathfrak{u}) + \nu\_F \left(\mathfrak{g}\_{33}\right)^{\frac{1}{2}} (\mu\_\alpha \mathfrak{u}^\alpha)^{-\frac{1}{2}} T\_3^3(\mathfrak{u}) \mathfrak{u}^\beta \le 0.1$$

Unlike the rigid substrate case, this equation is extremely difficult to impose on a free contact boundary between shellmembrane and substrate in its present form. Therefore, a more computationally tractable displacement-based approximation is sought. Assume that u <sup>β</sup> ≥ 0 and contract the above equation with u β . Noting that in our framework g<sup>33</sup> = 1, we thus find

$$\begin{split} \mu \left( \mu^{\alpha} \bar{\nabla}\_{\alpha} \mu^{3} + \frac{1}{2} \bar{\nabla}\_{3} (\mu\_{\alpha} \mu^{\alpha}) \right) &+ \ \nu\_{F} (\mu\_{\alpha} \mu^{\alpha})^{\frac{1}{2}} \\ & \quad \quad \quad \quad \quad \quad \quad (\lambda \bar{\nabla}\_{\alpha} \mu^{\alpha} + (\lambda + 2\mu) \bar{\nabla}\_{3} \mu^{3}) \leq 0 \ . \end{split}$$

Now assume that this body is in contact with an elastic foundation, thus it permits normal displacements at the boundary, so we assert that only normal derivatives are of any consequence. Hence, we may approximate the above relation as

$$
\mu \vec{\nabla}\_3 (\mu\_\alpha \mu^\alpha)^{\frac{1}{2}} + \nu\_F (\lambda + 2\mu) \vec{\nabla}\_3 \mu^3 \le 0 \dots
$$

To simplify matters further, we assert that the condition is independent of any elastic properties of the overlying body. This may be achieved by assuming λ = 0, and thus we find

$$
\bar{\nabla}\_3 \left( (\mu\_\alpha \mu^\alpha)^{\frac{1}{2}} + 2 \upsilon\_F \mu^3 \right) \le 0 \,\,\,\,\, \theta
$$

By approximating the above condition even further, we arrive at the following hypothesis:

Hypothesis 1. A shell supported by an elastic foundation with a rough contact area, that has thickness small relative to the radius of mean curvature, satisfies the following displacement-based friction condition

$$
\mu^3 \le -\frac{1}{2\nu\_F} (\mu\_\alpha u^\alpha)^{\frac{1}{2}},\tag{11}
$$

where ν<sup>F</sup> is the coefficient of friction between shell and the foundation, and **u** is the displacement field of the shell. If 2νFu <sup>3</sup> + (uαu α ) 1 <sup>2</sup> < 0, then we say that the the shell is bonded to the foundation, and if 2νFu <sup>3</sup> + (uαu α ) 1 <sup>2</sup> = 0, then we say that the shell is at limiting equilibrium.

The justifications for introducing the hypothesis are as follows. Kikuchi and Oden (1988) assert that the variational form of a linear elastic body subjected to Coulomb's law of static friction, is non-convex and non-differentiable. Thus, the existence of a (unique or otherwise) solution is an open question that cannot be proven with conventional means. But we can show that the variational form, i.e., the energy functional of a linear elastic body, subjected to the displacement-based friction condition from this hypothesis is convex, coercive and differentiable, and thus proving the existence of solutions is perfectly possible. Also, unlike the model of Kikuchi and Oden (1988), our displacement-based condition is independent of the regularization parameter ε and it is well defined for all finite values of **u**. Furthermore, we show that our problem can be numerically modeled without an initial guess of the purely normal stress, which is something Kikuchi and Oden's model is incapable of. Note, however, that we can guarantee that the condition from our hypothesis will hold as we have already asserted that the lower-surface of the shell is not hyperbolic and its mean curvature is positive. Thus, for sensible boundary conditions, we can always expect the normal displacement to be non-positive.

Does our hypothesized displacement-based friction law make physical sense? Well, consider two elastic blocks in a zerogravity scenario, block-A at the top and block-B at the bottom, where the base of block-B satisfies the zero-Dirichlet boundary condition. Now assume that the contact area of both blocks is rough and that one is applying forces to both the top and to one

side of block-A to mimic respectively compression and shear in the contact region. The higher the compression, the higher the normal displacement is toward the bottom, i.e., u <sup>3</sup> < 0, and, the higher the shear, the higher the tangential displacement is in the direction of the applied tangential force. Just as for the case of Coulomb's friction, where the bodies are in respective equilibrium given that the magnitude of the normal stress is above a certain factor of the magnitude of the tangential stress, i.e., |T 3 3 (**u**)| > ν−<sup>1</sup> F |T α 3 (**u**)T 3 α (**u**)| 1 <sup>2</sup> , we assert that the bodies are in equilibrium given that the normal displacement is below a certain factor of the magnitude of the tangential displacement, i.e., u <sup>3</sup> < − 1 2 ν −1 F (uαu α ) 1 <sup>2</sup> , if u <sup>3</sup> < 0. Note that this factor may or may not be <sup>1</sup> 2 ν −1 F , but this is the most mathematically logical factor we have derived.

#### 3.2. The Governing Equations

Let ⊂ R <sup>3</sup> be a connected open bounded domain that satisfies the segment condition with a uniform C 1 (R 3 ; R 2 ) boundary ∂ such that ω, ∂<sup>0</sup> ⊂ ∂ with ω¯ ∩ ∂ ¯ <sup>0</sup> = Ø and meas(∂0; R 2 ) > 0, and let ω ⊂ R <sup>2</sup> be a connected open bounded plane that satisfies the segment condition with a uniform C 1 (R 2 ; R) boundary ∂ω. Let **X**¯ ∈ C 2 (¯ ; **E** 3 ) be a diffeomorphism and σ ∈ C 3 (ω¯; **E** 3 ) be an injective immersion. Let **f** ∈ **L** 2 (), **f**<sup>0</sup> ∈ **L** 2 (ω) and τ<sup>0</sup> ∈ **L** 2 (∂ω).

For this section, we assume that **u** ∈ C 2 (; R 3 ), u α |<sup>ω</sup> ∈ C 3 (ω), u 3 |<sup>ω</sup> ∈ C 4 (ω) and 2νFu <sup>3</sup> + (uαu α ) 1 <sup>2</sup> ≤ 0 everywhere in ω. For the elastic foundation we define T ij(**u**) = A ijklEkl(**u**) to be the second Piola-Kirchhoff stress tensor, Eij(**u**) = 1 2 (gik∇¯ ju <sup>k</sup> + gjk∇¯ iu k ) to be the linearized Green-St Venant stress tensor, A ijkl = λ¯ g ijg kl + ¯µ(g ikg jl + g ilg jk) to be the elasticity tensor, λ¯ = (1 − ¯ν − 2ν¯ 2 ) −1 ν¯E¯ as the first Lamé parameter, µ¯ = 1 2 (1 + ¯ν) <sup>−</sup>1E¯ as the second Lamé parameter, E¯ as the Young's modulus and ν¯ as Poisson's ratio of the elastic foundation. Furthermore, **f** is some external force density field acting on the elastic foundation. The covariant first fundamental form tensor of σ with respect to the curvilinear coordinates is defined as

$$F\_{[1]\alpha\beta} = \partial\_{\alpha}\sigma\_i\partial\_{\beta}\sigma^i \text{ , } \forall \ \alpha, \beta \in \{1, 2\} \text{ .}$$

Also we regard the indices α, β, γ , δ ∈ {1, 2}. Furthermore, F[I]αγ F γβ [I] = δ β <sup>α</sup> , ∀ α, β ∈ {1, 2}. The second fundamental form tensor is as defined above in Section 2.2.

The governing equations of the elastic foundation are given by

$$
\vec{\nabla}\_i T^i\_j(\mathfrak{u}) + f\_j = 0 \,, \forall \, j \in \{1, 2, 3\} \,,
$$

with the following boundary conditions:

$$\begin{aligned} \mathfrak{u}|\_{\partial\Omega\_0} &= \mathbf{0}, \\ \bar{n}\_i T\_j^i(\mathfrak{u})|\_{\{\partial\Omega \mid \{\alpha \cup \partial\Omega\_0\}\}} &= \mathbf{0}, \; \forall j \in \{1, 2, 3\}, \end{aligned}$$

where **n**¯ is the unit outward normal to the boundary ∂ in curvilinear coordinates.

Turning now to the overlying body, we assume that the body is so thin, and the bending moments are so small, that it can be approximated by a shell-membrane. For the shell-membrane we define τ αβ(**u**) = B αβγδǫγδ (**u**) to be the stress tensor and η αβ(**u**) = B αβγδργδ (**u**) to be the negative of the change in the moments density tensor.

$$\epsilon\_{\alpha\beta}(\mathfrak{u}) = \frac{1}{2} \left( \nabla\_{\alpha} (\mu\_{\beta}|\_{\alpha}) + \nabla\_{\beta} (\mu\_{\alpha}|\_{\alpha}) \right) - F\_{[\Pi]\alpha\beta} (\mathfrak{u}^3|\_{\alpha}) \mathfrak{u}$$

is then half the change in the first fundamental form tensor,

$$\begin{split} \rho\_{\alpha\beta}(\mathfrak{u}) &= \nabla\_{\alpha}\nabla\_{\beta}(\mathfrak{u}^{\mathfrak{d}}|\_{\alpha}) - F\_{\text{[II]}\alpha\underline{\gamma}}F\_{\text{[II]}\beta}^{\underline{\gamma}}(\mathfrak{u}^{\mathfrak{d}}|\_{\alpha}) + F\_{\text{[II]}\beta\underline{\gamma}}\nabla\_{\alpha}(\mathfrak{u}^{\mathfrak{d}}|\_{\alpha}) \\ &+ F\_{\text{[II]}\alpha\underline{\gamma}}\nabla\_{\beta}(\mathfrak{u}^{\mathfrak{d}}|\_{\alpha}) + \left(\nabla\_{\alpha}F\_{\text{[II]}\beta\underline{\gamma}}\right)(\mathfrak{u}^{\mathfrak{d}}|\_{\alpha}) \end{split}$$

is the change in the second fundamental form tensor,

$$B^{\alpha\beta\cdot\gamma\delta} = \frac{2\lambda\mu}{\lambda+2\mu} F\_{\text{[I]}}^{\alpha\beta} F\_{\text{[I]}}^{\gamma\delta} + \mu (F\_{\text{[I]}}^{\alpha\gamma} F\_{\text{[I]}}^{\beta\delta} + F\_{\text{[I]}}^{\alpha\delta} F\_{\text{[I]}}^{\beta\gamma}))$$

is the elasticity tensor, λ = (1 − ν − 2ν 2 ) −1 νE is the first Lamé parameter, µ = 1 2 (1 + ν) <sup>−</sup>1E is the second Lamé parameter, E is the Young's modulus and ν is Poisson's ratio of the frictionally coupled shell. Tr(T 3 j (**u**)) = T 3 j (**u**)|<sup>ω</sup> is the normal stress of the foundation in the contact region, and **f**<sup>0</sup> is some external force density field acting on the overlying shell.

The governing equations for the shell-membrane are determined by considering a variational problem based on the following energy functional:

$$\begin{split} J(\mathfrak{u}) &= \int\_{\Omega} \frac{1}{2} A^{jkl} E\_{ij}(\mathfrak{u}) E\_{kl}(\mathfrak{u}) - f^i u\_i \, d\Omega \\ &+ \int\_{\alpha} \frac{1}{2} B^{\alpha\beta\cdot\delta} \left( h \epsilon\_{\alpha\beta}(\mathfrak{u}) \epsilon\_{\gamma\delta}(\mathfrak{u}) + \frac{1}{3} h^3 \rho\_{\alpha\beta}(\mathfrak{u}) \rho\_{\gamma\delta}(\mathfrak{u}) \right) \\ &- h f^i\_0 u\_i \, d\alpha - \int\_{\partial\omega} h \tau^i\_0 u\_i \, d(\partial\alpha) \,, \end{split}$$

where **u** is subject to the displacement-based friction condition Equation (11) and the region in which the shell is at limiting equilibrium is unknown prior to solving the problem. Following the detailed analysis in chapter 4 of Jayawardana (2016), it is possible to obtain the following governing equations for the shell-membrane:

**if** - 2νFu <sup>3</sup> + (uαu α ) 1 2 |<sup>ω</sup> < 0 (the bonded case), **then**

$$\begin{split} \nabla\_{\alpha} \boldsymbol{\pi}\_{\beta}^{\alpha}(\mathfrak{u}) + \frac{2}{3} h^{2} F\_{\text{[III]}\beta} \nabla\_{\boldsymbol{\gamma}} \boldsymbol{\eta}\_{\alpha}^{\boldsymbol{\gamma}}(\mathfrak{u}) + \frac{1}{3} h^{2} \left( \nabla\_{\boldsymbol{\mathcal{V}}} F\_{\text{[III]}\beta}^{\alpha} \right) \boldsymbol{\eta}\_{\alpha}^{\boldsymbol{\gamma}}(\mathfrak{u}) \\ - \frac{1}{h} \text{Tr} (T\_{\beta}^{3}(\mathfrak{u})) + f\_{0} \boldsymbol{\rho} = \boldsymbol{0}, \ \boldsymbol{\forall} \ \boldsymbol{\beta} \in \{1, 2\}, \\ F\_{\text{[III]}\alpha} \boldsymbol{\tau}\_{\boldsymbol{\gamma}}^{\alpha}(\mathfrak{u}) - \frac{1}{3} h^{2} \nabla\_{\alpha} \left( \nabla\_{\boldsymbol{\gamma}} \boldsymbol{\eta}^{\alpha \boldsymbol{\gamma}}(\mathfrak{u}) \right) + \frac{1}{3} h^{2} F\_{\text{[III]}\alpha}^{\delta} F\_{\text{[III]}\boldsymbol{\gamma}} \boldsymbol{\eta}\_{\delta}^{\boldsymbol{\gamma}}(\mathfrak{u}) \\ - \frac{1}{h} \text{Tr} (T\_{\beta}^{3}(\mathfrak{u})) + f\_{03} = 0 \ ; \end{split}$$

**if** - 2νFu <sup>3</sup> + (uαu α ) 1 2 |<sup>ω</sup> = 0 (the limiting equilibrium case), **then**

$$\begin{split} \upsilon\_{F}\nabla\_{\alpha}\tau^{\alpha}\_{\beta}(\bar{u}) - \frac{1}{2}\frac{u\_{\beta}}{(\iota\_{\alpha}u^{\alpha})^{\frac{1}{2}}}\mathsf{F}^{\gamma}\_{\mathrm{II}]\alpha}\tau^{\alpha}\_{\gamma}(\bar{u}) \\ + \frac{2}{3}\,\upsilon\_{F}h^{2}\mathsf{F}^{\frac{\alpha}{2}}\_{\mathrm{II}]\beta}\nabla\_{\gamma}\eta^{\gamma}\_{\alpha}(\bar{u}) + \frac{1}{6}h^{2}\frac{u\_{\beta}}{(\iota\_{\alpha}u^{\alpha})^{\frac{1}{2}}}\nabla\_{\alpha}\nabla\_{\gamma}\eta^{\alpha\gamma}(\bar{u}) \\ + \frac{1}{3}\,\upsilon\_{F}h^{2}\left(\nabla\_{\gamma}\mathsf{F}^{\frac{\alpha}{2}}\_{\mathrm{III}]\beta}\right)\eta^{\gamma}\_{\alpha}(\bar{u}) \\ - \frac{1}{6}h^{2}\frac{u\_{\beta}}{(\iota\_{\alpha}u^{\alpha})^{\frac{1}{2}}}\mathsf{F}^{\frac{\beta}{2}}\_{\mathrm{II}]\alpha}\mathsf{F}^{\frac{\alpha}{2}}\_{\mathrm{III}]\gamma}\eta^{\gamma}\_{\delta}(\bar{u}) \\ - \frac{\upsilon\_{F}}{h}\mathrm{Tr}(T^{3}\_{\beta}(\bar{u})) + \frac{1}{2h}\frac{u\_{\beta}}{(\iota\_{\alpha}u^{\alpha})^{\frac{1}{2}}}\mathrm{Tr}(T^{3}\_{\beta}(\bar{u})) \\ + \nu\_{\delta}f\_{0\beta} - \frac{1}{2}\frac{u\_{\beta}}{(\iota\_{\alpha}u^{\alpha})^{\frac{1}{2}}}f\_{0\beta} = 0, \qquad \qquad \forall\ \beta\in\{1,2\}, \end{split}$$

where **u**¯|<sup>ω</sup> = (u 1 , u 2 , − 1 2 ν −1 F (uαu α ) 1 <sup>2</sup> )|<sup>ω</sup> and (∂3u¯ 1 , ∂3u¯ 2 , ∂3u¯ 3 )|<sup>ω</sup> = (∂3u 1 , ∂3u 2 , ∂3u 3 )|ω. Finally, the boundary conditions of the overlying shell are

$$\begin{split} \left[ n\_{\alpha} \pi\_{\beta}^{\alpha} (\mathfrak{u}) + \frac{2}{3} h^{2} n\_{\mathcal{V}} \widehat{\pi}\_{\text{[II]} \beta}^{\alpha} \eta\_{\alpha}^{\mathcal{V}} (\mathfrak{u}) \right] |\_{\partial \boldsymbol{\alpha}} &= \mathfrak{u} \boldsymbol{\beta}, \; \forall \; \beta \in \{1, 2\} \;, \\ -\frac{1}{3} h^{2} n\_{\mathcal{V}} \nabla\_{\alpha} \eta^{\alpha \mathcal{V}} (\mathfrak{u}) |\_{\partial \boldsymbol{\alpha}} &= \mathfrak{u}\_{0} \; , \\ \partial\_{\boldsymbol{\beta}} (\boldsymbol{u}^{3}|\_{\boldsymbol{\alpha}})|\_{\partial \boldsymbol{\alpha}} &= \mathbf{0}, \; \forall \; \boldsymbol{\beta} \in \{1, 2\} \;, \end{split}$$

where **n** is the unit outward normal vector to the boundary ∂ω in curvilinear coordinates and τ<sup>0</sup> is the external traction field acting on the boundary of the overlying shell.

#### 3.3. Numerical Experiments

To conduct numerical experiments, assume that we are dealing with a shell-membrane of thickness h, supported by an elastic foundation, where the unstrained configuration of the foundation has an annular cross-section, characterized by the diffeomorphism X¯(x, θ,r) = (x,r sin(θ), r cos(θ))E, where (x 1 , x 2 , x 3 ) = (x, θ,r), x ∈ (−L, L), θ ∈ (−π, π], and r ∈ (a0, a), and assume that the contact region lies within x ∈ (−ℓ, ℓ), θ ∈ (− 1 2 π, 0), where 0 < ℓ < L. Let the sufficiently smooth field **u** = (u 1 (x, θ,r), u 2 (x, θ,r), u 3 (x, θ,r)) be the displacement field of the foundation. With some calculations one can find the metric tensor to be **g** = diag(1,r 2 , 1) and the covariant derivatives to be

$$\begin{aligned} \bar{\nabla}\_1 u^1 &= \partial\_1 u^1, & \bar{\nabla}\_1 u^2 &= \partial\_1 u^2, & \bar{\nabla}\_1 u^3 &= \partial\_1 u^3, \\ \bar{\nabla}\_2 u^1 &= \partial\_2 u^1, & \bar{\nabla}\_2 u^2 &= \partial\_2 u^2 + r^{-1} u^3, & \bar{\nabla}\_2 u^3 &= \partial\_2 u^3 - r u^2, \\ \bar{\nabla}\_3 u^1 &= \partial\_3 u^1, & \bar{\nabla}\_3 u^2 &= \partial\_3 u^2 + r^{-1} u^2, & \bar{\nabla}\_3 u^3 &= \partial\_3 u^3. \end{aligned}$$

With further calculations, one can express the governing equations of the foundation as

$$\begin{aligned} & (\bar{\lambda} + \bar{\mu}) \partial^1 (\bar{\nabla}\_i u^i) + \bar{\mu} \bar{\Delta} u^1 = 0 \,, \\ & (\bar{\lambda} + \bar{\mu}) \partial^2 (\bar{\nabla}\_i u^i) + \bar{\mu} \bar{\Delta} u^2 = 0 \,, \\ & (\bar{\lambda} + \bar{\mu}) \partial^3 (\bar{\nabla}\_i u^i) + \bar{\mu} \bar{\Delta} u^3 = 0 \, . \end{aligned}$$

The boundary of the foundation can be decomposed into subboundaries as

$$\begin{aligned} \partial\Omega &= \bar{\phi} \cup \partial\Omega\_0 \cup \partial\Omega\_f \\ \omega &= \{a\} \times \left( -\frac{1}{2}\pi, 0\right) \times \left( -\ell, \ell \right), \\ \partial\Omega\_0 &= \left\{ \{a\_0\} \times \left( -\pi, \pi \right] \times \left[ -L, L \right] \right\} \cup \left\{ \left( a\_0, a \right] \times \left( -\pi, \pi \right] \right\} \\ &\times \left\{ \left( -L \right) \cup \left\{ L \right\} \right\}, \\ \partial\Omega\_f &= \left\{ \left\{ a \right\} \times \left( -\pi, \pi \right] \times \left( -L, L \right) \right\} \,\,\bar{\phi} \,\,. \end{aligned}$$

Thus, we can express the boundary conditions of the foundation as

$$\begin{aligned} \mathfrak{u}|\_{\partial\Omega\_0} &= 0 \text{ (zero-Dirichlet)},\\ \left[\partial\_3\mathfrak{u}^1 + \partial\_1\mathfrak{u}^3\right]|\_{\partial\Omega\_f} &= 0 \text{ (zero-Robin)},\\ \left[r^2\partial\_3\mathfrak{u}^2 + \partial\_2\mathfrak{u}^3\right]|\_{\partial\Omega\_f} &= 0 \text{ (zero-Robin)},\\ \left[\bar{\lambda}(\partial\_1\mathfrak{u}^1 + \partial\_2\mathfrak{u}^2 + r^{-1}\mathfrak{u}^3) + (\bar{\lambda} + 2\bar{\mu})\partial\_3\mathfrak{u}^3\right]|\_{\partial\Omega\_f} &= 0 \text{(zero-Robin)}.\end{aligned}$$

Let **u**|<sup>ω</sup> = (u 1 (x, θ, a), u 2 (x, θ, a), u 3 (x, θ, a)) be the displacement field of the shell-membrane. With some calculations, one can find the first fundamental form tensor to be **F**[I] = diag(1, a 2 ) and the covariant derivatives to be

$$\begin{aligned} \nabla\_1 u^1 &= \partial\_1 u^1 \,, & \quad \nabla\_1 u^2 = \partial\_1 u^2 \,, \\ \nabla\_2 u^1 &= \partial\_2 u^1 \,, & \quad \nabla\_2 u^2 = \partial\_2 u^2 \,. \end{aligned}$$

Considering the case h 2ρ γ <sup>α</sup> (**u**)ρ α γ (**u**) ≪ ǫ γ <sup>α</sup> (**u**)ǫ α γ (**u**), with some further calculations one can express the governing equations of the shell-membrane as: 1

If [2νFu <sup>3</sup> + (u 1u <sup>1</sup> + r 2u 2u 2 ) <sup>2</sup> ]|<sup>ω</sup> < 0, then

$$\begin{aligned} \left(\Lambda + \mu\right)\partial^{1}(\nabla\_{\alpha}u^{a}) + \mu\Delta u^{1} + \frac{1}{a}\Lambda\partial^{1}u^{3} - \frac{1}{h}\text{Tr}(T\_{3}^{1}(u)) &= 0, \\ \left(\Lambda + \mu\right)\partial^{2}(\nabla\_{\alpha}u^{a}) + \mu\Delta u^{2} + \frac{1}{a}(\Lambda + 2\mu)\partial^{2}u^{3} \\ &- \frac{1}{h}\text{Tr}(T\_{3}^{2}(u)) = 0 \\ \Lambda\partial\_{1}u^{1} + (\Lambda + 2\mu)\left(\partial\_{2}u^{2} + \frac{1}{a}u^{3}\right) + \frac{a}{h}\text{Tr}(T\_{3}^{3}(u)) &= 0 \ . \end{aligned}$$

If [2νFu <sup>3</sup> + (u 1u <sup>1</sup> + r 2u 2u 2 ) 1 <sup>2</sup> ]|<sup>ω</sup> = 0, then

(3 + µ)∂ 1 (∇αu α ) + µ1u <sup>1</sup> − 3 2aν<sup>F</sup> ∂ 1 (u 1 u <sup>1</sup>+a 2 u 2 u 2 ) 1 2 − 1 h Tr(T 1 3 (**u**)) − (3 + µ) 4aν 2 F u <sup>1</sup> − (λ¯ + ¯µ) 4hν 2 F u 1 + 1 2ν<sup>F</sup> u 1 (u 1u <sup>1</sup>+a 2u 2u 2 ) 1 2 3∂1u <sup>1</sup> + (3 + 2µ)∂2u 2 + a h λ¯(∂1u <sup>1</sup> + ∂2u 2 ) + (λ¯ + 2µ¯)∂3u 3 = 0 , (3 + µ)∂ 2 (∇αu α ) + µ1u <sup>2</sup> − (3 + 2µ) 2aν<sup>F</sup> ∂ 2 (u 1 u <sup>1</sup>+a 2 u 2 u 2 ) 1 2 − 1 h Tr(T 2 3 (**u**)) − (3 + µ) 4aν 2 F u <sup>2</sup> − (λ¯ + ¯µ) 4hν 2 F u 2 + 1 2ν<sup>F</sup> u 2 (u 1u <sup>1</sup>+a 2u 2u 2 ) 1 2 3∂1u <sup>1</sup> + (3 + 2µ)∂2u 2 + a h λ¯(∂1u <sup>1</sup> + ∂2u 2 ) + (λ¯ + 2µ¯)∂3u 3 = 0 .

The boundary of the shell-membrane can be decomposed into sub-boundaries as

$$\begin{aligned} \partial\boldsymbol{\omega} &= \partial\boldsymbol{\omega}\_{T\_0} \cup \partial\boldsymbol{\omega}\_{T\_{\text{max}}} \cup \partial\boldsymbol{\omega}\_f, \\ \partial\boldsymbol{\omega}\_{T\_0} &= [-\ell,\ell] \times \left\{-\frac{1}{2}\pi\right\}, \\ \partial\boldsymbol{\omega}\_{T\_{\text{max}}} &= [-\ell,\ell] \times \{0\}, \\ \partial\boldsymbol{\omega}\_f &= \{\{-\ell\} \cup \{\ell\}\} \cup \left(-\frac{1}{2}\pi,0\right). \end{aligned}$$

Thus, we can express the boundary conditions of the shellmembranes as

$$\begin{aligned} \left[\Lambda\partial\_1 u^1 + (\Lambda + 2\mu)(\partial\_2 u^2 + a^{-1}u^3)\right]|\_{\partial\alpha\_{\overline{T}\_0}} &= \tau\_0 \text{ (transition)},\\ \left[\Lambda\partial\_1 u^1 + (\Lambda + 2\mu)(\partial\_2 u^2 + a^{-1}u^3)\right]|\_{\partial\alpha\_{\overline{T}\_{\text{max}}}} &= \tau\_{\text{max}} \text{ (trace)},\\ \left[\partial\_2 u^1 + a^2 \partial\_1 u^2\right]|\_{\partial\alpha\_{\overline{f}}} &= 0 \text{ (zero-Robin)},\\ \left[(\Lambda + 2\mu)\partial\_1 u^1 + \Lambda(\partial\_2 u^2 + a^{-1}u^3)\right]|\_{\partial\alpha\_{\overline{f}}} &= 0 \text{ (zero-Robin)}.\end{aligned}$$

A second-order accurate finite-difference method is again employed in conjunction with Newton's method for nonlinear systems. A modestly fine grid is chosen and the iterative process is terminated once |1 − ||**u**m+1||<sup>ℓ</sup> <sup>2</sup> /||**u**m||<sup>ℓ</sup> <sup>2</sup> | falls below a certain value (10−<sup>10</sup> in the calculations shown here). We attempt to model a stiff shell-membrane on a relatively flaccid foundation with a large coefficient of friction. To do so, we keep the values ν<sup>F</sup> = 1, h = 0.001, a = 1, ℓ = 1 4 , L = 1 2 , E = 10<sup>3</sup> , ν = 1 4 , ν¯ = 0, τ<sup>0</sup> = 1 and τmax = 2 fixed for all experiments. Note that some preliminary work in chapter 4 of Jayawardana (2016) found that if (i) the overlying body is relatively thin, (ii) it has a relatively high Young's modulus, (iii) the coefficient of friction is high, and (iv) the mean curvature is a constant, then the solution for the shell model with friction is in relatively good agreement with a numerical model using Kikuchi and Oden's original friction law in chapter 10 of Kikuchi and Oden (1988), thus justifying the choice of our parameters.

.

**Figure 8** shows the total surface displacement and the total surface shear at the outer boundary of the foundation for varying E¯ (the Young's modulus of the foundation). The total displacement is calculated by |**u**||r=<sup>a</sup> = ( P {1x 1 ,1x 2 } uiu i ) 1 <sup>2</sup> |r=a, for i ∈ {1, 2, 3} and the total shear is calculated by | P {1x 1 ,1x 2 } T 3 α (**u**)T α 3 (**u**)| 1 <sup>2</sup> |r=a, for α ∈ {1, 2}. For these simulations, we asserted that a<sup>0</sup> = 1 4 and E¯ = {10, 20, 30, 40, 50, 60, 70, 80, 90, 100}. From **Figure 8** one can see that as the Young's modulus of the foundation increases, the shear stress experienced on the underlying body surface increases, but the total displacement of the body decreases. Only the decrease in displacement, as the Young's modulus of the foundation increases, seems intuitive as when the Young's modulus increases, one would need to increase the amount of force applied to achieve the same amount of displacement. However, the increase in total surface shear is a more interesting result suggesting higher shear forces occur on the underlying surface as the body becomes stiffer despite the coefficient of friction remaining fixed.

**Figure 9** shows the total surface displacement and the total surface shear at the outer boundary of the foundation for varying a0. For these experiments, we asserted that a<sup>0</sup> ∈ { 5 20 , 6 20 , 7 20 , 8 20 , 9 20 , 10 20 , 11 20 , 12 20 , 13 20 , 14 20 , 15 <sup>20</sup> } and <sup>E</sup>¯ <sup>=</sup> 10. From **Figure 9**, one can see that as the thickness of the foundation increases, both the total displacement and the shearing stress on the surface of the body increases. The increase in displacement as the thickness of the foundation increases seems intuitive as there is more deformable matter available to be displaced but this appears again to lead to higher shear stress in the contact region. To explore this further, our final figure (**Figure 10**) shows the total surface stress (shear and normal stresses) at the outer boundary of the foundation. The total surface stress is calculated by | P {1x 1 ,1x 2 } T 3 i (**u**)T i 3 (**u**)| 1 <sup>2</sup> |r=a, where i ∈ {1, 2, 3}. These plots indicate that not only does a thicker foundation experience higher shear stresses but increasingly higher normal stresses with a consequent increase in the likelyhood of potential damage to the underlying body.

#### 4. CONCLUSIONS

Two numerical models have been presented here that can provide insight into the forces generated by a compliant sheet of material under tension in contact with a rigid or deformable object. The aspects of geometry and deformability of the underlying body were examined separately by the mathematical models presented in Sections 2 and 3 respectively.

The model in Section 2 determines the coefficient of friction in the contact region between a thin membrane pulled dynamically at a constant speed and a rigid underlying body. The model of Kikuchi and Oden (1988) for Coulomb's law of static friction was extended to curvilinear coordinates, and a numerical model was used to investigate how the calculated coefficient of friction varies with different material and physical parameters. For parameters such as Poisson's ratio of the membrane, Young's modulus of

the membrane and the speed of the membrane there was no significant variation in the determined coefficient of friction; this indicates that a capstan-type approach lacking dependence on these parameters should still produce accurate results. However, changes to the mass density of the fabric and the lateral (and thus Gaussian) curvature of the underlying body appear to lead to significant variations in the determined coefficient of friction which would not be captured by a capstan model. Even a remarkably small mass density of the fabric (which is assumed to be negligible in the capstan model) leads to the capstan model overestimating the actual coefficient of friction. This effect can be, to a certain extent, attributed to the fabric weight contributing to the overall tension and how this impacts via the logarithmic relation between the coefficient of friction and the tension ratio.

In varying the Gaussian curvature of the underlying body, the numerical model suggests that for a saddle-type geometry (as seen in real experiments such as **Figure 2**) the capstan approach may lead to a significant underestimate of the coefficient of friction. On the other hand, for a barrel-type geometry with positive Gaussian curvature the converse is true with the capstan equation potentially overestimating the coefficient of friction. The numerical model also indicates the intriguing possibility of an optimal barrel-type geometry where the coefficient of friction is minimized—a surprising result that certainly requires further investigation. We note, in passing, that the effect curvature appears to have on the coefficient of friction here possesses some similarities to how Kelvin's equation governs equilibrium vapor pressure over a curved surface (Skinner and Sambles, 1972; Fisher and Israelachvili, 1981) and how grain shape and size modify the rate of complex matter agglomeration (Gadomski and Rubı, 2003).

The model developed in Section 3 is for a thin shell-membrane under tension in frictional contact with an elastic foundation where static friction is imposed in the region of contact. The fact that our frictional law (e.g., Coulomb's law of static friction) must now be imposed on a free boundary because we no longer know a priori the location of the contact region significantly increases the computational complexity. To combat this, a modified, more computationally tractable, displacement-based static friction condition is derived from the model of Kikuchi and Oden (1988) for Coulomb's law of static friction in curvilinear coordinates. We can show that a set of governing equations for a two-body contact problem that incorporates this displacement-based static friction condition yields a unique solution. A numerical scheme for the two-body static friction contact problem is then developed where, this time, the coefficient of friction is specified. Using this model, we examine how the normal and tangential stresses and displacements computed by the model vary as we vary the stiffness and thickness of the underlying body. For a shellmembrane supported by an elastic foundation subjected to static friction we observe the following: (i) that as the Young's modulus of the elastic body increases, the magnitude of the displacement of the body surface decreases; (ii) that as the Young's modulus of the elastic body increases, the magnitude of the tangential shear stresses acting on the surface of body increases; (iii) that as the thickness of the elastic body increases, the magnitude of the surface displacements of the body increases; and (iv) that as the thickness of the elastic body increases, the magnitude of the tangential shear stresses acting on the body surface increases.

The shell-membrane in contact with a deformable elastic foundation model appears to indicate that both elastic and geometrical properties of the elastic foundation significantly affect the stress and deformation of the underlying tissue. This is reflected in the numerical results via a strong positive correlation between the thickness of the foundation and the amount of stress transferred from the shell-membrane to deforming the foundation. Indeed, with the applied tension and coefficient of friction fixed in the numerical model, the amount of stress experienced by the underlying elastic body appears to depend rather significantly on its geometry (thickness and curvature) and elastic properties, which are features that are neglected by capstan-type model approaches.

The models presented in this paper highlight some very interesting results and lead to a number of questions which should be pursued in terms of experimental design and to improve quantification of the frictional forces generated by nonwoven fabrics. One important issue is to examine how to measure the curvature of the contact region and designing experiments that can see how curvature affects the relationship between normal and tangential forces in the contact region. It would also be wise to test if the experimental results are indeed sensitive to variations in the mass density of the fabric. An extended capstan model that incorporates mass density is proposed in Jayawardana (2016) and other modifications to the capstan model could be considered. Experiments involving deformable underlying bodies with wellknown material parameters would be useful to validate our second model and further our understanding about how experiments on quantifying frictional forces involving human subjects should deal with significant skin deformation, such as in the examples shown in **Figure 2**.

Some future work is planned for the models presented, including proving the existence of a unique solution in the model of Section 2 and validating the implementation of the model of Coulomb's law of friction in Kikuchi and Oden (1988) for dynamic friction. For the elastic foundation model (Section 3), further theoretical research on rigorously applying static and dynamic friction at a contact region which is a free boundary, on introducing finite deformations, and on how such a numerical model can be effectively used to provide insight from experimental data are under investigation. Finally, in terms of accurately modeling the response of the skin and underlying tissue, some consideration should be given to exploring beyond Coulomb's law of friction to incorporate the adhesive and repulsive intermolecular forces (such as via hydration) occurring across a variety of spatial scales, leading to Derjaguin-type frictional laws (Gadomski, 2015).

# ETHICS STATEMENT

This paper is fundamentally a mathematical paper but there is one figure (**Figure 2**) containing images of experiments on human subjects from a previous project that is shown as motivation for the mathematical modeling work. The images show deformation in the skin and underlying soft tissue generated during measurements of friction between strips of nonwoven fabric and the volar forearms of female volunteers. The work was conducted with the approval of London Stanmore Research Ethics Committee and The Whittington Hospital NHS R&D office, September 2011. It was carried out in accordance with the recommendations of IRAS (Integrated Research Application System) Help guidance notes for HRA approval (Health Research Authority) with written informed consent from all subjects. All subjects gave written informed consent in accordance with the Declaration of Helsinki. Investigators interacting with elderly and disabled participants underwent DBS checks prior to REC submission and as a requirement for study approval.

# AUTHOR CONTRIBUTIONS

Research Design: KJ, NO, and AC. Mathematical Modeling and Analysis: KJ and NO. Clincial data analysis and guidance: AC. KJ, NO, and AC all contributed to the writing of the manuscript.

# FUNDING

This work was supported by The Dunhill Medical Trust [grant number R204/0511]; UCL Impact Studentship.

# ACKNOWLEDGMENTS

We are most grateful to Dr. Sabrina Falloon and Skevos Karavokiros for providing their insight regarding the experiments on measuring friction as well as for the data and images provided. We are also very grateful to The Dunhill Medical Trust and UCL for funding KJ's Ph.D. studentship.

### REFERENCES


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2017 Jayawardana, Ovenden and Cottenden. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Electrode Position and Current Amplitude Modulate Impulsivity after Subthalamic Stimulation in Parkinsons Disease—A Computational Study

Alekhya Mandali <sup>1</sup> , V. Srinivasa Chakravarthy <sup>1</sup> \*, Roopa Rajan<sup>2</sup> , Sankara Sarma<sup>3</sup> and Asha Kishore<sup>2</sup>

*<sup>1</sup> Department of Biotechnology, Bhupat and Jyoti Mehta School of Biosciences, Indian Institute of Technology Madras, Chennai, India, <sup>2</sup> Department of Neurology, Comprehensive Care Centre for Movement Disorders, Trivandrum, India, <sup>3</sup> Achutha Menon Centre for Health Science Studies, Sree Chitra Tirunal Institute for Medical Sciences and Technology, Trivandrum, India*

#### Edited by:

*Krasimira Tsaneva-Atanasova, University of Exeter, UK*

#### Reviewed by:

*Paolo Melillo, Second University of Naples, Italy Alok Ranjan Nayak, International Institute of Information Technology, India*

> \*Correspondence: *V. Srinivasa Chakravarthy schakra@iitm.ac.in*

#### Specialty section:

*This article was submitted to Computational Physiology and Medicine, a section of the journal Frontiers in Physiology*

Received: *20 August 2016* Accepted: *14 November 2016* Published: *29 November 2016*

#### Citation:

*Mandali A, Chakravarthy VS, Rajan R, Sarma S and Kishore A (2016) Electrode Position and Current Amplitude Modulate Impulsivity after Subthalamic Stimulation in Parkinsons Disease—A Computational Study. Front. Physiol. 7:585. doi: 10.3389/fphys.2016.00585* Background: Subthalamic Nucleus Deep Brain Stimulation (STN-DBS) is highly effective in alleviating motor symptoms of Parkinson's disease (PD) which are not optimally controlled by dopamine replacement therapy. Clinical studies and reports suggest that STN-DBS may result in increased impulsivity and *de novo* impulse control disorders (ICD).

Objective/Hypothesis: We aimed to compare performance on a decision making task, the Iowa Gambling Task (IGT), in healthy conditions (HC), untreated and medically-treated PD conditions with and without STN stimulation. We hypothesized that the position of electrode and stimulation current modulate impulsivity after STN-DBS.

Methods: We built a computational spiking network model of basal ganglia (BG) and compared the model's STN output with STN activity in PD. Reinforcement learning methodology was applied to simulate IGT performance under various conditions of dopaminergic and STN stimulation where IGT total and bin scores were compared among various conditions.

Results: The computational model reproduced neural activity observed in normal and PD conditions. Untreated and medically-treated PD conditions had lower total IGT scores (higher impulsivity) compared to HC (*P* < 0.0001). The electrode position that happens to selectively stimulate the part of the STN corresponding to an advantageous panel on IGT resulted in de-selection of that panel and worsening of performance (*P* < 0.0001). Supratherapeutic stimulation amplitudes also worsened IGT performance (*P* < 0.001).

Conclusion(s): In our computational model, STN stimulation led to impulsive decision making in IGT in PD condition. Electrode position and stimulation current influenced impulsivity which may explain the variable effects of STN-DBS reported in patients.

Keywords: impulsivity, sub thalamic stimulation, Parkinson's disease, Iowa gambling task, reinforcement learning

# INTRODUCTION

Deep brain stimulation (DBS) of the subthalamic nucleus (STN), is a surgical technique now widely applied for the treatment of Parkinson's disease (PD) when dopamine replacement therapy fails to provide sustained relief of motor symptoms or induces drug-induced dyskinesias (Benabid, 2003). Though the exact mechanism of action of DBS is not well-established, it is known that stimulation disrupts (Rosa et al., 2012) the increased synchrony and bursting activity in the β band (8–30 Hz; Kühn et al., 2006) of the STN neurons.

Several reports have highlighted the development of new onset, often transient, impulse control disorders (ICDs) following STN stimulation (Hershey et al., 2004; Smeding et al., 2007; Combs et al., 2015). This was thought to be, due to stimulation of the cognitive sub territory of STN or the spread of stimulation to adjacent parts of the cortico-limbic circuits. In support of this theory, stimulation parameters such as current spread and electrode position were shown to affect the outcome in cognitive tasks in PD patients (Sudhyadhom et al., 2007; York et al., 2009; Witt et al., 2013). Stimulation of the ventral STN decreased the number of correct hits and increased the number of errors on commission in Go-No Go task, when compared to stimulation to dorsal STN (Hershey et al., 2010). STN stimulation can also increase risk taking behavior in Iowa Gambling Task (IGT; Evens et al., 2015). Patients with STN-DBS tended to overestimate their performance with a preference toward competitive environments (Florin et al., 2013). On the other hand, pre-existing ICDs were reported to resolve following STN-DBS, as a result of reduction in dopaminergic medication (Castrioto et al., 2015). Thus STN-DBS may lead to varying net effects on impulsivity in PD through different mechanisms. Clinical studies aimed at dissecting out the effect of STN stimulation on impulsivity have been limited by the confounding effects of therapeutic reduction of dopaminergic medication following STN-DBS.

Computational models provide an opportunity to overcome this limitation by simulating the effect of variations in stimulation and medication protocols individually which cannot be easily applied in human subjects. We hypothesized that electrode position and stimulation parameters affect the decision making ability of PD patients who received STN-DBS. We used a spiking network model of basal ganglia (BG; Mandali et al., 2015) to test the performance under various conditions on the standard gambling task, IGT (Evens et al., 2015). It is known that IGT captures certain impulsive features such as the risk taking ability (Fukui et al., 2005) and lack of premeditation (Zermatten et al., 2005) during decision making. PD patients are known to have poor IGT performance, especially during dopaminergic medication (Poletti et al., 2011; Gescheidt et al., 2012).

Simulating IGT requires learning, which was incorporated in the proposed model using Reinforcement learning (RL; Chakravarthy et al., 2010). RL describes the manner in which an agent learns stimulus-response (S-R) relations based on action outcomes: S-R pairs associated with rewarding outcomes are reinforced while those associated with punishment are attenuated (Dayan and Abbott, 2001). Experimental evidence shows that dopamine (DA) codes for reward prediction error or the temporal difference error term ("δ") in RL (Niv, 2009).

Using the spiking network model of BG (described in Section Materials and Methods), we studied the performance of the model in IGT in normal, PD with and without medication [L-DOPA and Dopamine Agonists (DAA)] and STN-DBS conditions. Our results show that model in normal condition was able to learn from bad choices during the initial trials and improved its performance as the trials progressed. This was observed to be absent in PD with medication (both L-DOPA and DAA) condition.

We then studied the effect of STN stimulation alone on learning and performance by comparing it PD with and without medication. The simulation results show that during the initial trials, the stimulation current interferes with the learning which is reflected as poor performance. We then proceeded to study the factors of stimulation such as electrode position, amplitude of the current and spread that are specific to a patient. The simulation results indicate that electrode position played a significant role in altering performance in the model. We also observed that the model's performance improved for a narrow band of current amplitude.

# MATERIALS AND METHODS

#### Spiking Neuron Model of the Basal Ganglia

The network model of BG (Mandali et al., 2015; **Figure 1A**) was built using 2-variable Izhikevich spiking neurons (Izhikevich, 2003) where each nucleus was modeled as a 2D array (=50 × 50). Parameters for each of the nuclei [STN, Globus Pallidus internus (GPi) and externus (GPe) were chosen such that the model neurons display firing patterns (in terms of firing rate and firing patterns such as rebound firing) of their biological counterparts (Mandali et al., 2015). STN and GPe neurons were bi-directionally connected (Plenz and Kital, 1999) where projections from GPe (STN) are inhibitory (excitatory). Each GPi neuron received both glutamatergic projections from STN and GABAergic projections from D1R-expressing medium spiny neurons (MSNs) of the striatum (Gerfen and Surmeier, 2011). The final action selection was done at thalamus which was simulated as a race model (Vickers, 1970). The activities of both D1- and D2-receptor expressing, striatal MSNs that receive input from cortex (Tritsch and Sabatini, 2012) were modeled as Poisson spike trains. The full set of equations and module sizes related to the model are described in Appendix A and Table A.I (Datasheet in Supplementary Material). The input from cortex to STN, also known as hyper-direct pathway and the GABAergic projection from GPe to GPi, were not included in the model as their functional significance has not been fully explored. The list of acronyms and parameters used in the model are listed in **Table 1** and Tables A.II, IV (Supplementary).

$$\frac{d\nu^{\chi}\_{\vec{ij}}}{dt} = 0.04(\nu^{\chi}\_{\vec{ij}})^2 + 5\nu^{\chi}\_{\vec{ij}} - \mu^{\chi}\_{\vec{ij}} + 140 + I^{\chi}\_{\vec{ij}} + I^{\mathcal{Y}^n}\_{\vec{ij}} \tag{1}$$

$$\frac{d u^{\mathbf{x}}\_{ij}}{dt} = a(b\nu^{\mathbf{x}}\_{ij} - u^{\mathbf{x}}\_{ij}) \tag{2}$$

$$\text{if } \nu\_{ij}^{\times} \succeq \nu\_{peak} \left\{ \begin{array}{l} \nu\_{ij}^{\times} \gets c \\ u\_{ij}^{\times} \gets u\_{ij}^{\times} + d \end{array} \right\} \tag{3}$$

where, v x ij = membrane potential, u x ij = membrane recovery variable, I Syn ij = total synaptic current received, I x ij = external current applied to neuron "x" at location (i, j), vpeak = maximum voltage (+30 mv) with x being STN/GPe/GPi neuron.

#### Behavioral Task-IGT

The task involved presentation of four decks of cards wherein each of the decks A/B/C/D is associated with a combination of reward and penalty. IGT was conducted for a total of 100 trials (5 bins of 20 trials each).The net outcome of a certain card selection (reward + penalty) in each trial was calculated. The probability and amount of penalty varied from deck to deck as explained in Table A.III (Datasheet in Supplementary Material). Over a few trials, one can observe that cards from the decks A and B (C and D) were disadvantageous (advantageous) as the corresponding expected value is negative (positive) The performance was measured as IGT total score (number of selections from "C," "D"—number of selections from "A," "B")

# Simulating IGT Using Spiking Neuron Network Model

Since IGT consists of 4 decks, each nucleus [STN/GPe/GPi/ Striatum (both D1 and D2)] in the network was divided equally into 4 quadrants, where each quadrant received input from one of the decks (**Figure 1B**).The expected value of each card was represented by the cortico-striatal weight which was modulated by DA term "δ." The input to GPe and GPi (i.e., the output of D2 and D1 striatum) was modeled as Poisson spike train (Reti, 2015), whose frequency was proportional to the cortico-striatal weight (w D1 i,k , w D2 i,k ) of the corresponding card (i) and trial (k). The striatal neuronal firing rate was restricted to 2–40 Hz as per experimental literature (Kravitz et al., 2010).Since DA is known to modulate plasticity in cortico-striatal conditions, the error term "δ" (in the model) was used to update the cortico-striatal synapses (Surmeier et al., 2007). DA also modulated the synaptic strengths within various BG nuclei such as STN (Cragg et al., 2004), GPe (Smith and Kieval, 2000).

#### Cortico Striatal Weight Update and Temporal Difference Error

Each deck was associated with 2 cortico-striatal weights (w D1 i,0 , w D2 <sup>i</sup>,0 ) which were initialized with random values from a uniform distribution over (0, 1). The two cortico-striatal weights were

trained as,

$$
\Delta \boldsymbol{\omega}\_{i,k+1}^{D1} = \eta \delta\_k \boldsymbol{\omega}\_{i,k}^{inp} \tag{4}
$$

$$
\Delta \boldsymbol{\omega}\_{i,k+1}^{D2} = -\eta \delta\_k \boldsymbol{\omega}\_{i,k}^{inp} \tag{5}
$$

i,k

The expected value (V<sup>k</sup> ) for kth trial was calculated as,

$$V\_{\mathbf{k}} = \sum\_{i=1}^{4} \boldsymbol{\omega}\_{i,\mathbf{k}}^{D1} \ast \boldsymbol{\omega}\_{i,\mathbf{k}}^{\text{sel}} \tag{6}$$

The reward (Re<sup>k</sup> ) for kth trial was calculated as,

$$\text{Re}\_{\mathbf{k}} = \sum\_{i=1}^{4} r\_{i,\mathbf{k}} \bowtie x\_{i,\mathbf{k}}^{sel} \tag{7}$$

The loss (L<sup>k</sup> ) for the kth trial was calculated as,

$$L\_{\mathbb{k}} = \sum\_{i=1}^{4} l\_{i,\mathbb{k}} \bowtie x\_{i,\mathbb{k}}^{sel} \tag{8}$$

The error (δ) for kth trial was defined as,

$$\delta\_k = \text{Re}\_k + L\_k - V\_k \tag{9}$$

where, w D1 i,k+1 (w D2 i,k+1 ),w D1 i,k (w D2 i,k ) were the cortico-striatal weights of D1 (D2) striatum for ith card in k+1th and kth trial, ri,<sup>k</sup> and li,<sup>k</sup> were the reward and loss obtained for the selected ith card in kth trial, x inp was the input binary vector representing the 4 decks, x sel was the binary vector representing the selected card e.g., if the card "A" is selected x sel = [1 0 0 0].

#### Simulating Untreated PD and Medically Treated PD Conditions

Bearing in mind that "δ" is similar to DA activity (Schultz, 1998; Niv, 2009) and there is loss of DA neurons in PD, we simulated PD condition by clamping the "δ" value (Equation 9) to a low limit (δlim) which resembles the untreated PD condition (Equation 10).

$$\delta\_{\rm lim} = \min(\delta, DA\_{ceil}) \tag{10}$$

Where min (y,a) is defined as <sup>z</sup> <sup>=</sup> y if y <sup>&</sup>lt; <sup>a</sup> a if y > a and DAceil is the upper limit of "δ." Medically treated PD condition clinically involves external intake of dopamine precursors such as L-DOPA which was simulated by adding a positive "δmed" term to the δlim (Mandali and Chakravarthy, 2016) (Equation 10).

$$
\delta\_{new} = \delta\_{\text{lim}} + \delta\_{med} \tag{11}
$$

Another class of medication prescribed to PD patients is DAA, which has differential affinity for dopamine receptors. We simulated DAA with preferential affinity for D2 receptors, also known to be linked to impulsivity (MacMahon and Macphee, 2008). The δnew in the Equation (11) was used to update only D2 cortico-striatal weight (w D2) unlike for L-DOPA where both w D1 and w D2 were updated.

#### DBS Current

An external current which mimics the clinically delivered DBS current was applied to the STN neurons in the model. The parameters (frequency, pulse duration, and amplitude) of the stimulation current were chosen to be similar to the typical values used in a clinical setting (Garcia et al., 2005) [Appendix A (Datasheet in Supplementary Material)]. The spread of current over network of neurons spatially is known to follow a Gaussian distribution (Lukasiewicz and Werblin, 1990). Apart from that, Foutz and McIntyre (2010) have simulated various stimulation waveforms and observed that non-rectangular waveforms are more efficient (Foutz and McIntyre, 2010). The stimulation current was applied to the entire/part of STN module in the form of Gaussian distribution (Foutz and McIntyre, 2010). The mean of the Gaussian coincides with the lattice position (i<sup>c</sup> , jc) which was assumed to be the center of the electrode and extent of current spread was controlled by the variance parameter (σ).

$$I\_{ij}^{DBS} = A\_{DBS} \ast e^{\frac{-((i-i\_c)^2 + (j-j\_c)^2)}{\sigma^2}} \tag{12}$$

where I DBS ij is the current received by the neuron at position (i, j) added to equation 1 of STN neurons, ADBS is the amplitude of the current (pA), σ controls the current spread and (i<sup>c</sup> , jc) is the mean/center point of the electrode. The effect of electrode position (i<sup>c</sup> , jc) and stimulation parameters ADBS and σ on STN activity and on behavior was explored.

Values of the model parameters used for simulating various conditions (PD, medication and stimulation) are given in Appendix A (Datasheet in Supplementary Material).

# Statistical Analysis

The IGT score obtained from each of the conditions [Healthy Controls (HC), untreated PD, medically treated PD, with and without STN stimulation] were compared using repeated measures of ANOVA. ANOVA (which stands for analysis of variance) is a popular statistical technique used to check if the means of two or more populations are equal and come from same distribution. It calculates the variance of means between the groups rather than intra group variance to determine whether the groups come from the same distribution.

Once the statistical test such as ANOVA has been performed, it is important to determine the underlying patterns in the data i.e., groups which could be representatives of different populations and the detects used are generally called as the a-posteriori tests. One such simple yet powerful test is the Bonferroni test which uses the "p-value" to determine the significance of the result. We have used the post-hoc Bonferroni test to study the effect of stimulation, medication on performance. All statistical tests were performed using IBM SPSS Statistics for Windows, Version 21.0, and Armonk, NY: IBM Corp., USA.

# RESULTS

# De-Synchronization by DBS Current

The membrane potential of STN neurons PD untreated condition (**Figures 2B,D**) showed bursting activity and frequency content showed a peak at around 4 Hz with high synchrony level (=0.67; Mandali et al., 2015) which was absent in healthy condition (**Figures 2A,D**). On stimulating the STN neurons in PD condition, the peak around tremor frequency (=4 Hz) was significantly reduced (**Figure 2D**; P < 0.00001). Similarly the bursting activity in **Figure 2B** was overridden and suppressed by the stimulating current (**Figure 2C**). The synchrony level, R sync (Equation A.13) in the presence of DBS current (**Figure 2E**) decreased from 0.67 (in PD condition and stimulation-OFF) to

0.42 (stimulation-ON; **Figure 2F**) but increased at higher current amplitudes.

# IGT-Healthy vs. PD Condition

Total IGT scores for HC, untreated PD, medically treated PD (L-DOPA and DAA) were significantly different [F(3, 36) =9.813, P < 0.0001] [**Figure 3A**, Table A.V (Datasheet in Supplementary Material)]. Post-hoc analysis indicated a significant difference between HC and the other three conditions: untreated PD (P = 0.003), medically treated PD L-DOPA (P = 0.001) and DAA (P < 0.0001). No statistically significant difference was observed between the two medically treated PD (L-DOPA and DAA) conditions.

In HC condition, the score was negative in the 1st bin, but changed to positive at the 2nd bin and continued to increase, a trend that was absent in both untreated and medically treated PD (both L-DOPA and DAA) conditions (**Figure 3B**). The mean scores [Table A.V Datasheet in Supplementary Material)] for 1st bin of HC, untreated PD, medically treated PD L-DOPA and DAA were similar (F = 0.684, P = 0.568). For both 2nd and 3rd bins medically treated PD condition performed worse than healthy controls [bin 2 showed significant difference between HC, medically treated PD L-DOPA and medically treated PD DAA (P < 0.005); bin 3: The scores were significantly different between HC, medically treated PD L-DOPA (P = 0.01) and medically treated PD DAA, P = 0.025)]. Though untreated PD had lower scores compared to HC and higher scores compared to treated PD in both the bins, these differences did not reach statistical significance. The mean scores for 4th bin among the four conditions were not significantly different. For the 5th bin, except HC, all other conditions showed poor performance (F = 8.744, P < 0.0001). The individual bin scores obtained in each of the above described condition are given in **Table 2**.

The mean IGT score values (**Figure 4A**) for HC, obtained from experiment (Gescheidt et al., 2012) and simulation (**Figure 4B**) were not significantly different (P = 0.19). Similarly the mean IGT score values obtained from medically treated PD subjects (Gescheidt et al., 2012) and simulation were not significantly different (P = 0.74).

# IGT-PD Condition with and without Stimulation

Total IGT score was negative for untreated PD, untreated PD with stimulation, medically treated PD and medically treated PD with stimulation (**Figure 5A**) with a significant difference among them [F(3, 36) = 7.24, P = 0.001]. Post-hoc analysis revealed that medically treated PD (P = 0.001) and medically treated PD with stimulation (P = 0.004) had worse performance compared to untreated PD. No significant difference was observed between medically treated PD with and without stimulation.

A significant difference was observed among the 4 conditions (**Figure 5B**) for bin 1[F(3, 36) = 3.24, P = 0.033] [Tables A.VI, A.VII (Datasheet in Supplementary Material)]. Post-hoc analysis indicated a significant difference only between untreated PD and untreated PD with stimulation (P = 0.039). For the 2nd bin [F(3, 36) = 5.58, P = 0.003], medically treated PD performed worse compared to untreated PD (P < 0.008), no significant effect of stimulation was noted. No significant difference was observed for IGT score for the last three bins. The individual bin scores obtained in each of the above described condition are given in **Table 3**.

The mean IGT score values obtained from medically treated PD (PD "ON") subjects from experiment (Oyama et al., 2011) and simulation (**Figures 6A,B**) were statistically similar (P = 0.42). Similarly the mean IGT score values for PD with STN-DBS (DBS "ON") experiment and simulation were similar (P = 0.55).

# Effect of DBS Electrode Parameters on IGT Score

When the electrode position (positions explained in **Figure 7** legend) was changed such that stimulation (for untreated PD) is given selectively to a part of the STN module corresponding to each deck in IGT, we observed a significant variation in the IGT score (**Figure 7B**) (P < 0.0001).On changing the spread of DBS current (**Figure 7C**), there was a trend toward better performance with lower radius of spread (σ = 10), which, however, did not reach statistical significance (P = 0.67). We also observed a lower IGT score at higher (=300 pA) and lower (=70 pA)

currents compared to that obtained from optimal current (=100

varied (D) the effect of DBS current amplitude (70, 100, and 300 pA) on IGT scores when the electrode is placed in position 3.

pA) (P < 0.001) (**Figure 8A**). Interestingly, the current range where the highest IGT score was obtained was nearly the same range where the synchrony in STN neurons was observed to be lowest (**Figure 8B**). The underlying cause for such an effect (**Figures 7D**, **8A,B**), was investigated by observing the spiking activity of STN for both optimal (=100 pA) and high (=300 pA) current scenarios. At optimal current levels, DBS desynchronized the activity of the STN neurons (**Figure 2F**) that received the stimulation, leading to the selection of the stimulated panel (**Figure 2D**; Figure A1; purple line) and a positive IGT score. On the contrary, at non-optimal (=300 pA) current amplitudes, the corresponding STN neuron activity is increased leading to de-selection of the panel "D" (Figure A1).

# DISCUSSION

We built a computational spiking basal ganglia network model to understand the effects of STN stimulation on impulsivity. We first tested the spiking network model, by comparing its simulated activity with the known pathophysiological alterations in PD. It has been observed that desynchronized, irregular STN activity observed in HC changes into synchronized bursting behavior in PD (Wilson and Bevan, 2011), which is also observed in STN neurons of MPTP treated primates (Bergman et al., 1994). This bursting oscillatory activity (Plenz and Kital, 1999) was also observed in our spiking model which correlated with tremor frequency, suggesting that the spiking model has the ability to reflect pathological STN activity.

# Effect of PD and Dopaminergic Therapy on IGT Performance

Overall IGT performance was poor in untreated and medically treated PD conditions compared to HC. Medically treated PD condition did worse than HC and untreated PD, regardless of the type of medication used. Analysis of bin scores revealed that learning of the task was poor in all PD conditions compared to HC. Medically treated PD condition resulted in lower bin scores

from the 2nd bin onwards, suggesting significantly impaired learning of the task, while in untreated PD; a significantly low bin score compared to HC was seen only in the last bin. The model in treated PD condition does not learn from its action outcomes (rewards/punishments) and wanders among the decks, which is reflected in the negative IGT score (**Figure 3**). This behavior is similar to that previously reported in a probabilistic action selection paradigm where PD "ON" subjects fail to avoid punitive choices (Frank et al., 2007). Physiologically this behavior is attributed to excess DA levels in striatum (Frank et al., 2007). In the model, striatal weights were positively updated even in punishment situation due to dopaminergic medication (δmed), leading to the selection of wrong choice. Clinical studies have identified dopamine agonists to be associated with higher risk of impulsivity, mediated through their D3 receptor affinity. We did not specifically model the D3 receptor activity and did not observe a higher risk with DAA (which



selectively increased the D2 weight) compared to L-DOPA in our model.

#### Effect of STN Stimulation

Stimulation applied to the entire STN module did not result in a significant deterioration of overall IGT performance (**Figure 5A**). However, stimulation significantly impaired performance in the 1st bin when applied to untreated PD condition (**Figure 5B**). It is during these early trials that most of the learning regarding deck's reward pattern happens and this learnt information is used for future card selection. Stimulation seems to affect this learning ability, making the model performance worse as trials progress. Similar behavior is also reported in clinical experiments, where it was observed that PD patients with stimulation tend to overestimate their choices (Florin et al., 2013). No significant change was observed between medically treated PD with and without stimulation. This could be due to an overriding effect of dopaminergic medication over stimulation.

A few aspects of DBS that are specific to a PD patient who receives stimulation are the active contact point in the electrode, the amplitude of the current and the spread due to the current. Keeping this mind, we first changed the position of electrode within the STN nucleus (**Figures 7A,B**) and observed


TABLE 2 | The mean IGT scores with standard deviation for HC, untreated PD, medically treated PD (L-DOPA and DAA) at total and individual bin levels and total IGT score.

TABLE 3 | The mean IGT scores with standard deviation for untreated PD, medically treated PD (L-DOPA) with and without stimulation at total and individual bin levels.


a significant change in IGT score. Physiologically, it might be possible that stimulation of different active points (which are 0.5– 1 mm apart in the electrode) could lead to differential activation of neurons. This activation not only leads to overall difference in the current spread (McIntyre et al., 2004) but also in the activity of neurons that receive input from a different sources. This controlled activation might eventually changes the behavior (Witt et al., 2013).

Apart from position, another DBS parameter thought to significantly influence cognition is current amplitude. Various computational and experimental studies showed that the volume of tissue activated is dependent on the stimulating current amplitude (McIntyre et al., 2004; Arle et al., 2007; Yousif et al., 2010). We observed that a high stimulation current can increase the firing rate of STN neurons that received the stimulation whereas sufficiently low amplitude current would just desynchronize the activity. This change in STN, in terms of spiking activity during optimal and non-optimal (Figure A1; for a fixed position) stimulation currents might be the reason behind conflicting results observed at behavioral level (**Figure 7D**). For instance, de-selection of panel "D" increases probability of selection of other panels. In Figure A1, the selected panel was "A" (blue line) is a highly rewarding panel when viewed at a shorter time scale but punitive in long term. The model selects this panel due to its inability to learn the punishments associated with it. This selection of other panels (A and B are disadvantageous, and only C is rewarding) gave rise to a negative IGT score with high current.

With the above results one can consider the possibility that stimulation current when applied to the corresponding topographical areas of the panels within STN might lead to inhibition/facilitation of the corresponding panel selection depending on the current amplitude. To relate the above results physiologically, we suggest a role for the parallel functional loops within BG nuclei (Alexander and Crutcher, 1990) (motor, cognitive, and the limbic loops) and topographical mapping in impulsive behavior. A coarse functional organization is also observed within STN nucleus with motor and cognitive areas being adjacent to each other (Temel et al., 2005). We suspect that the highly variable cognitive outcomes in experimental studies could be correlated to electrode position and current spread (**Figures 7B–D, 8A**). These results are similar to those observed in the clinical study where a decrease in performance (hit rate) was observed when the position of the electrode was changed (Hershey et al., 2010).

Limitations of our model include the connectivity pattern within BG nuclei (GPe-GPi connection and hyper direct pathway is not included) and neuronal number. More elaborate modeling studies are required to further explore the effect of electrode position and stimulus waveform on motor and cognitive aspects of the PD patients. Although the human STN is known to be organized into motor and cognitive sub territories, a further topographical division into separate areas for various choices is yet unconfirmed. Nevertheless, our model reflects the pathophysiology of STN in PD and predicts behavioral changes similar to clinical data. Our results yield valuable information on the effect of electrode position and current amplitude on behavioral and cognitive outcomes of STN stimulation in PD that may help in the development of optimal stimulation protocols in a clinical setting.

### AUTHOR CONTRIBUTIONS

AM: Designed the study, built the computational model and performed the simulations, Manuscript Preparation and Analysis. VSC: Designed the study, built the computational model and performed the simulations, Manuscript Preparation and Analysis. RR: Designed the study, Manuscript Preparation and Analysis. AK: Designed the study, Manuscript Preparation and Analysis. SS: Analysis.

# SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: http://journal.frontiersin.org/article/10.3389/fphys. 2016.00585/full#supplementary-material

#### REFERENCES


outcomes of subthalamic neurostimulation in Parkinson's disease: results from a randomized trial. Brain 136, 2109–2119. doi: 10.1093/brain/ awt151


Ment. Dis. 193, 647–650. doi: 10.1097/01.nmd.0000180777.41 295.65

**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2016 Mandali, Chakravarthy, Rajan, Sarma and Kishore. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Waning Immunity Is Associated with Periodic Large Outbreaks of Mumps: A Mathematical Modeling Study of Scottish Data

Dalila Hamami <sup>1</sup> \*, Ross Cameron<sup>2</sup> , Kevin G. Pollock <sup>2</sup> and Carron Shankland<sup>3</sup>

*<sup>1</sup> Department of Computing Science, University of Oran1 Ahmed BenBella, Oran, Algeria, <sup>2</sup> Health Protection Scotland, Glasgow, UK, <sup>3</sup> Department of Computing Science and Mathematics, University of Stirling, Stirling, UK*

Vaccination programs for childhood diseases, such as measles, mumps and rubella have greatly contributed to decreasing the incidence and impact of those diseases. Nonetheless, despite long vaccination programmes across the world, mumps has not yet been eradicated in those countries: indeed, large outbreaks continue. For example, in Scotland large outbreaks occurred in 2004, 2005, and 2015, despite introducing the MMR (Measles-Mumps-Rubella) vaccine more than 20 years ago. There are indications that this vaccine-preventable disease is re-emerging in highly vaccinated populations. Here we investigate whether the resurgence of mumps is due to waning immunity, and further, could a booster dose be the solution to eradicate mumps or would it just extend the period of waning immunity? Using mathematical modeling we enhance a seasonally-structured disease model with four scenarios: no vaccination, vaccinated individuals protected for life, vaccinated individuals at risk of waning immunity, and introduction of measures to increase immunity (a third dose, or a better vaccine). The model is parameterised from observed clinical data in Scotland 2004–2015 and the literature. The results of the four scenarios are compared with observed clinical data 2004–2016. While the force of infection is relatively sensitive to the duration of immunity and the number of boosters undertaken, we conclude that periodic large outbreaks of mumps will be sustained for all except the second scenario. This suggests that the current protocol of two vaccinations is optimal in the sense that while there are periodic large outbreaks, the severity of cases in vaccinated individuals is less than in unvaccinated individuals, and the size of the outbreaks does not decrease sufficiently with a third booster to make economic sense. This recommendation relies on continuous efforts to maintain high levels of vaccination uptake.

Keywords: mumps, vaccination, waning immunity, mathematical and computational modeling and simulation, Bio-PEPA

# INTRODUCTION

To prevent, control and eradicate childhood diseases, vaccination programs have been adopted throughout the world. For example the trivalent measles-mumps-rubella vaccine (MMR) (Harling et al., 2005; Le Menach et al., 2014; Cordeiro et al., 2015) has been highly successful for both measles and rubella reduction in many countries.

#### Edited by:

*Krasimira Tsaneva-Atanasova, University of Exeter, UK*

#### Reviewed by:

*Maria Vittoria Barbarossa, Heidelberg University, Germany Mauricio Lima, Pontifical Catholic University of Chile, Chile*

> \*Correspondence: *Dalila Hamami dalila.hamami@univ-mosta.dz; dhamami8@gmail.com*

#### Specialty section:

*This article was submitted to Computational Physiology and Medicine, a section of the journal Frontiers in Physiology*

Received: *31 January 2017* Accepted: *03 April 2017* Published: *25 April 2017*

#### Citation:

*Hamami D, Cameron R, Pollock KG and Shankland C (2017) Waning Immunity Is Associated with Periodic Large Outbreaks of Mumps: A Mathematical Modeling Study of Scottish Data. Front. Physiol. 8:233. doi: 10.3389/fphys.2017.00233*

**252**

Despite near eradication of both measles and rubella (Isaacs and Menser, 1990; Glass and Grenfell, 2004; Ueda, 2016), elimination of mumps has not been achieved and could be considered to be re-emerging, despite initial early success in reducing mumps cases. In the last decade, many countries, such as Belgium (Abrams et al., 2014), Korea (Park, 2015), the Netherlands (Snijders et al., 2012), and the US (Dayan et al., 2008) have reported a dramatic increase in the incidence of mumps. In Scotland, 2004/2005 saw a sudden high resurgence in mumps with approximately 4500 cases, 8 years after the second dose of MMR was included in the vaccination program (which was predicted to substantially reduce mumps outbreaks Anderson and May, 1992). One hypothesis is that the resurgence was related to declining vaccine coverage (Nardone et al., 2003; van Boven et al., 2013), in particular, a widespread scare related to autism which led to some parents refusing to vaccinate their children. This can be easily debunked: the herd immunity threshold is estimated at 75–86% (Donaghy et al., 2006) and mumps vaccination levels have stayed above that level (e.g., in Scotland, ranging from 87 to 94% pre-2004). In addition Donaghy et al. (2006) argues that those infected during the 2004/2005 epidemics are characterized by low uptake of a single dose of MMR (catch-up campaign) and being of school age at time when the mumps virus had greatly reduced circulation in that group, delaying infection. The study undertaken by DeStefano et al. (2013) analyzing the number of antigens in both children with and without autism, shows that there is no association between receiving vaccine and developing autism.

A second hypothesis is to link vaccination status and age, e.g., proposing that outbreaks continue in the older population but die out in the increasingly vaccinated population. However, while age structure has shown to be informative in many models of traditionally childhood diseases (Andreasen, 1993; Ferguson et al., 1996; Hethcote, 2000; Brisson et al., 2010), current studies suggest that age is not the key determinant in mumps. Snijders et al. (2012) do not find any significant interaction between these two features. In addition, several studies of different outbreaks occurring at different times and locations in the US and Canada (Centers for Disease Control and Prevention, 2009) indicate that there is no evidence that age is the main factor leading to mumps spread. For instance, the outbreaks occurring in New York (Sulivan, Brooklyn, Rockland county and Orange county), New Jersey and Canada show variable average of infected age groups (Sullivan: 12 years, Brooklyn: 14 years, Rockland county: 12 years, Orange county: 18 years, New Jersey: 19.5 years and Canada: 27.5). However, it was confirmed that all cited cases were related to religious events or camping in Sullivan, with the majority fully vaccinated. It was also reported that the series of outbreaks were due to one fully vaccinated child aged 11 years who had been infected during his travel to UK. Snijders et al. (2012) analyzed a group of infected whose ages ranged in 3–13 years. The authors find out that no significant difference between the attack rate of the group aged 10–13 years and 3–5 years. Considering Scotland specifically, Donaghy et al. (2006) argued that the shift of ages observed in the epidemic in Scotland suggests that the propagation of mumps is becoming more widespread and diverse as the targeted population becomes more dynamic and mobile.

Having rejected the first two hypotheses, the arguments used lead to the third and more plausible hypothesis: MMR vaccine efficacy against mumps reduces over time (van Boven et al., 2013). In 2015 67% of those infected in Scotland were fully vaccinated individuals (1 and 2 doses confounded). Moreover, most primary cases occurred in adolescent and young adults, in contrast to the pre-vaccine era where outbreaks were among children of primary school age. Similar patterns can be found for Belgium in 2012 (Abrams et al., 2014) and in the US in 2006 (Dayan et al., 2008). Serological studies (Heffernan and Keeling, 2009; Park, 2015) show that susceptibility level increases (immunity wanes) as time from vaccination increases; however, the antibody threshold defining the protective level is not well specified for mumps (LeBaron et al., 2009). Even using two doses of the MMR vaccine, existing analyses (Cameron and Smith-Palmer, 2015; Park, 2015) stress that some of the population will remain at risk of disease unless additional control strategies are adopted.

We investigate the hypothesis of waning immunity using mathematically-based computational modeling. The basic model is a seasonal compartmental SEIR model (Anderson et al., 1984, 1987; Keeling and Grenfell, 2002), to which vaccination and immunity is added. We first show that the model produces comparable results to observed mumps data in Scotland<sup>1</sup> , matching endemic levels of mumps with occasional larger epidemics, as in 2005 and 2015. Having established the accuracy of the model with historical data, we use it predictively to better understand the relationship between immunity and transmission, to illuminate long-term patterns of resurgent outbreaks, and to determine whether these can be controlled by extending immunity duration (e.g., by using another booster). While modeling has been previously used to investigate mumps and vaccination (Anderson et al., 1987; Abrams et al., 2014; Edmunds et al., 2000), the novelty of our approach lies in consideration of waning immunity and associated optimal control strategies. Our model shows clearly that waning immunity is a driver for a long period of oscillating outbreaks. Moreover, by working with epidemiologists to use mathematics to understand the observed clinical data, we illustrate the power of mathematics to inform public health policy through multi-disciplinary collaboration.

#### MUMPS EPIDEMIOLOGY IN SCOTLAND

During the period 1988–2015, Health Protection Scotland (HPS), the national surveillance center for Scotland, reported 10943 mumps cases. 10486 of these cases were between 2004 and 2015. Vaccination was introduced in 1988, with a second dose introduced in 1996. **Figure 1** shows the epidemic curve of mumps, and the vaccination uptake curves for both vaccines (MMR1 and MMR2). Observe the initial success of the vaccine (1988–2003) contrasted with a long potential cycle from 2004 to 2015, possible with sub-cycles (2005–2009, 2009–2012, 2012– 2015). The 2004/2005 outbreak was related only partly to the

<sup>1</sup>Department of Health Protection Scotland, National surveillance center.

decrease in vaccination coverage shown in **Figure 1** (Donaghy et al., 2006). The majority of cases (94%) were born before 1990 (aged 15+ years), with only a few of them receiving only one dose of MMR (around 1%) or none at all. Similarly for the outbreaks in 2009 and 2012. In 2015 the highest incidence of mumps (63%) was related to the group born 1991–2000 (aged 15–24 years). Cameron and Smith-Palmer (2015) argue that the 2015 outbreak was the first where the majority of cases were fully vaccinated. Transmission is a complex feature to model as it can be influenced by many factors (vaccination history, current immunity status, age, opportunity for social mixing, geography, and so on). Moreover, some of these factors are confounded (e.g., age and vaccination history). We propose in this model that vaccination history is used as a proxy for these combined effects. Therefore, the main question arising is: why are vaccinated individuals being infected? Here we focus on the long curve (2005–2015) relating to the long inter-epidemic period. We explore these features within the model presented in Methods, using the Bio-PEPA plugin tool (Ciocchetta and Hillston, 2009) and deterministic simulation to provide time series prediction of the number of infected individuals. The model is parameterised and validated on data up to 2015, and then to further validate its predictive performance it is shown to match 2016 data provided by HPS. The advantages for using the Bio-PEPA formalism (a mathematically-defined computational modeling approach called process algebra) have been fully argued in many works (Ciocchetta and Hillston, 2009; Benkirane et al., 2012; Hamami and Atmani, 2013). Here, the advantages are: formal structuring of interactions between components, a compositional approach to building the epidemiological model, and a range of analysis techniques to support the modeler in understanding the system. The underlying semantics of Bio-PEPA is a continuous time Markov chain.

# METHODS

# Model Structure, Epidemiological Assumptions, and Parameter Estimates

We consider a compartmental structure for a model of mumps formulated as an extended SEIR (Anderson and May, 1992) model including seasonality and waning immunity: natively susceptible (S1), vaccinated individuals with MMR1 only (V1), vaccinated individuals with both MMR1 and MMR2 (V2), modified susceptible who are vaccinated individuals who have become susceptible (S2), exposed individuals (E), infected individuals (I) and recovered individuals who are regarded as immune for life (R) (Anderson and May, 1992; Greenhalgh and Sfikas, 2003). **Figure 2** shows how these compartments interact.

Our goal is to provide as simple a model as is necessary to demonstrate the impact of waning immunity, therefore we have ignored features which others have chosen to include. For example, the models of Glass and Grenfell (2003) and Barbarossa and Röst (2015) include immunity levels and immune-boosting through vaccination and interactions with infected. Since we have no data on antibody levels as individuals interact we choose not to include this, choosing the simpler scenario which can be parameterised through observed data. Neither do we include agestructure, as mumps has ceased to be a mainly childhood disease. As shown in several works (Donaghy et al., 2006; Centers for Disease Control and Prevention, 2009; Brockhoff et al., 2010; Fanoy et al., 2011), the range of those infected with mumps has become more diverse due to a more mobile susceptible population. Therefore, rather than stratifying the population by age, we assume a more homogenously-mixed population, with routine vaccination, and transmission based on seasonality and immunity status.

This model is general and could be parameterised for any seasonal disease with up to two vaccinations. We use data from

Health Protection Scotland (HPS) from 2004 to 2016<sup>2</sup> and some parameters from the literature (Anderson and May, 1992; Keeling and Rohani, 2008). These are detailed in Table 1 Appendix 3 in Supplementary, with some explanatory text.

• Demographic estimation

Birth and death rate (µ) estimated from Scottish demographic data<sup>3</sup> .

• Immigration rate estimation (λ)

As the net migration to Scotland is insignificant (typically 15,000 per year), the model has been simplified by having neither mass emigration nor immigration of susceptible individuals. A small constant rate of immigration of infected individuals is required to prevent the disease dying out entirely. This is justified by the knowledge that there is immigration, and there are many populations in the world where mumps is more prevalent and the global population is more mobile, transmitting disease between countries. A small rate of immigration of infectious individuals is estimated as in Finkenstädt et al. (1998) and Benkirane et al. (2012).

• Vaccination rates estimation (µ1, µ2, µ3)

According to vaccination data<sup>4</sup> , our basic assumption is an average of 94% MMR1 vaccination coverage (1988–2016) for children aged 0 to 2 years and 90% MMR2 vaccination coverage (1996–2016) for children aged 3 to 5 years. According to past vaccination history (Morgan-Capner et al., 1988; Public Health England, 2013), we estimate the susceptible portion of the remaining unvaccinated population at 20%. Within that proportion of susceptible we consider 11% of those to be aged 10 years or over according to current demographics. It would be more realistic to consider a varying vaccination rate each year; however, we did not want this to confound the patterns obtained through simply waning immunity. We do investigate scenarios in which these average vaccination rates are varied across the simulation period, to show how this affects the pattern of outbreaks.

• Waning immunity estimation (τ, δ)

Our basic assumption is individuals vaccinated with MMR1 and MMR2 (resp. only MMR1) are temporarily protected and that immunity wanes toward susceptibility at constant rate δ (resp. τ). LeBaron et al. (2009) report low antibody levels 4–9 years after MMR1 only, and 7–12 years after MMR2 administration. We also investigate scenarios in which these rates are varied.

• Transmission rate estimation (β1, β2, β3)

In our model, the transmission rate depends on two features: seasonality (High, Low) and type of susceptible (native susceptible, modified susceptible) giving four rates: β1 (High season and native susceptible), β2 (high season and modified susceptible, β3 (low season and native susceptible), β4 (low season and modified susceptible). For seasonality, data report higher number of cases October to May, and fewer between June and September<sup>5</sup> . As most cases occurs in 17–24 year-olds this seasonality is further supported through an assumption that many of that group are likely to be in full-time education, and mixing more in semester-time than in the holiday. As the total number of infected at low season is small we assume β3 = β4. In addition, we assume β2 > β1 (transmission in modified susceptible is higher than in native susceptible). This follows from the model of Scherer and McLean (2002), and is supported by the report of Cameron<sup>6</sup> that within 205 confirmed cases related to two health boards, 137 (67%) individuals were fully vaccinated. As transmission rate is based on the basic reproduction number R<sup>0</sup> (see Table 1 Appendix 3 in Supplementary), a range of proposed values were collected from literature (Anderson et al., 1987; Anderson and May, 1992; van Boven et al., 2013), where R<sup>0</sup> is ranged [4–11]. See Sensitivity Analysis for sensitivity analysis of the particular choices of these rates.

<sup>2</sup>Department of Health Protection Scotland, National surveillance center.

<sup>3</sup>Department of Health Protection Scotland, National surveillance center.

<sup>4</sup> Information Service Division. Childhood Immunisation Statistics, Available online at: http://www.isdscotland.org/Health-Topics/Child-Health/Immunisation/

<sup>5</sup>Department of Health Protection Scotland, National surveillance center.

<sup>6</sup>R. Cameron. Health Protection Scotland, Immunization Team. Personal communication. (Accessed Jan 21, 2016).

• Incubation rate α and recovery rate γ

Established empirical studies (Anderson et al., 1987; Anderson and May, 1992) estimate the incubation period between 12 and 25 days and the infectious period between 7 and 9 days (Anderson et al., 1987). For modeling convenience, we assume the same period of infection and incubation (Public Health England, 2013) for both natively susceptible and modified susceptible.

• Initial conditions

The initial mix of susceptible, vaccinated, exposed, infected and recovered is calculated for 1996 according to the above assumptions about population based on vaccination beginning in 1988. See Appendix 1 in Supplementary (model component).

The description of the model and parameters above can be summarized by seven ordinary differential equations:

$$\begin{aligned} \frac{dS1}{dt} &= \mu 1N - \frac{\beta \text{ (t)} \,\text{S1 I}}{N} - \mu \text{ }S1\\ \frac{dV1}{dt} &= \mu 2N - \text{\text{\textdegree I}}{N} - \mu \text{ }V1\\ \frac{dV2}{dt} &= \mu 3N - \delta V2 - \mu \text{ }V2\\ \frac{dS2}{dt} &= \delta V2 + \text{\textdegree I}{}\_{1} - \frac{\beta \text{ (t)} \,\text{S2 I}}{N} - \mu \text{ }S2\\ \frac{dE}{dt} &= \frac{\beta \text{ (t)} \,\text{S1 I}}{N} + \frac{\beta \text{ (t)} \,\text{S2 I}}{N} - \alpha E - \mu \text{ }E\\ \frac{dI}{dt} &= \alpha E - \gamma I - \mu I + \lambda\\ \frac{dR}{dt} &= \gamma I - \mu R \\ \text{Where:} \end{aligned}$$

$$\begin{aligned} & \beta\ (t) \left(\text{resp. } \beta'\ (t)\right) \\ &= \begin{cases} \beta 1 \text{ (resp. } \beta 2 \text{ (t))} \text{ if } Time \in \left[\text{Octover} - \text{May}\right] \\\ \beta 3 \text{ if } Time \in \left[\text{func} - \text{Septerm}\right] \end{cases} \end{aligned}$$

This model is coded in Bio-PEPA (see Appendix 1 in Supplementary). Analysis of the model is performed through deterministic simulation. Stochastic simulation was used to guide model development but does not provide additional information when identifying long term trends.

# Model Scenarios

To capture the impact of vaccination efficacy and the effect of waning immunity on the population of Scotland for future projection of epidemics, the history of mumps epidemics (from pre-vaccine to post-vaccine era) are reproduced where four strategies are considered:


# RESULTS

According to observed mumps data in Scotland in **Figure 1**, and in conjunction with observed mumps data in England and Wales in Figures A1, A2 (see Appendix 3 in Supplementary), three different periods of an epidemiological shift in incidence are observed: pre-vaccine, successful post-vaccine and waning immunity period. **Figure 3** depicts time series results for infected cases under scenarios 1–3. Overall, it is clear that mumps occurs every year, regardless of vaccination or waning immunity; however, those factors control the amplitude of the epidemic and the frequency of the highest peaks driving a long term damping oscillation of large outbreaks. After 100 years the difference between the high and low of the cycle is around 25 cases.

#### Scenario One (No Vaccination = Pre-Vaccine Era)

We begin by checking model performance without vaccine. **Figure 3A** shows an inter-epidemic period of 3 years within an oscillatory pattern of mumps cases. This matches parameter values of incubation period of 13 days, infectious period of 7 days and a mean age of infection of 5 years (all within the ranges of Table 1 Appendix 3 in Supplementary). This is supported by the incidence of mumps in England and Wales (Anderson et al., 1987) and observations in the literature reporting cycles of 2–5 years (Galazka et al., 1999).

We point out that predicted cycles do not damp out during 100 years of simulations. By varying seasonality parameter of the model, including removing seasonality altogether, we observed that after a long period the model reaches an endemic state. To further reinforce the suitability of the model we considered R<sup>0</sup> ranging from [7 to 14]. **Figure 4** (see Appendix 3 in Supplementary) shows that increasing R<sup>0</sup> leads to decreasing the inter-epidemic period from 5 to 3 years.

#### Scenario Two (up to two vaccinations and immunity is permanent = immediate post-vaccine era)

Turning to the successful post-vaccine era (and assuming life-long immunity), **Figures 3B,D** show a massive decrease of mumps infections consistent with observed data 1988–2003, where waning immunity was not yet an important factor and the number of cases overall dramatically decreased due to the decreased pool of susceptibles, in turn due to vaccination.

3-Vaccination with waning immunity, (D) Predicted-Observed data for mumps from 1996 to 2016.

Again, this helps to confirm that the model successfully models historical data.

#### Scenario Three (up to two vaccinations and immunity wanes)

**Figure 3C** (resp. **Figure 3D**) shows model prediction against observations from Scotland in the post-vaccine era (2004–2016, resp. 1996–2016). **Figure 3C** shows pattern of mumps outbreaks from 2004 to 2016 as waning immunity begins to be more relevant. The simulated data (black solid line) displayed in **Figure 3C** depicts patterns of mumps dynamics qualitatively similar to observed data (gray solid line). Mumps is notoriously under-reported (Takla et al., 2013) as, especially for those in whom immunity has waned, the disease is often milder (and infected do not seek medical attention). Our model has no notion of "level" of infection, therefore sub-clinical, mild, and serious infections are all counted and contribute to disease transmission. Observed data is scaled by two to compensate for under-reporting of mumps. This is a conservative estimate, based on higher uptake of vaccine in Scotland than in Germany (Takla et al., 2013). This is discussed further in the Discussion.

**Figure 3D** shows that 2005/2015 years were the dominant period reflecting the highest peaks of mumps infection. Some notable gaps are observed (2009, 2010 and 2012); the observed mumps dynamics are inherently stochastic and noisy. **Figures 3C,D** depicts that the simulated data for the year 2016 follows the same patterns as observed data, where the number of infected start to decrease. Qualitatively, the simulation results show that even if vaccination is applied, mumps is occurring each year, where the seasonal patterns of our model depict that the infection increases rapidly over the last few months of the year and the high peak is reached early at the start of the year. This is broadly in agreement with observed data.

Vaccination coverage dips in this period, but this is not the main factor leading to the resurgence and sustainability of mumps, nor is seasonality on its own (as above). We investigate the variability of vaccination coverage by ranging its value from [75 to 95], where 75% is the minimum value related to the threshold level and 95% is the maximum value of applied vaccine coverage in Scotland. **Figure 5** (see Appendix 3 in Supplementary) shows that increasing vaccine coverage leads to a decrease in the peak of infected<sup>7</sup> (from 1694 to 1413). This is 16%, and still produces a large number of cases. Therefore, increasing

the vaccination coverage does not prevent disease occurrence. In addition, we note that all experiments (vaccination coverage ranging from [80 to 95]) settle into a 10 year pattern of gently damping oscillations (100 years of simulation), where the large oscillations are up to 2045, and thereafter the outbreaks become more and more regular in height.

To further investigate the impact of waning immunity **Figure 6** depicts separately those infected-unvaccinated and those infected-vaccinated against natively susceptible and modified susceptible over 100 years of temporal prediction. As expected, due to increasing levels of vaccinated individuals in the population, the number of natively susceptibles and infected-unvaccinated decreases over time, reaching a steady state of infection of around 200 individuals. Conversely, waning immunity leads to an increase in the number of modified susceptible and infected-vaccinated, settling into a 10 year pattern with peaks of between 800 and 1200. Therefore, waning immunity and its effects are the dominant portion of any epidemic.

#### Scenario Four (additional booster-up to three vaccinations and immunity wanes)

Further, we consider scenario 4: the impact of increasing the period of immunity by applying an additional dose of MMR (R and . Cameron , 2016). This could be similarly done by increasing immunity by increasing the efficacy of the vaccination (Public Health England, 2013). We investigate increasing immunity duration in steps from 10 to 80 years (broadly, life expectancy). **Figure 7** compares these scenarios and shows that the average of the number of infected individuals at the peak of each outbreak decreases with increasing duration of immunity, as expected.

# SENSITIVITY ANALYSIS

The results above depend on precise parameter values, therefore we used sensitivity analysis to show that the qualitative results of periodic large outbreaks hold across the range. We identify significant parameters reproducing first the observed data, and second leading to the low level endemic state. Table 2 Appendix 3 in Supplementary shows the impact on epidemic amplitude and the periodicity of damping cycles of a series of experiments during 100 years of simulation varying model parameter values for: transmission rates (β1, β2, β3), infectious period (γ),

<sup>7</sup>Average number of infected corresponds to the average of the highest peaks during 100 years of simulations.

incubation period (α), immunity duration (τ, δ), and vaccination rate (µ1, µ2, µ3). The values of the remaining parameters (birth rate, death rate and immigration rate) are fixed.

For all analysis we used ANOVA as implemented in Minitab (Minitab 17 Statistical Software, 2010). The full details of the analysis are in Appendix 2 in Supplementary: as expected, only varying transmission rates and immunity duration impact on results. Increasing R<sup>0</sup> leads to a decrease in period between large outbreaks and therefore an increase in the number of oscillations (see **Figure 8**, Appendix 3 in Supplementary). Smaller immunity durations increase the pool of susceptibles faster and therefore lead to larger and earlier epidemics.

# DISCUSSION

Our analysis shows that mumps epidemics will continue, with larger outbreaks of ∼1200 every 10 years as shown in **Figure 6**, eventually settling into an endemic state. This is despite high vaccination coverage against mumps (87–95%) since 1988 in Scotland<sup>8</sup> (well above the estimated herd immunity threshold of 75–86% Donaghy et al., 2006).

In this paper, we have presented the results of mathematical modeling using Bio-PEPA, identifying the impact of vaccination and waning immunity in the mumps component of the MMR vaccine. Even though vaccination has been ongoing since 1988, thus largely preventing mumps in children, our results show that waning immunity is the main factor in a repeated pattern of outbreaks. Simulations and analysis undertaken showed that waning immunity over 10 years leads to the highest number of infected and to the longest inter-epidemic period for larger outbreaks.

The first part of this study was to build a seasonal model which reproduces the patterns of the observed data in three scenarios: no vaccination, initial post-vaccine period with immunity for life, and with waning of vaccine-induced immunity as suggested by several sources (Dayan et al., 2008; Snijders et al., 2012; Abrams et al., 2014; Park, 2015). Those show that mumps is present in previously vaccinated individuals with the majority of those affected being university students. While based on Scottish data this is not a peculiarly Scottish phenomenon: for example, in the US (Harling et al., 2005), Korea (Park, 2015) and the Netherlands (Snijders et al., 2012) adolescent individuals were notified as infected despite high vaccine coverage. In these countries, it was observed that the majority of cases were in young adult (18– 25 years) who have been fully vaccinated. In the US, where the first dose of MMR was introduced in 1977 and the second dose in 1990, the outbreak occurring in 2006 reached 6584 cases, 63% of whom received two doses of vaccine. For this country it was reported that in 1982 the incidence rate was reduced to 97% and the 3 year cycles observed in the pre-vaccine era disappeared. Moreover, in 2005, 1 year before the resurgence of the outbreak occurred in 2006, the incidence rate was damped to up to 99% where the vaccine coverage reached 91.5%. In the Netherlands, the large epidemic which occurred in 2004 led to the reintroduction of mumps as a notifiable disease. This followed its removal from the notifiable disease register in 1999 as a consequence of low outbreaks and vaccination coverage of at least one dose of MMR of at least 93% since the introduction of routine vaccine in 1987. In Korea, the epidemic of 2013–2014 showed that 99% of infected individuals aged from 13 to 18 years have been fully vaccinated. It is worth noting that Korea is not that different from other countries as in the pre-vaccine era the epidemic cycles were identified at 4 to 5 years and the mean age of infection at 4 to 6 years which shifted to teenagers in the recent outbreaks (2007 and 2013) in time when vaccination coverage rose to 90%.

Waning immunity is expressed in our model by including an additional compartment of modified susceptible, which is

<sup>8</sup>Department of Health Protection Scotland, National surveillance center.

increased by vaccinated individuals (MMR1 and MMR2) losing their immunity. We find that assuming 5 years of MMR1 vaccineinduced immunity (resp. 10 years of MMR2 vaccine-induced immunity) generates simulation results consistent with more recent mumps post-vaccine data from Scotland (2004–2015). In addition, as our model suggests a 10-year-long gradually damping oscillation, the following trajectory of mumps disease would show a decrease in 2016 and so on, building back up from 2020 to another high peak in the year 2025. The most recent data provided by HPS has confirmed this prediction, where the year 2016 depicts 215 cases compared to 2015 which defines 836 cases. Although our estimates of the amplitude of mumps epidemics are higher than observed data, we conjecture that this can be explained by a low level of reporting. Anecdotally, cases of mumps in vaccinated individuals have much milder symptoms and therefore may be undetected (Public Health England, 2013; Takla et al., 2013; Cordeiro et al., 2015; Gouma et al., 2016).

By considering different values of immunity duration (scenario 4) we can estimate the time needed to reverse the epidemic trend and eliminate mumps. This models the situation that, for example, a new, more effective, vaccine is introduced, or a third vaccine dose is introduced into the national programme. This is shown in **Figure 7**. Even extending immunity to 80 years, a reasonable lifespan, mumps outbreaks still occur. Only by further increasing immunity duration to 150 years eliminates mumps outbreaks, assuming no perturbations occur, such as a new vaccine or new strain of mumps.

It is worth noting that the basic reproductive number R<sup>0</sup> for the pre-vaccine era is estimated at 10.5 which falls in the range [7–14] as cited in literature (Anderson and May, 1992; Keeling and Rohani, 2008) and for the post-vaccine era R<sup>0</sup> is estimated at 6 where in the literature it is quoted at [4–7] (Anderson and May, 1982). Recall that R<sup>0</sup> indicates the number of secondary infections, clearly showing that the number of doses of vaccination and immunity duration has a great impact on decreasing infectious contacts.

Cumulatively, our findings suggest that the more "unprotected" individuals (who were either never vaccinated or lost their immunity), the shorter the period between two high peaks of epidemic outbreak (note the number of cycles in Table 2 for varying values of R0). In addition,

in both cases related to scenarios 1 and 3 (No vaccination and waning immunity), an earlier high peak of mumps is expected. This occurs because the pool of susceptibles is increasing faster as those vaccinated lose their immunity and move to the susceptible state (scenario 3), or the pool of susceptibles is decreasing faster when no vaccination is applied and R<sup>0</sup> is higher (scenario 1). Clearly, controlling the number of susceptible individuals has a great impact on controlling disease. As argued by Gay (1998): to achieve elimination of an epidemic, low levels of susceptible individuals should be maintained, leading the basic reproductive number (R0) to be <1. We do this here by adjusting immunity duration.

These conclusions illustrate an enhanced understanding of mumps disease in response to mass immunization gained through mathematical modeling. Further, our multidisciplinary team could explore the potential impact of further vaccination on cyclic outbreaks. Our conclusion for public health services is that they should urge vaccine uptake in those eligible since a high degree of protection is offered by the vaccine overall for those under 18. Considering the possible economic cost/benefit of a third vaccine dose, it seems that while there would be an increased period of immunity, the cyclic outbreaks would continue at about 2/3 the current level, therefore this would not offer significant advantages over the present situation. The Joint Committee on Vaccination and Immunization<sup>9</sup> do not consider these large outbreaks of particular concern, since there has been no formal discussion to introduce a 3rd vaccine dose into the national programme.

We suggest further study with this model could include vaccination programmes targeted to those subject to waning immunity or at higher risk due to social mixing in a diverse population (as in higher education). Such a model might also include economic factors to allow the effect of targeted programmes to be more precisely evaluated. Another interesting facet would be to bring more attention to the level of immunity by analyzing the vaccine/virus content and detect eventual

<sup>9</sup>UK body advising government health policy on vaccination and immunisation.

discrepancy between vaccine strain and mumps outbreak. This might also be linked with a data science approach to analyzing serology of confirmed cases. There are further opportunities to use data science to analyse other features, such as geographic distribution. These developments would allow an enhanced version of **Figure 6** showing waves of outbreaks related to waning immunity, evolution of strains of mumps, and locality.

### AUTHOR CONTRIBUTIONS

The Conception or design of the work: CS, DH, and KP. Data collection: RC, KP. Data analysis and interpretation: CS, DH, and KP. Drafting the article: DH, CS. Critical revision of the article: CS, KP, DH, and RC. Final approval of the version to be published: CS, KP, DH, and RC.

#### REFERENCES


### FUNDING

DH is grateful to the Algerian government, the University of Mostaganem and to the University of Oran for supporting her series of research visits to the University of Stirling.

### ACKNOWLEDGMENTS

We thank the reviewers for their many helpful comments to improve this paper.

# SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: http://journal.frontiersin.org/article/10.3389/fphys. 2017.00233/full#supplementary-material


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2017 Hamami, Cameron, Pollock and Shankland. This is an openaccess article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Patient-Specific, Multi-Scale Modeling of Neointimal Hyperplasia in Vein Grafts

Francesca Donadoni <sup>1</sup> \*, Cesar Pichardo-Almarza<sup>1</sup> , Matthew Bartlett <sup>1</sup> , Alan Dardik 2, 3 , Shervanthi Homer-Vanniasinkam1, 4, 5 and Vanessa Díaz-Zuccarini <sup>1</sup>

<sup>1</sup> Mechanical Engineering, University College London, London, UK, <sup>2</sup> The Department of Surgery, Yale University School of Medicine, New Haven, CT, USA, <sup>3</sup> Veteran Affairs Connecticut Healthcare System, West Haven, CT, USA, <sup>4</sup> Leeds Vascular Institute, Leeds General Infirmary, Leeds, UK, <sup>5</sup> Division of Surgery, University of Warwick, Warwick, UK

Neointimal hyperplasia is amongst the major causes of failure of bypass grafts. The disease progression varies from patient to patient due to a range of different factors. In this paper, a mathematical model will be used to understand neointimal hyperplasia in individual patients, combining information from biological experiments and patient-specific data to analyze some aspects of the disease, particularly with regard to mechanical stimuli due to shear stresses on the vessel wall. By combining a biochemical model of cell growth and a patient-specific computational fluid dynamics analysis of blood flow in the lumen, remodeling of the blood vessel is studied by means of a novel computational framework. The framework was used to analyze two vein graft bypasses from one patient: a femoro-popliteal and a femoro-distal bypass. The remodeling of the vessel wall and analysis of the flow for each case was then compared to clinical data and discussed as a potential tool for a better understanding of the disease. Simulation results from this first computational approach showed an overall agreement on the locations of hyperplasia in these patients and demonstrated the potential of using new integrative modeling tools to understand disease progression.

#### Edited by:

Zbigniew R. Struzik, University of Tokyo, Japan

#### Reviewed by:

Christian Vergara, Politecnico di Milano, Italy Giuseppe D'Avenio, Istituto Superiore di Sanità, Italy

#### \*Correspondence:

Francesca Donadoni francesca.donadoni.14@ucl.ac.uk

#### Specialty section:

This article was submitted to Computational Physiology and Medicine, a section of the journal Frontiers in Physiology

Received: 09 December 2016 Accepted: 30 March 2017 Published: 18 April 2017

#### Citation:

Donadoni F, Pichardo-Almarza C, Bartlett M, Dardik A, Homer-Vanniasinkam S and Díaz-Zuccarini V (2017) Patient-Specific, Multi-Scale Modeling of Neointimal Hyperplasia in Vein Grafts. Front. Physiol. 8:226. doi: 10.3389/fphys.2017.00226 Keywords: neointimal hyperplasia, vein grafts, remodeling, shear stress, computational fluid dynamics, multi-scale modeling

# 1. INTRODUCTION

Peripheral bypasses are amongst the most common vascular interventions; however, the reality is that millions of these bypasses fail due to vascular remodeling and this is a real burden for National Health Systems. In the UK alone, costs for surgery exceed 200 M (Peach et al., 2012). Why bypasses fail is a critical issue in vascular surgery today, however, traditional approaches have not provided answers to this problem. It is essential to mention that as of today, animal experiments to study peripheral grafts have failed dramatically. To put it simply, there are no animal models which would provide useful data to understand lower extremity venous bypass failure in humans and other, novel approaches are urgently required. The ideal vascular conduit remains the "Holy Grail" of vascular surgery (Byrom et al., 2010). Variability and uncertainty of the outcome are serious issues and the consequences of a failed graft for every patient, just dire. When a bypass graft blocks, blood supply is usually worse than before bypass surgery. In these circumstances amputation can be inevitable unless the graft can be salvaged and the blood supply restored. Recent randomized controlled trials showed that 40% of lower extremity vein grafts occlude or develop significant stenosis within the

**263**

first year after implantation (Owens et al., 2015). Results for more complex procedures to the calf vessels have usually slightly worse prognosis, with resultant serious morbidity and mortality. These figures have largely remained unchanged for the past several decades. One can read (Owens et al., 2015), "...it is discouraging to consider that 5 decades of high-powered science has not effectively changed bypass graft outcomes." Improving results of long-term limb salvage remain critically dependent on understanding the mechanisms of successful vein graft adaptation to the arterial environment. In a clinical landscape with ever-increasing and more aggressive bypass procedures, the use of novel mathematical modeling and simulation tools to understand venous adaptation to the arterial environment would help preventing the significant numbers of excess complications, mortality and cost of re-interventions and alternative therapies. This is key in order to devise a personalized "best management plan" and to deliver the best treatment for a specific patient, at that crucial time.

This paper is focused on the development of a simulation framework to elaborate the mathematical tools needed to understand and model key properties of a multi-scale, clinical problem, i.e., bypass failure, at different levels. The vision is that we can use multi-scale mechanistic models to understand and disentangle the complexity of lower limb graft failure and to start making some headway toward patient stratification in this regard.

Neointimal hyperplasia (NIH) is one of the processes leading to restenosis (and ultimately graft failure). NIH is, in simple terms, the re-narrowing of a blood vessel after a stenting or vein grafting surgical procedure, due to tissue growing at the site of injury (Murphy and Boyle, 2010). Upon the start of the formation of NIH, smooth muscle cells change their phenotype from contractile to synthetic, making them more proliferative and resulting in thickening of the arterial tissue. The new conditions in the vascular tissue trigger a reaction which also leads to the release of growth factors and cytokines, including transforming growth factor beta (TGF-β) (Guerri-Guttenberg et al., 2013), platelet derived growth factor (PDGF) (Huang et al., 2002), fibroblast growth factor (FGF-2) (Nabel et al., 1993) and a group of inflammatory cytokines (Collins et al., 2012). Furthermore, other hypotheses have been studied such as that the fibroblasts in the adventitia might move to the media in the form of smooth muscle cells or that bone marrow-derived progenitor cells coming from the bloodstream might also turn into smooth muscle cells to form part of the vascular tissue (Collins et al., 2012). In addition to smooth muscle cells, other types of cells, such as monocytes, are also involved in intimal volume growth (Stark et al., 1997), in response to the inflammatory process triggered by the surgical cut.

However, studies have shown that biological mechanisms alone are not the sole process leading to changes in the morphology and geometry of the artery. A number of mechanical forces in the arterial wall, and shear stress also play a role (Owens, 2010). As opposed to artificial stents, in the case of vein grafts it is the whole conduit, and not just the arterial tissue around it, to be made of living tissue and thus to respond to the surrounding environment. As a result, it is particularly important to consider both the mechanical and the biological mechanisms behind remodeling. Given the complex interplay between biological mechanisms and mechanical stimuli, mathematical modeling can offer much needed help.

Previous research has shown the potential of computational modeling to describe the connection between hemodynamic factors and NIH. One of the first studies to model the relation between blood flow dynamics and NIH appeared in 2001 (Hill and Spendiff, 2001). In this model, the relationship between flow and tissue growth was described by modeling the permeability of the endothelium as a function of wall shear stress (WSS). Tran-Son-Tay et al. (2007) and Tran-Son-Tay et al. (2008) investigated the problem by setting up an experiment using a rabbit model. Subsequently, they developed a mathematical model and compared it to the results obtained from the animal model, using a differential equation where the rate of change of the intima thickness is proportional to the shear stress. The model by Budu-Grajdeanu et al. (2008) includes cellular and chemical mechanisms and takes the change in lumen radius into account. Other studies (Dexter et al., 2009; Boyle et al., 2010; Hwang et al., 2011, 2013; Garbey and Berceli, 2013) developed agentbased models of the disease. Wu and Cassel (2013) modeled NIH as a diffusion process of smooth muscle cells using a feedbackcontrol system, and simulated a reduction in NIH. Finally, in the model by Fok (2012), intimal thickening was modeled as a free boundary problem, which was an accurate description of cell and chemical dynamics, but did not include the flow characteristics which are an essential aspect in this study. A key feature of the work shown in this paper is that the simulations presented here are compared against patient-specific clinical data, which to the authors' knowledge, has not been done before in the context of NIH and bypass failure. Deidentified patientspecific data for this study was obtained with approval of the institutional human investigation committee (approval AD0009, Veterans Affairs Connecticut Healthcare System, West Haven, CT, USA) as part of an ongoing collaboration between UCL and Yale University.

This paper is organized as follows: the second section describes the methods used to develop the computational framework, including specific details about the different biological scales considered in the model, the mechanisms through which mechanical stimuli combine with biological changes, a description of how blood flow is modeled and how the two aspects of the model are combined in a framework. The third section presents specific simulation results for two vein grafts of a human patient and validation with clinical data. Finally, the fourth and fifth sections show a discussion of these initial simulation results and the conclusions, respectively.

This study is part of a broader research activity aimed at developing multi-scale, patient-specific models of cardiovascular disease (Alimohammadi et al., 2015; Di Tomaso et al., 2015), and in particular, vascular remodeling problems. A similar approach has been successfully implemented for the case of atherosclerotic disease (Díaz-Zuccarini et al., 2014). The patient-specific, multiscale modeling framework proposed here uses a feedback approach to connect biological and mechanical mechanisms in vascular remodeling and details are provided below. In addition, this study seeks to establish the groundwork toward translational studies to understand the relationship between different multiscale mechanisms and factors in patient-specific studies of vein graft failure, and to develop predictive tools for NIH-prone areas in vein grafts.

# 2. METHODS

This paper brings together mathematical techniques and biological data to produce a model of NIH that describes the link between hemodynamic forces and significant biological mechanisms, according to the literature to date. The mathematical framework presented here relies on patientspecific imaging and hemodynamic measurements, which enables the possibility to study in detail the characteristics of the blood flow in patient-specific geometries. These data, together with literature findings (Humphrey, 2002; Budu-Grajdeanu et al., 2008; Duru et al., 2015), enable to identify areas most prone to NIH. In a nutshell, the process is as follows: patient-specific blood flow simulations on the vein graft prior to the formation of NIH (right after the procedure) are performed by means of a computational fluid dynamics software (ANSYS CFX). Subsequently, time averaged wall shear stress (TAWSS) results from the simulations are used as an input to a biochemical model of cell growth. This is mainly based on the association between WSS and the production of nitric oxide which has been established in previous research (Andrews et al., 2010), in addition to various other biological mechanisms, which will be explained in this section. A remodeling process occurs and a new geometry is then obtained. Finally, the steps are repeated until the time for comparison with clinical data is reached. Details are provided in the following sections.

### 2.1. Biochemical Model

NIH occurs about 6–24 months after vascular intervention in the form of thickening of the tunica intima. This is one of the layers that constitutes the vascular tissue together with the tunica media and adventitia. While the adventitia is mainly composed of fibroblasts, the media and intima have very similar composition as they mostly contain smooth muscle cells. Normally, the tunica media is thicker than the tunica intima. However, after injury, smooth muscle cells, which are not normally characterized by high rates of proliferation, turn into a more synthetic type, with higher cell turnover and migration from the media to the intima, causing the vessel lumen to narrow (**Figure 1**). This causes the intimal volume to increase and often further surgical treatment is needed to avoid blockage (Collins et al., 2012).

This model considers smooth muscle cells proliferation in the intima to be the most critical response from vascular tissue after injury to the endothelium (Boyle et al., 2010; Model and Dardik, 2012). After injury, the start of the inflammatory process causes them to migrate from the media to the intima, accumulating there and causing the intima to thicken (Kohler, 2005; Model and Dardik, 2012).

The model allows for the calculation of intimal volume growth and lumen occlusion. The first process to affect smooth muscle cells in NIH is a change from the contractile phenotype, in which the production is very low, to a synthetic one with a much higher turnover (Collins et al., 2012). In the model proposed in this paper, the rate of production of quiescent cells (Q) is described by means of a logistic growth equation. The subscript i refers to the tunica intima, while m refers to the tunica media. The only inhibiting factor to the proliferation of the cells is the volume available to them, so the maximum numbers of cells Qimax and Qmmax are calculated based on the maximum volume available in the intima and media, respectively. This approach to cell modeling has previously been used in mathematical models of tumor growth (Marusic et al., 1994; Kozusko and Bourdeau, 2007). Quiescent cells in the intima and media were modeled as

$$\frac{dQ\_i}{dt} = \beta \times Q\_i \times (1 - \frac{Q\_i}{Q\_{i\_{\text{max}}}}) \text{ in } \Omega\_i \tag{1}$$

$$\frac{dQ\_m}{dt} = \beta \times Q\_m \times (1 - \frac{Q\_m}{Q\_{m\_{\text{max}}}}) \text{ in } \Omega\_m,\tag{2}$$

with β = coefficient of quiescent cells turnover [day−<sup>1</sup> ], Q = quiescent cells [cells], and Qmax = maximum number of quiescent cells [carrying capacity, cells]. is function's domain, with <sup>i</sup> = tunica intima domain, and <sup>m</sup> = tunica media domain. Smooth muscle cells production is dependent upon phenotype change from quiescent cells, migration from the intima, presence of growth factors and production and apoptosis of cells that is in turn dependent on nitric oxide (NO). In the tunica media, they are assumed to be deriving from change in phenotype and cell production, and to degrade according to the amount migrating to the media and undergoing apoptosis. The coefficient of quiescent cells turnover β was estimated based on the assumption that the turnover rates in the intima and media are similar at a quiescent stage (Davies and Hagen, 1994).

$$\frac{d\mathbb{S}\_i}{dt} = \mathbb{y} \times \mathbb{Q}i + \left( (p\_i - a\_i) \times \mathbb{S}\_i + m \times \mathbb{S}\_m \right)$$

$$+ \phi \times \left( G\_P + G\_F \right) \text{in } \Omega\_i \tag{3}$$

$$\frac{d\mathbb{S}\_m}{dt} = \chi \times Q\_m + (p\_m - a\_m - m)\mathbb{S}\_m \text{ in } \Omega\_m,\tag{4}$$

with γ = coefficient of differentiation from quiescent cells [day−<sup>1</sup> ], p<sup>i</sup> , p<sup>m</sup> = coefficient of smooth muscle cell proliferation [day−<sup>1</sup> ], a<sup>i</sup> , a<sup>m</sup> = coefficient of smooth muscle cells apoptosis [day−<sup>1</sup> ], m = coefficient of smooth muscle cells migration [day−<sup>1</sup> ], φ = coefficient of production due to growth factors [cells/(ng×day)], S = smooth muscle cells [cells]. In the tunica media, p<sup>m</sup> and a<sup>m</sup> have been kept as proliferation and apoptosis coefficients as these are the values for standard conditions (high values of TAWSS) and the media was not considered to be affected by the TAWSS.

The collagen turnover is based on the approach proposed by Cilla et al. (2013) adjusting the parameter values according to the units used in the present model.

$$\frac{dC\_i}{dt} = \mathbf{S}\_i \times \boldsymbol{\lambda} - \mathbf{C}\_i \times \boldsymbol{\chi} \text{ in } \Omega\_i \tag{5}$$

$$\frac{dC\_m}{dt} = S\_m \times \lambda - C\_m \times \chi \text{ in } \Omega\_m,\tag{6}$$

with λ = coefficient of production from smooth muscle cells [g/(day×cell)], χ = coefficient of degradation [day−<sup>1</sup> ] and C = collagen [g]. The response of smooth muscle cells to PDGF and fibroblast growth factor (FGF-2) based on WSS is modeled according to experimental data. This means that, although the general form of the equation is similar to that used in Budu-Grajdeanu et al. (2008), the equation's coefficients have been estimated from literature, to reflect the behavior of specific PDGF and FGF-2 growth factors.

$$\frac{d(G\_P)}{dt} = \zeta\_\S - \zeta\_d \times G\_P \text{ in } \Omega\_i \tag{7}$$

$$\frac{d(G\_F)}{dt} = \theta\_\S - \theta\_d \times G\_F \text{ in } \Omega\_m,\tag{8}$$

with ζ<sup>g</sup> = PDGF production coefficient (ng/day), ζ<sup>d</sup> = PDGF degradation coefficient (day−<sup>1</sup> ), θ<sup>g</sup> = FGF-2 production coefficient (ng/day) and θ<sup>d</sup> = FGF-2 degradation coefficient (day−<sup>1</sup> ). **Table 1** reports the equations used to compute the parameters which vary with WSS. Finally, new volumes are calculated to find the growth of the tissue,

$$V\_i = (\mathbf{S}\_i + \mathbf{Q}\_i) \times \rho\_s^{-1} + \mathbf{C}\_i \times \rho\_c^{-1} \tag{9}$$

$$V\_m = (\mathbf{S}\_m + \mathbf{Q}\_m) \times \boldsymbol{\rho}\_s^{-1} + \mathbf{C}\_m \times \boldsymbol{\rho}\_c^{-1},\tag{10}$$

with ρ<sup>s</sup> = cell density [cells/m<sup>3</sup> ], ρ<sup>c</sup> = collagen density [g/m<sup>3</sup> ] and V = volume [m<sup>3</sup> ]. The model considers that cells have a uniform spatial behavior and the volume of vascular tissue is considered to be homogeneous. The tissue is divided in two layers, the tunica intima and tunica media, whose behavior is described by means of eight main equations. The volume of the tunica media remains constant as a result of the balance between apoptosis and proliferation of smooth muscle cells being steady, as this layer is not assumed to be affected by WSS. It has nevertheless been modeled to show the migration of smooth muscle cells from the media to the intima, and the turnover of cells in the tissue. Growth factors in the media have been disregarded. The biochemical species accounted for in the model are smooth muscle cells and collagen since they are the major constituents of the layers of vascular tissue considered (Humphrey, 2002). **Tables 1**, **2** summarize the model parameters and their respective values.

Additionally the model describes the regulation of smooth muscle cells apoptosis, migration and proliferation through a direct relationship between WSS and shear stress-dependent biomolecules that regulate the vasculature. Numerous studies have shown that mechanical factors (e.g., WSS) influence cell response and thus the formation of NIH (Paszkowiak and Dardik, 2003; Caro et al., 2013; Tarbell et al., 2014; Jia et al., 2015). According to Humphrey (2002), NO, PDGF and FGF-2 play an important role in controlling smooth muscle cells turnover. While the relationship between growth factors and WSS has been modeled following the expression


used in the study by Budu-Grajdeanu et al. (2008), multiple results from literature were considered when analyzing the relationship between WSS and NO (as described in the next section).

#### 2.2. Relationship between WSS and NO

One of the ways cell production is influenced by WSS is through the relation between WSS and NO. When present near the vascular wall, a higher WSS causes a higher production of NO (Plata et al., 2010), which inhibits NIH (Ahanchi et al., 2007; Pearce et al., 2008).

Multiple models of NO production in response to WSS have been proposed using different approaches, for example, Chen et al. (2011) used a linear model, Fadel et al. (2009) used a hyperbolic model and Plata et al. (2010) a sigmolidal model, all of them derived from experimental data. The linear model was selected as it was shown in previous research (Andrews et al., 2010) to give results close to experimental values for shear stresses lower than 0.5 Pa, the threshold under which NIH is more likely to develop (Meirson et al., 2015). Nitric oxide production rate (RNO) was expressed as:

$$R\_{NO} = 1.74 + 7.52 \times WSS,\tag{11}$$

with WSS = wall shear stress.

The rate of production of NO has been shown to relate to the balance between proliferation, apoptosis and migration of cells. Studies have shown that low WSS, leading to a reduction of NOs, mRNA and protein expression, hinders the apoptosis signaling pathway, and induces platelet-derived growth factor (PDGF) and matrix metalloproteinase-2 (MMP-2) signaling pathway which leads to higher smooth muscle cells proliferation and migration (Qiu et al., 2013).

Experimental findings have shown NO production to be associated with the inhibition of cell growth in a linear (Marks

#### TABLE 2 | List of constant parameters.


et al., 1995; Nishio et al., 1996) or hyperbolic (Krick et al., 2002) way. In the particular case of the model presented here, linear relationships between NO production and the apoptosis (a), proliferation (p) and migration (m) coefficients are proposed:

$$a\_i = a\_m - a\_m \times \frac{R\_{NO}}{R\_{NO\_{MAX}}},\tag{12}$$

$$p\_i = p\_m - p\_m \times \frac{R\_{NO}}{R\_{NO\_{MAX}}} \tag{13}$$

$$m = m\_0 - m\_0 \times \frac{R\_{NO}}{R\_{NO\_{MAX}}} \tag{14}$$

also reported in **Table 1**.

# 2.3. Patient Data Characteristics

Deidentified computed tomography (CT) scans were obtained from a patient that underwent bilateral peripheral vein graft bypasses procedures (approval from the institutional human investigation committee: AD0009, Veterans Affairs Connecticut Healthcare System, West Haven, CT, USA); in the right leg a femoro-popliteal bypass was performed from the mid-thigh to the mid-calf, and in the left leg a femoro-distal bypass was performed from the groin to the mid-calf. A diagram of the procedure leading to the implantation of vascular grafts is shown in **Figure 2**. Doppler ultrasound and noninvasive pressure measurements were also taken at different locations along both legs. All studies were performed as part of the standard clinical care for the patient after surgery. Consent for the studies was obtained.

### 2.4. Image Processing

Images of the patient-specific arterial geometries were extracted from the CT scans using ScanIP (Simpleware Ld., Exeter, UK). The post-operation images were processed using ScanIp to obtain two different 3D geometries for each of the bypasses (right leg and left leg): one immediately after surgery and another a few months after surgery (4 months for the right leg and 8 months for the left leg bypass). While the geometries a few months after surgery were part of the clinical dataset, the geometries immediately after surgery were not available within the dataset, and were obtained by locating the areas where NIH had formed as exemplified in Budoff and Shinbane (2010), and virtually removing them. The right side graft had an initial radius of 2.72 mm at the inlet, while the outlet radius measured 2.67 mm. The length of the selected volume for the analysis (from inlet to outlet) was 30 cm. The left side graft had a more complex geometry due to the

presence of multiple outlets, the greater length of the graft itself, and the difference in dimensions between different parts of the geometry. The radii of inlet and main outlet were 6.29 and 0.96 mm, respectively, while the outlets of the peripheral vessels measured 2.71 mm (deep anterior femoral), 1.63 mm (deep posterior femoral), 1 mm (posterior distal—proximal outlet). The length of the volume considered for the left side graft was 90 cm.

# 2.5. Computational Fluid Dynamics Model

A commercial software, ANSYS CFX 17 (Ansys Inc, PA, USA) which is based on Finite Volume Methods was used for CFD simulations. The Navier Stokes equations were spatially discretized using a high resolution upwind scheme. A second order implicit backward formula (called second order backward Euler scheme by the software) was used for the temporal discretization (Ferziger and Peric, 2013); the time-step size was 0.0025 s. Blood was assumed to be an incompressible, homogeneous and Newtonian fluid. Blood characteristics were a density of 1,050 kg/m<sup>3</sup> and viscosity of 0.0035 Pa·s. For each of the simulations in the remodeling cycle, an unstructured mesh of 400,000 elements was created, each with 7 prismatic layers. A grid sensitivity analysis was carried out for the first simulation on a transient case to compare values of TAWSS. A 7% difference in TAWSS was found between the 400,000 elements and 1M elements meshes. The difference in velocities at the outlet was less than 2%. This was considered enough for mesh convergence. The same approach to meshing was used for the left leg bypass geometry, which led to finding an optimum number of elements at 2.9 million elements. It is worth noticing that the mesh for the left leg graft had an increased number of elements, partly due to the fact that the simulated section is significantly extended since femoro-distal bypasses run along the whole leg (**Figure 2**), but also to the fact that the very small collateral vessels were also meshed and this required an appropriate element size.

# 2.6. Boundary Conditions

At the inlet, a pulsatile parabolic velocity boundary condition was applied. This was obtained from the Doppler ultrasound data available as part of the clinical dataset. The curves were smoothed in Matlab (The MathWorks Inc., Natick, MA, USA) to avoid nonphysiological oscillations and used to compute mass flow rate curves. The standard Matlab smooth function based on a moving average filter was used to smooth the data. An example of the level of smoothing applied to the images can be seen in **Figure 3**. The ultrasound measurements obtained from the clinical dataset presented quite a significant amount of noise which had to be removed, as shown in the figure. A velocity profile at the inlet of the flow domain was then obtained from the estimated mass flow rate at the location of the ultrasound measurement and used for creating a parabolic profile according to the equation (Munson et al., 2002):

$$u(r) = V\_{\max} \times (1 - \frac{r^2}{R\_{\max}^2}),\tag{15}$$

where u(r) = velocity profile along the radius, Vmax = maximum velocity curve from Doppler data, r = position along the lumen radius, Rmax = lumen radius.

#### 2.6.1. Right Leg

For the right leg graft, which featured only two openings (an inlet and an outlet), a zero pressure-gauge boundary condition was used at the outlet, with no further pressure or boundary conditions. For a rigid wall model with no bifurcations, this is considered to be a reasonable approximation (Wood et al., 2006). Four cardiac cycles, each 1.1 s long, were simulated to reach periodic steady-state.

#### 2.6.2. Left Leg

The left leg bypass featured five different openings, of which one inlet and four outlets. The presence of the four outlets was confirmed by estimating the flow difference between the graft and the common femoral artery. This was done with the help of Doppler data provided as part of the clinical dataset. The velocity curves were extracted from the Doppler ultrasound images for the common femoral artery, the graft and the popliteal artery. Mass flow rates were calculated for each of the vessels. Collateral vessels were modeled as openings following a mass-conservation analysis (**Figure 4**). In the case of multiple outlets, lumped parameter models have been shown to provide a good estimate of the conditions at boundaries allowing to relate pressure to flow through the use of resistance, capacitance and inductance parameters (resistance, compliance and inertance the hydraulic equivalent) which are based on the analogy of fluid systems to electrical circuits (Westerhof et al., 2008; Alimohamadi et al., 2015). In order to estimate model parameters for peripheral resistance and compliance, a lumped parameter model of the vasculature was built using 20-sim (Controllab Products, Enschede, the Netherlands). The 3-D domain was simplified by modeling segments of the geometry with corresponding values for resistance and inductance parameters as seen in Westerhof et al. (1969). This model was used to test and adjust parameters for arterial resistance and compliance to adapt them to the geometry specific to this case. To calculate the resistance, mean pressure difference over mean flow (both part of the clinical dataset) was evaluated to get an estimate as a starting point for finding the appropriate values, and compliance was computed starting from values found in the literature (Stergiopulos et al., 1992). Using the starting parameters (from the literature) for resistance and compliance, the 0-D model was run in 20-sim to obtain flow curves at the outlets. These were compared against the curves from clinical data using the hydraulic-electrical analogy. Changing the values of resistance and compliance parameters at the outlets allowed to adjust the flow curves obtained via 20-sim until matching those from the clinical dataset. A summary of the final parameters used is shown in **Table 3**. After this first calibration, the 0D and 3D models were coupled using Ansys CFX 17 (Ansys Inc., Canonsburg, PA, USA). A diagram of the lumped parameter model and the application of boundary conditions to the 3D model is shown in **Figure 4**.

#### TABLE 3 | List of parameters used in the 0D model.


#### 2.7. Remodeling Cycle

In order to simulate the occlusion of the vessel based on the flow characteristics and biochemical processes considered, a remodeling cycle was used to combine the two approaches in such a way that the biochemical model could feed back into the fluid dynamics model and conversely information on WSS would inform the biochemical model (**Figure 5**). After the first CFD simulation, results were obtained for TAWSS at each mesh node. These values were used to calculate tissue growth at each node by means of the biochemical model discussed above. As it was not possible to run enough cardiac cycles (on a timescale of seconds) in the CFD model to cover the whole timespan of disease development (on a timescale of months), an assumption had to be made to couple the two timescales while remaining within a reasonable computational time. The CFD model was run for 4 cardiac cycles at a time-step size of 25 ms, which allowed to reach periodic steady state. After obtaining the CFD results, the biochemical model was run using the results for wall shear stress from the previous CFD simulations, time averaged over one cardiac cycle in order to capture the variation of the variable over time. A new geometry file accounting for the growth was then created using the results from the biochemical model. This was run with a time-step size of 1 day, either for the full length of time from surgery to clinical data acquisition (in the case of the left leg bypass), or until reaching a significant change in geometry (after 2 months, in the case of the right leg bypass). Finally, a new CFD simulation was run to obtain values of TAWSS and other hemodynamics indicators to model the flow in the new lumen geometry, and the cycle either finished or repeated in case the total time had not yet been achieved.

At the end of the remodeling cycle, results were obtained for comparison with clinical data. The analysis was carried out using ScanIp (Simpleware Ld., Exeter, UK) and Ansys CFD-Post (Ansys Inc., Canonsburg, PA, USA). The lumen occlusion was computed according to the formula below.

Percentage of lumen volume occluded =

100 × Initial cross sectional area – Final cross sectional area Initial cross sectional area

#### 3. RESULTS

From the fluid dynamics simulations, the variable of interest for the model is TAWSS, as this is the mechanical factor that has an influence on the turnover of cells. Results of TAWSS were extracted after each step of the remodeling cycle, in order to be used as input data for the biochemical model. **Figure 6A** shows TAWSS in the right side graft at the beginning of the cycle, and after 2 and 4 months, while contour maps of the TAWSS values in the left side graft at time zero and at 8 months are shown in **Figure 6B**. It can be noticed that in both cases WSS distribution changes as the remodeling occurs, and that low TAWSS areas tend to be located in the proximity of bifurcations or turns (as shown by the zoomed in areas in the figure). For instance, in the right side graft case, the areas where TAWSS is lowest are at the proximal and distal anastomosis, with the latter showing a much more extended area where TAWSS is less than 0.5 Pa. Zones with low TAWSS are also present along the graft. However it is interesting to note that due to the remodeling process these areas tend to decrease when not situated near parts of the graft where the flow is likely to be deflected. The same is also valid for the left side graft, in which low TAWSS is located where the diameter of the graft is exposed to sudden changes, or before bifurcations. To illustrate the importance of the remodeling of the graft and how it changes the WSS distribution, a comparison between TAWSS contour plots and hyperplasia growth was made (**Figure 7**). After the remodeling, the changes in the geometry of the graft have a significant impact on the hemodynamics, which in some cases reduced the areas of low TAWSS, while in others it further reduced TAWSS thus causing the area to become even more prone to growth. By affecting the geometry, the locations of NIH growth have a significant impact on the next remodeling cycle. To illustrate the importance of remodeling in the graft, the areas of critical TAWSS (<0.5 Pa) were measured before and after the full remodeling cycle (**Figure 7**). In the right side graft, this showed that the remodeling led to a decrease in the areas of critical TAWSS with the total area dropping from 17.79 to 4.75 cm<sup>2</sup> . However, the area close to the distal anastomosis changed more slowly from 7.2 to 3.5 cm<sup>2</sup> , so the extent of low TAWSS remained significant, causing a critical amount of growth in the area near the distal part of the graft. This part of the graft was also where the lowest values of TAWSS were found, with TAWSS as low as 0.2 Pa before remodeling took place. On the other hand, the proximal part of the anastomosis was subject to a change from 1.72 to 0.35 cm<sup>2</sup> of the low TAWSS area, which, while still causing some degree of growth, did not have such an impact as in the distal segment. A similar behavior was observed in the left graft, with the total area of low TAWSS decreasing from 105.82 to 72.13 cm<sup>2</sup> , the area near the proximal anastomosis from 33 to 21.71 cm<sup>2</sup> , near the distal anastomosis from 77.45 to 44.26 cm<sup>2</sup> . Both grafts present a curvature toward the end, which has an effect on the velocity profile. When going through a turn, the flow at the center for inertia moves slower than the flow near the wall, causing it to move away from the center of the curvature (Giordana et al., 2005). This results in a different flow pattern compared to the case for a straight artery, leaving space for the formation of secondary flows and also slowing down the main flow, both of which are factors causing lower levels of shear stress. For further analysis, the relative residence time (RRT) index, which has previously been used for other cardiovascular diseases to identify areas where TAWSS was low and oscillatory shear index was high, for

instance in the study of atherosclerosis (VanderLaan, 2004), was also considered. As shown in **Figure 8**, this index also indicates the locations of the most severe hyperplasia progression, which suggest high oscillatory shear stress might also play an important role in the development of the disease.

As can be seen from the streamline plots in **Figure 9**, low TAWSS is located mostly in the proximity of areas where the flow is disturbed due to the presence of bifurcations. In the right side graft, at the onset of systole (point 1), flow was steady and some recirculation zones were observed only after the proximal anastomosis, due to the presence of the occluded vessel. At peak systole (point 2), the velocity was uniform although it decreased at the proximal anastomosis and through the slightly larger diameter of the cross section near the distal anastomosis. As the flow slows down and goes into the diastolic phase (points 3, 4, 5), a different distribution can be seen, with a disturbance of the flow at the distal anastomosis, which will likely cause lower values of WSS, especially where changes in diameter occur. This agrees with previous research showing changes in the intensity of vortices after the flow passes from the graft to the artery (Doorly et al., 2002). A similar behavior can also be noticed in the left side graft, where the flow is more streamlined when at peak systole (point 1), except after the narrowing distal to the bifurcation at the lower end, where some reversed flow can be noticed even at systole. When the flow slows down and goes into diastole (points 2, 3), in addition to separation and reversed flow especially before and after bifurcations and where the graft joins the artery, some helicoidal flow can be seen right after the very narrow section at the distal location.

As expected, the disease progression had a different impact at different locations along the graft. **Figure 10** shows five different cross sections of the right hand side bypass with their respective lumen occlusion as obtained from the radiological images. The lumen geometry at time zero is shown in pale yellow, whereas the geometry at 4 months post-operation is marked in dark red. As can be seen, the disease progression was different for the cross sections selected, with most hyperplasia forming in proximity of the anastomoses. **Figure 11A** shows the locations

where the most critical values of NIH development were found in the CT scans in the right bypass, while the critical locations on the left bypass are shown in **Figure 11B**. As restenosis is defined as the development of 50% or more luminal narrowing in the graft (Hill, 2004), this was chosen as the threshold to select the locations of critical NIH development. The measurements show that the locations of growth were reproduced by the model.

#### 4. DISCUSSION

As mentioned in the introduction, several computational approaches have been proposed in the literature to model NIH or restenosis. While some studies focused on developing agentbased models (Dexter et al., 2009; Boyle et al., 2010; Hwang et al., 2011, 2013; Garbey and Berceli, 2013), others (Budu-Grajdeanu et al., 2008) used a deterministic approach based on ordinary differential equations. More recently, the increasing use of computational modeling in medical applications and the latest advancements in these techniques have allowed to study the disease from different angles. Some studies have been using CFD as a tool for analyzing the behavior of the flow in pathological cases. In many cases, studies have been conducted to analyze the flow in stents and grafts and to build hypotheses on its effect on NIH and restenosis. This has confirmed the hypothesis of low shear stress and abnormal flow patterns leading to NIH in the pulmonary artery (Berdajs et al., 2015), carotid artery (Harrison et al., 2014), coronary arteries (Guerciotti et al., 2016) peripheral arteries (McGah et al., 2012; Rivera et al., 2014). In addition, some studies have focused on specific applications, such as vascular access for hemodialysis, looking at minimizing NIH by finding the optimum conditions for performing arteriovenous fistulae operations. Research has been carried out in this area on hemodynamic factors such as WSS (Jia et al., 2015) and oscillatory shear index (Ene-Iordache et al., 2015), but also on variables such as blood flow rate and needle tip position (Fulker et al., 2016). CFD has also been used to analyze possible design optimization routes for grafts for both hemodialysis (Canneyt et al., 2013), coronary artery stents looking at the flow within stent struts (Gundert et al., 2012), comparing different stent geometries (Gogas et al., 2014) and the effect of deformable stents (Martin et al., 2014), peripheral bypasses (Grus et al., 2016), and novel grafting systems such as an endograft for aneurysm repair (Aristokleous et al., 2015) and a mechanism of external vein graft support (Meirson et al., 2015). Moreover, new studies looking at the interaction between mechanical forces and cell response for instance used agent based (Ziraldo et al., 2013), hybrid agent based-continuum (Garbey et al., 2015), coupled agent-based and finite element (Zahedmanesh and Lally, 2011), and mechanistic approaches (Goodman et al., 2016).

The model of NIH presented in this paper shows an intrapatient comparison of two different bypass geometries and the applicability of a patient-specific, multi-scale approach to each case. The physiological parameters used by this model have been tested in previous studies to quantify the behavior of SMC and collagen and its influence on NIH progression. The ability of the model to produce results and to formulate and test hypothesis on a macroscopic scale using data from experiments conducted on a cellular level shows the applicability of a quantitative approach to the interpretation of biological data and processes. The model confirmed that the behavior of the biochemical species play an important role in the disease progression. The model validation was carried out against a patient-specific, unique clinical dataset by detecting the locations of restenosis in the CT scans from the patient, and then comparing them against the simulated results. As can be seen in **Figure 11**, the locations of restenosis in the patient corresponded to those estimated by the model.

In the right side graft, growth estimated for each point ranges from 0.85 mm (region 3) to 1.2 mm (region 1) whilst in the CT scans, an approximate growth of 1.5 mm (region 3) to 2.6 mm (region 1) is observed. In the left side graft, values of growth at the critical locations measured 0.3–0.4 mm in the simulated geometry, while in the real case were estimated to be around 0.8 mm. It is however difficult to correctly measure NIH development due to the quality of the CT scans and uncertainty in the measurements. Additionally, some sections of the artery occluded in the real case and although the location was relatively well predicted, growth was underestimated by the model. There is

a complex interplay between the remodeling/growth in the model and the hemodynamic parameters and this warrants further investigation.

This shows that combining a model describing biochemical interactions and a mechanical model describing the hemodynamics is a promising approach to describe the behavior of the patient's disease. This type of model has a potential use as a clinical tool to inform clinicians on the progression of the disease, and future modifications will allow not only to locate NIH but also to inform on the severity of the growth. The applicability of the model to patient-specific geometries was also evaluated on two different types of vein graft, for both of which the model located areas with the most NIH development. In addition, the study also lead to a further analysis of the importance of including collateral vessels in the model, which were found to have an influence on the fluid dynamics results and consequently on the biochemical processes as well.

This work has shown the potential of using experimental data to develop integrative in silico models to study NIH progression. A combination of data coming from in vivo and in vitro experiments was particularly useful to get a better understanding of the disease and design a computational model able to produce quantitative information related to relevant dynamical processes of NIH progression. In particular, from the fluid dynamics aspect it was important to be able to apply an appropriate type of boundary conditions. Although for the right side bypass this was relatively straightforward due to the presence of only one inlet and one outlet and the validity of the stress-free boundary condition at the outlet, the conditions were more complicated on the left graft. As stated above, boundary conditions for this model were provided by a lumped parameter model. This allowed to account for the effects of resistance of the peripheral vessels, through the resistance parameter, as well as vessel elasticity through the compliance parameter (Shi et al., 2011). Applying a zero-pressure boundary condition in this case would have led to a non-physiological flow split, with most of the flow being directed toward the path of least resistance. In addition, accounting for these aspects of the flow allows to obtain physiological pressure curves (Kim et al., 2010).

Finally in order to study the performance of the whole multiscale modeling approach proposed in this paper, the sensitivity of NIH development with respect to key parameters in the

biochemical model was evaluated. In this initial analysis the kinetic rates related to proliferation and apoptosis of smooth muscle cells were selected (given the key role of these cells in NIH progression).

As seen in **Figure 12**, NIH growth was sensitive to changes in both the apoptosis and proliferation coefficients. In particular the growth rate was more sensitive to the increase in the apoptosis parameter, with values increasing by up to three folds when the parameter was increased by 20%.

These results show how important is the influence of the biochemical parameters on the final clinical endpoint simulated by this model. So far, with these results it is not possible to conclude about how different smooth muscle cell behaviors could have a different impact on disease progression. However, if a more detailed biochemical model is proposed, the same kind of sensitivity analysis might enable us to draw conclusions about the influence of specific biological/physiological mechanisms (e.g., intra/extracellular mechanisms or processes related to inflammation) that play a key role in NIH.

The multi-scale modeling approach proposed here, has shown how using a mathematical description capturing different biological and physiological scales related to NIH can be used to have a better holistic understanding of the disease allowing the use of advanced computational tools to generate dynamic results

related to disease progression. This kind of approach is not only useful for simulation (through validation with clinical data) but also for further analyses (e.g., sensitivity analyses) that can be linked to testing specific biological hypotheses.

# 4.1. Limitations

The voxel size of the CT images was 0.8 mm, which is very close to the dimensions of the arterial geometry analyzed. In addition, although CT scans were available of both pre and post-surgery, only post-surgery scans were used, and the geometry right after surgery was recreated as described in the method section. While this method to obtain the initial images is subject to inaccuracies due to both the remodeling occurring within the tissue and the quality of CT scans, the resolution of the images was the highest available for this study, as no other data acquisition techniques outside of those already part of the standard clinical procedure were used.

Results from the multi-scale approach using the biochemical model proposed in this paper are very encouraging even if only one type of cells was considered, i.e., smooth muscle cells. A next extension of this part of the model might consider additional types of cells contributing to NIH (e.g., fibroblasts, macrophages) (Model and Dardik, 2012).

An additional extension to this model could include the description of the endothelium and different functions related to this layer, for example the release of molecules that affect cell response and growth (Cui et al., 2014), the regulation of leukocyte adhesion after injury (Tseng et al., 2014), and the control of the uptake of immune cells, i.e., monocytes that later differentiate into macrophages in the vascular wall (Jaipersad et al., 2014).

As this is a first attempt to test the model, the time taken for simulations could be further improved. For instance, in the case of the left leg bypass the large number of mesh elements caused the process to be more computationally expensive compared to the case of the right leg bypass. In order to minimize computational time so that it would remain within values for potential clinical use, only one remodeling cycle was performed for the left side graft. Although this should not have affected

the results, optimizing simulation time is key for improving the model.

The introduction of the impedance condition could also be added as part of the boundary conditions (Du et al., 2015). In addition, although the femoral artery is considered a large artery and a Newtonian model of blood flow should be sufficient, it could be interesting to compare simulations run using different non-Newtonian models against the current Newtonian model to evaluate whether there are any differences in the results. The use of non-Newtonian models has been shown to lead to different results in arterial flow studies. Newtonian models have already been shown to overestimate WSS in a previous study (Xiang et al., 2011), and the same might be happening in this case, especially at points where the diameter of the vessel becomes very small.

Another feature that might improve the model could be the introduction of moving walls, to mimic the pulsating movement of the arterial wall. Including the movement of solid boundaries in CFD simulations of arteries has been shown to improve results in previous models of cardiovascular disease (Alimohammadi et al., 2015), and the same is likely to be valid for NIH. It should be noted that the boundary conditions at the inlet also affect the outcome of the simulation. Limitations linked to the data acquisition and quality of ultrasound measurements themselves can be significant and there is additional uncertainty introduced due to the smoothing of the ultrasound data.

Finally, it is important to mention that the issue of patientspecific biomarkers is essential when trying to understand peripheral arterial disease and graft failure in individual patients. In these patients circulating biomarkers have been found to correlate with disease presence and severity (Owens et al., 2007). There is also evidence that certain circulating biomarkers may predict development of restenosis/graft disease; potential

candidates include C-reactive protein, inflammatory cytokines, growth factors and adhesion molecules (Willigendael et al., 2005; Wildgruber et al., 2007). Further work will consist on collecting longitudinal datasets including imaging data and biomarkers in order to better represent and understand the biological aspects of the disease and to correlate them with hemodynamic variables.

# 5. CONCLUSION

The study presented a multi-scale model of the behavior of peripheral vein grafts affected by NIH when subject to WSS, the first to date to simulate the disease progression based on the dynamics of the flow in patient-specific geometries. A comparison intra-patient between a femoro-popliteal and femoro-distal graft was carried out which showed the potential but also, limitations of the applicability of the model to different cases. By means of this model it has been possible to analyze NIH progression and its mechanisms from a hemodynamics and a biochemical standpoint. In the future, the model will be used for the analysis of more patients in order to test its applicability on different cases of vein graft NIH. Moreover, the remodeling framework produced could be expanded in future work, for instance by extending the model to other components of vascular tissue. Mathematical models already exist that describe the cellular behavior in atherosclerosis (Cilla et al., 2013; Díaz-Zuccarini et al., 2014; Pichardo-Almarza et al., 2014). These could be modified to describe behavior of additional cells such as macrophages in NIH, in order to include a better description of the inflammation processes triggered in the disease. In addition, the endothelial layer could be added as already seen in previous literature (Pichardo-Almarza et al., 2014; Alimohammadi et al., 2016) to describe the transport of biochemical species into the vascular tissue. Modeling the endothelium would also enable to take into account the effect of WSS on the behavior of monocytes, as previously done for atherosclerosis (Díaz-Zuccarini et al., 2014), thus considering a further aspect of the mechanical response of cells. Further analysis could be done by considering other shear stress indices, as some of these have been shown to give better predictions in models of other vascular diseases such as atherosclerosis (Alimohammadi et al., 2016), which is also a disease affected by mechanical forces similarly to NIH, although through different cell species and processes. More flow characteristics could be included to describe recirculation and separation zones, all of which have been shown to affect the development of NIH (Murphy and Boyle, 2010). Finally, future work should investigate the use of fluid structure interaction in order to capture the movement of the arterial wall when subject to hemodynamic forces, which might have an influence in the results.

# ETHICS STATEMENT

This study was carried out in accordance with the recommendations of the Institutional Human Investigation Committee (Veterans Affairs Connecticut Healthcare System, West Haven, CT, USA) with written informed consent from all subjects. All subjects gave written informed consent in accordance with the Declaration of Helsinki. The protocol was approved by the Institutional Human Investigation Committee (Veterans Affairs Connecticut Healthcare System, West Haven, CT, USA).

# AUTHOR CONTRIBUTIONS

FD, CP, and VD conceived the study and carried out simulations and wrote the manuscript. MB created the geometries from CT scans, AD acquired the clinical data and provided medical input. SH conceived the study and provided medical input.

# FUNDING

This project is supported by the Engineering and Physical Sciences Research Council (EPSRC) through the Doctoral Training Programme "Healthcare Engineering for an aging population." The authors are grateful for the support of the EPSRC Network "POEMS" (EP/L001101/1) for facilitating the visit of AD to UCL. The authors gratefully acknowledge support by the EPSRC grant "Personalized Medicine Through Learning in the Model Space" (grant number EP/L000296/1) and the Leverhulme Trust Senior Research Fellowship "Exploring

#### REFERENCES


the Unknowable Using Simulation: Structural Uncertainty in Multiscale Models" (Fellowship number RF-2015-482). This work was supported by the resources and the use of facilities at the Veterans Affairs Connecticut Healthcare System (West Haven, CT).


from a proof-of-concept. Glob. Cardiol. Sci. Pract. 2014:56. doi: 10.5339/gcsp. 2014.56


shear stress. Matrix Biol. 29, 586–593. doi: 10.1016/j.matbio.2010. 07.003


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2017 Donadoni, Pichardo-Almarza, Bartlett, Dardik, Homer-Vanniasinkam and Díaz-Zuccarini. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

digital media

of impactful research

article's readership