PATHOPHYSIOLOGY OF THE BASAL GANGLIA AND MOVEMENT DISORDERS: GAINING NEW INSIGHTS FROM MODELING AND EXPERIMENTATION TO INFLUENCE THE CLINIC

EDITED BY: Daniela Andres, Olivier Darbin and Marcelo Merello PUBLISHED IN: Frontiers in Human Neuroscience

#### *Frontiers Copyright Statement*

*© Copyright 2007-2017 Frontiers Media SA. All rights reserved. All content included on this site, such as text, graphics, logos, button icons, images, video/audio clips, downloads, data compilations and software, is the property of or is licensed to Frontiers Media SA ("Frontiers") or its licensees and/or subcontractors. The copyright in the text of individual articles is the property of their respective authors, subject to a license granted to Frontiers.*

*The compilation of articles constituting this e-book, wherever published, as well as the compilation of all other content on this site, is the exclusive property of Frontiers. For the conditions for downloading and copying of e-books from Frontiers' website, please see the Terms for Website Use. If purchasing Frontiers e-books from other websites or sources, the conditions of the website concerned apply.*

*Images and graphics not forming part of user-contributed materials may not be downloaded or copied without permission.*

*Individual articles may be downloaded and reproduced in accordance with the principles of the CC-BY licence subject to any copyright or other notices. They may not be re-sold as an e-book.*

*As author or other contributor you grant a CC-BY licence to others to reproduce your articles, including any graphics and third-party materials supplied by you, in accordance with the Conditions for Website Use and subject to any copyright notices which you include in connection with your articles and materials.*

> *All copyright, and all rights therein, are protected by national and international copyright laws.*

*The above represents a summary only. For the full conditions see the Conditions for Authors and the Conditions for Website Use.*

ISSN 1664-8714 ISBN 978-2-88945-317-7 DOI 10.3389/978-2-88945-317-7

#### About Frontiers

Frontiers is more than just an open-access publisher of scholarly articles: it is a pioneering approach to the world of academia, radically improving the way scholarly research is managed. The grand vision of Frontiers is a world where all people have an equal opportunity to seek, share and generate knowledge. Frontiers provides immediate and permanent online open access to all its publications, but this alone is not enough to realize our grand goals.

#### Frontiers Journal Series

The Frontiers Journal Series is a multi-tier and interdisciplinary set of open-access, online journals, promising a paradigm shift from the current review, selection and dissemination processes in academic publishing. All Frontiers journals are driven by researchers for researchers; therefore, they constitute a service to the scholarly community. At the same time, the Frontiers Journal Series operates on a revolutionary invention, the tiered publishing system, initially addressing specific communities of scholars, and gradually climbing up to broader public understanding, thus serving the interests of the lay society, too.

#### Dedication to Quality

Each Frontiers article is a landmark of the highest quality, thanks to genuinely collaborative interactions between authors and review editors, who include some of the world's best academicians. Research must be certified by peers before entering a stream of knowledge that may eventually reach the public - and shape society; therefore, Frontiers only applies the most rigorous and unbiased reviews.

Frontiers revolutionizes research publishing by freely delivering the most outstanding research, evaluated with no bias from both the academic and social point of view. By applying the most advanced information technologies, Frontiers is catapulting scholarly publishing into a new generation.

#### What are Frontiers Research Topics?

Frontiers Research Topics are very popular trademarks of the Frontiers Journals Series: they are collections of at least ten articles, all centered on a particular subject. With their unique mix of varied contributions from Original Research to Review Articles, Frontiers Research Topics unify the most influential researchers, the latest key findings and historical advances in a hot research area! Find out more on how to host your own Frontiers Research Topic or contribute to one as an author by contacting the Frontiers Editorial Office: researchtopics@frontiersin.org

# **PATHOPHYSIOLOGY OF THE BASAL GANGLIA AND MOVEMENT DISORDERS: GAINING NEW INSIGHTS FROM MODELING AND EXPERIMENTATION TO INFLUENCE THE CLINIC**

Topic Editors:

**Daniela Andres,** National University of San Martin (UNSAM), Argentina **Olivier Darbin,** University South Alabama, United States and National Institute for Physiological Sciences, Japan **Marcelo Merello,** Raul Carrea Institute for Neurological Research, Argentina

Original cover art by Daniela Andres

The basal ganglia constitute a group of subcortical structures, highly interconnected among themselves, as well as with the cerebral cortex, thalamus and other brain areas. These nuclei play a central role in the control of voluntary movement, and their specific pathology comprises the group of diseases known as movement disorders, including Parkinson's disease, Huntington's disease, dystonia and Gilles de la Tourette syndrome, among others. Additionally, the presence of a number of circuits within the basal ganglia related to non-motor functions has been acknowledged. Currently, the basal ganglia are thought to participate in cognitive, limbic and learning functions. Moreover, disorders related to the basal ganglia are known to involve a number of complex, non-motor symptoms and syndromes (e.g. compulsive and addictive behavior). In the light of this evidence, it is becoming clear that our knowledge about the basal ganglia needs to be revised, and that new pathophysiological models

of movement disorders are needed. In this context, the study of the pathophysiology of the basal ganglia and the treatment of their pathology is becoming increasingly interdisciplinary. Nowadays, an appropriate approach to the study of these problems must necessarily involve the use of complex mathematical modeling, computer simulations, basic research (ranging from biomolecular studies to animal experimentation), and clinical research.

This research topic aims to bring together the most recent advances related to the pathophysiology of the basal ganglia and movement disorders.

**Citation:** Andres, D., Darbin, O., Merello, M., eds. (2017). Pathophysiology of the Basal Ganglia and Movement Disorders: Gaining New Insights from Modeling and Experimentation to Influence the Clinic. Lausanne: Frontiers Media. doi: 10.3389/978-2-88945-317-7

# Table of Contents

*05 Editorial: Pathophysiology of the Basal Ganglia and Movement Disorders: Gaining New Insights from Modeling and Experimentation, to Influence the Clinic*

Daniela S. Andres, Marcelo Merello and Olivier Darbin

#### **Section I: Basal ganglia models and theory**


Moran Gilat, Simon J. G. Lewis and Ahmed A. Moustafa

*39 A Mathematical Model of Levodopa Medication Effect on Basal Ganglia in Parkinson's Disease: An Application to the Alternate Finger Tapping Task* Chiara Baston, Manuela Contin, Giovanna Calandra Buonaura, Pietro Cortelli and Mauro Ursino

# **Section II: Basal ganglia functions**


Kenji Kato, Fusako Yokochi, Hirokazu Iwamuro, Takashi Kawasaki, Kohichi Hamada, Ayako Isoo, Katsuo Kimura, Ryoichi Okiyama, Makoto Taniguchi and Junichi Ushiba

#### **Section III: Molecular pathways and neurotransmission**

*82 More than a Rumor Spreads in Parkinson's Disease*

Natalia C. Prymaczok, Roland Riek and Juan Gerez


#### **Section IV: Non-motor symptoms of basal ganglia disorders**

*116 Preliminary Evidence of Apathetic-Like Behavior in Aged Vesicular Monoamine Transporter 2 Deficient Mice*

Aron Baumann, Carlos G. Moreira, Marta M. Morawska, Sophie Masneuf, Christian R. Baumann and Daniela Noain

*126 Action Experience and Action Discovery in Medicated Individuals with Parkinson's Disease*

Jeffery G. Bednark, John N. J. Reynolds, Tom Stafford, Peter Redgrave and Elizabeth A. Franz

#### **Section V: Signal analysis and clinical approaches**

*136 Structure Function Revisited: A Simple Tool for Complex Analysis of Neuronal Activity*

Federico Nanni and Daniela S. Andres

*146 Parkinsonian Balance Deficits Quantified Using a Game Industry Board and a Specific Battery of Four Paradigms*

Olivier Darbin, Coral Gubler, Dean Naritoku, Daniel Dees, Anthony Martino and Elizabeth Adams


Niklas König, Navrag B. Singh, Christian R. Baumann and William R. Taylor

#### **Section VI: Deep brain stimulation (DBS)**


Takashi Morishita, Masa-aki Higuchi, Kazuya Saita, Yoshio Tsuboi, Hiroshi Abe and Tooru Inoue

*204 Probing the Role of Medication, DBS Electrode Position, and Antidromic Activation on Impulsivity Using a Computational Model of Basal Ganglia* Alekhya Mandali and V. Srinivasa Chakravarthy

# Editorial: Pathophysiology of the Basal Ganglia and Movement Disorders: Gaining New Insights from Modeling and Experimentation, to Influence the Clinic

#### Daniela S. Andres <sup>1</sup> \*, Marcelo Merello<sup>2</sup> and Olivier Darbin3, 4

**Editorial on the Research Topic**

<sup>1</sup> Laboratory of Neuroengineering, Science and Technology School, National University of San Martín, San Martín, Argentina, <sup>2</sup> Movement Disorders Section, Neuroscience Department, Raul Carrea Institute for Neurological Research, Fundación para la Lucha contra las Enfermedades Neurológicas de la Infancia (FLENI), Buenos Aires, Argentina, <sup>3</sup> Department of Neurology, University South Alabama, Mobile, AL, United States, <sup>4</sup> Division of System Neurophysiology, National Institute for Physiological Sciences, Okazaki, Japan

Keywords: Parkinson's disease, movement disorders, Huntington's disease, basal ganglia, temporal structure, personalized therapies, alpha synuclein (α syn), non-motor symptoms

#### Edited by:

Camillo Porcaro, Istituto di Scienze e Tecnologie della Cognizione (ISTC) – CNR, Italy

#### Reviewed by:

Sebastien Helie, Purdue University, United States Wolf-Julian Neumann, Charité Universitätsmedizin Berlin, Germany

#### \*Correspondence: Daniela S. Andres dandres@unsam.edu.ar

Received: 01 August 2017 Accepted: 05 September 2017 Published: 20 September 2017

#### Citation:

Andres DS, Merello M and Darbin O (2017) Editorial: Pathophysiology of the Basal Ganglia and Movement Disorders: Gaining New Insights from Modeling and Experimentation, to Influence the Clinic. Front. Hum. Neurosci. 11:466. doi: 10.3389/fnhum.2017.00466 **Pathophysiology of the Basal Ganglia and Movement Disorders: Gaining New Insights from Modeling and Experimentation, to Influence the Clinic**

The human brain is complex at every level, from the scale of single neurons to microcircuits and large neuronal networks. Although this complexity is well known and studied in neuroscience, few tools or concepts of complex analysis have been transferred to the clinic yet. In the case of the basal ganglia, there has been much debate about the necessity to include nonlinear concepts into pathophysiology models (Andres and Darbin, in press). However, for new approaches to make an impact on the clinic active research is needed on many fronts: new clinical, experimental and modeling insights are crucial. In other words, research in the field of basal ganglia and related disorders is becoming increasingly interdisciplinary. On this research topic different authors challenge classic paradigms of basal ganglia pathophysiology and movement disorders in 6 areas of research. Main findings and breakthroughs are summarized in the next paragraphs.

# BASAL GANGLIA MODELS AND THEORY


• Action selection concepts and pharmacokinetics combined in a mixed modeling approach can be used to study how dopamine affects a motor task at different stages of PD (Baston et al.).

# BASAL GANGLIA FUNCTIONS


# MOLECULAR PATHWAYS AND NEUROTRANSMISSION


# NON-MOTOR SYMPTOMS OF BASAL GANGLIA DISORDERS


# SIGNAL ANALYSIS AND CLINICAL APPROACHES


# DEEP BRAIN STIMULATION (DBS)


The results published in this topic promise great advancements in coming years in the field of basal ganglia pathophysiology and related disorders.

# AUTHOR CONTRIBUTIONS

DA, MM, and OD are responsible for the full content of this article.

# REFERENCES

Andres, D. S., and Darbin, O. (in press). Complex dynamics in the basal ganglia: health and disease beyond the motor system. J. Neuropsy. Clin. Neurosci.

**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2017 Andres, Merello and Darbin. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Modeling and Theories of Pathophysiology and Physiology of the Basal Ganglia–Thalamic–Cortical System: Critical Analysis

#### Erwin B. Montgomery Jr. 1,2 \*

<sup>1</sup> Greenville Neuromodulation Center, Greenville, PA, USA, <sup>2</sup> Departments of Neuroscience and Philosophy, Thiel College, Greenville, PA, USA

Theories impact the movement disorders clinic, not only affecting the development of new therapies but determining how current therapies are used. Models are theories that are procedural rather than declarative. Theories and models are important because, as argued by Kant, one cannot know the thing-in-itself (das Ding an sich) and only a model is knowable. Further, biological variability forces higher level abstraction relevant for all variants. It is that abstraction that is raison d'être of theories and models. Theories "connect the dots" to move from correlation to causation. The necessity of theory makes theories helpful or counterproductive. Theories and models of the pathophysiology and physiology of the basal ganglia–thalamic–cortical system do not spontaneously arise but have a history and consequently are legacies. Over the last 40 years, numerous theories and models of the basal ganglia have been proposed only to be forgotten or dismissed, rarely critiqued. It is not harsh to say that current popular theories positing increased neuronal activities in the Globus Pallidus Interna (GPi), excessive beta oscillations and increased synchronization not only fail to provide an adequate explication but are inconsistent with many observations. It is likely that their shared intellectual and epistemic inheritance plays a factor in their shared failures. These issues are critically examined. How one is to derive theories and models and have hope these will be better is explored as well.

#### Edited by:

Olivier Darbin, National Institute for Physiological Sciences (NIPS), Japan

#### Reviewed by:

Satomi Chiken, National Institute for Physiological Sciences (NIPS), Japan Mesbah Alam, Hannover Medical School, Germany

\*Correspondence: Erwin B. Montgomery ebmontgomery@wisc.edu

Received: 03 June 2016 Accepted: 06 September 2016 Published: 21 September 2016

#### Citation:

Montgomery EB Jr. (2016) Modeling and Theories of Pathophysiology and Physiology of the Basal Ganglia–Thalamic–Cortical System: Critical Analysis. Front. Hum. Neurosci. 10:469. doi: 10.3389/fnhum.2016.00469 Keywords: basal ganglia–thalamic–cortical system, model, theory, globus pallidus interna rate theory, betaoscillations theory, increased synchronization theory, principles of causational and informational synonymy, logical fallacies

# THE IMPACT OF THEORY IN THE CLINIC

Theories, particularly the prevailing theory, have an enormous impact on clinical practice and medical science, the latter often determines clinical practice. Consider the Globus Pallidus Interna (GPi) Rate theory, which posits that overactivity of the GPi suppresses intended movements, resulting in hypokinetic disorders consequent to the inhibitory influence of GPi neurons on neurons of the ventral thalamus pars oralis (Vop). Further, underactivity of GPi neurons results in unintended movements by abnormal disinhibition of thalamic neurons, resulting in hyperkinetic disorders. This theory is demonstrably incorrect, and contrary evidence has been available ever since the initial publications of the theory (reviewed in Montgomery, 2007). Consider the following consequences of the GPi Rate theory.

# Continued Arguments that High-Frequency Deep Brain Stimulation (DBS) Reduces the Output of the GPi, Improving the Hypokinetic Symptoms of Parkinson's Disease by Reducing Abnormal Inhibition of Vop Neurons

While there is considerable evidence to the contrary, this theory does not explain why high-frequency deep brain stimulation (DBS) also is effective for hyperkinetic disorders. This hypothesis was based on the logical Fallacy of Pseudotransitivity, which presumes synonymy between the neurophysiological mechanisms between DBS in the vicinity of the GPi and subthalamic nucleus (STN) and pallidotomy and subthalamotomy, respectively (Montgomery, 2012). It is important to note that the judicious use of logical fallacies is critical to the advancement of science when its use generates hypotheses for subsequent experimental vindication. However, it is injudicious to use such fallacies to argue in support of any theory or to argue against other alternative theories. In this author's opinion, this theory has slowed progress in understanding the mechanisms of action of DBS and, importantly, theories of pathophysiology and physiology of the basal ganglia–thalamic–cortical system. It is likely that these arguments also delayed the development of better therapeutic approaches. Indeed, the intuitive appeal of both the GPi Rate theory and the postulated mechanisms of action of DBS became mutually re-enforcing, creating a circularity of explanatory theory, making alternative theories much more difficult to gain traction. At least the consideration of alternatives has the potential of being right.

# Dichotomization of DBS into High Frequencies, Associated with Improvement, and Low-Frequency DBS Associated with Worsening of Symptoms of the Disease Treated

There is considerable evidence to the contrary, yet this notion persists, particularly if insufficient caution and attention are paid to sampling issues with regard to DBS frequencies (Huang et al., 2014). However, the assumption of a dichotomization between high and low frequencies based on the suspect onedimensional push–pull dynamics of the GPi Rate theory (see earlier discussion) persists (di Biase and Fasano, 2016). The consequence is that the full clinical potential of DBS at a wide range of frequencies has not yet been explored.

# Beyond the Specifics of the GPi Rate Theory, the Underlying Enabling Presumptions are a Dichotomization of the Mechanisms into Two Opposing Contrary Stable States

In the case of the GPi Rate theory, the two states of high neuronal activity vs. low neuronal activity in the GPi dominate theoretical explanations. However, one just as easily could substitute increased beta-oscillatory activity or increased synchronization of the GPi for the overactivity of the GPi neurons in hypokinetic disorders. The one-dimensional push–pull dynamics for which the two states have been the poles that constituted the basis for understanding the effects of various treatments. The onedimensional push–pull dynamics presupposes the existence to two steady states.

Consider viewing a grayscale that ranges from white to black at the extremes. How many colors (shades of gray) are in the grayscale? Since Aristotle and his notion of the Contraries (Montgomery, 2012), the answer continues to be just two—white and black. All the other shades are admixtures of varying degrees of black and white. In the case of the GPi Rate theory, the dimension has overactivity and underactivity at the extremes of the continuum. Disease is occasioned when the actual condition is at an extreme and normality is at some intermediate condition.

The Cholinergic/Dopaminergic Imbalance theory of the movement disorders in the 1970s posited two one-dimensional systems that complemented each other. One dimension was organized on the relative excess or a deficiency of acetylcholine in the basal ganglia. The other dimension had relative excess and relative deficiency of dopamine at its poles. Interestingly, with the ascendency of the GPi Rate theory, any theories to explain the obvious benefit of anticholinergic medications seem to have evaporated, perhaps not unlike the memory holes in the Ministry of Truth in Orwell's novel 1984 (Orwell, 1949).

The one-dimensional system surviving the Cholinergic/Dopaminergic Imbalance theory was a continuum between relative deficiency and excess dopamine in the striatum, resulting in hypokinetic and hyperkinetic disorders, respectively. The stable states relative to dopamine content thus influenced the notion that dopamine release in the striatum was relatively constant, giving rise to tonic dopamine activity—this despite the demonstration by Schultz et al. (2015) of a very rapid but brief increase in dopamine neuronal activities during a motor task (Schultz, 1986). This theory led directly to dopamine replacement therapies that provided constant application of dopamine (Obeso and Olanow, 2011). However, the failure of tonic dopamine replacement therapy is seen in patients with Parkinson's disease where increased amounts of presumably tonic dopamine were in the striatum as a result of fetal dopamine transplantation. Many, if not most, patients did not improve. It has been only recently that renewed interest in the dynamic or phasic operations of dopamine neurons has been rekindled (Schultz et al., 2015).

# Presumption of the One-Dimensional Push–Pull Dynamics Influencing Biomedical Research

Research attempting to improve the hypokinetic symptoms of Parkinson's through genetic manipulation directed at the STN to reverse the state of the STN neurons from one end of the single dimension (excitatory) to the other end of the same dimension (inhibitory) presumes the GPi Rate theory (LeWitt et al., 2011). One might argue ''doesn't the fact that pallidotomy and genetic reversal of the STN neurotransmitter effect demonstrate the validity of the GPi Rate theory?'' As will be seen, these evidences do not. Theories that seek vindication by a demonstration of predictions derived from the theories are the Fallacy of Confirming the Consequence. Note that this is not to say that the theory cannot be true, but only that the demonstration of its predictions does not assure that it is true (discussed in greater detail later).

# Presumption of the One-Dimensional Push–Pull Dynamics and Symptoms and Signs in the Clinic

The notion of a one-dimensional dichotomy goes back to Aristotle's notion of Contraries and was fundamental to Galen's notion of disease. The concept found a home and credibility in the writing of John Hughlings Jackson, called the Father of English Neurology, in his dichotomization of symptoms and signs into positive and negative. Paralysis was a negative sign representing insufficient activity in the motor systems, while seizures and spasms were attributed to excessive activity in the motor systems. Extended to disorders of the basal ganglia, the symptoms and signs were attributed to deficiency and excess of basal ganglia function, respectively. As there is little corroborating evidence sufficient to prove the case, it must be taken as a theory. However, the clinical dichotomization resonates with the GPi Rate theory, as well as with the predecessor and successors of the theory, again demonstrating the intuitive appeal and power of the simple one-dimensional push–pull dynamics.

Parkinson's disease is considered the archetype of hypokinetic syndromes. The diametric opposite in the one-dimensional dichotomy are the hyperkinetic syndromes, such as Huntington's disease. It is a testament to the power of theory, particularly those that are intuitive and appealing, that contrary evidence would be trumped. Patients with Parkinson's disease can have bradykinesia as well as hyperkinesia simultaneously. Similarly, patients with chorea from Huntington's disease are bradykinetic on volitional tasks.

There likely are many reasons for the power of theories such as the GPi Rate theory and other one-dimensional push–pull theories, such as Beta-Oscillations or Increased Synchronization theories. These range from the polemical, where proponents of popular theories make it difficult for insurgent theories (Kuhn, 1965) by not funding grants, accepting publications, or inviting advocates for alternatives to present, to ways of observing phenomena and adjudicating what is relevant and acceptable evidence (Montgomery, 2012). Abraham Maslov wrote, ''I suppose it is tempting, if the only tool you have is a hammer, to treat everything as if it were a nail'' (Maslow, 1966). Think of popular theory as a very big hammer.

# THEORIES ARE NECESSARY BUT ALL THE MORE REASON FOR DILIGENCE

Discussions of theory in biomedical research and clinical science are fraught with difficulty due to misconceptions about the nature of theory. While theory is readily appreciated in physics, chemistry and psychology, often theory is a pejorative term in biomedical research. In the latter, often the presumption is that theory is just so much metaphysical speculation and that data speak for itself. Theory is unnecessary. Disabusing any reader holding this position is beyond the scope of this article. For those readers, this author can only ask forbearance. The position held here is that, at the very least, the facts-of-the-matter, such as observations and evidence as related to the pathophysiology and physiology of the basal ganglia–thalamic–cortical system, are inadequate for a complete explication of the altered behaviors associated with disorders of the basal ganglia–thalamic–cortical system. A certain amount of ''connecting the dots'' is necessary and thus, the necessity of theory to do so. At the very least, the theoretical connections ''between the dots'' become the testable hypotheses for subsequent experimentation and thus new knowledge.

Models are a form of theory. Models typically are procedural rather than declarative. They explain by doing but are theories nonetheless. The great advantage is that models can succeed in a procedural sense without the necessity of declarative explications. The latter is discovered post hoc. Thus, what is epistemically true of any model is also true of theory.

Science is remarkably effective in the accumulation of facts-of-the-matter. Advances in scientific technology truly are breathtaking, and other fields of human endeavor to discover new knowledge are left wanting. However, truth be told, science is poor at what is most fundamental, that is, the generation of hypotheses that would be subjected to the Scientific Method. This is to be expected as hypotheses necessarily extend beyond the facts-of-the-matter, whether by interpolation or extrapolation, and thus beyond scientific technology. Theories, and their specification in models, require the application of reasoning rather than discovery. Herein lies a problem. While scientists are happy to discuss technology, questioning their reasoning seems beyond the pale. Those that do question reasoning are labeled judgmental and are dismissed. However, any scientific experiment is only as good as the hypothesis it seeks to support or refute. Some scientists just think wrongly and it would be a disservice to look the other way. Any attempts to generate good hypotheses are, first and foremost, exercises in reasoning and not technology.

Whatever theory is, it is not logical deduction and, consequently, does not carry the certainty of deduction. Deduction, either propositional of the form if **a** implies **b** is true and **a** is true, then **b** is true or syllogistic deduction of the form all **a**'s are **b**'s and **c** is an **a**, the **c** is a **b**, provides the highest certainty. The epistemic utility of deduction is that one can be assured true conclusions from a deductive argument with valid propositions and true premises. At the very least, one proposition or premise in any theory cannot be taken as valid or true, respectively, as otherwise the theory would be fact, law, or principle. In an important sense, it is good that theories are not deductions, as deductions do not provide new knowledge, which is the purpose of theory.

Fallacies of deduction can provide for new knowledge as they generate hypotheses that combine to form theories. Indeed, the Scientific Method, when used to assert a positive claim, as opposed to denying the claim, is the Fallacy of Confirming the Consequence. However, as necessary logical fallacies, theories require great caution in their construction and use, as will be demonstrated.

Induction is an alternative that could provide new knowledge but requires presuppositions that risk tautology. For example, if it is observed that every case of increased GPi neuronal activity is consistently associated with Parkinsonism, then one can induce that all cases of Parkinsonism are due to overactivity of the GPi neurons. However, such experience does not preclude the possibility that some case of Parkinsonism are not associated with overactivity of the GPi. The problem is what were the circumstances that allowed observations of overactivity of GPi neurons? If it is limited to high, perhaps excessive, doses of the neurotoxin n-methyl-4-phenyl-1,2,3,6-tetrahydropyridine (MPTP), then every examined case using such high doses would have increased GPi neuronal activities. However, as has been known since 1986, one can produce parkinsonism in nonhuman primates, demonstrating neurometabolic changes demonstrated by others as being associated with parkinsonism, without causing the changes in neuronal activities predicted by the GPi Rate theory (Montgomery et al., 1986; subsequently confirmed by Wang et al., 2009). Indeed, the use of dopamine antagonists and electrolytic lesions of the nigral–striatal pathway produced Parkinsonism in nonhuman primates without causing overactivity in the GPi (Percheron et al., 1993).

The same doubts attend the theory that increased betaoscillations produce Parkinsonism, as 15–20% of patients with Parkinson's disease do not have increased beta-oscillations. If induction from observations in patients with Parkinson's disease was used, then the claim is invalid. Note that this is not to say that increased beta-oscillations cannot cause Parkinsonism (hence, a sufficient cause). Rather, the evidence shows that increased beta-oscillations cannot be a necessary cause of Parkinsonism. Similarly, DBS that improves Parkinsonism increases synchronization, which means that it cannot be true that increased synchronization is causal to Parkinsonism. Electroencephalographic-evoked potentials in response to DBS likely would not be detectable if DBS caused desynchronization (Baker et al., 2002; Walker et al., 2012). Similarly, a study of M-wave responses in normal subjects and patients with Parkinson's disease, with and without DBS, suggests an increased synchronization of lower motor neuronal activity in patients with untreated Parkinson's disease but even greater synchronization with therapeutic DBS (Aldewereld et al., 2012).

Perhaps the only other alternative is the use of logical fallacies, particularly the Fallacy of Confirming the Consequence and the Fallacy of Pseudotransitivity. The Scientific Method, when used to affirm a claim, is the Fallacy of Confirming the Consequence, which is of the form if **a** implies **b** is true and **b** is true then **a** is true. For example, if overactivity of the GPi neurons (or increased beta-oscillations or increased synchrony) implies Parkinsonism and Parkinsonism is found, then there must be GPi neuronal overactivity (or increased beta-oscillations or increased synchrony). It is important to note that had **b** in the formal statement been false, then **a** would have to be false or the proposition that **a** implies **b** would have to be invalid. Thus, if Parkinsonism was not found using methods that produced overactivity, increased betaoscillations or increased synchrony, then the latter cannot be causal to Parkinsonism. That Parkinsonism was demonstrated is at least a partial victory, assuming that the victory is held with skepticism. Such results are better termed vindication rather than verification.

It would be a disservice to merely point out where some current theories and models regarding the pathophysiology and physiology of the basal ganglia–thalamic–cortical system are inconsistent with or contrary to facts. Rather, it is important to understand the factors that lead to the creation of the theory in the first place, least one repeats the errors. Theories do not emerge fully formed as Athena from the head of Zeus, they have a history. As Santayana (1905) said, ''Those who do not remember the past are condemned to repeat it.''

# METAPHORS TO ADVANCE UNDERSTANDING OF THE PHYSIOLOGY AND PATHOPHYSIOLOGY OF THE BASAL GANGLIA–THALAMIC–CORTICAL SYSTEM

A fundamental question, almost entirely ignored, is where do the hypotheses that constitute theories come from? Very frequently, hypotheses derive from metaphors. For example, it was known that pallidotomy, presumed to silence the output of the GPi, improves Parkinsonism. DBS in the vicinity of the GPi likewise improves Parkinsonism. Thus, the reasonable theory can be constructed that DBS silences the GPi neurons based on the metaphor, pallidotomy is to improved Parkinsonism as DBS in the vicinity of the GPi is to improved Parkinsonism. As pallidotomy silences GPi neurons, DBS in the vicinity of the GPi must silence neurons.

From the metaphor derives a hypothesis that DBS in the vicinity of the GPi should silence neurons. It is now a matter of experimentation to demonstrate the case by recording neuronal activities within the GPi. As this proved to be problematic, the alternative was to use a recording of thalamic neurons, as they would manifest the consequence of increased or decreased GPi neuronal activities. Unfortunately, proposals to do just that found hostility. This is an example where the metaphor relating pallidotomy to DBS in the vicinity of the GPi went from enabling research, as a method of hypothesis generation, to disabling research as the metaphor was used as proof against the alternatives.

# METAPHORS AND MODELS

There is a real question of whether any human can ever have a complete and explicit knowledge of the brain given its complexity. All one might know is some approximation of the brain. Thus, at best, all any human can know is some model of the brain and that model must serve as a metaphor for the brain, as the brain itself is unknowable.

Even if it were possible to know everything about an individual's brain, another's brain is likely to be different. Even if it were possible to know everything about the brains of each of the two individuals, it would be impossible to know what it is about their brains that leads to an understanding of behaviors in common. One could say that specific arrangements of neurons in person **A** result in behavior **X**. One also could say that a specific arrangement of neurons in person **B** also results in behavior **X**. As those arrangements are different, one would have to either say that there are as many behaviors **X** as there are different arrangements of neurons in every individual person or say that there is something in the arrangements of both persons **A** and **B** that transcends the actual arrangements of neurons in order to produce the same behavior **X**. The question is what that transcendent entity would look like.

The notion that there is some transcendent entity that accounts for the same functions or behaviors despite entirely different instantiation in the actual neural structure is called Functionalism. The analogy is that the same word processing operations can be implemented in a variety of hardware. The intriguing question is whether that transcendent entity is a better explanation of brain functions than the actual neural physiological architecture in any actual instantiation. Then would not modeling of the transcendent entity have a higher probability of providing insight than explicit examination of each instantiation in each individual?

Whether an explicit understanding based on the exact instantiation of neuronal physiologic architecture is not possible or whether the transcendent functional entity is a better target for understanding, the net result is that only modeling is possible. Modeling is the only option to gain any understanding.

Models are used as metaphors—the operations are to the brain as are the operations are to the model. Metaphors contain a target and a source domain. The target is the statement that needs explication, while the source domain is the statement that suggests an explication. For example, consider the metaphor DBS is to improvement of Parkinsonism as pallidotomy is to improvement. The source domain pallidotomy is to improvement provides a suggestion as to the nature of the target domain, DBS is to improvement. Whatever mechanisms by which pallidotomy produces improvement are used to suggest the mechanisms by which DBS provides improvement. While this is not verification, at the least it offers a hypothesis for subsequent experimentation.

Every metaphor contains epistemic risk, that is, the metaphor may lead to a wrong hypothesis. The mechanisms underlying pallidotomy need not be the same as underlying DBS and, indeed, they are not. Thus, any modeling is only as good as the enabling metaphor. The question is how to evaluate the potential of any metaphor, and thus its epistemic risk.

Epistemic risk involves epistemic distance and epistemic degrees of freedom. Epistemic distance is the degree of dissimilarity between the target domain and the source domain. For example, consider the metaphor kainic acid injections in the GPi are to improvement as pallidotomy is to improvement. Although, to the knowledge of this author, kainic acid lesions of the GPi have not been studied for its ability to improve movement disorders improved by pallidotomy, the hypothesis that kainic acid injections into the GPi may improve movement disorders is not unreasonable and suggests that experimental vindication may be worthwhile. The epistemic distance between pallidotomy, which destroys all the neuronal elements in the target, including neuronal cell bodies, as well as axons in passage, is not that different from kainic acid lesions that destroy the neuronal cell bodies, sparing (relatively) axons in passage.

Epistemic degrees of freedom relate to how many modifications would be required to increase the similarity between the target and the source domain. Consider the metaphor DBS is to improvement as pallidotomy is to improvement. The metaphor seeks to ''equate'' DBS to pallidotomy. Pallidotomy results in the wholesale destruction of neuronal elements at the site. DBS involves applying electrical pulses to the site. The question is how many twists and turns of logic would be required to make wholesale destruction equivalent to applying electrical pulses? This is very difficult because virtually every other experience of applying electrical stimulation has been to excite rather than suppress neuronal activity (Montgomery and Baker, 2000). Even studies that demonstrate a reduction of action potentials back propagated into the soma likely are due to the activation of presynaptic terminals that release neurotransmitters, which result in hyperpolarization of the postsynaptic membrane. Thus it is not easy to translate electrical stimulation to inhibition of neuronal activity as the link to pallidotomy would suggest. The epistemic degrees of freedom would have to be very high, thereby resulting in a high degree of epistemic risk when using the metaphor. That high epistemic risk has been borne out by observations that DBS does not result directly in suppressing neuronal activity at the neuronal membrane. DBS can generate action potentials at the initiating segment even if activation of presynaptic terminals result in hyperpolarization of the soma. Certainly, the direct physiological effects of the changes in the electrical fields involve a variety of voltage gated ionic conductance changes, such as sodium and calcium channels, as well as affecting NMDA receptors (extensively discussed in Montgomery, 2016).

# EPISTEMIC RISK AND ARISTOTLE'S PRINCIPLE OF CAUSATIONAL SYNONYM

Epistemic risk, based on epistemic distance and degrees of freedom, is a refinement of Aristotle's Principle of Causational Synonymy. This principle holds that whatever mechanisms are contained within a cause must be the same as the mechanisms contained in the effect. For example, consider how the movement of a hand in a pool of water can cause the water to move. The hand is solid and the water a liquid. It may not seem that there is any causational synonymy between the hand and the water. However, the electrons in the orbits around the atoms of the surface of the hand repel the electrons in the orbits around the molecules of water. Thus, there is clear synonymy in terms of the repulsive forces between electrons.

If one accepts that the Principle of Causational Synonymy applies to causal models of pathophysiology of movement disorders and the physiology of the basal ganglia–thalamic–cortical system, then current popular models fail. The GPi Rate and the beta-oscillations theories hold that there is a gate-like mechanism and that the pathophysiology of Parkinson's disease is due to the gate being closed excessively. This notion is clear and direct in the GPi Rate theory, where the overactivity of neurons of the GPi shut down activity in Vop and consequently in the motor cortex. The excessive beta-oscillatory activity is thought to prevent activations in the remainder of the basal ganglia–thalamic–cortical system to produce movement. Thus, the cause is some steady-state manifest by excess GPi neuronal activities or beta-oscillations.

The effect is not at all a steady state. Parkinsonism is not the relative absence of movements, automatic (habitual) or otherwise. Rather, the movements themselves are abnormal. The normal recruitment order of motor units, with small units activated first, followed by progressively larger motor units, is disrupted in Parkinson's disease. The normal triphasic pattern of electromyographic activities associate with rapid ballistic movements is abnormal. The normal reciprocal activities between agonist and antagonist are disrupted. Higher level synergies of muscular activities over multiple joints with complex movements are abnormal (for a review, see Montgomery, 2013). Thus, very little synonymy exists between the mechanisms of pathophysiology of the basal ganglia attributable to these prevailing theories and the mechanisms by which motor unit orchestration is disrupted. The dynamics of the theories are one dimensional and push–pull. The actual dynamics represented by motor unit activities are far more complex.

The great problem for those attempting to explicitly model activities within the basal ganglia lies in the dynamics that the models are intended to capture. If these models are based on currently popular theories, then the models are going to be one-dimensional push–pull dynamics. A great number of such models have been offered typically under the notion of Actor–Critic models.

# PRINCIPLE OF INFORMATIONAL SYNONYMY

The Principle of Causational Synonymy can be extended to the Principle of Information Synonymy. One can define information as nonrandom changes in states, either in parallel or in sequence. The patterns of motor unit recruitment in normal movement are not random and therefore contain information. Further, information contained in the pattern of motor unit recruitment and de-recruitment must be contained in the information provided to the lower motor neurons, although the information may be altered based on the properties of the lower motor neuron.

The classical notion is that the motor cortex—the predominant source of information to the lower motor neurons, particularly those that project to the distal musculature—specifies the time course and the magnitudes of the forces to be generated. The biophysical properties of the lower motor neurons then convert that information into the orchestration of motor unit recruitment according to the Henneman Size Principle. However, studies have demonstrated that Parkinson's disease disrupts the orchestration of motor unit recruitment, violating the Henneman Size Principle, which is normalized with therapeutic DBS (Huang et al., 2012). Thus, any model of the function of the basal ganglia–thalami–cortical system must account for the orchestration of motor unit recruitment.

The GPi Rate, the beta-oscillator, and the increased synchronization theories do not account for these dynamics and hence cannot be considered adequate. The information encoded in the basal ganglia–thalamic–cortical system, perhaps with other descending inputs to the lower motor neurons, must be at least equal to the information contained in the orchestration of motor unit recruitment and derecruitment. It cannot be the case that the lower motor neurons themselves create additional information, as this would be a violation of the Second Law of Thermodynamics as applied to information. The Second Law of Thermodynamics holds that in any closed system, entropy (considered the converse of information) cannot decrease. This means that in any closed system, net information cannot increase. Information in the activities of the lower motor neuron that drives motor unit activity cannot be generated from the lower motor neuron independently of its inputs, such as from the basal ganglia–thalamic–cortical system. What can occur is either a loss of information, perhaps manifesting as disease, or a conversion of information from one form to another that preserves the overall information content. Onedimensional push–pull dynamics, such as underlie the GPi Rate, beta-oscillator, and hypersynchronization theories, are just insufficient.

# MODELS AND METAPHORS NEEDED THAT APPROXIMATE THE DYNAMICS AND INFORMATION OF THE SYSTEMS MODELED

Any theory, model, or metaphor whose fundamental dynamics are one-dimensional push–pull will be inadequate. Certainly, such simple models are more tractable and one might argue that is a place to start. The program is to start with these simple one-dimensional systems and to increase the complexity progressively. Thus, the models can extrapolate from the simpler precursors in an incremental manner. Unfortunately, this will not work and holding to this incrementalist agenda is only likely to delay significant breakthroughs. Experience with other physical systems demonstrates that at some point in increasingly complex models, the quantitative evolution becomes a qualitative revolution in which the prior inferences are unlikely to hold.

Consider Newton's laws of motion and gravitation. These apply well to single bodies, such as an object in motion not subject to external forces. They apply equally well to two body problems, such as the moon orbiting the earth. However, these laws fail in their ability to predict the three body problem precisely, such as the orbit of the moon about the earth and both orbiting around the sun. Rather, this three body problem demonstrates Complexity, as first suggested by Henri Poincare. The fundamental property of Complex Systems is their unpredictability. At some point, increasing complexity incrementally likely will result in a qualitative change where inferences from preceding systems that are just an iota less complex fail to predict and thus explain the behavior of the complex system.

The facts just described mean that a reductionist approach, in empirical as well as in theoretical science—which includes modeling—is profoundly limited. Rather, in Complex Systems, such as would describe the pathophysiology and physiology of the basal ganglia–thalamic–cortical system, one needs to begin with the complex system. Modeling must look to examples of Complex Systems for the metaphors that will inform future modeling endeavors, which will be critical to any subsequent successful empiric understanding. There are beginnings, such as models based on networks of loosely coupled polysynaptic re-entrant nonlinear discrete oscillators (Montgomery, 2004, 2007). While it is far too soon to know whether this work will prove to explicate the complex dynamics underlying the pathophysiology and physiology of the basal ganglia–thalamic–cortical system, at the very least it represents a radical departure from the one-dimensional push–pull dynamics of popular conceptions.

Unfortunately, many recent models continue to view the basal ganglia–thalamic–cortical system as a sequential processing system rather than in parallel (Schroll and Hamker, 2013), which likely is an error (Montgomery and Buchholz, 1991). It is interesting that in their review, Schroll and Hamker (2013) describe both open loop, perhaps analogous to sequential processing, and closed loop, which would not be analogous to sequential processing given the dynamics of information transfer within the basal ganglia–thalamic–cortical system. However, they and others fail to appreciate or explore the implications for information processing within the basal ganglia–thalamic–cortical system in the context of behavior.

As shown in **Figure 1**, effects of a DBS pulse in the vicinity of the STN causes responses with very short and highly consistent latencies in the cortex consistent with antidromic activation of cortical projections to the STN. There is a response in putamen neurons at approximately 2.5–3 ms. This likely represents orthodromic activation of putamen neurons, and one source would be collateral branches of antidromically activated cortico–subthamic nucleus axons that project to the putamen. Similarly, there are responses in GPi and externa at 4 and 3 ms, respectively, which are perhaps monosynaptic activations from stimulated STN neurons.

The key point is that information transfer between nodes within the basal ganglia–thalamic–cortical system is very fast, on the order of 3–4 ms. If a bit of information is generated in the motor cortex, it would activate putamen neurons 3 ms later and STN neurons at 3 ms. GPi neurons would be activated at 7 ms, which then would produce posthyperpolarization rebound in neurons of the Vop at 10.5 ms and then back to the cortex at 14 ms. Next consider a movement that is executed over 1 s. This means that the motor cortex can drive the remainder of the nodes 71 times during the course of the movement. It also means that any structure within the basal ganglia–thalamic–cortical system can drive the motor cortex 71 times during the behavior. Further, the motor cortex, for example, receiving information through what is called the indirect route, could send information to what is called the direct route or what is called the hyperdirect route 71 times during the course of a 1-s behavior. Thus, the time course of information percolating though the basal ganglia–thalamic–cortical system relative to the time course of the behavior, called the duty cycle, is very small. The analogous situation is the fact that each pixel on a computer screen is ''painted'' sequentially but is so fast relative to human perception that events appear simultaneously on the screen. Clearly, given these dynamics, does it make sense to talk or model the basal ganglia–thalamic–cortical system as separate and discrete pathways operating as open loops in a sequential manner?

# MODELING TO NORMAL FROM ABNORMAL AND TELEOLOGICAL THINKING

The problematic nature of inferring normal function from alternations in the normal subject was well known to the ancient Greeks and a substantial reason for the reticence toward vivisection. Research involving lesions in animals is a study of abnormal animals. One cannot assume that inferences from lesioned animals or disordered humans will translate easily into an understanding of normal function. The British neurologist Francis M. R. Walsh in the early 1900s likened the situation to the circumstance of gear teeth in the differential gear of an automobile breaking and causing a clunking sound. It would be an error in reasoning to then think that the purpose of the differential gear is to prevent clunking sounds.

It would be a similar misreasoning to infer that because disorders such as a stroke involving the STN result in involuntary movements that the function of the STN, specifically, and the basal ganglia–thalamic–cortical system is to prevent involuntary movements. There are alternative explanations (see Montgomery and Baker, 2000). However, just such a presumption is made in many of the so-called Actor–Critic models of the basal ganglia–thalamic–cortical system. These Actor–Critic models also presumed the highly improbable theory proposed by Mink and Thach (1993), whose only support appears to be lifting the center-surround antagonist physiology of the

visual system. A critical review is beyond the scope of this article, but it is suffice to point out that cross-correlation studies of neurons recorded within various structures of the basal ganglia–thalamic–cortical system fail to demonstrate the type of negative correlation that would be reasonably expected according to the model proposed by Mink and Thach (1993).

Teleological thinking and its attendant problems are epidemic in models of the basal ganglia–thalamic–cortical system, as can be seen in the review of computational models by Schroll and Hamker (2013). The method is to begin with the putative purposes of the basal ganglia–thalamic–cortical system and then to demonstrate that the model behaves in a manner consistent with the putative purposes. With all due respect to Aristotle, the champion of such teleological thinking, and his subsequent conceptual heirs, this reasoning is to put ''the cart before the horse.'' The problems that ensue have been alluded to previously.

# THE MYTH THAT OBSERVATIONS OR DATA SPEAK FOR ITSELF AND THE LOGIC OF MODELING

Unlike physics originally and chemistry historically later, biology has not fully embraced formal modeling, particularly in its quantitative or mathematical sense. There may be many factors, such as it is difficult to explicitly describe complex biological phenomena in such a way that the mathematics give some intuitive sense of the underlying reality, particularly in a mechanistic sense. Also, there may be a presumption that mathematical explication is unnecessary, as the data ''speak for itself.'' Upon careful epistemological analysis, such a presumption is a myth, but further discussion of the mythological nature is beyond the scope of this article. However, if data did ''speak for itself, '' there would be no need of theory. As discussed previously, there clearly is a need for theory.

In the case of popular current theories of the pathophysiology and physiology of the basal ganglia–thalamic–cortical system, it may be that the simple one-dimensional push–pull dynamics appear not in need of mathematical explication. This does not mean that mathematical modeling has not been done for the basal ganglia–thalamic–cortical system. Rather, the mathematical modeling generally is a demonstration or proof of concept. These models demonstrate that when organized in the right way with the appropriate initial conditions, the behavior of the model will be analogous to what the pre-existing theories predict. This reasoning suffers from a conjunction of two fallacies. First is the Fallacy of Confirming the Consequence, which in this case is constructed as if **model A of parkinsonism** implies **increased neuronal activity in the GPi and increased neuronal activity in the GPi** is found, then **model A is true of parkinsonism**. The same outcome attends having **b** = **increased beta oscillation** or **b** = **increased synchronization**. The discussions provided previously prove these arguments false. Note that the goal of modeling is not necessarily to demonstrate increased GPi neuronal activity, betaoscillations, or increased synchrony. This would be, at the very least, an empty exercise and, at the very worst, sophomoric and misleading. The argument could be recast as if and only if model **A** implies **increased neuronal activity in GPi** and **increased neuronal activity in GPi** is true then model **A** is true. Note the claim relating to model **A** to Parkinsonism has been dropped. This model, if demonstrated empirically (in vitro, in vivo, or in silico), would be interesting and informative. The next important question is what is it about model **A** that it only can result in increased activity in the GPi?

A major problem with the very large majority of modeling of the basal ganglia–thalamic–cortical system has been models presuming a kind of if and only if mentality in the context of one-dimensional push–pull and sequential dynamics. Increased neuronal activity in the GPi, necessarily and only, results in decreased neuronal activity in Vop. Yet, this is not true. To be sure, there is an initial reduction in Vop neuronal activity with DBS in the vicinity of the GPi activating GPi efferent axons, but in the majority of cases, there is posthyperpolarization rebound excitation. The posthyperpolarization rebound results in a net increase in neuronal activity for many neurons. Added to this is a further increase in neuronal activity likely from positive feedback from activated cortical neurons projecting back to Vop neurons (Montgomery, 2006).

Ignoring or ignorance of these complex additional dynamics risks the Fallacy of Limited Alternatives. This fallacy is of the form if **a** inclusive-or **b** inclusive-or **c** is true and **b** and **c** are found false then **a** is true. (In Probability theory this fallacy is known as the Gambler's Fallacy.) Note the use of the inclusiveor, which allows any or all of the entities be true so long as one is true. Thus, the truth or falsehood of **a** is independent of **b** and independent of **c** and **b** is independent of **c**. Also, note that with **b** and **c** removed for being false, the result is the Fallacy of Confirming the Consequence. Failure to consider that the actions of GPi neurons on Vop neurons are more akin to delayed excitation rather than inhibition would result in the Fallacy of Limited Alternatives.

The Fallacy of Limited Alternatives is a tremendous challenge to mathematical and computational modeling. The mathematical and computational tools are extremely powerful. For example, genetic and neural network computing do not even require any a priori knowledge of how the computational solution should be arrived at. Even if one were to constrain the degrees of freedom of a proposed computational modeling by prescribing that the neural elements of the computational process obey the constraints found in biological neurons, the process is still incredibly powerful and can produce a great variety of computational processes that arrive at the same solution (Marder and Taylor, 2011).

The great, perhaps greatest, value of modeling is the demonstration that the model has to be the one to predict the behavior of interest, not that it could predict the behavior. The reason is that demonstrating that a model has to be the one that predicts the behavior greatly increases the likelihood that the model explicates the behavior. Certainly, in practice, it may not be possible to say that one and only one model can predict the behavior. However, the effort should be to minimize the number of candidate models and then search for commonalities and differences in the dynamics of the model to find those mechanisms of the model that likely are explanatory, following from Mill (1843) Joint Method of Agreement and Difference for Induction.

It is worth considering why modelers have been so susceptible to the Fallacy of Limited Alternatives. At least one answer is that the susceptibility is in the language. The vernacular has terms such as excitation and inhibition, which are incomplete and consequently misleading. Rather, depolarization or hyperpolarization should be used instead. Hyperpolarization at least connotes the possibility of posthyperpolarization rebound excitation, and depolarization connotes the possibility of a depolarization blockade. Further, the one-dimensional push–pull mental predisposition is re-enforced by conflating neurotransmitters with actual electrophysiological dynamics (Valenstein, 2005). Neurotransmitters are the messenger, not the message. To hold that the neurotransmitter is the determinant of neural functions, and thus neural functions can be inferred from neurotransmitters, is like saying the operations of a computer can be inferred directly from the properties of an electron. Note that this is not to say that electrons are not the fundamental element that underlies electronic computers, but it is the same electrons that underlie televisions and smartphones. Thus, the properties of an electron are necessary but not sufficient to understand the operations of a computer. Most neurological and psychiatric disorders are disorders of information. Neurotransmitters (and gap or electric junctions) are necessary but insufficient for information or misinformation. Holding that neural behaviors can be inferred from the actions of specific neurotransmitters would be an example of the Mereological Fallacy (see Montgomery, 2012).

Similarly, talk of the pathophysiology and physiology of the basal ganglia–thalamic–cortical system, and thus by extension, neural models, is couched in terms of neurons. However, use of the term neuron carries the connotation that the neuron is the fundamental unit, anatomically, and by extension physiologically. This similarly represents a Mereological Fallacy.

# REDUCTIONISM AND THE NEED FOR NEW METAPHORS

Reductionism often is a choice when one wants an understanding of complex behaviors that goes beyond purely descriptive (sometimes referred to as the ''stamp collecting'' approach to science). The choice to go beyond the merely descriptive necessitates that the set of behaviors to be understood (the explanandum) must have a set of explanations (explanans) with fewer elements. If the required elements of the set of explanans equaled that of the set of explanandums, the consequence would be purely descriptive. Thus, reductionism is the necessary consequence. Further, in science the relationship between the explanans and the explanandum typically is causal in nature.

Mathematical and computational modeling is no different. Even when employing methods such as genetic and neural network computing—methods that are not necessarily reductive—once the computational solution is determined, it is ''dissected'' to understand how its ''components, '' sometimes referred to as motifs in network or systems theory (Alon, 2007), represent an economical set of causal mechanisms.

Reductionism is relatively easy—all one needs, metaphorically speaking, is a bigger hammer, sharper knife, or more powerful computer. The value of reductionism lies not in the reduction, but in the reconstruction from the economical set of explanans; a notion often forgotten or never appreciated by scientists. The goal is to understand the behavior; an understanding the reduced preparation, be it a tissue slice or a mathematical or computational model, is the means, not the ends, at least in the context of any understanding of the basal ganglia–thalamic–cortical system with the purpose of effecting a benefit in the clinic. Even aside from utilitarian considerations, studying a ''reduced'' preparation on its own is not synonymous with the notion of reductionism.

A successful reductionism is when the explanandum can be reconstructed from the explanans. One could reduce, methodologically, the basal ganglia–thalamic–cortical system to a system of neurotransmitter fluxes, but it is highly unlikely that one could explicitly reconstruct the normal and abnormal orchestrations of motor unit behaviors in health and disease. The failure of any reconstruction, defined by deficiency or error, indicates that information was lost. If the method by which information was lost is irreversible, then by the Second Law of Thermodynamics as applied to information, the reduction will never allow a reconstruction.

In the case of the basal ganglia–thalamic–cortical system, one such reduction is to view the system as a sequential hierarchical system. This is implicit in all descriptions of the system where the putamen is viewed as the input and the GPi and substantia nigra pars reticulata are viewed as the output. However, an alternative conception is that the basal ganglia–thalamic–cortical system is a set of loosely coupled polysynaptic re-entrant nonlinear discrete oscillators for which there is considerable supporting evidence (Montgomery, 2016). Assuming that the theory is true, then reducing the basal ganglia–thalamic–cortical system to a sequential hierarchical system will eliminate the important re-entry oscillator dynamics. Consequently, any theory based on a sequential hierarchical organization will be unable to ever reconstruct the behavior due to the basal ganglia–thalamic–cortical system.

The critical question then becomes what is the nature of a reconstruction? It may be a matter in practice or in principle that a full reconstruction is impossible; more on this subsequently. However, one might be able to appeal to the possibility or approximation of such a reconstruction. For example, it may just be impossible to reconstruct, in a fully explicit way, the behavior of any individual given the biological variability and complexity of the nervous system. However, one can make ''approximations'' by appealing to the average of a set of explanans and set of explanandums, with the presumption or assumption that the average represents the Central Tendency; the latter itself is a problematic notion—witness the distinction among mean, median, and mode as they vive for the claim of the Central Tendency.

Perhaps the Reductionism counterpart to the average is the asymptote. In this case, reconstruction using the economical set of explanans converges onto the explanandum. In other words, the limit of the reconstruction as the effort and sophistication approximates perfection becomes the reconstruction. This is analogous to limit theory used in differential calculus where some value **y** becomes **dx/dt** as **dt** approaches zero, for example, where **y** is the instantaneous velocity, **dx** is the change in distance, and **dt** is the change in time.

Another example is Galileo's demonstration of inertia using inclined planes. A ball rolling down an inclined plane will roll up another incline plane to the same height from which it descended. As the angle of the second incline is reduced, the ball has to roll further along the incline to reach the same height. As the second incline is continually reduced, the ball rolls further. The presumption is that, independent of friction, at a zero angle on the second incline, the ball would roll forever; hence, Galileo's law of inertia. Yet, clearly it would be impossible to demonstrate this fact empirically. Consequently, Galileo's argument forever would be a metaphor, but nonetheless a very convincing one. This argument is a Process Metaphor. In this case the target domain, that being Galileo's law of inertia, gets its credibility from the source domain, the experiment of successively reducing the angle of the second inclined plane.

The Process Metaphor is endemic to modeling, particularly mathematical and computational. Typically, one begins with the neuron as a model, its dynamics described by Hodgkin and Huxley constructions of voltage and ligand-gated ionic conductance channels or other abstractions that are more computationally tractable, such as the FitzHugh–Nagumo model or simpler integrate-fire neuron models. One then progressively increases the complexity of the models with the expectation that the model outputs will converge on a dynamic or mechanism that can be thought representative of the underlying biology. As it is unlikely that even an infinitely complex model, defined here as the practically infinitely complex, then the credibility of the model is dependent on the manner in which the model was made more complex, that is the Process Metaphor.

# FAILURE OF REDUCTIONISM AND CHAOS AND COMPLEXITY

There are notable examples where a Reductionist account has failed to reconstruct the behavior of interest. For example, using Newton's laws of motion and gravitation, one can explicitly determine the motion of one planet about another, such as the moon about the earth, but not about the moon orbiting the earth as both orbit the sun, as described earlier. This is called the Three Body Problem, which Henri Poincare, and others, demonstrated that no analytical solution was possible, although the system could be approximated later and, in some analyses, an asymptotic solution is possible. Poincare's analysis was one of the first indications of the unique situation of Chaos.

It is important to realize that Chaos and Complexity, which came along later, are not random systems. They are determinant in that they are based explicably on specific laws and principles. For example, the formation of a snowflake is an example of a Complex System. One of the hallmarks of Complex Systems is their unpredictability. Indeed, the actual geometry, other than having six points, of any particularly large snowflake (containing very large number of water molecules) is different from any other. Thus, it would be hard to predict any particular large snowflake, although mathematical and computational modeling of the growth, as distinct from origin, is advancing (Barrett et al., 2012).

Many chaotic and complex systems do demonstrate ''structure, '' hence information. Note that the structures of snowflakes are not random. How ''structure'' arises and how it can be recognized in chaotic and complex systems is the challenge (see Strogatz, 2014). The structure in these systems may be evident in various ''attractors'' and bifurcations. Other examples of the dynamics of chaotic and complex systems include a dependence on initial conditions and bifurcations to and between metastable states.

Whether or not the basal ganglia–thalamic–cortical system is a chaotic and complex system remains to be determined. According to at least one theory, called the Systems Oscillators theory, the basal ganglia–thalamic–cortical system has all the ''ingredients'' for a chaotic and complex system, including the highly nonlinear dynamics of discrete oscillators and the vastness of the potential interactions. There is some preliminary supportive evidence (Montgomery, 2016). However, what is clear is that any reductive method or reduced theory is not going to be able to demonstrate chaos and complexity, and thus not be able to leverage the dynamics of chaotic and complex systems for an adequate theory of the pathophysiology and physiology of the basal ganglia–thalamic–cortical system. At the very least, the Chaotic and Complex Systems theory should be considered a metaphor for further research, particularly modeling, of the basal ganglia–thalamic–cortical system.

Developments in physics, mathematics and physical chemistry are advancing dramatically, as evidenced by the Chaos and Complexity theory. There are a number of other physical–mathematical concepts that, if appreciated by neuroscientists, could advance our understanding of the basal ganglia–thalamic–cortical system greatly. It always is hard to predict which nascent concept will have future impact—witness the initial tepid reception to the transistor and personal computer. Potential areas for fruitful scholarly efforts lie in discrete oscillators (not the kind that use digital signal processing to generate oscillators that approximate continuous harmonic oscillators), as these are more realistic in biological neural oscillators (Montgomery, 2016). Another area that could prove illuminating is nonequilibrium steady states, particularly as how they may underlie metastable states associated with chaotic and complex systems.

# AN ALTERNATIVE MODEL FOR THE PURPOSE OF DEMONSTRATING CONTRASTS WITH CURRENT MODELS

The Systems Oscillators theory (Montgomery, 2016) begins with the same loop-like architecture that underlies most anatomical concepts of the basal ganglia–thalamic–cortical system but closes them into reentrant oscillators. In doing so, it becomes clear that the basal ganglia–thalamic–cortical system can be seen as a network of loosely coupled reentrant oscillators. Each oscillator contains nodes defined as sets of neurons in the various anatomical structures that make up the basal ganglia–thalamic–cortical system. Neurons within a node are defined by their shared inputs and projections to the same set of neurons in the subsequent node. Different oscillators can share the same nodes thus resulting in a network of coupled oscillators.

The next conceptual difference is that the Systems Oscillators theory addresses the dynamics of neuronal activities at relevant time scales. For example, based on a 3.5 ms time for a ''bit of information'', such as an action potential, to pass from one node to the next, the time to traverse a four node loop, such as motor cortex–putamen–GPi–Vop, is in the order of 14 ms. This corresponds to a reentrant oscillator frequency of 71 Hz. This means that the motor cortex can influence the GPi 71 times during the course of a 1 s behavior. Similarly, the GPi can influence the motor cortex 71 times. Thus, does it make any sense to call the putamen an input stage or the GPi as an output stage? Further, information processing does not occur in a manner restricted to a given anatomical structure. Thus, does it make any sense to ascribe a ''behavioral function'' to any specific anatomical structure, such as the GPi? Does it make any sense to construct a model with structures identified as input or output stages or with anatomical structure assigned a specific behavioral function? The dynamics also demonstrate that a bit of information received by the motor cortex that came through the globus pallidus externa (GPe) could affect the putamen or the STN and then the GPi many times during the course of a behavior. Does it make sense to talk about the physiology unique to the direct, indirect or hyperdirect pathways?

Finally, while most of the neurotransmitters involved in the anatomical connections within the basal ganglia–thalamic–cortical system utilize GABA and produce hyperpolarizations, research has shown that such hyperpolarizations are followed by rebound excitation. In some cases, the rebound excitation results in a net increase in action potential generation, not a decrease. Therefore, does it make sense to talk about inhibitory actions within the basal ganglia–thalamic–cortical system? It makes little sense in modeling the basal ganglia–thalamic–cortical system in terms of excitation and inhibition. The dynamics are far more complex and because of the oscillators, dynamics more likely reflect positive and negative stochastic resonance, stochastic coherence, phase changes, phase entrainment and ''noisy'' synchronization. Perhaps modeling of the basal ganglia–thalamic–cortical system might benefit from incorporating such dynamics.

Evidence supporting these conceptualization are addressed and more extensive discussion is provided elsewhere (Montgomery, 2016).

# SUMMARY AND FUTURE DIRECTIONS

Modeling of the basal ganglia–thalamic–cortical system is fundamental and critical for many epistemic reasons. Modeling, considered as a procedural form of theory, helps to understand when direct knowledge is not possible because of incomplete direct knowledge. Ideally, it is the models that generate strong hypotheses that, once vindicated by experimentation, expands knowledge. Indeed, it can be argued reasonably that models are the only route to any understanding.

Modeling for the intent of generating hypotheses necessarily requires the judicious use of logical fallacies. Injudicious use not only leads to flawed models but also inhibits development of new models. Further, models and other forms of theory to not arise spontaneously but are legacies of prior models and theories and risk inheriting many of the presuppositions and misconceptions. Among the misconceptions are dynamics that are one-dimensional push-pull and conflating the hyperand depolarizing effects of neurotransmitters with inhibition and excitation respectively. Also, the Mereological fallacy of attributing to the part, the function of the whole is rampant as is reductionism.

The primary but unappreciated value of reductionism is not the ability to reduce phenomena to simpler forms, but rather in the ability to reconstruct the phenomena from the reduced forms. However, there is a fundamental limit to the ability to reconstruct based on Chaos and Complexity theory. At some point incremental increases in complexity go from a quantitative

#### REFERENCES


to a qualitative change at which point the inferences from the just slightly simpler reconstruction is no longer applicable. Thus, the ultimate goal is to have a model whose complexity is at a level of complexity on par with the phenomena the model is intended to explicate. It is not clear that a reductionist approach will be successful.

If one wants to change the future, in this case to a future of greater knowledge and understanding, then one must know the present. To know the present, one must know the past. The future of modeling will not likely succeed if it inherits misconceptions. Modelers of the future will need to rid themselves of the misconceptions and look to new metaphors by which to fashion future hypotheses and theories. Fortunately, rapid advances in physics and mathematics can help, such as Chaos, Complexity, Percolation theory, nonequilibrium steady states and networks of discrete nonlinear oscillators.

# AUTHOR CONTRIBUTIONS

The author was responsible for all aspects of the article.

# FUNDING

This effort was fully funded by the Greenville Neuromodulation Center, Greenville, PA, USA.


Orwell, G. (1949). Nineteen Eighty-Four [1984]. London: Secker and Warburg.

Percheron, G., Filion, M., Tremblay, L., Fenelon, G., Francois, C., and Yelnik, J. (1993). The role of the medial pallidum in the pathophysiology of akinesia in primates. Adv. Neurol. 60, 84–87.

Santayana, G. (1905). The Life of Reason: Five Volumes in One. Echo Library.


**Conflict of Interest Statement**: The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

The reviewer SC and handling Editor declared their shared affiliation, and the handling Editor states that the process nevertheless met the standards of a fair and objective review.

Copyright © 2016 Montgomery. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution and reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# A Neurocomputational Model of the Effect of Cognitive Load on Freezing of Gait in Parkinson's Disease

Vignesh Muralidharan<sup>1</sup> , Pragathi P. Balasubramani <sup>1</sup> , V. Srinivasa Chakravarthy <sup>1</sup> \*, Moran Gilat <sup>2</sup> , Simon J. G. Lewis <sup>2</sup> and Ahmed A. Moustafa<sup>3</sup>

<sup>1</sup> Department of Biotechnology, Indian Institute of Technology, Chennai, India, <sup>2</sup> Parkinson's Disease Research Clinic, Brain and Mind Research Institute, University of Sydney, Sydney, NSW, Australia, <sup>3</sup> MARCS Institute for Brain and Behaviour and School of Social Sciences and Psychology, Western Sydney University, Sydney, NSW, Australia

Experimental data show that perceptual cues can either exacerbate or ameliorate freezing of gait (FOG) in Parkinson's Disease (PD). For example, simple visual stimuli like stripes on the floor can alleviate freezing whereas complex stimuli like narrow doorways can trigger it. We present a computational model of the cognitive and motor cortico-basal ganglia loops that explains the effects of sensory and cognitive processes on FOG. The model simulates strong causative factors of FOG including decision conflict (a disagreement of various sensory stimuli in their association with a response) and cognitive load (complexity of coupling a stimulus with downstream mechanisms that control gait execution). Specifically, the model simulates gait of PD patients (freezers and non-freezers) as they navigate a series of doorways while simultaneously responding to several Stroop word cues in a virtual reality setup. The model is based on an actor-critic architecture of Reinforcement Learning involving Utility-based decision making, where Utility is a weighted sum of Value and Risk functions. The model accounts for the following experimental data: (a) the increased foot-step latency seen in relation to high conflict cues, (b) the high number of motor arrests seen in PD freezers when faced with a complex cue compared to the simple cue, and (c) the effect of dopamine medication on these motor arrests. The freezing behavior arises as a result of addition of task parameters (doorways and cues) and not due to inherent differences in the subject group. The model predicts a differential role of risk sensitivity in PD freezers and non-freezers in the cognitive and motor loops. Additionally this first-of-its-kind model provides a plausible framework for understanding the influence of cognition on automatic motor actions in controls and Parkinson's Disease.

Keywords: Parkinson's disease, freezing of gait, basal ganglia, cognitive, motor, utility, conflict, cognitive load

# INTRODUCTION

Deterioration of gait in Parkinson's Disease (PD) is of major concern as it severely affects the quality of life of patients. Some characteristics of gait disturbance include increased double support time, reduced stride length and velocity (Hausdorff et al., 1998; Morris et al., 1998). In addition to these features, freezing of gait (FOG) is a paroxysmal phenomenon where patients feel they are glued to the ground despite the desire to walk (Giladi et al., 2001; Nutt et al., 2011). A diverse range of

#### Edited by:

Marcelo Merello, Fundación para la Lucha contra las Enfermedades Neurológicas de la Infancia, Argentina

#### Reviewed by:

Summer Sheremata, Florida Atlantic University, USA Zhen Yuan, University of Macau, China Wei Zhang, City University of New York, USA

> \*Correspondence: V. Srinivasa Chakravarthy schakra@iitm.ac.in

Received: 26 March 2016 Accepted: 08 December 2016 Published: 09 January 2017

#### Citation:

Muralidharan V, Balasubramani PP, Chakravarthy VS, Gilat M, Lewis SJG and Moustafa AA (2017) A Neurocomputational Model of the Effect of Cognitive Load on Freezing of Gait in Parkinson's Disease. Front. Hum. Neurosci. 10:649. doi: 10.3389/fnhum.2016.00649 environmental contexts can trigger FOG such as passing through narrow and confined spaces like doorways (Almeida and Lebold, 2010; Cowie et al., 2010; Shine et al., 2013a), turning and increased cognitive processing such as dual tasking (Schaafsma et al., 2003; Spildooren et al., 2010). The specific contribution of set-shifting, attention, visuo-spatial processing, sensory integration and emotions like anxiety have also been found to trigger FOG in PD (Lewis and Barker, 2009; Nutt et al., 2011; Martens et al., 2014).

Multiple neural networks are involved in gait processing and freezing including the sensory and motor cortices along with the association and prefrontal cortices, anterior cingulate cortex, basal ganglia and the brainstem (Shine et al., 2013b). Gait impairment in PD indicates a role for basal ganglia (BG) in these processes (Hausdorff et al., 1998; Morris et al., 1998). The cortico-basal ganglia system is organized as parallel loops associated with specific functional domains (Alexander et al., 1986; Parent and Hazrati, 1995; Graybiel, 1998). Competitive interactions among the cortico-basal-ganglia loops are thought to be a major factor for triggering freezing (Lewis and Barker, 2009). This is due to the impaired balance in demands to resources, that is, increased demands from cognitive, motor and limbic loops under depleted dopamine resources leading to the inhibition of brainstem locomotor systems (Lewis and Barker, 2009; Shine et al., 2013d). Several studies have investigated and reported deficits related to the effect of cognitive processes on gait in PD subjects especially freezers. The studies investigate the ability of these subjects to resolve conflict in the cognitive aspect of the task or in its interaction with a motor activity. Some examples include tasks such as the attention network task (ANT), the virtual reality gait task (Shine et al., 2013a), "timed up and go" task (TUG) (Weiss et al., 2010; Herman et al., 2011), object avoidance (Snijders et al., 2010; Pieruccini-Faria et al., 2014), and dual tasking (Yogev et al., 2005; Springer et al., 2006).

Our previous work (Muralidharan et al., 2013) on modeling PD gait explored the possibility of neuromodulator deficiency in PD freezers, particularly serotonin and norepinephrine. The model could explain deceleration and gait (step/stride length) changes observed in experiments involving walking through doorways with variable widths (Almeida and Lebold, 2010; Cowie et al., 2010). However, the role of cognitive factors was not included in our previous gait model. Furthermore, the prior model was only a Value based model, and did not consider the possible contributions of Risk (expected uncertainty), a quantity representing ambiguity associated with decision-making and is significant for understanding PD (Balasubramani et al., 2015). The current paper models gait performance in healthy controls and PD patients using Utility-based decision making that combines Value and Risk estimates for generating the decision variable toward executing behavior. Risk (or expected uncertainty) estimates capture the variance in reward outcome observed due to decision making (Balasubramani et al., 2014). Moreover, the sensitivity of the subjects to risk has previously been hypothesized to be a correlate of serotonin (5HT) function in the striatum. Based on the hypothesis that striatal serotonin levels code for risk-sensitivity, computational models have been able to unify several existing theories of serotonin function into a single theory (Balasubramani et al., 2014, 2015). In the experimental tasks modeled here, a word cue is associated with a specific motor action (walk/stop) resulting in a specific outcome (reward/punishment). Increased inconsistency in the relationship between cue and actions, results in greater reward variance or greater risk. Our model is built on the lines that this uncertainty in outcomes during the presentation of complex cue (also reflected as cognitive load as it demands the recruitment of more cognitive resources to achieve optimal behavior) facilitates the generation of a risk estimate. Along with the expected reward (or value) estimates, risk combines to generate the utility measure of decision process. The objective of the study is then to understand the interaction between cognitive and motor aspects in gait control, and analyze roles of conflict and load in the task design on simulated subjects (agent) as they approach a doorway. We also want to see whether the risk sensitivity of the subjects has any role in explaining the freezing behavior of certain PD patients and the implications of these measures on the therapeutic strategies. The results of the simulation are compared to behavioral results from PD patients (Matar et al., 2013; Shine et al., 2013d).

## METHODS

#### Computational Modeling of the Virtual Reality Gait Task Setup

The model simulates performance in a series of virtual reality experiments conducted on controls, PD non-freezers, and PD freezers (Matar et al., 2013; Shine et al., 2013d) to study the effects of conflict and cognitive load in PD patients. These behavioral experiments used a modified version of the Stroop task (Treisman and Fearnley, 1969) where there is an association of a color-word stimulus to a specific motor action (i.e., to walk or to stop) while subjects navigate a series of doorways. These virtual reality (VR) tasks assessed gait performance of the subjects, which require effective interaction between the corticobasal ganglia circuits. The task setup for the model is inspired by two experiments (Matar et al., 2013; Shine et al., 2013d).

#### Patient Description

In the Matar et al. (2013) experiement there were 18 healthy controls, 37 PD patients classified as non-freezers and 36 PD patients classified as freezers. The freezers were identified reliably using the item 3 of the FOG questionnaire (FOG-Q3—"Do you feel as if your feet are glued to the floor while walking, making a turn or while trying to initiate walking?"). Patients also performed the Mini mental State Examination (MMSE) and none had dementia according to the Movement Disorders Society PD dementia criteria.

The Shine et al. (2013d) experiment considered only PD nonfreezers (n = 10) and freezers (n = 10) as part of the study. Since the study took place in both the ON and OFF conditions, clinically defined OFF condition was a minimum of 18 h without dopaminergic medication, with an average of 22.5 ± 3.1 h. Apart from the FOG questionnaire, the PD patients were also required to perform a few timed up and go trials with 180◦ left and right turns to identify patients with freezing behavior.

#### Model Setup

In the simulation, the track that the agent navigates consists of 300 doorways (each doorway appearing at a distance of 4 length units). The agent (simulated subject; circular with about 1 unit in diameter) navigates a series of doorways (wide—3 length units and narrow—2 length units), while simultaneously performing the cognitive task of responding to word cues (**Figure 1**). At any time step, the output exhibited by the agent includes performing a forward motion (dubbed as a "step"), associated with a specific latency estimated as the number of time steps required to reach a decision threshold. The experiments gave the subjects, who were seated in front of a computer monitor, a first person view of the virtual reality (VR) environment which they could interact using a set of foot pedals. Alternate pressing of the foot pedals leads to forward motion in the VR setting, simulating the experience of locomotion, while the word cues were presented at the bottom of the monitor (**Figure 1B**). The model simulates this by executing a forward motion in the virtual environment. Additionally, to indicate walking or stopping within a trial, the task utilizes a set of simple and complex cues. The simple cues include the word "WALK" (usually presented in green) that indicated the subject to walk and the word "STOP" (usually presented in

(adapted from Shine et al., 2013a).

red) that indicated stopping. Simple cues are also presented in neutral color (e.g., BLACK). The task complexity is increased by interlacing simple cues with blocks of Stroop's words (complex cues). These words could be congruent (word and color are the same) or incongruent (word and color are different). Here, we use words RED, GREEN, and BLUE and the colors red, green and blue with their combinations providing a set of 13 different cues (see **Figure 1** and **Table 1**). To represent a word stimulus, we adopt the following format "WORD (color)." The metric used for assessing freezing is inter-step length latency, defined as the time period between two consecutive alternating (left-right-left) presses of the foot pedals. Using this measure the following gait parameters are defined (Matar et al., 2013).


#### Model Architecture

The proposed cortico-basal ganglia model simulates the interaction between the motor and cognitive loops (using a "Motor Module" and "Cognitive Module"). Both the Cognitive and Motor Modules of the proposed BG model are based on the Actor-Critic architecture, each having its respective Critic and Actor. Evidences from the two modules are combined to execute the final output. These two modules build their respective evidences based on different sensory stimuli—the Motor Module based on visual appearance of the doorway, and the Cognitive Module based on the word cue. The first evidence (EI) involves the Cognitive Module identifying the salience of a word cue upon its presentation (**Figure 2**). Since a word cue does not appear at every moment, EI is taken only upon the presentation of the word cue. The second evidence (EII) involving the Motor

TABLE 1 | The list of cues used in the virtual reality paradigm and the actions associated with their appearance as used in the experiments and the model.


\*The blocks are counterbalanced so that half of the patients associate a congruent cue to walk and incongruent to stop and vice versa for the other half.

Module takes the visual appearance of the doorway as input and computes the direction of the step as well as the latency associated with it as outputs (**Figure 2**). EII is computed at every time step. The GEN (Go/Explore/Nogo) policy, which is the Actor, adopts hill-climbing over the Utility landscape to calculate the velocity of the agent. The evidences of the two modules are combined subsequently to get the step latency.

Below we describe the following modeling components: (1) Utility-based decision making, (2) Cognitive Module, (3) Motor Module, (4) computations of gait parameters, and (5) Modeling PD condition.

#### UTILITY-BASED DECISION MAKING

The Value function "Q" under a policy π, which is defined as the expected discounted sum of rewards, associated with a state, "s," and an action, "a," pair, at time, "t" is given as,

$$Q^{\pi} \text{(s,a)} = E\_{\pi} (r(t+1) + \nu \ r(t+2) + \nu^2 \ r(t+3))$$

$$+ ... | s(t) = s, a(t) = a) \tag{1}$$

where, r is a scalar reward obtained at time t, γ , is the discount factor controlling the time scale of reward prediction (Sutton and Barto, 1998). The update form of the Value function is as follows:

$$Q(t+1) = Q(t) + \eta\_Q \delta(t) \tag{2}$$

where, η<sup>Q</sup> is the learning rate and 'δ' is the temporal difference (TD) error.

$$\delta(t) = r(t) + \gamma Q(t+1) - Q(t) \tag{3}$$

or

$$
\delta(t) = r(t) - Q\left(t\right) \tag{4}
$$

for instantaneous rewards.

The Utility function formulation combines the Value function defined above with the Risk function, h, which represents reward variance (Bell, 1995; d'Acremont et al., 2009). In recent work, we showed that the Utility function formulation can be effectively used to model the interactions between dopamine and serotonin in BG (Balasubramani et al., 2014, 2015).

The Risk function is updated as follows:

$$h(t+1) = h(t) + \eta\_h \xi(t) \tag{5}$$

where, ξ (t) is the Risk prediction error given by:

$$
\xi(t) = \delta(t)^2 - h(t) \tag{6}
$$

We proposed a slightly modified form of Utility "U," expressed as a combination of the Value function and the Risk function weighted again by the function sign(Q), as follows:

$$U(t) = Q(t) - \alpha \operatorname{sign}(Q(t)) \sqrt{h(t)}\tag{7}$$

where, α that controls the risk sensitivity representing the functioning of serotonin (5HT) in the BG (Balasubramani et al., 2014). The sign() term in Equation (7) represents the non-linear risk sensitivity (Balasubramani et al., 2014).

# THE COGNITIVE MODULE

The Cognitive Module is a two-layer neural network (**Figure 3**) that receives its input as the word cues, which is a color-word pair defined in the format WORD (color). It returns as output the Utility associated with the action "walk." The word stimulus is represented as a 9-dimensional vector S cog k ). The first 5 bits correspond to the word inputs (STOP, WALK, RED, GREEN, and BLUE) and the last 4 bits correspond to the color associated with that word (red, green, blue and neutral). The respective bits are turned to 1's for a specific color-word stimulus. The input is fed into an association layer (M cog j ) which consists of 5 nodes, with sigmoidal non-linearity. The output (Q cog i ) consists of 2 nodes representing the Action Values for walking and stopping, i ε [w, s]. In accordance to the experiments, upon visualization of a cue, the agent has two actions to select from, that is, to walk or to stop.

#### Critic

The simulated agents performing the VR experiments are trained to associate the cues with actions to at least an accuracy level of 95%. The network gives the Action Values for the cue that is Q<sup>w</sup> and Q<sup>s</sup> . The weights (W cog jk and W cog ij ) of the network are randomly initialized and upon presentation of a cue (S cog k ), one of the output nodes (Q cog i ) is selected via competitive dynamics. Here S cog represents the input word vector, and Mcog denotes the hidden layer (association layer) of the neural network.

$$M\_j^{\text{og}}(t) = \lg(W\_{jk}^{\text{og}} S\_k^{\text{og}}(t)) \tag{8}$$

$$Q\_i^{\text{og}}(t) = A\_Q^{\text{og}} \mathcal{g}(\mathcal{W}\_{ij}^{\text{og}} \mathcal{M}\_j^{\text{og}}(t)) \tag{9}$$

where g(x) = 1 1+e−<sup>λ</sup> cog x and λ cog is the slope of the sigmoid function. The Qs are initially trained by selecting a node through the forced alternative choice method. A reward (r cog ) is obtained upon selecting an action (r cog = 1 for the correct action; r cog = 0 for the incorrect action). The weights for the corresponding node are updated with learning rate η cog using the following rule:

$$
\Delta W\_{ij}^{\rm cog} = \eta^{\rm cog} \delta\_i^{\rm cog}(t) M\_j^{\rm cog} \tag{10}
$$

$$
\Delta W\_{jk}^{\text{ogg}} = \eta^{\text{cog}} \delta\_j^{\text{cog}\_2}(t) S\_k^{\text{cog}} \tag{11}
$$

where the prediction error defined as, δ cog (t) = r cog (t) − Q cog i (t), is used to update the output weights (W cog ij ) and is backpropagated as δ cog2 j (t) = 6 i W cog ij g ′ (W cog jk S cog k (t))δ s i (t) for updating the weights W cog jk . The prediction error δ cog is an analog of temporal difference error correlated with dopamine signaling in the Cognitive Module (Schultz, 2010).

The training procedure is as follows:


After learning the Q-Values, the network can in parallel be used to compute the Utility for the cues using the following approach. Since there is an uncertainty in the task toward the identification of the appropriate action for a Stroop word, this uncertainty is calculated by the probability of walking (pw) for a particular cue.

$$p\_{\mathcal{W}} = \frac{Q\_{\mathcal{W}}}{Q\_{\mathcal{W}} + Q\_{\mathcal{S}}} \tag{12}$$

The Risk function (h cog ) is then estimated by the following expression.

$$h^{\text{org}}(t) = \frac{p\_{\text{w}}(1 - p\_{\text{w}})}{a\_{\text{p}}} \tag{13}$$

a<sup>p</sup> is a constant used to scale up the Risk function so that the maximum Risk is 1. The parameter a<sup>p</sup> is set to be 0.25 in all simulations. The Utility is then defined for the cognitive network as a combination of the Value (Q cog ) and the uncertainty/Risk function (h cog ). The amount of Risk taken into account for the

Utility computation is controlled by the sensitivity factor α cog similar to the manner done in (Balasubramani et al., 2014).

$$U^{\rm cog}(t) = Q^{\rm cog}(t) - \alpha^{\rm cog} \text{sign}(Q^{\rm cog}(t)) \sqrt{h^{\rm cog}(t)} \tag{14}$$

The Utility is used as a measure of the salience of a cue and is the EI of the task (**Figure 2**). It represents information regarding the decision to be taken toward a specific stimulus upon its presentation. The Utility for walking (U cog <sup>w</sup> ) is further passed on to the motor network to estimate step latency (See Supplementary Material S2: Table S1 for the list of parameter values to simulate the Cognitive Module).

#### THE MOTOR MODULE

The ability of the agent to navigate through a series of doorways based on the visual appearance of the doorway is controlled by the Motor Module. The virtual reality paradigm used several different characteristics of the doorway such as wide and narrow doorways, wide and narrow passages and also sliding doorways which open upon approaching the doorway (Matar et al., 2013; Shine et al., 2013d). For simplicity, in the model we consider only two different types of doorways, a wide and a narrow one; sliding doorways are omitted. We assume that the only property of the doorway that determines freezing or non-freezing is its width.

The agent is associated at every point on the track with a heading direction/ velocity 1Z(t) = - 1X(t) 1Y(t) (where X is the dimension along the track in the forward direction and Y is perpendicular to it (**Figure 1**) which points to the direction in which the agent is moving (or looking) at the moment. Importantly, a successful passage through a doorway yields a reward (r mot = 1) and collision with the sides leads to punishment (r mot = −1). Using such a reward scheme, the agent constructs a Value function and navigates through the virtual corridor. So although in the experiments the subjects looked straight and experienced only forward motion, in the model the agent had a 2D motion. It was necessary for the virtual agent to successfully pass through the doorway which yielded a positive reward and bump with the sides that gave it negative reward or punishment, which gave rise to a speedaccuracy tradeoff close to the doorway resulting in the agent slowing naturally as it approaches the doorway. In previous studies, this approach helped explain deceleration of PD freezers as they approached a narrow doorway (Muralidharan et al., 2013).

Since the step latency is the desired parameter it is estimated near the doorway using the following approach. A region of around 0.1 length units on either side of a doorway is considered as a significant distance to isolate the effect of the doorway on reaction times. In this region, the maximum latency exhibited by the agent is averaged across trials for the two doorway types. In order to study the effect of cues on the latency near the doorways, the agent is made to navigate through the track while simultaneously presented with different cues. Within a single trial (from the start point until encountering a doorway) the cues always appear at a distance of about 2 length units before the doorway. Unlike the Cognitive Module where the action occurs at discrete steps when a word cue is presented, the actions of the Motor Module are made every time step, since the doorway is continuously visible to the agent (**Figure 4**).

#### Cue-Visual Input

The visual information represented by the "view vector," φ, acts as the state for the Critic of the Motor Module. The agent can see around 120◦ along the width and 90◦ along the height. Both the horizontal and vertical fields of vision are split into 50 sectors each. Thus, the visual vector is a 100-dimensional binary vector [φ ε (−1 1)], where the first 50 bits code the width of the doorway visualized by the agent and the last 50 code for the height of the doorway. The heading direction vector (H) determines the direction the agent is looking, at a particular time step. The corresponding bits in the view vector are switched on (=1) whenever a doorway is in the field of vision of the agent (**Figure 5**). The factor of height played a role in distinguishing the discrepancy in the code which might occur in certain conditions. Since the height of the doorways remained the same throughout the simulations, the view of a narrow doorway visualized close by can be differentiated from seeing a wide doorway from far away. The number and location of 1's in the visual field becomes a function of the position and the orientation of the agent and forms an implicit code for representing space. A conceptual illustration and the construction of the visual vector is seen in **Figure 5**. See Supplementary Material S1 for the construction of the view vector.

FIGURE 4 | The subcomponents of the Motor Module. The input to the network is the "view vector" associated with the doorway. The Critic Module computes Action Value and Risk, and combines the two into Utility function. The next step of the agent [1Z = (1X, 1Y)] is computed by using a policy (GEN Module). The next step information is converted to Step Latency, which represents the evidence (EII) of the Motor Module. Utility from the Cognitive Module modulates the Step Latency whenever a word cue is presented.

#### Critic

The Critic in the Motor Module is slightly different from that of the Cognitive Module. The Critic of the Motor Module, unlike the Critic in the Cognitive Module, estimates both Value and Risk as a function of the view vector. This distinction is present since the views close to the doorways themselves are sufficient to encode the attribute of Risk because of high probability of hitting the sides on approaching a doorway. The Critic computes the Value "Q mot" for the view vector [φ(t)]. It is defined as an estimation of the predicted reward at any time, t, for that state ?(t). We model Q mot(t) as in Equation (15):

$$Q^{\rm mot}(t) = A\_Q^{\rm mot} f(\sum W\_l^{\rm value}(t)\phi\_l(t))\tag{15}$$

where f(x) = 1 1+e−λmotx and λ cog is the slope of the sigmoid function. The update equation for the weights in the above approximation (having weight vector, Wvalue) is given by Equation (16):

$$
\Delta W^{\text{value}} = \eta^{\text{not}} \delta^{\text{not}} \phi(t) \tag{16}
$$

Here, η mot is the learning rate for the critic and "δ mot" denotes the temporal difference (TD) error in Value function, that has been linked to dopamine signaling (Schultz, 2010). It is given by Equation (17) in which γ is the discount factor.

$$\delta^{mot} = r^{mot}(t) + \chi Q^{mot}(t) - Q^{mot}(t-1) \tag{17}$$

The Risk function for the Motor Module is approximated using the Equation 18. The weights for the Risk computation are updated using (δ mot) 2 (Equation 19). Note that this same quantity is estimated as uncertainty in the Cognitive Module (Equation 13).

$$h^{\rm mot}(t) = A\_h^{\rm mot} f(\sum W\_i^{risk}(t)\phi\_i(t))\tag{18}$$

$$
\Delta \mathcal{W}^{risk} = \eta^{mat} (\delta^{mat})^2 \phi(t) \tag{19}
$$

The Utility function combines the Value function and α mot controlled Risk function.

$$U^{\rm mot}(t) = Q^{\rm mot}(t) - \alpha^{\rm mot} \text{sign}(Q^{\rm mot}(t)) \sqrt{h^{\rm mot}(t)} \tag{20}$$

#### Actor/Go Explore NoGo Policy

The Actor in the model computes the direction of movement for the agent using the latency is estimated. We assume in the model that there must be an actor only for the Motor Module, and the Cognitive Module influences this information based on the word stimulus that appeared during the trial, thereby controlling the latency of the motor actions. The Actor dubbed as GEN or the GO/EXPLORE/NOGO policy is a type of action selection mechanism which performs a stochastic hill-climbing over the Value function space. This type of action selection has been shown to model a range of BG functions in healthy controls and PD patients (Sridharan et al., 2006; Magdoom et al., 2011; Kalva et al., 2012; Gupta et al., 2013; Muralidharan et al., 2013). In the present model, however, the GEN policy is used to maximize the Utility rather than the Value function, as in our prior models (Balasubramani et al., 2014, 2015). This is achieved by modifying the GEN equations of Muralidharan et al. (2013) and extending them to the Utility function as follows:

$$\delta\_U = U^{mot}(t) - U^{mot}(t-1) \tag{21}$$

The 3 regimes of action selection (GO/EXPLORE/NOGO) can be represented as a function of δ<sup>U</sup> by the following expression

$$\begin{aligned} \Delta Z(t) &= A\_G \text{sig}(\lambda\_G \delta\_U) \,\Delta Z(t-1) \\ &+ A\_E \chi \, \exp(-\delta\_U^2 / \sigma\_E^2) \\ &- A\_N \text{sig}(\lambda\_N \delta\_U) \,\Delta Z(t-1) \end{aligned} \tag{22}$$

where AG, AN, A<sup>E</sup> are the gains of GO, NOGO and EXPLORE regimes respectively, λ<sup>G</sup> and λNare the sensitivities of GO and NOGO regimes and σ <sup>E</sup> is the parameter controlling the extent of exploration. 1Z(t) = - 1X(t) 1Y(t) represents the change in position at the time step t which includes both the components of velocity, using which the current position of the agent is updated. The GEN policy can give rise to negative velocities and thus can hamper the agent's movement by inducing backward motion in the simulations. In order to prevent the agent from doing this, the y component of the velocity [1Y(t)] is passed through a sigmoidal function before addition to the position [Z(t)]

$$
\widetilde{\Delta Y}(t) = \frac{1}{1 + e^{(-\lambda^{\text{rel}} \Delta Y(t))}} \tag{23}
$$

$$
\Delta Z(t) = \begin{bmatrix} \Delta X(t) & \tilde{\Delta} Y(t) \end{bmatrix} \tag{24}
$$

$$Z(t+1) = Z(t) + \Delta Z(t) \tag{25}$$

Here Z = (X,Y) and denotes the position of the agent on the track. This gives rise to a different orientation and a view vector and thus the cycle continues. The 1Z(t) represents EII of the task (**Figure 2**) and is used to estimate the step latency. (See Supplementary Material S2: Table 1 for the list of parameter values to simulate the Motor Module)

#### ESTIMATING THE LATENCY OF MOTOR ACTIONS

The final output of the model is the step latency which is dependent on the outputs of both the Cognitive and Motor Modules (**Figure 4**). Since the words appear only at certain instants in the task, their contribution to the latency is maximal only at the time of their presentation. A decision variable, s, which can be thought to accumulate evidence for an action is used to get the reaction times from the model. From the Motor Module, the GEN output is used to estimate the latency at any given point in time as,

$$\mathbf{\dot{s}} = \kappa \ast \left\| \Delta Z(\mathbf{t}) \right\|\tag{26}$$

$$\kappa = U\_W^{\text{og}} + b \tag{27}$$

In Equation (26), the variable "s" is defined as "intent for walking" (product of the Utility from the Cognitive Module and the GEN output from the Motor module) as its rate of change indicates how fast the agent would take a forward step, and has to cross a threshold (th = 1) for the action to be executed. The time taken for "s" to cross the threshold is the "step latency." The velocity term 1Z(t) comes from the Motor Module, and the coefficient "κ" comes from the Cognitive Module. Upon the appearance of a word, the U cog <sup>W</sup> is produced by the Cognitive Module which is used along with the velocity to compute the latency. At instants when there is no word cue, κ is set to the default value of b (Equation 27). Thus, the walking latency is determined by contributions from both Motor and Cognitive Modules.

#### MODELING PD

Parameters that represent PD conditions (freezers and nonfreezers) in the model include the temporal difference errors in both motor (δ mot) and cognitive (δ cog ) modules and sensitivity parameters for the Risk function in the Utility computation (α mot and α cog ). In agreement with previous modeling efforts, the temporal difference error is appropriately clamped to simulate dopamine deficient conditions (Gupta et al., 2013; Muralidharan et al., 2013; Balasubramani et al., 2014), using the factor δ ∗ . Therefore, if [a, b] represents the range of dopamine levels in healthy controls then [a, δ ∗ ] represents the PD OFF condition, where δ <sup>∗</sup>< b. The PD ON condition is modeled by the addition of a medication factor δmed to the existing dopamine level.

$$\text{PD OFF: } \begin{array}{l} \text{If } \delta > \delta^\* \\ \delta = \delta^\* \end{array} \tag{28}$$

$$\begin{array}{c} \text{If } \delta \text{ } \succ \delta^\* \\ \text{PD ON:} \begin{array}{c} \delta = \delta^\* + \delta\_{med} \\ \delta s \end{array} \\ \delta = \delta + \delta\_{med} \end{array} \tag{29}$$

In addition to these parameters, from our previous work on modeling PD freezers, the exploration factor σ <sup>E</sup> (Equation 22) is also considered as a factor contributing to FOG behavior (Muralidharan et al., 2013).

Model parameters representing the Motor and Cognitive modules in (1) healthy controls, (2) PD ON, and (3) PD OFF conditions, are estimated as follows. The critical parameters including the gains of the critic network in both modules (A mot Q , A mot h , A cog Q ), the sensitivities of the critic networks (λ mot , λ cog ), the risk sensitivities (α mot , α cog ), discount factor (γ ) and the parameters of the GEN (AG, AN, AE, λG, λN, and σ <sup>E</sup>) needed to simulate the model are first optimized for healthy controls (**Figure 6**) using genetic algorithm (See Supplementary Material S2: Table 2 for Genetic Algorithm conditions). Once optimized for healthy controls, the parameters are then also used for the simulation of PD conditions (both OFF and ON). Furthermore the parameters (mentioned above) used to simulate the PD conditions are further optimized using a grid search algorithm (Supplementary Material S2: Table 3) to best fit the experimental behavior.

A repeated measures ANOVA was done to estimate statistical differences among subject groups (Controls, PD non-freezers and PD freezers) in different conditions (doorway and cues). Bonnferroni correction was applied to correct for type I error inflation. Additionally planned t-tests were conducted to measure statistical significance in specific cases. In simulations each subject group for a particular task condition was run for 50 trials and the averaged results are presented. All the simulations were done in MATLAB R2013a (Mathworks Inc.).

#### RESULTS

This section is organized as follows: We explain: (1) the effect of Utility and Risk observed in both the Cognitive and Motor Modules, (2) the effect of cognitive cues on gait and their contribution to step latency under conflicting situations such as while approaching the doorways, and finally (3) effect of cues as a source of cognitive load. These effects have been modeled in healthy controls, and PD patients under ON and OFF medication conditions.

#### Value and Risk as Functions of the Cues

As mentioned above, Matar et al. (2013) investigated the effect of the modified Stroop cues on gait latency. The subject groups tested were healthy controls, PD non-freezers and freezers. In our model simulations of Matar et al. (2013), the agent is trained to "walk" (as a response) for a congruent cue (stimulus) and "stop" for an incongruent cue. Utility associated with the cues represents the goodness associated with an action in presence of the said cue. We derive Value, Risk, and Utility measures associated with each of the cues. Analyzing the effects of simple and complex cues, we found the following: the association of a complex cue (congruent or incongruent) to the VR task can decrease the certainty associated with the action, as these associations are not pre-learnt. In **Figures 7A–C**, in PD freezers and non-freezers, it is evident that complex cues are associated with low Utility for both actions, walk [F(8, 2) = 95.79, p < 0.05] and stop [F(8, 2) = 97.53, p < 0.05)], suggesting increased uncertainty in their responses. Furthermore, the Utility is much lower in the freezers compared to the non-freezers (t = 5.04, p < 0.05).

The conflict in the association between cues and actions determines the Risk and Utility magnitudes of a cue component. For example, the presentation of "RED (red)" makes the agent continue walking, but the red color is initially primed to the response stop. The weightage of influence provided by the ink color and word meaning, and their magnitude of conflict within these components for the different classes of cues (simple, congruent and incongruent) can be analyzed by their Risk magnitudes as seen in **Figure 8A**. The model estimates higher Risks for all three subject groups for congruent and the incongruent cues in comparison to the simple cues [F(36, 2) = 21.74, p < 0.05]. The congruent cues have higher Risk which arises as a result of training the cue RED (red) to respond to "walk" whereas the inherent priming of the red stimulus (color or word) is to "stop." This can be seen in **Figure 8B** which represents the Risk estimated by PD freezers for congruent cues in which RED (red) shows the highest Risk in comparison to BLUE (blue) and GREEN (green). The increased Risk observed in the model affects the Utility through Equation 14. So in order to show behavioral differences the risk sensitivity is modulated among the groups. PD freezers have been modeled to have higher risk sensitivity (α cog ) in the cognitive loop (see **Table 2**).

#### Utility and Risk Functions in Relation to Gait

The variation among controls, PD non-freezers and PD freezers is analyzed by examining the Utility (U mot) and the Risk (h mot) functions obtained from the Motor Module. The differences are presented as a function of the distance from the doorway in **Figure 9**.

The amount of uncertainty that the model estimates is computed for every subject group using Equation 18. The Utility measure is the highest for controls followed by PD non-freezers and then PD freezers (**Figure 9A**). While analyzing the subjective Risk measures from the model for different subject groups, and the corresponding α measure, we see that the subjective Risk computed is very high for healthy controls as seen in **Figure 9B**. The values of the sensitivity factor (α mot) are in **Table 2**. The temporal difference error (correlate of dopamine) is clamped in PD non-freezers and freezers, with the freezers having a stronger clamp than non-freezers. Additionally the Risk function seems to peak closer to the doorway for PD subjects, suggesting its role in controlling latency near the doorways.

# Effect of Cognitive Cues on Motor Activity

On extending the Cognitive Module's contribution to the Motor Module, the model predicts the conflict among the different cues, which can be estimated as the response (step latency) of the agent upon the presentation of a Stroop word.

#### Behavioral Results

The model simulates the result of Matar et al. (2013) to understand the effect of cognitive cues on motor activity. Modal latency in **Figure 10A** shows no change in the latency among controls, PD non-freezers and PD freezers, similar to experimental results. This also augments the validity of the results obtained as the behavior in the model is not affected by the result of changes in the modal latency. The experimental results (**Figure 10B**) show that cues like GREEN (green) which have an implicit salience for "walk" response, evoke little or no change in the step latency for PD freezers. The RED (red) cue which has an implicit salience to "stop" response seems to increase the step latency of the PD freezers.

#### Model Results

The model replicates this effect where the BLUE (blue) [F(49, 2) = 1376.88, p < 0.05] and RED (red) [F(49, 2) = 1048.01, p < 0.05] cues produced maximum footstep latency (MFSL) in the freezers in comparison to controls and non-freezers (**Figure 10C**). Consequently these are the two cues for which the Risk given by the Cognitive Module is high (**Figure 8B**). Several

#### TABLE 2 | Parameter values for simulating the behavior seen in the Matar et al. (2013) and Shine et al. (2013d) experiments.

the other two congruent cues. Abbreviations: CONG, congruent cue; INCONG, incongruent. (##p < 0.05).


Shine et al., 2013d


The bold values highlight parameters differences in PD freezers in comparison to controls and PD non-freezers.

studies propose the presence of higher uncertain component of the environment as a reason for the inability to inhibit such latent behavior (Vandenbossche et al., 2011, 2012). From observing only the Cognitive Module, the uncertainty in the simulated PD condition arises in the model due to (a) training the Cognitive Network under clamped δ (dopamine) conditions (see δ ∗ in **Table 2**), and (b) by controlling the agent's sensitivity toward the Risk (h cog ) associated with the cue using the parameter α cog . The risk sensitivity (α cog )in freezers is set higher than the other two groups, suggesting that the Risk taken into account for computing the Utility for a specific cue could be higher in freezers (**Table 2**). So, besides the Risk estimation being high, its accountability for behavior is also found to be high through our model.

# The Influence of Doorways on Step Latency

#### Behavioral Results

The experiments reported in Matar et al. (2013) suggest that the PD freezers exhibit higher step latencies in both the wide and narrow doorway cases, with the narrow doorway being more significant than the wide doorway (**Figure 11A**) compared to controls and non-freezers. There seemed to be a doorway width and group interaction, which is enhanced in case of narrow doorways. Moreover, in the experiment the latency during navigating narrow doorways had good correlation to item 3 of FOG questionnaire score (FOG-Q3—"Do you feel as if your feet are glued to the floor while walking, making a turn or while trying to initiate walking"). There is no significant doorway- word cue interaction in the study suggesting that the cues might not be involved in affecting the doorway latency within the subject groups.

#### Model Results

As mentioned previously in the Methods section all the cues presented to the agent appear in the region of 0–2 length units from the doorway. We study the effects of doorways on the step latency as the agent passes through them. The model captures the trends seen in the experiment in relation to the step latency exhibited by each subject group as they encountered a doorway of a specific type (**Figure 11B**). The wide [F(49, 2) = 136.5, p < 0.05] and narrow [F(49, 2) = 163.5, p < 0.05] doorways

provoked increased step latencies in PD freezers compared to the controls and non-freezers. The behavioral performance in PD non-freezer and PD freezer conditions is simulated by using parameters described in **Table 2**. The temporal difference errors in both Cognitive and Motor Modules are clamped to represent PD conditions, although the level of clamping was different for the two modules (**Table 2**). In PD freezers the values of both the exploration factor (σ <sup>E</sup>) and the sensitivity of the motor Risk function (α mot) are lesser than in case of the controls and the nonfreezers (**Table 2**), in contrast to the cognitive loop where the α cog is higher for the PD freezers compared to the other subjects.

# Cognitive Load and Motor Arrests

The effect of cognitive load on motor responses is a result of the ability of the subject to map cues to appropriate actions, depending on the nature of the cues. In this respect, simple cues are easily associated with their corresponding—walking or stopping. This is different for complex cues as mapping to actions is not straightforward.

#### Behavioral Results

In the Shine et al. (2013d) experiment, PD non-freezers and freezers were presented with cues both in the OFF and the ON medicated conditions. The trials were also counterbalanced among the patients such that a congruent cue is associated to "walk" and incongruent to "stop" and vice versa. According to the experiments, which were conducted on both PD freezers and non-freezers, the outcome of loading is evident from the number of motor arrests observed. The experiments were conducted with patients ON and OFF their dopamine medications. The PD freezers (OFF) showed the highest number of motor arrests (**Figure 12C**), with the tendency of freezing about 2.7 times more than the non-freezers. Though, PD freezers were generally more likely to suffer a motor arrest (both OFF and ON) compared to the non-freezers, the high load situation triggered more arrests in the PD freezers (**Figure 12C**). Similar to the previous experiment, there were also no significant differences in the modal latency between the PD freezers and the non-freezers.

FIGURE 11 | Maximum footstep latency (MFSL) observed for both the narrow and the wide doorways in the experiments (A) and the model (B). The MFSL is higher for both the doorways for the PD freezers compared to the non-freezers and controls. (##p < 0.05).

#### Model Results

In the model, a similar strategy is imposed and the trials including the low and the high load cues are extracted. The number of motor arrests is estimated using the distribution of the step latency (**Figures 12A,B**). As previously defined in the Methods, a motor arrest is any event with step latency that is twice the modal (preferred) step latency of the subject. The PD freezers seem to have higher frequency of higher step latency events especially in the regions of 30–50 in **Figure 12B**.

The modeling results are similar to experimental results (**Figure 12D**), where under high load scenario, the PD freezers OFF medication show maximum motor arrests, which is comparatively less in the low load case [F(49, 1) = 4.30, p < 0.05]. Similar to the previously simulated experiment, the temporal difference errors (δ mot and δ cog ) in both the modules are clamped along with appropriate modulation of the sensitivities (α mot and α cog ) of the Risk function. The introduction of medication seems to bring down motor arrests in the freezers, suggesting that the DA medications play a role in lowering the number of such spontaneous events though its mechanism of action is still unknown. However, in the model, the addition of a medication factor (δmed) in the TD error eqn. (Equation 28) did not produce the same effects as the experiments. In addition to δmedfactor, the sensitivity toward the uncertainty in the cues (α cog ) had to be significantly reduced to simulate this behavior (**Table 2**). Such effects are not seen in the PD non-freezer case and the model predicts no significant changes in the motor arrests in both the medication as well as the load states.

Additionally the involvement of the Cognitive Module in motor arrests can also be ascertained by analyzing the Utility of the Cognitive and Motor Modules at the time of a motor arrest (**Figure 13A**). Since the Cognitive Module is active only during the presentation of words, instances where a motor arrest is elicited when a word cue is given are extracted and the contribution of the Motor and the Cognitive Module is visualized. It is clear that the average values of the Utility of walking (U cog <sup>w</sup> ) of the Cognitive Module is much lower than Utility of the Motor Module (U mot) in case of PD freezers (t = 49.3, p < 0.05) compared to non-freezers. This further strengthens the claim that there is a shift to more cognition based

decision during a freeze episode and understanding the role of these areas would lead to further insights into the phenomenon.

#### The Influence of Risk Sensitivities (α mot and α cog) and Medication (δmed) on Motor Arrests

The contribution of each parameter to the normalized motor arrests also reveals several trends (**Figure 13B**). Although lower α mot values are used to simulate PD freezer conditions and are the optimal range for accounting for the experimental data, the trends suggest that higher α mot would lead to a high number of motor arrests. The role of α cog to elicit motor arrests seems to be more effective under conditions of low α mot values where there is an increase in the number of motor arrests as α cog increases. The model thus predicts an increase of α cog to differentiate a control from a non-freezer but an increase in α mot to increase the number of motor arrests. Medications are also found to play a role in decreasing the number of motor arrests, and in particular, a case of low α mot is shown to better responds to medications (**Figure 13C**).

#### DISCUSSION

The current study simulates the impact of cognition on gait in healthy and PD subjects while navigating a virtual reality environment. In this task, an agent navigates a virtual corridor consisting of doorways, while simultaneously responding to word cues (Matar et al., 2013; Shine et al., 2013a,c,d). The word cues could be either simple (direct) or complex like Stroop words (color-word pair) mapped to an appropriate action (walk or stop). The classical Stroop task involves either naming the ink color of a color word or just reading the color out loud. There seems to be facilitation in the responses (indicated by shorter reaction times) when the word and its ink color match (a congruent case). In the case of an incongruent cue, which consists of a mismatch between the word and its color, there is an inhibition of response and increased reaction times (MacLeod, 1991). Conflict in this association arises due to the mismatch in association of action to WORD and COLOR aspects of complex cues. Thus, our study attempts to model the effect these interactions on gait execution. The model uses an actorcritic based reinforcement learning (RL) model which performs Utility-based decision making. An extensive literature supports the function of the BG system as a reinforcement learning engine (Albin et al., 1989; Frank et al., 2004; Chakravarthy et al., 2010; Chakravarthy and Balasubramani, 2014).

The main observations from the model are as follows

1. Modal latency is the same among all the subject groups and match experimental results. This stresses the fact that the results obtained from the model are a consequence of changes due to the introduction of task parameters (cues and doorways) and not due to inherent differences among the subject groups. Freezing is a paroxysmal event such that observed motor arrests come as a result of a shift to higher latencies. This can be considered as the involvement of higher cognitive areas over the action which in the model can come from the intervention of the cognitive module (Vandenbossche et al., 2012). This is also evident from **Figure 13A** where the Utility for the cognitive network during motor arrests in PD freezers is very low compared to the non-freezers suggesting higher involvement of cognition in an event of freezing.


# Utility Codes Conflict

We use a Utility-based decision making model to analyze the effects of cognitive load on gait. The sole output of the proposed Utility-based decision making system is step latency. Step latency is selected to maximize Utility and minimize Risk. The Utilitybased decision making approach has been previously shown to explain PD motor impairment in precision grip and freezing of gait (Gupta et al., 2013; Muralidharan et al., 2013). As seen from Equation (7), Utility in the model is a weighted combination of Value and Risk functions. Value (Q ∗ ) is the expectation of rewards associated with the state (word/view), and Risk captures the uncertainty associated with the state and rewards. Risk in the Cognitive Module is modeled to measure the uncertainty associated with the state, indirectly from the probability of selecting an action (p<sup>w</sup> and ps) obtained from the Value function (Q cog ). Here, the state uncertainty comes due to the Stroop task where the mapping of the color and word to the actions walk and stop can be conflicting for several input stimuli. In the Motor Module the Risk function captures the action uncertainty as a function of the view vector (φ). The action uncertainty is often a result of the tradeoff between accuracy and speed, in this case manifest as high step latency close to the doorway. So the reason for the higher Utility estimated in the model for healthy controls could be that their Value computation is higher, or contribution of Risk to Utility computation could be lower. Risk computed here is very specific to patient groups as well as sensory states (word vs. view). This is seen in the **Figure 9B** where Risk function represents the environmental context in the healthy controls effectively, but becomes sub-optimal when simulated under PD condition with clamped dopamine signal. Also note that this Risk function can replicate experimental data only when scaled by the optimal α values (**Table 2**). Hence the effective Risk function is the product of the risk sensitivity (α mot/α cog ) and the Risk measure itself.

In the case of Cognitive Module, uncertainty is especially high for complex cues. In **Figure 8B**, the congruent cues seem to have high Risk and specifically RED (red) has the highest. The association of the RED (red) to the action walk as set in the described experimental task contradicts the implicit heuristic of associating the color red to the action "stop." In the model simulations with the incongruent cues, BLUE (red) and RED (blue) seem to have a low measure of Risk (**Figure 8C**). The experimental setup associates the incongruent cues to action stop. The facilitation of the red word and color to the action stop when presented as a part of an incongruent cue, leads to a low Risk. In the case of blue, we propose that its neutrality may not contradict the mapping to the associated action as strong as red, and hence develops a low Risk measure. Thus, Utility forms a strong indicator of conflict and its proper estimation is necessary for optimal behavior.

#### Model Predictions

#### Role of the Cognitive Module in Motor Arrests

It is evident from **Figure 13** that there is an increase in the contribution of the Cognitive Module to action selection in comparison to the Motor Module. The model predicts that PD freezers tend to rely more on the cognitive areas to perform the task. It is known that there is a shift toward more cortical resources during the situations that give rise to a freeze episode. Brain imaging studies have shown the recruitment of the posterior parietal cortex, dlPFC, and vlPFC (dorsolateral and ventrolateral prefrontal cortex), the anterior insula and the dorsal cingulate while performing the virtual reality task (Shine et al., 2013b). The role of anterior cingulate in resolving conflict has been well studied and especially thought to mediate process selection in the Stroop task (Pardo et al., 1990). Interestingly the anterior insular cortex has been investigated as a potential source for conscious error monitoring (Preuschoff et al., 2008; Ullsperger et al., 2010). This brain region was found to be implicated in generating autonomic responses in relation to balancing effortful tasks. Since the Utility in our model helps resolve the level of conflict in the task, we presume that these areas must be overcompensating for estimating the correct responses and thus preventing the subjects, especially the PD freezers, from proper gait execution.

A close analysis of the pattern of dopamine loss in PD patients indicates a relatively higher dopamine loss in putamen which controls the motor loop, compared to the caudate which controls the cognitive loop (Kish et al., 1988). This distinction is reflected in model parameters δ <sup>∗</sup>mot and δ <sup>∗</sup>cog , which denote maximum permissible dopamine levels in motor and cognitive loops respectively. Note that δ mot values are smaller compared to the δ cog values in both the PD non-freezers and freezers (**Table 2**). This pattern of loss could be exaggerated in PD freezers especially in the motor areas, leading to increased postural defects, increased incidence of falling and gait variability (Bloem et al., 2004; Jacobs et al., 2009). Medication seems to be another factor whose effect on freezing behavior seems to be difficult to comprehend. There have been instances in PD freezers where dopamine medications have alleviated the symptoms, though in some cases it had no effect, and in others the symptoms worsened like in ON state freezing (Nonnekes et al., 2015). This is an important aspect of freezing pathology that has to be understood in order to improve therapeutic strategies. The model predicts that there could be conditions where medications could be more effective than in other cases (**Figure 13C**). It is possible from the two cases seen in the figure (Case 1: α mot =0.1; α cog =7 and Case 2: α mot =7; α cog =7) that we can categorize the patients based on theses parameters, i.e., dopamine responsive regime (Case 1) and dopamine insensitive regime (Case 2). These parameters reflecting the risk sensitivity of the patients need to be deeply investigated for their neurobiological correlates. The significance of Risk Sensitivity parameters in the model is elucidated further below.

#### Risk Sensitivity Parameters (α mot and α cog)

The parameters α mot and α cog which correspond to risk sensitivity in each module seem to bring about the behavioral differences between controls, non-freezers and freezers. These Risk measures capture the variability in rewards (high when associating a cognitive or motor cue with a corresponding response) sampled through time due to the agent's responseexecution strategy. On analyzing the cognitive and motor loops, the following trends for α emerge. The healthy controls are more risk averse than PD patients with respect to the Motor Module (**Table 2**). Contrastingly, the PD patients are highly risk aversive with respect to the Cognitive Module.

Our earlier modeling studies on motor functions of PD (Gupta et al., 2013; Muralidharan et al., 2013) show a lower magnitude of the risk sensitivity parameter in PD patients. Modeling efforts suggest that risk sensitivity correlates with the levels of serotonin in the motor areas (Balasubramani et al., 2014, 2015). Incidentally the concentration of serotonin and its derivatives in PD have been shown to be lower in the cerebro-spinal fluid, with a strong correlation with freezing of gait (Tohgi et al., 1993). But, we need to tease apart risk sensitivity measures for the motor and cognitive loops individually. We see reduced risk sensitivity measure in the motor loop, accounting for the reduced risk aversive and increased risk seeking nature of the PD patients. In contrast, we see increased risk aversiveness in the Cognitive Module. If the previous hypothesis of reduced serotonin levels to reduced risk aversiveness in the motor areas is generalized, the cognitive areas should also have reduced serotonin levels. But as our model predicts increased risk aversiveness in cognition, the following are some possibilities of their pathophysiology.

It could be that there are differential changes in serotonin levels in the cognitive and motor areas, the former containing higher serotonin levels associated with risk aversive behavior, and the latter containing reduced serotonin levels. There have been reports showing differential loss of the expression of certain serotonin markers in the caudate and the putamen in PD (Kish et al., 2008). The caudate could be considered to be part of the cognitive loop due to its projections from frontal areas and putamen part of the motor loop as it receives projections from the motor cortex (Parent and Hazrati, 1995). Hence increased risk aversiveness reported in the study for the cognitive loop relates to increased risk sensitivity measure in these areas. This might give rise to an altered paradigm of decision making in the cognitive loop where subjects take more Risk into account upon the introduction of other tasks while walking and maintaining posture. On the other hand this could also force subjects to adopt a posture–second strategy where the importance given to postural maintenance is less compared to cognitive ability suggesting increased risk-seeking behavior in PD patients in the motor side (Bloem et al., 2006). Therefore, besides dopamine, this model suggests the need to conduct experiments to measure serotonin levels within the Cognitive and Motor Modules, and how they relate to the risk sensitivity. Many studies relate different kinds of uncertainty to the effects of neuromodulators such as acetylcholine (Ach) to the expected uncertainty, norepinephrine (NE) to the unexpected uncertainty both in the cortex and basal ganglia (Yu and Dayan, 2005), serotonin to modulate expected uncertainty in the BG (Balasubramani et al., 2014, 2015). It is plausible that these neuromodulators have differential action on PD gait and therefore merit a close and comprehensive study.

#### Model Limitations and Future Directions

Although the model replicates the behavior of controls and PD subjects under this paradigm, several additional neural level details as listed below can be potentially included. The model does not have an explicit representation of the cortex as it only includes the representation of cortical inputs for both the networks in the form of a word vector for the cognitive loop and the view vector for the motor loop. The Cognitive Module might involve the dorsolateral prefrontal cortex (DLPFC), the posterior parietal cortex and the caudate of the BG, while the Motor Module might include the motor cortex (M1), premotor area (PMA), and the putamen of the BG (Shine et al., 2013b). The sub-cortex receiving inputs from the cortex controls the gait centers in the brainstem through the output nucleus globus pallidus interna (GPi). GPi in turn controls the downstream brain stem regions (Mesencephalic locomotor regions) responsible for rhythm generation and maintenance of gait (Shine et al., 2013b). The model compares results based on the behavior to the VR tasks which only simulate the effect of locomotion. Although there seems to be a good correlation between the VR tasks and the timed up and go tasks in PD freezers (Shine et al., 2013a), especially the duration of motor arrests, it is necessary to quantify the model for actual walking tasks. Furthermore, in modeling perspective we could introduce a downstream gait model to understand changes in the dynamics of locomotion in PD patients.

Furthermore, the basal ganglia module in our model is an abstract version that could be developed to a more detailed neural network model (Balasubramani et al., 2015) elaborating the role of the different nuclei in eliciting freezing behavior. Additionally the interaction between the different cortical loops in our model occurs only at the level of the output of the basal ganglia (at the GPi/thalamic level), though interactions at cortex, striatum have also been reported to encompass cortico-striatal convergence (Guthrie et al., 2013). A network model including these areas would provide a better understanding of PD gait. It may also suggest neural targets for drug delivery in a patient-specific manner.

#### AUTHOR CONTRIBUTIONS

VM, Conceiving, developing the model, data analysis and manuscript preparation. PB, Conceiving, developing the model,

#### REFERENCES


data analysis and manuscript preparation. VC, Conceiving, developing the model, data analysis and manuscript preparation. MG, reference data and manuscript preparation. SL, reference data and manuscript preparation. AM, reference data and manuscript preparation.

#### SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: http://journal.frontiersin.org/article/10.3389/fnhum. 2016.00649/full#supplementary-material

disease. J. Neural Transm. 108, 53–61. doi: 10.1007/s0070201 70096


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2017 Muralidharan, Balasubramani, Chakravarthy, Gilat, Lewis and Moustafa. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# A Mathematical Model of Levodopa Medication Effect on Basal Ganglia in Parkinson's Disease: An Application to the Alternate Finger Tapping Task

Chiara Baston<sup>1</sup> , Manuela Contin2, 3, Giovanna Calandra Buonaura2, 3, Pietro Cortelli 2, 3 and Mauro Ursino<sup>1</sup> \*

<sup>1</sup> Department of Electrical, Electronic, and Information Engineering "Guglielmo Marconi," University of Bologna, Bologna, Italy, 2 IRCCS, Institute of Neurological Sciences of Bologna, Bellaria Hospital, Bologna, Italy, <sup>3</sup> Department of Biomedical and Neuromotor Sciences, University of Bologna, Bologna, Italy

Malfunctions in the neural circuitry of the basal ganglia (BG), induced by alterations in the dopaminergic system, are responsible for an array of motor disorders and milder cognitive issues in Parkinson's disease (PD). Recently Baston and Ursino (2015a) presented a new neuroscience mathematical model aimed at exploring the role of basal ganglia in action selection. The model is biologically inspired and reproduces the main BG structures and pathways, modeling explicitly both the dopaminergic and the cholinergic system. The present work aims at interfacing this neurocomputational model with a compartmental model of levodopa, to propose a general model of medicated Parkinson's disease. Levodopa effect on the striatum was simulated with a two-compartment model of pharmacokinetics in plasma joined with a motor effect compartment. The latter is characterized by the levodopa removal rate and by a sigmoidal relationship (Hill law) between concentration and effect. The main parameters of this relationship are saturation, steepness, and the half-maximum concentration. The effect of levodopa is then summed to a term representing the endogenous dopamine effect, and is used as an external input for the neurocomputation model; this allows both the temporal aspects of medication and the individual patient characteristics to be simulated. The frequency of alternate tapping is then used as the outcome of the whole model, to simulate effective clinical scores. Pharmacokinetic-pharmacodynamic modeling was preliminary performed on data of six patients with Parkinson's disease (both "stable" and "wearingoff" responders) after levodopa standardized oral dosing over 4 h. Results show that the model is able to reproduce the temporal profiles of levodopa in plasma and the finger tapping frequency in all patients, discriminating between different patterns of levodopa motor response. The more influential parameters are the Hill coefficient, related with the slope of the effect sigmoidal relationship, the drug concentration at half-maximum effect, and the drug removal rate from the effect compartment. The model can be of value to gain a deeper understanding on the pharmacokinetics and pharmacodynamics of the medication, and on the way dopamine is exploited in the neural circuitry of the basal ganglia in patients at different stages of the disease progression.

#### Edited by:

Daniela S. Andres, Swiss Federal Institute of Technology (ETH) and University of Zurich (UZH), Switzerland

#### Reviewed by:

Hidenao Fukuyama, Kyoto University, Japan Henning Schroll, Charité - Universitätsmedizin Berlin; Chemnitz University of Technology, Germany

> \*Correspondence: Mauro Ursino mauro.ursino@unibo.it

Received: 12 April 2016 Accepted: 25 May 2016 Published: 17 June 2016

#### Citation:

Baston C, Contin M, Calandra Buonaura G, Cortelli P and Ursino M (2016) A Mathematical Model of Levodopa Medication Effect on Basal Ganglia in Parkinson's Disease: An Application to the Alternate Finger Tapping Task. Front. Hum. Neurosci. 10:280. doi: 10.3389/fnhum.2016.00280

Keywords: mathematical model, neural network, Basal Ganglia, Levodopa, Parkinson's disease, finger tapping

# INTRODUCTION

One of the main disabling features of Parkinson's disease (PD) is bradykinesia, defined as the progressive reduction in speed and/or amplitude of repetitive actions. Several rating scales are currently used to assess the clinical severity of PD, which include also behavioral tasks where the patient performs repetitive movements. Among the others, the finger tapping task (Picillo et al., 2016) is one of the simplest, being able to provide information about the severity of bradykinesia. In particular, frequency of alternate finger tapping on two separate keys has been shown to correlate strongly with part III of the Unified Parkinson's Disease rating scale (UPDRS III; Fahn and Elton, 1987), namely with the bradykinesia subscore (Homann et al., 2000; Taylor Tavares et al., 2005) and to have the highest sensitivity to discriminate PD patients from the general population (Pal et al., 2001).

The usefulness of the alternate finger tapping task for specific PD motor assessment has been proven by evidences showing correlation with the extent of loss of neurons in the substantia nigra, assessed in vivo with [18F]-6-fluoro-L-dopa (6-FD) PET (Pal et al., 2001). The correlation just mentioned is considered relevant since the substantia nigra is the brain structure of the basal ganglia (BG) in which the death of dopaminergic neurons is responsible of the major motor symptoms of the disease.

At present, neither pure causal treatment nor neuroprotective mechanisms are available for PD. As previously reported (Contin and Martinelli, 2010) alternate finger tapping test seems appropriate for the evaluation of levodopa (LD) effects on bradykinesia, proving to be consistently affected in PD subjects compared with a control group and a sensitive and reproducible indicator of drug effect. As the disease advances, the practical benefits of levodopa are hindered by modifications in drug kinetic and dynamic mechanisms, resulting in a fluctuating response during the day. Oral doses of levodopa at first achieve a "long duration effect" persisting longer than the plasma half-life of the drug, but with the progression of the disease the clinical response becomes more dependent upon the rise and fall of plasma LD concentrations. No differences in LD pharmacokinetics have been observed in the shift from a "stable" to a "fluctuating" response to LD doses (Contin et al., 2001).

The relevance of the information provided by the finger tapping task, and its association with the pattern of the response to levodopa, is usually assessed with empirical equations, which assume a non-linear relationship (similar to the Hill law) between levodopa concentration in the brain and the frequency of tapping (Sheiner et al., 1979; Contin et al., 2001; Chan et al., 2004). Of course, this information is incomplete and risks of being under-utilized if not related with the neural circuitry of the basal ganglia, directly involved with the disease and responsible for movement's initiation and termination. In particular, it is well known that the response to levodopa affects the balance between Go (direct) and No Go (indirect) circuitry in the BG (Albin et al., 1989; Frank, 2005; Schroll and Hamker, 2013) which, in turn, results in the observed bradykinesia and the appearance of motor fluctuations. Indeed, the key point to understand the symptoms of PD, and their temporal deterioration, should be searched in the relationship between the activity of neurons in the BG (especially in the striatum) and dopamine (or levodopa) levels in the brain.

Neurocomputational models, inspired by biology, represent a powerful tool to quantify the main mechanisms involved in a complex neural system, and to relate behavioral patterns with the underlying neural circuitry. These models can also mimic the plastic changes induced by experience (such as the effect of reward and punishments on synapse potentiation and depotentiation) and the role of tonic and phasic alterations in neurotransmitter levels. The past years have seen a richness of neurocomputational models of the BG, with the emphasis on different neurophysiological or clinical problems (Frank, 2005; Wiecki and Frank, 2010; Schroll et al., 2012; Helie et al., 2013). Recently, we developed a neural model of the BG (Baston and Ursino, 2015a), which represents a good compromise between simplicity and accuracy. The model includes the three main routes operating in the BG circuitry (that is, the direct (Go), indirect (No Go), and hyperdirect pathways). Furthermore, it incorporates the role of dopamine (both tonic and phasic, i.e., dopamine peaks or dips during reward and punishment), synapse plasticity, and the role of the cholinergic interneurons (affected by dopamine levels themselves). Preliminary simulations performed in conditions of altered dopamine (Baston and Ursino, 2015b) show that, in the model, the time required to accomplish an action crucially depends on the tonic dopamine level, in a way coherent with the present knowledge of PD symptoms. Hence, we claim this model may be a suitable innovative tool to simulate bradykinesia (and in particular the tapping frequency) by relating the neural mechanisms with dopamine/levodopa levels in the brain.

Accordingly, the aim of this work is to quantify the connection between levodopa levels and finger tapping performances by means of biologically inspired models and computer simulations. In particular, we want to show that the model is even able to reproduce the qualitatively different finger tapping pattern in time of PD patients with no motor fluctuations (here referred as group 1) and PD patients showing motor fluctuations (referred as group 2). With this objective in mind, the neurocomputational BG model has been linked with a classic model of levodopa pharmacokinetics and pharmacodynamics. In particular, the BG model used has the same structure as the model presented in Baston and Ursino (2015a) including the main neural structures (cortex, Go, and NoGo neurons in the striatum, subthalamic nucleus, globus pallidus pars interna and externa, thalamus) involved in action selection by the BG. The difference is that here we assumed only two possible actions to be selected, and we simulated a dynamic shift between these actions to mimic an alternate finger tapping task. In other words, the model is used to perform a dynamic task; conversely, in our previous works, the model was used to perform a static task, in which just a single static choice had to be selected. The global model aspires to simulate the entire chain of mechanisms involved in the patient behavior, from levodopa administration to the response of motor cortical neurons.

With the model, we first analyzed how different dopamine levels may affect the frequency of tapping. Then, we simulated the temporal patterns of levodopa concentration in plasma and the temporal pattern of tapping frequency in six PD patients, during 4 h after levodopa administration.

#### MATERIALS

#### Patients

We retrospectively modeled levodopa (LD) test results obtained from six PD patients referred to the Institute of Neurological Sciences for therapeutic drug monitoring (TDM; Contin et al., 2001). Patients had given their written informed consent to personal data processing for research purposes. These patients were divided in two groups, on the basis of motor fluctuations: group 1 included patients without motor fluctuations, and group 2 patients showing motor fluctuations.

Clinical characteristics of patients are reported in **Table 1**.

#### Levodopa Kinetic-Dynamic Test

On the morning of TDM the patients received an oral fasting dose of LD/benserazide (100/25 mg) after a 12 h washout of LD.

Blood venous samples (2 ml) for measurements of plasma LD concentrations were drawn by an indwelling catheter immediately before the drug dose, at 15 min intervals for the first 90 min, then on the half hour up to 3 h after dosing. Blood specimens were collected and processed for plasma LD analysis as reported previously (Baruzzi et al., 1986).

Patients' motor response to the LD test dose was assessed by the alternate finger tapping test simultaneously with blood sample collection (Contin et al., 2001) up to 4 h post-dosing. This test objectively measured the number of times the patient could alternately tap two buttons 20 cm apart in 60 s with the most affected hand, using a touch-screen computerized system. Patients were comfortably seated in an armless chair and instructed to alternate tap the two touch-sensitive buttons as fast and as accurately as possible.

Latency to onset of a clinical significant motor response elicited by the LD test dose is defined as the time to increase in tapping frequency of ≥15% of baseline values. Duration of the tapping effect was calculated as the difference between the time to return to <15% of baseline values and time to onset of response.

For the objective of the present modeling application, patients were defined "stable" or "wearing-off " responders when no return or return to baseline tapping performances, respectively, was observed within the 4 h length of examination.

#### MODEL DESCRIPTION

The overall model consists of:


The relationships between the different parts of the global model are depicted in **Figure 1**, and illustrated qualitatively below. Equations can be found both in the main body of the manuscript and in the Appendix in Supplementary Material.

#### Modeling the Levodopa Pharmacokinetics

The kinetics of levodopa was simulated using an approach similar to that used in former papers (Sheiner et al., 1979; Contin et al., 2001; Chan et al., 2004). The overall model (subdivided in a plasma model and an effect compartment) is illustrated in **Figure 2**.

A two-compartment model (blocks 1 and 2 in **Figure 2**) was adopted to describe plasma levodopa concentration. This corresponds to the following equations:

$$V\_1 \frac{dc\_1}{dt} = -\left(k\_{21} + k\_{31} + k\_{\epsilon 1}\right)c\_1 + k\_{12}c\_2 + i \tag{1}$$

$$V\_2 \frac{dc\_2}{dt} = k\_{21c1} - k\_{12c2} \tag{2}$$

The first represents a central compartment, where levodopa is administered and plasma concentration is measured. The second is a peripheral compartment, representing the interaction between plasma and other body fluids. As shown in **Figure 1** and in the equations, the model contains five parameters: the intercompartment rate constants (k<sup>12</sup> and k21), the total body rate constant (ketot = ke<sup>1</sup> + k31), and the compartment volumes (V<sup>1</sup> and V2). It is worth noting that, compared with some previous models (Chan et al., 2004, for instance) we adopted two different values for the inter-compartment rate constants. This choice is necessary to obtain good fitting of the measured plasma concentration values (see Section Results). Assuming the same rate (i.e., k<sup>12</sup> = k21) results in poor fitting of real data.

The parameters describing the levodopa plasma kinetics were estimated in individual patients by minimizing a least-square criterion function of the difference between model predictions and in vivo data of plasma concentration, during 4 h after levodopa administration. Minimization was achieved using the Nelder-Mead algorithm (Press et al., 2007), which performs a direct search in the parameter space and does not require the computation of gradient. However, in order to reduce the number of estimated parameters (thus reducing the risk of overfitting) we maintained V<sup>1</sup> and V<sup>2</sup> at constant values, and only parameters k12, k21, and ketot were estimated individually. The fixed values of V<sup>1</sup> and V<sup>2</sup> were mean values taken from Chan et al. (2004), who essentially considered a subject with a standard weight of 70 kg. A personalization on the individual weights may be adopted in future works; however, we deem this specialization uninfluential at present, since our aim was to achieve a good fitting of plasma concentration, to be used for the downstream effect compartment.

The input to the central compartment, representing the levodopa administration, was maintained constant during a certain period, to mimic a progressive oral assimilation. The duration of this period corresponds to the period in which the measured plasma levodopa concentration increased progressively. The constant value was computed so that the


TABLE 1 | Clinical characteristics of each patient: age [years], Parkinson's disease symptom duration [years], levodopa therapy duration [years], anti-parkinsonian cotherapy dose (PRA stands for pramipexole, ROP for ropinirole and RAS for rasagiline) [mg/day], levodopa dose per day [mg/day], Unified Parkinson's disease Rating Scale (UPDRS) III scores, Hoehn and Yahr scores.

overall amount of administered levodopa was equal to the experimental one.

solution of Equation (3):

$$V\_3 \frac{d c\_3}{d t} = k\_{31} c\_1 - k\_{e3} c\_3 \tag{3}$$

#### Modeling the Levodopa Pharmacodynamics

In order to simulate the effect of levodopa on the basal ganglia, and therefore on the finger tapping response, we needed to calculate the levodopa concentration in the "effect compartment." We used the model proposed by Sheiner et al. (1979); this model assumes that the drug concentration in plasma and in the effect compartment tends to become proportional in steady state conditions (i.e., when all transient phenomena have exhausted, and all quantities settle at a constant equilibrium value), and that no levodopa comes back from the effect compartment to the central one. Hence, thanks to the last assumption, the effect compartment does not affect the kinetics of the plasma compartment, provided a single parameter (ketot) is used to describe total body clearance. As shown in **Figure 2**, the effect compartment contains three parameters: k31, ke3, and V3, which describe the drug absorption from the central to the effect compartment, the drug removal from the effect compartment, and the compartment volume, respectively. However, these parameters are not independent, since a combination of them produces the same model output. First, only two parameters actually appears in the equations, i.e., k31/V<sup>3</sup> and ke3/V3. Furthermore, it can be demonstrated that the shape of the c<sup>3</sup> (the concentration of levodopa in the effect compartment) temporal pattern depends only on the ratio k31/V3. In fact, the general can be written as follows (assuming no levodopa in the effect compartment at the instant t = 0):

$$c\_3(t) = \frac{k\_{31}}{V\_3} \int\_0^t c\_1(\tau) e^{-\frac{k\_{e3}}{V\_3}(t-\tau)} d\tau \tag{4}$$

Hence, the ratio k31/V<sup>3</sup> represents only a proportionality factor for the previous equation. This can be accounted for by a different value of parameter Dc<sup>50</sup> in the subsequent equation (Hill law, law, Equation (6)), without affecting the overall fitting procedure. Therefore, without a loss of generality, we used a fixed value for the parameter k31/V<sup>3</sup> in all trials, and only the remaining parameter ke3/V<sup>3</sup> was assigned individually. This parameter was estimated, together with the other parameters describing the pharmacodynamics, by fitting the global model to the finger tapping frequency values (see below).

Finally, we needed a law describing how the concentration in the effect compartment (i.e., c3) affected the activity of the neurons in the striatum. In fact, as described in the sub-section "Modeling the basal ganglia" below, our neurocomputational model assumes an input quantity (named D) representing how dopamine modulates the activity of the Go and NoGo neurons.

First, in order to account for the observed delay between plasma concentration and the clinical response, we introduced

FIGURE 2 | Compartment model used to simulate plasma pharmacokinetics (compartments 1 and 2) and the drug effect in the brain (compartment 3).

a pure delay (say T) between the computed concentration in the effect compartment and its action on the striatum neurons. The delayed concentration will be named c3delay below. We have:

$$c\_{3delay}(t) = c\_3(t - T) \tag{5}$$

A classic way to describe the binding of a molecule with a receptor (or a reaction with cooperative effects, where an enzyme can bind one or more substrate molecules) is the Hill law (Keener and Sneyd, 2009). We can write:

$$D = D\_0 + \frac{D\_{\text{max}} c\_{3ddelay}^N}{D\_{c50}^N + c\_{3ddelay}^N} \tag{6}$$

where D<sup>0</sup> represents the basal value (i.e., the effect immediately before the beginning of levodopa administration) and the second term, with a sigmoidal shape, represents the effect induced by a levodopa concentration c3, delayed by the time T. Dmax is the maximum effect that levodopa can produce, Dc<sup>50</sup> is the levodopa concentration which produces 50% of the maximum effect, and N is the Hill coefficient, which determines the slope of the concentration-effect relationship.

All the parameters in the previous equation (D0, Dmax, Dc50, and N) together with the parameter ke<sup>3</sup> in the effect compartment, and the delay T, were assigned to simulate the tapping frequency values measured on patients during 4 h after levodopa administration. At present, the estimation was performed manually, through trial and error adjustments of the parameters. The reason for this choice is that the tapping frequency exhibits significant outliers, which preclude a reliable automatic fitting. An automatic better fitting will be attempted in future works. Indeed, the aim of this preliminary work was not to achieve parameter estimation automatically, but rather to show that the present neurocomputational model of the BG can simulate the behavior of disparate patients, at different stages of PD severity.

#### Modeling the Basal Ganglia

A significant difference of this model compared with previous ones (for instance, Contin et al., 2001; Chan et al., 2004) is that Equation (6) was not used to fit the tapping frequency directly, but is used as an external input for the neural network model described below. The advantage is that, after parameter estimation, the model may be used to simulate other tests in the same patients, or to make additional predictions on the patient behavior, thus providing a much more flexible interpretation of the neurological status (see Section Discussion). We are aware that different parts of the basal ganglia may have different functions (for instance cognitive in the basal part and motor in the dorsal part) and that these may be differently damaged in different patients. Since we simulate a motor test (the finger tapping) we can assume that the model, and the estimated parameters, refer to the dorsal portion of the BG.

The specific structure of the network is depicted in **Figure 3A**, with further details clarified in **Figure 3B**. The detailed description of the BG computational model, including its equations, is presented in the Appendix in Supplementary Material. Parameters and synaptic weights of these equations are reported respectively in **Table 4** and **Table 5**.

Each neuron in the model is represented as a computational unit, which calculates its activity from the weighted sum of inputs. The output activity of each neuron is in the range [0, 1], representing a normalized firing rate. In particular, we used a sigmoidal static relationship to represent the presence of a lower threshold and upper saturation for neuronal activity, and a first order low-pass dynamics to mimic the integrative capacity of neuron membrane.

The model includes a sensory representation (S), which represents the external context, and the corresponding motor representation in the cortex (C). This representation considers several actions in mutual competition, each represented by a segregated channel. In the previous model version (Baston and Ursino, 2015a,b) we assumed four actions for simplicity. In the present work, devoted to a simulation of the alternate finger tapping test, only two actions are considered and sufficient to represent the task in an adequate way: tapping down (action 1) and lifting the same finger up while moving it to the other position (action 2). For this reason, just two different and segregated action channels are shown in **Figure 3A**, each coding for a different alternative choice. Of course, including more channels, the network can also be used to simulate more complex alternative choices.

Moreover, the model includes the thalamus (T), the striatum, functionally divided according to dopamine (DA) receptor expression (D1: Go or G, D2: NoGo or N), the subthalamic nucleus (STN), the globus pallidus pars externa (Gpe or E) and an output part represented by the globus pallidus pars interna (Gpi or I) and the substantia nigra pars reticulata (SNr) taken together. This framework implements the three main pathways (direct, indirect, and hyperdirect; Albin et al., 1989; Nambu

et al., 2002) used by the BG. In absence of sufficient stimuli, all actions are inhibited, due to a prevalence of the No Go pathway on the Go one, which leads to inhibition of the thalamus. In the presence of a sufficient external stimulus, the cortex can select a response based on a competition among cortical neurons. This competition is realized with a winner-takes-all (WTA) process, implemented via lateral inhibition in the cortex and a positive feedback from the thalamus. In particular, if the Go pathway prevails on the No Go within an action channel, the corresponding neuron in the thalamus is disinhibited, thus providing an excitatory input to the corresponding cortical neuron. This positive loop maintains the winner neuron to a high level (the action is selected) while the other rival neurons in the cortex are inhibited.

A simplification in the model consists in the use of the dopamine effect (D) directly as a modulating input factor, without explicitly representing the dopaminergic neurons in the substantia nigra pars compacta, which are responsible for the release of the dopaminergic neurotransmitter. This choice allows simple simulations of normal and pathological conditions, in which dopamine levels can be artificially altered by the disease or by external intervention. It is worth noting that D does not represent the real tonic dopamine level, but an input that modulates the working point of the Go and NoGo neurons in the striatum. High values of D mean high dopamine effect on receptors in the striatum. Conversely, low values of D indicate a poor dopamine effect. Experimental studies show that dopamine can exert different effects depending on the receptor (Hernández-López et al., 1997; Hernandez-Lopez et al., 2000). In particular, the effect of D is different within the striatum, being primarily excitatory for the Go part and inhibitory for the NoGo part. As a result, high values of D favor a rapid selection of actions, whereas low values of D are associated with a prevalence of the No Go, and so with slow or inhibited actions. Furthermore, we also included a contrast enhancement effect on the Go neurons (Frank, 2005), i.e., the quantity D is able to excite only the neurons in the Go pathway with a high excitation, thus further potentiating their response, but has an inhibitory effect on Go neurons with poor excitation. This mechanisms further help the WTA dynamics, accelerating the choice of a winner action.

A novelty of this model, presented for the first time in Baston and Ursino (2015a), is the description of the cholinergic pathway. Indeed, dopamine exerts its effect on the striatum not only directly (i.e., by exciting Go neurons with high activity and depressing both Go neurons with poor activity and NoGo neurons) but it also inhibits the cholinergic pathway (see **Figure 3B**). The latter, in turn, has an opposite effect on the striatum, favoring the No Go pathway, and depressing the Go pathway. Since the cholinergic system is active at rest, and is inhibited by a dopamine increase, the two mechanisms work in synergy. In particular, we observed that a change in dopamine level per se is insufficient to have appropriate responses, without the potentiation induced by the synergistic cholinergic effect. A lesion of the cholinergic mechanisms in the model would reduce the dopaminergic influence, thus resulting in a further bradykinesia.

Finally, evidences (Schultz, 1998) show that BG are able to modify their synaptic weights, in particular those entering into the Go and NoGo striatal neurons. This relies on dopamine and acetylcholine (Ach) changes. In particular, plasticity occurs in case of punishment or reward, when phasic changes in dopamine (a transient peak during rewards; a transient dip during punishments) induce a synaptic change via Hebbian mechanisms. A dopamine peak, with the consequent fall in Ach, further excites the winner Go neurons, and depresses the NoGo neurons, thus causing Hebbian potentiation of the winning action and depotentiation of all other actions. The opposite effect occurs during punishment.

In this study, however, differently from our previous work, we only test different tonic dopamine levels, avoiding the analysis of phasic changes and neglecting possible synapse plasticity during the trials.

In conclusion, the BG, through the combined action of the three pathways described above, modulate the inhibition provided from the Gpi to the thalamus, thus consenting or blocking the WTA process. Ultimately, it is the imbalance between the two pathways (Go and No Go), due to different values of the synapses and of dopamine level, that modulates the activity of the Gpi. If the Go pathway prevails, the Gpi provides less inhibition to the corresponding neuron of the thalamus (i.e., the BG "let go" the response). On the contrary, if the No Go pathway is more active, the Gpi provides more inhibition to the thalamus (i.e., the BG "stop" the response). Challenging situations, characterized by a high conflict among alternative actions, are managed by the hyperdirect pathway, carried out by the STN: its role is to provide an overall stop signal to all the units of the Gpi in order to prevent many simultaneous cortical winners and let the cortex more time to solve the conflict.

#### RESULTS

In the first part of this section, we will present some computer simulations results, to show how the model can simulate the alternate finger tapping test, and how the pattern of cortical motor neurons reflects the level of tonic dopamine.

In the second part, we will present the results of parameter estimation, obtained by fitting the overall model (pharmacokinetic-pharmacodynamic and neurocomputational) to the data obtained on six patients.

#### Simulation of a Finger Tapping Test

**Figures 4**, **5** show the temporal pattern of activity in the two cortical motor neurons, during two different simulations. We assume that the first neuron of the cortex (top panel of **Figures 4**, **5**) encodes the tapping, i.e., the movement downwards of the finger in either position, while the activity of the second neuron of the cortex (bottom panel of **Figures 4**, **5**) encodes its movement upwards together with the spatial shift. The first simulation was performed using a high value of D, typical of an healthy individual. The second was performed using a low value of the parameter D, typical of a PD patient.

In our model the external sensory input is used to denote the action to be selected (1 means that the action is strongly preferred, 0 no preference for this action). During these simulations we first presented a stimulus S = [1 0], which means excitation to the first channel and inhibition to the second. This provokes the activation of the first channel, inducing the finger to go down in the first position, preventing at the same time its movement upwards (deactivation of the second channel). However, for the movement to initiate, we need the Go pathway to prevails on the No Go one, to activate the thalamus and to allow the corresponding cortical neural activity to reach a level close to 1. In fact, we assumed that an action starts only when the cortical activity exceeds a given threshold, close to the maximum. This requires a transient period, clearly evident in **Figures 4**, **5**.

Furthermore, we assumed that, when an action has been selected, the input stimulus is reversed (in the previous example, we now provide an input S = [0 1] which excites the second action channel) to represent that the subject is now trying to perform the second action, which induces the movement of the finger upward and the shift toward the second position. A physiological time delay of 100 ms has been included after the beginning of the first action, to account for the physiological time necessary to initiate the movement, detect the action and communicate it to the central neural system. Similarly but opposite to the previous case, this time the finger has to lift up and shift, avoiding the tapping down. Again, after a transient period, the activity of the second neuron of C reaches its action threshold level, inducing the lifting of the finger and a new reverse of the input stimuli.

We are aware that simulating the tapping task as a simple choice between two actions (finger down in either position, finger up and shift) is a strong simplification. Of course, the real movements consist of a sequence of simpler movements, suitably chained. We summarized this chain of movements using only two macroscopic choices to reduce model complexity to a minimum. Moreover, with this assumption we are able to fully reproduce clinical data of the finger tapping task.

The activities of the two neurons of the cortex follow this iterative pattern, establishing recurring alternate signals.

In **Figure 4**, parameter D is 0.55 and the frequency of the neural signals is 2.89 Hz (173 taps/min), reflecting the short time required for the winner neuron to reach the action threshold.

**Figure 5** shows the same recurring alternate pattern for cortical activities, but with a tonic D = 0.22. Now the frequency of the neural signals is just 1.00 Hz (60 taps/min), indicating a severe stage of bradykinesia. In this last case it is worth noting the very long time required for the winner neuron to reach the threshold level that initiates the action: this is the consequence of the low levels of the input D, which inhibit the Go neurons and potentiate the NoGo ones.

Iterating the previous procedure several times, i.e., establishing a level of tonic D and evaluating the frequency of the neural activities patterns of the cortex (and so the alternate finger tapping task), a curve that maps D into the tapping frequency, and therefore tapping score, can be built.

As it is evident from **Figure 6**, this curve has a clear monotonic trend, with an upper saturation. An increase in D corresponds to an increase in the tapping frequency, more pronounced for lower values of D and nearly negligible for higher values of D, where the curve reaches saturation.

In our model, tonic D-values typical of Parkinson's disease may be between 0.2 and 0.3; in this range the curve obtained predicts that the tapping frequency for PD subject should be between 1 and 2 Hz. On the contrary, tonic D-values typical of healthy subjects in our model are set between 0.4 and 0.5, and the same curve predicts that the tapping frequency should be around 3 Hz. In other words, PD subjects are slower at performing the finger tapping task. This finding is coherent with the well-known bradykinesia of PD subjects compared to the normal motor behavior of healthy subjects (Contin et al., 1990).

#### Parameter Estimation in PD Patients

As previously specified, the patients were subdivided in two groups, on the basis of the duration of the tapping response.

Parameters describing the pharmacokinetics (k12, k21, and ketot) were fitted on the plasma concentration curves, while the other parameters (ke3/V3, T, D0, Dmax, Dc50, and N) were assigned to simulate the finger tapping frequency.

The results are shown in **Figure 7**, for what concerns the three patients of the first group, and in **Figure 8** for the patients of the second group. The values of all estimated parameters are reported in **Table 2**, for what concerns pharmacokinetics, and **Table 3**, for what concerns the drug effect.

The model, with a suitable choice of parameters, is able to simulate the patterns of levodopa concentration in plasma and the tapping frequency in all patients. Looking at the curves and at the estimated parameter values, no significant differences can be found between the two groups for what concerns the plasma concentration of levodopa. Conversely, significant differences can be observed in the relationship between the effect compartment concentration and its effect on the striatal neurons.

In particular, the patients in the first group have a much lower value of the parameter N (as low as 2 for all three) whereas patients in the second group have much higher values of N (7 or 8). This means that the relationship is much steeper for patients in the second group, and therefore that even a moderate reduction in levodopa concentration can produce a quick reduction of the tapping frequency.

Another parameter that seems higher in patients of the second group is Dc50, i.e., the levodopa concentration at half of the maximum effect. This implies that the effect of levodopa starts to be reduced at higher level, and higher concentrations values

TABLE 2 | Parameters of the compartmental model obtained using the Nelder-Mead algorithm for minimization and corresponding cost functions: k21, k12, and ketot are in [L/min].


V<sup>1</sup> and V<sup>2</sup> values were set a priori at 12 L and 32 L, respectively, corresponding to mean values typical of a 70 kg man, as done by Chan et al. (2004). The last column represents the criterion function after minimization, i.e., the sum of the square errors.

FIGURE 7 | Results concerning the three patients in the first group (subjects with no motor fluctuations). The first line represents the subject 1, the second the subject 2, and the third the subject 3. The first column reports the experimental data of plasma levodopa concentration (red dot and dashed line), the corresponding fitting curve (blue line) and the delayed estimated brain levodopa concentration curve (green line) vs. time. All concentrations are represented in µg/ml. The second column represents the tapping frequency [taps/min] vs. time [min]. Experimental data are represented with red dots, the fitting curve with a blue line. The third column represents the tapping frequency [taps/min] plotted vs. brain levodopa concentration [µg/ml]: experimental data are represented with red full points, estimated values with a blue line.

TABLE 3 | Parameters of the effect model, i.e., the drug removal rate ke3 /V3 , the delay T, and the parameters of the Hill equation tuned in order to fit the alternate finger tapping frequency pattern in each subject.

symbols is the same as in Figure 7.


The absorption rate (k31/V3) was maintained at the same value (0.01) for all patients. ke3/V<sup>3</sup> is in [min−<sup>1</sup> ], T is in [min], Dc50 is in [µg/ml], D0, Dmax , and N are dimensionless.

are necessary to obtain a sustained effect. However, differences in this parameter are less evident between the two groups.

A parameter that seems to have a certain role is also the drug removal rate from the effect compartment, i.e., ke3/V3. This parameter is higher in the second group than in the first, indicating a greater reabsorption rate in more severe PD patients.

It is worth noting that patient 3 in the first group exhibits value of Dc<sup>50</sup> and of ke3/V<sup>3</sup> more similar to those of the second group, suggesting that this patient is borderline between the two groups. TABLE 4 | Parameter values of the basal ganglia computational model. The parameters refer to the complete description and equations of the computational model represented in the Appendix section.


# DISCUSSION

In recent years, we developed a simple model of action selection in the basal ganglia, which incorporates the main physiological mechanisms acting in the BG and represents a good compromise between simplicity and accuracy (Baston and Ursino, 2015a).

TABLE 5 | Synaptic values of the basal ganglia computational model. The synaptic values refer to the complete description and equations of the computational model represented in the Appendix section.


The most important aspect of that model was the reproduction, although in a simplified form, of the dopamine effect on the BG (in particular on the Go and NoGo neurons in the striatum) taking into account both the basic level (i.e., tonic dopamine) and phasic changes (peaks and dips during rewards or punishments). An important novelty was also the inclusion of the dopamine-acetylcholine balance on the striatum, which allowed a much better reproduction of the dopamine influence (especially during a fall in dopamine, potentiated by an excitation of the cholinergic system).

Previously, other neurocomputational models contributed to our understanding of both cognitive and motor deficits in PD. They mainly focused on the notion that reduced dopamine increases the activity and causes long-term potentiation in the indirect pathway of the BG, as observed by Wiecki and Frank (2010). These authors also state that this view can account for progressive motor degeneration as well as cognitive issues. Frank (2005) tried to explain cognitive tests results of medicated and unmedicated PD subjects, and particularly avoidance behavior in unmedicated PD on both motor and cognitive sides, using a computational framework (the LEABRA network) that, although incomplete (the hyperdirect pathway was actually lacking), is able to provide some explanations on clinical results. In a subsequent work the same authors included all the three main routes (direct, indirect, hyperdirect), using the same mathematical approach (Frank, 2006). The model by Moustafa and Gluck (2011) instead focused specifically on PD's issues in a peculiar cognitive task, the "weather prediction" task, pointing out a key role of prefrontal cortex dopamine in addition to striatal dopamine in affecting the action selection process. Stocco et al. (2011) described a model quite similar to the present, considering general interneurons as well. PD condition was simulated lesioning dopaminergic neurons. On the pure motor side, the influence of BG in the motor program selection functions and dysfunctions has been investigated by means of a spiking neural model (i.e., detailed neural description) by Humphries et al. (2006). Schroll et al. (2014) proposed a completely new viewpoint with respect to the majority of the computational models in literature, proposing that the motor impairments and pathway imbalances assessed in PD might also result from dysfunctional synaptic plasticity in the BG. In providing evidence of their original position, they used a detailed mathematical model, faithful to biological knowledge of BG, and different learning rules according to different BG structures driving the synaptic plasticity. In the motor domain, reaching has been studied, being a simple motor task: Magdoom et al., 2011 provided an original model, deviating from the classical Go/No Go model of the BG by adding an intermediate regime called the "explore regime," used to control the stochasticity of action selection. PD was simulated reducing the dopamine level and affecting the indirect pathway.

Compared to other models, our computational model belongs to the class of those that describe neuron dynamics with more simple and compact equations, still remaining constrained by the neurobiological architecture, being able at the same time to simulate both motor and behavioral aspects. This may contribute to understanding the nature of the computation performed by entire brain regions. Hereafter, we will specifically compare our mathematical implementation, our novelties and results with previous computational models in literature. Ashby and Crossley (2011) also introduced cholinergic interneurons, but using a detailed mathematical approach and interpreting these neurons only as a "switch" for learning events. Stocco et al. (2011) considered general interneurons, but did not account for their differential effect on Go or NoGo neurons of the striatum. In fact, the authors state that their role is only to release inhibition on projection neurons when proper cortical inputs are detected, thus allowing the incoming cortical signals to be processed. Other models (Frank, 2006; Wiecki and Frank, 2010; Cavanagh et al., 2011) share some similarities with the present one, especially in the physiological knowledge incorporated, but use different mathematical representations. The main differences are observed between our model and the model by Magdoom et al. (2011), which uses a much more simplified view of the basal ganglia and network structure, recalling more of an actor-critic model. Other relevant models lack some BG pathways and/or structures, compared with ours (Frank, 2005; Moustafa and Gluck, 2011; Stocco et al., 2011; Schroll et al., 2012).

Going back to our model, previous preliminary simulations (Baston and Ursino, 2015b) endorsed the potential usefulness of the model in the study of PD patients, for whom the relationship between dopamine level and action selection in the BG plays a crucial role. In particular, we showed that a reduction in dopamine levels induces a slowdown in the action time (Baston and Ursino, 2015b). A subsequent necessary step toward model application is now to test its behavior on real data, to verify the capability to reproduce effective responses in PD patients with different chronic levels, and to point out the relationships with the underlying neural circuitry.

Accordingly, the objective of this work was to simulate patient responses during a simple test (the alternate finger tapping) routinely used in the clinical practice. Hence, a simple BG model with only two action channels seemed suitable to grasp the main aspects of the test.

The first simulations, performed by varying the tonic dopaminergic input from a normal level to a level leading to severe bradykinesia (**Figures 4**–**6**), confirmed the existence of a clear monotonic relationship between the input and the velocity of movement selection (quantified by means of the tapping frequency). An important contribution emerging from **Figure 6** is that this relationship is quite flat at high values of the input (D > 0.4 in **Figure 6**), indicating a stable behavior. This points out that even quite large fluctuations in the input induce only minor fluctuations in the action selection velocity, as is typical for a healthy subject. Conversely, at low values of D this relationship becomes quite steep: at this position, even small changes in D can cause large alterations in the action selection velocity. We think this is the zone more disabling for a PD patient.

With this scenario in mind, we exploited the neurocomputational BG model within a more comprehensive integrated model, to mimic the overall chain of events from levodopa administration to the alternate finger tapping response. Using this global model, we then simulated the behavior of six PD patients, providing individual estimation of the most influential parameters. Furthermore, to account for patients' variability, three patients were chosen from a first group, showing no motor fluctuations, and three from a second group, with motor fluctuations.

In building the global model, we used a classic representation of pharmacokinetics and pharmacodynamics, already used in previous studies (Sheiner et al., 1979; Contin et al., 2001; Chan et al., 2004). Indeed, these former compartment models were adequate to simulate levodopa temporal effects, hence we did not deem necessary to modify them.

Actually, the great novelty of the present work is in the connection between the pharmacodyamic compartment and the BG neurocomputational model (see **Figure 1**). In other words, the output of the pharmacodynamic model is not directly related with the tapping frequency, as done in previous papers, but is added to the dopaminergic input of the neural model, thus acting on the excitation/inhibition balance between the Go and No Go pathways. This balance, in turn, affects the tapping frequency.

We wish to emphasize that, in our model, the wearing-off of levodopa effect depends on two concomitant factors, while only the first is usually considered in previous models:

(i) the sigmoidicity of the concentration/effect relationship (Equation (6)), mainly influenced by parameters Dc<sup>50</sup> and N. A steep relationship (i.e., the Hill curve) means a large influence of small levodopa changes, hence a possible unstable behavior; a high value of Dc<sup>50</sup> indicates the need for high levodopa doses;

(ii) the position of the input quantity, D, on the global curve depicted in **Figure 6**. The latter aspect, which reflects the behavior of the neurocomputational model, depends, above all, on the dopaminergic basal value D0, which sets the central working point on this curve. A subject with a high basic dopamine works positioned in the high segment of this curve (like a healthy subject with physiological high D values), and therefore exhibits a stable behavior independently on fluctuations of the dopaminergic input. In fact, as we demonstrated in our previous work too (Baston and Ursino, 2015a), in this case the velocity of the action selection is almost unaffected by moderate fluctuation in the input D. Nevertheless, changes in D in these subjects (as occurring during rewards and punishments) can have a significant effect on synapse learning, by potentiating or depotentiating a synapse via Hebbian plasticity. On the other hand, a subject with a low basal value of D (as typical of PD subjects) works in the steep portion of the curve in **Figure 6**, hence he is naturally more prone to large fluctuations in the action selection velocity. This is the zone where therapeutic interventions are more effective, but also where instability is more easy to occur, and where the use of the model may provide significant benefits in future applications (as discussed in the last part of this session).

To elucidate these concepts, and look for model clinical use, in this work we undertook a preliminary model validation in a clinical setting by estimating parameters on six patients. We are aware that six patients are too few to attempt a critical analysis of the estimated parameter values. Indeed, the aim of this work was not to provide an exhaustive statistical investigation on parameter estimates (that would require a large dataset and an automatic estimation algorithm) but rather to show that the model is able to simulate the temporal patterns of the alternate finger tapping response in various patients, with potential clinical applications. Nevertheless, we can derive some preliminary indications, which of course require a subsequent confirm on a larger dataset.

An automatic procedure was chosen only for fitting the parameters in the compartment model. In fact, in this model, due to the regularity of plasma levodopa, the fitting was unique and reliable, quite independently on the initial guess. Conversely, an automatic fitting for the Hill parameters in the effect portion of the model was not appropriate, at least in the present initial work, since, due to the large noise on the experimental data, the obtained parameter values were crucially dependent on the initial guess and influenced by some outliers. Hence, we preferred a manual investigation in the parameter space. We are aware that our conclusions may be not unique, since other parameter combinations might produce quite similar results. In future works, with more data available, we will develop a constrained automatic fitting procedure, to solve the aforementioned problems.

Among the estimated parameters, three of them seem to have a more relevant role in discriminating between stable and wearingoff patients. The most important is the Hill coefficient N, which sets the slope of the sigmoidal curve (Equation (6)). Critical patients in the second group have higher values of N, suggesting a steeper relationship between levodopa and its effect (see also the last column in **Figures 7**, **8**). This signifies that even a small change in levodopa concentration may induce the passage from saturation to the lower level.

Another parameter that seems to play a role is Dc50, which represents the value at which levodopa exerts half of its maximal effect. The greater values found in two patients of the second group signify that higher doses of levodopa are required to have sustained beneficial effects.

Finally, the parameter ke3/V3, which describes the levodopa removal from the effect compartment, is also of interest, setting the velocity at which concentration in the effect compartment decreases. Two patients of the second group have higher values.

We expect that the previous parameters may have a clinical impact. In particular, a high value of N and, although less important, a high removal rate mean that a patient may rapidly decay from quite a stable behavior (corresponding to a high value of D in our model) to a bradykinetic behavior. This is related with the duration of the tapping response and, more generally, with the length of clinical effects. A high value of parameter Dc<sup>50</sup> implicates a smaller impact of levodopa on the patients, hence the necessity of higher doses. Indeed, patients with wearingoff phenomena may require almost two-fold higher levodopa concentrations (Contin et al., 1993).

The present results, although obtained on a small number of cases, emphasize the potential model benefits, and agree reasonably well with those found in former papers, where the pharmacodynamic model was directly linked to the tapping frequency. For instance, Contin et al. (1993, 2001) observed that patients with wearing-off phenomena required almost twofold higher levodopa concentration (a result related with the parameter Dc<sup>50</sup> in our model) and that regression toward a more unstable response to levodopa was associated with an increase in the sigmoidicity index N. Moreover, these authors found no clear role for the maximum magnitude effect (Dmax) with the progression of the disease. All these results agree with our observations, as evident looking at **Table 3**. It is worth noting that all the previous parameter changes have a substantial implication on the relationship between plasma levodopa concentration and its effect. Although plasma levodopa kinetics may remain substantially unchanged, the same pattern may produce shorter and more elusive effects in the advanced patients, being associated with a more rapid onset but also with a curtailed duration of motor response.

#### REFERENCES


As also discussed in Contin et al. (1993, 1994) this knowledge may have strong practical effects. It might assist in designing a better dose regimen, for instance by reducing the dose of levodopa and increasing its frequency in more critical patients. A challenge for future works may be to find the optimal curve for levodopa administration, after the individual parameters have been estimated, in order to reduce the bradykinesia periods but maintaining levodopa to a minimum. In patients like those of the second group, for whom even a small fall in levodopa level may induce a large change in responsiveness, a model application may be to predict the moment when the wearing-off is starting. A further aspect that may be studied with the model in future work is dyskinesia, which may depend on an altered balance between Go and No Go pathways, hence on dopamine levels too. Finally, patients with a poor responsiveness may also exhibit an unbalance between reward and punishment during learning tasks. The latter aspect may also have a strong practical impact.

In conclusion, we claim that this approach is original and may have important benefits in future works. Specifically, once parameters have been estimated on the individual patient, the global model can be used not only to simulate results of the alternate finger tapping test, but also to investigate the patient's behavior in other conditions of clinical interest. For instance, an excessive response to levodopa (as occurring in patients with elevated Hill coefficient N) may excessively trigger the Go pathway, resulting in hyperkinesia when levodopa concentration is high, or in bradykinesia, with a delayed or absent response, when the levodopa concentration falls down. Future works may concentrate on these aspects, for instance also evaluating the impact of some external noise (which is always present both in the inputs and in the neuronal responses) on the action selection, or the effect on synapse plasticity during tasks which require learning, as a function of the estimated levodopa effect parameters.

# AUTHOR CONTRIBUTIONS

CB and MU wrote the paper, developed the model, and performed the simulations. MC, PC, and GC provided the patients' data and contributed to data analysis and to the interpretation of results.

#### SUPPLEMENTARY MATERIAL

Supplementary Material presents the Appendix of the paper. The Appendix for this article describes in detail the Latest version of the Appendix can be found online at: http://journal.frontiersin. org/article/10.3389/fnhum.2016.00280

learning. J. Cogn. Neurosci. 23, 1549–1566. doi: 10.1162/jocn.2010. 21523

Baruzzi, A., Contin, M., Albani, F., and Riva, R. (1986). Simple and rapid micromethod for the determination of levodopa and 3-O-methyldopa in human plasma by high-performance liquid chromatography with coulometric detection. J. Chromatogr. B Biomed. Sci. Appl. 375, 165–169. doi: 10.1016/S0378-4347(00)83705-6


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2016 Baston, Contin, Calandra Buonaura, Cortelli and Ursino. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Time Processing and Motor Control in Movement Disorders

Laura Avanzino<sup>1</sup> , Elisa Pelosin<sup>2</sup> , Carmelo M. Vicario3, 4, Giovanna Lagravinese<sup>1</sup> , Giovanni Abbruzzese<sup>2</sup> \* and Davide Martino5, 6

<sup>1</sup> Department of Experimental Medicine, Section of Human Physiology and Centro Polifunzionale di Scienze Motorie, University of Genoa, Genoa, Italy, <sup>2</sup> Department of Neuroscience, Rehabilitation, Ophthalmology, Genetics and Maternal Child Health, University of Genoa, Genoa, Italy, <sup>3</sup> School of Psychology, University of Tasmania, Hobart, TAS, Australia, <sup>4</sup> Wolfson Centre for Clinical and Cognitive Neuroscience, School of Psychology, Bangor University, Bangor, UK, <sup>5</sup> International Parkinson's Centre of Excellence, King's College and King's College Hospital, Denmark Hill Campus, London, UK, <sup>6</sup> Queen Elizabeth Hospital, Woolwich, Lewisham, and Greenwich NHS Trust, London, UK

The subjective representation of "time" is critical for cognitive tasks but also for several motor activities. The neural network supporting motor timing comprises: lateral cerebellum, basal ganglia, sensorimotor and prefrontal cortical areas. Basal ganglia and associated cortical areas act as a hypothetical "internal clock" that beats the rhythm when the movement is internally generated. When timing information is processed to make predictions on the outcome of a subjective or externally perceived motor act, cerebellar processing and outflow pathways appear to be primarily involved. Clinical and experimental evidence on time processing and motor control points to a dysfunction of the neural networks involving basal ganglia and cerebellum in movement disorders. In some cases, temporal processing deficits could directly contribute to core motor features of the movement disorder, as in the case of bradykinesia in Parkinson's disease. For other movement disorders, the relationship between abnormal time processing and motor performance is less obvious and requires further investigation, as in the reduced accuracy in predicting the temporal outcome of a motor act in dystonia. We aim to review the literature on time processing and motor control in Parkinson's disease, dystonia, Huntington's disease, and Tourette syndrome, integrating the available findings with current pathophysiological models; we will highlight the areas in which future explorations are warranted, as well as the aspects of time processing in motor control that present translational aspects in future rehabilitation strategies. The subjective representation of "time" is critical for cognitive tasks but also for motor activities. Recently, greater attention has been devoted to improve our understanding of how temporal information becomes integrated within the mechanisms of motor control. Experimental evidence recognizes time processing in motor control as a complex neural function supported by diffuse cerebral networks including cortical areas, cerebellum, and other subcortical structures (Ivry and Spencer, 2004; Coull and Nobre, 2008). Timing is an essential component of motor control primarily within two types of motor tasks: (i) when producing sequential rhythmic movements or sustained movements of a definite duration (explicit timing); (ii) when the temporal information is used implicitly, such as when coordinating our movements to those of moving objects or individuals within the

#### Edited by:

Marcelo Merello, Fundación Para la Lucha contra las Enfermedades Neurológicas de la Infancia (FLENI), Argentina

#### Reviewed by:

Kristina Aurousseau, LaNeP3, CRIUGM, Canada Marta Bienkiewicz, ´ Aix Marseille Université, Centre National de la Recherche Scientifique, ISM, France

> \*Correspondence: Giovanni Abbruzzese

> giabbr@unige.it

Received: 27 July 2016 Accepted: 25 November 2016 Published: 12 December 2016

#### Citation:

Avanzino L, Pelosin E, Vicario CM, Lagravinese G, Abbruzzese G and Martino D (2016) Time Processing and Motor Control in Movement Disorders. Front. Hum. Neurosci. 10:631. doi: 10.3389/fnhum.2016.00631 external environment (implicit timing). In this review, we will provide a brief description of the neural network supporting motor timing focusing only on instrumental information to explain the link between timing and motor control in movement disorders. Then we will review available data on motor timing in Parkinson's disease, dystonia, Huntington's disease, and Tourette syndrome, and discuss how this body of evidence integrates with the available information on the pathophysiology of these movement disorders. Finally, we will discuss the translational aspects of the explored neural mechanisms with respect to future rehabilitation strategies.

Keywords: timing and time perception, movement disorders, neural pathways, motor control, rehabilitation

# THE NEURAL NETWORK OF MOTOR TIMING

#### Explicit Timing

When performing explicit timing tasks, we make an explicit use of temporal information (e.g., estimates of the duration of different stimuli or of their inter-stimulus intervals [ISI]) in order to represent precise temporal durations through a sustained or periodic motor act (Coull and Nobre, 2008). Functional magnetic resonance imaging (fMRI) (Bengtsson et al., 2004, 2005; Lewis et al., 2004; Jahanshahi et al., 2006; Bueti et al., 2008) and noninvasive brain stimulation (Koch et al., 2003; Dusek et al., 2011; Vicario et al., 2013) studies of motor timing have consistently identified brain regions crucial to time processing, in particular supplementary motor area (SMA), basal ganglia (BG), cerebellum and right-inferior frontal and parietal cortices.

The synchronization-continuation paradigm has been frequently used to study the neural substrates of explicit motor timing. This paradigm allows for the study of both externally triggered and internally triggered movements as it involves: (i) a synchronization phase, in which subjects are asked to tap in synchrony with a train of tones separated by a constant ISI, and (ii) a continuation phase, in which subjects are requested to continue tapping at the previous rate in the absence of the auditory cue. Using fMRI paradigms, Rao et al. (1997) showed that the continuation (internally triggered) phase, but not the synchronization (externally triggered) phase, activates the SMA, the left-caudal putamen, and the left ventrolateral thalamus. These findings support the existence of a hypothetical "internal clock" that beats the rhythm when the movement is internally generated (Pastor et al., 1992b; Meck and Benson, 2002; François-Brosseau et al., 2009), and involves the sensorimotor circuit of the BG (**Figure 1**). Indeed, it has been proposed that several parallel, segregated loops through the BG connect back to distinct cortical areas (namely motor, associative, and limbic network) (Alexander et al., 1986). The sensorimotor loop connects the motor areas in the cortex with the caudolateral striatum territories playing a role in stimulus-response habitual control (Redgrave et al., 2010).

The role of the BG in timing was further substantiated by demonstrating that they are activated during motor, but also perceptual, time processing tasks (Bueti et al., 2008) involving both sub-second and supra-second temporal intervals (Jahanshahi et al., 2006). Noteworthy, the associative basal ganglia circuit is particularly active during externally triggered tasks (**Figure 1**).

The cerebellum, instead, is more often activated (i) during motor than perceptual explicit timing tasks (Bengtsson et al., 2005; Jahanshahi et al., 2006; Bueti et al., 2008), (ii) when a synchronization to an external rhythm is required (Del Olmo et al., 2007), and (iii) when processing involves sub-second rather than supra-second intervals (Lewis and Miall, 2003a,b).

#### Implicit Timing

Timing is implicit when temporal information can optimize performance on a non-temporal task. The processing of timedependent features of a movement is crucial in predicting whether the outcome of the movement will be consistent with its ultimate goal, or will result in an execution error. When temporal information is processed to predict the outcome of self-executed or externally-driven movements, cerebellar outflow pathways are primarily involved. Indeed, cerebellar networks optimize self-executed actions by recalibrating predictions, capturing the sensory consequences of the same actions (Bell et al., 2008; Synofzik et al., 2008; Izawa et al., 2012). When subjects use temporal information inherent to the spatial-temporal trajectory of a dynamic visual stimulus to predict its final position, fMRI studies revealed activation in the left inferior parietal cortex (Assmus et al., 2005), sensorimotor regions of the premotor and parietal cortices (Field and Wann, 2005), and cerebellum (O'Reilly et al., 2008). We recently showed that the cerebellum is engaged when temporal information is processed to predict the temporal outcome of a motor act (Avanzino et al., 2015a). We developed an ad hoc task in which participants were required to observe a movement in a video and then predict the end of the same movement (Avanzino et al., 2013, 2015a; Martino et al., 2015). Crucially, a few seconds after its onset, the video was darkened for a given time interval, thus the task could be performed only by extrapolating time-related features of observed motion sequence. By this task, we have shown that inhibiting the activity of the lateral cerebellum with 1 Hz-repetitive transcranial magnetic stimulation induced a deterioration of timing performance selectively when subjects were asked to estimate the duration of a body segment movement (handwriting) and not of a movement involving an inanimate object (Avanzino et al., 2015a). This finding may suggest that cerebellar sub-regions are specifically involved in processing

temporal features of movements belonging to the human motor repertoire.

# EVIDENCE ON MOTOR TIMING ABNORMALITIES IN MOVEMENT DISORDERS (BOX 1)

#### Parkinson's Disease

The notion that the mechanisms underlying time processing are abnormal in PD has been suggested by some pivotal observations. Pastor et al. (1992a,b) reported that the accuracy in timing of repetitive alternating wrist movements, time estimation and time reproduction on a verbal task were impaired in PD patients "off " medication, indicating timing mechanisms abnormalities in PD already at a perceptual level and possibly related to dysfunction of an "internal clock" that is modulated by dopamine.

Afterwards, impaired time reproduction was confirmed during paced finger tapping in PD (O'Boyle et al., 1996; Elsinger et al., 2003) and assessed in "on" and "off " conditions (Jones et al., 2008). Using the synchronization-continuation task to investigate both de novo and treated PD subjects, Jones et al. (2011) showed that treated PD patients tapped ahead of the beat at the 250 ms rate, respect to de novo and healthy subjects without any difference between "off " and "on" conditions. This observation, resembling the clinical phenomenon of festination that typically occurs in PD for fast repetitive movements, suggested an influence of either chronic pharmacological treatment or disease progression. Altogether, the continuation phase highlighted major differences likely reflecting the difficulties of PD subjects with internally generated movements.

More recently, using a modified version of the synchronization-continuation paradigm with a motor imagery task, we showed that PD patients exhibited a selective deficit in motor timing for internally triggered sequential movements separated by a supra-second interval, and that this deficit was better explained by a defect of motor planning (Avanzino et al., 2013). Bienkiewicz and Craig (2015) showed that abnormalities in sensorimotor synchronization in PD are not limited to finger tapping task but rather extend in the context of a beat interception task based on aiming movements. The type of task required prospective motor control (i.e., coupling movement to neural based dynamic information that helps anticipate when the beat is going to sound) suggesting that BG circuitry might undermine also the temporal prediction ability.

Overall, the motor timing abnormalities observed in PD indicate that the BG are mainly involved in explicit timing mechanisms (see also Martinu et al., 2012). Accordingly, a


recent study showed that the implicit timing of salient events seems largely unaffected by PD (de Hemptinne et al., 2013). It is worthwhile to underline that the tight interaction between difficulties with cognitive timing and spatio-temporal control of movement emerges not only from experimental findings, but also from the clinical aspects of PD. Bradykinesia (slowness of movement initiation and execution) is particularly evident for internally generated sequential movements (McIntosh et al., 1997; Heremans et al., 2012), commonly occurs in gait and can manifest as slow shuffling strides, an accelerating gait, or highly variable and random stride times (Baltadjieva et al., 2006; Almeida et al., 2007).

Related to the neural network supporting motor timing abnormalities in PD, evidence demonstrated decreased activation within the sensorimotor cortex, cerebellum, and medial premotor system in PD patients compared to controls during paced finger tapping (Elsinger et al., 2003). Dopamine supplementation did not improve task performance, but partially "normalized" brain activation patterns. In a PET study PD patients presented a significantly enhanced activation of the cerebellum, thalamus and substantia nigra reticulata during a synchronization-continuation task, compared to healthy subjects (Jahanshahi et al., 2010). Moreover, pallidal over-activation and cortical regions under-activation were observed in PD subjects in the "off " state, but switching to the "on" state was associated with increased striato-frontal functional connectivity. This study and others (Husárová et al., 2013, 2014) supported the idea that motor timing networks are modulated by dopaminergic stimulation and that PD patients activate an alternative timing network relying more on cerebellar activation than healthy controls. Summarizing the behavioral and imaging evidence on the influence of dopaminergic medication on motor timing, so far results are mixed and therefore conclusions are unclear. The role of the BG oscillatory activity in motor timing has been evidenced in a group of 12 PD patients implanted for subthalamic nucleus deep brain stimulation (STN-DBS) (Wojtecki et al., 2011). Supra-second time reproduction performance was significantly improved during "on" stimulation (130-Hz) compared to no stimulation, whereas milliseconds timing was not affected.

# Huntington's Disease

Huntington's disease (HD) is a hereditary neurodegenerative disorder, caused by an expansion of CAG triplet repeats in the Huntingtin gene on chromosome 4. The monogenic etiology of HD allowed investigating timing in both symptomatic and presymptomatic individuals. Explicit timing abilities progressively deteriorate in mutation carriers as they approach clinical disease onset. Early-moderate HD patients exhibited marked irregularity of tapping rates on the synchronization-continuation paradigm (Freeman et al., 1996). In pre-symptomatic HD mutation carriers, reproduction of supra-second intervals accuracy decreases with estimated time to onset, Hinton et al. (2007) and Rowe et al. (2010). A self-paced variant of the same tapping task was also used to investigate the relationship between timing performance and disease progression. Performance on self-paced finger tapping was associated with disease progression (Michell et al., 2008; Tabrizi et al., 2009; Bechtel et al., 2010), showing high sensitivity in separating mutation carriers from non-carriers already from the earliest stages (Tabrizi et al., 2009; Bechtel et al., 2010). A similar pattern of timing deterioration was observed on non-rhythmic motor reproduction of supra-second intervals, which became less precise as disease onset approached and progressed further as disease advanced (Beste et al., 2007; Wild-Wall et al., 2008; Rao et al., 2014).

Functional magnetic resonance imaging (fMRI) confirmed that performance on paced finger tapping was associated with reduced neural activation in subcortical regions (putamen, caudate, thalamus) in pre-symptomatic HD mutation carriers closer to disease onset (Zimbelman et al., 2007). This subgroup of pre-symptomatic patients under-activated the SMA, left-anterior insula and right-inferior frontal gyrus during the finger tapping task. Conversely, those far from onset exhibited more efficient compensatory activation of other brain areas (sensorimotor cortex, medial frontal cortex and cerebellum) (Zimbelman et al., 2007).

Overall, the progressive involvement of cortico-basal ganglia sensorimotor and associative loops in HD patients is reflected by a similarly progressive timing deficit that is more selective for explicit, cognitive-controlled timing tasks rather than implicit timing tasks.

# Dystonia

Dystonia is a network movement disorder associated with maladaptive plasticity of motor output coupled to disordered sensorimotor integration. Dysfunction in both cortico-basal ganglia and cerebello-thalamo-cortical networks is believed to give rise to the involuntary tonic and phasic muscle contractions that characterize dystonia. Timing has been assessed mainly in task-specific forms of dystonia, triggered by complex movements like musical playing or writing. Kinematic analyses of scales or finger tapping performed on a digital piano by pianists with dystonia showed inaccuracies in tone and interval duration and rhythmic inconsistency (Jabush et al., 2004; Furuya and Altenmüller, 2013). This may represent a mere consequence of the motor overflow during musical performance, since part of these measures improved with botulinum toxin therapy (Jabush et al., 2004; Furuya et al., 2014). Further, explicit timing performance in these patients appeared similar to control subjects (Van Der Steen et al., 2014), suggesting that direct extrapolation of temporal properties while performing timed movements may be preserved in dystonia. On the other hand, we showed that patients with either task-specific (writer's cramp) (Avanzino et al., 2013) or non-task-specific (cervical) dystonia (Martino et al., 2015) are less accurate in predicting the temporal outcome of a visually perceived movement, and that this is observed for human body, but not inanimate object, motion. This supports the view that patients with dystonia manifest subtle differences in extrapolating temporal properties during the perception of hand/arm movements. Such deficit may be linked to an abnormal internal model of motor commands reflecting dysfunction of cerebellar outflow pathways. The selective implicit timing task abnormalities support the notion that dystonia is a broader network disorder, in which the crucial nodes are located

not only in the BG and the sensorimotor cortex, but also in the cerebellum.

#### Tourette's Syndrome

Tourette's syndrome (TS) is associated with the abnormal maturation of cortico-basal ganglia loops. Timing abilities are underexplored in TS. In an early report, discrimination between different time intervals in untreated patients was comparable to healthy subjects (Goldstone and Lhamon, 1976). More recently Goudriaan et al. (2006) reported that time estimation and reproduction of time intervals spanning from 2 to 20 s did not differ between TS and healthy adults. However, these authors measured time intervals using a manual stopwatch, which is poorly accurate. A study by Vicario et al. (2010) has documented higher accuracy in the processing of supra-second intervals in TS children with lower tic severity, compared to age-matched healthy volunteers (Vicario et al., 2010). This result was explained by suggesting the existence of an effective compensatory modulation phenomenon, in correspondence of the prefrontal cortex, leading to better tic inhibition and increased precision in explicit processing of supra-second time intervals. Dopaminergic modulation seems also relevant to temporal discrimination in TS patients, as dopamine receptor blockers like pimozide may improve patients' variability on this task (Vicario et al., 2016).

Overall, TS has been investigated mainly for perceptual aspects of timing, revealing greater accuracy of temporal discrimination of supra-second intervals associated with lower tic severity. As already suggested for other behavioral abnormalities in TS (Mueller et al., 2006; Jackson et al., 2007), this behavioral gain appears to be more linked to adaptive mechanisms facilitating tic suppression rather than to core pathogenic mechanisms.

# TRANSLATIONAL ASPECTS IN REHABILITATION STRATEGIES

Knowledge on abnormalities in time processing for motor control has been already applied to plan rehabilitative strategies in movement disorders. For PD, rehabilitation approaches to compensate defective internal rhythm generation by the BG have been designed. Rhythm-related interventions (such as external sensory cueing or music) have been extensively used in motor rehabilitation. Most studies showed that gait impairment in PD could be significantly improved by rhythmic (auditory) stimulation (Lim et al., 2005, 2010) or musically cued gait training (Benoit et al., 2014). Rhythmic external cueing was also found to improve upper limb movements of PD patients (Vercruysse et al., 2012), and even assist motor learning (Nieuwboer et al., 2009). However, synchronization with a fixed-tempo may be critical in

# REFERENCES

PD subjects (Bienkiewicz and Craig, 2015; Young et al., 2016) and the effect may be very limited in time since learning may be specific to cued rather than non-cued performance. Thus, novel strategies have been recently proposed involving interactive systems providing adaptive rhythmic stimulations (Hove and Keller, 2015) video (Pelosin et al., 2013) or mirror visual feedback training (Bonassi et al., 2016).

The motor timing abnormalities observed in hyperkinetic disorders might have a role in disease monitoring (HD), development of novel rehabilitative strategies (dystonia), or monitoring of existing interventions (tic disorders). Self-paced finger tapping and time reproduction of supra-second intervals have proved to be sensitive sub-clinical markers of disease progression in HD, and their application to experimental trials of disease-modifying treatments is potentially very rewarding.

Several experimental approaches aimed to promote the re-organization of sensory-motor regions to improve motor control and symptoms of dystonia (Avanzino et al., 2015b). Available information in dystonia shows reduced accuracy in extrapolating temporal properties during the perception of body movements. Sensory-motor integration is central to new physiological models of motor control, whereby motor outflow generated by the primary motor cortex is associated with the generation of efferent copies that allow anticipating the sensory consequences of the motor program (Perruchoud et al., 2014). This anticipatory ability could be crucial to motor performance, and could be influenced also by perceptual experiences of internal and external movements. Within this theoretical framework, temporal expectations tasks applied to external or internal movements might be exploited as rehabilitative strategies to improve the ability to anticipate "on-line" the sensory consequences of motor output in dystonia.

Finally, cognitive control over tics is an important component of behavioral treatment strategies (Piacentini et al., 2010; Van de Griendt et al., 2013). Predictors of response to behavioral treatments for tics are still uncertain, and performance on tasks assessing cognitive control of motor actions may represent a useful predictive marker. Interestingly, preliminary studies of time processing in TS suggest a possible link between cognitively controlled tasks and timing, indicating that time processing tasks might be considered in future studies on cognitive predictors of response to behavioral treatment (Jung et al., 2013; Vicario et al., 2013).

# AUTHOR CONTRIBUTIONS

LA, GA, and DM contributed to literature search, writing of preliminary draft, text revision, and finalization. EP, CV, and GL contributed to literature search, writing of preliminary draft.

Almeida, Q. J., Frank, J. S., Roy, E. A., Patla, A. E., and Jog, M. S. (2007). Dopaminergic modulation of timing control and variability in the gait of Parkinson's disease. Mov. Disord. 22, 1735–1742. doi: 10.1002/mds.21603

Assmus, A., Marshall, J. C., Noth, J., Zilles, K., and Fink, G. R. (2005). Difficulty of perceptual spatiotemporal integration modulates the neural

Alexander, G. E., DeLong, M. R., and Strick, P. L. (1986). Parallel organization of functionally segregated circuits linking basal ganglia and cortex. Annu. Rev. Neurosci. 9, 357–381. doi: 10.1146/annurev.neuro.9.1.357

activity of left inferior parietal cortex. Neuroscience 132, 923–927. doi: 10.1016/j.neuroscience.2005.01.047


and externally-triggered finger movements in the dominant and nondominant hand. Eur. J. Neurosci. 29, 1277–1286. doi: 10.1111/j.1460-9568.2009. 06671


a randomized controlled trial. JAMA 303, 1929–1937. doi: 10.1001/jama. 2010.607


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2016 Avanzino, Pelosin, Vicario, Lagravinese, Abbruzzese and Martino. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Do Dopaminergic Impairments Underlie Physical Inactivity in People with Obesity?

Alexxai V. Kravitz 1, 2 \*, Timothy J. O'Neal <sup>1</sup> and Danielle M. Friend<sup>1</sup>

*<sup>1</sup> National Institutes of Health, National Institute of Diabetes and Digestive and Kidney Diseases, Bethesda, MD, USA, <sup>2</sup> National Institutes of Health, National Institute on Drug Abuse, Baltimore, MD, USA*

Obesity is associated with physical inactivity, which exacerbates the negative health consequences of obesity. Despite a wide consensus that people with obesity *should* exercise more, there are few effective methods for increasing physical activity in people with obesity. This lack is reflected in our limited understanding of the cellular and molecular causes of physical inactivity in obesity. We hypothesize that impairments in dopamine signaling contribute to physical inactivity in people with obesity, as in classic movement disorders such as Parkinson's disease. Here, we review two lines of evidence supporting this hypothesis: (1) chronic exposure to obesogenic diets has been linked to impairments in dopamine synthesis, release, and receptor function, particularly in the striatum, and (2) striatal dopamine is necessary for the proper control of movement. Identifying the biological determinants of physical inactivity may lead to more effective strategies for increasing physical activity in people with obesity, as well as improve our understanding of why it is difficult for people with obesity to alter their levels of physical activity.

#### Edited by:

*Daniela S. Andres, ETH Zurich, Switzerland*

#### Reviewed by:

*Ramalingam Vetrivelan, Harvard Medical School, USA Aaron G. Roseberry, Georgia State University, USA Yinghua Yu, University of Wollongong, Australia*

> \*Correspondence: *Alexxai V. Kravitz lex.kravitz@nih.gov*

Received: *26 July 2016* Accepted: *28 September 2016* Published: *14 October 2016*

#### Citation:

*Kravitz AV, O'Neal TJ and Friend DM (2016) Do Dopaminergic Impairments Underlie Physical Inactivity in People with Obesity? Front. Hum. Neurosci. 10:514. doi: 10.3389/fnhum.2016.00514* Keywords: obesity, dopamine, exercise, physical activity, physical activity promotion, Parkinson's disease, movement disorders

# INTRODUCTION

Obesity is associated with reductions in motor output, often termed "physical inactivity" (Tudor-Locke et al., 2010; Bouchard et al., 2015), although whether this relationship is causal remains a point of debate (Simon et al., 2008; Haskell et al., 2009; Dwyer-Lindgren et al., 2013; Swift et al., 2014). Despite the importance of physical activity for health, there are few effective methods for increasing physical activity levels in people with obesity, leading some researchers to conclude that, "there are presently no evidence-based interventions that can reliably and sustainably increase the level of physical activity among obese adults" (Ekkekakis et al., 2016). This point is reflected in our limited understanding of the cellular and molecular determinants of physical inactivity in people with obesity. We believe that a cellular understanding of why obesity is associated with physical inactivity is needed to understand, and ultimately alter, the relationship between obesity and physical inactivity. In this review, we propose that impairments in striatal dopamine contribute to physical inactivity in obesity, akin to classic movement disorders such as Parkinson's disease.

The striatum is a forebrain structure that controls movement, as well as learning and emotional states. There are two main projection cell types in the striatum, the "direct" and the "indirect" pathway medium spiny neurons (dMSNs and iMSNs), as well as several classes of interneurons. dMSNs and iMSNs exhibit distinct protein expression patterns, projection targets, and support distinct behavioral functions (Alexander and Crutcher, 1990; DeLong, 1990; Gerfen et al., 1990; Graybiel et al., 1994; Le Moine and Bloch, 1995; Obeso et al., 2000; **Figure 1A**). dMSNs express the excitatory Gs-coupled dopamine D<sup>1</sup> receptor (D1R), while iMSNs express the inhibitory Gi-coupled dopamine D<sup>2</sup> receptor (D2R; Gerfen et al., 1990). Dopamine can facilitate movement by binding to D1Rs and enhancing the output of dMSNs, or binding to D2Rs and inhibiting the output of iMSNs (Sano et al., 2003; Buch et al., 2005; Durieux et al., 2009; Kravitz et al., 2010). In this way, dopaminergic signaling controls the downstream signaling of dMSNs and iMSNs, and resulting motor output. We have simplified this discussion for the purposes of this review, but striatal function is also influenced by several additional layers of complexity (Mink, 1996; Calabresi et al., 2014). For example, the dorsal striatum is commonly linked to motor control, while the ventral striatum is linked to motivation and effortful movement (Mogenson et al., 1980; Voorn et al., 2004; Kreitzer and Malenka, 2008).

The importance of dopamine for proper control of movement is evident in neurological disorders. Hypokinetic states such as Parkinson's disease are the result of too little striatal dopamine (Hornykiewicz, 2010), whereas hyperactive states such as bipolar mania are associated with too much (Logan and McClung, 2016). Drugs that increase dopamine release (e.g., amphetamine) increase motor output (Schindler and Carmona, 2002) and dopamine antagonists (used clinically to reduce manic episodes) often result in motor impairments as a side effect (Janno et al., 2004; Parksepp et al., 2016). Genetic manipulations in animals further support the role of striatal dopamine transmission in motor control, as mice lacking dopamine receptors have reduced movement (Drago et al., 1994; Xu et al., 1994; Baik et al., 1995; Kelly et al., 1997; Beeler et al., 2016), whereas those that overexpress dopamine receptors are hyperactive (Ikari et al., 1995; Ingram et al., 1998; Dracheva et al., 1999; Thanos et al., 2001; Trifilieff et al., 2013). In particular, cell-type specific reductions of the D2R in iMSNs reduce open field movement, demonstrating the sufficiency of the D2R to regulate physical activity, by controlling the output of iMSNs (Anzalone et al., 2012; Lemos et al., 2016). In summary, striatal dopamine promotes movement in animals, due to actions on its striatal target neurons.

Obesity is associated with impairments in striatal dopamine function. Reported impairments include deficiencies in dopamine synthesis and release, as well as alterations in striatal dopamine receptors. While alterations in striatal DA transmission are commonly discussed in relation to reward processing (Kenny et al., 2013; Volkow et al., 2013), we hypothesize that these impairments may also contribute to the link between obesity and physical inactivity (**Figure 1B**).

# OBESITY AND PHYSICAL INACTIVITY

An inverse relationship between weight gain and physical activity has been observed in humans (Hemmingsson and Ekelund, 2007; Chaput et al., 2012; Hjorth et al., 2014), non-human primates (Wolden-Hanson et al., 1993), domesticated animals (Morrison et al., 2013), and rodents (Jürgens et al., 2006; Bjursell et al., 2008). The cross-species nature of this relationship indicates that it is a conserved phenomenon that may stem from the evolutionary benefit of storing energy in times of caloric excess, a state that is rare in nature. However, in modern environments physical inactivity exacerbates the negative health effects of

Schematic is replicated in lean (left) and obese (right) conditions, to show reported dopaminergic alterations in obesity. Inlay: Dopaminergic synapse onto striatal iMSNs. GPe, globus pallidus; STN, subthalamic nucleus; SNr, substantia nigra; VTA, ventral tegmental area. (B) Hypothetical progression by which diet induced obesity is associated with impaired striatal dopamine transmission, leading to physical inactivity.

obesity, increasing the risk of cardiac disease and diabetes (Al Tunaiji et al., 2014; Bao et al., 2014; Bouchard et al., 2015). It is possible that physical inactivity precedes, and thereby contributes to, weight gain (Jürgens et al., 2006; Haskell et al., 2009). Indeed animals with high levels of spontaneous physical activity are partially protected against diet-induced obesity (Teske et al., 2012; Zhang et al., 2012). While pre-existing differences in activity levels may contribute to the relationship between obesity and physical inactivity, at a cellular level it remains unclear why people with obesity are inactive.

Part of the difficulty in understanding this relationship stems from the multifaceted nature of the two variables. For instance, the weight of excess adiposity restricts joint and muscle mobility and increases joint pain, which may make it more difficult for people to move (Belczak et al., 2014; Muramoto et al., 2014). However, weight alone does not appear sufficient to explain physical inactivity in people with obesity. Several researchers have tracked physical activity levels across periods of weight loss, to see whether physical activity levels increase as people lose weight, and experience fewer mobility-restricting effects of excess adiposity. Surprisingly, weight loss is generally associated with decreases, and not increases, in physical activity (de Boer et al., 1986; de Groot et al., 1989; Martin et al., 2007; Redman et al., 2009). These results have been explained in terms of metabolic adaptations, as the body seeks to reduce energy expenditure to compensate for the caloric deficit induced by the diet. However, when subjects were tracked during maintained periods of weight loss lasting a year, physical activity levels still did not increase above pre-diet obese levels (Camps et al., 2013). Similar results have been reported following gastric bypass surgery. Despite large amounts of weight loss (>30 kg), objectively measured physical activity levels did not increase in patients that received gastric bypass surgery, even up to 12 months after the peak of the weight loss (Bond et al., 2010; Ramirez-Marrero et al., 2014; Berglind et al., 2015, 2016). Studies in animals also support these conclusions, as loss of adiposity is again associated with decreases, and not increases, in physical activity (Sullivan and Cameron, 2010; Morrison et al., 2014; Vitger et al., 2016). We conclude that the weight of excess adiposity does not sufficiently explain the association between obesity and physical inactivity. Rather, the evidence suggests that obesity-induced adaptations continue to contribute to physical inactivity, even after weight loss. While these adaptations may include chronic mobility issues in joints or muscles, we hypothesize that motor circuitry in the brain is also a large contributor. Specifically, we hypothesize that deficits in striatal dopaminergic signaling contribute to the persistent reductions in physical activity in obesity.

Further supporting the conclusion that the weight of adiposity does not adequately explain physical inactivity in obesity, not all groups of obese animals, or people with obesity, have low levels of physical activity. Even in studies that report deficits in striatal dopamine, physical activity levels can remain unaltered (Davis et al., 2008). Similar findings have been reported under controlled conditions in humans as well. In an 8-week study in which subjects were over-fed by 1000 calories per day, subjects significantly increased their spontaneous physical activity, despite gaining an average of 4.7 kg. The authors linked this increase to a mechanism for dissipating excess energy to preserve body weight (Levine et al., 1999). A similar increase in physical activity was reported in an 8-week over-eating study, despite an average weight gain of 5.3 kg (Apolzan et al., 2014). While physical inactivity is a correlate of obesity in large populations, there is considerable variability on this point among individuals. This variability may be another avenue for unraveling the cellular underpinnings of the relationship between physical activity and obesity.

# OBESITY AND DISRUPTIONS IN DOPAMINE PRODUCTION AND RELEASE

A wealth of animal research has described alterations in the dopamine system in obesity. The majority of studies in obese rodents have focused on dopamine transmission in the nucleus accumbens (NAc), which resides in the ventral striatum and is involved in effortful movement (Salamone et al., 2007; Schmidt et al., 2012). Based on this role, the NAc may be particularly important for explaining the lack of vigorous physical activity in obesity (Ekkekakis et al., 2016). Long-term ad libitum high-fat diet decreased tonic dopamine in the NAc of mice (Carlin et al., 2013) as well as dopamine turnover in the NAc of rats (Davis et al., 2008). This specific deficit was distinct from adiposity, as rats that were fed an iso-caloric amount of high-fat diet also had decreased dopamine turnover (Davis et al., 2008). Whereas both chow and high-fat diet increased phasic dopamine in the NAc of lean rats, obese rats had a blunted response to these diets (Geiger et al., 2009). Chronic exposure may be necessary for deficits in phasic dopamine signaling, as they are seen following 6, but not 2, weeks of high-fat diet (Cone et al., 2013). Similar to differences observed in phasic dopamine release in the NAc of obese animals, rats that were bred to be prone to weight gain had reduced dopaminergic responses to both chow (Geiger et al., 2008) and high-fat diet (Rada et al., 2010).

The above deficits in dopamine release may be explained by alterations in genes involved in the synthesis and metabolism of dopamine. Midbrain dopamine regions including the substantia nigra and the ventral tegmental area (VTA) provide the main dopaminergic innervation to the striatum (**Figure 1**). Expression of tyrosine hydroxylase, the rate-limiting enzyme in dopamine synthesis, is reduced in the VTA of mice fed a high-fat diet (Vucetic et al., 2012; Carlin et al., 2013). Again, this did not depend on fat storage, as similar effects were observed in mice that were pair-fed a high fat diet (Li et al., 2009). The effect of high-fat diet on co-acetyl methyl transferase (COMT), a key enzyme responsible for the degradation of dopamine is less clear, with studies reporting either decreased (Carlin et al., 2013) or unchanged (Alsio et al., 2010; Vucetic et al., 2012) expression following diet-induced obesity. Interestingly, in humans, polymorphisms that confer low activity of monoamineoxidases (the other main enzyme responsible for degrading dopamine) have been linked to obesity (Camarena et al., 2004; Ducci et al., 2006; Need et al., 2006). Overall, the evidence supports two conclusions: (1) exposure to high-fat diets can impair dopamine synthesis and striatal dopamine release and processing, but (2) heterogeneity exists among these reports, indicating that the impact of high-fat diets on the dopamine system is complex and may occur differently among different individuals.

# OBESITY AND DYSFUNCTION OF DOPAMINE RECEPTORS

Multiple researchers have observed alterations in dopamine receptors in people with obesity. Individuals with at least one copy of the drd2 Taq1A allele have reduced brain D2R availability of ∼30–40% (Noble et al., 1991; Thompson et al., 1997) and an increased prevalence of obesity (Blum et al., 1996; Stice et al., 2008, 2010; Davis et al., 2009; Carpenter et al., 2013). An inverse relationship between obesity and D2R availability, assayed via positron emission tomography (PET), has also been reported in humans. This was first reported by Wang et al. (2001) and was initially supported by others (Volkow et al., 2008; de Weijer et al., 2011; Kessler et al., 2014; van de Giessen et al., 2014). However, several other groups have failed to replicate this finding (Dunn et al., 2012; Caravaggio et al., 2015; Cosgrove et al., 2015; Karlsson et al., 2015, 2016; Tuominen et al., 2015), or found opposing associations in different regions of the striatum (Guo et al., 2014). Interestingly, Guo and colleagues noted a negative relationship between body mass index (BMI) and D2R binding only in the ventral striatum, which may be linked to effortful movements (Salamone et al., 2007; Schmidt et al., 2012). Several possibilities may account for the discrepancy among studies of D2R binding and BMI. Different D2R radio-ligands were used among these studies, which may bind differentially to D2R or D3Rs (Gaiser et al., 2016). Changes in striatal dopamine tone could impact binding potential (Horstmann et al., 2015). Finally, experimental factors including the amount of time after meal consumption or individual variability among subjects may contribute to observed differences (Small et al., 2003).

Animal studies have more consistently linked impairments in D2Rs to obesity, via analysis of mRNA (Mathes et al., 2010; Zhang et al., 2015), protein (Johnson and Kenny, 2010; Adams et al., 2015), and receptor binding (Huang et al., 2006; Hajnal et al., 2008; Thanos et al., 2008; Michaelides et al., 2012; van de Giessen et al., 2012, 2013; Narayanaswami et al., 2013). Interestingly, rats maintained on an iso-caloric high-fat (but not high-sugar) diet also had lower levels of D2Rs in ventral (but not dorsal) striatum (Adams et al., 2015), supporting the conclusion that exposure to high-fat diet may be a better predictor of dopaminergic dysfunction than weight gain itself (van de Giessen et al., 2013). To date, no published work has examined associations between D1-type dopamine receptors (D1Rs) and obesity in humans, so an evaluation of potential changes here is limited to a small number of animal studies. D1R mRNA was decreased in obese rats relative to lean controls (Vucetic et al., 2012; Zhang et al., 2015), while another study reported a decrease in D1Rs only in female rats (Ong et al., 2013). We conclude that reduced function of D2Rs appears to be a particularly important alteration in obesity, although there is considerable variability in D2R alterations among studies and individuals. Unfortunately, studies of the D1R are too sparse to make strong conclusions about its relationship to obesity.

# DO ALTERATIONS IN DOPAMINE FUNCTION RECOVER WITH WEIGHT LOSS?

It is unclear whether changes in dopamine signaling in people with obesity persist after weight loss. The few studies that exist on this topic point to dopaminergic alterations being at least partly resistant to change, and at times even exacerbated by weight loss. High-fat diet reduced the levels of several enzymes involved in dopamine production in the VTA and NAc, and switching these obese mice to low-fat chow caused even further decreases in these enzymes (Carlin et al., 2013; Sharma et al., 2013). Two PET imaging studies reported a lack of recovery of D2R binding following Roux-en-Y gastric bypass surgery (RYGB) in humans, with one showing an even further decrease in binding (Dunn et al., 2010; de Weijer et al., 2014). A small study of five women reported a partial recovery of D2R binding 6-weeks after RYGB (Steele et al., 2010). An increase in D2R binding was also reported during food restriction and associated weight alterations in obese rats (Thanos et al., 2008). Although the data on this topic are limited, it appears that diet-induced changes in dopamine function are at least partly persistent following weight loss. Consistent with this conclusion, physical activity levels remain low in people with obesity, even months after the peak of weight loss (Bond et al., 2010; Camps et al., 2013; Ramirez-Marrero et al., 2014; Berglind et al., 2015, 2016). Again, the small number of studies of this topic precludes firm conclusions, and underscores the need for further research on the persistence of dopaminergic alterations in people with obesity.

# OBESITY AND PHYSICAL INACTIVITY: CONCLUSIONS

Chronic exposure to obesogenic diets is associated with changes in both physical activity levels and dopaminergic function. Dietinduced changes in the dopamine system may be sufficient to explain the development of physical inactivity in people with obesity. Increased understanding of obesity-related changes in dopamine and related systems may support evidence-based approaches for increasing physical activity in people with obesity. In addition, such an understanding may reveal genetic or environmental contributions to dopaminergic dysfunction, and physical inactivity, in obesity.

# AUTHOR CONTRIBUTIONS

AK, TO, and DF conceived of the idea and wrote and edited this manuscript.

# ACKNOWLEDGMENTS

This work was funded by the NIH Intramural research program. We thank Kavya Devarakonda for comments on this manuscript.

#### REFERENCES


after bariatric surgery: preliminary findings. Brain Res. 1350, 123–130. doi: 10.1016/j.brainres.2010.03.064


in healthy human volunteers. Neuroimage 19, 1709–1715. doi: 10.1016/S1053- 8119(03)00253-2


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2016 Kravitz, O'Neal and Friend. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Frequency-Specific Synchronization in the Bilateral Subthalamic Nuclei Depending on Voluntary Muscle Contraction and Relaxation in Patients with Parkinson's Disease

Kenji Kato1,2 , Fusako Yokochi <sup>1</sup> \*, Hirokazu Iwamuro<sup>3</sup> , Takashi Kawasaki <sup>3</sup> , Kohichi Hamada<sup>3</sup> , Ayako Isoo<sup>3</sup> , Katsuo Kimura<sup>1</sup> , Ryoichi Okiyama<sup>1</sup> , Makoto Taniguchi <sup>3</sup> and Junichi Ushiba<sup>2</sup>

<sup>1</sup> Department of Neurology, Tokyo Metropolitan Neurological Hospital, Tokyo, Japan, <sup>2</sup> Department of Biosciences and Informatics, Faculty of Science and Technology, Keio University, Kanagawa, Japan, <sup>3</sup> Department of Neurosurgery, Tokyo Metropolitan Neurological Hospital, Tokyo, Japan

The volitional control of muscle contraction and relaxation is a fundamental component

of human motor activity, but how the processing of the subcortical networks, including the subthalamic nucleus (STN), is involved in voluntary muscle contraction (VMC) and voluntary muscle relaxation (VMR) remains unclear. In this study, local field potentials (LFPs) of bilateral STNs were recorded in patients with Parkinson's disease (PD) while performing externally paced VMC and VMR tasks of the unilateral wrist extensor muscle. The VMC- or VMR-related oscillatory activities and their functional couplings were investigated over the theta (4–7 Hz), alpha (8–13 Hz), beta (14–35 Hz), and gamma (40–100 Hz) frequency bands. Alpha and beta desynchronizations were observed in bilateral STNs at the onset of both VMC and VMR tasks. On the other hand, theta and gamma synchronizations were prominent in bilateral STNs specifically at the onset of the VMC task. In particular, just after VMC, theta functional coupling between the bilateral STNs increased, and the theta phase became coupled to the gamma amplitude within the contralateral STN in a phase-amplitude cross-frequency coupled manner. On the other hand, the prominent beta-gamma cross-frequency couplings observed in the bilateral STNs at rest were reduced by the VMC and VMR tasks. These results suggest that STNs are bilaterally involved in the different performances of muscle contraction and relaxation through the theta-gamma and beta-gamma networks between bilateral STNs in patients with PD.

Keywords: subthalamic nucleus, Parkinson's disease, neuronal oscillations, coherence, cross-frequency coupling, deep brain stimulation

# INTRODUCTION

In the execution of volitional movement in the human motor repertoire, not only voluntary muscle contraction (VMC), but also voluntary muscle relaxation (VMR) is a fundamental component. Previous electrophysiological studies with electroencephalograms (EEGs) in healthy participants reported that muscle relaxation is preceded by a cortical

#### Edited by:

Olivier Darbin, University of South Alabama, USA and National Institute for Physiological Sciences, Japan

#### Reviewed by:

Wolf-Julian Neumann, Charité-University Medicine Berlin, Germany Daniel Dees, University of South Alabama, USA

> \*Correspondence: Fusako Yokochi fyokochi-tmnh@umin.net

Received: 22 December 2015 Accepted: 11 March 2016 Published: 30 March 2016

#### Citation:

Kato K, Yokochi F, Iwamuro H, Kawasaki T, Hamada K, Isoo A, Kimura K, Okiyama R, Taniguchi M and Ushiba J (2016) Frequency-Specific Synchronization in the Bilateral Subthalamic Nuclei Depending on Voluntary Muscle Contraction and Relaxation in Patients with Parkinson's Disease. Front. Hum. Neurosci. 10:131. doi: 10.3389/fnhum.2016.00131 preparatory activity at the primary motor cortex and supplementary motor areas (Terada et al., 1995, 1999; Rothwell et al., 1998; Alegre et al., 2003). Imaging studies with functional magnetic resonance imaging have also demonstrated an increase in the blood-oxygen level-dependent signal, at least in the primary motor cortex and supplementary motor areas, during VMR, probably through the contribution of corticospinal tracts targeting spinal inhibitory interneurons or inhibitory cortical neurons (Terada et al., 1995; Toma et al., 1999; Pope et al., 2007). Despite this evidence for cortical involvement in VMR, the details of the subcortical neural mechanisms of VMR remain largely unknown.

The subthalamic nucleus (STN), which is one of the key subcortical motor centers mediating the cortico-basal gangliathalamo-cortical loop, is well known to play an important role in voluntary movements such as VMC and VMR. Previous electrophysiological studies investigating STN neurons in the monkey during arm movements have found that some neurons show an increased discharge rate, and other neurons show a decreased discharge rate with increased movement amplitude (Georgopoulos et al., 1983). Although large variabilities in positive or negative correlations between the discharge rate and movement amplitude were observed, consistent results from monkey neural recording and human imaging studies have demonstrated that subthalamic activities are scaled with the dynamic parameters of grip force output such as amplitude and rates (DeLong et al., 1984; Spraker et al., 2007; Prodoehl et al., 2009). On the other hand, the STN has also been implicated in suppressing an initiated go response in a stop signal response task (Aron and Poldrack, 2006). This is probably because subthalamic activation leads to a broad inhibition of thalamocortical projections, resulting in a global movement inhibition for stopping impulsive responses (Aron, 2011). Considering previous findings, the STN may be involved in both movement excitation and inhibition. However, the characteristics of human STNs in the fundamental domain of movements such as VMC and VMR are still unknown.

Recently, the characteristics of human STNs has been intensively demonstrated by investigating the movement-related local field potentials (LFPs) recorded from the STN through electrodes implanted for deep brain stimulation (DBS) to treat Parkinson's disease (PD; Kühn et al., 2004; Loukas and Brown, 2004; Androulidakis et al., 2007; Kempf et al., 2007; Ray et al., 2012). Utilizing this opportunity, previous studies have shown that the magnitude of alpha (8–13 Hz) and beta (14–35 Hz) bands in the STN decreases in response to VMC in terms of event-related desynchronization (ERD), and following that, an increase occurs in these frequencies in terms of event-related synchronization (ERS; Kühn et al., 2004; Ray et al., 2012). On the other hand, gamma ERS (40–100 Hz) is also simultaneously observed after movement initiation (Cassidy et al., 2002; Alonso-Frech et al., 2006; Devos and Defebvre, 2006; Androulidakis et al., 2007). These frequency-specific movement-related ERD and ERS occur in the two STNs across hemispheres (Alegre et al., 2005), suggesting that bilateral STNs are involved with the initiation of a unilateral motor program. In addition, the alpha functional coupling between bilateral STNs increases prior to the onset of unilateral movement (Darvas and Hebb, 2014). This finding suggests that frequency-specific bilateral networks between the two STNs are activated for a unilateral motor program of VMC. As for the subcortical role in VMR, one previous study reported that the alpha/beta ERD and subsequent ERS recorded from the contralateral STN occur in both VMC and VMR tasks in patients with PD (Hsu et al., 2012; Tan et al., 2013). However, how functional coupling is bilaterally activated through the different VMC and VMR tasks is largely unclear.

Moreover, recent studies have demonstrated that functional couplings between different frequency bands, in terms of phase-amplitude cross-frequency coupling (PAC), may play a physiologically fundamental role in regulation of motor behavior (de Hemptinne et al., 2013; von Nicolai et al., 2014). In the ''OFF'' medication state in patients with PD, gamma oscillation is entrained by the phase of excessive beta oscillation in the STN and sensorimotor cortex (López-Azcárate et al., 2010; Yang et al., 2014; de Hemptinne et al., 2013, 2015). This beta-gamma PAC is reduced by medication (López-Azcárate et al., 2010), DBS treatment (de Hemptinne et al., 2013, 2015), and voluntary movement (de Hemptinne et al., 2013, 2015), suggesting that it may reflect the neural constrained states of an inflexible pattern of corticosubthalamic activities, leading to parkinsonian motor symptoms such as bradykinesia and rigidity. On the other hand, other researchers have demonstrated that in the striatum of healthy rats, increased gamma activities during motor behavior are entrained not by the beta phase but the theta phase, suggesting that the theta-gamma PAC may be directly related to the functional domain of motor behavior in the cortico-basal ganglia system (von Nicolai et al., 2014).

From these previous findings, frequency-specific subthalamic activities and their functional couplings, in terms of coherence or PAC, are differently involved in VMC and VMR, although the details of the distinct characteristics of each frequency component (i.e., theta, alpha, beta, and gamma bands) are largely unknown. Our motivation for this study was to clarify the distinct features of the subthalamic oscillations and their functional couplings over the theta, alpha, beta, and gamma ranges during the different motor controls of VMC and VMR in patients with PD. In this study, VMC- and VMR-related synchronization and desynchronization in the bilateral STNs were investigated when patients were performing externally paced VMC and VMR tasks of unilateral wrist extensor muscles. Coherence was analyzed to evaluate the movement-related functional coupling between the bilateral STNs in response to VMC and VMR tasks. Moreover, the PAC was analyzed to investigate the crossfrequency interaction in the bilateral STNs depending on the different motor processes of the VMC and VMR tasks.

#### MATERIALS AND METHODS

#### Participants and Surgical Procedure

The participants were seven right-handed PD patients (4 males and 3 females; age, 57.0 ± 8.6 years; disease duration,

#### TABLE 1 | Clinical characteristics of Parkinson's disease (PD) patients.


UPDRS: Unified Parkinson's disease rating scale, On: on phase of wearingoff phenomenon, Off: off phase of wearing-off phenomenon, LED: Levodopa equivalent dose. LED 100 mg = 100 mg standard levodopa = 133 mg of controlledrelease levodopa = 10 mg bromocriptine = 1 mg pergolide = 1 mg premipexole = 5 mg ropinirole.

15.2 ± 5.3 years) undergoing surgery for implantation of DBS electrodes into the STN. These patients corresponded to Patients 4, 5, 6, 7, 8, 10, and 11 in the previous work by Kato et al. (2015). All patients received bilateral electrodes. The present study was approved by the ethics committees of the Tokyo Metropolitan Neurological Hospital (Tokyo, Japan) and the Faculty of Science and Technology, Keio University, Kanagawa, Japan. All patients provided their written informed consent before the recording. Each patient was diagnosed as having PD by the UK PD Society Brain Bank clinical diagnostic criteria (Daniel and Lees, 1993). Stereotactic surgery was performed according to the following procedures reported previously (Kato et al., 2015). Patients were withdrawn from their anti-Parkinsonian medication at least 12 h prior to the recordings, and the recordings were performed with the patients in the ''OFF'' medication state. The details of medication and Unified PD rating scale (UPDRS) scores in each patient are summarized in **Table 1**.

# Local Field Potentials and Electromyogram (EMG) Recordings

LFPs in bilateral STNs were recorded from the four contacts of each DBS electrode (Model 3389; Medtronic Neurological Division, Minneapolis, MN, USA) referenced to linked ears 2 days after the operation. The surface electromyogram (EMG) was simultaneously recorded bilaterally using pairs of Ag/AgCl disk electrodes (10 mm diameter) attached to the muscle bellies (20 mm interelectrode distance) of the following muscles: sternocleidomastoid, biceps brachii, triceps brachii, flexor carpi radialis, and extensor carpi radialis. Then, LFPs and EMG data were band-pass filtered (LFP data, 0.5–400 Hz; EMG data, 20–400 Hz) and digitized (sampling frequency, 1000 Hz) using a biosignal recorder (Neurofax EEG 1200; Nihon Kohden Corporation, Tokyo, Japan). Data were analyzed using Matlab (Mathworks, Natick, MA, USA). STN LFPs were calculated for DBS contact pairs 0–1, 1–2, and 2–3 by subtracting the signal of the contact pair from the signal of the adjacent contact pair.

#### Movement Task

Patients were awake and instructed to lie in the supine position with their eyes closed. To confirm that patients made no involuntary movements during the task, the EMG was monitored in all patients throughout the experiments. Taking into account the feasibility of the movement task, each patient performed the movement task using the less-affected side (4 patients on the right side; 3 patients on the left side). They were asked to perform tonic extension and relaxation of the wrist. After an auditory cue, they dorsiflexed the wrist as quickly as possible and sustained the posture by contracting the wrist extensor muscles for 10 s. The duration of the auditory cue for the initiation of VMC was 1 s. During this phase, the patients were instructed not to contract muscles in other limbs. After the VMC phase for 10 s, a sound in another pitch was generated, and the patients were then instructed to place the palm back into the original flat position by relaxing their wrist muscles as soon as possible. The duration of the auditory cue for the initiation of VMR was 1 s. VMC and VMR sessions were repeated over 30 trials. The onset of muscle contraction and relaxation was defined as the time to exceed five standard deviations (SD) of the baseline muscle activities, averaged from 4 s to 2 s before the VMC onset. In six out of seven patients, the timing of the cue signal was simultaneously recorded, and the response time of muscle contraction and relaxation was calculated for each patient.

#### Power Spectral Analysis

To evaluate the frequency-specific power changes in bilateral STNs, the discrete Fourier transform, calculated in accordance with the method of Halliday et al. (1995), was used for power spectral analysis. The LFP data recorded from the adjacent contacts of the electrodes (i.e., contacts 0–1, 1–2, and 2–3) for each trial were segmented into epochs of 1024 data points of the same duration of 1.024 s, yielding a frequency-resolution of 0.987 Hz. Each data segment was Hanning-windowed to reduce spectral leakage. The VMC- or VMR-related oscillatory changes in the bilateral STNs were investigated by computing the time-frequency map of the power spectrum over a sliding short-time (1.024 s) window moved in 0.05 s steps. As for the time periods for the analysis of power changes in the VMC and VMR tasks, 8 s durations ranging from 4 s before to 4 s after the VMC or VMR task were extracted in each task. Movementrelated power was then averaged across contacts and trials, and the percentage values were calculated in relation to the baseline period, which was defined as 100%, ranging from 4 s to 2 s before the VMC.

The frequency range for the latter quantitative analyses was set at 2–100 Hz, including the theta (4–7 Hz), alpha (8–13 Hz), beta (14–35 Hz), and gamma (40–100 Hz) bands. The periods used in the statistical procedure were segmented as follows: from 0 s to 1 s after the VMC and VMR as the duration of the movement, and from 2 s to 4 s after VMC as the duration of holding VMC. Movement-related power changes in the respective frequency bands were assessed using factorial analysis of variance (ANOVA) with Bonferroni adjustment for multiple comparisons. The four factors were patient, period (i.e., 0–1 s after VMC, 2–4 s after VMC, and 0–1 s after VMR), side (i.e., ipsilateral and contralateral sides), and frequency (i.e., theta, alpha, beta, and gamma bands).

#### PAC

To investigate the cross-frequency interaction during the VMC and VMR tasks, PAC between phase values of the low frequency including the theta, alpha, and beta ranges and the amplitude value of high frequency of the gamma range components was evaluated based on the methods developed by Canolty et al. (2006). The amplitude of phase-amplitude coupling was determined by computing a phase-modulation-index (MI) based on mutual information as described in Hurtado et al. (2004) and Tort et al. (2010). The periods used in the statistical procedure were segmented as follows: from −2 s to −1 s of the VMC onset as the duration of the resting state, from 0 s to 1 s after the VMC and VMR as the duration of the movement, and from 2 s to 4 s after VMC as the duration of holding VMC. Then, the phase-MI was averaged across contacts and patients. Movement-related changes in the phase-MI were assessed using factorial ANOVA with Bonferroni adjustment for multiple comparisons. The four factors were patient, period, side, and frequency.

## Coherence

To investigate the functional coupling between the bilateral STNs in the VMC and VMR tasks, coherence (Rosenberg et al., 1989; Halliday et al., 1995; Ushiyama and Ushiba, 2013; Kato et al., 2015) was evaluated between the following couplings: left STN contacts 3–2 vs. right STN contacts 3–2, left STN contacts 2–1 vs. right STN contacts 2–1, and left STN contacts 1–0 vs. right STN contacts 1–0. To minimize the false connectivity arising from volume conduction, the imaginary part of coherence was calculated as true functional coupling between bilateral STNs during VMC and VMR tasks (Nolte et al., 2004). Time-dependent imaginary coherence was calculated by integrating 1 s time windows across 30 trials (i.e., total time length was 30 s). Time-dependent coherence was then averaged across couplings and computed from −2 s to 4 s after the onset of VMC and VMR, in 0.05 s steps. The threshold for significant coherence at p < 0.05 was calculated based on the standard method developed by Halliday et al. (1995).

# RESULTS

In this study, the movement-related STN oscillations and their functional couplings between bilateral STNs during externally paced VMC and VMR of the wrist were investigated in seven PD patients. During the movement task, no involuntary movements such as tremors were observed over the recorded muscles in any patient. The activities of the wrist extensor were modulated in response to the onset of VMC (**Figure 1Aa**) and VMR (**Figure 1Ab**). The response times for the onset of VMC and VMR were 327.1 ± 83.1 ms and 363.8 ± 121.5 ms in the VMC and VMR tasks, respectively. Following VMC and VMR, the power in the bilateral STNs recorded from each adjacent contact pair (i.e., Contacts 1–0, 2–1, 3–2) showed frequency-specific ERS or ERD over the theta (4–7 Hz), alpha (8–13 Hz), beta (14–35 Hz), and gamma (40–100 Hz) bands across bilateral STNs around the time of onset of VMC (**Figure 1Ba**) and VMR (**Figure 1Bb**). Overall, alpha and beta ERDs were clearly observed in each contact of the bilateral STNs in both the VMR and VMC tasks. At the same time, theta and gamma ERSs also appeared especially in the VMC task. We observed no significant changes in the magnitudes of ERD or ERS among the contact pairs of Contacts 3–2, 2–1, and 1–0 (0–1 s after the onset of VMC or VMR). To further characterize the observed frequency-specific ERD and ERS, the time-dependent power changes in the bilateral STNs were compared between the phases of VMC and VMR, divided into the alpha and beta ERDs and the theta and gamma ERSs, as below.

# ERD in the Alpha and Beta Frequency Bands

The averaged powers in the bilateral STNs across patients and contacts (i.e., Contacts 0–1, 1–2, and 2–3) in the alpha and beta bands were plotted against the onset of VMC and VMR (**Figures 2A,B**). The alpha ERD was observed bilaterally in both VMC and VMR tasks (**Figures 2Aa,b,Ca**). The magnitude of the alpha ERD was not different between ipsilateral and contralateral STNs in the VMC and VMR tasks. In the ipsilateral STN, however, the alpha ERD after VMC was shifted to ERS after 2 s of VMC (p < 0.05, **Figure 2Ca**).

In the beta band, ERD and the subsequent ERS (often described as ''beta rebound'') were bilaterally observed after the VMC and VMR tasks (**Figures 2Ba,b**). The magnitudes of ERDs in the bilateral STNs 1 s after onset were greater in the VMR than in the VMC task (p < 0.05, **Figure 2Cb**). On the other hand, the beta rebound 2–4 s after VMC onset was greater in the ipsilateral STN than in the contralateral STN (p < 0.05, **Figure 2Cb**). Overall, the alpha and beta ERDs in bilateral STNs appeared consistently after the VMC and VMR tasks.

#### ERS in the Theta and Gamma Frequency Bands

As well as the ERD in the alpha and beta bands, ERS in the theta and gamma bands emerged in the bilateral STNs after the VMC and VMR tasks, as shown in **Figure 3**. In the theta band, the ERS in bilateral STNs occurred transiently just around the onset of the VMC and VMR tasks (**Figures 3Aa,b**). However, the magnitude of theta ERS was greater in the VMC than in the VMR task (p < 0.01, **Figure 3Ca**), although these were not significantly different between the ipsilateral and contralateral STNs in each task (**Figure 3Ca**).

In the gamma band, the ERS was clearly observed in the bilateral STNs after the VMC task, but not after the VMR task (**Figures 3Ba,b,Cb**). Unlike the transient increase in the theta

ERS, the gamma ERS after VMC increased continuously over 4 s after the onset. The magnitude of gamma ERS in the VMC task was not significantly different between the ipsilateral and contralateral STNs (**Figure 3Cb**).

bilateral STNs in both VMR and VMC tasks. Theta/gamma synchronization also appears, especially in the VMC task.

Overall, theta ERSs were bilaterally present in the VMC and VMR tasks, but they were more prominent in the VMC than in the VMR tasks. On the other hand, the bilateral gamma ERSs were continuously observed only in the VMC task.

# Theta-Gamma and Beta-Gamma Phase-Amplitude Coupling

The time-dependent power changes in bilateral STNs clearly showed that the theta/gamma ERSs occurred simultaneously in response to the onset of the VMC and VMR tasks. In such cases, the ERSs in the theta and gamma bands may interact with each other via cross-frequency coupling, as previously demonstrated in the corticostriatal axis during motor behavior in rats (von Nicolai et al., 2014). To investigate the movementrelated PAC over broad frequency bands, the PAC between the phases of low-frequency bands (i.e., theta, alpha, beta bands) and the amplitude of high-frequency bands (i.e., gamma band) were computed in bilateral STNs during the VMC and VMR tasks.

A significant increase in theta-gamma PAC was found in the contralateral STN during 0–1 s after the VMC phase (**Figure 4A**). In particular, during this period of VMC phase, the theta phase was interacted with the amplitude of lower part of the gamma band (i.e., 40–60 Hz). Across patients and contact pairs (i.e., Contacts 0–1, 1–2, 2–3), the averaged

represents 100% of the relative power in the bilateral STNs. Overall, the beta ERD and subsequent event-related synchronization (ERS) appear in bilateral STNs in response to VMC and VMR. (C) The magnitudes of the synchronization and desynchronization in the contralateral STN (black) and the ipsilateral STN (gray) in each of the selected time periods (i.e., 0–1 s before VMC, 2–4 s after VMC, and 0–1 s after VMR) are shown in the alpha (a) and beta (b) bands. The asterisks (<sup>∗</sup> , ∗∗) indicate a significant difference in relative power. Error bars represent standard errors.

phase-MI of theta-gamma PAC increased significantly in the contralateral STN during 0–1 s after VMC (**Figure 4B**). For the ipsilateral STN, we observed no significant change in thetagamma PAC during VMC. On the other hand, in the VMR phase, significant changes in the theta-gamma PAC were not found in the bilateral STNs (**Figure 4B**), suggesting that the theta and gamma oscillations were jointly coupled via phase-amplitude cross-frequency interactions within the contralateral STN just around the onset of VMC.

In addition to the VMC-related change in the theta-gamma PAC, the beta-gamma PAC was also observed in the STN at rest (**Figure 4Ca**). In particular, the phase of the low beta band (i.e., 14–20 Hz) was interacted with the amplitude of higher part of the gamma band (i.e., 80–100 Hz) at rest. In addition, this betagamma PAC decreased in each time period of the movement phase of VMC (i.e., 0–1 s after VMC), the holding phase of VMC (i.e., 2–4 s after VMC), and movement phase of VMR (i.e., 0–1 s after VMR; **Figures 4Cb,c**). The decreases in the betagamma PAC during VMC and VMR tasks occurred bilaterally (**Figure 4D**).

Overall, the gamma oscillations were entrained by the beta phase at rest. However, this beta-gamma PAC decreased with VMC and VMR tasks. In particular, the gamma oscillations in the contralateral STN switched from being entrained from the

synchronization and desynchronization in the contralateral STN (black) and the ipsilateral STN (gray) in each of the selected time periods (i.e., 0–1 s before VMC, 2–4 s after VMC, and 0–1 s after VMR) are shown in the theta (a) and gamma (b) bands. The asterisks (∗∗) indicate a significant difference in relative power. Error bars represent standard errors.

beta phase to being entrained from the theta phase just after the VMC phase.

#### Coherence Between Bilateral STNs

Similar features in the movement-related power changes in the bilateral STNs further suggested the synchronous relationship between ipsilateral and contralateral STNs over the selected frequency bands. These synchronizations may occur by functional coupling between bilateral STNs. To demonstrate the functional connectivity between bilateral STNs, the time-dependent change in coherence between ipsilateral and contralateral STNs in response to VMC and VMR onset was computed. To further characterize the directionality of the functional coupling between bilateral STNs, the time-dependent changes in Granger causality were evaluated.

The time-frequency mappings of VMC- and VMR-related imaginary coherence between bilateral STNs were plotted (**Figures 5Aa,b**). These showed that the coherence between

FIGURE 4 | Theta-gamma and beta-gamma phase-amplitude cross-frequency couplings (PAC) in the bilateral STNs related to VMC and VMR tasks. (A) Representative color maps of the theta-gamma phase-modulation-index (MI) in the contralateral STN in the selected time periods before and after the VMC task (i.e., 1–2 s before VMC (a), 0–1 s after VMC (b), and 2–4 s after VMC (c)) are shown. The increase in the theta-gamma PAC can be observed after 0–1 s of the VMC onset. (B) The averaged MIs between the theta and gamma bands in the contralateral STN (black) and ipsilateral STN (gray) are plotted in the selected time periods of the VMC and VMR tasks (i.e., 1–2 s before VMC, 0–1 s after VMC, 2–4 s after VMC, and 0–1 s after VMR). Error bars represent standard errors. Horizontal bars indicate a significant difference in MI (p < 0.05). (C) The representative color maps of the beta-gamma MI in the contralateral STN in the selected time periods before and after the VMC task (i.e., 1–2 s before VMC (a), 0–1 s after VMC (b), and 2–4 s after VMC (c)) are shown. The decrease in the beta-gamma PAC after 0–1 s of VMC onset can be observed. (D) The averaged MIs between the beta and gamma bands in the contralateral STN (black) and ipsilateral STN (gray) are plotted in the selected time periods before and after the VMC task (i.e., 1–2 s before VMC, 0–1 s after VMC, 2–4 s after VMC, and 0–1 s after VMR). Error bars represent standard errors. Horizontal bars indicate significant difference in MI (p < 0.05).

ipsilateral and contralateral STNs (iSTN-cSTN coherence) in the theta band increased transiently just around the onset of the VMC and VMR tasks (**Figures 5Aa,b**). In the other frequency bands, we found no significant coherence over time. The time-dependent change in theta coherence is shown in **Figures 5Ba,b**, suggesting that peaks of theta iSTN-cSTN

coherence were more prominent after the onset of VMC than VMR (p < 0.05).

# DISCUSSION

This study demonstrated that bilateral STNs modulated their frequency-specific oscillatory activities in response to VMC and VMR in PD patients. The alpha/beta ERDs in the bilateral STNs occurred during both VMC and VMR tasks, whereas the theta/gamma ERSs in the bilateral STNs were prominent specifically in the VMC task. Particularly, just after the VMC task, the theta functional coupling between the bilateral STNs increased transiently, during which the gamma activities in the contralateral STN were entrained by the theta phase via PAC. These results suggest that bilateral STNs are closely involved in the execution of both VMC and VMR through the theta-gamma networks between bilateral STNs.

# ERDs in Alpha and Beta Bands Over Bilateral STNs During Voluntary Muscle Contraction and Relaxation Tasks

In this study, the alpha/beta ERDs and subsequent ERSs occurred similarly in response to externally paced VMC and VMR tasks. Similar movement-related STN oscillatory changes in the alpha/beta bands have been extensively described in previous studies using VMC (Kühn et al., 2004; Loukas and Brown, 2004; Kempf et al., 2007; Tan et al., 2013) and VMR (Hsu et al., 2012; Tan et al., 2013) tasks in patients with PD. As with the self-paced movements, alpha/beta ERDs are generally believed to occur prior to the onset of the VMC (Cassim et al., 2000) and VMR (Hsu et al., 2012), suggesting that the STN has a feed-forward role in the voluntary control of muscle contraction and relaxation in patients with PD. In particular, the alpha/beta ERDs prior to the onset of VMR suggest that the STN plays an active role in the inhibition of on-going muscle contraction. We also confirmed that the antagonist wrist flexor was almost silent throughout the experiment. Thus, VMR-related oscillatory changes likely come mainly from relaxation of the wrist extensor and minimally from contraction of the wrist flexor muscle via reciprocal inhibition during VMR.

These movement-related alpha/beta ERDs also occur at least in the contralateral primary motor cortex and supplementary motor area before and during VMC (Toro et al., 1994; Ohara et al., 2000) and VMR (Labyt et al., 2006) tasks in healthy subjects. From these findings, the emergence of alpha/beta ERDs in the STN may reflect a physiological phenomenon, rather than being a characteristic marker in patients with PD. However, of note, in PD patients in the ''OFF'' medication state, the ERD in the alpha band starts later (Defebvre et al., 1996; Magnani et al., 1998), but becomes nearly normal in the ''ON'' medication state (Defebvre et al., 1998; Magnani et al., 2002) or when treated with STN DBS (Devos et al., 2004). In agreement with these studies, the present results during the ''OFF'' medication state showed a delayed response of alpha ERD in both the VMC and VMR tasks, supporting the idea that the delayed alpha ERD may reflect a pathophysiological feature of PD.

Additionally, alpha/beta oscillatory changes were present bilaterally in response to VMC as well as VMR, suggesting that the STNs were bilaterally involved with different unilateral motor programs of VMC and VMR. Indeed, as with VMR, previous studies have confirmed that in elderly healthy subjects compared to young healthy subjects, the distribution of alpha/beta ERDs is more widespread over the bilateral sensorimotor cortex, beginning earlier over contralateral frontocentral and parietocentral regions (Derambure et al., 1993; Labyt et al., 2004). These observations indicate that aging can also be associated with bilateralization of the alpha/beta ERDs, probably due to a compensatory strategy to correct the motor program. Moreover, another study demonstrated that even in healthy subjects, the alpha ERD recorded over the sensorimotor cortex by EEG or magnetoencephalograms (MEG) has contralateral preponderance during the early premovement phase, and then becomes symmetrically bilateral near the beginning of the movement (Derambure et al., 1993). These results may indicate that the motor commands for unilateral movement can be sent in a parallel processing manner between bilateral cortical levels. Anatomical evidence has demonstrated that the STN has rich connections to the motor cortex via cortical inputs to the striatum and the external segment of the globus pallidus via the ''indirect pathway'', and without relay via the ''hyper-direct pathway'' (Alexander et al., 1986; Alexander and Crutcher, 1990; Nambu et al., 1996), whereas subthalamic output reaches the motor cortex via the ventrolateral thalamus (Hoover and Strick, 1999). As a consequence of these findings, a unilateral motor program such as VMC and VMR may be processed bilaterally both at cortical and subcortical levels such as the cortico-basal ganglia-thalamo-cortical loops across bilateral hemispheres.

# Synchronization in the Theta and Gamma Bands Over the Bilateral STNs

In addition to the alpha/beta ERD, the theta/gamma ERS was simultaneously observed specifically in response to the VMC task. The gamma ERS has been thought to arise from increasing asynchronous spiking activity of the STN local neuronal population (Ray et al., 2008; Manning et al., 2009; Miller et al., 2010; Ray and Maunsell, 2011), although another study demonstrated that the firing of neurons in the upper STN and the bordering zona incerta tends to be locked to gamma activity in the LFP (Trottenberg et al., 2006). In either case, the present results of VMC-specific theta/gamma ERS suggest that VMC may require more spike excitation than VMR, indicating that novel processes relevant to VMC and VMR were elicited in the bilateral STNs.

VMC-related theta/gamma ERS was also present bilaterally, as confirmed in previous studies (Alegre et al., 2005). However, the gamma ERS over the motor cortex, recorded from electrocorticography, was observed during movements in the contralateral side, but not the ipsilateral side (Crone et al., 1998). This result implies that VMC-specific theta/gamma oscillatory changes in bilateral STNs could be explained by bilateral cortico-striatal projections (Rouzaire-Dubois and Scarnati, 1985; Mouroux et al., 1995; Parent and Hazrati, 1995), assuming that the high gamma activity was generated primarily in the contralateral motor cortex in response to VMC.

One of the novel findings of the present study was that theta and gamma oscillations interacted with each other via cross-frequency coupling in the contralateral STN during the VMC task. This means that the phase of theta oscillation in the contralateral STN entrained the gamma amplitude, especially during the initiation of VMC. Originally, the idea of PAC between theta low frequency and gamma high frequency was thought to be a general mechanism for regulation of the neural ensemble in various functions such as working memory, visual perception, and reinforcement learning (Lakatos et al., 2005; Canolty et al., 2006; Siegel et al., 2009; Fujisawa and Buzsáki, 2011; Spaak et al., 2012; de Hemptinne et al., 2013). Recently, von Nicolai et al. (2014) have further shown that corticostriatal theta and gamma oscillations are strongly modulated via cross-frequency coupling by simple motor behavior in the rat, suggesting that the coordination of fast gamma oscillations through coherent phase-amplitude coupling may be a general mechanism for regulation of motor behavior. In excellent agreement with these studies, the present study first demonstrated that such theta-gamma cross-frequency coupling was also prominent within the contralateral STN and was especially related to the initiation of VMC in patients with PD.

In addition to movement-related theta-gamma PAC, we also confirmed movement-related decreases in the beta-gamma PAC in the bilateral STNs, as reported in previous studies (de Hemptinne et al., 2013, 2015). From the abundant evidence, the abnormal excessive beta oscillations in the STN at rest may be characteristic of the ''OFF'' medication state in PD (Brown, 2003; Bronte-Stewart et al., 2009; Kato et al., 2015). Thus, several previous studies have suggested that prominent beta-gamma PAC at rest reflects the inflexible and locked neural states in the STN, which may be related to parkinsonian motor impairments (de Hemptinne et al., 2013, 2015). After movement initiation, however, the neural constrained states due to the beta-gamma PAC over the bilateral STNs were temporarily canceled, and instead, the theta-gamma PAC emerged within the contralateral STN to accomplish VMC. To our knowledge, this is the first study to show that betagamma and theta-gamma PACs exist simultaneously in the STN in PD and are differentially modulated by VMC and VMR movements. However, whether the observed movement-related theta-gamma PAC in the STN plays a physiological role in regulation of voluntary movement remains unclear, because we recorded subthalamic activities in the ''OFF'' medication state. Therefore, the present findings may be specific to PD. Nevertheless, well-known anatomical evidence suggests that the striatum has rich connections to the STN via the ''indirect pathway'' and the ''direct pathway'', as described above. Based on this evidence, the present findings may suggest that such a VMC-related increase in theta-gamma PAC may occur within the corticostriatal axis (von Nicolai et al., 2014), as well as the STN, to regulate motor behavior in a physiological manner, rather than being a pathophysiological biomarker in PD.

Additionally, theta functional coupling between bilateral STNs was found to increase just around the onset of VMC. As yet, no physiological evidence exists that shows a direct connection between bilateral STNs. From this evidence, bilateral functional coupling in the theta band may occur from the influence of a third nucleus other than the two STNs. For example, the thalamus is a candidate because it is well connected to multiple nuclei in the basal ganglia, and therefore, is capable of driving phase synchrony between bilateral STNs. Interestingly, although synchronized theta oscillation between the bilateral STNs was observed, thetagamma cross-frequency coupling only emerged within the contralateral STN. The mechanism of the selected crossfrequency coupling between ipsilateral and contralateral STNs is still unclear. Further studies to elucidate this mechanism are needed. The present study, however, demonstrated that even though theta-gamma ERS occurred similarly in the ipsilateral and contralateral STNs, the theta-gamma interaction became selective to the contralateral STN just around the onset of VMC and may play a key role in the initiation of the movement.

# Study Limitations

Our experimental approach has some important limitations. First, the present findings are difficult to directly compare with normal subjects due to the invasive nature of LFP recordings in the STN. Recording in the ''ON'' medication state would have improved the generalizability. However, our research opportunity was limited during the ''OFF'' medication state in which the subthalamic activity would be severely impaired. Therefore, the present findings could be characteristic of PD. Although our study suggested that frequency-specific STN oscillatory changes are differentially involved in VMC and VMR in the ''OFF'' medication state in PD, further comparisons between the ''ON'' and ''OFF'' medication states will be necessary to explore the functional meaning of each frequency oscillatory change.

Second, although group level analysis was performed in this study because consistent results were found, the number of patients should be increased to allow more robust statistics, especially for the PAC and coherence analysis. As for the movement-related changes in PAC and coherence, these changes were demonstrated after we carefully checked for similar robustness in previous findings about VMC-related alpha/beta ERDs and theta/gamma ERSs. Therefore, we feel that our findings about PAC and coherence represent a generalizable phenomenon.

Lastly, from the anatomical perspective, the STN receives many types of inputs: GABAergic and inhibitory inputs from the globus pallidus, excitatory and glutamatergic inputs from the cerebral cortex in the motor cortex, and neuromodulatory inputs from dopaminergic axons from the substantia nigra pars compacta (Alexander et al., 1986 and Alexander and Crutcher, 1990). Considering these findings, the present results of the STN oscillatory patterns modulated by VMC and VMR may reflect the fact that individual types of excitatory and inhibitory synaptic connections contribute differentially to each of the executions of VMC and VMR. Of course, elucidating the functional meaning of each frequency-specific oscillation in this study is difficult, but will be clarified by combining the many different types of movement experiments, such as voluntary vs. passive movements, self-paced vs. externally paced voluntary movements, or movement with preparation vs. without preparation.

# AUTHOR CONTRIBUTIONS

KK and FY designed research, KK and FY performed the experiment. FY, HI, TK, KH, AI, KK, RO, and MT performed the operation and treated patients pre-and post-operation. KK analyzed the data, KK, FY, JU wrote the article.

# REFERENCES


primary motor cortex in Parkinson disease. Proc. Natl. Acad. Sci. U S A 110, 4780–4785. doi: 10.1073/pnas.1214546110


in the human subthalamic nucleus may be a pathophysiological mechanism in Parkinson's disease. J. Neurosci. 30, 6667–6677. doi: 10.1523/JNEUROSCI. 5459-09.2010


**Conflict of Interest Statement**: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2016 Kato, Yokochi, Iwamuro, Kawasaki, Hamada, Isoo, Kimura, Okiyama, Taniguchi and Ushiba. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution and reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# More than a Rumor Spreads in Parkinson's Disease

Natalia C. Prymaczok <sup>1</sup> , Roland Riek <sup>2</sup> and Juan Gerez <sup>2</sup> \*

<sup>1</sup> Laboratorio de Neurofisiología del Instituto Multidisciplinario de Biología Celular, Argentine Research Council (CONICET), National University of La Plata and Scientific Research Commission, Province of Buenos Aires (CIC-PBA), La Plata, Buenos Aires, Argentina, <sup>2</sup> Laboratory of Physical Chemistry, D-CHAB, ETH Zurich, Zurich, Switzerland

As Parkinson's disease progresses, a massive loss of dopaminergic neurons is accompanied by accumulation of alpha-Synuclein (αSyn) neuronal inclusions called Lewy bodies and Lewy neurites. Inclusions first appear in olfactory bulb and enteric neurons then in ascendant neuroanatomical interconnected areas, and finally, in late stages of the disease, Lewy bodies are observed in a substantia nigra pars compacta with clear signs of neuronal loss. It is believed that the spreading of Lewy bodies through the nervous system is a consequence of the cell-to-cell propagation of αSyn, that can occur via sequential steps of secretion and uptake. Certain pathological forms of transmitted αSyn are able to seed endogenous counterparts in healthy recipient cells, thus promoting the self-sustained cycle of inclusion formation, amplification and spreading, that ultimately underlies disease progression. Here we review the cell-to-cell propagation of αSyn focusing on its role in the progression of Parkinson's disease.

#### Edited by:

Marcelo Merello, Fundación para la Lucha contra las Enfermedades Neurológicas de la Infancia, Argentina

#### Reviewed by:

Kristina Aurousseau, Centre de recherche de l'Institut universitaire de gériatrie de Montréal (CRIUGM), Canada Martin Radrizzani, National University of San Martin, Argentina

#### \*Correspondence:

Juan Gerez juan.gerez@bc.biol.ethz.ch

Received: 29 August 2016 Accepted: 14 November 2016 Published: 02 December 2016

#### Citation:

Prymaczok NC, Riek R and Gerez J (2016) More than a Rumor Spreads in Parkinson's Disease. Front. Hum. Neurosci. 10:608. doi: 10.3389/fnhum.2016.00608 Keywords: Parkinson's disease, alpha-Synuclein, neurodegeneration, cell-to-cell propagation, Lewy bodies, prion-like diseases

# ALPHA-SYNUCLEIN AND PARKINON'S DISEASE

Parkinson's disease (PD) is a complex degenerative disorder that is pathologically characterized by a massive loss of dopaminergic neurons in the substantia nigra pars compacta (SNpc) and the progressive accumulation of Lewy bodies and Lewy neurites (LBs/LNs), two forms of inclusions rich in filaments of aggregated alpha-Synuclein (αSyn) (Spillantini et al., 1997). Although a causative role remains to be formally established, the facts that LBs/LNs are present in virtually all sporadic and familial forms of PD (Poulopoulos et al., 2012), that point mutations and multiplications of the αSyn-encoding gene, SNCA, lead to early onset PD (Polymeropoulos et al., 1997; Krüger et al., 1998; Singleton et al., 2003; Chartier-Harlin et al., 2004; Zarranz et al., 2004) and that SNCA polymorphisms positively correlate with PD risk (Satake et al., 2009; Simón-Sánchez et al., 2009; Edwards et al., 2010) attest an irrefutable link between PD and αSyn.

Since the discovery that αSyn is abundant in LBs in the late 90s, a tremendous effort has been made to determine the precise 3D conformations adopted by this protein under physiological conditions. It is clear now that in aqueous solution αSyn behaves as an intrinsically disordered protein, lacking a defined or stable structure (Uversky and Eliezer, 2009; Drescher et al., 2012). Although still a matter of extensive debate, an emerging consensus indicates that within healthy cells αSyn exists as soluble low molecular weight species that play important roles in intra and extracellular vesicle trafficking and dynamics (Burre et al., 2010; Bartels et al., 2011; Fauvet et al., 2012; Theillet et al., 2016). In disease-related contexts, however, αSyn is also found as β-sheetenriched amyloid aggregates that reside within and constitute the building blocks of LBs/LNs (Spillantini et al., 1997; Baba et al., 1998; Conway et al., 1998, 2000). Compelling evidence indicates that the culprits of toxicity are oligomers and higher order assemblies of αSyn such as amyloid fibrils (El-Agnaf et al., 1998; Winner et al., 2011; Rockenstein et al., 2014).

# THE CELL-TO-CELL TRANSMISSION OF αSYN

Early neuroanatomical studies conducted mainly by Braak and co-workers revealed that LBs appear first in the olfactory bulb and enteric neurons and that only after several years they are found in certain areas of the midbrain such as SNpc and eventually neocortex (Wakabayashi et al., 1988; Braak et al., 2003a, 2006; Braak and Del Tredici, 2008). Thus, during the progression of the disease, LBs are found in a stereotypical and topographical distribution in the nervous system. This highly predictable pattern of LB distribution was not taken into deep consideration until the subsequent discovery (in 2008) that healthy neurons would acquire LBs when grafted into the brains of PD patients (Kordower et al., 2008a,b; Li et al., 2008, 2010). A few years later, the demonstration that αSyn is transmitted from cell-to-cell led to the unifying hypothesis that the transcellular transmission of certain forms of αSyn underlies LB pathogenesis and spreading, and by extension, PD progression (Dunning et al., 2012). This hypothesis was originally supported by clinical evidence suggesting "host-to-graft" transmission of pathological αSyn forms: when embryonic mesencephalic neurons were grafted into PD patient's brains, they developed LBs several years after grafting (Kordower et al., 2008a,b; Li et al., 2008, 2010). The in vitro evidence supporting the cell-to-cell propagation of αSyn is its release by unconventional secretion (Emmanouilidou et al., 2010) and the uptake of extracellular αSyn (both natural and recombinant forms) by active mechanisms involving endocytosis (**Figure 1**; Sung et al., 2001; Liu et al., 2007; Lee et al., 2008a). αSyn can also be transmitted trans-synaptically and through tunnel-like structures that connect the cytosol of neighbor cells (Danzer et al., 2011; Abounit et al., 2016). In vivo evidence includes the slow but persistent acquisition of LB-like inclusions by healthy neuronal cells that have been grafted into the brains of mice predisposed to develop LB-pathology spontaneously, such as αSyn transgenic mice (Desplats et al., 2009; Hansen et al., 2011). Similarly, an early onset and widespread LBlike pathology is observed in animals that had received

FIGURE 1 | Hypothetical model of α-Synuclein cell-to-cell transmission. In pathological conditions, αSyn is found as β-sheet-enriched amyloid aggregates and fibrils that reside within Lewy bodies and Lewy neurites (LBs/LNs). Neurons containing LBs (left) could release αSyn aggregates and seeds into the extracellular milieu by different mechanisms such as non-classical exocytosis or via exosomes (1). Extracellular αSyn is then internalized by endocytosis by neighbor neurons as well as glial cells (2). Due to its amyloidogenic nature, uptake of exogenous seeds promotes the structural corruption of endogenous counterpart in healthy recipient neuronal cells (right). Thus, monomers of the recipient cell are converted into aggregates and fibrils by direct action of exogenous seeds (3) and new LBs are formed (not shown). Neuronal viability is severely affected by two mechanisms; (i) the intrinsic cytotoxic properties of intracellular αSyn aggregates and (ii) indirectly by action of proinflammatory molecules released by glial cells activated upon exposure to extracellular αSyn seeds (4).

an intracerebral dose of brain homogenates of diseased αSyn transgenic mice (Luk et al., 2012b). While other factors present in brain homogenates could be involved, αSyn alone is sufficient to initiate LB-like pathology and its subsequent spreading: a single intracerebral injection of synthetic αSyn preformed fibrils leads to pathogenesis and progressive accumulation LB-like inclusions in neuroanatomically-interconnected areas accompanied by pathological features of PD such as neurodegeneration, neuroinflammation and motor deficits (Luk et al., 2012a; Sacino et al., 2014). This induction of LB-like pathology by intracerebral administration of αSyn aggregates strictly depends on the presence of αSyn in the host recipient cell, as no pathology can be induced in αSyn knock out mice (Luk et al., 2012b; Mougenot et al., 2012). The requirement of endogenous αSyn on LB-like inclusion spreading is explained by the observation in cell cultures that upon uptake, preformed αSyn fibrils promote the structural corruption of endogenous αSyn and its recruitment into newly formed inclusions (Luk et al., 2009; Waxman and Giasson, 2010; Volpicelli-Daley et al., 2011, 2014). Thus, host αSyn would be essential for the amplification of inclusions, an idea that needs to be further challenged (Helwig et al., 2016). Recruitment of host αSyn by exogenous αSyn has also been demonstrated in vivo in most mouse models in which LB pathogenesis is induced by administration of exogenous αSyn seeds (Luk et al., 2012b).

#### BRAAK'S HYPOTHESIS IN PD PROGRESSION

Braak and colleagues discovered that in sporadic PD, Lewy body pathology is first observed in the lower brainstem and anterior olfactory structures and that it then ascends following a caudo-rostral pattern from the dorsal motor nucleus through susceptible areas of the medulla, pontine tegmentum, midbrain, basal forebrain, reaching in some extreme cases the cerebral cortex (Braak et al., 2003a). These observations elegantly detailed LB spreading along the central nervous system (CNS), but they did not explain where and how the inclusions are originated. Thus, the provocative idea that LB-pathology begins when a neurotropic pathogen enters the nervous system and then spreads in a retrograde-axonal and transneuronal manner from one vulnerable brain region to the next was then introduced (Braak et al., 2003b; Hawkes et al., 2007). While in transit, this pathogen induces formation of αSyn-positive inclusions in these traceable CNS areas. It is now evident that certain species of αSyn can fulfill the requirements for Braak's neurotropic pathogen, and that both the olfactory and gastric tracks are largely compatible with the putative entry routes. In support of this, LB pathology is found in both the anterior olfactory nucleus as well as olfactory bulb mitral cells, the projection neurons that receive inputs from the olfactory epithelium (Daniel and Hawkes, 1992; Braak et al., 2003a). LB pathology has long been known to occur in the gastrointestinal tract of PD patients and is well documented in all the stages of PD (Braak et al., 2006; Lebouvier et al., 2008; Pouclet et al., 2012). Importantly, Braak and colleagues also provided the explanation for the observed non-random distribution of LBs along the nervous system by showing that different cell types have different susceptibilities to the development of inclusions. They concluded that neurons with a long, thin and poorly myelinated axon are highly susceptible to develop inclusions (Braak and Braak, 2000; Braak et al., 2003b). Importantly, these features are found in the enteric vagal preganglionic neurons that are susceptible to developing LB-pathology in early asymptomatic stages of the disease (Braak and Del Tredici, 2004).

# MECHANISMS OF TOXICITY

Like many aspects of its intercellular transmission, very little is known on the mechanisms of toxicity inherent to cellto-cell transmitted αSyn. Likewise, how αSyn transmission is modulated or whether PD-related familial mutations or somatic copy number variations of the SNCA gene influence αSyn transmission remains to be determined. Taking into consideration our current knowledge on the consequences of αSyn misregulation, it is conceivable that cell-to-cell transmitted and endogenous αSyn share cytotoxic mechanisms that directly impact neuronal survival. These include, but are not limited to, the loss of function of endogenous αSyn as a consequence of its seeded aggregation, a phenomenon that massively affects neuronal physiological processes such as vesicle trafficking including neurotransmitter release and recycling (Jenco et al., 1998; Abeliovich et al., 2000; Murphy et al., 2000; Cabin et al., 2002; Chandra et al., 2005), the impairment of mitochondrial activity that perturbs not only a plethora of metabolic processes but also degradative pathways (Martin et al., 2006; Devi et al., 2008; Liu et al., 2009; Chinta et al., 2010; Loeb et al., 2010), and the disruption of vesicular transport mechanisms, in particular those that trigger endoplasmic reticulum stress (Cooper et al., 2006; Gitler et al., 2008; Thayanidhi et al., 2010). However, it is still possible that cell-to-cell transmitted αSyn has its own particular repertoire of cytotoxic properties, in addition to its probable distinct physiological functions. The fact that cellto-cell transmitted αSyn is released to the extracellular milieu allowed neuroscientists to develop cellular and animal models based in administration of exogenous αSyn species that would recapitulate key mechanistic aspects of αSyn transmission such as its internalization and downstream events. Data obtained from glial cells might constitute the first evidence that endogenous and exogenously acquired αSyn might behave differently. It is well known that glial cells normally do not express αSyn mRNA (Reyes et al., 2014) and instead acquire the protein from the extracellular milieu (Liu et al., 2007; Lee et al., 2008b, 2010; Park et al., 2009). Thus, uptake of exogenous αSyn not only would explain the source of αSyn in glial cytoplasmic inclusions (GCIs) in multiple system atrophy (MSA), a progressive neurodegenerative disease related to PD and other synucleinopaties (Tu et al., 1998), but also would uncover a unique role for extracellular αSyn in the context of αSyn-deficient cells. Furthermore, it was shown that extracellular αSyn activates astrocytes and microglia in vitro and in vivo resulting in a neuroinflammatory response reminiscent to that observed in PD (Zhang et al., 2005; Lee et al., 2010; Alvarez-Erviti et al.,

2011; Halliday and Stevens, 2011; Luk et al., 2012a). Noteworthy, neurons are highly susceptible to glial-derived proinflammatory factors, therefore representing an alternative neurotoxic process triggered specifically by cells that have acquired αSyn from the extracellular milieu. Although the neuron-glia interaction might help to elucidate specific and non-redundant roles for intracellular and extracellular cell-to-cell transmitted αSyn, the molecular and biochemical determinants that presumably make these two forms of αSyn different remain completely unexplored.

#### THE PRION HYPOTHESIS

As a consequence of Braak's model formulation, the idea that PD behaves as a prion disease has emerged, leading some to refer to it as a "prion-like" disorder. Of note, unlike prions, transmissibility between individuals of pathological forms of αSyn has not been demonstrated and thereby αSyn is currently considered as a non-infectious protein (Aguzzi and Rajendran, 2009; Beekes et al., 2014). The analogy to prion diseases stems for the fact that in in vivo experiments, involving mostly rodents and in some cases non-human primates, intracerebral administration of exogenous αSyn (either αSyn-containing brain material or synthetic αSyn proteins) is sufficient to trigger LB pathogenesis, amplification and spreading. These processes are usually accompanied by the phenotypic changes naturally observed in PD such as neurodegeneration, neuroinflammation, and motor deficits. Furthermore, it has been shown that in some cases the particular structural conformation of the exogenous αSyn seed, normally referred to as the "conformational strain", is transmitted from the exogenously administered aggregates to host αSyn and thereby to the newly formed inclusions (Bousset et al., 2013; Guo et al., 2013; Peelaerts et al., 2015). This supports the idea of that the applied exogenous αSyn acts as seeds that template the aggregation of homotypic molecules of the host, a phenomenon characteristic of prions. The evidence of PD as a prion-like disorder is accumulating, however, there are still several unsolved questions that should be addressed before this terminology is broadly accepted. These questions arise from the inherent limitations of animal models for the full recapitulation of the human condition. In this sense, a constantly growing effort is being made to better characterize the brain material that contains neurotoxic αSyn species and the synthetic αSyn aggregates that are administrated intracerebrally to trigger PDlike pathology (Bousset et al., 2013; Tuttle et al., 2016). To uncover the molecular similarities and differences between the αSyn used in such in vivo experiments and those contained in Lewy bodies and Lewy neurites is critical to comprehensively understand the scopes and limitations of such animal models and the resultant hypotheses. As an example of such a gap of information, it has long been reported that certain forms of

#### REFERENCES

Abeliovich, A., Schmitz, Y., Fariñas, I., Choi-Lundberg, D., Ho, W. H., Castillo, P. E., et al. (2000). Mice lacking α-synuclein display functional deficits in the nigrostriatal dopamine system. Neuron 25, 239–252. doi: 10.1016/S0896-6273(00)80886-7

αSyn are found in the cerebrospinal fluid (CSF) of both healthy subjects and PD patients (Borghi et al., 2000; Tokuda et al., 2006; Mollenhauer et al., 2010; Parnetti et al., 2011; Foulds et al., 2012). However, it remains an enigma which structural species of αSyn are found in CSF and whether these molecules correspond to the transmitted species that mediate the pathogenic process that underlies LB pathogenesis and spreading. Elucidating this will help to reconcile clinical evidence arguing against the concept of prion-like progression in Parkinson's disease and related synucleinopaties (Hallett et al., 2014).

# CONCLUDING REMARKS

The discovery of the cell-to-cell propagation of αSyn and in particular its role as mediator of disease progression has opened new therapeutic avenues for the treatment of PD and related neurological disorders, and novel therapies targeting extracellular αSyn aimed to delay or stop disease progression are currently being explored. The therapeutic potential of passive immunotherapy targeting aberrant forms of αSyn, for instance, has recently been investigated showing that it efficiently interferes with uptake of extracellular αSyn seeds preventing downstream effects such as amplification and transmission of pathological aggregates (Tran et al., 2014). Similarly, administration of rationally engineered antibodies robustly promotes degradation and neutralization of internalized αSyn preventing cell-to-cell aggregate transmission and neuronal loss (Bae et al., 2012; Spencer et al., 2014). Nevertheless, the elucidation of the mechanisms involved in αSyn transcellular transmission will be instrumental not only for the development of novel therapies for PD but also for the understanding of the "prion-like" properties of amyloid-beta (Aβ), tau and Huntingtin, all of them transmissible aggregation-prone proteins with a long history in neurodegenerative diseases such as Alzheimer and Huntington's diseases, respectively (Brettschneider et al., 2015).

# AUTHOR CONTRIBUTIONS

NP, RR, and JG carried out literature searches, assisted in generation of figures and writing of the manuscript. All authors read and approved the manuscript.

#### ACKNOWLEDGMENTS

NP and JG are supported by a "Wilhelm Hurka" Stiftung. RR and JG are supported by a grant of the Swiss National Science Foundation (Sinergia 154461). JG is supported by an EMBO postdoctoral fellowship (ALTF-254-2012). The authors acknowledge to Jason Greenwald for critical reading of the manuscript.

Abounit, S., Bousset, L., Loria, F., Zhu, S., de Chaumont, F., Pieri, L., et al. (2016). Tunneling nanotubes spread fibrillar α-synuclein by intercellular trafficking of lysosomes. EMBO J. 35, 2120–2138. doi: 10.15252/embj.201593411

Aguzzi, A., and Rajendran, L. (2009). The transcellular spread of cytosolic amyloids, prions, and prionoids. Neuron 64, 783–790. doi: 10.1016/j.neuron.2009.12.016


in mammalian cells by antagonizing ER/Golgi SNAREs. Mol. Biol. Cell 21, 1850–1863. doi: 10.1091/mbc.E09-09-0801


pathology leading to synaptic dysfunction and neuron death. Neuron 72, 57–71. doi: 10.1016/j.neuron.2011.08.033


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2016 Prymaczok, Riek and Gerez. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Contribution of Neuroepigenetics to Huntington's Disease

Laetitia Francelle<sup>1</sup> \*, Caroline Lotz<sup>2</sup> , Tiago Outeiro<sup>1</sup> , Emmanuel Brouillet<sup>3</sup> and Karine Merienne<sup>2</sup>

<sup>1</sup> Department of NeuroDegeneration and Restorative Research, University Medical Center Goettingen, Goettingen, Germany, <sup>2</sup> CNRS UMR 7364, Laboratory of Cognitive and Adaptive Neurosciences, University of Strasbourg, Strasbourg, France, <sup>3</sup> Commissariat à l'Energie Atomique et aux Energies Alternatives, Département de Recherche Fondamentale, Institut d'Imagerie Biomédicale, Molecular Imaging Center, Neurodegenerative diseases Laboratory, UMR 9199, CNRS Université Paris-Sud, Université Paris-Saclay, Fontenay-aux-Roses, France

Unbalanced epigenetic regulation is thought to contribute to the progression of several neurodegenerative diseases, including Huntington's disease (HD), a genetic disorder considered as a paradigm of epigenetic dysregulation. In this review, we attempt to address open questions regarding the role of epigenetic changes in HD, in the light of recent advances in neuroepigenetics. We particularly discuss studies using genome-wide scale approaches that provide insights into the relationship between epigenetic regulations, gene expression and neuronal activity in normal and diseased neurons, including HD neurons. We propose that cell-type specific techniques and 3Dbased methods will advance knowledge of epigenome in the context of brain region vulnerability in neurodegenerative diseases. A better understanding of the mechanisms underlying epigenetic changes and of their consequences in neurodegenerative diseases is required to design therapeutic strategies more effective than current strategies based on histone deacetylase (HDAC) inhibitors. Researches in HD may play a driving role in this process.

Keywords: neuroepigenetics, Huntington's disease, epigenomics, transcriptomics, neuronal activity, HDAC inhibitors, neurodegenerative diseases

# INTRODUCTION

Huntington's disease (HD) is a genetic disease affecting preferentially medium spiny neurons (MSN) of the striatum. Increasing numbers of studies provide evidence for altered epigenetic regulations in HD. Here, after a summary of general epigenetic mechanisms in neurons, we review the state-of-the-art of epigenetic changes in HD. We discuss the mechanisms underlying these changes and their consequences, particularly on expression of neuronal identity genes. Current challenges to improve our understanding of the role of epigenetic mechanisms in HD are discussed, including the development of new technologies to profile neuronal epigenomes at single-cell or cell-type specific levels. Epigenetic modifications represent promising therapeutics. We discuss the need of identifying new epigenetic targets for therapy, representing alternatives to histone deacetylase (HDAC) inhibitors.

#### Epigenetic Mechanisms: General Rules

The current definition of epigenetics relates to "the study of phenomena and mechanisms that cause chromosome-bound, heritable changes to gene expression that are not dependent on

Edited by:

Daniela S. Andres, ETH Zurich, Switzerland

#### Reviewed by:

Anthony John Hannan, Florey Institute of Neuroscience and Mental Health, Australia Elizabeth A. Thomas, Scripps Research Institute, USA Martin Radrizzani, National University of San Martín, Argentina

#### \*Correspondence:

Laetitia Francelle laetitia.francelle@hotmail.fr

Received: 07 November 2016 Accepted: 10 January 2017 Published: 30 January 2017

#### Citation:

Francelle L, Lotz C, Outeiro T, Brouillet E and Merienne K (2017) Contribution of Neuroepigenetics to Huntington's Disease. Front. Hum. Neurosci. 11:17. doi: 10.3389/fnhum.2017.00017

changes to DNA sequence" (Deans and Maggert, 2015). Epigenetic mechanisms actually regulate several DNA/RNAmediated processes, including transcription, DNA repair and DNA replication, through modulation of the structure of chromatin, a macromolecular complex of DNA, RNA and proteins such as histones. Two major epigenetic mechanisms influence chromatin structure: histone modifications and DNA methylation (Allis and Jenuwein, 2016) (**Figure 1**).

In the nucleus, DNA is wrapped around core particles of chromatin, the nucleosomes, which are formed of octamers of histones, including H2A, H2B, H3, and H4. Histones are subject to various post-translational modifications (PTM) such as acetylation, methylation, phosphorylation and ubiquitylation. These histone modifications modulate the degree of compaction of nucleosomes, thereby affecting chromatin accessibility to various factors, particularly transcriptional regulators. (Jenuwein and Allis, 2001; Borrelli et al., 2008; Wang and Jin, 2010; Tsompana and Buck, 2014; Xu et al., 2014). Importantly, histone residues can be modified in a combinatorial, reversible, and targeted manner (**Figure 1**). For instance, histone acetyltransferases (HAT) and HDAC are involved in acetylation and deacetylation of specific histone residues, respectively (**Figure 1**). Similarly, methylases and demethylases regulate the addition and removal of methyl groups on histone residues (**Figure 1**). The combinatorial, reversible and targeted nature of histone modifications is the basis of the so-called 'histone code'. It permits to achieve specificity in the outcome, through the action of proteins interpreting the code (e.g., "readers") (Jenuwein and Allis, 2001; Bannister and Kouzarides, 2011; Jones et al., 2016). One major outcome is transcription. General rules of the "transcriptional" histone code are relatively well defined. For instance, histone acetylation, whatever the histone or the residue modified, promotes relaxed chromatin and is associated with transcriptional activation, whereas histone methylation, depending on the specific residue that is modified, can lead to transcriptional repression or activation. For instance, H3K9 methylation is associated with transcription repression, whereas H3K4 methylation correlates with transcription activation. Moreover, the genome comprises different regulatory regions, including promoters and enhancers (i.e., regulatory regions distal to promoters) playing specific roles in transcriptional regulation. These regions display specific histone modifications. For instance, promoters and enhancers of active genes are enriched in trimethylated H3K4 (H3K4me3) and in acetylated H3K27 (H3K27ac) respectively, further illustrating the targeting precision permitted by the histone code.

DNA methylation, another important epigenetic mechanism, consists in adding a methyl group to cytosines by DNA methyltransferases (DNMT), particularly at C5 position of cytosine in cytosine-guanine dinucleotide sequences (CpG), creating 5-methylcytosine (5-mC). DNMT1 is implicated in the maintenance of methylation patterns during DNA replication. In contrast, DNMT3a and DNMT3b have been associated with de novo DNA methylation (Jeltsch, 2006; Feng et al., 2010). Though DNA methylation was long considered as a stable process, it is now clear that post-mitotic cells can undergo active DNA demethylation, through a mechanism implicating TET proteins (Pastor et al., 2013). TET proteins induce hydroxylation of 5-mC (leading to 5-hmc) and are further involved in their oxidation. Oxidized 5-hmC are then processed by DNA repair mechanisms and converted back to their unmethylated state (Guo et al., 2011; Feng et al., 2015). 5-hmC is particularly extended in neurons, suggesting that regulation of DNA methylation is highly dynamic in these cells (Song et al., 2011; Szulwach et al., 2011). 5-mC are bound by several classes of methyl-binding proteins (such as MeCP2), which associate with other protein partners, including HDAC, forming co-repressor complexes (Urdinguio et al., 2009; Du et al., 2015). DNA methylation at gene promoters, generally enriched in CpG sites and forming so-called CpG islands, is an important mechanism involved in gene repression (Deaton and Bird, 2011) (**Figure 1**).

The collection of epigenetic modifications (e.g., the epigenome) can be assessed at genome-wide scale, using next generation sequencing-based techniques such as chromatinimmunoprecipitation-sequencing (ChIP-seq). Generation and integration of epigenomic data with transcriptomic and/or functional data to fully decode epigenomes at cell-type specific level is a major current challenge that requires the development of new techniques and analysis methods (Marconett et al., 2013; Shin et al., 2014). In particular, methods to process low cell numbers/single-cell need to be developed as well as powerful bioinformatics tools (Bock et al., 2016).

#### Neuroepigenetics: Why are Neurons Specific?

Epigenetic mechanisms, interfacing individual genomes with environmental factors, are essential to the regulation of fundamental biological processes. Historically, epigenetic mechanisms have been primarily explored in the context of development, including cellular differentiation and the establishment of stable cellular identity (Holliday, 2006; Roth and Sweatt, 2011; Boland et al., 2014). In fact, cell state transitions during development require massive epigenetic changes, whose stabilization enables the maintenance of cell-type specific identity. The mechanisms involve the establishment of defined transcriptional programs. As a result, the epigenome is a gatekeeper of cell-type specific identity. However, this view of stable epigenetics has been challenged, due to evidence showing that massive and dynamic epigenetic changes can be implicated in the regulation of cellular activity. Such "plasticity" of epigenetic regulations is particularly critical to neuronal cell activity; hence the concept of neuroepigenetics (Borrelli et al., 2008; Riccio, 2010; Sweatt, 2013; Deans and Maggert, 2015).

Neuronal excitability is one major property of neuronal cells. In response to environmental experience, including learning, drug exposure, psychological and physical stress environmental signals, neurons undergo reversible transitions from resting to active (or excited) states, which influence synaptic plasticity and promote adaptive behavior, such as learning and memory (Sultan and Day, 2011; Sweatt, 2016). These processes are highly dynamic, and can also be long-lasting. Increasing evidence indicates that epigenetic mechanisms regulate the transition from resting to active neuronal state (Korzus, 2010; Landgrave-Gomez et al., 2015; Meadows et al., 2016).

FIGURE 1 | Epigenetic regulations are dynamic. Chromosomal DNA is wrapped around histone octamers (comprising dimers of H2A, H2B, H3, H4), forming nucleosomes, the core unit of chromatin. Protruding N-terminus histone tails can undergo post-translational modifications (PTM) that influence chromatin state (e.g., relaxed vs. compact state) Epigenetic writers add PTM on histone tail residues. For example, histone acetyltransferases (HAT) add acetyl- (Ac) groups, and histone methyltransferases (HMT) add methyl- (Me) groups. In a dynamic manner, these marks are removed by epigenetic erasers, such as histone deacetylases (HDAC) and lysine demethylases (KDM), removing, respectively, Ac- and Me-groups. DNA methylation at cytosines, particularly at CpG dinucleotides, is performed by DNA methyltransferases (DNMT). DNA demethylation involves hydroxylation and oxydation of methylated cytosines by Tet proteins (Tet), which are then repaired by DNA repair mechanisms, including base excision repair (BER).

Learning and memory processes are associated with synaptic plasticity, leading to the rapid formation of new synapses, which can be strengthened or lost over time. Synaptic plasticity correlates with dynamic changes at the level of histone modifications. Specifically, changes in acetylation of histones, including H3 and H4 have been associated with early formation of new synapses (Federman et al., 2009; Chatterjee et al., 2013; Graff and Tsai, 2013; Peixoto and Abel, 2013; Benito et al., 2015). Moreover, neuronal activity can be associated with memory storage and consolidation, which may involve long-term remodeling of neuronal networks at system levels. Epigenetic regulations have also been implicated in these processes. Upon learning paradigms, the chromatin in brain tissues implicated in memory formation and/or consolidation, such as the hippocampus and the cortex, undergoes extensive modifications, including increased histone acetylation and DNA methylation changes (Day and Sweatt, 2011; Bousiges et al., 2013; Zovkic et al., 2013; Halder et al., 2016). DNA methylation and histone modifications both control memory processes, through transcriptional effects that comprise the activation of synaptic plasticity genes, including immediate early genes (IEGs) like Fos, Egr1, or Arc (Minatohara et al., 2015; Thakurela et al., 2015). Remarkably, these transcriptional effects that are experiencedriven and epigenetically regulated can be long lasting. Activation of specific signaling pathways, such as the cAMP/CREB/CREBbinding protein (CBP) pathway, is involved in the coupling between epigenetic and transcriptional responses, promoting the recruitment of protein complexes at target genes, which induces chromatin remodeling and drive transcription (Cedar and Bergman, 2009; Tie et al., 2014; Alberini and Kandel, 2015; Ortega-Martinez, 2015).

However, although tight coupling between neuronal-activityregulated epigenetic and transcriptional changes remains a favored hypothesis, recent data, integrating epigenomic, and transcriptomic data suggest that the link between both events may not be as clear as anticipated (Lopez-Atalaya and Barco, 2014; Liu et al., 2015; Valor, 2015b). Recently, contextual fear conditioning in mice was used as a learning paradigm to examine the spatiotemporal correlation between epigenetic and transcriptional modifications in neurons (Halder et al., 2016). Specifically, DNA methylation and seven histone modifications were assessed at a genome-wide scale using hippocampal and cortical neurons of mice that were subjected to contextual fear conditioning. To specify the timing of epigenetic changes in these tissues, analyses were performed at different time points with respect to contextual fear conditioning, associated with different memory processes, including memory formation and consolidation. Transcriptomic analyses were also performed

on the same tissues. Generally, histone modifications were extensively modulated during memory formation, and these widespread changes weakly correlated with gene-selective transcriptional changes. In contrast, DNA methylation changes were rather locus-specific and correlated with transcriptional changes, including splicing events (Lev Maor et al., 2015; Halder et al., 2016). Thus, the coupling between epigenetic and transcriptional changes may depend on chromatin modifications. Additional studies are required to specify causal relationship between the two events. Other neuronal cells/tissues and experimental paradigms may be used, which would permit to investigate the interplay between epigenetics and transcription in brain cells/tissues associated with additional brain functions, including other cognitive functions, motor functions and functions linked to emotion and motivation regulation.

Thus, it appears that epigenetic mechanisms in neurons not only control their identity, like in other cell types, but also regulate neuronal activation, including the ability to undergo dynamic plasticity in response to environmental signals. Then, the question arising is what is going on in pathological situations, when neuronal function is impaired? Does it result from altered activation capacity of affected neurons or from a loss of neuronal identity? Are impaired epigenetic regulations implicated in pathological processes? These questions are particularly relevant to neurodegenerative diseases where specific neuronal populations (or identities) are preferentially affected. In the following sections, we have chosen to focus on one such disease, HD, playing a pioneering role in the understanding of the epigenetic regulation mechanisms in brain diseases.

# EPIGENETICS IN HD

Huntington's disease is a neurodegenerative disease caused by an unstable expanded CAG repeats (>35–39 repeats) in the Huntingtin gene (HTT), which results in the production of mutant protein (mHtt) with a toxic polyglutamine (polyQ) tract (Landles and Bates, 2004). HD is characterized by specific symptoms, including motor impairment (e.g., chorea, bradykinesia, gait abnormalities, dystonia), cognitive deficits (motor skill learning deficits, planning, and attention troubles) and psychiatric alterations (depression, mania, apathy, suicide), usually appearing at adulthood (see as review, Roos, 2010). Since polyQ expansion toxicity is correlated with repeat size, HD patients with longer CAG expansions are more severely affected, showing earlier onset of symptoms and faster pathology progression. HD is characterized by a preferential and primary degeneration of two structures of the basal ganglia: the caudate nucleus and the putamen that form the neostriatum. However, additional brain regions, particularly the cortex, degenerate as the pathology progresses (Rosas et al., 2008). Remarkably, in HD striatum, selective neuronal populations, the GABAergic MSN, are more particularly vulnerable, whereas the large cholinergic interneurons, the medium size GABAergic interneurons and glial cells appear spared (Ferrante et al., 1987a,b; Cicchetti et al., 1996). From a biochemical point of view, polyQ-Htt presents a high propensity to misfold and aggregate, which leads to the formation of nuclear inclusions, particularly in neurons (DiFiglia et al., 1997; Li L. et al., 2016). These aggregates, a disease hallmark, recruit a number of proteins, including transcriptional regulators (Steffan et al., 2000; Arrasate and Finkbeiner, 2012).

A long period of neuronal dysfunction precedes the death of neurons sensitive to the HD mutation. Several cellular processes are believed to contribute to neuronal dysfunction, including Htt cleavage and aggregation, abnormal protein-protein interactions, dysfunctional calcium signaling, abnormal axonal transport, impaired energy metabolism, dysregulation of neuronal activity, and altered transcription (see as reviews Landles and Bates, 2004; Zuccato et al., 2010; Labbadia and Morimoto, 2013; Saudou and Humbert, 2016).

# The "Neuronal" Signature of the HD Transcriptome

Transcriptional dysregulation plays a central role in HD pathogenesis (Seredenina and Luthi-Carter, 2012). Major transcriptional changes have been reported in the brains of HD patients (Hodges et al., 2006). This is also observed in HD mouse models, including transgenic and knock-in mice (Luthi-Carter et al., 2000; Luthi-Carter et al., 2002; Zucker et al., 2005; Roze et al., 2008). Transcriptomic studies using mice showed that transcriptional changes are progressive, CAG repeat-lengthdependent and most extended in the striatum (Desplats et al., 2006; Kuhn et al., 2007; Becanovic et al., 2010; Langfelder et al., 2016). In the striatum of HD models, transcriptional changes occur in both directions: many genes are down- and up-regulated (Seredenina and Luthi-Carter, 2012; Francelle et al., 2014). Remarkably, down-regulated genes display a neuronal signature, since decreased genes in HD striatum are enriched in genes that define striatal neuron identity and function, such as Darpp32, Rgs9, Drd1 or Drd2 (Hodges et al., 2006; Kuhn et al., 2007; Vashishtha et al., 2013; Achour et al., 2015; Langfelder et al., 2016). In the other affected brain regions such as the cortex, fewer genes are down-regulated. However, they also present a neuronal signature, reflecting tissue identity (Vashishtha et al., 2013; Langfelder et al., 2016).

Interestingly, a recent study using RNAseq revealed a developmental signature associated with differentially expressed genes in post-mortem prefrontal cortex of HD patients. Specifically, Hox genes and additional homeobox genes were re-expressed in HD brains, suggesting that the transcriptome of HD neurons resemble that of immature neurons (Labadorf et al., 2015). Thus, these results support the view that the transcriptional program involved in the maintenance of neuronal identity is impaired in HD neurons. Up-regulated genes in the striatum or cortex of HD patients were also enriched in biological processes linked to inflammation (Hodges et al., 2006; Labadorf et al., 2015). However, up-regulation of immune response genes is not that clear in mouse models (Achour et al., 2015; Langfelder et al., 2016). Thus, down- and up-regulated genes in HD brain tissues display distinct functional signatures, suggesting that different mechanisms may operate.

The mechanisms underlying mutant Htt-induced transcriptional effects are unclear. However, they are thought to

involve both altered regulation of transcription regulators and histone-modifying enzymes.

# HDACi in HD: What are the Targets?

The hypothesis of an epigenetic origin of transcriptional dysregulation in HD, particularly implicating histone acetylation, has received increasing support over the years (Jaenisch and Bird, 2003; Lee et al., 2013; Glajch and Sadri-Vakili, 2015). The assumption that altered histone acetylation might contribute to HD was made in the early 2000s, when it was found that the HAT CREB-binding protein CBP is recruited into aggregates of mutant Htt, and that HDAC inhibitors (HDACi) improve phenotypes of drosophila and mouse models of HD (Nucifora et al., 2001; Steffan et al., 2001; Kazantsev et al., 2002). As a result, epigenetic strategies using HDACi to increase histone acetylation have been early considered for HD. Several preclinical studies have been performed using broad-spectrum HDACi (e.g., suberoxylanilide hydroxamic acid (SAHA), Trichostatin A (TSA), phenylbutyrate, sodium butyrate (NaB)] that target nonselectively HDAC of class I and II (Ferrante et al., 2003; Hockly et al., 2003; Gardian et al., 2005; Sadri-Vakili et al., 2007; Sharma and Taliyan, 2015b).

Histone deacetylase inhibitors improve some phenotypes of HD mice, including neuropathology and motor function. However, it is unclear whether beneficial effects require increased histone acetylation. Instead, studies suggest that the mechanism may involve acetylation of non-histone proteins. In support to this view, inactivation of a target of SAHA, Hdac 4, ameliorates neurodegeneration in HD mice through an apparently, transcription-independent mechanism, acting on mutant Htt aggregation process (Mielcarek et al., 2011). Non-histone-mediated beneficial effects of HDACi have also been documented in models of Parkinson disease (PD; Godena et al., 2014), suggesting common mechanisms between several neurodegenerative diseases.

New compounds have been developed in an attempt to generate more selective HDACi and with less toxic side effects (Herman et al., 2006; Thomas et al., 2008). The compound HDACi 4b, which was reported to ameliorate disease phenotype of HD mice, show high potency for inhibiting HDAC1 and HDAC3 (Thomas et al., 2008; Jia et al., 2012). However, physicochemical properties and metabolic profile of this compound were found suboptimal for investigation of HDAC inhibition in mice per oral administration (Beconi et al., 2012). The effect of RGFP966, an HDAC3-selective inhibitor, was recently investigated using HD mice (Jia et al., 2016). The results suggest that the compound limits glial cell response, diminishing markers of glial cell activation. Surprisingly however, a heterozygous inactivation of the Hdac3 gene in HD mice did not ameliorate disease-related phenotypes (Moumne et al., 2012), suggesting that more than 50% knock-down of the Hdac3 gene might be needed to see a beneficial effect. More recently, beneficial transgenerational effects have been reported using HDACi 4b in HD mice (Jia et al., 2015). Thus, despite some beneficial effects, the mode of action of HDACi in HD models remains elusive and may not systematically implicate histoneand transcription-dependent mechanisms.

It is still unclear whether expression of neuronal identity genes in HD brain tissues is restored upon HDACi treatment (Coppede, 2014; Wang et al., 2014; Sharma et al., 2015; Valor, 2015a). Clinical studies using HDACi are ongoing and the results are awaited. So far, Phase II studies provide indication for safety and tolerability of several compounds, including phenylbutyrate (Hogarth et al., 2007; Westerberg et al., 2015).

But what exactly is the status of the HD epigenome? Is acetylation the only histone modification impaired in HD affected tissues, and to which extent? Are acetylated histone residues all affected the same way by the HD mutation? Is DNA methylation also impaired? Is the chromatin structure globally altered and repressed or is it altered at specific genomic regions or gene loci? What is the underlying mechanism? What are the consequences of chromatin structures changes? Do they underlie transcriptional changes? Addressing these questions is certainly a prerequisite to the development of new epigenetic therapies for HD.

#### HD Epigenome

Many studies have already been performed that attempt to address these issues (Lee et al., 2013; Valor and Guiretti, 2014; Glajch and Sadri-Vakili, 2015). It is expected that the use of genome-scale approaches, which has remained so far rather limited in the HD field, will improve our knowledge of HD epigenome as well as provide insights into the mechanism responsible for altered epigenetic regulation in HD.

#### Relationship between Epigenetic and Transcriptional Changes in HD

#### **Histone acetylation**

Extensive changes in histone acetylation levels were observed in cellular systems based on mutant Htt overexpression (Steffan et al., 2001; Igarashi et al., 2003). However, global levels of H2B, H3 and H4 acetylation appeared unchanged between brain tissues of HD R6/2 and control mice (Hockly et al., 2003; Sadri-Vakili et al., 2007). Studies using the striatum of HD mice further suggested that decreased H3 acetylation occurs at selective gene loci, particularly at promoters of down-regulated genes such as Drd2, Penk1, Actb, or Grin1 (Sadri-Vakili et al., 2007). Using chromatin immunoprecipitation paired with microarray hybridization (ChIP-chip), McFarland et al. (2012) assessed histone acetylation changes at genome-wide scale in the striatum of HD R6/2 transgenic mice. H3K9 and H3K14 acetylations (H3K9,14ac) and transcriptional changes between HD and WT striatum were poorly correlated in the striatum of HD R6/2 mice, suggesting that variation in H3K9,14ac levels alone may not be sufficient to account for gene expression changes in HD mice. Valor et al. (2013) reached similar conclusions by investigating changes in H3K9,14 ac and H4K12ac using a more powerful method –ChIPseq- on hippocampus and cerebellum of the HD transgenic N171-82Q mouse model (**Figure 2**). However, whereas data obtained by McFarland et al. (2012) suggest broad changes in histone acetylation in HD mouse striatum when compared to WT striatum, the results reported by Valor et al. (2013) indicate that changes are restricted to few loci. The absence of bulk changes in histone acetylation is further supported by a

study showing promoter deacetylation of H3 at specific loci in HD models (Guiretti et al., 2016).

H3K27ac, a mark of active enhancers, was also selectively decreased in the striatum of HD R6/1 mice (Achour et al., 2015) (**Figure 2**). Integrating H3K27ac ChIPseq with RNA polymerase II (RNAPII) ChIPseq and RNAseq data, Achour et al. (2015) showed that H3K27ac and RNAPII are decreased at regions enriched in down-regulated genes, providing evidence for a strong correlation between decreased H3K27ac, decreased RNAPII and gene down-regulation in HD mouse striatum. Moreover, enhancer regions presenting reduced H3K27ac in HD R6/1 striatum were enriched in super-enhancers, a category of broad enhancers, regulating genes that define cell type-specific identity and function (**Figure 3**). In fact, striatal super-enhancer genes down-regulated in HD mouse striatum were enriched in genes controlling neuronal activity, including neuronal plasticity and transmission (Achour et al., 2015). Thus, the results suggest that selective decrease in super-enhancer activity underlies HD neuronal transcriptomic signature (e.g., down-regulation of genes that define neuronal identity and function, referred to as neuronal identity genes thereafter). This supports an epigenetic origin of gene down-regulation in HD.

#### **Other histone modifications**

The above results support the view that specific regulatory elements, super-enhancers, are sensitive to the HD mutation. Recent data indicate that selective promoters are also impaired in the striatum of HD mice and patients. Investigating H3K4me3, a mark of active promoters, down-regulated genes in the striatum and cortex of HD R6/2 mice were found to preferentially associate with broad promoters, regulating genes enriched in biological processes linked to neuronal function (Vashishtha et al., 2013). Moreover, H3K4me3 appeared decreased at downregulated genes in R6/2 vs. WT tissues (**Figure 2**). Thus, these results show that specific broad promoter signature associates with decreased expression of neuronal genes in the striatum and cortex of HD mice. It is very likely that target genes of broad promoters and super-enhancers in brain tissues overlap

and are enriched in neuronal identity genes. Moreover, in a study using ChIPseq on embryonic stem cells (ESC) and neural progenitor cells (NPC) expressing mutant Htt with various CAG sizes, a correlation was reported between CAG-repeat-dependent changes in gene expression and in H3K4me3 levels, particularly in NSC (Biagioli et al., 2015). This raises the hypothesis that mutant Htt might alter epigenetic regulation at early stage of neuronal differentiation. Finally, H3K4me3 has been investigated in the prefrontal cortex of HD patients using ChIPseq (Bai et al., 2015; Dong et al., 2015). In contrast to mouse data, human data indicate that epigenetic and transcriptional changes in HD vs control tissues are poorly correlated (Bai et al., 2015; Dong et al., 2015). However, in human experiments, sequencing depth appears suboptimal, which may have resulted in underestimation of H3K4me3 signals. Interestingly however, the study by Dong et al. (2015) indicates that promoters differentially enriched in H3K4me3 associate with genes involved in pathways or network linked to neuronal activity and inflammation, suggesting that transcriptional changes affecting inflammatory genes, in addition to those affecting neuronal genes, involve epigenetic mechanisms (**Figure 2**). The hypothesis that induction of inflammatory genes in HD neuronal tissues associates with loss of identity of glial cells would need to be investigated (Gabel et al., 2016).

Additional epigenetic modifications at regulatory regions might also be impaired. Increased levels of H3K9me2, a mark associated with heterochromatin, have been reported in the striatum of HD patients and R6/2 mice, using immunohistological analyses (Ryu et al., 2006). Whether specific loci are more particularly sensitive to increased H3K9 methylation in HD models has yet to be investigated using genome-wide approach. Moreover, the level of H3K27me3, a repressive histone modification that can mark promoters and enhancers, was modulated by CAG-repeat size in both ESC and NPC (Seong et al., 2010; Biagioli et al., 2015). However, the transcriptional consequences of this effect were unclear. Finally, H2A ubiquitylation (H2Aub) was found increased at downregulated genes in HD R6/2 mice (Kim et al., 2008), and ChIPon-chip analysis of H2Aub changes in R6/2 striatum indicated that histone changes were not restricted to dysregulated genes (McFarland et al., 2012).

#### **DNA methylation**

Genome-wide analysis of DNA methylation was performed in HD cell models (Ng et al., 2013). Changes in DNA methylation in response to mutant Htt were observed at both promoter proximal and distal regulatory regions. Interestingly, a large fraction of the genes that changed in expression upon mutant Htt expression displayed changes in DNA methylation, suggestive of a causal relationship (**Figure 2**). DNA methylation was also profiled using post-mortem cortex and liver from HD patients (De Souza et al., 2016). The results revealed minimal evidence of HD-associated DNA methylation. However, the HTT

gene was methylated in a tissue-specific manner, which might lead to tissue-specific regulation of HTT promoter activity. Additionally, 5-hydroxymethylcytosine (5-hmC) and 7-methyl guanine (7-MG) were globally reduced in brain tissues of HD mouse models, including YAC128 mice (5-hmC study), R6/2 and CAG140 knockin mice (7-MG study) (Thomas et al., 2013; Wang et al., 2013). While these studies further provide evidence for altered DNA methylation in HD brain tissues, transcriptional consequences of such impairments remains elusive. Analysis of DNA methylation has also been used to assess the epigenetic clock of HD patients (Horvath et al., 2016). Horvath recently developed an epigenetic measure of tissue age, so-called epigenetic age, estimated from DNA methylation levels of 353 CpG sites (Horvath, 2013). The study using brain tissues from HD patients showed accelerated epigenetic aging in HD brain, particularly in cortical tissues. However, this was not the case for striatal tissues, possibly due to excessive neuronal loss (Horvath et al., 2016). While transcriptional significance of accelerated aging in HD brain is unclear, the data might reveal an age-dependent alteration of epigenetic regulation. Finally, treatment of HD mice with the HDAC inhibitor HDACi 4b led to transgenerational effects, possibly mediated by increased DNA methylation at CpG sites associated with Kdm5d (Jia et al., 2015). Thus, DNA methylation may be a therapeutic target for HD.

# Mechanisms Involved in Altered Epigenetic Regulation in HD

As mentioned above, altered histone acetylation in HD has been proposed to result from decreased activity of the HAT CBP, due to CBP recruitment into aggregates of mutant Htt, CBP depletion in neurons expressing mutant Htt and/or through an aberrant interaction of CBP with soluble mutant Htt (Steffan et al., 2000; Nucifora et al., 2001; Jiang et al., 2006; Seredenina and Luthi-Carter, 2012; Glajch and Sadri-Vakili, 2015) (**Figure 4**). The effect appears specific to CBP since CBP-related HAT p300 was unable to rescue cell toxicity in overexpression assays (Nucifora et al., 2001). However, although studies manipulating CBP levels in HD models support a role for CBP in pathogenesis (Steffan et al., 2000; Klevytska et al., 2010), including cognitive deficits (Giralt et al., 2012), it is still unclear whether altered CBP underlies HD neuronal transcriptional signature.

The activity of additional chromatin modulators was also found affected by Htt and/or mutant Htt. Htt was associated with polycomb repressive complex 2 (PRC2) in cell nucleus, which facilitated stimulation of PRC2 activity (**Figure 4**). Noticeably, CAG-expanded Htt further enhanced PRC2 activity in cell models (Seong et al., 2010). However, although H3K27me3 ChIPseq analyses revealed that Htt null mutation in ESC globally decreased H3K27me3, a result consistent with Httmediated stimulation of PRC2, the trend was opposite in NPC, suggesting that Htt may also be implicated in the process of H3K27me3 removal, and these different roles for Htt on H3K27me3 regulation may depend on cell differentiation state (Biagioli et al., 2015). Thus, the role of mutant Htt on genomewide regulation of H3K27me3 needs to be further investigated, particularly in mature neurons. It is especially important to specify this issue since PRC2 deficiency in adult neurons leads to molecular, electrophysiological and behavioral phenotypes reminiscent to those seen in HD mice (von Schimmelmann et al., 2016). In fact, inactivation of PRC2 in striatal neurons resulted in re-expression of transcription factors involved in neuronal differentiation and in down-regulation of striatal identity genes, suggesting that transcriptional signatures resulting from PRC2 inactivation and from the expression of mutant Htt in the striatum share similarities (von Schimmelmann et al., 2016).

Additional enzymes modulating histone methylation were also deregulated in HD models. ESET/SETB1, a H3K9 methyltransferase, was increased in the striatum and cortex of HD R6/2 mice due to SP1/SP3-mediated transcriptional activation, which resulted in increased histone H3K9 methylation (Ryu et al., 2006) (**Figure 4**). Moreover, KDM5C/JARID1C, an enzyme involved in demethylation of H3K4me3, was upregulated in the striatum and cortex of HD R6/2 mice and proposed to contribute to decreased H3K4me3 in HD brain tissues (Ng et al., 2013; Vashishtha et al., 2013) (**Figure 4**). Other studies further suggest that Htt and/or mutant Htt affect chromatin structure through modulation of the activity of enzymes regulating histone methylation, including H3K9 and H3K4 methylation (Ooi and Wood, 2007; Lee et al., 2013; Dietz et al., 2015).

As already mentioned, transcription factors also contribute to remodel the chromatin through the recruitment of histone modifying enzymes, including enzymes involved in histone acetylation and methylation. For instance, following binding to their cognate DNA response element, several transcription factors can interact with CBP in a stimulus-dependent manner, thereby increasing CBP concentration and histone acetylation at selective gene regulatory elements (e.g., enhancers and/or promoters) (Sterner and Berger, 2000). Thus, impaired level or activity of transcription factors in HD may also affect epigenetic regulations. Many transcription factors, including REST, SP1, TAF130, p53, were found deregulated in cells expressing the HD mutation, due to aberrant interaction with mutant Htt or altered transcriptional regulations (Nucifora et al., 2001; Bae et al., 2005; Ravache et al., 2010; Zuccato et al., 2010; Moumne et al., 2013; Langfelder et al., 2016) (**Figure 3**). It is tempting to speculate that some of these factors, particularly those regulating neuronal differentiation or maintenance of neuronal fate (this is for instance the case of REST), might contribute to HD epigenetic and transcriptional alterations.

Additionally, the ability of transcription factors to interact with chromatin-modifying enzymes depends in some cases on the activation of specific signaling pathways, which permits to optimize the coupling between epigenetic and transcriptional responses after stimulation. This is the case for CREB, which needs to be phosphorylated in response to activation of cAMP signaling pathway, to bind and recruit CBP on chromatin (Cardinaux et al., 2000). The Ras/MAPK/MSK1 signaling pathway is another pathway, where transcriptional and epigenetic responses are coordinated through phosphorylation events targeting transcription factors and histones (e.g., ELK1 and H3 at serine 10, respectively) (Brami-Cherrier et al., 2009; Bahrami and Drablos, 2016). These pathways, which control

genes. ESET, ERG-Associated Protein with SET Domain; HTT, Huntingtin; KDM5C, Lysine Demethylase 5C; PRC2, polycomb repressive complex 2; REST, RE1 Silencing Transcription Factor.

the dynamics of IEGs expression such as Fos, Egr1, and Arc, play a key role in the regulation of neuronal activity, including plasticity and excitability (Besnard et al., 2011; West and Greenberg, 2011; Lopez-Atalaya and Barco, 2014). Several studies suggest that these pathways are impaired in HD striatum, raising the hypothesis that the coupling between epigenetic and transcriptional mechanisms regulating neuronal activity is altered in HD (Besnard et al., 2011; Langfelder et al., 2016), which might contribute to altered epigenetic regulations (Chawla et al., 2003).

Despite these hypotheses, it remains unclear how mutant Htt leads to selective alteration of the chromatin associated with



CBP, CREB-binding protein; EZH2, Enhancer of Zeste Homolog 2; DNMT, DNA MethylTransferase; HAT, Histone AcetylTransferase.

neuronal identity genes. The underlying mechanism may be the addition of direct effects of mutant Htt on the chromatin and transcription regulators described above (e.g., CBP, PRC2, ESET/SETB1, KDM5C/JARID1C, REST, SP1, CREB, TAF130, p53). Alternatively, it may be the result of an indirect mechanism caused by mutant Htt on neuronal homeostasis and activity that is yet to be identified.

## EMERGING PICTURE AND PERSPECTIVES

Here, we reviewed the current understanding of the role of neuroepigenetics in HD, in an attempt to uncover the significance of epigenetic changes in HD brain, a prerequisite to the rational design of epigenetic therapies.

Three main questions underlie the review: (1) What are epigenetic changes in HD? (2) What is the consequence of epigenetic changes in HD, particularly on transcriptional regulation? (3) What mechanism(s) cause(s) epigenetic changes in HD? Although, we still lack complete answers to any of these questions, an emerging picture arises, supporting the view that maintenance and/or establishment of neuron-specific chromatin identity is altered in HD, which leads to down-regulation of neuronal identity genes. This suggests that mutant Htt might interfere with neuronal differentiation, in agreement with recent studies (Molero et al., 2016). To investigate this hypothesis, it will be crucial to assess HD epigenome and transcriptome across time, including developmental stages. If altered epigenetic regulation in HD brain results in down-regulation of neuronal identity genes, it is very likely to disturb neuronal activity, including synaptic plasticity and neuronal excitability, controlling learning and memory processes. This will need to be investigated. The issue of tissue-/cell-specificity of epigenetic alterations in HD also remains to be investigated. It is unclear whether the HD mutation induces similar epigenetic changes between cell types (e.g., neurons vs. glial cells) or between different tissues (e.g., striatum vs. cortex, hippocampus or cerebellum, neuronal tissue vs. non-neuronal tissue). It will also be crucial to investigate the timing of establishment of super-enhancer signatures relative to transcriptional changes to specify the relationship between epigenetic and transcriptional changes in HD brain tissues. Addressing these questions will certainly provide insights into the mechanism causing epigenetic and transcriptional changes in HD, which may be a key to the identification of new therapeutic targets.

Comprehensive analysis of the HD epigenome using various HD models and genome-wide techniques, including techniques that permit to investigate epigenomes at cell type-specific or single-cell scale and in 3D (using chromosome conformation capture- (3C)-based methods) is necessary to address these questions. Another challenge will be to develop bioinformatics methods to analyze the data and extract meaning. This may require the development of approaches that reduce complexity, such as network-based methods (Langfelder and Horvath, 2008; Parmentier et al., 2013).

Epigenetic therapies to treat neurodegenerative diseases have been first considered for HD, when Steffan et al. (2000) showed

that HDAC inhibitors improve the phenotype of HD flies. In fact, preclinical studies to evaluate the effect of HDAC inhibitors in HD have inspired other neurodegenerative diseases, including additional trinucleotide repeat (TNR) diseases (for review see Evans-Galea et al., 2013) PD, Alzheimer disease (AD), and amyotrophic lateral sclerosis (ALS) (Xu et al., 2011; Lazo-Gomez et al., 2013; Sharma and Taliyan, 2015a). Beneficial effects have been observed in preclinical models for these different diseases, though the effects of HDAC inhibitors appear partial and underlying mechanisms are unclear (Benito et al., 2015; Harrison et al., 2015). In the specific case of TNR diseases, HDAC inhibitors may affect disease progression through two independent mechanisms: (1) through a general effect on gene expression program, (2) through modulation of the stability of TNRs, which are subject to epigenetic regulations (Goula et al., 2012; Evans-Galea et al., 2013). To overcome limitations associated with the use of HDCAi, including the lack of specificity and toxicity, HDACi-alternative epigenetic therapies are currently being developed that are based on HAT activators, histone methyltransferase inhibitors and DNMT modulators (**Table 1**).

Epigenomic studies on HD models might benefit from studies in other neurodegenerative diseases and reciprocally. Indeed,

#### REFERENCES


the similarity of epigenomic and transcriptomic signatures between HD and AD models (e.g., neuronal and inflammation signatures) is intriguing and might suggest that common epigenetic mechanisms to several neurodegenerative diseases might operate (Gjoneska et al., 2015).

#### AUTHOR CONTRIBUTIONS

LF and KM developed the literature review. KM, LF, and EB wrote the manuscript. LF and CL built the figures. All the authors read and commented the article throughout the writing, and collaborated in the revision of the final manuscript before submission.

#### FUNDING

This work was supported by funds from the Centre National de la Recherche Scientifique (CNRS). CL is a Ph.D. student supported by the French Research Ministry. LF is supported by a Postdoctoral Fellowship of the Alexander-von-Humboldt Foundation. TO is supported by the DFG Center for Nanoscale Microscopy and Molecular Physiology of the Brain (CNMPB).


and HDAC5. J. Neurochem. 85, 151–159. doi: 10.1046/j.1471-4159.2003. 01648.x


via NF-kappaB activation. Mol. Neurobiol. 53, 5041–5055. doi: 10.1007/s12035- 015-9428-3



in a mouse model of Huntington's disease. PLoS ONE 7:e31080. doi: 10.1371/ journal.pone.0031080


Proc. Natl. Acad. Sci. U.S.A. 110, E3027–E3036. doi: 10.1073/pnas.13113 23110


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2017 Francelle, Lotz, Outeiro, Brouillet and Merienne. This is an openaccess article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Corticostriatal Dysfunction in Huntington's Disease: The Basics

#### Kendra D. Bunner\* and George V. Rebec\*

Department of Psychological and Brain Sciences, Program in Neuroscience, Indiana University, Bloomington, IN, USA

The main input to the basal ganglia, the corticostriatal pathway, shows some of the earliest signs of neuropathology in Huntington's disease (HD), an inherited neurodegenerative condition that typically strikes in mid-life with progressively deteriorating cognitive, emotional, and motor symptoms. Although an effective treatment remains elusive, research on transgenic animal models has implicated dysregulation of glutamate (Glu), the excitatory amino acid released by corticostriatal neurons, in HD onset. Abnormalities in the control of Glu transmission at the level of postsynaptic receptors and Glu transport proteins play a critical role in the loss of information flow through downstream circuits that set the stage for the HD behavioral phenotype. Parallel but less-well characterized changes in dopamine (DA), a key modulator of Glu activation, ensure further deficits in neuronal communication throughout the basal ganglia. Continued analysis of corticostriatal Glu transmission and its modulation by DA, including analysis at the neurobehavioral level in transgenic models, is likely to be an effective strategy in the pursuit of HD therapeutics.

#### Edited by:

Daniela S. Andres, Swiss Federal Institute of Technology (ETH) and University of Zurich (UZH), Switzerland

#### Reviewed by:

Carlos Cepeda, David Geffen School of Medicine at the University of California, Los Angeles, USA Eugene A. Kiyatkin, National Institute on Drug Abuse, USA

#### \*Correspondence:

Kendra D. Bunner kdbunner@indiana.edu; George V. Rebec rebec@indiana.edu

Received: 21 April 2016 Accepted: 13 June 2016 Published: 28 June 2016

#### Citation:

Bunner KD and Rebec GV (2016) Corticostriatal Dysfunction in Huntington's Disease: The Basics. Front. Hum. Neurosci. 10:317. doi: 10.3389/fnhum.2016.00317 Keywords: glutamate, dopamine, cortiostriatal circuitry, electrophysiology, Huntington's disease

# INTRODUCTION

Aberrant function of basal ganglia circuitry plays an important role in multiple neuropathological conditions (Reiner, 2010). Here we focus on Huntington's disease (HD), a dominantly inherited and ultimately fatal neurodegenerative disorder characterized by near-total loss of cognitive, emotional, and motor control (Cepeda et al., 2007; Zuccato et al., 2010). Early signs of HD neuropathology emerge in the striatum, the main input structure of the basal ganglia, and the cerebral cortex, which supplies the striatum with the associative, limbic, and motor information necessary to select and guide appropriate behavioral responses despite widely varying circumstances and contexts (Kreitzer and Malenka, 2008; Milnerwood and Raymond, 2010). In effect, the corticostriatal system supports the production of skilled yet flexible behavioral actions that define a healthy life. HD disrupts this system by interfering with the mechanisms by which cortical and striatal neurons communicate.

#### SYMPTOMS AND NEUROPATHOLOGY OF HUNTINGTON'S DISEASE

HD afflicts approximately 1 in 10,000 persons of European descent. Other populations are also affected but at a slightly lower incidence (Harper, 2001; Rawlins et al., 2016). The largest and best characterized population with HD is localized to the region of lake Maracaibo, Venezuela. In fact, it was with this population the underlying genetic cause was discovered (Gusella et al., 1983).

HD is caused by an expansion of the polyglutamine (CAG) repeat in the huntingtin (htt) gene located on exon 1 of chromosome 4 (The Huntington's Disease Collaborative Research Group, 1993), which encodes the protein huntingtin (HTT). In healthy individuals, htt alleles contain 6–30 CAG repeats; more than 30 repeats and the gene is considered mutated. An expansion of 36 or more CAG repeats will result in disease onset with full penetrance occurring at 40 or more repeats (Rubinsztein et al., 1996). An intermediate number of repeats (e.g., 30–36) is associated with germline instability and can lead to de novo HD, indicating a higher prevalence of HD in general populations with longer CAG repeats (Bates et al., 2015). Although there is considerable variability, the onset and severity of symptoms is inversely correlated with the number of CAG repeats; the greater the CAG expansion, the earlier the onset of symptoms (Nørremølle et al., 1993; Harper and Jones, 2002; Kumar et al., 2010). Symptoms typically begin to manifest around midlife, 35–45 years of age, with the majority of patients having between 40 and 50 CAG repeats. An expansion greater than 50 CAG repeats results in juvenile onset (Telenius et al., 1993).

Following clinical diagnosis, usually triggered by motor symptom onset, HD is fatal within 15–20 years. During the initial stages of the disorder, HD patients experience cognitive deficits in mood, attention, procedural memory and executive functions (Lemiere et al., 2004; Estrada-Sánchez et al., 2015a). They also experience mild motor abnormalities such as tremor or tics. Following these mild changes, a hyperkinetic stage emerges. Motor abnormalities, primarly chorea, the defining feature of HD, manifests with abrupt and uncontrolled exaggeration of gestures and spontaneous movements of the trunk, face, and limbs. These movements cannot be volunatrly suppressed and worsen during periods of heighted stress (Kremer, 2001). Motor skills continue to deteriorate affecting gait, speech and swallowing. During this time patients also suffer dramatic weight loss and muscle wasting despite maintaining a high caloric diet (Estrada-Sánchez et al., 2015a). Eventually choreic movements are replaced by bradykinesia and rigidity and ultimately results in death primarily due to pneumonia and heart disease (Estrada-Sánchez and Rebec, 2012).

Neuropathological features of HD include: reduced brain weight, enlarged lateral ventricles, and decreased striatal and cortical volume. The gross atrophy of the caudateputamen (striatum) results mainly from loss of medium spiny neurons (MSNs), which account for >90% of the striatal neuronal population. Striatal neuronal loss is usually accompanied by loss of cortical pyramidal neurons (CPNs), primarily in motor and premotor areas (Aylward et al., 1998; Rosas et al., 2001; Menalled et al., 2002). Because CPN targets include MSNs, the degeneration and loss of these neurons indicate impaired corticostriatal connectivity and dysfunctional information processing (Hamilton et al., 2003; Miller et al., 2011; Estrada-Sánchez and Rebec, 2012). Thus, although mutant HTT is widely expressed in brain and body, primary neuropathology occurs in the cerebral cortex and the striatum. This review will focus on changes in corticostriatal circuitry and how glutamate (Glu), the transmitter released by CPNs, and its interaction with dopamine (DA), a key modulator of striatal function, affect this circuitry in HD. Multiple animal models of HD developed soon after identification of the htt gene form the basis for this line of research.

# ANIMAL MODELS

A few years following identification of the gene, transgenic rodent models were developed in order to investigate mechanisms underlying the HD phenotype. Genetic murine models can be separated into three groups: transgenic truncated models, transgenic full-length models, or knock-in (KI) models. Truncated models solely express the first exon of the htt gene, where the CAG expansion occurs. Full-length models express the entire human mutant htt gene. Both of these transgenic models contain two non-mutated endogenous htt alleles making the mutant HTT to HTT ratio differ from that seen in humans. This raises translational concerns as some cellular alteration in the truncated and full-length models may not be an accurate reflection of the human condition. KI models avoid this translational confound by directly inserting the CAG expansion into exon 1 of the endogenous htt gene; thereby mirroring the genetic construct of HD patients.

#### Truncated Models

The truncated R6 line is the oldest and one of the most studied transgenic mouse models. R6/2 mice, with a CAG repeat length of ∼150, have an aggressive phenotype. R6/2 mice have a shortened lifespan of 3–5 months with symptom onset beginning as early as 1 month of age (Mangiarini et al., 1996). R6/1 mice have a longer life expectancy than R6/2 mice. With 110 to 115 CAG repeats, R6/1 mice become symptomatic between 4 to 5 months with more robust motor symptoms emerging around 6 to 7 months of age followed by death at 10 to 14 months (Mangiarini et al., 1996; Naver et al., 2003).

The R6 line has neuronal intracellular and intranuclear aggregates contacting the mutant HTT fragment (Davies et al., 1997); as well as reduced striatal volume (Stack et al., 2005). Motor deficits including tremor, reduced climbing, and clasping has been observed in R6 truncated models (Mangiarini et al., 1996; Miller et al., 2008). Susceptibility to epileptic seizures (Mangiarini et al., 1996) and decreased learning has also been observed in these truncated models (Murphy et al., 2000). These behavioral changes are correlated to neurochemical changes in basal ganglia circuitry (Bibb et al., 2000; Ariano et al., 2002; Johnson et al., 2006; Miller et al., 2008; Ortiz et al., 2010).

# Full-Length Models

Two full-length transgenic HD models have been developed: the yeast artificial chromosome (YAC) and bacteria artificial chromosome (BAC). Several lines of the YAC mouse model have been generated based on the number of CAG repeats: YAC18 (control), YAC46, YAC72, and YAC128. Compared to truncated models, YAC and BAC mice have a slower disease progression. As is the case with humans, the YAC128 mouse model displays striatal followed by cortical atrophy as well as hyperkinetic followed by hypokinetic activity in an agedependent manner (Slow et al., 2003; Van Raamsdonk et al., 2005). The BAC model, which carries 97 CAA-CAG repeats, also exhibits a progression motor deficits, decreased striatal and cortical volume, and decreased MSN synaptic activity (Gray et al., 2008). BACHD mice have also shown abnormal striatal and cortical firing patterns (Estrada-Sánchez et al., 2015b).

The YAC128 mouse model is one of the few transgenic models exhibiting both the hyper- and hypoactive phenotype seen in humans as well as striatal followed by cortical atrophy (Slow et al., 2003; Van Raamsdonk et al., 2005). In the YAC128 mouse model, by 3 months of age a hyperkinetic phenotype emerges this is then followed by progressive motor deficits. By 9 months of age, motor deficits are apparent and the hypokinetic phenotype is accompanied by striatal atrophy; when the animal reaches 12 months of age cortical atrophy also starts to occur (Slow et al., 2003). Because the onset of symptoms does not occur as rapidly in the YAC128 model as truncated models; initial symptoms seen in humans as well as neurochemical changes throughout the progression of this biphasic (hyper- and hypo-) active disorder can be studied in more depth.

#### Knock-in Models

Multiple HD KI models have been created. KI models carry the expanded CAG repeats within the native murine htt gene, closely mimicking the genetic context of patients with HD. KI models provide stronger construct validity compared to other transgenic rodent models. However the protracted phenotype observed in KI models has limited their use. Current KI mouse models show subtle behavioral, histopathological, and molecular phenotypes compared to the transgenic models that over express mutant HTT (truncated and full-length models; Chang et al., 2015). The CAG140 KI mouse model has a normal lifespan, with overt symptoms emerging after 20 months of age (Hickey et al., 2008). CAG140 mice do have the characteristic nuclear aggregates and decreased striatal volume seen in other models (Menalled et al., 2003; Hickey et al., 2008). The Q175 KI model was derived by a spontaneous germline CAG expansion from the previously constructed CAG140 KI mouse model (Menalled et al., 2012). Initial characterization of this new mouse model showed decreased body weight, body tremor, abnormal gait, and activity level. Decreased striatal and cortical volume has also been observed along with decreased cognitive ability (Heikkinen et al., 2012; Menalled et al., 2012). These behavioral changes observed in HD KI models mirror changes seen other mouse models and HD patients (Mangiarini et al., 1996; Slow et al., 2003; Miller et al., 2011; Estrada-Sánchez et al., 2015a).

#### CORTICOSTRIATAL PATHWAY OVERVIEW

As the primary input nucleus of the basal ganglia, the striatum receives excitatory afferents from the entire cortical mantel as well as the thalamus (Kreitzer and Malenka, 2008). Because the primary neuropathology of HD involves the loss of MSNs and CPNs, the HD behavioral phenotype is likely due to dysfunction of the cortical-basal ganglia system. Therefore a greater understanding of the corticostriatal pathway and its activity is necessary for a fuller understanding of both HD pathology and behavioral phenotype.

#### Cortex

In humans, the cortical mantel is divided into six layers with astrocytes homogenously distributed throughout. The different types of cortical neurons make local cortical connections between cells of different layers as well as send and receive projections from other brain regions, such as the striatum. Layer III, V, and VI all contain CPNs. CPNs from layer III and V are the primary input to the basal ganglia, brain stem, and spinal cord. CPNs from layer VI send projection to the thalamus (McGeorge and Faull, 1989; Shipp, 2007; Estrada-Sánchez and Rebec, 2013). Post-mortem tissue taken from HD patients has a 30% reduction in CPNs for these three cortical layers (Cudkowicz and Kowall, 1990; Hedreen et al., 1991; Sotrel et al., 1991; Heinsen et al., 1994). While the entire cortical mantel projects to the striatum, this review will be referring to CPNs of the motor cortex due to the motor cortex's involvement in execution and control of voluntary movements. This reduction of CPNs as well as studies showing dysfunction CPNs neuronal firing (Walker et al., 2008) potentially underlies the HD behavioral phenotype.

Two types of striatal-projecting CPNs have been identified: (1) pyramidal tract (PT)-type neurons; and (2) intratelencephalically projecting (IT)-type neurons. PT-type neurons project to ipsilateral striatum, thalamus, and subthalamic nucleus (STN) as well as the regions of the spinal cord and brain stem. IT-type neurons project to the ipsilateral and contralateral striatum and other cortical layers (Shepherd, 2013). PT-type neurons are mainly found in the lower cortical layer V and IT-type neurons are primarily located in layer III and upper half of layer V. The different morphological features and location of PT- and IT-type neurons suggest these two types of neurons differentially target striatal MSNs, with IT-type neurons differentially targeting MSN of the direct pathway and PT-type neurons preferentially innervate MSNs of the indirect pathway (Reiner et al., 2003; Parent and Parent, 2006; Chen et al., 2013).

#### Striatum

MSNs are GABAergic neurons that constitute 95% of the neuronal population of the striatum. Interneurons in the striatum are also primarily GABAergic, with some cholinergic interneurons (Kreitzer and Malenka, 2008). There are two major subtypes of striatal MSNs based on their protein expression and their axonal projections: (1) Striatonigral MSNs express substance P (subst P), dynorphin, and D1-like DA receptors. They project directly to the basal ganglia output nuclei: internal globus pallidus (GPi) and substantia nigra pars reticulata (SNr); and (2) Striatopallidal MSNs contain enkephalin (enk) and express D2-like DA receptors. Striatopallidal MSNs projects to the external globus pallidus (GPe) and are part of the indirect pathway. Both striatonigral and striatopallidal MSNs integrate glutamatergic cortical and thalamic inputs and relay that information to downstream basal ganglia nuclei (Kreitzer and Malenka, 2008).

These two subtypes of MSNs have been used to segregate basal ganglia circuitry into two pathways: the direct pathway containing striatonigral D1-enriched MSNs and the indirect pathway containing striatopallidal D2-enriched MSNs. These pathways are believed to act in opposing ways to control movement. According to this view, the direct pathway, which consists of the striatonigral MSNs, initiates movement; whereas, the indirect pathway, consisting of striatopallidal MSNs, inhibits movement (Alexander and Crutcher, 1990). While these pathways are not completely isolated (Haber et al., 2000; Cui et al., 2013), for simplicity, we will discuss them as parallel circuits, depicted schematically in **Figure 1**.

The direct pathway is thought to facilitate movement (Albin et al., 1989; DeLong, 1990). Striatonigral MSNs in the striatum receive excitatory, glutamatergic projections from the cortex and thalamus. This results in excitation of the striatal GABAergic MSNs, which in turn inhibits the GABAergic projection in the GPi/SNr. The inhibition of the inhibitory GABAergic neurons in the GPi/SNr results in less inhibition on the thalamus glutamatergic neurons. This results in excitation of the motor cortex and initiation of voluntary movement. By this schematic, dysregulation of striatonigral MSN projection neurons would result in rigidity and bradykinesias.

The indirect pathway is presumed to inhibit movement (Albin et al., 1989; DeLong, 1990; Durieus et al., 2009;

FIGURE 1 | Simplified schematic representation of basal ganglia circuitry highlighting cortical and striatal outputs. Open arrows represent excitation, closed arrows represent inhibition. Neurotransmitters are indicated in parenthesis. Intra-telencephalically projecting (IT)-type CPNs project to the striatongiral D1-enriched MSNs of the direct pathway. Pyramidal tract (PT)-type CPNs project to the stratopallidal D2-enriched MSNs of the indirect pathway. Abbreviations: Glu, glutamate; enk, enkephalin; subst P, substance P.

Kravitz et al., 2010). Striatopallidal MSNs inhibit GABAergic neurons in the GPe. This in turn results in less inhibition on the glutamatergic projection in the STN. The glutamatergic projections from the STN excite the GABAergic neurons in the GPi/SNr which results in greater inhibition to the thalamus and decreased signaling to the motor cortex (Alexander and Crutcher, 1990). Thus dysregulation of MSNs in the indirect pathway result in uncontrollable voluntary movements, such as chorea and tremor (Bateup et al., 2010).

Glu projections from the cortex are not the only influence on basal ganglia circuitry. DA projections from the substantia nigra pars compacta (SNc) play a modulatory role. Evidence suggests D1-like receptor signaling facilitates glutamatergic signaling in direct pathway, striatonigral MSNs. In contrast, D2 like receptor activation inhibits MSNs in the indirect pathway (**Figure 1**; Surmeier et al., 2007). Therefore DA projections from the SNc results in excitation (D1-like) or inhibition (D2 like) of the direct or indirect pathway, respectively, the ultimate results is facilitation of movement. Therefore a dysregulation of Glu or DA, which modify the excitatory responses induced by Glu (Kiyatkin and Rebec, 1999; Cepeda and Levine, 2006), could results in uncontrolled involuntary movements or bradykinesia.

#### DYSREGULATION OF THE CORTICOSTRIATAL PATHWAY

Motor function is in part shaped by CPNs projecting to striatal MSNs in the so-called corticostriatal pathway. The cerebral cortex and striatum are the most affected brain areas in HD, with massive MSN loss in the striatum. Not all MSNs however are equally vulnerable (Vonsattel and DiFiglia, 1998; Galvan et al., 2012). Morphological evidence shows time-dependent, differential alterations in the two populations of MSNs. MSN in the indirect pathway tend to be preferentially lost prior to MSN in the direct pathway. Evaluation of indirect striatopallidal MSNs and their projections (i.e., enk) are lost in postmortem tissue of symptomatic patients and in presymptomatic and early symptomatic brains of HD mouse models. In contrast, direct striatonigral MSNs projecting subst P are not as greatly affected and in some cases appear unaffected till later, advance stages of the disease (Albin et al., 1992; Sapp et al., 1995; Menalled et al., 2000).

In keeping with the direct/indirect model of basal ganglia movement control, the preferential loss of MSN in the indirect pathway can result in the inability to control voluntary movement, resulting in a hyperkinetic phenotype (chorea). The dysfunction of the direct pathway in the later stage of the disease would result in an inability to facilitate movement; resulting in a rigidity and bradykinesia (see above). Thus changes in output from striatal MSNs, which receives Glu input from the cortex, correlates with the behavioral phenotype of HD.

Particular focus has been placed on the neuronal activity of the cortex and striatum. Regardless of genetic construct and phenotypic onset, changes ranging from membrane properties to impaired neuronal processing in the corticostriatal pathway have been identified in multiple mouse models of HD (Walker et al., 2008; Miller et al., 2011; Heikkinen et al., 2012; Estrada-Sánchez et al., 2015a,b). Abnormal MSN activity has been observed in presymptomatic and symptomatic transgenic mouse models (Rebec et al., 2006; Miller et al., 2008, 2011; Cayzac et al., 2011; Estrada-Sánchez et al., 2015b). Abnormal CPN activity has also been observed in HD mouse models (Walker et al., 2008, 2011). Multiple transgenic mouse models show increased firing rate with decreased burst firing and firing variability in the dorsal striatum and primary motor cortex. A decrease in synchronous firing and coherent bursting was also observed in the cortex and striatum of transgenic murine models (Walker et al., 2008; Höhn et al., 2011; Miller et al., 2011; Estrada-Sánchez et al., 2015b). Changes in neuronal activity level and variability along with changes in burst firing, which is involved in synaptic plasticity and enhanced information transmission, demonstrate neuronal dysfunction in the striatum and cortex thereby indicating a communication problem between the cortex and dorsal striatum.

In vitro electrophysiology analysis showed resting membrane potential of MSNs are more depolarized in Q175 mice as well as show significantly less rheobasic current (the minimum current amplitude that results in depolarization) in HOM mice (Heikkinen et al., 2012), indicating Q175 striatal MSNs become progressively excitable and abnormal corticostriatal neuronal activity emerges as the animals age. Collectively, these studies suggest that dysregulation of MSN firing patterns are a cardinal feature of HD.

Besides abnormal electrophysiological properties of CPN and MSN, dysfunctional network activity has been described both in HD patients and HD models. Neurons operate in a coordinated way and large neuronal population activity can be monitored by local field potentials (LFPs). Spectral analysis of LFPs recorded from the striatum of freely behaving R6/2 mice revealed an increase in power in theta (7–14 Hz) and gamma (35–45 Hz) bands (Miller et al., 2011). Increased power in theta/alpha (4–12 Hz) and low gamma (35–45 Hz) bands also has been observed in LFP activity recorded in the globus pallidus of HD patients (Groiss et al., 2011; Hong et al., 2012). Thus, expression of mutant HTT cause changes in corticostriatal processing that may underlie HD cognitive and behavioral deficits.

Interneurons also may play a role in corticostriatal dysfunction in HD. GABAergic control of striatal MSNs comes primarily from feedforward inhibition derived from local inhibitory interneurons (Tepper et al., 2004). Specifically, parvalbumin-expressing (PV) fast-fast spiking (FS) GABAergic interneurons are the main source of this feedforward inhibition (Gittis et al., 2010). Because PV FS interneurons receive strong cortical innervation (Ramanathan et al., 2002) and respond with faster latency to cortical stimulation than MSNs (Mallet et al., 2005), PV FS cells are able to make feedforward inhibition work effectively. Furthermore, PV FS interneurons project strongly to MSNs (Taverna et al., 2007) with a slight targeting preference for direct-pathway MSNs (Gittis et al., 2010).

In a recent study, significant decreases in striatal PV FS interneurons were observed in postmortem tissue of HD patients with varying degrees of atrophy (Reiner et al., 2013). These changes in interneurons could result in disrupted direct-pathway MSN communication to the GPi. In HD, therefore, local inhibition on the subst P-containing striatonigral D1-enriched MSNs (direct pathway) is limited. According to the highly schematic and simplified circuitry shown in **Figure 1**, decreased inhibition on the direct pathway would result in more GABA release in GPi, less GABA release in thalamus, and thus more excitation of motor cortex.

Increased firing of FS interneurons has also been observed in HD mouse models. For example, increased GABAergic synaptic activity in symptomatic HD mouse models was observed, primarily from feedforward inhibition of indirect pathway MSNs (Cepeda et al., 2013). Thus, an increase in activity in the indirect pathway from striatum would result in decreased movement during the later stages of HD. In short, inhibitory interneurons act locally in dorsal striatum to influence basal ganglia output.

#### GLUTAMATE IN HUNTINGTON'S DISEASE

Pioneering studies by Wong et al. (1982) demonstrate perturbation in the synthesis of Glu by corticostriatal neurons. Since then further investigation into Glu receptors and Glu uptake have been investigated in relation to deviations in the corticostriatal pathway.

One major hypothesis is that excitotoxicity underlies striatal neurodegeneration in HD (DiFiglia, 1990). Excitotoxicity can be a result from: (1) an increase in responsiveness of Glu receptors; or (2) an increase in synaptic Glu. The responsiveness of Glu receptors can change due to either an increase in the number of receptors or receptor density or a change in receptor composition or signaling properties. An increase in synaptic Glu could be due to an increase in release or a decrease in uptake. The literature primarily focuses on N-methyl-D-aspartate (NMDA) receptors and removal of excess Glu via glutamate transporter 1 (GLT1).

#### NMDA Receptors in HD

NMDA receptors are ionotropic Glu receptors that serve as essential mediators of neuronal function, synaptic transmission, synaptic plasticity, and aspects of neural development (Purves et al., 2008; Iversen et al., 2009). In HD degeneration of MSNs occurs, and while the mechanism behind this selective degeneration is not well understood, convergent evidence supports the role for NMDA receptor mediated excitotoxicity (Beal et al., 1986; Ferrante et al., 1993; Cepeda et al., 2007; Fan and Raymond, 2007).

Striatal injections of NMDA receptor agonists in rodent and non-human primates reproduce the pattern of neuronal damage seen in HD (Beal et al., 1986; Ferrante et al., 1993). Furthermore electrophysiological assessment of presymptomatic and symptomatic R6/2 mice found larger NMDA currents and NMDA-induced Ca2<sup>+</sup> influx compared to littermate controls (Cepeda et al., 2001). It was speculated the striatal neuronal subpopulation that displayed the most elevated response to NMDA application corresponded to indirect MSNs. The increase in NMDA response as well as enhancement of intracellular calcium suggests changes in NMDA receptors signaling could result in excitotoxicity and neuronal death.

NMDA exposure to the MSNs cultured cells resulted in a potentiation in apoptosis of YAC72 and YAC128 MSNs compared to the healthy YAC18 control MSNs. Moreover producing a reduction in NMDA receptor mediated current and calcium influx in YAC72 MSNs to levels seen in YAC18 MSNs resulted in a reduction of NMDA receptor-mediated apoptosis (Shehadeh et al., 2006). Thus, controlling NMDA signaling brought the rate of apoptosis in HD mice to a comparable level seen in WT controls.

However NMDA receptor activation has been shown to promote both neuronal cell survival as well as neuronal cell death (Hardingham and Bading, 2010). Synaptic NMDA receptor activity has been shown to reduce mhtt toxicity and act as a neuroprotectant while extrasynaptic NMDA receptors have been associated with promoting cell death (Okamoto et al., 2009; Milnerwood et al., 2010). Okamoto et al. (2009) found synaptic NMDA receptor activity reduces mutant htt toxicity by increasing the formation of non-toxic mutant htt inclusions by a process involving the up-regulation of protein chaperones thereby rendering neurons more resistant to mhtt-mediated cell death. In contrast, stimulation of extrasynaptic NMDA receptors increased the vulnerability of neurons to cell death by impairing a neuroprotective CREB-PGC-1α cascade. Furthermore treatment with lower doses of memantine, which blocks extrasynaptic but not synaptic NMDA receptors, improves neuropathological and behavioral manifestations of HD (Okamoto et al., 2009). Also presymptomatic YAC128 mice treated at a low level dose that preferentially targets extrasynaptic NMDA receptors resulted in reversal of early signaling and motor deficits (Milnerwood et al., 2010). Though at higher doses, when memantine also blocks synaptic NMDA receptors, memantine worsened the manifestation of HD (Okamoto et al., 2009). This perturbation in the balance between synaptic NMDA and extrasynaptic NMDA receptor activity could contribute to excitotoxicity and neuronal dysfunction in HD.

#### Glutamate Uptake in HD

Inadequate Glu uptake has been reported for HD patients as well as transgenic mouse models of HD. The removal of Glu from the synaptic cleft is controlled by several transport proteins; including: GLT1, L-glutamate/L-aspartate transporter (GLAST), and excitatory amino-acid carrier 1 (EAAC1). Note GLT1 is also known as in EAAT2 in humans and is primarily located on glial cells. Glu transporters play a vital role in maintaining the level of extracellular Glu by removing Glu from the synapse; dysregulation of this Glu uptake can result in slow Glu clearance prompting Glu spillover and increased receptor activation (Beart and O'Shea, 2007; Estrada-Sánchez et al., 2008). These high concentrations of Glu can result in excitotoxicity, a pathological process in which neuronal cells are damaged or killed due to excessive stimulation. Therefore the functionality of the Glu transporters plays a critical role in preserving the local integrity of excitatory synaptic transmission (Marcaggi and Attwell, 2004).

Astrocytes not only play a role in Glu uptake via Glu transporters, but recent evidence indicates that mutant htt can affect astrocytes in a way that increases Glu release and MSN excitability (Lee et al., 2013; Khakh and Sofroniew, 2014; Tong et al., 2014). Calcium and Glu imaging of astrocytes in the full-length BACHD mouse model showed increased release of Glu into extracellular space. Furthermore BACHD mice have increased de novo synthesis of Glu (Lee et al., 2013). This ability of mutant htt to increase Glu production and release could further contribute to the excitotoxicity seen in HD. Additionally, a reduction in astrocyte functional proteins, including GLT1and Kir4.1, has been associated with onset of neurological symptoms (Tong et al., 2014). Restoration of Kir4.1 channels through viral vector administration attenuated MSN excitability, improved gait, and increased the lifespan of HD mice (Khakh and Sofroniew, 2014; Tong et al., 2014). For a more thorough review of astrocytes involvement in circuitry and disease models, see Khakh and Sofroniew (2015).

GLT1 is responsible for 90% of extracellular Glu uptake and is either down regulated or dysfunctional in HD mouse models (Estrada-Sánchez and Rebec, 2012). Although the extracellular concentration of Glu is similar in HD mice to that of WT controls (NicNiocaill et al., 2001; Gianfriddo et al., 2004), the length of time Glu remains in the synapse is prolonged in HD models. A microdialysis study found Glu release is larger and takes longer to return to basal levels in the striatum of R6/1 mice (NicNiocaill et al., 2001), indicating a change in Glu uptake. Moreover, Lievens et al. (2001) showed a decrease of GLT1 mRNA in the striatum and cortex of R6/2 mice accompanied by a concomitant decrease in Glu uptake without a change in GLAST and EAAC1. Other studies have confirmed the decrease in GLT1 protein (Estrada-Sánchez et al., 2009, 2010; Sari et al., 2010), but some of these studies also found a decrease GLAST (Estrada-Sánchez et al., 2009, 2010). A study using the CAG140 KI mouse model found functional changes in EAAC1, but the overall protein level of EAAC1 in HD mice was not significantly different from the level in WT controls (Li et al., 2010). Thus, not only is Glu uptake decreased, suggesting a functional change, but expression of the Glu transporter proteins, especially GLT1, is also decreased, indicating an underlying change in the uptake machinery itself. Further discussion of this machinery and the Glu synapse, including the role of astrocytes in Glu regulation, is described elsewhere (Estrada-Sánchez et al., 2015a).

Because the experiments using the R6 mouse models were carried out before the overt neurological phenotype emerged, the down-regulation of GLT1 appears to persist throughout the progression of HD and is a likely contributor to neurodegeneration. Furthermore, stimulation of layer V afferents innervating the striatum evoked excitatory postsynaptic currents (EPSCs) mediated by Glu. The evoked EPSCs are decreased in Q175 mice, indicating dysregulation of Glu transmission in this model (Menalled et al., 2012).

Dysregulation of Glu also has been observed in human postmortem studies. In 1986, Cross et al. (1986) studied the binding of [3H]-aspartic acid to the high-affinity Glu uptake system and found a significant reduction in [3H]-aspartic acid binding in both the caudate nucleus and putamen of HD subjects. A reduction in the transporter GLT1 mRNA was also observed in the caudate and putamen of HD patients, suggesting a loss of Glu transporter (Arzberger et al., 1997). In a recent study, uptake of [3H]-glutamate was reduced by over 50% in the prefrontal cortex of HD patients with no change in GLAST but a decrease in EAAT2 (Hassel et al., 2008), further demonstrating a dysregulation in Glu clearance. Postmortem evaluation of HD patients revealed a grade-dependent decrease in striatal GLT1 expression (Faideau et al., 2010). Increasing evidence implicates deficits in Glu uptake in HD pathology and that this decrease is, at least in part, due to down regulation of the GLT1 transporter.

Manipulation of GLT1 has been tested as a therapeutic target. Acute administration of ceftriaxone, a β-lactam antibiotic known to elevate GLT1 expression (Rothstein et al., 2005), resulted in attenuation of behavioral deficits. Paw clasping and twitching were reduced in HD mice compared to WT control (Miller et al., 2008). These results not only indicated impaired Glu uptake is a major factor underlying HD pathophysiology and symptomology, but restoration of GLT1 could be a viable treatment for HD. Furthermore ascorbic acid, which is associated with GLT1, appears decreased in HD mice. Following treatment with ceftriaxone, ascorbic acid was increased in HD mice (Miller et al., 2012). These results suggest a dysfunction of the Glu system in HD. An increase in Glu uptake via GLT1 is a viable therapeutic strategy because it removes excess Glu from the synapse which can result in excitotoxicity.

#### DOPAMINE IN HUNTINGTON'S DISEASE

Aberrant DA signaling underlying behavioral abnormalities in HD was first proposed in 1970 based on evidence that asymptomatic offspring of individuals with HD developed dyskinesia following levodopa (L-DOPA) administration (Klawans et al., 1970). Further studies of HD patients suggest increased DA release induces chorea while a reduction in DA leads to akinesia. Levels of DA were found to be 11-fold higher in the cerebrospinal fluid (CSF) of early-stage HD patients compared to healthy individuals (Garrett and Soares-da-Silva, 1992), but decreased DA was found in patients studied at a later stage in the disease (Kish et al., 1987). In line with this finding, recent work has shown biphasic changes in DA release in the R6/1 mouse model of HD: enhanced DA release during pre-symptomatic stages and severely attenuated DA release in symptomatic animals (Dallérac et al., 2015). Collectively, these results suggest a biphasic, age-dependent change in DA transmission in HD.

The progressive reduction in striatal DA levels during later stages of HD has been confirmed in multiple studies (Hickey et al., 2002; Johnson et al., 2006; Ortiz et al., 2010, 2011; Callahan and Abercrombie, 2011; Dallérac et al., 2015), but it remains unclear if early stages promote DA excess. One study using transgenic HD rats showed increased striatal tyrosine hydroxylase expression and increased DA neurons during the early symptomatic stage (Jahanshahi et al., 2010). These rats, however, also have been shown to have impaired DA release (Ortiz et al., 2012). Further work is required to clarify DA changes during the hyperkinetic stages of HD.

Deficits in DA levels and/or release during late HD stages in rodent models have been attributed either to impaired vesicle loading (Suzuki et al., 2001) or to a reduction in the DA readily releasable pool (Ortiz et al., 2010). In vitro studies of R6/2 and R6/1 mice have shown a decrease in extracellular DA concentrations as well as a decrease in DA release (Johnson et al., 2006) and uptake (Ortiz et al., 2011). The decrease in DA release emerged during symptom onset and became significant as the animals aged. Importantly, both R6/2 and R6/1 mice have late-stage reductions in DA release, but only R6/1 mice have decreased uptake. Moreover, the decrease in uptake occurred towards the end of the R6/1 lifespan, suggesting that the R6/2- R6/1 uptake difference could be explained by the increased lifespan of the R6/1 model (Mangiarini et al., 1996).

In both HD patients and mouse models, D1 and D2 receptors are compromised during the early as well as late stages of the disease (Chen et al., 2013). An autoradiography study found a decrease in both DA receptor families (Richfield et al., 1991). Positron emission tomography studies have also found decreased striatal D1 and D2 receptors in asymptomatic carriers of mutant htt (Weeks et al., 1996; van Oostrom et al., 2009). Multiple studies using the R6 line have also found decreased D1 and D2 receptors signaling and receptor binding (Cha et al., 1998; Bibb et al., 2000; Ariano et al., 2002). A significant reduction in D1 and D2 mRNA was also found in YAC128 mice (Pouladi et al., 2012). Collectively these studies indicate DA receptors, which are involved in modulating basal ganglia circuitry, are disrupted throughout HD progression and in some cases prior to disease onset.

The traditional, albeit simplified view of basal ganglia circuitry, proposed behavioral abnormalities in HD are the result of initial dysregulation of the striatopallidal MSNs of the D2-enriched indirect pathway, resulting in hyperkinetic choreic movements. This is then followed by dysregulation of the striatonigral D1-enriched MSNs of the direct pathways, resulting in hypokinesia (Spektor et al., 2002). However recent studies have suggested a counter view, the direct pathway is affected first by D1 receptor activation. Striatal MSNs receive glutamatergic inputs modulated by DA (Cepeda and Levine, 2006). Different DA receptors are localize in different striatal cell populations with less than 10% of D1 and D2 MSNs co-localized (Deng et al., 2006; Lobo et al., 2006; Cepeda et al., 2008). This lead to the idea that DA modulates the direct and indirect pathway by opposing action, with DA release increasing activity in the direct pathway and reducing activity in the indirect pathway (see above; DeLong and Wichmann, 2007).

However it has been difficult to parcel out the exact mechanism by which D1 and D2 DA receptors influence the direct and indirect pathway. Recently animal expressing enhanced green fluorescent protein (GFP) in either D1 or D2 receptors have been studied. Using electrophysiology with these mice a DA receptor-specific modulation of EPSC and NMDA and AMPA currents was found. D1 receptor agonist increased EPSC in D1 MSNs projecting to the direct pathway whereas D2 receptor agonist decreased EPSCs in D2 cells of the indirect pathway. Changes in NMDA and AMPA current were also DA receptor-specific. A D1 agonist resulted in increased current in direct pathway MSNs, and a D2 receptor agonist resulted in decreased NMDA and AMPA current in the indirect pathway MSNs (André et al., 2010). Thus Glu receptor-mediated responses in MSNs are modulated by the type of DA receptor abundantly expressed in that cell.

Additionally, these results suggest D1 receptor activation leads to increased Glu receptor response on MSNs in the direct pathway, which would result in increased GABA release to the GPi resulting in hyperkinetic movement. An electrophysiological study testing full-length HD mouse models crossed with mice expressing enhanced GFP found direct pathway MSNs received more excitatory inputs than control animals during the early hyperkinetic stage. The indirect pathway MSNs were not as affected during this early stage, and during the hypokinetic stage both pathways received less excitatory inputs compared to WT controls (André et al., 2011; Galvan et al., 2012). Further suggesting increased DA in the early stages of HD contributes to the hyperkinetic symptoms of HD by modifying the direct pathway first.

Morphological assessments have found the indirect pathway to be affected initially in HD (see above). Electrophysiological

#### REFERENCES


data from mice expressing GFP now suggest the direct pathway is affected in the early stages of HD via D1 receptor activation. More than likely it is a combination of D1 receptor activation in the direct pathway in conjunction with decreased function of the indirect pathway. Since both theories would result in increased GABAergic output to the direct pathway, the ultimate result is an exacerbated HD phenotype.

#### CONCLUSION

HD is a multifaceted disorder displaying cognitive, psychiatric, and motor deficits. The dysregulation in the corticostriatal pathway contributes to these associated characteristics of HD and potentially underlie the HD phenotype. Dysregulation of Glu and DA have been documented in HD patients as well as rodent models, and the abnormal electrophysiology, due to these neurotransmitters, further indicates that disrupted communication in the corticostriatal pathway in HD.

#### AUTHOR CONTRIBUTIONS

Both authors contributed to this review article. KDB: prepared a draft, which was updated and edited by GVR.

#### FUNDING

We gratefully acknowledge support from the CHDI Foundation (Grants A-7449).


of Huntington disease. J. Neurosci. Res. 66, 525–539. doi: 10.1002/ jnr.1244


neurotoxicity in the R6/2 Huntington's disease mice. Neurochem. Res. 35, 1156–1163. doi: 10.1007/s11064-010-0168-5


and asymptomatic carriers of the HD mutation–a longitudinal follow-up study. J. Neurol. 251, 935–942. doi: 10.1007/s00415-004-0461-9


in preclinical Huntington's disease. Eur. J. Neurol. 16, 226–231. doi: 10.1111/j. 1468-1331.2008.02390.x


of Huntington's disease. Ann. Neurol. 40, 49–54. doi: 10.1002/ana.4104 00110


**Conflict of Interest Statement**: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2016 Bunner and Rebec. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution and reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Preliminary Evidence of Apathetic-Like Behavior in Aged Vesicular Monoamine Transporter 2 Deficient Mice

Aron Baumann, Carlos G. Moreira, Marta M. Morawska, Sophie Masneuf , Christian R. Baumann and Daniela Noain\*

Department of Neurology, University Hospital of Zurich, Zurich, Switzerland

Apathy is considered to be a core feature of Parkinson's disease (PD) and has been associated with a variety of states and symptoms of the disease, such as increased severity of motor symptoms, impaired cognition, executive dysfunction and dementia. Apart from the high prevalence of apathy in PD, which is estimated to be about 40%, the underlying pathophysiology remains poorly understood and current treatment approaches are unspecific and proved to be only partially effective. In animal models, apathy has been sub-optimally modeled, mostly by means of pharmacological and stress-induced methods, whereby concomitant depressive-like symptoms could not be ruled out. In the context of PD only a few studies on toxin-based models (i.e., 6-hydroxydopamine (6-OHDA) or 1-methyl-4-phenyl-1,2,3,6 tetrahydropyridine (MPTP)) claimed to have determined apathetic symptoms in animals. The assessment of apathetic symptoms in more elaborated and multifaceted genetic animal models of PD could help to understand the pathophysiological development of apathy in PD and eventually advance specific treatments for afflicted patients. Here we report the presence of behavioral signs of apathy in 12 months old mice that express only ∼5% of the vesicular monoamine transporter 2 (VMAT2). Apathetic-like behavior in VMAT2 deficient (LO) mice was evidenced by impaired burrowing and nest building skills, and a reduced preference for sweet solution in the saccharin preference test, while the performance in the forced swimming test was normal. Our preliminary results suggest that VMAT2 deficient mice show an apathetic-like phenotype that might be independent of depressive-like symptoms. Therefore VMAT2 LO mice could be a useful tool to study the pathophysiological substrates of apathy and to test novel treatment strategies for apathy in the context of PD.

#### Edited by:

Marcelo Merello, Fundación para la Lucha Contra las Enfermedades Neurológicas de la Infancia, Argentina

#### Reviewed by:

Francesca Morgante, University of Messina, Italy Gianfranco Spalletta, Fondazione Santa Lucia (IRCCS), Italy

> \*Correspondence: Daniela Noain daniela.noain@usz.ch

Received: 25 April 2016 Accepted: 03 November 2016 Published: 18 November 2016

#### Citation:

Baumann A, Moreira CG, Morawska MM, Masneuf S, Baumann CR and Noain D (2016) Preliminary Evidence of Apathetic-Like Behavior in Aged Vesicular Monoamine Transporter 2 Deficient Mice. Front. Hum. Neurosci. 10:587. doi: 10.3389/fnhum.2016.00587 Keywords: apathy, depression, PD mice, vesicular monoamine transporter 2, goal-directed behaviors

#### INTRODUCTION

About 40% of all patients with Parkinson's disease (PD) are estimated to suffer from apathy (den Brok et al., 2015), a behavioral syndrome that is characterized by loss of or diminished motivation for goal-directed actions, affecting behavior, emotions and cognition (Robert et al., 2009; Pagonabarraga et al., 2015). Apathy in PD is of clinical relevance, because of its association with poorer prognosis and raised caregiver burden (Pedersen et al., 2009b; Schiehser et al., 2013), and because it is often a chronic condition which is difficult to treat (van Reekum et al., 2005; Benito-León et al., 2012; Skorvanek et al., 2013).

Apathy in PD is highly comorbid with depression and can easily be misinterpreted as such (van Reekum et al., 2005). However, the progression of apathy in PD is rather linked to the progression of the typical motor-symptoms than with depression (Zahodne et al., 2012), probably indicating independent roots for both conditions in the context of PD. In this line, apathy might be associated with a hypodopaminergic state in the mesocorticolimbic pathway in PD (Czernecki et al., 2008; Pagonabarraga et al., 2015), although lesions and neuroimaging studies suggest the dysfunction of frontal-subcortical circuits, especially those linking the ventromedial prefrontal cortex to related regions in the basal ganglia, also to be relevant for the development of apathy (van Reekum et al., 2005; Levy and Czernecki, 2006; Chase, 2011; Santangelo et al., 2013; Robert et al., 2014).

Despite the increasing awareness on the importance of apathy as a comorbid condition in PD, specific treatment strategies for patients are lacking. Furthermore, assessment tests in rodents for preclinical study of apathy are poorly developed. Since apathetic-like symptoms in rodents might be mistaken for depressive-like signs, tests to distinctively assess those symptoms would be favorable. We therefore selected three established behavioral tests, the burrowing test, the nest building test and the saccharin preference test, which we hypothesize might measure apathetic-like symptoms in mice. The burrowing behavior probably constitutes a motivationdependent behavioral need, which is based in the behavior itself and not on its functional consequences (Sherwin et al., 2004). This finding is akin to human apathetic patients, who show drastically reduced self-generated voluntary and purposeful behaviors (Levy and Dubois, 2006). Moreover, it is suggested that burrowing represents a measure of well-being (Jirkof, 2014) and a murine form of the ability to perform goal-directed ''activities of daily living'', like taking care of oneself and others (Deacon, 2009; Jirkof, 2014). The same has been suggested for the nest building behavior (Deacon, 2009; Jirkof, 2014). Because dysfunction of motivation is a core feature of apathy (Robert et al., 2009) and decreased well-being and goal-directed activities are also significantly impaired in patients with apathy (Marin, 1991), we speculate that decreased performance in these tests could possibly reflect an apathetic-like phenotype in mice. The preference for sweet solutions has been proposed to measure anhedonia (Malatynska et al., 2012; Overstreet, 2012; Klein et al., 2015), which is not only a feature of depression, but also of apathy (Pagonabarraga et al., 2015) and has been shown to be more closely associated with apathy than with depression in PD (Kaji and Hirata, 2011). To differentiate apathetic-like from depressive-like symptoms we used the forced swimming test, a widely used test to assess signs of depression in rodents (Borsini and Meli, 1988; Bourin et al., 1998; Cryan et al., 2005; Petit-Demouliere et al., 2005).

In this line, selecting tests that measure dopamine-modulated goal-directed and/or reward-related behaviors is key to reliably assess apathy in animals. For instance, it has been shown that nest building behavior in rodents is disrupted by the administration of dopamine receptor antagonists and restored by application of agonists (Giordano et al., 1990; Silva et al., 2001) or gene therapy for dopaminergic restoration in dopamine-deficient mice (Szczypka et al., 2001). Moreover, the impaired nest building performance in 1-methyl-4-phenyl-1,2,3,6-tetrahydropyridine (MPTP)-lesioned C57Bl/6 mice (Sedelis et al., 2000; Hofele et al., 2001), and the consequential improvement after L-DOPA injections (Sager et al., 2010), emphasize dopaminergic involvement in the test. In the same manner, it has been reported that dopaminergic systems are involved in the preference for a sweetened solution (Muscat and Willner, 1989; Brenes and Fornaguera, 2008), therefore allowing to hypothesize that the saccharin preference test might reflect apathetic-like symptoms by impairments in the reward system.

Here we aim at characterizing an established murine model of PD in terms of apathetic- and possible depressive-like phenotypes in order to potentially provide a novel biological tool for the study of apathy in the context of PD. Thus, we used 12 months old vesicular monoamine transporter 2 (VMAT2) deficient mice (VMAT2 LO mice or LO mice), a well- established animal model of PD (Mooslehner et al., 2001; Caudle et al., 2007; Taylor et al., 2009), to test for an apathy-like phenotype by means of the burrowing test, nest building test and saccharin preference test, and for a possible depressive-like phenotype by means of the forced swimming test. Our preliminary results suggest that for 12 months VMAT2 LO mice show a rather apathetic than depressive-like phenotype, therefore becoming a potential tool for the study of neurobiological substrates and specific treatments for apathy in the context of PD.

# MATERIALS AND METHODS

#### Animals

In this study we tested two different mouse genotypes. The first group consisted of VMAT2 deficient mice (LO, n = 15) on a C57Bl/6J background (Mooslehner et al., 2001; Caudle et al., 2007; Taylor et al., 2009), originally obtained by embryonic recovery from Jackson laboratories with the kind permission of Prof. Dr. G. Miller (Emory University, Atlanta, GA, USA) and bred in-house (BZL, University Hospital, Zurich, Switzerland). The second group consisted of wild-type control C57Bl/6 mice (WT, n = 19) either littermates or obtained from a supplier (Janvier Labs, Le Genest-Saint-Isle, France). All mice were male and their mean age at the start of the experiment was 371 days with a standard deviation of 8 days and a range of 355–380 days. We single-caged the mice in clear-transparent Eurotype II L plastic cages (Indulab AG, Gams, Switzerland), with food and water available ad libitum 7 days prior to behavioral testing. The room temperature was maintained constantly at 21–22◦C, and on a 12 h light-dark cycle starting at 8.00 or 9.00 a.m., according to season. Both animal housing conditions and experimental manipulations were approved by the Ethics Committee of the Cantonal Veterinary Office of Zurich, Switzerland under license no. ZH 205/2012, and were in accordance with Swiss Animal Protection Ordinance.

## Genotyping and PCR

We permanently marked VMAT2 LO mice with ear punches following a standard mice numbering procedure. We then used the ear biopsies to extract genomic DNA (DNeasy<sup>r</sup> Blood and Tissue Kit, Qiagen, Germany) and performed the genotyping by polymerase chain reaction (PCR) experiments. Genotypes were further confirmed from DNA extracted from tail samples taken immediately prior to the sacrifice of the animals. Briefly, the PCR protocol was performed with the forward primer Forward\_WT (5<sup>0</sup> -GCGAATATTCCAGTCCTCCA-3 0 ) and the reverse primer Rev\_WT (5<sup>0</sup> - CAGGCAACACCAGAAACAAT-3<sup>0</sup> ) for VMAT2 WT and Rev\_KO (5<sup>0</sup> -GGAAAGTGAGCCACCATGTAG-3<sup>0</sup> ) for VMAT2 LO. Amplification was performed in a 25 µL reaction mixture with 9.5 µL nuclease free water, 1 µL forward primer (10 µM), 0.5 µl of each reverse primer (10 µM), 12.5 µl GoTaq green master mix (Promega, Madison, WI, USA) and 1 µL of DNA using a standard thermocycler (TProfessional; Biometra, Göttingen, Germany). PCR cycle started with 4 min of initial denaturation at 94◦C, followed by 25 repeated cycles of 1 min denaturation at 94◦C, 1.5 min annealing at 57◦C and 1 min of extension at 72◦C, and ended with 7 min of final extension at 72◦C. PCR products were stained with GelRed (Biotium, Hayard, CA, USA) separated with electrophoresis at 150 V for 1 h in a 3% agarose gel (w/v) in 1 × TAE buffer and visualized under UV light.

# Behavioral Testing

#### Burrowing Test

The burrowing test has been proven to be sensitive to brain lesions, infections, pharmacological agents, pain, models of inflammatory bowel disease, schizophrenia, anxiety, Alzheimer's Disease and other pathological states (Deacon, 2006b; Jirkof, 2014). Burrowing behavior is thought to constitute a motivationdependent behavioral need (Sherwin et al., 2004) that might represent a murine form of the ability to perform ''activities of daily living'' (Deacon, 2009; Jirkof, 2014). The burrowing test measures how much an animal burrows, i.e., how much burrowing material (e.g., food pellets) of a filled container is removed during a defined time window. Mice achieve the removal by kicks and coordinated hind- and forelimb movements. This burrowing behavior is quantified by comparing the amount of burrowing material in the container at the onset of the experiment with the amount left over after the experiment. We adapted the test to the description of previously published records (Deacon, 2006a; Jirkof, 2014). Briefly, we habituated all mice to the burrowing setup prior to the experiment and performed the test in the home cage. We placed two 250 mL plastic bottles (150 mm × 55 mm, l × Ø) that served as burrowing containers and placed them in the left and right corners of the cages' posterior walls. In each cage, we filled one container (alternatively left or right) with ∼140 g of regular food pellets that served as burrowing material. The other container was left empty and served as shelter. We measured the weight of the filled container before the test (at 18:00 h), 2 h later (2 h; 20:00 h) and on the next morning (overnight; 9:00 h). After the 2 h-measurement, the burrowing container was refilled up to the same weight as at test onset. Percentage of pellets burrowed was calculated separately for the 2 h and the overnight measurement. This was done by subtracting the weight of burrowing container after the test from the weight of burrowing container at test start. Absolute values were then transformed in percentages, resulting in the percentage burrowed for the 2 h and the overnight period. Initially, we tested n = 19 WT and n = 15 LO mice. After inspection of the data, good and poor burrowers of each genotype and for each test period were identified by median split in order to analyze the groups separately. Thus, proficient burrowers were defined as those presenting burrowing rates ≥ the group median: 2 h period 85% for WT and 73% for LO, overnight period 100% for WT and LO; and poor burrowers were the animals below the median of the respective group. Two outliers were identified using the Grubbs' test and excluded from the analysis: 1 LO mouse from the 2 h poor burrowers population and 1 WT mouse from the 24 h poor burrowers population. Finally, we performed 2-sided, unpaired t-tests comparing: (1) n = 19 WT and n = 15 LO mice in both periods (whole population); (2) n = 10 WT and n = 8 LO mice for the 2 h period, and n = 10 WT and n = 8 LO mice for the overnight period (proficient burrowers); and (3) n = 9 WT and n = 6 LO mice for the 2 h period, and n = 8 WT and n = 7 LO mice for the overnight period (poor burrowers).

#### Nest Building Test

The nest building test has been used to detect deficits in nest quality in murine models of PD, Alzheimer's Disease, brain lesions, obsessive-compulsive disorders, autism, pain, prion infection and other diseases (Greene-Schloesser et al., 2011; Deacon, 2012; Jirkof et al., 2013). Similar to apathy (Czernecki et al., 2008; Santangelo et al., 2013; Thobois et al., 2013), nest building behavior has been shown to depend on dopaminergic pathways (Giordano et al., 1990; Szczypka et al., 2001; Sager et al., 2010; Paumier et al., 2013) and is assumed to represent the ability to perform ''activities of daily living'' (Deacon, 2009; Jirkof, 2014). The quality of mice nests can be quantified with the nest building test (Deacon, 2006b), simply by providing nesting material (e.g., a compressed cotton piece) overnight and rating the nest quality on the next morning. Additionally, the weight of shredded nesting material can be used as a semi-independent measure of nest complexity (Deacon, 2012). Prior to the experiment, we habituated the mice to the nesting material and performed the nest building test in the home cage, based on previous reports (Deacon, 2006b; Jirkof, 2014). Briefly, we introduced a cotton Nestlet (5 cm × 5 cm, Indulab AG, Gams, Switzerland) in the center of the cage's frontal wall. In the following morning, we weighed the amount of untorn Nestlet, calculated the difference in weight to the Nestlet at test onset and expressed the value of material shredded in percentage. Also in the next morning, we scored the nest quality on a 5-point scale; 1: Nestlet > 90% intact (Nestlet not noticeably touched); 2: Nestlet 50–90% intact (Nestlet partially torn up); 3: Nestlet < 50% intact (Nestlet mostly shredded, but no identifiable nest site); 4: Nestlet < 10% intact (identifiable, but flat nest) and 5: Nestlet < 10% intact (a ''near'' perfect nest). Where criteria for the amount of shredded Nestlet and the nest quality did not agree (e.g., a ''near'' perfect nest with a Nestlet > 50% intact), the difference was split (Deacon, 2012). This way the nest scorings can also have half values. From the n = 19 WT and n = 15 LO mice tested, we identified active and inactive nesters and separated them by median-split. Thus, animals displaying naturally poor or inactive nesting activity (nesters below the median of the groups for both parameters assessed, i.e., nest score: 4 points for WT and 3 points for LO mice; percentage of Nestlet torn: 91% for WT and 51% for LO mice) were separated from active or good nesters. Finally, we performed 2-sided, unpaired t-tests comparing: (1) n = 19 WT and n = 15 LO mice for both parameters (whole population); (2) n = 10 WT and n = 10 LO mice for the nest score and n = 9 WT and n = 8 LO for the percentage of Nestlet torn (active nesters); and (3) n = 9 WT and n = 5 LO mice for the nest score and n = 10 WT and n = 7 LO for the percentage of Nestlet torn (inactive nesters).

#### Saccharin Preference Test

The saccharin preference test has been proposed to measure symptoms of anxiety, depression and anhedonia (lack of pleasure; Forbes et al., 1996; Brenes and Fornaguera, 2008; Malatynska et al., 2012; Klein et al., 2015). However, it is highly likely that the saccharin preference test rather measures apathetic-like symptoms. It has been shown that the preference for sweetened solutions in rodents drastically depends on the dopaminergic system (Muscat and Willner, 1989; Szczypka et al., 2001; Brenes and Fornaguera, 2008; Malatynska et al., 2012), which is also a key player in the pathology of apathy (Pedersen et al., 2009a,b; Zahodne et al., 2012). The preference for sweet solutions can be measured by means of the two-bottle preference test (Bachmanov et al., 2001; Sclafani, 2006), by providing one bottle with sweetened solution, and one bottle filled with water. After a defined time window, the intake of each solution is calculated and the preference for the sweet solution is expressed as a ratio in favor of the sweet solution. In the present study we tested the preference for saccharin, also a sweet compound, and based the protocol on previous reports (Bachmanov et al., 1996; Sclafani, 2006). First, we habituated the mice with the test cages, bottles and their content for 48 h prior to the experiment (Sclafani, 2006). The test took place in clear-transparent Eurotype III test cages (425 mm × 266 mm × 155 mm, l × w × h; Indulab AG, Gams, Switzerland). The cages were equipped with sawdust, regular nest building tissue, a bottle of water, a bottle of freshly prepared 0.1% w/w saccharin solution (Sigma-Aldrich Chemie GmbH, Buchs, Switzerland) and regular food ad libitum. Both tap water and saccharin solution were provided in 100 mL plastic drinking bottles with an 8 mm diameter, 2 mm opening and 6.5 cm length stainless steel sipper spout (Techniplast S.p.A., Buguggiate, Italy). We introduced the previously weighed bottles into the test cages and weighed them again 24 h later. We calculated the absolute liquid intake separately for water and saccharin solution by subtracting the weight of the bottles after the test from the weights of the bottles prior to the test. For further analyses, we transformed the weight values into the corresponding volumes and normalized the volumes by the body weight of the respective mice. The saccharin preference ratio was calculated by dividing the volume of saccharin solution drunk by the volume of total liquid consumption. Out of n = 19 WT and n = 15 LO mice tested, one data point was excluded from the analysis due to leakage of one of the testing bottles. We analyzed the data by a 2-sided, unpaired t-test for each variable, incorporating totally n = 19 WT and n = 14 LO animals.

#### Forced Swimming Test

The forced swimming test is widely used to assess depressive-like symptoms in rodents (Porsolt et al., 1979). In the forced swimming test, mice are forced to swim in a container filled with water that does not allow for escape. Normally they rapidly stop attempts to escape and become immobile, a behavior that is quantified as the time spent immobile, which is typically interpreted as behavioral despair and a measure of depressive-like behavior in rodents. We slightly adapted the forced swimming test as described previously (Porsolt et al., 1979; Taylor et al., 2009). Briefly, we tested the mice in batches of 3–4 animals at a time and their swimming activity was recorded during 4 min from above by an infrared camera. Raw data was acquired and analyzed with the software EthoVision XT 9.0 (Noldus Information Technology GmbH, Oberreifenberg, Germany) on a standard PC. The experimental setup consisted of a set of four square areas (each 53 cm × 53 cm) separated by wooden walls, allowing testing of four mice simultaneously. Immediately before testing, a 2000 mL glass beaker (19 cm × 13 cm, h × Ø) was placed in each area, and filled with 1500 mL fresh (25◦C) tap water. Swimming behavior was recorded for 6 min, but only the last 4 min were subjected to analysis (Taylor et al., 2009). We defined the time spent immobile as a floating behavior with a max speed of 3 cm/s, a value that became apparent of being a threshold value for active swimming behavior by detailed analysis of movement tracking. We tested a total of n = 19 WT and n = 15 LO mice. After inspection of the data, good and poor swimmers of each genotype were identified by median split in order to analyze the sub-groups separately. Thus, proficient swimmers were defined as those presenting immobility times ≤ the group median: 192 s for WT and 194 s for LO. Finally, we performed 2-sided, unpaired t-tests comparing: (1) n = 19 WT and n = 15 LO mice (whole population); (2) n = 9 WT and n = 7 LO mice (proficient swimmers); and (3) n = 10 WT and n = 8 LO mice (poor swimmers).

# RESULTS

# VMAT2 Deficient Mice Present Impaired Performance in Goal-Directed and Reward-Related Behavioral Tasks

To evaluate whether goal-directed behaviors were affected in VMAT2 mutant animals, we tested LO mice and their WT

controls in two different tasks, the burrowing test and the nest building test. First, we performed the burrowing test (**Figure 1**), by which we measured the amount of burrowed material during two different periods of time, 2 h (**Figures 1A–C**) and overnight (**Figures 1D–F**) for the whole population tested (**Figures 1A,D**), the proficient burrowers (**Figures 1B,E**) and the poor burrowers (**Figures 1C,F**). Our results showed no significant difference between LO and WT mice within the whole population in the percentage of burrowed food pellets 2 h or 24 h after the test start. In an attempt to reduce the variability and unmask possible differences in subpopulations of animals classified on the basis of their performance, we then identified and separated proficient from poor burrowers in both time periods. In the analysis of the good burrowers no significant differences were observed between genotypes at any test time point. However, in the analysis of the poor burrowers, we observed a non-significant trend towards reduced burrowing performance in LO mice at 2 h (2-sided, unpaired t-tests; 2 h WT vs. LO §p = 0.055). And we determined a significantly reduced percentage of material burrowed in LO mice, as compared to WT animals (2-sided, unpaired t tests; 24 h WT vs. LO <sup>∗</sup>p < 0.05) in the overnight measure. We then performed the nest building test (**Figure 2**), in which we assessed the nesting activity by measuring the amount of nesting material used (percentage of Nestlet torn, **Figures 2A–C**) and rated the quality of the nests (rating scores from 1 to 5 points, **Figures 2D–F**) for the whole population (**Figures 2A,D**), the active nesters (**Figures 2B,E**) and the inactive nesters (**Figures 2C,F**). Our results showed no significant differences for both the whole population and the inactive nesters groups for both tested parameters. However, we observed both significantly lower percentage of Nestlet torn and nest rating in LO mice, as compared to WT controls, in the active nesters group (2-sided, unpaired t-tests; percentage of Nestlet torn WT vs. LO <sup>∗</sup>p < 0.05, nest ratings WT vs. LO ∗∗p < 0.01).

To explore the reward-related behavioral performance in VMAT2 mutant animals, we subjected LO and WT mice to the saccharin preference test (**Figure 3**). In the SPT we measured water (**Figure 3A**), saccharin (**Figure 3B**) and total liquid intake (**Figure 3C**) in respect to body weight, as well as the saccharin preference ratio (**Figure 3D**) in both genotypes. Our results showed similar water intake of LO and WT mice. However, total liquid intake, saccharin intake and the saccharin preference ratio were significantly reduced in LO animals, as compared to their WT controls (2-sided, unpaired t-tests; water intake WT vs. LO p > 0.05, saccharin intake WT vs. LO ∗∗p < 0.01, total liquid intake WT vs. LO <sup>∗</sup>p < 0.05, saccharin preference ratio WT vs. LO <sup>∗</sup>p < 0.05).

#### VMAT2 Deficient Mice Do not Present Evidence of Depressive-Like Behavior

To determine the existence of depressive-like behavior in LO mice, we subjected them and their WT controls to the forced swimming test (**Figure 4**). Using an automated recording system, we measured the time spent immobile of each animal during a

forced swimming session of 4 min. Our results showed similar immobility times in LO and WT mice in the whole and the two sub-populations analyzed (2-sided, unpaired t-tests; time of immobility WT vs. LO p > 0.05).

# DISCUSSION

In this study we aimed at evaluating the usefulness of VMAT2 deficient mice as a tool to study apathy-like symptoms (i.e., reduction in goal-directed, reward-related behaviors) in the context of PD.

Although the pathological mechanisms of apathy are poorly understood and need further elucidation (Santangelo et al., 2013; Pagonabarraga et al., 2015), impairment in the dopaminergic pathways are assumed to underlie both PD and apathy (Pedersen et al., 2009a,b; Zahodne et al., 2012). Thus, animal models mimicking PD pathology are of special interest in the study of apathy. However, since apathy and depression are closely related but yet different constructs, depression can be a confounder in the study of apathy, especially in the context of PD, where depression and apathy are both common comorbid disorders (van Reekum et al., 2005). Thus, animal models of PD presenting apathy-like symptoms in the absence of depressive-like symptoms are desirable for the study of apathy's underlying processes and the development of specific treatments.

We evaluated the performance of 12 months old VMAT2 LO mice in four behavioral tests, namely the burrowing test, nest building test, saccharin preference test and forced swimming test, and observed a significantly lowered performance in LO mice in the first three behavioral tasks. The reduced burrowing and nest building activities suggest impairment in goal-directed, motivated behaviors in VMAT2 LO mice. Burrowing likely constitutes a motivation-dependent behavioral need (Sherwin et al., 2004) and nesting, as well as burrowing behavior, are suggested to represent a murine form of the ability to perform goal-directed ''activities of daily living'' (Deacon, 2009; Jirkof, 2014). These impairments might be akin to human apathetic patients, who show drastically reduced self-generated voluntary and purposeful behaviors (Levy and Dubois, 2006), and dysfunction in motivation (Robert et al., 2009). The saccharin preference test might also reflect apathetic symptoms in VMAT2 LO mice by reflecting the reduced preference for sweet solutions as a consequence of impaired mesolimbic dopaminergic reward pathways. Mice treated with dopamine receptor antagonists or injected with 6-hydroxydopamine (6-OHDA) into the nucleus accumbens exhibit significantly reduced preference for sweet solutions (Muscat and Willner, 1989; Malatynska et al., 2012). Vice versa, the restoration of dopamine production in the nucleus accumbens of dopamine-deficient mice could normalize the preference for sweet solutions (Szczypka et al., 2001). Lesions of the ventral tegmental area in mice also entailed diminished sucrose preference (Martínez-Hernández et al., 2006), and the consumption of sucrose in rats has been shown to induce the release of dopamine and to correlate with its concentration in the ventral striatum (Brenes and Fornaguera, 2008; de Araujo

et al., 2008). The circuit of dopaminergic projections from neurons of the ventral tegmental area to the ventral striatum, more precisely to the nucleus accumbens, is indeed, the best characterized reward circuit in the rodent brain (Russo and Nestler, 2013). However, dopamine is not the only afflicted neurotransmitter in apathy, neither in PD nor in VMAT2 LO mice. In VMAT2 LO mice norepinephrine concentrations are also drastically reduced, illustrated by progressive degeneration in the locus coeruleus, that even preceded degeneration in the substantia nigra pars compacta (Taylor et al., 2014). A recent study revealed that at the age of 4–6 months VMAT2 LO mice also exhibited dramatically reduced serotonin release capacity (Alter et al., 2015). Thus, reducing the VMAT2 LO model primary to a dopaminergic deficiency is oversimplifying, and the same criticism can be applied to the hypothesis of apathy being mainly caused by dopaminergic dysfunction. Apathy is a complex syndrome that is not limited to dopaminergic activity, as serotonergic, cholinergic and noradrenergic neurotransmitter systems are known to be involved in the same circuits that are affected in PD (Pagonabarraga et al., 2015).

On the other hand, our forced swimming test result showed that 12-months old VMAT2 LO and WT mice do not differ in the time spent immobile during the 4 min of the test, indicating a possible lack of depressive-like symptoms in the VMAT2 mutants. In addition, own results in the tail suspension test indicate a similar finding, with unchanged time spent immobile in the mutants as compared to controls (data not shown). These findings are conflicting with previously reported data from VMAT2 LO and HET mice, showing increased immobility times in the forced swimming test (Fukui et al., 2007; Taylor et al., 2009). Specifically, VMAT2 HET mice showed increased immobility times at the age of 3–5 months (Fukui et al., 2007), whereas the VMAT2 LO mice did not show increased immobility times at 4–6 months, but only later at the age of 12–15 months (Taylor et al., 2009). Despite these conflicting data, a series of aspects confirm the validity of our findings. First, the previously characterized VMAT2 HET mice differ in age and genotype from the LO mice tested here and, therefore, our results cannot be directly contrasted to those of Fukui et al. (2007). Second and in regard to the observations of Taylor et al. (2009), it can be speculated that depressive-like symptoms could start evidencing only after 12 months of age and therefore manifest later on in increased immobility times. Finally, our sample sizes were significantly larger than in the previous VMAT2 LO characterization, providing increased validity to our results.

Dysfunctions of the nigrostriatal, mesolimbic and mesocortical dopaminergic pathways are suggested to play an important role in the manifestation of apathy (Thobois et al., 2010; Santangelo et al., 2013). Indeed, several findings substantiate the importance of dopaminergic activity in the pathology of apathy. Dopaminergic treatment seems to decrease apathy in early stages of PD (Santangelo et al., 2014) and the severity of apathy changes in regard to the dopaminergic ''on'' and ''off'' treatment states (Lhommée et al., 2012; Castrioto et al., 2014). Since VMAT2 deficient mice are vastly impaired in their dopaminergic neurotransmission, it is conceivable that they would manifest symptoms of apathy.

The main limitation of our study is the missing longitudinal dimension in our experiments. Evaluating the progression of apathetic symptoms throughout the life-span of the mice, as well as its temporal relation with depressive-like symptoms would provide a more profound insight regarding the temporal window of apathy-based studies and the development of specific

#### REFERENCES


treatment options for each condition. An additional drawback of our design is that the applied behavioral tests may not be entirely specific for one symptom or another. In this line, the independence of apathy from depression has been matter of intense debate, as apathy has classically been considered as a core feature of depression (Kirsch-Darrow et al., 2006; Santangelo et al., 2013) and many definitions of depression comprise features of apathy (Cummings et al., 2015). Moreover, we explore apathy in an animal model of dopaminergic deficiency rather than implementing a specific test for apathy, which remains largely missing in the field. Thus, relying on behavioral tests that accurately measure the effect of dopaminergic dysfunction over goal-directed and reward-related behaviors in the context of PD, appears as a risky but promising strategy in the study of pure apathy.

In summary, 12 months old VMAT2 deficient mice likely do not express a depressive-like phenotype but rather show behavioral signs of apathy that might be related to dopaminergic dysfunction. Future research should validate our preliminary results and address potential modulatory effects of dopaminergic treatment on the performance in the burrowing, nest building and saccharin preference tests, and closely relate the development of the apathetic phenotype to the underlying neurochemical changes in VMAT2 deficient mice's brains.

#### AUTHOR CONTRIBUTIONS

DN and CRB designed research. AB, CGM, MMM and SM executed research. AB and DN analyzed data, prepared the figures and wrote the manuscript. All authors edited the manuscript.

#### FUNDING

The study was funded by the Clinical Research Priority Program ''Sleep and Health'' of the University of Zurich (UZH) and by the HSM-2 Initiative ''Stereotactic Neurosurgery'' of the Canton of Zurich.

#### ACKNOWLEDGMENTS

The authors would like to thank Dr. Paulin Jirkof for helpful consultation regarding behavioral tests.

disease: the ANIMO study. Mov. Disord. 27, 211–218. doi: 10.1002/mds. 23872


in two mouse models of temporal lobe epilepsy. Exp. Neurol. 263, 263–271. doi: 10.1016/j.expneurol.2014.09.004


Malatynska, E., Steinbusch, H. W. M., Redkozubova, O., Bolkunov, A., Kubatiev, A., Yeritsyan, N. B., et al. (2012). Anhedonic- like traits and lack of affective deficits in 18-month-old C57BL/6 mice: implications for modeling elderly depression. Exp. Gerontol. 47, 552–564. doi: 10.1016/j.exger.2012.04.010


untreated Parkinson's disease: a prospective longitudinal study. Eur. J. Neurol. 22, 253–260. doi: 10.1111/ene.12467


the locus ceruleus. Neuropharmacology 76, 97–105. doi: 10.1016/j.neuropharm. 2013.08.033


**Conflict of Interest Statement**: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2016 Baumann, Moreira, Morawska, Masneuf, Baumann and Noain. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution and reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Action Experience and Action Discovery in Medicated Individuals with Parkinson's Disease

Jeffery G. Bednark <sup>1</sup> \*, John N. J. Reynolds <sup>2</sup> , Tom Stafford<sup>3</sup> , Peter Redgrave<sup>3</sup> and Elizabeth A. Franz <sup>4</sup> \*

<sup>1</sup> Queensland Brain Institute, The University of Queensland, Brisbane St Lucia, QLD, Australia, <sup>2</sup> Department of Anatomy, Otago School of Medical Sciences, and The Brain Health Research Centre, University of Otago, Dunedin, New Zealand, <sup>3</sup> Department of Psychology, University of Sheffield, Sheffield, UK, <sup>4</sup> Department of Psychology and fMRIotago, University of Otago, Dunedin, New Zealand

Parkinson's disease (PD) is a neurodegenerative disorder that markedly affects voluntary action. While regular dopamine treatment can help restore motor function, dopamine also influences cognitive portions of the action system. Previous studies have demonstrated that dopamine medication boosts action-effect associations, which are crucial for the discovery of new voluntary actions. In the present study, we investigated whether neural processes involved in the discovery of new actions are altered in PD participants on regular dopamine treatment, compared to healthy age-matched controls. We recorded brain electroencephalography (EEG) activity while PD patients and agematched controls performed action discovery (AD) and action control tasks. We found that the novelty P3, a component normally present when there is uncertainty about the occurrence of the sensory effect, was enhanced in PD patients. However, AD was maintained in PD patients, and the novelty P3 demonstrated normal learning-related reductions. Crucially, we found that in PD patients the causal association between an action and its resulting sensory outcome did not modulate the amplitude of the feedback correct-related positivity (fCRP), an EEG component sensitive to the association between an action and its resulting effect. Collectively, these preliminary results suggest that the formation of long-term action-outcome representations may be maintained in PD patients on regular dopamine treatment, but the initial experience of action-effect association may be affected.

#### Keywords: Parkinson's disease, action discovery, ERPs, agency

# INTRODUCTION

Parkinson's disease (PD) is a neurodegenerative disorder caused by the progressive loss of dopamine neurons (Agid and Blin, 1987), which affects both motor and non-motor function. While dopamine medication (usually in the form of levodopa: L-dopa, a precursor of brain dopamine) is commonly used to restore motor function, exogenous elevation of dopamine has been linked to changes in dopamine-related signaling in the basal ganglia (e.g., Gotham et al., 1988; Cools et al., 2001, 2010; Frank et al., 2004; O'Reilly and Frank, 2006; Moustafa et al., 2008). Individuals with PD who are ON dopamine medication are shown to have increased sensitivity to reward feedback (Frank et al., 2004), exaggerated working memory updating

#### Edited by:

Daniela S. Andres, ETH Zurich, Switzerland

#### Reviewed by:

Francesca Garbarini, University of Turin, Italy Kristina Aurousseau, Centre de Recherche de l'Institut Universitaire de Montréal (CRIUGM), Canada

#### \*Correspondence:

Jeffery G. Bednark j.g.bednark@gmail.com Elizabeth A. Franz Lfranz@psy.otago.ac.nz

Received: 06 June 2016 Accepted: 09 August 2016 Published: 25 August 2016

#### Citation:

Bednark JG, Reynolds JNJ, Stafford T, Redgrave P and Franz EA (2016) Action Experience and Action Discovery in Medicated Individuals with Parkinson's Disease. Front. Hum. Neurosci. 10:427. doi: 10.3389/fnhum.2016.00427 (Moustafa et al., 2008), and enhanced action-outcome binding (Moore et al., 2010).

Crucially, the dopaminergic-system has a significant modulatory role in learning (Reynolds et al., 2001), and likely shapes the discovery of novel action-sensory outcome associations that make up adaptive behavior (Redgrave and Gurney, 2006; Redgrave et al., 2008, 2011; Franz, 2012). Based on the temporal dynamics of dopamine activity, it has been recently proposed that the dopamine-system provides both fast responses to salient events and slow responses that are only executed after the information has been highly processed (Joshua et al., 2009). This allows the dopamine-system to participate in both the monitoring of salient stimuli and the evaluation of information for modifying future behavior. These dynamic signals, however, may be affected in dopamine-medicated individuals with PD leading to changes in the monitoring and evaluation of behaviorally relevant stimuli.

Detecting the occurrence of salient external events and evaluating the cause of salient events is essential for learning and maintaining adaptive behavior (Redgrave and Gurney, 2006; Redgrave et al., 2008). Using event-related potentials (ERPs), derived from electroencephalography (EEG), we have previously shown that the discovery of novel action-outcome associations is dependent on an interaction between salience monitoring and evaluation of novel sensory information with regard to the preceding movement (Bednark et al., 2013). Two key ERPs were demonstrated to play a role: the feedback correct-related positivity (fCRP), and the novelty P3.

The fCRP, along with other reward-related potentials, is commonly associated with the monitoring of salient, positive and reward-related outcomes (Holroyd et al., 2008, 2011). Related to action, previous work has shown that the fCRP response to salient outcomes is enhanced when participants have a sense of control over the salient outcome (Li et al., 2011; Bednark et al., 2013; Bednark and Franz, 2014). Conversely, the novelty P3 is thought to reflect the engagement of evaluative processes necessary for learning when a novel stimulus is encountered (Friedman et al., 2001; Jongsma et al., 2006; Sailer et al., 2010). We have shown that the learning of novel actions directly modulates the novelty P3, with the amplitude of the novelty P3 in response to the behaviorally relevant outcome decreasing as task performance improves and participants acquired a novel action (Bednark et al., 2013). Importantly, both of these brain potentials have been associated with dopamine activity (de Bruijn et al., 2005; Poceta et al., 2006; Polich and Criado, 2006; Potts et al., 2006; Holroyd et al., 2008) and may separately reflect dopaminerelated salience monitoring and evaluative processes.

In the present study, we use these brain potentials to investigate whether the dynamics of the dopamine-system, associated with the discovery of new actions, are altered in individuals with PD while ON regular dopamine treatment (forms of L-dopa), compared to healthy age-matched controls. To test this, medicated individuals with PD and age-matched controls performed action discovery (AD) and action control tasks similar to our previous study (Bednark et al., 2013). Given that dopamine is implicated in mediating both the fCRP and the novelty P3, we expect that both potentials will be enhanced in PD patients compared to controls. However, dopamine-medication boosting of action-effect associations is hypothesized to alter AD.

# MATERIALS AND METHODS

#### Participants

Eight patients with mild to moderate PD were recruited from the participant pool of the Action, Brain, and Cognition Laboratory at Otago, a program that works in close association with the Otago Parkinson's Society, Dunedin, New Zealand (see **Table 1** for patient details). The inclusion criteria were: no known dementia (Mini-Mental State Exam score ≥27), no current depressive symptoms (Geriatric

TABLE 1 | Demographic, pathology, and drug details in Parkinson's disease (PD) patients and controls.


Depression Scale score ≤5), and currently on dopamine medication.

Eight age- and sex-matched healthy controls were recruited from a University of Otago database of older adults participants (**Table 1**). Inclusion criteria were: no known neurological or psychiatric illnesses, no known dementia (MMSE score ≥27), and no current depressive symptoms (GDS score ≤5). The Lower South Otago Regional Ethics Committee approved all procedures. All participants volunteered and gave their fully informed written consent.

#### Experimental Procedure

Prior to the experimental session, patients and controls were interviewed. During the interview, current medical history was obtained, including current PD treatment medication for patients. Neuropsychological tests were also administered to screen for dementia and depression. The motor portion of the UPDRS was administered to patients.

#### Tasks

For the AD and two action control tasks, participants were informed that their goal was to cause a green circle to appear in the center of the screen by moving the cursor on the screen using a tracking-ball mouse. They were further instructed to look at the fixation cross throughout the trials.

For the AD task, participants were not given any specific instructions about how to elicit the green circle, but they were told that they would learn how to do so over the course of the task. A diagram of the AD task can be seen in **Figure 1A**. The green circle was elicited by the participants' movements when the cursor was moved into a specific location or ''hot spot''. The location of the ''hot spot'' remained the same within a block of 30 trials or 30 presentations of the green circle, but each block had a different location for the ''hot spot''. The cursor was re-located to a new random starting point after each presentation of the green circle. This was done so that the final position, and not the initial position of the movement, was the critical determinant of eliciting the presentation of the green circle. Participants had to then re-locate this location for each trial.

During the AD task, the monitoring and evaluation of the sensory effect was required to identify the movement(s) eliciting the sensory outcome. Over the course of the AD task, participants learned a set of efficient rules that guided action selection or a movement heuristic. This AD task was compared to two non-learning goal-directed motor tasks to investigate how the monitoring and evaluation of sensory outcomes may differ during AD. As a final control task, we used a stimulus-response (SR) task to investigate the monitoring of behaviorally relevant sensory effects that were unrelated to the preceding voluntary movement.

In the initial cue (IC) action-control task, location of the ''hot spot'' for eliciting the green circle was cued at the start of the block with a gray outline of a circle (**Figure 1B**). Participants were instructed that moving the cursor into this screen location would elicit the green circle, but that the gray outline circle would not be present when they performed the task. The cue was then removed and participants began this IC task. This IC provided participants with sensory information for performing the necessary actions for achieving the task goal prior to the start of the task, thereby serving as a control for learning novel actions during the AD task.

The continuous cue (CC) action-control task served as a control for any memory processes or uncertainty (particularly with regards to the novelty P3) associated with remembering the cued location in the IC task or the learnt location in the AD task. At the start of the CC task, participants were instructed that moving the cursor into the location defined by a gray outline of a circle on the screen would elicit the green circle (**Figure 1C**). For action-outcome task (**Figures 1A–C**) the cursor was re-positioned to a different location on the screen following each presentation of the green circle.

The final control task was a SR task (**Figure 1D**). For the SR task, participants were informed that their movements did not elicit the sensory effect. Rather, they were instructed to move the cursor around the screen (so participants were still making voluntary movements) and respond to the presentation of the green circle by moving the cursor to the fixation cross (so the green circle was still salient) and then resume moving the cursor around the screen. To control for the reduction in the inter-stimulus interval, which is known to affect P3 amplitude (e.g., Gonsalvez et al., 2007) that is observed with learning in the AD task, we controlled the timing of green circle presentation. The timing of the green circle in each block of the SR task was computer-controlled to have the same interstimulus interval as the corresponding block of the movementlearning task performed by the same participant (thereby using yoked timing conditions). As a result, any variation or decrease in the timing interval of the sensory effects in the movementlearning task would be directly controlled for by the SR task.

The behavioral tasks that were administered during the experimental session were previously conducted in normal undergraduate participants (Bednark et al., 2013). The theoretical importance of our AD task has also been highlighted elsewhere (Stafford et al., 2012). There are, however, a few key differences between the tasks used in the present study and those used in our previous ERP study (Bednark et al., 2013). To reduce the difficulty of the tasks, we increased the size of the ''hot spot'' (4.04◦ visual angles) and other visual stimuli (i.e., fixation cross: 0.8◦ ; cursor: 0.8◦ ; green sensory effect: 4.04◦ ). Additionally, while the three different control tasks were previously conducted in two separate experiments, the same group of participants performed all control tasks in the present study. The order of the tasks was as follows: ABC-ABC-ABC-DDD or ACB-ACB-ACB-DDD. A total of 12 blocks was conducted, with a block defined by 30 presentations of the green circle. The tasks were performed on a 54 cm display with a black background. MatLab software (MathWorks, Inc.) was used for all stimulus presentation and collection of behavioral responses.

# Behavioral Analysis

For the AD task and the two action control tasks, the time it took the participants to elicit each green circle was recorded. These times were used to determine hit rate, or the number of green circles presented per 2 s interval. The hit rate was used as a behavioral measure of goaldirected motor performance and was calculated for the first half of green circle occurrences (1–15; F15) and second half of green circle occurrences (16–30; L15) of each block in order to investigate changes in performance within a block.

For the SR task, the time it took the participants to respond to each green circle by moving to the fixation cross, and the number of responses made were recorded. The mean response time and the mean number of responses were computed for the first half of green circle occurrences (1–15; F15) and second half of green circle occurrences (16–30; L15) of each block.

# EEG Data Acquisition and Analysis

EEG and electrooculography (EOG) data were collected continuously using a 32-channel Ag-Ag/Cl sintered Quickcap and a Neuroscan Synamps amplifier, interfaced with a Dell Intel computer running Scan 4.3 software. Data were sampled at 1000 Hz with a band pass of 0.5–200 Hz, and the gain was ×500. The 28 scalp electrode sites were referenced to linked mastoid electrodes, with AFz as the ground. Horizontal EOG data were recorded from two electrodes placed on the outer canthi of the two eyes. Vertical EOG data were recorded from linked electrodes on the infraorbital and supraorbital ridges of the left eye. Impedances were maintained below 5 k.

EEG data analysis was conducted offline using purposewritten MatLab scripts. Continuous EEG data were epoched with respect sensory event onset (200 ms prior and 1000 ms after) in the behavioral task, and baseline corrected relative to the 200 ms period prior to sensory event onset. Prior to averaging, epochs containing ocular artifacts were corrected (Gratton et al., 1983). To remove movement artifact associated with PD, EEG data were then wavelet decomposed to level 9 using a ''Daubechies 6'' discrete wavelet transformation, and reconstructed with 1–25 Hz-frequency range. Based on visual inspection of the averaged waveforms, the mean amplitude of the fCRP for both PD and AMC groups was measured at the Cz electrode site and averaged across the time window 210–290 ms. For the novelty P3 (measured at CZ) we created a mean amplitude measure for the peak amplitude (averaged 25 ms before and after the peak) found within the 300–450 ms time window. This was done to reduce variation in the latency of the novelty P3 component observed in the patient and control group when the waveforms were visually Bednark et al. ERPs of Movement-Outcome Learning in PD

inspected. For plotting purposes, EEG data were smoothed using a one-dimensional digital filter with a 25 ms time window.

#### Statistical Analysis

The hit rate behavioral measure was entered into a mixed analysis of variance (ANOVA) with the within-subjects factors of Task (AD, CC, IC), Block-Half (F15, L15) and Block (Block 1, Block 2, Block 3), and between-subject factor of Group (patients, controls). Similar mixed ANOVAs were conducted for fCRP and novelty P3 amplitude, but the SR task added to the Task factor. To tease apart specific effects of interactions, additional repeated-measures ANOVAs and planned comparisons were used where appropriate to test our hypotheses. Effects were considered significant if p < 0.05. Greenhouse–Geisser corrections were also applied to p-values where appropriate. Effect sizes are shown using partial eta squared (η 2 ). For the assessment of behavioral performance during the SR task, the mean time to response to the stimulus and the mean number of responses to the stimulus were entered into paired-sample t-tests that compared the PD and a control group. All statistical tests were conducted using SPSS (version 18.0) software.

#### RESULTS

#### Behavioral Results

For both PD and controls, the hit rate in each task increased across blocks (Block, F(2,28) = 18.78, p < 0.001, partial η <sup>2</sup> = 0.57) and within blocks (Block-Half, F(1,14) = 37.44, p < 0.001, partial η <sup>2</sup> = 0.73). There was a significant main effect of Task (F(2,28) = 72.48, p < 0.001, partial η <sup>2</sup> = 0.84) demonstrating that the hit rate during the AD task (M = 0.85, SE = 0.08) was significantly smaller than in the CC task (M = 1.64, SE = 0.09, p < 0.001) and the IC task (M = 1.68, SE = 0.09, p < 0.001). A significant Task × Block-Half interaction (F(2,28) = 33.59, p < 0.001, partial η <sup>2</sup> = 0.71) suggested that performance was improving at a greater rate during the AD task. Verifying learning-related improvement, post hoc paired-samples t-tests showed that the change in hit rate within a block was significantly larger for the AD task (M = 0.40, SE = 0.05) compared to the CC task (M = 0.10, SE = 0.04, t(47) = 4.69, p < 0.001) and the IC task (M = −0.06, SE = 0.05, t(47) = 8.54, p < 0.001).

Regarding the difference in behavioral performance between groups, we did not find any significant difference between groups in hit rate (**Figure 2A**). Though the hit rate of the PD group was generally less than that of the control group. There were also no significant differences between groups in mean response time (F(1,14) = 1.69, p = 0.21, partial η <sup>2</sup> = 0.11) and number of responses (F(1,14) = 0.19, p = 0.67, partial η <sup>2</sup> = 0.01) during the SR task (**Figures 2B,C**).

#### fCRP Results

The average ERP waveforms for each task and group are presented in **Figure 3**. We found a significant Task × Group

interaction in the amplitude of the fCRP (F(3,42) = 3.73, p = 0.03, partial η <sup>2</sup> = 0.21). As shown in **Figure 4A**, subsequent post hoc analysis revealed that the magnitude of the fCRP during the SR task was significantly larger in amplitude in the PD group (M = 6.53 µV, SE = 0.80 µV) compared to the control group (M = 3.28 µV, SE = 0.80 µV; Group, F(1,14) = 8.40, p = 0.012, partial η <sup>2</sup> = 0.375; Bonferroni corrected α-level = 0.0125). Initial analysis also suggests that the magnitude of the fCRP response for both groups demonstrated a reduction across Block-Half (F(1,14) = 4.84, p = 0.045, partial η <sup>2</sup> = 0.26) and Block (F(2,28) = 5.15, p = 0.013, partial η <sup>2</sup> = 0.27). However, when each group was assessed individually, only the PD group demonstrated significant reductions in fCRP amplitude across Block-Half (F(1,14) = 4.84, p = 0.045, partial η <sup>2</sup> = 0.26) and Block (F(2,28) = 5.15, p = 0.013, partial η <sup>2</sup> = 0.27). In the control group, there was a significant main-effect of Task (F(3,21) = 4.48, p = 0.033, partial η <sup>2</sup> = 0.39) with the difference observed between the AD task (M = 5.14 µV, SE = 0.60 µV) and the SR task (M = 3.28 µV, SE = 0.60 µV, p = 0.022; Bonferroni corrected α-level = 0.016), as shown in **Figure 4A**.

across the two groups was similar. Error bars indicate standard error.

#### Novelty P3 Results

As shown in **Figure 4B**, the magnitude of the novelty P3 response in the PD group was significantly larger than the novelty P3 response in the control group (Group, F(2,14) = 4.48, p = 0.034, partial η <sup>2</sup> = 0.43). However, while the novelty P3 amplitude for the PD group was larger than in the control group, the general trend of novelty P3 response is similar for the two

CC task, (C) the IC task, and (D) the SR task.

groups (**Figure 4C**). There was also a significant main-effect of Task overall, (F(3,42) = 4.48, p = 0.034, partial η <sup>2</sup> = 0.43). Bonferroni pair-wise comparisons revealed that the novelty P3 response during the AD task (M = 11.77 µV, SE = 0.89 µV) was significantly larger than in the IC task (M = 7.03 µV, SE = 0.49 µV, p < 0.001) and CC task (M = 7.53 µV, SE = 0.48 µV, p < 0.001). The SR task (M = 10.65 µV, SE = 0.96 µV) was also significantly different from the IC task (p = 0.003) and the CC task (p = 0.017). However, no difference between the AD task and the SR task was observed. The novelty P3 response also demonstrated a significant reduction across Block-Half (F(1,14) = 4.48, p = 0.034, partial η <sup>2</sup> = 0.43), and Block (F(2,28) = 4.48, p = 0.034, partial η <sup>2</sup> = 0.43).

This pattern of significant main effects was observed in both PD and control groups, as confirmed by separate analyses on each group. Accordingly, in the PD group, there were significant main effects of Task (F(3,21) = 14.13, p = 0.001, partial η <sup>2</sup> = 0.67), Block-Half (F(1,7) = 27.27, p = 0.001, partial η <sup>2</sup> = 0.80), and Block (F(2,12) = 6.04, p = 0.021, partial η <sup>2</sup> = 0.46). The pattern was similar for the control group: Task (F(3,21) = 7.36, p = 0.006, partial η <sup>2</sup> = 0.51), Block-Half (F(1,7) = 13.59, p = 0.008, partial η <sup>2</sup> = 0.66), and Block (F(2,14) = 6.30, p = 0.027, partial η <sup>2</sup> = 0.47).

#### DISCUSSION

The present study investigated the neural processes mediating the discovery of novel actions in PD patients currently receiving dopamine treatment. We found no significant behavioral differences in AD between PD patients and age-matched controls. However, ERP analysis suggests that there may be significant alterations to the neural processes involved in the formation of action-outcome associations in PD patients compared to controls.

#### Novelty P3

As hypothesized, the amplitude of the novelty P3 was larger in people with PD compared to age-matched controls. This effect is complementary to previous work that has demonstrated a reduction in novelty P3 amplitude in PD patients OFF dopamine medication (Poceta et al., 2006). Indeed, evidence from multiple studies suggests that dopamine is a crucial neuromodulator of the novelty P3 response (Polich and Criado, 2006; Polich, 2007). This would suggest that dopamine levels modulate the response to salient unexpected sensory events, which is in keeping with the dopamine errorsignaling hypothesis (Schultz, 1997, 1998). Additionally, this

enhancement of the novelty P3 provides further evidence for the proposed over-abundance of dopamine in relatively preserved portions of the basal ganglia (Gotham et al., 1988; Swainson et al., 2000; Cools et al., 2001; Shohamy et al., 2006). In contrast to previous EEG studies, our results suggest that the mechanisms governing the reduction of the novelty P3 are unaffected or restored in medicated individuals with PD.

In both PD patients and age-matched controls, the novelty P3 demonstrated learning-related reductions within and across blocks of the AD task. Previously, we hypothesized that the novelty P3 reflects the engagement of attentional mechanisms necessary for identifying the movements responsible for the unexpected sensory event (Franz, 2012; Bednark et al., 2013). As the responsible movements are identified and a heuristic is formed to guide future action selection, the need to engage attentional mechanisms is reduced. In the present study, we found that the novelty P3 response in PD patients still demonstrated learning-related reductions. This would suggest that neural processes mediating action-outcome anticipation might be unaffected by changes to the magnitude of the dopamine response or that dopamine medication restores function to this system.

According to previous AD proposals of the dopamine error signal, the formation of long-term associations between actions and outcomes are likely established in structures outside the basal ganglia (Redgrave and Gurney, 2006; Redgrave et al., 2008). One such brain structure is anterior cingulate cortex (ACC). Recent evidence indicates that the ACC is responsible for governing the value of actions based on their previous reinforcement history (Walton et al., 2004; Rushworth et al., 2007; Holroyd and Coles, 2008). The ACC is the proposed source generator of the novelty P3 (Polich, 2007), and is thought to receive the dopamine error signal from the basal ganglia (Holroyd and Coles, 2002, 2008). While original proposals of this ACC-dopamine error signaling (Holroyd and Coles, 2002) have focused on prediction errors that are derived from subcortical indications of stimulus novelty (for a review see Redgrave et al., 2008, 2011), recent evidence from animal studies suggests that the dopamine error signal (with a longer latency) can be elicited from cortical indications of stimulus novelty (Bromberg-Martin et al., 2010; Nomoto et al., 2010). Along these lines, there is evidence that the ACC can directly modulate dopamine release (Gariano and Groves, 1988). Indeed, it has been proposed that different time scales of the dopamine response may control different aspects of motor behavior (Joshua et al., 2009). Thus, the ACC may govern the learningrelated reductions in the novelty P3 response through its modulation of dopamine signaling. This possible top-down mechanism appears to remain moderately intact or is restored to normal function in PD patients on regular dopamine treatment.

# fCRP

In the motor tasks in which participants' actions were responsible for the occurrence of the sensory event, there was no significant difference between PD patients and agematched controls. Crucially, when the participants' actions were unrelated to the occurrence of the sensory event, we observed an enhancement in the fCRP response in PD patients compared to age-matched controls. As highlighted in the Introduction, previous studies have demonstrated that the fCRP response is enhanced by the perceived responsibility over Li et al. (2011) and coupling of an intentional action with a sensory event (Bednark et al., 2013; Bednark and Franz, 2014). Additionally, it has been previously demonstrated that PD patients on regular dopamine treatment perceive actions and sensory events as occurring closer in time than when OFF dopamine treatment (Moore et al., 2010). These converging lines of evidence would suggest that PD patients had an exaggerated experience of association between their actions and the sensory effect during the SR task. This is despite explicit knowledge that their actions were not responsible for the sensory effect. Presumably this could be because dopamine treatment used to restore dopamine levels in PD patients may actually reduce the dynamic range of the dopamine response (Frank, 2005). This, in turn, may reduce the fCRP differentiation between action-related and action-unrelated sensory effects.

In contrast to the novelty P3, the pattern of fCRP response across and within tasks in PD patients varied significantly from the fCRP pattern observed in age-matched controls. This would suggest that the processes involved in the fCRP response might be altered in PD patients. A recent source localization study that used principal component analysis to remove the influence of other ERPs demonstrated that the basal ganglia is the likely source generator of the reward-related fCRP response (Foti et al., 2011). Applied to our findings, this would suggest that alterations to the fCRP response in PD patients may occur at the level of the basal ganglia. Thus, in PD patients ON medication, it appears that over-abundant dopamine levels in relatively preserved portions of the basal ganglia causes the behavioral relevance of the sensory effect to be maintained across all tasks.

An alternative possibility is that the enhanced fCRP may be related to increased task motivation. Others have suggested that task motivation can modulate brain activity in the time-range of the fCRP (Hajcak et al., 2005; Boksem et al., 2006; Sailer et al., 2010). However, it is unlikely that task motivation in the PD patients was only enhanced during the SR task. Nevertheless, reduced task motivation may explain the observed reduction in fCRP amplitude within and across blocks in the PD patients.

# Limitations and Future Directions

The low number of individuals with PD available for this study and the lack of an OFF-medication state in the PD group limits the extent to which these results can be generalized; the present findings must be viewed as preliminary. However, it is important to note that despite the small sample size and heterogeneity of the PD observed in these patients, we were still able to find significant differences in our ERP measures. Thus, this study provides initial evidence for the use of ERPs in exploring PD. Future studies with access to larger patient populations and ON/OFF design should be conducted to determine the extent to which these ERPs are influenced by dopamine medication during agent-based action learning. However, performing a study with ON/OFF design may prove to be difficult given the learning nature of this task. Differences between the ON and OFF state may not be detected because carry-over effects (e.g., learning the task) affect performance in the task from one state to the next. Conducting this experiment with de novo PD patients who still have a normal level of motor control could potentially reveal if action learning mechanisms are affected before gross motor control effects are visible.

# Conclusion

This is an initial study investigating whether the ability to identify new actions might be affected in dopamine medicated PD participants. AD is maintained in PD patients on regular dopamine treatment despite a potential over-abundance of dopamine in relatively preserved portions of the basal ganglia. This highlights the importance of structures outside the basal ganglia for the formation of long-term action-outcome representations. However, the initial experience of actionoutcome association appears to be affected by increased dopamine levels in the basal ganglia.

# AUTHOR CONTRIBUTIONS

JGB: designed and conducted study, analyzed data, and wrote the manuscript. JNJR: contributed to study design and writing of the manuscript. TS: contributed to study design, the development of the experimental task, and editing the manuscript. PR: contributed to task design and experimental concept; contributed to the editing of the manuscript. EAF: provided the facilities for conducting the study; contributed to study design, data analysis, and writing of the manuscript.

#### ACKNOWLEDGMENTS

This work was supported in part by a grant from the Parkinson's Society of New Zealand (to EAF) and in part by a Marsden grant

#### REFERENCES


from the Royal Society of New Zealand (to JNJR), a European Community 7th Framework Programme (FP7/2007–2013, grant agreement No. ICT-IP-231722, to TS and PR) and a Wellcome Trust grant 091409 (to PR). During the preparation of this article, JNJR received support from a Rutherford Discovery Fellowship from the Royal Society of New Zealand, and EAF also received funding from her Marsden grant from the Royal Society of New Zealand.


**Conflict of Interest Statement**: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2016 Bednark, Reynolds, Stafford, Redgrave and Franz. This is an openaccess article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution and reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Structure Function Revisited: A Simple Tool for Complex Analysis of Neuronal Activity

#### Federico Nanni and Daniela S. Andres \*

Science and Technology School, National University of San Martin (UNSAM), San Martin, Argentina

Neural systems are characterized by their complex dynamics, reflected on signals produced by neurons and neuronal ensembles. This complexity exhibits specific features in health, disease and in different states of consciousness, and can be considered a hallmark of certain neurologic and neuropsychiatric conditions. To measure complexity from neurophysiologic signals, a number of different nonlinear tools of analysis are available. However, not all of these tools are easy to implement, or able to handle clinical data, often obtained in less than ideal conditions in comparison to laboratory or simulated data. Recently, the temporal structure function emerged as a powerful tool for the analysis of complex properties of neuronal activity. The temporal structure function is efficient computationally and it can be robustly estimated from short signals. However, the application of this tool to neuronal data is relatively new, making the interpretation of results difficult. In this methods paper we describe a step by step algorithm for the calculation and characterization of the structure function. We apply this algorithm to oscillatory, random and complex toy signals, and test the effect of added noise. We show that: (1) the mean slope of the structure function is zero in the case of random signals; (2) oscillations are reflected on the shape of the structure function, but they don't modify the mean slope if complex correlations are absent; (3) nonlinear systems produce structure functions with nonzero slope up to a critical point, where the function turns into a plateau. Two characteristic numbers can be extracted to quantify the behavior of the structure function in the case of nonlinear systems: (1). the point where the plateau starts (the inflection point, where the slope change occurs), and (2). the height of the plateau. While the inflection point is related to the scale where correlations weaken, the height of the plateau is related to the noise present in the signal. To exemplify our method we calculate structure functions of neuronal recordings from the basal ganglia of parkinsonian and healthy rats, and draw guidelines for their interpretation in light of the results obtained from our toy signals.

Keywords: Parkinson's disease, neuronal activity, interspike intervals, 6-hydroxydopamine, alertness, basal ganglia, complexity, temporal structure

# INTRODUCTION

The nervous system is complex at many levels. It is built as a network of nonlinear elements with complex dynamics themselves: the neurons (Rulkov, 2002; Korn and Faure, 2003). As a result, the output of the nervous system exhibits complex dynamics at multiple scales, which is reflected in neural signals from the level of single neurons, to microcircuits and larger neuronal networks.

#### Edited by:

Felix Scholkmann, University Hospital Zurich, University of Zurich, Switzerland

#### Reviewed by:

Luciano Zunino, Consejo Nacional de Investigaciones Científicas y Técnicas (CONICET), Argentina Axel Hutt, German Weather Service, Germany

> \*Correspondence: Daniela S. Andres dandres@unsam.edu.ar

Received: 18 May 2017 Accepted: 25 July 2017 Published: 14 August 2017

#### Citation:

Nanni F and Andres DS (2017) Structure Function Revisited: A Simple Tool for Complex Analysis of Neuronal Activity. Front. Hum. Neurosci. 11:409. doi: 10.3389/fnhum.2017.00409 This complexity can be measured from different kinds of data, for instance microelectrode recordings (MER), electroencephalograms (EEG) and functional magnetic resonance imaging (fMRI) (Mpitsos et al., 1988; Elger et al., 2000; Yang et al., 2013a,b). Although the theoretical concepts used to characterize these different signals are not scale specific (chaoticity, entropy, nonlinearity, fractality), the tools of analysis required to calculate complexity measures from different signals do differ, and need to be tuned for each particular case.

Our motivation comes from the observation of complex properties in the neuronal activity of the basal ganglia (Darbin et al., 2006; Lim et al., 2010; Andres et al., 2011). Different complexity measures of basal ganglia activity show a correlation to different physiologic/pathologic conditions, like arousal level or the presence of specific pathologies (dystonia, parkinsonism) (Sanghera et al., 2012; Andres et al., 2014a; Alam et al., 2015). Even more, some of these measures can be modified with therapeutic interventions, suggesting that a correct characterization of basal ganglia complexity has potentially high clinical impact (Dorval et al., 2008; Lafreniere-Roula et al., 2010). However, complexity measures are not always easily transferred to the clinic. Nonlinear tools are sensitive not only to parameters' settings, but also to the number of data analyzed (length of the recordings), noise and linear correlations present in the signals. Because of that, the correct implementation of nonlinear tools depends critically on the behavior of the tool at hand for the particular system under study (Schreiber, 1999).

In this context, the temporal structure function emerged as a powerful tool for the analysis of basal ganglia activity. In previous work we tested this tool on human, animal and simulated neuronal data, and observed a correlation of abnormalities of the temporal structure of basal ganglia spike trains with parkinsonism (Andres et al., 2014b, 2015, 2016). Importantly, the calculation of temporal structure functions of spike trains is robust and efficient computationally. However, the interpretation of results remains difficult, partly because the use of temporal structure functions for the characterization of spike trains is relatively new. Here, we perform an analysis of the temporal structure of toy signals, i.e., signals with known properties, as a means to characterize the behavior of the tool. This task has been partly attempted before, but the analysis did not include a study of any complex system or a comparison with neuronal data, and is therefore incomplete to our purpose (Yu et al., 2003). Our goal is to draw general guidelines for the interpretation of structure functions of neuronal data. To illustrate the method, we calculate the structure function of spike trains obtained from the basal ganglia of healthy and parkinsonian rats during the transition from deep anesthesia to alertness.

# THE TEMPORAL STRUCTURE FUNCTION

#### Definition

Consider the following time series:

$$I(t) = \{I\_{t1}, I\_{t2}, \dots, I\_{tn}\} \tag{1}$$

where Itn are successive recordings of the variable I at the times t1, t2 to tn. The length of the time series is n. For this time series, the temporal structure function is a function of the scale τ and of order q, defined as the average of the absolute value of the differences between elements of the time series separated by time lags corresponding to the scale τ , elevated to the power of q (Lin and Hughson, 2001):

$$S\_q\left(\mathbf{r}\right) = \left\langle \left| \Delta I(\mathbf{r}) \right|^q \right\rangle \tag{2}$$

Here, 1 denotes the difference, |·| denotes the absolute value and h·i denotes the average.

It is often the case for complex systems that structure functions of increasing order follow a relation like (Lin and Hughson, 2001):

$$S\_q\left(\mathfrak{r}\right) \propto \mathfrak{r}^{\xi\left(q\right)}.\tag{3}$$

This justifies the plotting of structure functions with double logarithmic axes, since the exponent ζ can be recovered from this plot as the slope of the linear regression, because:

$$
\log \left( \mathbb{S}\_q \left( \mathbb{r} \right) \right) = \mathbb{X}\_q \cdot \log \left( \mathbb{r} \right) . \tag{4}
$$

Therefore, when discussing the slope of S<sup>q</sup> (τ ) plotted in log-log, we are in fact making reference to the exponent ζ . In addition, the double logarithmic plotting enhances the visualization of small scales.

An analogous definition applies to spatial series of data, where I (x) are recordings obtained simultaneously in a spatial arrangement, instead of at successive times (Stotskii et al., 1998). In that case, the structure function is called spatial structure function, in opposition to the temporal structure function, in which we are interested here. In the case of a velocity field, a linear transformation (i.e., the velocity) relates the spatial and the temporal structure functions.

#### Step by Step Algorithm for the Calculation of the Temporal Structure Function

In this section, we introduce briefly a step by step algorithm for the calculation of the temporal structure function from a given signal. The procedure is illustrated in **Figure 1**. To calculate the temporal structure function of a signal I (t), proceed as follows.

Step 1: Define the range of scales of interest, going from the minimum scale τmin to the maximum scale τmax.

Step 2: Define the range of orders of interest, going from the minimum order qmin to the maximum order qmax.

Step 3: Calculate the difference between each element of the time series I<sup>t</sup> and the element separated from it by a number of elements equal to τmin:

$$I\_t - I\_t + \pi\_- \min. \tag{5}$$

The number of values obtained is equal to the length of the time series minus the scale (n − τmin).

Step 4: Calculate the absolute value of the differences obtained from step 3.

FIGURE 1 | How to apply the algorithm for the calculation of the temporal structure function to a sample time series. In this example, we calculate the structure function of order 1, therefore the power step is not needed. The number of differences computed for each scale (used as the denominator to calculate the average) is equal to the length of the time series minus the scale. We show how to calculate three points of a temporal structure function (S(τ1,2,3)), corresponding to scales 1–3. Typically, large scale ranges are of interest to characterize short and long term dynamics (for instance S(τ1−1000)).

Step 5: Elevate the values obtained from step 4 to the power of qmin.

Step 6: Calculate the average of all the values obtained from step 5.

Step 7: Increase τ and repeat steps 3–6 until τmax is reached. In this way, one obtains Sq\_min (τ ).

Step 8: To calculate structure functions of higher order, repeat steps 3–7 until qmax is reached.

In this paper we focus on temporal structure functions of order 1, in which case steps 5 and 8 are needless. The behavior of the slope of the structure function at increasing q indicates the kind of fractal structure present in the signal: monovs. multifractal properties (Lin and Hughson, 2001). These properties might indeed be useful for a better characterization of neuronal signals. However, in previous work we found that the structure function of pallidal neurons of the rat shows no great differences for orders up to q = 6 (Andres et al., 2014b). Therefore in this paper we chose to stay at order one, for simplicity. As is good practice, we normalized S (τ ) by dividing by the initial value, and therefore all the structure functions shown in this paper start at S (1) = 1.

If the signal of interest is a neuronal recording of spiking activity, some previous conditioning is needed before applying the algorithm described. Previous steps include: (1) isolate single neuronal activity employing some spike sorting algorithm (see for example Quiroga et al., 2004), (2) detect the times of occurrence of spikes, and (3) build a time series of interspike intervals (ISI) to obtain the time series I (t). **Figure 2** illustrates the whole transformation process from the raw neuronal recording to the structure function.

# TEMPORAL STRUCTURE OF TOY SYSTEMS

#### Random Time Series

Theoretical work demonstrates that the structure function of random signals has mean slope equal zero (Lin and Hughson, 2001). To test the practical implementation of this statement, we generated 30 random signals that followed a normal distribution (mean = 1, SD = 0.1) with a length of 10<sup>4</sup> numbers. We obtained S (τ ) of these random signals and calculated the slope of S (τ ) with a linear regression (**Figure 3**, upper panel). The slope had a value close to zero for all cases (8.07·10−<sup>8</sup> ± 2.31·10−<sup>7</sup> : mean ± standard deviation; SD).

A well-known measure in time series analysis is the autocorrelation function. In the case of random series, the autocorrelation falls rapidly to zero indicating independent behavior of the elements of a signal. This behavior is hardly differentiated from the rapid loss of autocorrelation of complex nonlinear systems with highly variable output, like the Lorenz system. On the contrary, the zero-slope behavior of the structure function of random signals is clearly different from the temporal structure of complex systems.

#### Oscillatory Signals

We calculated S(τ ) from sin(x) (**Figure 3**, middle panel). Structure functions of oscillatory signals oscillate steadily between a lower and an upper bound. In general, the frequency of oscillation is translated into the structure function following the rule

$$
\omega(\mathfrak{r}) = \omega(I(t)) / samp,\tag{6}
$$

where I(t) is the original signal and samp is the sampling rate. As is the case with random signals, the mean slope of S(τ ) of perfectly oscillatory signals is close to zero (linear regression, slope = −1.18 x 10−<sup>4</sup> ).

To test the effect of adding increasing noise levels to an oscillatory signal, we multiplied a time series of random numbers with normal distribution (0 ± 1, mean ± SD) by a factor of 1.0, 1.5 and 2.0 successively, and added it to sin(x). We calculated S (τ ) from the three resulting time series. The results of adding increasing noise levels are plotted in blue, red and black in the middle panel of **Figure 3**. The slope of the temporal structure function S(τ ) remains around zero as the noise level is increased, while its amplitude diminishes, due to the effect of the random variable on the average term of the structure function. It needs to be noted that the slope of S(τ ) of periodic signals is not zero, if scales smaller than the period are considered. However, the mean slope of S(τ ) of periodic oscillatory signals is close to zero, if a time sufficiently longer than the period itself is measured, which is observed in our results.

#### Nonlinear Systems

To analyze the structure function of nonlinear systems with complex properties we obtained a signal representative of the time evolution of the well-known, chaotic Lorenz attractor (Strogatz, 1994). We integrated numerically the Lorenz equations with the Euler method, then extracted the temporal variable x(t) and finally calculated the structure function of x(t) (**Figure 3**, lower panel). The results show a structure function with a positive slope at small scales and a clear breaking point. At this breaking or inflection point, which we have called τ<sup>1</sup> in previous work, the function turns into a plateau, turning more or less abruptly into a zero-sloped function (Andres et al., 2015). This behavior is related to the loss of autocorrelation of the system, associated to its chaoticity. However, the autocorrelation function C(τ ) is very similar for random and complex systems. On the contrary, complex systems with long range, nonlinear correlations (like the Lorenz system) exhibit structure functions S(τ ) dramatically different from random and oscillatory signals. The main difference is observed in the mean slope of S(τ ), which is no longer zero at every scale.

To test the effect of added noise on this nonlinear system, we followed a similar procedure as with the oscillatory signals. We multiplied a time series of random numbers with normal distribution (0 ± 1, mean ± SD) by a factor of 1.0 and 2.0 successively, and added it to the nonlinear time series. The position of the breaking point in the function does not change as noise is added to the signal (**Figure 3**, lower panel, blue line: no noise, red and black lines: increasing noise levels). However, the height of the plateau of the structure function is sensitive to the noise level, and as a consequence the slope of the function tends to zero as noise is added.

#### NEURONAL RECORDINGS

To exemplify the implementation of the method, we analyzed neuronal recordings of the entopeduncular nucleus of the rat (analogous to the internal segment of the globus pallidus in the primate/human: GPi). The experimental protocol was revised and approved by FLENI Ethics Committee, Buenos Aires, Argentina. Recordings belonged to two groups of animals: healthy and parkinsonian rats. Detailed methodological information can be found in Andres et al. (2014a). Briefly, in adult Sprague-Dawley rats we induced Parkinsonism implementing the 6-hydroxydopamine (6-OHDA) partial retrograde lesion of the nigrostriatal pathway. We recorded spontaneous neuronal activity of the GPi under intraperitoneal anesthesia with chloral-hydrate and at increasing levels of

(Continued)

#### FIGURE 2 | Continued

calculated as shown (ISI=time elapsed between the occurrence of a spike and the next). (Middle) Sample time series of interspike intervals, obtained from a neuronal recording like the one shown in the upper panel. The vertical axis indicates ISI duration (ms) and the horizontal axis indicates ISI number (position in the time series). Notice the high variability of the ISI, typical of complex systems. (Lower) Temporal structure function obtained from a time series of ISI like the one shown in the middle panel. The vertical axis is the value of the function S (τ ) and the horizontal axis is the scale τ . In pallidal neurons it is common to observe a positive slope of the function at lower scales, followed by a breakpoint and a plateau at higher scales, also typical of complex systems. The double logarithmic scale helps visualization of smaller τ .

alertness. Alertness levels are as described in Andres et al. (2014a): level (1) deep anesthesia; level (2) mild alertness; level (3) full alertness. We analyzed a total of 45 neuronal recordings, belonging to the following groups: from 11 healthy animals, 22 neuronal recordings (level 1: n = 5; level 2: n = 11; level 3: n = 6) and from 9 parkinsonian animals, 23 neuronal recordings (level 1: n = 7; level 2: n = 9; level 3: n = 7). In **Figure 4** we show sample structure functions of neuronal recordings, to illustrate the occurrence in neuronal activity of features such as those of toy systems (randomness, oscillations and nonlinear properties). A majority of the recordings (64%) showed marked nonlinear behavior (corresponding to type A neurons of Andres et al., 2015). Additionally, 13% of the recordings presented clear oscillations. A minority of neurons (33%) presented a zero-slope of the structure function at all scales, indicating random behavior. These percentages did not vary significantly between the control and the parkinsonian group or at different levels of alertness, but these results need to be further tested with greater numbers of recordings.

Two characteristic numbers can be extracted from S(τ ) to quantify its behavior, when a change of slope typical of nonlinear systems is observed: the inflection point τ1, where the slope of the function changes, and the height of the plateau, which we call Sp. In previous work we developed an algorithm for the calculation of τ1, and observed a higher τ<sup>1</sup> in GPi neurons with nigrostriatal lesion (Andres et al., 2015). This observation was done under conditions of full alertness, i.e., animals were under local anesthesia plus analgesia, alert and head restrained at the moment of the surgery. Now we calculated τ<sup>1</sup> applying the same algorithm from neuronal recordings obtained during the whole arousal process, going from deep anesthesia, to mild and full alertness. All statistical comparisons were calculated applying the Kolmogorov-Smirnov test; results were considered statistically significant when p < 0.05. Results show that τ<sup>1</sup> is higher in the parkinsonian group at all alertness levels, with a more pronounced effect as alertness increases (i.e., as the animal awakens from anesthesia; **Figure 5**, right panel). These results were not statistically significant (p > 0.05) and need to be tested on more experimental data.

We calculated the plateau height S<sup>p</sup> as the mean value of S(τ ) for 100 < τ < 200, a range where all the neurons analyzed had reached a plateau, if this was present. We have shown in the previous section that the plateau height is sensitive to the amount of noise added to a signal. In this sense S<sup>p</sup> might not be reliable as a raw measure to compare neuronal data corresponding to different experimental groups. This disadvantage can be overcome by studying variations of S<sup>p</sup> instead of raw values, i.e., subtracting a given S<sup>p</sup> from a previous value of itself obtained under the same recording conditions. In our study case we recorded activity from single neurons during long periods of time (1–3 h), and we can safely assume that environmental conditions (electrical noise and any other source of interference) did not vary during the whole recording. We calculated S<sup>p</sup> from isolated segments of activity obtained at the beginning, middle and end of the recording, corresponding to deep anesthesia, mild alertness and full alertness, respectively. Thus, we obtained the following values of Sp: Sp1−2, as the difference between the plateau height at mild alertness minus the plateau height at deep anesthesia, and Sp2−3, as the difference between the plateau height at full alertness minus the plateau height at mild alertness. In the control group we did not observe any difference between Sp1−<sup>2</sup> and Sp2−3, whereas under parkinsonian conditions Sp1−<sup>2</sup> was significantly higher than Sp2−<sup>3</sup> (p < 0.01; **Figure 5**, left panel).

# SUMMARY AND CONCLUSION: GUIDELINES FOR THE INTERPRETATION OF THE TEMPORAL STRUCTURE S(τ ) OF NEURONAL SIGNALS

Nonlinear properties of neuronal activity are critical for normal basal ganglia functioning and deteriorate in specific ways in disease, in particular in movement disorders (Parkinson's disease, dystonia, and others) (Montgomery, 2007; Darbin et al., 2013; Alam et al., 2015). Even more, therapeutic interventions are able to restitute such properties to normal, suggesting the clinical importance of quantifying nonlinear features of neuronal activity (Rubin and Terman, 2004; Lafreniere-Roula et al., 2010). However, up to now the community has not agreed on any method as a gold standard to quantify nonlinear properties of the basal ganglia. This is partly due to difficulties in the implementation of nonlinear methods of analysis, which are typically sensitive to a wide range of parameters. Opposed to that, the temporal structure function S(τ ) is a nonlinear tool of analysis easy to implement, and robust to short recordings, but it is not well known and therefore difficult to interpret. We analyzed the behavior of S(τ ) from signals with known properties (toy systems), and observed that: (1) S(τ ) has zero-slope at every scale for random systems; (2) S (τ ) is oscillatory and bounded for oscillatory systems, and the frequency of oscillations can be recovered from S(τ ) if the sampling rate is known; and (3) S(τ ) has a positive slope at small scales for nonlinear systems, which changes to a plateau with zero-slope at large scales.

In the light of our observations for toy systems we analyzed a number of neuronal recordings of healthy and parkinsonian basal ganglia at different levels of alertness (from deep anesthesia to full alertness). In a majority of the neurons studied, nonlinear behavior was clearly present. In these cases we extracted two

FIGURE 3 | Structure function of toy signals, with and without added noise. (Upper) Random signal, before (black) and after (red) applying a low-pass filter. The slope calculated with a linear regression lies around zero for both cases. (Middle) Oscillatory signal (sin(x)), with increasing levels of added noise, (blue: x1.0, red: x1.5, black: x2.0; see the text). The slope is zero on average, but the oscillations of the signal are clearly translated into the structure function. As added noise increases, the amplitude of the oscillations diminishes. (Lower) Lorenz system (x variable; parameters: σ = 10, ρ = 28, β = 8/3), with increasing levels of added noise, (blue: no noise, red: x1.0, black: x2.0; see the text). The breaking point lies around 40 < τ1 < 110 for this example. Observe (Continued)

#### FIGURE 3 | Continued

that the position of the breaking point in the structure function does not change as added noise increases, but the height of the plateau diminishes. In the limit, the initial ascending phase disappears and the slope of the structure function is zero at all scales, as random dynamics prevail over the nonlinear system.

FIGURE 4 | Different types of structure functions from neuronal recordings show features of random, oscillatory, and nonlinear systems. (Upper) Sample neuron with a structure function showing zero-slope at all scales, indicating random behavior. (Middle) Sample neuron with a structure function showing oscillations. (Lower) This case is the most representative of all the neurons analyzed (64%). The structure function has clear, nonlinear behavior.

characteristic numbers from S(τ ) to quantify its behavior: (1) the height of the plateau (Sp), and (2) the scale of the inflection point or slope change (τ1). Since the plateau height is sensitive to the noise added to the signal, we used it in a relative way, measuring the changes of S<sup>p</sup> for alertness transitions within single neuronal recordings, when we can assume that recording conditions were stationary. For parkinsonian neurons the change of plateau height from deep anesthesia to mild alertness was significantly higher than from mild to full alertness (Sp1−<sup>2</sup> > Sp2−3, p < 0.01). This difference in the variation of S<sup>p</sup> was not observed in the control group. The fact that Sp1−<sup>2</sup> is significantly higher than Sp2−<sup>3</sup> in parkinsonian animals is in agreement with previous observations indicating that the basal ganglia of animals with dopamine depletion do not handle well the awakening process (Andres et al., 2014a). Importantly, as a consequence of the averaging process S<sup>p</sup> is independent from the frequency of discharge of the neurons by definition, solving a previous controversy about the structure function method (Darbin et al., 2016). Regarding the inflection point τ1, it was higher in Parkinson's disease than in control neurons with a more pronounced effect at higher alertness levels, but this effect was not statistically significant (p > 0.05) and needs to be further tested in larger studies. Although preliminary, our findings are relevant for understanding results obtained from human surgery on Parkinson's disease, usually performed with the patient awake, under local anesthesia only. We report on the inability of pallidal neurons with Parkinson's disease to handle normally the transition from anesthesia to alertness, which might be a key finding to better understand the pathophysiology of the basal ganglia.

In previous work, we determined that the positive slope at small scales of the log-log temporal structure function is associated with particular properties of neuronal dynamics. Specifically, in a neuronal network with nonlinear properties we showed that the slope depends on the coupling strength (Andres et al., 2014b). This indicates that the temporal structure function captures critical properties of the underlying dynamics of the system. Importantly, in our previous modeling study we observed that a smaller percentage of neurons behave in random fashion, which seems to be related to the stability of the system (Andres et al., 2014b). This finding is now reproduced in our experimental results.

Finally, we would like to draw some attention to other nonlinear tools that are available for the characterization of neurophysiologic signals (Amigó et al., 2004; Pereda et al., 2005; Song et al., 2007). Every tool shows advantages and shortcomings, making them more or less suitable for the study of specific neurologic systems. Recently Zunino et al. introduced two methods that seem to be particularly powerful for the analysis of physiologic time series (Zunino et al., 2015, 2017). A detailed review and comparison of the performance of the temporal structure function with these other tools is beyond the scope of this paper. Nevertheless, we are still interested in a detailed analysis of the temporal structure function, because it has previously shown to be useful for the characterization of neuronal spike trains obtained from human patients with Parkinson's disease, which tend to be short (around 5,000 data per time series) (Andres et al., 2016). Another limitation of our work is that we have studied the temporal structure function of only one nonlinear system (i.e., the Lorenz attractor). However, our results are supported by results from other well-known nonlinear systems, which exhibit similar behavior (Lin and Hughson, 2001).

To conclude, we would like to draw some brief guidelines for the interpretation of the temporal structure function of neuronal activity. The most important feature distinguishing S(τ ) of random vs. complex signals is its slope. If S(τ ) has zero slope for every τ , randomness can be assumed at the scales analyzed, meaning that the order of the events in the time series (in this case interspike intervals) is not different from what would be observed for an independent variable. On the other hand, if S(τ ) has a first segment with positive slope then turning into a plateau, the behavior falls in the category of complex systems. In this case, one can look at τ<sup>1</sup> and Sp. The breaking point τ<sup>1</sup> is related to the memory limit of the system, and therefore to its chaoticity. For scales below τ<sup>1</sup> the order of events (ISI) is not random, and therefore memory or temporal organization is present. In other words, at scales smaller than τ<sup>1</sup> the times of occurrence of single spikes are not independent from each other, but nonlinear organization plays a role in the signal. At scales larger than τ<sup>1</sup> the system behaves in random fashion. Regarding the neural code, it can be assured that at scales larger than τ<sup>1</sup> only a rate code or other averaged coding scheme can be used, since complex time patterns cannot be transmitted from neuron to neuron beyond the memory limit of the system (Bialek et al., 1991; Ferster and Spruston, 1995). The second quantitative measure that can be obtained from the temporal structure function is Sp. While τ<sup>1</sup> is related to the memory limit and is robust to noisy signals, S<sup>p</sup> is sensitive to added noise. Therefore it is necessary to be cautious when comparing S<sup>p</sup> between experimental data, if it cannot be assured that the data were obtained under similar conditions, in particular regarding external noise and interference. If recording conditions can be safely assumed to be stationary, then variations of S<sup>p</sup> indicate a change in the power of the random components of the system. Finally, oscillations of the original signal are translated into the structure function, and the original frequency can be recovered from S(τ ) if the sampling rate is known.

#### REFERENCES


#### ETHICS STATEMENT

All animal experiments and procedures were conducted with adherence to the norms of the Basel Declaration. The experimental protocol was revised and approved by our local ethics committee CEIB, BuenosAires, Argentina.

#### AUTHOR CONTRIBUTIONS

DA designed the study and conducted the experiments. FN processed and analyzed the data. Both authors contributed to the preparation of the manuscript and approved its final version.

#### FUNDING

The work of Daniela Andres is supported by the Ministry of Science and Innovative Production, Argentina, and by National University of San Martin.

#### ACKNOWLEDGMENTS

The authors acknowledge the work of the technical personnel at National University of San Martin that made this work possible.


mode network is correlated with cognitive function in normal male elderly: a multiscale entropy analysis. Neurobiol. Aging 34, 428–438. doi: 10.1016/j.neurobiolaging.2012.05.004


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2017 Nanni and Andres. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Parkinsonian Balance Deficits Quantified Using a Game Industry Board and a Specific Battery of Four Paradigms

Olivier Darbin1, 2, 3, 4 \*, Coral Gubler 4, 5, Dean Naritoku<sup>1</sup> , Daniel Dees <sup>1</sup> , Anthony Martino<sup>6</sup> and Elizabeth Adams 4, 7 \*

*<sup>1</sup> Department of Neurology, University of South Alabama, Mobile, AL, USA, <sup>2</sup> Division of System Neurophysiology, National Institute for Physiological Sciences, Okazaki, Japan, <sup>3</sup> Animal Resource Program, University of Alabama at Birmingham, Birmingham, AL, USA, <sup>4</sup> Vestibular Research, University of South Alabama, Mobile, AL, USA, <sup>5</sup> Department of Physical Therapy, University of South Alabama, Mobile, AL, USA, <sup>6</sup> Department of Neurosurgery, University of South Alabama, Mobile, AL, USA, <sup>7</sup> Department of Speech Pathology and Audiology, University of South Alabama, Mobile, AL, USA*

This study describes a cost-effective screening protocol for parkinsonism based on combined objective and subjective monitoring of balance function. Objective evaluation of balance function was performed using a game industry balance board and an automated analyses of the dynamic of the center of pressure in time, frequency, and non-linear domains collected during short series of stand up tests with different modalities and severity of sensorial deprivation. The subjective measurement of balance function was performed using the Dizziness Handicap Inventory questionnaire. Principal component analyses on both objective and subjective measurements of balance function allowed to obtained a specificity and selectivity for parkinsonian patients (vs. healthy subjects) of 0.67 and 0.71 respectively. The findings are discussed regarding the relevance of cost-effective balance-based screening system as strategy to meet the needs of broader and earlier screening for parkinsonism in communities with limited access to healthcare.

Keywords: center of pressure, oscillations, irregularity, dispertion, self-reported symptoms, fall, movement disorders

# INTRODUCTION

Parkinson disease (PD) is a progressive disorder that affects both peripheral and central nervous systems. Current treatment strategy include dopaminergic-replacement and deep brain stimulation (Connolly and Lang, 2014); ongoing research suggests that treatment aimed to slow down or block the progression of the disease may become available (Schapira et al., 2014). The successful implementation of these current and future treatment strategies depends, at least for a part, on the screening for subjects with high risk for parkinsonism, especially those depending on community health system with time, resource, and staffing constraints (Bennett et al., 1996; Birbeck et al., 2015). In the current pilot study, we evaluated a cost-effective screening system (low equipment and personal costs) based on balance monitoring to identify subjects with high risk for parkinsonism-related disorders.

Balance control is a multisystem function relying on the integration of vestibular, somatosensory, and visual inputs. In patients with movement disorders, both abnormal static posture (Del Din et al., 2016) and pathological dysfunctions in sensory-motor circuitry contribute

#### Edited by:

*Mikhail Lebedev, Duke University, USA*

#### Reviewed by:

*Fady Alnajjar, Riken Brain Science Institute, Japan Thorsten Stein, Karlsruhe Institute of Technology, Germany Susan Elizabeth Esposito, Life University, USA*

#### \*Correspondence:

*Olivier Darbin olivieredarbin@gmail.com Elizabeth Adams eadams@southalabama.edu*

Received: *28 May 2016* Accepted: *11 August 2016* Published: *30 August 2016*

#### Citation:

*Darbin O, Gubler C, Naritoku D, Dees D, Martino A and Adams E (2016) Parkinsonian Balance Deficits Quantified Using a Game Industry Board and a Specific Battery of Four Paradigms. Front. Hum. Neurosci. 10:431. doi: 10.3389/fnhum.2016.00431* to the loss in balance function and the increased risk for fall (Gatev et al., 2006; Darbin, 2012; Darbin et al., 2013a; Schrag et al., 2015). The integration of vestibular information involves a large circuitry. Briefly, the pedunculo-pontine nucleus (PPN) receive primary (Woolf and Butcher, 1986; Hazrati and Parent, 1992) order neurons from the vestibular nucleus and project diffuse acethylcholinergic fibers the basal ganglia–thalamo– cortico loops [Striatum, STR (Saper and Loewy, 1982; Lavoie and Parent, 1994a); Subthalamaic nucleus, STN (Nomura et al., 1980; Saper and Loewy, 1982; Edley and Graybiel, 1983; Hammond et al., 1983; Sugimoto and Hattori, 1984; Scarnati et al., 1987; Lavoie and Parent, 1994a; Bevan and Bolam, 1995; Muthusamy et al., 2007; Kita and Kita, 2011); Substantia nigra mostly compacta, SNc (Saper and Loewy, 1982; Lavoie and Parent, 1994a,b); Globus Pallidus (Saper and Loewy, 1982; Lavoie and Parent, 1994a; Muthusamy et al., 2007) interna and externa, GPe (Saper and Loewy, 1982; Scarnati et al., 1987) and GPi (Saper and Loewy, 1982; Scarnati et al., 1987); Thalamus (Saper and Loewy, 1982; Scarnati et al., 1987; Smith et al., 1988, 2004; Muthusamy et al., 2007) including the centro-median part, CM (Sugimoto and Hattori, 1984) and Parafasicular nucleus, Pf (Sugimoto and Hattori, 1984; Scarnati et al., 1987); and primary Motor Cortex, M1 (Muthusamy et al., 2007)] in addition to brainstem nuclei, cerebellum, hypothalamus and spinal cord (Martinez-Gonzalez et al., 2011). The locus coeruleus (A6) receive primary (Fung et al., 1987) and secondary neurons [via the ventrolateral medulla (Nishiike et al., 2001; Holstein et al., 2011)] from the vestibular organ and project acethylcholinergic fibers the basal thalamo– cortico loops (Thalamus including the intralaminal complex and motor Cortex) in addition to brainstem nuclei, cerebellum, hypothalamus, and spinal cord (Fornai et al., 2007).

In the parkinsonian brain, Braak et al. (Braak et al., 1995, 1996, 2002, 2005; Braak and Braak, 2000; Del Tredici et al., 2002; Burke et al., 2008; Fujishiro et al., 2008) have suggested that pathology begins in post-ganglionic neurons and nuclei in the brainstem and progresses to higher centers (Wakabayashi and Takahashi, 1997; Orimo et al., 2005, 2008). Regarding the balance function in parkinsonism, uncertainty resides on whether or not the acethylcholinergic vestibular efferent neurons are affected (de Waele et al., 1995; Rabbitt and Brownell, 2011). However, most studies on parkinsonism have reported loss of neurons in the pedonculo-pontine nucleus (Del Tredici and Braak, 2012), the locus coeruleus (Mann and Yates, 1983; Halliday et al., 1990; German et al., 1992; Braak et al., 2003) and, emblematic to the condition, the substantia nigra compacta (Mann and Yates, 1983; Halliday et al., 1990; German et al., 1992; Braak et al., 2003). Degeneration of the pedonculo-pontine nucleus is cause for depletion in acetylcholine into the thalamo-basal ganglia circuitry, brainstem nuclei, cerebellum spinal cord, and hypothalamus. Degeneration of the locus coeruleus is cause for depletion in noradrenaline into the thalamo-cortical circuitry, brainstem nuclei, cerebellum spinal cord, and hypothalamus. The degeneration of the substantia nigra causes a depletion of dopamine in the sensory motor part of the basal ganglia circuitry. Higher areas, such as the insula and other areas of cortex, are also affected by PD pathology (Braak et al., 2003; Bertrand et al., 2004).

Data raised above indicate that in parkinsonian state, balance circuitry is altered by multiple lesions and balance dysregulations result from declines in vestibular (Bertolini et al., 2015), proprioceptive (Bekkers et al., 2014) and visual functions (Redfern et al., 2001; Schrag et al., 2015). Importantly, synergy and compensatory mechanisms between these systems contribute to partially compensate the decline in balance function as disease progresses (Rinalduzzi et al., 2015).Therefore, the decline in balance function related to parkinsonian condition is a complex resultant of decline in sensorimotor components (Reichert et al., 1982; Becker-Bense et al., 2015), changes in compensatory mechanisms between these systems (Shumway-Cook and Horak, 1986; Bronstein et al., 1990; Bekkers et al., 2014) and effects of treatments (Hely et al., 2005; Collomb-Clerc and Welter, 2015; Curtze et al., 2015).

The sensitivity of balance function to parkinsonian state and its dynamic with disease progression make standing stability a putative marker to screen test subjects with high risk for this condition. However, the access to routine monitoring of balance function remains limited because of its high equipment and personnel costs as trained staff is often needed for the analyses of data. Cost-effective strategies are needed for broad and frequent evaluation of balance function which can also be beneficial to provide adapted recommendations, adjust treatments to safer levels and reduce the psychosocial disabilities related to the risk for fall related to parkinsonism.

In the current study, we have investigated a cost effective system to screen test subjects with parkinsonism using a combination of subjective and objective measures of balance functions (Kahle and Highsmith, 2014). Subjective deficits were evaluated by the Dizziness Handicap Inventory; objective deficits were measured using a low cost balance board (Clark et al., 2010; Holmes et al., 2013) to extract features of the center of pressure in time, frequency and non-linear domains. Automated multivariate analyses of the subjective and objective measurement was developed and showed satisfactory selectivity and selectivity to screen test PD patients from healthy population.

# MATERIALS AND METHODS

# Population and Testing Session

The testing was performed in one 60-min session at either the Vestibular Research Laboratory in the Department of Speech Pathology and Audiology at the University of South Alabama or at the University of South Alabama Neurology Clinic. In order to complete the experimental tasks, all participants were required to possess the strength and stamina to stand unsupported for the duration of 60-s intervals, in multiple test conditions. The test procedure (see below) required approximately 10 min to administer and both healthy subjects and patients had a positive attitude during these testing paradigms.

Exclusionary criteria for the control group included history of previous neurologic and otologic disease, and additional significant medical history. Exclusionary criteria for the PD group included history of previous otologic disease, other neurologic disorders, and additional significant medical history. All participants read and signed a statement of informed consent approved by the Institutional Review Board at the University of South Alabama.

Sixteen individuals grouped based on their history of diagnosed Parkinson's disease (PD) participated in the present study Three males and two females served as participants in the control group (n = 5). Individuals in the control group ranged in age from 48 to 69 years (61 yo; 59–68), and were negative for history of neurologic incident or diagnosis, balance disturbance, vertigo, and significant otologic pathology. The PD group (n = 11) consisted of seven males and four females, and ranged in age from 60 to 83 years (65 yo; 60.5–71). Time since PD diagnosis preceding experimental testing ranged from 1 to 34 years (4 years; 3–7.5). The two groups did not significantly differed in age (P > 0.1).

#### Procedures

In order to assess the impact of dizziness, imbalance, or unsteadiness on daily life activities, all participants completed the Dizziness Handicap Inventory (DHI; Jacobson and Newman, 1990). The DHI questionnaire contains 25 items, identified in three subscales: Functional limitations, physical movement, and emotional well-being. Items in the functional limitations subscale assess the extent to which dizziness, imbalance, and unsteadiness limit participation in normal daily activities. The physical subscale assesses the impact of specific head and body movements in precipitating and exacerbating feelings of dizziness, imbalance, and unsteadiness. Items within the emotional subscale assess the impact of the dizziness, imbalance, or unsteadiness on the individual's emotional well-being, such as feelings of depression, social isolation, and the effect of the problem on personal relationships. Participants were required to respond verbally to each item using the forced-choice, closedset response alternatives "yes," "sometimes," or "no." Participants were instructed to respond to each item as it pertained only to dizziness, imbalance, or unsteadiness experienced, separating this from the effects of Parkinson's disease as much as possible. Scoring of the DHI was completed such that "yes" responses were assigned four points; "sometimes" responses were assigned two points; and "no" responses were assigned zero points. Therefore, the highest possible score on the DHI was 100 points, indicating significant handicapped related to dizziness, imbalance, or unsteadiness. Total scores on the DHI close to zero indicate little to no impact of dizziness, imbalance, and unsteadiness on daily life.

The modified Clinical Test for Sensory Interaction on Balance (mCTSIB; Shumway-Cook and Horak, 1986) was used to assess each participant's functional ability to maintain balance in four conditions. Participants were instructed to maintain a quiet stance with bare feet approximately shoulder width apart on: (1) a firm surface with eyes open; (2) a firm surface with eyes closed; (3) a compliant surface with eyes open; and (4) a compliant surface with eyes closed. Each condition was maintained for 60 s, or until a corrective step was made or fall was imminent. This duration of testing was chosen to improve the sensitivity in the dynamical analyses of the center of pressure. During the firm surface conditions, participants stood directly on the Balance Board. High-density foam (18′′ × 24′′ × 6 ′′), marked for correct foot placement, was used in the two compliant surface conditions. The foam was of sufficient density to prevent the participant from sinking to the firm surface below, thereby eliminating reliable somatosensory cues. Slight modification of the foam, wherein a small overhang was added to the foam block, allowed for direct placement of the foam atop the Balance Board without movement during testing. Sensitivity of the Balance Board to force on each sensor was maintained during testing with the high-density foam. Conditions were tested in a standardized order: (1) hard floor and open eyes, (2) hard floor and closed eyes, (3) foam and open eyes and finally (4) foam and closed eyes. Patients were allowed to relax between testing. Special marks on the board were used to reduce the variability in foot positioning between patients or when patients needed a break between conditions.

Conditions in the mCTSIB selectively distort or eliminate the visual and somatosensory input used to maintain balance. Specifically, visual input was removed in the two eyes closed conditions (conditions 2 and 4), and somatosensory input was made unreliable in the two high-density foam conditions (conditions 3 and 4). Therefore, in condition 2, the participant was only able to rely on somatosensory and peripheral vestibular input to maintain balance, and in condition 3, the participant was only able to rely on visual and peripheral vestibular input to maintain balance. In condition 4, the visual input was removed (eyes closed) and the somatosensory input was distorted and unreliable (foam); therefore, the participant had only input from the peripheral vestibular apparatus to maintain balance and upright posture.

#### Device and Software

During the mCTSIB, measurements of center of pressure were made using a Nintendo Wii Balance Board. The Balance Board has a sensor on each of the four corners, which measure the force of each foot in the specific quadrants of the board. Information gathered from the sensors was sent to the interfaced computer and software, based on the BlueTooth Toolbox (http://forums.ni. com/ni/attachments/ni/170/265158/1/wiimote.zip). The values (expressed in Kg) from the four (left front, LF; right front, RF; left back, LF; right back, RB) sensors were captured at a sampling rate of 20 Hz and normalized by the weight of the patient. The weight of the participant (Kg) at each time was calculated according to the following equation:

$$Weight\,\,(\text{Kg}) = \frac{\sum \,\text{values\,of\,\,each\,\,sensor}}{102} \tag{1}$$

The time series generated by each sensor were recorded in a TMS file (labview format) and stored on the hard disk for off-line analyzes in matlab environment.

Specifically, four successive sequences of recording were run for the four paradigms previously described. Each recording sequence was started by the investigator and ended automatically at the end of the 60-s test interval. The investigator stopped the recording if a corrective step was made or fall was imminent.

#### Pre-processing

All the analyses were performed off-line in Matlab environment.

The euclidian coordinates of the center of pressure were first normalized by:

$$sm\text{ (\$t\$)} = \frac{s~(t) - mean\text{ (\$S\$)}}{std\text{ (\$S\$)}}\text{ with S defined either by X or Y}\text{ (\$2)}$$

In which Sn is the normalized signal, S is the original signals, mean is the average and std is the standard deviation of the original signals. Preliminary study showed this technique useful to reduce the variability between patients and causal to their differences in weight.

Polar coordinates were then calculated from the normalized Euclidian for orienting geometry to the position of center of pressure between [−π, +π] with a zero angle at the median of the frontal segment. The polar angle (or azimuth) is defined by:

$$
\alpha = \operatorname\*{atang2}\left(\mathbf{Y}, \mathbf{X}\right) \tag{3}
$$

The azimuth was used for the analyses described below.

#### Analyses

Preliminary principal component analyses allowed to select a limited number of features in time, frequency and non-linear domains based on independence.

#### Time Domain Analyses

The median absolute deviation (MAD) was used as a feature for dispersion of the azimuth and was defined by:

$$\text{MAD} = \text{median}\left(\text{abs}\left(\alpha\_t - \text{median}\left(\alpha\right)\right)\right) \tag{4}$$

#### Frequency Domain Analyses

The Fourier Transformed (FFT, n = 100 pts, 5 s non-overlapping window) was used to calculate the power spectrum of the azimuth time series (αt). The low (0.4–2 Hz) and high frequency bands (2–5 Hs) were calculated.

#### Non-linear Domain Analyses

We used the Approximate Entropy (ApEn) parameter as an indicator of statistical irregularity. ApEn quantifies the randomness of fluctuations in a given data stream (Pincus, 1995). Previous studies have used this parameter to describe the level of complexity of fluctuations of autonomic functions such as the pulse rate (Pincus and Viscarello, 1992; Darbin et al., 2002; Naritoku et al., 2003), fluctuation in EEG and neuron discharge (Darbin et al., 2006, 2016; Lafreniere-Roula et al., 2010) or movement dynamic (Vaillancourt and Newell, 2000; Vaillancourt et al., 2001). Following the method of Pincus to calculate ApEn (Pincus, 1991), we used three parameters in computing the ApEn value: The number of points in the time series (N), the embedding dimension m, and the vector comparison length r. Because ApEn is dependent on the recording length (N), the length of the data streams was fixed to be 20 points (equivalent to 1 s for a sampling rate of 20 Hz) and this running window was applied without overlapping along the recording. The median of the windows was used as final entropy feature. In line with previous studies (Pincus, 1995), the use of small m and moderate r ensures the reliability of ApEn and provides better accuracy for comparisons between samples. Therefore, the embedding dimension m was empirically set to 2 and the parameter r was calculated for each recording as 15% of the SD of the time series (Pincus, 1995). As the first step in the calculation of the ApEn value, the "correlation integral" was computed as the number of vectors whose distance from the vector under study was less than r, using a lag of 1. The natural logarithm of the correlation integral was averaged over the N points. This process was repeated m+1 times, and the ApEn value was finally computed as the difference between the values at m and m+1 (Pincus, 1991). Low ApEn values are indicative of low irregularity, while high ApEn values indicate high irregularity (for discussion see (Pincus, 1995; Darbin et al., 2013b, 2016). ApEn was chosen because in respect to m and r fixed, ApEn require low number of point to compare irregularity between groups (Pincus, 1991, 1995).

#### Principal Component Analyses and Binary Classification Test

In order to reduce the impact of the differences between the distribution of the features, we applied the principal component analyses on the rank transformed data (Jackson, 2005). A table including the DHI scores and the features for each standing testing condition (standing duration, Low Frequency, High Frequency, ApEn, Median absolute deviation) was constructed and the ranks calculated for each features over the population of subjects (healthy subjects and PD patients). The three first components of the PCA were investigated in the present study.

Sensitivity and specificity were used as measure of the performance of the first three principal components to classify the PD patients from the tested population. True positives (TP; PD patients identified as PD patients), false positive (FP; healthy subject identified as PD patients), true negative (TN; healthy subjects identified as healthy subjects), and false negative (FN; healthy subjects identified as PD patients) were calculated for every cut off values defined between the lower and upper limits of the principal components.

Sensitivity was calculated as the number of true positive divided by the number true positive and the number of false negative.

$$sensitivity = \frac{number\text{ of true positive}}{number\text{ of true positive} + number\text{ of false negative}}\tag{5}$$

Specificity relates to the test's ability to correctly detect patients without a condition. Consider the example of a medical test for diagnosing a disease. Specificity of a test is the proportion of healthy patients known not to have the disease, who will test negative for it. Mathematically, this can also be written as:

$$\text{specificity} = \frac{number \text{ of true negative}}{number \text{ of true negative} + number \text{ of false positive}} \tag{6}$$

Specificity and selectivity were selected at the cut off value giving the maximal accuracy (acc) for each components tested and accuracy was defined by the ration of true positive and true negative to the total number of subjects:

$$Accuracy = \frac{number\ of\ true\ positive + number\ of\ true\ negative}{number\ of\ heathly\ subjects + number\ of\ PT\ patients} \tag{7}$$

#### Statistics

Data were expressed by the median and the 25th and 75th percentiles. We used the Kruskal–Wallis test for inter group comparisons and the Mann–Whitney U–test for intragroup comparisons with a threshold at 0.05 (Siegel and Castellan, 1989). PCA on quantile normalized values from the analyses of the center of pressure and the DHI was used to identify groups of correlated features.

# RESULTS

#### Dizziness Handicap Inventory

Possible scores on the DHI range from 0 to 100, with higher scores indicating greater self-reported difficulty with dizziness and/or imbalance. The DHI scores for the control group ranged from 0 to 4, (0;0–5.5) while the DHI scores for the PD group ranged from 0 to 44 (6, 0–14) (P < 0.05). Two participants in the control group indicated intermittent difficulty on two separate DHI questions. One control participant indicated intermittent dizziness or imbalance with quick movements of the head (item 11), and the other participant indicated intermittent dizziness or imbalance when bending over (item 25). These results are in stark contrast to the DHI results from the PD group. About two\third of the PD population reported score below the pathological threshold (60%, score< = 6) while the third of participants in the PD group (40%) scored above pathological threshold (score > 6). In the PD population, a majority of patients (60%) reported at least some level of difficulty with dizziness and/or imbalance on multiple items of the DHI. Twenty-one of the 25 DHI items were noted as problematic for at least some of the participants with PD (see Supplemental table 1); however, the difficulties noted by the participants with PD was quite variable across DHI items. The four items most frequently noted by the participants with PD to cause difficulty were items 11, 19, 25, and 5. Items 11 and 25 are within the physical subscale of the DHI, which assesses factors that precipitate and exacerbate symptoms of dizziness and/or imbalance. These are the same two items identified by two participants in the control group (quick head movements and bending over); however, a greater number of participants in the PD group noted difficulty with these situations. Participants with PD also frequently identified limitations with ambulating in the dark (item 19) and with getting into and out of bed (item 5), suggesting functional limitations in the daily activities. Difficulty with items on the emotional subscale of the DHI was noted by a small number of PD participants. These participants reported feeling embarrassed (item 10), frustrated (item 2), handicapped by their dizziness or imbalance (item 21), and afraid others may perceive them as intoxicated (item 15). A few of the participants with PD also reported that their dizziness and/or imbalance caused them to restrict participation in social activities (item 6) and travel for business or recreation (item 3), as well as to experience functional limitations related to household and job responsibilities (items 14 and 24). Overall, there was variability in the items noted by the participants in the PD group as causing difficulty. These results suggest that individuals with PD experience functional limitations and negative emotional responses to dizziness and/or imbalance experienced; however, the exact effects of the dizziness and/or imbalance is not standard across individuals with PD.

# Standing Time as Function of the Paradigm

All control subjects were able to perform the four mCTSIB conditions for the duration of 60 s. In the PD group, all subjects performed well with normal proprioception (firm surface) and 81.9% of them succeeded the standing test with sensorial deprivation. Only 18.2% of PD patients lost their balance before 60 s when tested on the foam (see Supplemental table 2). For those patients who lost balance on the foam, the loss of visual input drastically reduced their standing duration from 23.33 s (8.9–37.8) to 4.52 s (4.45–4.6) (p < 0.05) (see Supplemental table 3).

# Screen Testing Based on Multivariate Analyses of Subjective and Objective Measurement

Screen testing strategy used in this study was based on a multivariate analysis of subjective measurement (DHI scoring) and objective measurement of features in the time, frequency, and non-linear domain calculated standing balance performance and the dynamic of the center or of pressure collected during the paradigms tested.

The Euclidian space defined by the first three components of the PCA is presented **Figure 1** and the abbreviation used to label the binomial (paradigm, feature) are indicated in **Table 1**. The three first components explained individually more than 15% of the variance and, combined, 63% of the variance (see **Table 2**).

The contributions (weight) of each variable in the first three principal components are given in **Table 3**. The first component (C1, 26.24%) opposed irregularity and duration (FCApEn, FOApEn, FCT > 0.25; F:foam, C: closed eye, O: open eyes, ApEn: irregularity, T: duration of standing) to oscillations and dispersion (FCMAD,FCHF,FCLF < −0.25; MAD: median absolute deviation, HF: high frequency, LF: low frequency) mostly for paradigms on with combined sensorial deprivation (foam and closed eyes).

The second component opposed irregularity and high frequency oscillations (HCApEn, FOHF, HOApEn > 0.3) to low frequency and dispersion (HOLF, HCLF, FOMAD < −0.3) mostly on the firm surface.

Subjective measurement of balance function poorly contributed poorly to these two first components as indicated by the weight of the DHI scoring contained between 0.15 and −0.15 for both C1 and C2.

In contrast, component 3 presented relevant clinical significance as it opposed self-reported symptoms (SRS > 0.25) to duration (T < −0.11). C3 also opposed dispersion and high frequency (HCMAD, HCHF, HOHF, FCHF, FOMAD > 0.25) to LF and ApEn (FOLF, HCApEn < −0.15). Paradigms with visual deprivations mostly contributed to this component (HCMAD, HCHF, HOHF, FCHF opposed to HCApEn, FCT).

FIGURE 1 | (A) Shows the features calculated on the dynamic of the center of pressure monitored during the four paradigms and expressed in the space defined by the first three components of the PCA. Abbreviations for the binomes (feature,paradigm) are detailed in Table 1. SRS indicate the scoring to the DHI. (B–D) Shows the specificity and selectivity of component 1, 2, and to discriminate parkinsonian patients from healthy subjects. TP, true positive; TN, true negative; FP, false positive; and TN, true negative.

TABLE 1 | This table indicate the list of the abbreviations for each paradigm and feature used for the processing of the principal component analyses.


*The four paradigms of the modified Clinical Test for Sensory Interaction on Balance (mCTSIB) are indicated in rows: HO, hard surface with eyes opened; HC, hard surface with eyes closed; FO, foam and eyes open; FC, foam and eyes closed. The features processed on the azimuth of the center of pressure are indicated in columns: LF, low frequency oscillations in the azimuth of the center of pressure; HF, high frequency oscillations in the azimuth of the center of pressure; ApEn, Approximate Entropy from the azimuth of the center of pressure; T, duration of the subject's stand during the paradigm.*

Analyses of the selectivity and specificity between the three first components to differentiate PD subjects from healthy subjects showed, unsurprisingly, that component 3 discriminated best these two populations (**Figure 1C**, sensitivity: 0.67, specificity: 0.71; **Figure 1**).

# DISCUSSION

#### General Discussion on the Study

We have investigated a new paradigm to screen patients with parkinsonism based on a multivariate analyses of subjective and objective measurements of the balance function. The system includes features from the DHI questionnaire and features of on the dynamic of the center of pressure monitored during a 4-paradigm standing test and using a low cost force board. An automated analyses based on principal component allowed to achieve a sensitivity of 0.67 and selectivity of 0.71.

In PD population, the obvious variability in score to the DHI and self-reported dizziness, unsteadiness, or imbalance may relate to many factors ranging from severity of the disease (Lee et al., 2016), age (Harris et al., 2015; Venhovens et al., 2016), response to treatment (Odin et al., 2015), occupational activity (Frazzitta et al., 2015), and awareness on the condition (van der Kooij et al., 2007). Another possible explanation is that variability among patients may exist on the time course of lesion along the circuitries related to balance functions resulting in large variability on subjective balance deficit. These findings support the views that routine screening is needed among the population with risk for parkinsonism.

About half the population of PD patients reported balance deficit in a pathological range (DHI score above six). This result shows the relevance of subjective measurement to screen for parkinsonism (Leddy et al., 2011) and also the need of complementary objective measurements for those unaware of

TABLE 2 | Percent of variability explained by the first 10 components.


*The first 4 components, cumulated, explained 72.26 percent of the total variability. This table shows the percents of variability explained by each component (center column) and the cumulative variance explained by the first 10 components (right column).*

their deficits (van der Kooij et al., 2007; Kahle and Highsmith, 2014). Principal component analyses shows that for the most discriminating component, objective measurements exhibit stronger weigh than the DHI and reinforce previous investigation suggesting that subjective and objective measurement are rather complementary in the appreciation of balance deficits (van der Kooij et al., 2007; Rossi-Izquierdo et al., 2015).

The use of paradigms with sensorial deprivation appeared as an important point of our system as shown by the weight of some features calculated on the center of pressure in conditions with sensorial deprivations. In addition, a small subpopulation of PD participants show reduced functional balance only when proprioceptive input is unreliable, which is compounded when visual input is eliminated. Healthy controls did not show the same difficulty maintaining balance on a compliant surface, with and without visual input. The most discrimant component showed positive relationship between self-reported symptoms, dispersion and high frequency; these features were opposed to duration of standing, irregularity, and slow oscillation. This finding suggests that PD patients and healthy subjects differ in their strategy to compensate sensorial deprivation. The compensatory mechanisms underlying these findings may be major contributors to patient unawareness on their individual balance deficits and point on the importance to "bypass" these compensatory mechanisms when testing balance functions in this population. In addition, this finding shows that vestibular dysfunction is likely a relevant feature to test screen subject with high risk for parkinsonism. In regard to Braak hypothesis on the predominance of peripheral lesions in the early stage of parkinsonism (Braak et al., 1995, 1996, 2002, 2005; Braak and Braak, 2000; Del Tredici et al., 2002; Burke et al., 2008; Fujishiro et al., 2008), the vestibular organ and its connection to central balance circuitry need to be further investigated as a putative hallmark for early diagnosis.

The multivariate analyses on subjective measurement of balance deficit and on objective measurement of center of pressure during a 4 paradigm tests allowed discrimination between PD patients (vs. healthy control) with a selectivity of 0.67

TABLE 3 | This table shows the coefficients of the features for the first three components (component 1, component 2, component 3).


*(Continued)*

#### TABLE 3 | Continued


*Features processed from the standing test during the four paradigms are indicated in the left column and for each component. The abbreviations are detailed in* Table 1*. SRS indicates the scoring to the DHI. The coefficient attributed to each feature is indicated in the right column and for each component. For each component, the data are ordered based on the signed value of the coefficients (from the largest positive value to the largest negative value).*

and a specificity of 0.71. Inarguably, these statistical measures show the relevance of this testing strategy for screening purposes but not for diagnosis. The low duration of the testing (about 10 min), the automatization of the analyses (that reduces need of trained staff) and low cost of equipment highlight the costeffectiveness of such strategy for screen testing patients with parkinsonism. Such screening platform are good putative to meet the need for broad screen testing for PD in both private or public practice and especially in areas with social-economical constraint.

#### Limitation to the Study

While some studies have reported that game industry boards can adequately estimate the center of pressure (Clark et al., 2010; Holmes et al., 2013; Huurnink et al., 2013; Goble et al., 2014), others have reported some limitations (Wikstrom, 2012). Our protocol may certainly benefit from being interfaced with equipment that are technologically more advanced to monitor balance (Rahmatullah et al., 2015). However, the fact that this material is already broadly used in recreational activity facilitate its implementation in clinical environment and especially those with limited resources. For example, game industry boards have already gain a lot of attention in physical therapy (Goble et al., 2014; Harris et al., 2015) for training purposes; some of these

#### REFERENCES

Becker-Bense, S., Wittmann, C., Bloem, B., and Dieterich, M. (2015). P32. Prevalence of parkinson symptoms among patients with peripheral vestibular disorders. Clin. Neurophysiol. 126, e112–e113. doi: 10.1016/j.clinph.2015.04.170

training software's are now available for personal use at home (Esculier et al., 2012). Shortfalls of using balance board from game industry may include some limitations in the selectivity and sensitivity of the screening test and, certainly, some limitation in the utilization of the data collected for research purposes. Alternatives to the use of professional balance board may exist and include the implantation of digital filters to reduce the instability of sensors, the identification of features more robust to the limitation of the equipment's and the use of machine learning to improve the strategy of discrimination.

#### CONCLUSION

Components analysis using features in the time, frequency, and non-linear domains from the center of pressure of subjects measured on a game industry board allowed discrimination of a cohort of PD patients from healthy subjects with a selectivity of 0.67 and a specificity of 0.71. The obsergy between oscillation and complexity contributes to the distinction between parkinisonian and healthy subjects. Regarding parkinsonism, the relationship between features in the frequency domain and features in non-linear domain remains to be further investigated especially in the context of sensorial deprivation. This approach using low cost balance system, simple paradigms and automated multi-component analyses is a promising testing strategy to meet the needs of broader and earlier screening for parkinsonism especially in communities with limited access to healthcare.

#### AUTHOR CONTRIBUTIONS

All authors listed, have made substantial, direct and intellectual contribution to the work, and approved it for publication.

#### ACKNOWLEDGMENTS

We would like to express our gratitude to the subjects (and family members) who contributed to this study. This study was supported by the departments of Neurology, Neurosurgery, Speech Pathology and Audiology, and Physical Therapy of the University of South Alabama. Dr. OD was partially supported by the National Institute for Physiological Sciences (Okazaki–Japan) during this investigation.

#### SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: http://journal.frontiersin.org/article/10.3389/fnhum. 2016.00431

Bekkers, E. M., Dockx, K., Heremans, E., Vercruysse, S., Verschueren, S. M., Mirelman, A., et al. (2014). The contribution of proprioceptive information to postural control in elderly and patients with Parkinson's disease with a history of falls. Front. Hum. Neurosci. 8:939. doi: 10.3389/fnhum.2014.00939

Bennett, D. A., Beckett, L. A., Murray, A. M., Shannon, K. M., Goetz, C. G., Pilgrim, D. M., et al. (1996). Prevalence of parkinsonian signs and associated mortality in a community population of older people. N. Engl. J. Med. 334, 71–76. doi: 10.1056/NEJM199601113340202


Woolf, N. J., and Butcher, L. L. (1986). Cholinergic systems in the rat brain: III. Projections from the pontomesencephalic tegmentum to the thalamus, tectum, basal ganglia, and basal forebrain. Brain Res. Bull. 16, 603–637. doi: 10.1016/0361-9230(86)90134-6

**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2016 Darbin, Gubler, Naritoku, Dees, Martino and Adams. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# BrainCycles: Experimental Setup for the Combined Measurement of Cortical and Subcortical Activity in Parkinson's Disease Patients during Cycling

Maciej Gratkowski <sup>1</sup> \*, Lena Storzer <sup>2</sup> , Markus Butz <sup>2</sup> , Alfons Schnitzler <sup>2</sup> , Dietmar Saupe<sup>1</sup> and Sarang S. Dalal 3, <sup>4</sup>

<sup>1</sup> Department of Computer and Information Science, University of Konstanz, Konstanz, Germany, <sup>2</sup> Institute of Clinical Neuroscience and Medical Psychology, Medical Faculty, Heinrich Heine University Düsseldorf, Düsseldorf, Germany, <sup>3</sup> Center of Functionally Integrative Neuroscience, Department of Clinical Medicine, Aarhus University, Aarhus, Denmark, <sup>4</sup> Zukunftskolleg and Department of Psychology, University of Konstanz, Konstanz, Germany

Recently, it has been demonstrated that bicycling ability remains surprisingly preserved in Parkinson's disease (PD) patients who suffer from freezing of gait. Cycling has been also proposed as a therapeutic means of treating PD symptoms, with some preliminary success. The neural mechanisms behind these phenomena are however not yet understood. One of the reasons is that the investigations of neuronal activity during pedaling have been up to now limited to PET and fMRI studies, which restrict the temporal resolution of analysis, and to scalp EEG focused on cortical activation. However, deeper brain structures like the basal ganglia are also associated with control of voluntary motor movements like cycling and are affected by PD. Deep brain stimulation (DBS) electrodes implanted for therapy in PD patients provide rare and unique access to directly record basal ganglia activity with a very high temporal resolution. In this paper we present an experimental setup allowing combined investigation of basal ganglia local field potentials (LFPs) and scalp EEG underlying bicycling in PD patients. The main part of the setup is a bike simulator consisting of a classic Dutch-style bicycle frame mounted on a commercially available ergometer. The pedal resistance is controllable in real-time by custom software and the pedal position is continuously tracked by custom Arduino-based electronics using optical and magnetic sensors. A portable bioamplifier records the pedal position signal, the angle of the knee, and the foot pressure together with EEG, EMG, and basal ganglia LFPs. A handlebar-mounted display provides additional information for patients riding the bike simulator, including the current and target pedaling rate. In order to demonstrate the utility of the setup, example data from pilot recordings are shown. The presented experimental setup provides means to directly record basal ganglia activity not only during cycling but also during other movement tasks in patients who have undergone DBS treatment. Thus, it can facilitate studies comparing bicycling and walking, to elucidate why PD patients often retain the ability to bicycle despite severe freezing of gait. Moreover it can help clarifying the mechanism through which cycling may have therapeutic benefits.

#### Edited by:

Daniela S. Andres, ETH Zurich, Switzerland

#### Reviewed by:

Johanna Wagner, University of California, San Diego, USA Fernando Salvucci, Universidad Nacional de San Martin, Argentina

> \*Correspondence: Maciej Gratkowski maciej.gratkowski@gmail.com

Received: 01 August 2016 Accepted: 22 December 2016 Published: 10 January 2017

#### Citation:

Gratkowski M, Storzer L, Butz M, Schnitzler A, Saupe D and Dalal SS (2017) BrainCycles: Experimental Setup for the Combined Measurement of Cortical and Subcortical Activity in Parkinson's Disease Patients during Cycling. Front. Hum. Neurosci. 10:685. doi: 10.3389/fnhum.2016.00685

Keywords: DBS, LFPs, EEG, cycling, ergometer, freezing of gait, Parkinson's disease

# 1. INTRODUCTION

Recently, it has been demonstrated that bicycling ability remains surprisingly preserved in Parkinson's disease (PD) patients. In the first report on this phenomenon (Snijders and Bloem, 2010), a video of a 58-year-old man with a 10-year history of idiopathic PD suffering from severe freezing of gait (FOG) is presented. The patient had difficulties initiating gait, which resulted in forward festination and eventually in a fall to the ground. He was able to perform only a few steps when provided with a visual cue and the axial turning was not possible at all. However, the patient had no apparent problems riding and controlling a bike. Immediately after hopping off the bike the FOG episode recurred. Followup reports by these investigators showed that most PD patients, with and without FOG, maintain the ability to bicycle despite severe walking deficits (Snijders et al., 2011, 2012). It has been also demonstrated that the loss of bicycling ability early in the progression of disease strongly supports a diagnosis of atypical parkinsonism rather than PD (Aerts et al., 2011).

It has long been established that PD patients have abnormal basal ganglia function. As the basal ganglia are thought to be critical for all types of locomotion, the observation that bicycling and walking are differentially impacted in FOG is surprising. Snijders et al. (2011) propose possible explanations for this phenomenon: The bicycle's rotating pedals may act as an external pacing cue for the legs. Alternatively, gait and other activities like cycling, which involve moving the legs, might be differentially affected in PD. One may also speculate that the special conditions of bicycling, e.g., continuous resistance and angular momentum of the pedals, may provide feedback that is substantially different from walking. Bicycling has also been recently promoted as a viable therapy for PD (Mohammadi-Abdar et al., 2016), with evidence emerging that it may stimulate improvements in motor control (Ridgel et al., 2009, 2015) and cognitive performance (Alberts et al., 2011; Ridgel et al., 2011), as well as reduce severity of tremor, bradykinesia (Ridgel et al., 2012) and of orthostatic hypotension (Ridgel et al., 2016).

The neural mechanisms behind these phenomena are however not yet understood. One of the reasons is that investigation of neuronal activity during pedaling has been limited up to now to functional imaging and scalp EEG studies. Functional imaging restricts the temporal resolution of analysis, and scalp EEG is focused on cortical activation. Christensen et al. (2000) examined bicycling movements with PET in healthy subjects, observing activation of many typical motor structures (primary motor cortex, supplementary motor area, cerebellum), but notably not basal ganglia. Fukuyama et al. (1997) used SPECT to determine involvement of the basal ganglia in walking in healthy subjects, along with primary motor cortex, supplementary motor area, and cerebellum. SPECT, however, reflects the summation of all brain activity over several minutes and can therefore not reveal oscillatory activity or connectivity patterns. A recent study in dystonia patients measured local field potentials (LFPs) from the basal ganglia while they walked on a treadmill, and found increases in theta (4–8 Hz), alpha (8–12 Hz), and gamma (60– 90 Hz) power compared to rest, while beta (15–25 Hz) power was markedly reduced (Singh et al., 2011). Importantly, no power differences were noted between sitting and standing positions, reducing the likelihood that the seated configuration of bicycling could play a significant role in the context of the bicycling ability phenomena in PD patients. The neural activity during walking has been however mostly investigated using scalp EEG in healthy subjects. Sipp et al. (2013) found increased power in the theta band in anterior cingulate, anterior parietal, superior dorsolateral-prefrontal, and medial sensorimotor cortex as well as decreased beta power in sensorimotor cortex during walking on a balance beam compared with treadmill walking. Wagner et al. (2014) found that mu, beta, and lower gamma rhythms in premotor and parietal cortices are suppressed during conditions that require an adaptation of steps in response to visual input. In another study (Wagner et al., 2016) they found two distinct beta band networks active during gait adaptation to shifts in the tempo of an auditory pacing cue. Mean beta band power was suppressed in central midline and parietal regions and increased in medial prefrontal and dorsolateral prefrontal cortex. The beta suppression may be related to initiation and execution of movement, while the prefrontal beta increase to cognitive top-down control. Seeber et al. (2014) found that during active walking the upper mu (10–12 Hz) and beta (18–30 Hz) oscillations were suppressed compared to upright standing and the significant beta ERD activity was located focally in central sensorimotor areas. They also found that low gamma (24–40 Hz) amplitudes were modulated related to the gait phase. The gait phase dependent gamma modulation may be linked to sensorimotor processing or integration, while the decrease of beta may reflect the suppression of an inhibitory network that enables voluntary movement. In Seeber et al. (2015) they reported increased high gamma (60–80 Hz) amplitudes during human upright walking as compared to standing. The high gamma activity was located focally in central sensorimotor areas and was increased during the gait cycle, which may facilitate motor processing.

The first scalp EEG study of cortical activity as a function of instantaneous pedaling (Jain et al., 2013) reported that beta power over the motor cortex was significantly reduced in active pedaling as opposed to passive pedaling. Furthermore, they demonstrated a relationship between EEG power and EMG power of various leg muscles as a function of pedal position. Similar results were presented by Wagner et al. (2012) in a study of cortical activity related to lower limb movements in robot assisted gait. Power in the mu and beta bands over central midline areas was significantly reduced during active compared to passive walking and the decrease was dependent on gait cycle phases.

Recently, it has been shown using scalp EEG that bicycling relative to walking has a stronger sustained cortical activation and less demanding cortical motor control within the movement cycle (Storzer et al., 2016). This is probably due to the fact that walking demands more phase-dependent sensory processing and motor planning, because each leg is independent in altering stance and swing movement phases. In bicycling, pedals are locked to each other, imposing continuous movement of both legs.

No study to date has recorded deep brain activity during bicycle pedaling. Thus, there is a need for further investigation of cortical and deep brain structures to understand both the mechanism through which cycling ability is preserved in PD patients and the mechanism through which cycling may have therapeutic benefits for them. Electrodes implanted for deep brain stimulation (DBS) therapy in PD patients provide the unique chance to directly record basal ganglia activity.

DBS is an established therapeutic strategy that, for PD patients, involves neurosurgical implantation of electrodes directly in the basal ganglia to allow stimulation with electric current, somewhat analogous to a cardiac pacemaker. If externalized, the same electrodes can be used to also record LFPs, providing an opportunity to measure basal ganglia activity during motor and cognitive behavior.

In this paper, we present an experimental setup that we call BrainCycles. It allows combined investigation of basal ganglia LFPs and scalp EEG in conjunction with physiological and performance parameters during bicycling tasks in patients with implanted DBS electrodes. Furthermore, example data from pilot recordings with PD patients are presented.

## 2. MATERIALS AND METHODS

# 2.1. Experimental Setup Overview

The BrainCycles setup is a modified version of the Powerbike simulator, which was developed for data acquisition, analysis, and visualization of performance parameters in endurance cycling (Dahmen et al., 2011). The full Powerbike software suite incorporates cyclist and bicycle management, synchronized videos of cycling routes, an electronic gear shifter, and the recording and visualization of various performance parameters during a ride like speed, cadence, power, heart rate, and height profile. For the present experimental setup, the Powerbike's capabilities to record bicycling performance parameters as well as to actively control pedal resistance are used. Additionally, the setup was extended with a device for real-time measurement of pedal position. Custom software was developed for the control of the acquisition of the EEG, EMG and other physiological parameters important for experiment protocols involving PD patients. The main hardware component of the setup is a Cyclus2 ergometer (RBM elektronik-automation GmbH, Germany) with an eddy current brake. The brake force can be controlled through a serial port at a rate of 20 Hz using a custom-made software interface. A classic Dutch-style bike frame is mounted on the ergometer brake. The frame, saddle and handlebar were chosen taking the special needs of PD patients into consideration. They ensure that the patients are able to easily get on and get off the bike simulator and have a comfortable sitting position (see **Figure 1**).

An Arduino Due board (Arduino LLC, USA) controls the sensors that assess the crank phase angle and a TFT-display mounted on the handlebar. The Due board includes an Atmel SAM3X8E ARM Cortex-M3 microcontroller and features 54 digital input/output pins, 12 analog inputs, 4 hardware serial ports, an 84 MHz clock and 2 digital-to-analog converters (DACs). A vast number of extension boards (shields) and libraries for interfacing to a wide range of hardware is available

FIGURE 1 | The BrainCycles experimental setup. (Left) Classic Dutch frame and EEG amplifier mounted on the Cyclus2 ergometer. (Right) Handlebar display presenting information about the current and the desired target cadence in rpm.

for the Arduino platform. Thus, the board is especially suitable for easy and fast prototyping.

In this setup a TFT/SD shield (SainSmart, USA) is used to interface with a 3.2 TFT LCD touch-display (SainSmart, USA). The display is mounted on the handlebar and is used to present information and feedback for participants during experiments. For example, the display can show current and desired target cadence in revolutions per minute (rpm). If the target cadence is not being met, a colored arrow is shown, signaling the patient to pedal faster or slower. The color (red or yellow) of the arrow indicates the magnitude of deviation from the target cadence, with the thresholds adjustable by the experimenter.

The data acquisition in the BrainCycles setup is managed with the help of the OpenBCI software framework (Durka et al., 2012). OpenBCI is an open source software modular framework for brain-computer interfaces. The architecture of the framework is based on a centralized modular approach, where different modules communicate through a central multiplexer. The framework includes a module for online communication with signal acquisition hardware and a graphical module for signal review. The OpenBCI's tags manager module can be used to annotate the electrophysiological data with events like FOG episodes or start/stop signals, which are used to instruct participants to start or stop pedaling. The open source character of the framework enables the integration of cycling performance measures within the data stream. For example, twice per pedal revolution, data such as current brake force and cadence are saved as tags in the data stream. The analysis module enables the realization of feedback loops, e.g., in order to control pedal brake force depending on the parameters of electrophysiological signals like EEG or EMG.

# 2.2. Data Acquisition

All electrophysiological data is acquired by a 32-channel TMSi Porti amplifier (TMSi, Enschede, The Netherlands). This lightweight, compact amplifier is battery-powered and can therefore be worn by the patient on a belt and used for recording signals "untethered" for comparison of walking and pedaling conditions. It has a maximum sampling rate of 2048 Hz, and, unlike most other portable devices, is therefore capable of

capturing high frequency oscillations (100–500 Hz) particularly relevant for basal ganglia pathology in PD (Özkurt et al., 2011; Hirschmann et al., 2016). Additionally, it has several auxiliary channels, which can be used for the recording of performance data from the ergometer and other physical measurements. The amplifier provides 24 unipolar, 4 bipolar as well as 4 auxiliary channels. The 32-channels can be used to record from basal ganglia implants, which have 4–8 electrode contacts each, typically placed bilaterally for a total of 8–16 intracerebral channels. Scalp EEG can be recorded with an electrode montage optimized for capturing sensorimotor cortex activity (e.g., Fp1, Fz, Cz, C3, C4, and reference), along with bilateral EMG of the tibialis anterior (TA), rectus femoris (RF), and biceps femoris (BF) leg muscles. EMG and scalp EEG are recorded with actively shielded cables, which reduce sensitivity to motion artifacts and are therefore particularly suitable for recording during locomotion tasks. Furthermore, respiration rate, ECG, knee flexion and foot pressure can also be monitored. TMSi electronic goniometers and footswitches (TMSi, Enschede, The Netherlands) were successfully tested with the setup.

# 2.3. Measurement of LFPs

DBS treatment of PD involves the implantation of electrodes for deep brain stimulation in the subthalamic nucleus (STN) or the globus pallidus interna (GPi). At many DBS clinics, the electrodes are implanted in a first step, and the stimulator

device a few days later in a second step, allowing the chance to record LFPs from the externalized electrodes. Light physical activity is usually manageable for PD patients shortly after the implantation of electrodes. Thus, it is possible to record LFPs while the PD patient is pedaling a stationary bicycle in a laboratory environment. The LFPs can be recorded unipolarly or bipolarly using selected pairs of DBS electrode contacts. Bipolar measurements or bipolar digital rereferencing should be used in order to reduce contamination from distant sources as well as movement artifacts. Those could degrade true local LFPs, bias power spectral estimates, or influence coherence and correlation estimations between different brain regions (Whitmore and Lin, 2016). We have successfully tested Boston Scientific Vercise and St. Jude Medical Infinity DBS systems with the BrainCycles experimental setup. Both of them offer 8 contacts per electrode. Using directional leads they allow for stimulation and measurement in directions orthogonal to the lead trajectory.

## 2.4. Measurement of Pedal Position

One of the requirements of the BrainCycles setup is to provide a means to investigate possible relationships between EEG power and EMG power of various leg muscles as a function of pedal position. A custom made electronic circuit encodes the current crank position as an analog signal proportional to the angular position of the pedals. This analog signal is acquired together with electrophysiological signals using the auxiliary input of the EEG amplifier. An important advantage of this approach is that this eliminates the need to coregister data prior to analysis. Furthermore, the knowledge of the real-time pedal position opens new possibilities for experimental protocols with tasks or control parameters depending on the instantaneous position of the pedals. For example, the pedaling resistance at the push-down phase of the pedal motion could be independently set for each leg.

The main part of the rotary encoder are two forked light barriers consisting of infrared LEDs and phototransistors. The barriers are mounted next to the front chainring so that the teeth of the chainring pass through the light barriers, and the distance between the light axes of the barriers is smaller than the width of the teeth. The first light barrier acts as a key that changes the state when a tooth first breaks and then releases the light barrier. Thus, an inversed impulse is created every time a tooth passes through the barrier and temporarily interrupts the light. The falling edge of the impulse invokes a hardware interrupt in the microcontroller. The microcontroller counts each such event. The increase of the angular position of the cranks can be derived from this count and the number of teeth on the chainring. The orientation of the crank rotation can be determined with the help of the second barrier. If the pedaling is forward, the second barrier will be open at the time the tooth is entering the first barrier. If the pedaling is backwards, the second barrier will be closed.

The angular resolution of such encoder is directly proportional to the number of teeth on the chainring, because the teeth on the sprocket are equidistantly distributed. In our particular setup, the chainring has 42 teeth and thus the angular resolution is 360◦ /42 = 8.57◦ . In order to know the absolute position of the pedals a reset signal at position 0◦ is needed. The reset signal is provided by mounting a magnet on the crank and a magnetically actuated reed switch at position 0◦ , which in our case is the top position of the right pedal. The absolute position of the pedals is then converted to an analog signal by the DAC integrated on the Arduino Due board and connected to the auxiliary input of the EEG amplifier. The DAC output and the auxiliary input of the Porti amplifier are galvanically separated with an optocoupler for an additional layer of patient safety and to prevent ground loop interference.

# 2.5. Example Data

In order to demonstrate the utility of the setup, sample data from pilot recordings is presented. The shown data were recorded from a 56-year-old patient, who was diagnosed with PD in 2011 with motor symptoms predominantly on the right side. The patient had opted for bilateral DBS therapy (Boston Scientific Vercise system with 8 contacts per STN), and was recorded 1 day after the implantation of DBS electrodes while off any dopaminergic

slope profile of the simulated track. The Powerbike simulation software could serve as a virtual reality environment for the BrainCycles setup, enabling studies with

medication. Custom-made connectors were used to connect the DBS electrodes with the EEG amplifier.

visual feedback or simulating real-world bicycle navigation in patients with movement disorders.

The recording protocol consisted of 30 intermittent rest and pedaling periods guided by acoustic signals. The patient was provided with start and stop signals in form of a tone of 500 ms duration and frequency 1000 Hz in case of the start signal and 1500 Hz for the end signal. The duration of the rest and pedaling phases were around 10 seconds. The patient was instructed to pedal at a relaxed pace of his own preferred cadence. The brake force of the ergometer was constantly kept at a low level of 30 N. Individual gel-based electrodes were placed at Pz, Oz, and P3 to capture signals from somatomotor areas. The bandage left after the implantation of electrodes did not allow placement of electrodes more anterior. Subsequently, the recorded signals were digitally rereferenced to a bipolar montage of Pz-P3 and Pz-Oz. A water-based ground electrode (TMSi, Netherlands) integrated in a wristband was used. Bipolar surface EMG activity of TA, BF, and RF was recorded bilaterally using disposable Ag/AgCl electrodes. Electrodes were placed 2 cm apart on the belly of each muscle. EMG, scalp EEG and LFPs were recorded with actively shielded cables compensating for movement artifacts.

In previous studies, modulation of cortical oscillatory activity in the beta band during transitions between rest and pedaling conditions was observed (Storzer et al., 2016). Similarly, Wagner et al. (2012) and Seeber et al. (2014) showed a beta band decrease during walking compared to rest. Thus, we hypothesized that this modulation should also be seen in deeper brain regions. The recorded data were processed in Matlab (Mathworks, Inc., Natick, MA) using the FieldTrip open source Toolbox (Oostenveld et al., 2011). The bipolar LFP channel with the highest resting beta activity was chosen for further analysis. LFPs and EEG data were bandpass filtered with cutoff frequencies of 13 and 33 Hz. Next, the filtered data were Hilbert-transformed in order to compute their envelopes. The envelope samples from cycling and rest periods were statistically compared using Mann-Whitney U-test for equal medians (ranksum function, Matlab). Furthermore, the envelope signals were averaged taking 5 s of data before and 10 s of data after each movement initiation. Both ongoing and averaged envelope signals were filtered by a moving average filter of length 2500 samples, which represents around 1.2 s of data.

#### 3. RESULTS

**Figure 2** presents 110 seconds of continuously recorded data including the pedal angle (**Figure 2A**), the LFPs from the left STN (**Figures 2B,C**), the envelope of the beta band of the EEG Pz-Oz signal (**Figure 2D**) and the muscle activity of TA and BF (**Figures 2E,F**). Beta band activity in the STN decreased upon initiation of pedaling and rebounded after termination. This can already be seen in the filtered LPF data and its envelope (see **Figures 2B,C**). The beta increase is visible upon return to resting, when muscle activity is minimal. Thus, it is unlikely that the increase in the beta band is due to movement artifacts. The beta modulation can also be observed from the scalp EEG data (Pz-Oz) when averaged relative to movement initiation (see **Figure 3**). The Mann-Whitney U-test showed that the difference in the beta band between cycling and resting condition, both in EEG and LFPs is statistically significant (p < 0.001).

# 4. DISCUSSION

In this paper we presented an experimental setup for combined recording of cortical and deep brain oscillations during pedaling. We showed that direct recordings from the STN during bicycling are possible in a laboratory environment and that combining these signals with data from an ergometer is feasible, safe, and not prone to excessive noise or artifacts. Visual inspection suggests that the recorded signals have a very high SNR. The reduction of the beta power in the STN during bicycling is visible even in the raw data. A similar effect could be observed in the surface EEG. However, for the latter the change of the amplitude is much smaller and only visible after averaging the EEG locked to the movement initiation. We did not observe strong movement related artifacts in our LFPs and EEG data. However, data recorded at higher cadences and/or at higher resistive forces could exhibit strong rhytmic, movement related artifacts. Such artifacts could be minimized using a template regression procedure applied previously to EEG data recorded during walking and running (Gwin et al., 2010).

The presented experimental setup provides means to directly record basal ganglia activity not only during cycling but also during other movement tasks in patients who have elected DBS treatment. Thus, it can facilitate studies comparing bicycling and walking, to elucidate why PD patients often retain the ability to bicycle despite severe FOG. Moreover, it can help to clarify the mechanism through which cycling may have therapeutic benefits.

At present, the experiments can be performed between 1 and 6 days after electrode implantation, while the electrode cable is externalized and therefore allows recording of LFPs from the implanted structures. The next generation of DBS systems will potentially allow LFPs recording by the same unit that performs stimulation and storing the data internally for subsequent retrieval via wireless data transfer (Neumann et al., 2016). This will allow recording of data after electrode cables have been internalized, expanding opportunities to record LFPs, and eventually allowing recordings after patients have recovered from the surgical procedure and have become accustomed to the implant. Thus, task-related modulations of oscillatory activity and coupling could be monitored during a cycling training regimen and correlated to therapeutic outcomes. Importantly, several months after implantation, the patients would be able to participate in more strenuous activity, such as the forced pedaling regimens described in Alberts et al. (2011) and Ridgel et al. (2012).

The BrainCycle setup is linkable to a virtual reality environment provided by the Powerbike software, that enables simulation of bicycling on a given route, with the speed of pedaling controlling the speed of navigation and real-time modulation of brake force to simulate incline (see **Figure 4**). This feature can enable several new lines of research involving visual feedback or simulating real-world bicycle navigation in patients with movement disorders. For instance, the BrainCycles setup could be used to investigate visuomotor processing in PD patients. It has been hypothesized that PD is associated with a visuomotor disturbance and PD patients produce exaggerated responses to visual information (Cowie et al., 2010). For example, many patients slow down dramatically or even freeze when attempting to approach narrow doorways. Similarly, FOG episodes can be elicited by unexpectedly appearing obstacles (Delval et al., 2010). The Powerbike's ability to play videos synchronized to pedaling could be used to investigate how such sudden visual cues and obstacles are processed during cycling. Visuomotor processing during walking on a treadmill and cycling on an ergometer could be also compared in a lab environment.

# ETHICS STATEMENT

The study protocol was approved by the local ethics committee of the Medical Faculty of the Heinrich Heine University Düsseldorf (study number: 4294). All participants gave their prior written informed consent.

# AUTHOR CONTRIBUTIONS

All authors contributed to the design of the experiment setup and revised the manuscript. MG constructed and programmed the BrainCycles setup. LS acquired the data. MG, SSD, and LS were involved in data analysis and interpretation. All authors approved the final version of the manuscript, agreed to be accountable for all aspects of the work and qualify for authorship.

# FUNDING

This work was generously supported by a grant from the Jacques and Gloria Gossweiler Foundation. Further support was received from the Zukunftskolleg of the University of Konstanz [SSD], Heinrich Heine University Strategic Research Funds [MB], the Open Access Publication Fund of the University of Konstanz, and the research commission of the medical faculty of the Heinrich Heine University [9772562 to MB]. LS was supported by a travel grant from the Boehringer Ingelheim Foundation (BIF).

# ACKNOWLEDGMENTS

We thank Nitin Sharma (University of Konstanz) for assistance with constructing and programming the experimental setup.

# REFERENCES


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2017 Gratkowski, Storzer, Butz, Schnitzler, Saupe and Dalal. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Subcortical Volumes Differ in Parkinson's Disease Motor Subtypes: New Insights into the Pathophysiology of Disparate Symptoms

Keren Rosenberg-Katz 1,2 , Talia Herman<sup>1</sup> , Yael Jacob1,2,3 , Efrat Kliper 2,4 , Nir Giladi 1,3,4,5 and Jeffery M. Hausdorff 1,3,6 \*

<sup>1</sup> Center for the Study of Movement, Cognition and Mobility, Neurological Institute, Tel Aviv Sourasky Medical Center, Tel Aviv, Israel, <sup>2</sup> Functional Brain Center, Wohl Institute for Advanced Imaging, Tel Aviv Sourasky Medical Center, Tel Aviv, Israel, <sup>3</sup> Sagol School of Neuroscience, Tel Aviv University, Tel Aviv, Israel, <sup>4</sup> Neurological Institute, Tel Aviv Medical Center, Tel Aviv, Israel, <sup>5</sup> Department of Neurology, Sackler Faculty of Medicine, Tel Aviv University, Tel Aviv, Israel, <sup>6</sup> Department of Physical Therapy, Sackler Faculty of Medicine, Tel Aviv University, Tel Aviv, Israel

Objectives: Patients with Parkinson's disease (PD) can be classified, based on their motor symptoms into the Postural Instability Gait Difficulty (PIGD) subtype or the Tremor Dominant (TD) subtype. Gray matter changes between the subtypes have been reported using whole brain Voxel-Based Morphometry (VBM), however, the evaluation of subcortical gray matter volumetric differences between these subtypes using automated volumetric analysis has only been studied in relatively small sample sizes and needs further study to confirm that the negative findings were not due to the sample size. Therefore, we aimed to evaluate volumetric changes in subcortical regions and their association with PD motor subtypes.

Methods: Automated volumetric magnetic resonance imaging (MRI) analysis quantified the subcortical gray matter volumes of patients with PD in the PIGD subtype (n = 30), in the TD subtype (n = 30), and in 28 healthy controls (HCs).

Results: Significantly lower amygdala and globus pallidus gray matter volume was detected in the PIGD, as compared to the TD subtype, with a trend for an association between globus pallidus degeneration and higher (worse) PIGD scores. Furthermore, among all the patients with PD, higher hippocampal volumes were correlated with a higher (better) dual tasking gait speed (r = 0.30, p < 0.002) and with a higher global cognitive score (r = 0.36, p < 0.0001). Lower putamen volume was correlated with a higher (worse) freezing of gait score (r = −0.28, p < 0.004), an episodic symptom which is common among the PIGD subtype. As expected, differences detected between HCs and patients in the PD subgroups included regions within the amygdala and the dorsal striatum but not the ventral striatum, a brain region that is generally considered to be more preserved in PD.

Conclusions: The disparate patterns of subcortical degeneration can explain some of the differences in symptoms between the PD subtypes such as gait disturbances

#### Edited by:

Daniela S. Andres, ETH and University Zurich, Switzerland

#### Reviewed by:

Federico Nemmi, Karolinska Institutet, Sweden Miguel Wilken, Fundación para la Lucha contra las Enfermedades Neurológicas de la Infancia (FLENI), Argentina

> \*Correspondence: Jeffery M. Hausdorff jhausdor@tlvmc.gov.il

Received: 09 May 2016 Accepted: 29 June 2016 Published: 11 July 2016

#### Citation:

Rosenberg-Katz K, Herman T, Jacob Y, Kliper E, Giladi N and Hausdorff JM (2016) Subcortical Volumes Differ in Parkinson's Disease Motor Subtypes: New Insights into the Pathophysiology of Disparate Symptoms. Front. Hum. Neurosci. 10:356. doi: 10.3389/fnhum.2016.00356 and cognitive functions. These findings may, in the future, help to inform a personalized therapeutic approach.

Keywords: Parkinson's disease, tremor, postural instability gait difficulty, volumetric MRI, basal ganglia, globus pallidus, imaging, gait

#### INTRODUCTION

Patients with Parkinson's disease (PD) can be classified based on their motor symptoms into the Tremor Dominant (TD) or the Postural Instability Gait Difficulty (PIGD) subtype, depending on whether tremor or balance and gait disturbances are the most pronounced symptoms (Jankovic et al., 1990). Patients with the PIGD subtype have an increased risk for developing cognitive deterioration (Lewis et al., 2005; Burn et al., 2006, 2012; Herman et al., 2015), dementia (Aarsland et al., 2003, 2009; Williams-Gray et al., 2007) and mood disturbances such as depression or anxiety (Burn et al., 2012). Thus, in addition to the differences in PD motor symptoms, behavioral evidence suggests that the neurological substrate differs in the PIGD and TD subtypes.

These motor, cognitive, and behavioral differences among the PD subtypes likely reflect alternations in brain structure. Structural changes detected using Magnetic Resonance Imaging (MRI) have been used as biological markers of neurodegeneration in PD (Whitwell and Josephs, 2007; Pereira et al., 2012). Using Voxel-Based Morphometry (VBM), we recently reported on widespread gray matter reduction in the PIGD subtype, as compared to the TD subtype, in the frontal, parietal, occipital, and temporal lobes as well as in the parahippocampal gyrus, cerebellum, caudate nucleus and amygdala (Rosenberg-Katz et al., 2013). While VBM enables us to evaluate whole brain changes, it may suffer from errors that might arise from registration and segmentation procedures, especially in subcortical regions. Automated volumetric analysis is considered to be less sensitive to registration errors and to anatomical variability such as ventricular enlargement which is common in neurological diseases and aging (Khan et al., 2008). Therefore, while this technique also has its limitations, it has been considered to be the gold-standard approach for morphological analysis of MRI data in older adults (Fischl et al., 2002, 2004). Recently, a study using this automated approach found no differences in subcortical brain volumes between the PD subtypes (Nyberg et al., 2015), however, the relatively small number of subjects (12 and 9 for the PIGD and TD groups, respectively) might explain the negative findings. Additional study, using a larger cohort of patients with PD, is needed in order to determine if the previously reported absence of subcortical brain volume differences between the subtypes was due to a small sample size.

In the current study, we used automated volumetric analysis to evaluate differences in subcortical degeneration between the TD (n = 30) and PIGD (n = 30) subtypes. Based on our previous work, we expected to find a reduction in subcortical gray matter volume in the PIGD compared to the TD subtype. Specifically, we hypothesized that the PIGD subtype will have smaller gray matter volumes within the caudate nucleus (an area which is related to cognitive function), within the amygdala (an area which may be involved in the affective symptoms that are more common in this group), and within the globous pallidus (a brain area that might relate to the gait disturbances of the PIGD subtype as part of its role in the sensorimotor and the associative circuit; Tremblay et al., 2015).

#### MATERIALS AND METHODS

One-hundred and ten patients with idiopathic PD and 28 healthy controls (HCs) were recruited for this study. This is a secondary analysis of work designed to compare PD motor subtypes (Herman et al., 2013; Rosenberg-Katz et al., 2013). All patients were diagnosed by a movement disorders specialist as having idiopathic PD (as defined by the UK Brain Bank criteria). Patients and controls were excluded if they had major orthopedic disease, acute illness, history of stroke, a diagnosis of dementia based on DSM-IV criteria or a Mini Mental State Examination score (MMSE) < 24 (Folstein et al., 1983), had a diagnosed psychiatric disorder, or if they underwent brain surgery in the past. For the HCs, exclusion criteria also included any neurological disease.

#### Protocol Outline

All subjects provided informed written consent prior to participating in the study, as approved by the Human Research Ethics Committee of Tel Aviv Sourasky Medical Center. Patients were studied on two separate occasions: the first visit included a neurological and clinical examination. On a separate visit that took place within 2 weeks after the first visit, the participants underwent MRI testing in the ''ON'' medication state.

#### Clinical Evaluation

Patients underwent a clinical assessment that included the Unified Parkinson's Disease Rating Scale (UPDRS). The pull test (item 30 of the UPDRS) was used as a measure of balance and postural control. Usual and dual tasking gait speed (m/s) under single and dual task conditions (i.e., serial 3 subtracting) was determined as a measure of gait difficulties (Herman et al., 2014). These assessments were conducted in the ''OFF'' state after at least 12 h of overnight withdrawal of antiparkinsonian medications. A computerized cognitive battery (NeuroTrax Corp., Modiin, Israel) (Dwolatzky et al., 2003) was used in the ''ON'' state to sample a wide range of cognitive domains and to generate a normalized global cognitive score. To evaluate confidence while walking and fear of falling, the Activities-specific Balance Confidence (ABC) scale (Powell and Myers, 1995) was used. Emotional well-being and depressive

#### TABLE 1 | Demographic characteristics of the study participants.


LED, Levodopa equivalent dosage; UPDRS, Unified Parkinson's Disease Rating Scale; NA, not available.

#### TABLE 2 | Volumetric comparisons in the selected regions of interest (ROIs) between the postural instability gait difficulty (PIGD) subtype, tremor dominant (TD) subtype and HCs.


symptoms were assessed using the 15-item version of the Geriatric Depression Scale (Yesavage et al., 1982). Balance and postural control were evaluated using the Berg Balance Scale (BBS; Berg et al., 1992). Patients also completed the New Freezing of Gait Questionnaire (N-FOGQ; Nieuwboer et al., 2009). The daily levodopa equivalent dosage (LED) was calculated for each patient as previously described (Tomlinson et al., 2010).

#### Classification into PIGD and TD Subtypes

From the 110 patients that were recruited for this study, automated volumetric analysis was conducted on 105 patients (5 patients did not complete the MRI scans). These patients were classified into the PIGD or TD subtypes as previously described (Rosenberg-Katz et al., 2013). Briefly, symptoms were first quantified by summing specific items from the UPDRS to determine PIGD and TD scores that reflect gait and balance difficulties and tremor severity, respectively (Jankovic et al., 1990). Patients were classified to PIGD or TD group based on the ratio between the PIGD and tremor scores. Then, in order to stratify the patients into groups who best represented the two subtypes, with minimal overlap across symptom classes, patients were excluded from the TD group if they had a PIGD score higher than 3 or a tremor score lower than 4. Similarly, patients were excluded from the PIGD group if their PIGD score was lower than 4 or their tremor score was higher than 3 (Rosenberg-Katz et al., 2013). Patients who did not meet the criteria for one of the groups were considered as indeterminate. Thirty PIGD patients, 30 TD patients, and 45 indeterminate patients were identified.

#### MRI Acquisition

All of the MR images were acquired on a 3.0 T (GE) scanner using an 8-channel head coil. A high-resolution T1-weighted brain volume (BRAVO) acquisition was used with the following parameters: repetition time (TR) = 9000 ms, echo time (TE) = 3.6 ms, flip angle (FA) = 90◦ , voxel size = 1 × 1 × 1, matrix = 256 × 256, field of view (FOV) = 250 × 250 mm<sup>2</sup> .

#### Volumetric Analysis

The automated volume-based stream was performed on axial 3D T1-weighted BRAVO images using the FreeSurfer V5.1 image analysis suite, well documented and freely available software<sup>1</sup> (Fischl et al., 2002, 2004).

<sup>1</sup>http://surfer.nmr.mgh.harvard.edu/

Processing included affine transformation of each participant's T1 weighted image into Talairach space, probabilistic segmentation of gray and white matter structures, bias field intensity normalization, and automated labeling of anatomical regions of interest (ROIs) in both cerebral hemispheres (Fischl et al., 2002, 2004). The ROIs derived from this analysis were the same as those examined by Nyberg et al. (2015) and included the thalamus, the caudate nucleus, the putamen, the globus pallidus, the amygdala, the nucleus accumbens, and the hippocampus. While there is some disagreement about whether the hippocampus should be considered as a subcortical region, it was included as it is part of the limbic system and to compare with the work of Nyberg et al. (2015). For each patient, the ROIs contralateral to the more affected side, defined by the side with a higher score on the UPDRS, were examined. For the HCs, and patients with no dominant affected side, the mean value of both hemispheres was used for comparisons.

#### Statistical Analysis

All statistical analyses were two-sided and conducted using the Statistical Package for Social Sciences (Version 20; SPSS Inc., Chicago, IL, USA). Multivariate analysis of covariance (MANCOVA) adjusted for age and total intracranial volume was used to compare ROIs volumes across PIGD, TD subgroups and controls. Similar analyses which also included the indeterminate patients were also performed to evaluate if this group had ROIs volumes in between the values seen in the two subtypes. As these regions were predefined, no correction for multiple comparisons was performed in this analysis. For all correlation analyses, partial correlations were conducted adjusting for age and total intracranial volume. For patients with PD, correlations between gray matter brain volumes and PIGD score, tremor score, gait speed (usual and dual tasking), N-FOGQ, global cognitive score, ABC scale, and Geriatric Depression Scale (GDS) were conducted in the entire sample (n = 105), including the indeterminate group, and were adjusted for age, disease duration and total intracranial volume. Outliers were defined as values larger than 2 interquartile ranges above the 75th percentile or lower than 2 interquartile ranges below the 25th percentile, however, no outliers were detected. Bonferroni corrections for multiple comparisons were also applied for these correlation analyses.

#### RESULTS

#### Patient Characteristics

There were no significant differences between the PIGD and TD groups in their demographic characteristics including age, gender, years of education, UPDRS motor scores, disease duration, and LED (see **Table 1**). As expected, axial motor impairments were more severe in the PIGD group than in the TD group, as measured by the pull test and the BBS. Usual and dual tasking gait speeds were lower in the PIGD subtype. The executive function index from the computerized cognitive battery, Mini Mental Status Exam (MMSE) scores and global

cognitive scores tended to be lower in the PIGD group than in the TD group; however, these group differences were not significant (**Table 1**). Patients with the PIGD subtype also had worse scores on the ABC scale and GDS, respectively.

#### Volumetric Analysis

MANCOVA analysis including the PIGD group, the TD group, and HCs detected a significant GROUP effect within the caudate, putamen, globus pallidus and amygdala. The estimated marginal means adjusted for age and intracranial volume are presented in **Table 2** and **Figure 1**. Post hoc analysis showed that the PIGD patients had lower gray matter volumes in the globus pallidus and amygdala, as compared to the patients with the TD subtype.



n.s, non-significant.



Both the PIGD and TD subtypes had lower caudate nucleus and amygdala volumes, as compared to HCs. The PIGD subtype also had a lower putamen gray matter volume compared to the controls (see **Table 3** and **Figure 1**).

Additional MANCOVA analyses which included the indeterminate group as well as the PIGD subtype, the TD subtype, and HCs showed a similar pattern of results with a significant GROUP effect within the caudate, putamen, and amygdala (see **Tables 4**, **5** and **Figure 2**). While the globus pallidus only showed a small trend for a group effect (p = 0.19), post hoc comparisons still showed a significant difference between the PIGD and TD subtypes (p < 0.04, see **Table 5**) within this region. Interestingly, in the regions showing a significant GROUP effect, the volumes of the indeterminate group were between the volumes of the two subtypes (see **Table 4** and **Figure 2**). Furthermore, the amygdala volume was significantly larger in the indeterminate group than in the PIGD subtype, while no significant difference between the indeterminate group and the two PD subtypes were detected for the other regions (see **Table 5**).

Among all of the patients with PD (n = 105), higher hippocampal volumes were correlated with higher dual tasking gait speed (r = 0.30, p < 0.002) and with higher global cognitive score (r = 0.36, p < 0.0001; see **Figure 3**). Higher (worse) scores on the N-FOGQ were correlated with lower putamen gray matter volumes (r = −0.28, p < 0.004; see **Figure 3**). No significant correlations were detected between gray matter volumes and tremor or the PIGD scores, however, using a more liberal threshold, the PIGD score was mildly associated with lower globus pallidus volume (r = −0.22, p < 0.03; see **Figure 3**), but not when corrected for multiple comparisons.

#### DISCUSSION

In this study, greater amygdala and globus pallidus gray matter loss were detected in the PIGD, as compared to the TD subtype. In addition, the PIGD subtype had significantly higher putamen degeneration than controls. Interestingly, the putamen volumes in the TD subtype were not significantly different than controls. Furthermore, increased putamen degeneration was associated with a higher (worse) freezing of gait score, an episodic symptom which is more associated with the PIGD subtype and likely involves a motor-cognitive failure (Giladi et al., 2007; Fasano et al., 2011; Cohen et al., 2014; Maidan et al., 2015). As might be expected, indeterminate PD patients, who have a mixture of symptoms of both subtypes, had an intermediate volume between the subtypes in regions where a significant difference between the groups was observed.

Patients with PD had lower subcortical volumes within the amygdala and the dorsal striatum (caudate and putamen), as compared to HCs. The dorsal striatum is believed to be especially sensitive to PD, and this atrophy is considered to be a marker for neurodegeneration, as it is shown to be associated with the stages and severity of the disease (Pereira et al., 2012). In contrast, no differences were found within the ventral striatum (nucleus accumbens) which is considered to be more preserved in PD, based on the localized accumulation


of Lewy bodies (Braak et al., 2003). Based on the role of the caudate nucleus in cognition as part of the associative corticostriatal circuit (Alexander et al., 1986; Tremblay et al., 2015), we would expect to find reduced caudate nucleus volumes in patients of the PIGD subtype, as they have a higher risk for developing cognitive impairment (Lewis et al., 2005; Burn et al., 2006, 2012; Herman et al., 2015). Perhaps the role of the caudate nucleus in cognition is also affected by its functional connectivity with other brain regions, as previously demonstrated in patients with PD (Vervoort et al., 2016).

The increased pallidal degeneration in the PIGD subgroup, as compared to the TD subgroup, may partially explain a range of motor and non-motor symptoms common in the PIGD subgroup. Indeed, the globus pallidus is involved in the sensorimotor, associative and limbic corticostriatal circuits (Alexander et al., 1986). These three circuits play a role in many aspects of action planning, starting from motivational information for goal selection through the limbic circuit, continuing with action selection via the associative circuit, and concluding with movement selection and execution by the sensorimotor circuit (Tremblay et al., 2015). The gait disturbances, which are highly representative of the PIGD subtype, can be related to both the impairment of the sensorimotor and the associative circuit.

Amygdala degeneration was detected in both subtypes, as compared to HCs, with greater degeneration in the PIGD subtype, compared to the TD subtype and to the indeterminate group. This is consistent with a previous report that compared PD subjects without dementia to controls (Bouchard et al., 2008) and with our previous VBM analysis (Rosenberg-Katz et al., 2013). The detected changes in amygdala volume can be related to a number of affective (non-motor) symptoms in PD, including depression, apathy, compulsive behavior, and anxiety (Chaudhuri et al., 2006). Depression and anxiety were more severe in the PIGD subtype than in the TD subtype (Burn et al., 2012), which is consistent with our findings of greater level of fear of falling (reflected by the lower balance confidence scores) and higher depression scores in the PIGD subtype. Nonetheless, we could not detect any significant correlations between these measurements and amygdala gray matter volume. Perhaps a latent variable, possibly within the limbic system, mediates those relationships.

The detected correlations between hippocampal volumes with both global cognitive score and dual tasking gait speed are consistent with previously reported associations between lower hippocampus volume and reduced gait speed and step length in non-PD older adults (Callisaya et al., 2013). Similarly, lower hippocampal volume and metabolism (measured using MR spectroscopy) have been associated with poorer stride length in older adults without dementia (Zimmerman et al., 2009). Overall, our findings support the suggested relationships between gait and cognition (Amboni et al., 2013), possibly via the role of the hippocampus in spatial orientation and memory (O'Keefe et al., 1998). The lack of hippocampal differences between the groups is consistent with other volumetric studies in PD, which found lower hippocampus volumes in patients with PD-related dementia as compared to patients with no dementia (Zarei et al., 2013; Xu et al., 2016), while no changes were detected when comparing PD patients with mild cognitive impairment to PD patients with no cognitive impairment, suggesting that the hippocampal atrophy in PD is a gradual progressive process (Xu et al., 2016).

Interestingly, higher globus pallidus volumes were detected in the TD group, as compared to both controls and patients in the PIGD group. This observation is consistent with previous VBM studies which showed that PD tremor and essential tremor are associated not only with atrophy of brain regions but also with cortical enlargement. For example, enlargement of posterior parts of the thalamus was present in PD patients with tremor, as compared to controls (Kassubek et al., 2002; Lin et al., 2013). In our cohort, there was a trend for increased thalamic volume in the tremor subtype, although this was not significant. The direct relationship between the globus pallidus and the thalamus within the direct basal ganglia-thalamocortical circuit may explain the enlargement of both of these regions. This enlargement might reflect a compensatory mechanism in response to damage to the basal ganglia-thalamocortical circuit. Another possibility is that patients in the TD subtype are initially protected from globus pallidus and thalamic degeneration and this is why they have early tremor and not PIGD symptoms.

Indeed, a functional MRI study reported higher functional activation of the globus pallidus in patients with tremor dominant PD, as compared to those without (Prodoehl et al., 2013). The nature of this enlargement, either as a result of neuronal hypertrophy or by a higher density of neurons, can only be determined using post-mortem pathological investigations.

The volumetric differences that were found in the present study between the PD subtypes are not in agreement with a recent study which did not find volumetric changes in these same subcortical regions (Nyberg et al., 2015). Several possibilities might explain this discrepancy. First, our analysis included 30 patients in each group of the PD subtypes while Nyberg et al. (2015) included only nine PIGD and 12 TD patients. Second, we used additional restrictions for the traditional stratification into the PD subtypes (Jankovic et al., 1990), in order to exclude patients with mixed symptoms from the analysis (Rosenberg-Katz et al., 2013). This stricter stratification may enable the detection of subtle changes between the two PD subtypes. Further work is needed to confirm the results of the present study and better understand these differences.

Previous whole-brain VBM analysis on the same cohort (Rosenberg-Katz et al., 2013) detected widespread frontal, parietal, occipital, and temporal gray matter reduction in the PIGD subtype as compared to the TD subtype, as well as changes in the parahippocampal gyrus, cerebellum, caudate nucleus, and amygdala. While the changes in the amygdala were consistent using both types of analysis, we could not detect any group differences within the cerebellum and the caudate nucleus. This discrepancy could be related to the type of analysis that was used. Indeed, it has been suggested that automated volumetric analysis is less affected by anatomical variability, especially in subcortical structures (Fischl et al., 2002, 2004) than VBM.

To conclude, the detected gray matter atrophy in subcortical areas can potentially explain some of the motor, cognitive and affective symptoms that are different among the PD subtypes. These anatomical changes suggest that subcortical degeneration is not distributed similarly across the two PD motor subtypes and support the possibility that different therapeutic approaches should be considered for each of the motor PD subtypes.

#### AUTHOR CONTRIBUTIONS

KR-K, TH, JMH and NG designed the work; and KR-K, YJ, TH and EK analyzed the data. Participants were recruited and images were acquired by KR-K, YJ and TH. KR-K wrote the first draft. All authors revised the final version critically for important intellectual content. All authors approved the final submitted version, and agree to be accountable for its content.

#### REFERENCES


#### FUNDING

This study was supported by the Michael J. Fox Foundation for Parkinson's Research.

#### ACKNOWLEDGMENTS

We are grateful to all the patients who participated in this study. We thank Marina Brozgol for her help in clinical evaluation and Guy Heiman from the Functional Brain Center, Wohl Institute for Advanced Imaging for support in data acquisition.


**Conflict of Interest Statement**: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2016 Rosenberg-Katz, Herman, Jacob, Kliper, Giladi and Hausdorff. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution and reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Can Gait Signatures Provide Quantitative Measures for Aiding Clinical Decision-Making? A Systematic Meta-Analysis of Gait Variability Behavior in Patients with Parkinson's Disease

#### Niklas König<sup>1</sup> , Navrag B. Singh<sup>1</sup> , Christian R. Baumann<sup>2</sup> and William R. Taylor <sup>1</sup> \*

<sup>1</sup> Department of Health Sciences and Technology, Institute for Biomechanics, Swiss Federal Institute of Technology in Zurich (ETHZ), Zürich, Switzerland, <sup>2</sup> Department of Neurology, University Hospital Zürich, University of Zürich, Zürich, Switzerland

#### Edited by:

Olivier Darbin, University of South Alabama, USA

#### Reviewed by:

Annalisa Setti, University College Cork, Ireland Sue Lord, Newcastle University, UK

> \*Correspondence: William R. Taylor taylorb@ethz.ch

Received: 12 April 2016 Accepted: 13 June 2016 Published: 30 June 2016

#### Citation:

König N, Singh NB, Baumann CR and Taylor WR (2016) Can Gait Signatures Provide Quantitative Measures for Aiding Clinical Decision-Making? A Systematic Meta-Analysis of Gait Variability Behavior in Patients with Parkinson's Disease. Front. Hum. Neurosci. 10:319. doi: 10.3389/fnhum.2016.00319 A disturbed, inconsistent walking pattern is a common feature of patients with Parkinson's disease (PwPD). Such extreme variability in both temporal and spatial parameters of gait has been associated with unstable walking and an elevated prevalence of falls. However, despite their ability to discretise healthy from pathological function, normative variability values for key gait parameters are still missing. Furthermore, an understanding of each parameter's response to pathology, as well as the inter-parameter relationships, has received little attention. The aim of this systematic literature review and meta-analysis was therefore to define threshold levels for pathological gait variability as well as to investigate whether all gait parameters are equally perturbed in PwPD. Based on a broader systematic literature search that included 13′195 titles, 34 studies addressed Parkinson's disease, presenting 800 PwPD and 854 healthy subjects. Eight gait parameters were compared, of which six showed increased levels of variability during walking in PwPD. The most commonly reported parameter, coefficient of variation of stride time, revealed an upper threshold of 2.4% to discriminate the two groups. Variability of step width, however, was consistently lower in PwPD compared to healthy subjects, and therefore suggests an explicit sensory motor system control mechanism to prioritize balance during walking. The results provide a clear functional threshold for monitoring treatment efficacy in patients with Parkinson's disease. More importantly, however, quantification of specific functional deficits could well provide a basis for locating the source and extent of the neurological damage, and therefore aid clinical decision-making for individualizing therapies.

Keywords: gait variability, walking balance, dynamic stability, systematic review, meta-analysis, quality of movement, rhythmicity

# INTRODUCTION

Disturbance of normal walking patterns, caused by symptoms such as akinesia and loss of postural reflexes, is a wellacknowledged problem in patients with Parkinson's disease (PwPD) (Hausdorff, 2009). With the progression of the disease, balance, gait, and mobility are increasingly impaired, causing a loss of independence, and consequently a reduction in the quality of life (Damiano et al., 1999; Ellis et al., 2011). This loss of function, mobility, and independence is associated with further complications such as cognitive impairments, sleep disorders (Imbach et al., 2012), depression, cardiovascular diseases (Ton et al., 2010, 2012), and injuries and fatalities (Balash et al., 2005). Despite established pharmaceutical and surgical therapies for treating motor symptoms in PwPD, the disease poses immense challenges for clinicians to identify the disease onset at an early time point, provide a long-term objective evaluation and monitoring of therapies, but also to quantify differences between therapies.

Although, a number of recognized biomarkers for the clinical identification and evaluation of Parkinson's disease (PD) exist (Andersen et al., 2016; Salat et al., 2016), objective methods to measure human movement have become increasing available, and now provide the potential to complement clinical decisionmaking (Lord et al., 2011a, 2013). In an attempt to translate parameters derived from kinematics into an understanding of movement quality, several measures of both spatial and temporal gait have been investigated: summary measures based on the statistical mean (e.g., mean stride length; Faist et al., 2001; Ferrarin et al., 2005; Hausdorff, 2009), measures to quantify variability during walking based on the standard deviation (e.g., coefficient of variation of stride length; Blin et al., 1990, 1991; Hausdorff et al., 1998; Baltadjieva et al., 2006; Roemmich et al., 2012), measures to quantify bilateral symmetry of walking (e.g., phase coordination index; Plotnik et al., 2007; Plotnik and Hausdorff, 2008; Johnsen et al., 2009; Fasano et al., 2011), and non-linear algorithms that evaluate the structure of gait signals in relation to their temporal evolution (e.g., Lyapunov exponent; Dingwell and Cusumano, 2000; Bruijn et al., 2013; Roemmich et al., 2013). However, since PD is known to disturb rhythmicity of walking (constancy of step repetitions), mean measures for assessing degeneration of this temporal parameter simply result in averaging out any modifications and are therefore entirely insensitive. As a result, only parameters that highlight non-constancy over multiple repetitions can provide objective interpretation of modifications that occur with degeneration due to PD. While non-linear and bilateral symmetry measures of walking provide promising candidates for evaluating such degeneration, due to their simple implementation, parameters of gait variability have so far received the most attention (Hamacher et al., 2011; Lord et al., 2011a; Konig et al., 2014a). However, until now, thorough evaluation of measures of variability for understanding the quality of movement remains missing, and in particular, which threshold levels of gait variability can be considered physiological vs. pathological.

Generally, motor variability increases with aging and pathology while functional performance decreases. As a consequence, variability has been assumed to be detrimental for task performance (Hamacher et al., 2011; König et al., 2016). Traditionally, motor variability was thought to be the result of noisy signaling processes within the human sensory motor system (HSMS). However, recently it has been shown that motor variability can be adapted depending on the motor task requirements, and can also play an important role for successful motor learning (Roerdink et al., 2006; Dingwell and Kang, 2007; Wilson et al., 2008; Russell and Haworth, 2014; Wu et al., 2014; Pekny et al., 2015). As a result, it appears that motor variability is an integral aspect of human movement as well as a prerequisite for effective task performance. It has been argued that excessively high levels of variability render task performance unstable, whereas extremely low levels result in rigid motor performance, hampering the subject's ability to respond to changing environmental conditions. As a result, it seems plausible that an optimal level of variability for successful task performance exists (Todorov and Jordan, 2002; Stergiou et al., 2006). Indeed, in a large systematic review of the literature, it has recently been revealed that an optimal window of variability during walking and balancing exists, and which was somewhat consistent across a variety of neuromotor pathologies including stroke, brain injury, and disorders of the basal ganglia etc (König et al., 2016). However, while a consistent window of optimal levels of variability of stride time was presented, it remains unknown whether all parameters of variability respond in a similar manner to degenerated motor control in extrapyramidal diseases.

An efficient walking pattern is characterized by a plethora of concepts such as rhythmicity, regularity, bilateral, and intersegment coordination, balance control during one-legged and double limb support phases, and controlled forward progression etc., as well as the maintenance of boundary constraints such as sufficient toe clearance to avoid obstacles (Lord et al., 2013). However, such complex concepts to represent gait quality cannot be readily captured in any single parameter, and also not described in terms of quantity (less may not necessarily be better). An additional difficulty in quantifying the degeneration of gait quality due to pathology is that gait is controlled by the coordinated action of various central and peripheral neural circuits (Dietz, 1992, 2002, 2003; Arshavsky et al., 1997; Rosano et al., 2008). As a result, the primary understanding of many neural control mechanisms until now has come from clinical observations of various gait abnormalities (e.g., gait in hypokinetic vs. hyperkinetic disorders) in patients with known neuro-motor diseases, rather than an objective quantification of parameter deviations. For example, an observation of reduced step length in PwPD does not permit conclusions regarding the quality of walking, nor is it indicative of the underlying neural deficit. Importantly, these subjective observations generally lack sensitivity for the early identification of subtle and/or emerging pathologies.

In order to enable improved interpretation of gait metrics in clinical settings for diagnosis and assessment of gait quality and thus treatment effects in PwPD, the aim of this systematic literature review and meta-analysis was therefore to (1) establish whether different gait parameters are indicative of pathological disturbances of the HSMS in PwPD and (2) to define clear threshold values for healthy and pathological gait variability.

# METHODS

#### Literature Search and Selection

The data presented here comprises a follow-up analysis of a wider systematic review that compared motor variability in patients with various neurological diseases against the levels observed in asymptomatic subjects (König et al., 2016). While it achieved a new perspective on the effect of pathology on stride time variability, providing for the first time a window of optimal variability, it did not address the distinct effects of pathology on different measures of gait. In this current study, we therefore focus specifically on literature assessing measures to quantify variability during walking based on the standard deviation in PwPD.

The original systematic literature search was conducted with the aim to comprehensively identify studies in which measures of motor variability during walking and standing were collected in both a cohort of healthy elderly and a cohort of patients with a neurological pathology (König et al., 2016). There, a common search string was entered into four different databases (Pubmed, ISI Web of Knowledge, Embase and Ebsco). The search string contained Boolean operators such that an AND-combination of terms specified the task (e.g., walk<sup>∗</sup> ), measure (e.g., variability), and cohort (e.g., Parkin<sup>∗</sup> ). Within these categories synonyms as well as specifications of additional pathological cohorts were combined using the OR operator. The search was limited to original research articles published after the year 1980. Initially, the search revealed 13′ 195 publications potentially relevant for addressing the question of motor variability across neural pathologies. In two steps, eligibility of studies was assessed using the double-screening method (NK & NS) firstly on the titles and abstracts, followed by examining the methods section of each publication. Here, publications were selected according to pre-defined inclusion and exclusion criteria, and disagreement between reviewers was solved by consensus. Based on the 109 publications included in the original review, a further reduction of titles was undertaken in order to focus on studies addressing walking performance in PwPD, resulting in a total of 34 papers.

# Meta-Analysis

The aim of the meta-analysis was two-fold: Firstly, to determine threshold levels of gait variability that discriminates healthy controls (HCs) from PwPD, and secondly to identify whether all variability metrics are indeed elevated in PwPD. In order to achieve this, means and standard deviations (SD) of the various measures of gait variability for both asymptomatic and PwPD cohorts were extracted. In cases where standard error of the mean or 95% confidence intervals (95%CI) were presented, these values were translated into SD as recommended by Cochrane (Higgins, 2011). An effect size (ES) for each study was then determined according to Cohen (1988). In addition, each ES was corrected for sample size and adjusted to provide Hedges' g (Lipsey and Wilson, 2001). Finally, parameters were grouped to account for different reporting metrics when e.g., coefficient of variation vs. standard deviation of the same parameter was reported. In order to assess the effect of different gait parameter groups in PwPD, a mean ES for each gait parameter group was calculated (ES′ ), with studies weighted according to their standard error of measure. Heterogeneity was then assessed using Cochrane's Q and I 2 statistics.

A binary logistic regression (BLR) analysis was then performed on the most commonly reported gait parameter in order to assess how this parameter discriminates the two groups (i.e., HCs and PwPD). The logistic curve-fit was firstly analyzed using the Chi-square goodness-of-fit test, while the quality of the classification was evaluated using a receiver-operating characteristic (ROC) procedure. We then identified the optimal operating point, yoop with balanced levels of sensitivity as well as specificity. yoop was then used in an inverse binary logistic regression function in order to assess the optimal threshold value xoop for the most commonly reported gait measures (Equation 1):

$$\frac{\log\_{\mathfrak{e}}\left(\frac{\mathsf{y}\_{\text{oop}}}{1-\mathsf{y}\_{\text{oop}}}-b\_{0}\right)}{b\_{1}}=\mathsf{x}\_{\text{oop}}\tag{1}$$

# RESULTS

The 34 publications addressing walking performance in PwPD included a total of 800 PwPD (mean age: 65.6 ± 12.2 years) and 854 HCs (mean age: 65.5 ± 13.2 years). Clinically, disease severity was most commonly evaluated using the Unified Parkinson's Disease Rating Scale (UPDRS part III) (17 studies; range of study means: 6.2 to 50.2), followed by the Hoehn and Yahr scale (9 studies; range of study means: 1.6 to 2.8). Seven studies tested patients in the off-medication condition, one study tested both "on" and "off," and the remaining studies measured in the onmedication state (electronic Supplementary Table 1). All studies measured subjects during overground walking for the evaluation of gait, most commonly using footswitches (11 studies) followed by pressure sensitive mats (8 studies), but also by using optical motion capture systems (7 studies). Within the 34 publications, some studies reported multiple gait parameters, resulting in a total of 63 reported ES-values, with an overall I 2 -value of 42.3% and Cochrane's Q of 43.6. There were no significant differences between off-medication and on-medication trials (ES′ = 0.71 vs. ES′ = 0.75; p = 0.94). Eight parameter groups were identified, with variability of stride time (SrT; 21 studies), variability of stride length (SrL; 11 studies) and variability of step length (StL; 10 studies) being the most frequent. The majority of parameter groups showed a positive ES′ (ranging from SrL = 0.36 ± 0.19 to variability of double-limb support time = 1.30 ± 0.51), indicating a general increase in gait variability in PwPD compared to HCs (**Figure 1**). Only the parameter groups of step width variability (StW) (3 studies; ES = −0.54 ± 0.38) and stance time variability (SaT) (2 studies; ES = −0.24 ± 0.54) revealed a negative ES′ , indicative of lower levels of variability in PwPD as compared to HCs.

The BLR was conducted on the parameter SrT, including a total of 21 studies with 519 PwPD and 574 HCs, and revealed an area under the ROC curve of 0.74, with a sensitivity of

variability of PwPD compared to the HCs.



An alpha level of 10% was used to indicate significant differences in this meta-analysis (Higgins, 2011).

0.67 and a specificity of 0.78. In the inverse logistic regression, the corresponding xoop value revealed an optimal coefficient of variation of SrT of 2.4% [95%CI; 1.9 to 3.9] to discriminate Parkinsonian gait from asymptomatic walking.

#### DISCUSSION

Measures of variability have become a popular target for the assessment of gait function in PwPD. However, until now normative values for physiological variability have been missing and it remains unclear how specific gait parameters and their combinations reflect a healthy walking pattern. This systematic review and meta-analysis now provide evidence that 2.4% stride time variability discriminates healthy from pathological walking. More importantly, however, this analysis of the literature has, for the first time, considered how different gait parameters are indicative of pathological disturbances of the HSMS in PwPD. Here, contrary to the common observation of increased levels of variability during gait in these patients, the parameter of step width variability is decreased, hence indicating less flexible motor performance.

The majority of gait parameter groups showed positive ES between PwPD and asymptomatic elderly subjects. This indicates elevated levels of variability during walking in PwPD, which has been associated with reduced walking stability (Dingwell and Kang, 2007; Toebes et al., 2012) as well as a risk factor for falling (Hausdorff et al., 1997; Hamacher et al., 2011; Kobayashi et al., 2014; Konig et al., 2014a), which is a common problem in advanced PwPD cohorts (Wood et al., 2002; Schaafsma et al., 2003; Allen et al., 2011). Here, the parameter of stride time variability in particular has shown to be sensitive in the prediction of falls (Hamacher et al., 2011; Konig et al., 2014a), possibly since variability of temporal gait measures (e.g., stride and step time) depicts the concept of walking rhythmicity, where increased temporal variability is typically associated with observations of unsteady gait (Frenkel-Toledo et al., 2005; Lord et al., 2013). From a neurophysiological perspective, one of the functions of the basal ganglia, in particular the posterior putamen, substantia nigra and globus pallidus, is to maintain rhythmicity during repetitive motor tasks (Plotnik and Hausdorff, 2008; Takakusaki et al., 2008; Lord et al., 2011b; Wu et al., 2015), which is supported by the fact that intervention to these structures in the form of either dopamine replacement therapy or deep brain stimulation has been shown to reverse degenerative changes to temporal variability (Schaafsma et al., 2003; Hausdorff et al., 2009; Bryant et al., 2016). It therefore seems entirely plausible that malfunction or degeneration of specific basal ganglia structures might directly contribute to increased temporal variability during gait in PwPD.

Unexpectedly, two parameter groups showed average negative effect sizes, which is representative of reduced variability during walking in PwPD. While variability of stance time exhibited negative ES′ , this parameter was only represented by two extremely inconsistent studies and therefore clearly requires further investigation before meaningful conclusions can be drawn. For the parameter of step width variability, however, three studies revealed a highly consistent ES′ of −0.54. This indicates that the performance of the patients was less variable, or in other words more rigid, than HCs. Step width variability has been associated with balance performance during walking (Gabell and Nayak, 1984). It is well-established that in PwPD, static as well as dynamic balance is commonly disturbed (Allen et al., 2011; Park et al., 2015; Rinalduzzi et al., 2015) but also that dopamine replacement therapy usually lacks efficacy for reversing the effect (Rocchi et al., 2002; Benatru et al., 2008; Curtze et al., 2015), suggesting that neurophysiological structures other than dopasensitive cortico-basal circuits (Takakusaki et al., 2008) (i.e., basal ganglia) are involved in the control of balance during gait (Curtze et al., 2015; Mancini et al., 2015). Here, the pedunculopontine nucleus (PPN) has recently received considerable attention as a major protagonist involved in the control of balance (Hamani et al., 2007; Stefani et al., 2007; Takakusaki et al., 2008; Jahn and Dieterich, 2011; Mancini et al., 2015; Wu et al., 2015). Thus, we hypothesize that the observation of the opposite effect in step width variability in PwPD (i.e., reduced) as compared to parameters of temporal variability, could be explained by the selective effects of pathology on different neurological structures.

From a human movement perspective, however, the observed reduction in step width (spatial) variability, together with an increase in all temporal parameters of variability, might alternatively be explained by a motor compensation mechanism. Here, rather than the direct degeneration of specific neurophysiological structures that exclusively govern particular gait parameters, the loss of control over walking rhythmicity might be actively counter-balanced by tighter regulation of spatial parameters of movement, specifically step width. Here, it should be noted that the greatest effect sizes were observed in the parameter group of double-limb support time (DLS), which is a temporal parameter—therefore affected by disturbances to gait rhythmicity—but one that is also strongly associated with dynamic balance during walking (Lord et al., 2011b). It is therefore conceivable that patients with increased temporal variability compensate by regulating their step width. Such alternative control mechanisms would suggest that balance maintenance is a complex motor function that requires control of both spatial (e.g., step width) and temporal (e.g., DLS and stride time) domains (Todorov and Jordan, 2002), and therefore probably also requires the involvement of different neurophysiological structures. However, further investigation is clearly required to improve our understanding of the relationships between different gait characteristics and their interaction, as well as specific pathophysiology in PwPD.

There are certain limitations to this systematic review of the literature that must be considered when interpreting the presented results. Firstly, the reliability consistency of testing protocols applied in the studies should be considered. Specifically, it has been argued that at least 50 steps are required to ensure reliable measures for walking variability (Konig et al., 2014b), which was only fulfilled in 13 of the 34 (38%) studies. However, testing protocols within a study were similar for both the PwPD and HC groups, and therefore the approaches used possessed equal levels of reliability. Previously, it was shown that low reliability is caused by random error effects and its influence on the derived effect size is therefore negligible (Konig et al., 2014b). However, it is strongly suggested that walking protocols include the assessment of more than 50 steps in future studies. Secondly, a common limitation of reviews is their dependency on publication bias. It seems plausible that results contradicting common expectation on the relationship between pathological and healthy walking performance (i.e., PwPD should exhibit increased variability) are less likely to be published. This results in a relatively higher number of studies presenting positive ESs, which will overestimate the generalized effect of Parkinson's disease on walking performance. For the presented meta-analysis of the literature, such bias could have affected our estimation of the threshold level for stride time variability.

In conclusion, this systematic review and meta-analysis of the literature provides the highest level of evidence for a threshold of 2.4% CV of stride time to discriminate healthy from Parkinsonian gait. Furthermore, it has been shown, that not all gait parameters are equally increased in PwPD. In particular, a decrease in step width variability might be indicative of selective damage to specific neurophysiological structures or alternatively an important compensation mechanism for maintenance of gait stability. Although, the aetiology of disturbed gait characteristics remains to be elucidated, this systematic review is the most comprehensive study to examine the complex interplay between spatial and temporal control in gait in PwPD. However, an accurate analysis of neurophysiological damage in PwPD, together with a reliable and comprehensive assessment walking parameters could lay the foundations for improving our understanding of disturbed walking and control mechanisms during walking in man.

# AUTHOR CONTRIBUTIONS

NK, Data collection and analysis and writing of manuscript. NS, Data collection and analysis and statistical analysis. CB, Interpretation of results and writing of manuscript. WT, Conceptual design of study and writing of manuscript.

# SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: http://journal.frontiersin.org/article/10.3389/fnhum. 2016.00319

#### REFERENCES


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2016 König, Singh, Baumann and Taylor. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Patient-Specific Electric Field Simulations and Acceleration Measurements for Objective Analysis of Intraoperative Stimulation Tests in the Thalamus

Simone Hemm1,2 \* † , Daniela Pison<sup>1</sup>† , Fabiola Alonso<sup>2</sup> , Ashesh Shah<sup>1</sup> , Jérôme Coste3,4 , Jean-Jacques Lemaire3,4 and Karin Wårdell<sup>2</sup>

1 Institute for Medical and Analytical Technologies, School of Life Sciences, University of Applied Sciences and Arts Northwestern Switzerland FHNW, Muttenz, Switzerland, <sup>2</sup> Department of Biomedical Engineering, Linköping University, Linköping, Sweden, <sup>3</sup> Université Clermont Auvergne, Université d'Auvergne, EA 7282, Image Guided Clinical Neurosciences and Connectomics (IGCNC), Clermont-Ferrand, France, <sup>4</sup> Service de Neurochirurgie, Hôpital Gabriel-Montpied, Centre Hospitalier Universitaire de Clermont-Ferrand, Clermont-Ferrand, France

Despite an increasing use of deep brain stimulation (DBS) the fundamental mechanisms of action remain largely unknown. Simulation of electric entities has previously been proposed for chronic DBS combined with subjective symptom evaluations, but not for intraoperative stimulation tests. The present paper introduces a method for an objective exploitation of intraoperative stimulation test data to identify the optimal implant position of the chronic DBS lead by relating the electric field (EF) simulations to the patient-specific anatomy and the clinical effects quantified by accelerometry. To illustrate the feasibility of this approach, it was applied to five patients with essential tremor bilaterally implanted in the ventral intermediate nucleus (VIM). The VIM and its neighborhood structures were preoperatively outlined in 3D on white matter attenuated inversion recovery MR images. Quantitative intraoperative clinical assessments were performed using accelerometry. EF simulations (n = 272) for intraoperative stimulation test data performed along two trajectories per side were set-up using the finite element method for 143 stimulation test positions. The resulting EF isosurface of 0.2 V/mm was superimposed to the outlined anatomical structures. The percentage of volume of each structure's overlap was calculated and related to the corresponding clinical improvement. The proposed concept has been successfully applied to the five patients. For higher clinical improvements, not only the VIM but as well other neighboring structures were covered by the EF isosurfaces. The percentage of the volumes of the VIM, of the nucleus intermediate lateral of the thalamus and the prelemniscal radiations within the prerubral field of Forel increased for clinical improvements higher than 50% compared to improvements lower than 50%. The presented new concept allows a detailed and objective analysis of a high amount of intraoperative data to identify the optimal stimulation target. First results indicate agreement with published data hypothesizing that the stimulation of other structures than the VIM might be responsible for good clinical effects in essential tremor.

(Clinical trial reference number: Ref: 2011-A00774-37/AU905)

Keywords: deep brain stimulation (DBS), intraoperative stimulation tests, essential tremor, acceleration measurements, finite element method (FEM) simulations, ventral intermediate nucleus (VIM), patient-specific brain maps

#### Edited by:

Marcelo Merello, Fundación para la Lucha contra las Enfermedades Neurológicas de la Infancia, Argentina

#### Reviewed by:

Aasef G. Shaikh, Case Western Reserve University, USA Hoon-Ki Min, Mayo Clinic, USA

> \*Correspondence: Simone Hemm simone.hemm@fhnw.ch

†These authors have contributed equally to this work and should be considered as first authors.

Received: 25 July 2016 Accepted: 01 November 2016 Published: 25 November 2016

#### Citation:

Hemm S, Pison D, Alonso F, Shah A, Coste J, Lemaire J-J and Wårdell K (2016) Patient-Specific Electric Field Simulations and Acceleration Measurements for Objective Analysis of Intraoperative Stimulation Tests in the Thalamus. Front. Hum. Neurosci. 10:577. doi: 10.3389/fnhum.2016.00577

# INTRODUCTION

fnhum-10-00577 November 23, 2016 Time: 17:24 # 2

Deep brain stimulation (DBS) is a common neurosurgical procedure for relieving movement disorders such as those observed in Parkinson's disease (PD) (Benabid et al., 1993, 2009; Hemm and Wårdell, 2010), essential tremor (ET) (Benabid et al., 1991) and dystonia (Coubes et al., 2000; Cif et al., 2010). Despite an increasing use and an extension of the indications (Hariz et al., 2013), the fundamental mechanisms underlying stimulation-induced effects, either therapeutic or adverse, remain largely unknown. The exact anatomical regions or white matter fibers responsible for these effects are still subject of discussion (Herrington et al., 2015). During a typical surgical planning, the optimal implantation position for a specific target is first approached based on anatomical images. Intraoperatively, the micro contact of an exploration electrode is often used for microelectrode recordings (MER) (Coste et al., 2009) to evaluate the neuronal activity at previously planned positions of deep brain structures. In a further step, intraoperative stimulation tests are performed through the macro contact of the exploration electrode at different locations with help of the MER-system, and changes in the patient's symptoms are observed by clinical examination. The DBS electrode is finally implanted at the location with the highest therapeutic effect on the symptom with minimal stimulation amplitude and side effects, or with side effects occurring only for high stimulation amplitudes. This procedure is completely based on the physicians experience and will therefore vary depending on the clinical skills (Post et al., 2005).

A way to objectify this evaluation is to use accelerometer recordings of the movements. We have previously presented a method to support the physician's evaluation during surgery by quantifying intraoperatively obtained therapeutic effects on tremor (Shah et al., 2016b) and rigidity (Shah et al., 2016a) with the help of wrist acceleration measurements. These results suggest that mathematical parameters extracted from the acceleration signal are more sensitive to detect changes in tremor during intraoperative stimulation tests than the subjective neurologist's evaluation. An enhancement of this methodology would be to relate the wrist accelerometer measurements for the evaluation of intraoperative stimulation tests with the patient's own brain anatomy and patient-specific simulations of the EF around the stimulation electrode.

The finite element method (FEM) is commonly used to simulate the distribution of the EF around DBS electrodes often taking into account the individual patient's anatomical data (Åström et al., 2009; Chaturvedi et al., 2010; Wårdell et al., 2015). The established models have been applied to relate the results of long term chronic stimulation to anatomical structures surrounding the stimulating contact. However, the use of patientspecific models to simulate data acquired during intraoperative stimulation tests has not yet been proposed.

To support the patient-specific simulations and also the surgical planning, different brain atlases have been suggested over the years (Schaltenbrand and Bailey, 1959; Morel, 2007). This is especially important for brain nuclei generally not visible with current conventional magnetic resonance imaging (MRI). With specific sequences it is possible to detail most common substructures of the thalamus and of other deep brain regions (Zerroug et al., 2016). Lemaire et al. (2010) introduced a high resolution atlas of the thalamus which makes extraction of such nuclei possible.

The aim of the present study was to introduce a new methodology combining different patient-specific data to identify the optimal implant position of the chronic DBS lead: thalamic patient-specific brain maps, EF simulations for intraoperative stimulation tests based on patient-specific simulation models and the corresponding therapeutic effects quantitatively evaluated by wrist accelerometer recordings. To illustrate the feasibility of this methodology, it was applied to five patients with ET who underwent stimulation tests during targeting of the ventral intermediate nucleus of the thalamus (VIM). An exemplary way of analysis is presented by comparing the extension of stimulation for no/low and intermediate/high improvements.

# MATERIALS AND METHODS

An overview of the methodology including imaging, generation of patient-specific maps of the thalamic region, surgical planning, surgical procedure, stimulation tests, accelerometer measurements, patient-specific EF simulations and data analysis is presented in **Figure 1**.

# Surgical Protocol

Stereotactic exploration and lead implantation were performed at the Department of Neurosurgery, Clermont-Ferrand University Hospital, France, under local anesthesia in a two-day procedure.

The first day, the stereotactic frame was mounted on the patient's head (Leksell <sup>R</sup> G frame, Elekta Instrument AB, Sweden) under local anesthesia. T1 MRI (0.63 mm × 0.63 mm ×1.30 mm) and white matter attenuated inversion recovery images (WAIR, 0.53 mm × 0.53 mm × 2.00 mm) (Magnotta et al., 2000; Lemaire et al., 2007) were acquired (Sonata, 1.5T, Siemens, Germany). Using a stereotactic planning software (iPlan 3, Brainlab, Feldkirchen, Germany), the VIM and its anatomic neighbors were carefully identified and manually outlined on the coronal plane of the WAIR sequence (Lemaire et al., 2010; Zerroug et al., 2016). The nuclei identification followed the previously published nomenclature (Lemaire et al., 2010; Vassal et al., 2012) based on their relative positions, intrinsic MRI tissue contrasts on 1.5T WAIR images (see Figure 6 in Zerroug et al., 2016) and an in-house microscopic 4.7T 3D T1 MRI atlas (see Figure 1 in Vassal et al., 2012). Target coordinates and two parallel trajectories were defined according to the stereotactic reference system, without AC-PC referencing. **Figure 2** shows a stereotactic planning including the patient-specific brain map and the planned trajectory.

The second day, after repositioning of the frame and stereotactic computed tomography (CT) acquisition (0.59 mm × 0.59 mm × 1.25 mm), the planned trajectories were checked and adjusted if necessary with the stereotactic reference system of the CT after rigid image fusion of WAIR and CT data sets. The target region was then explored intraoperatively

(MicroGuide Pro; Alpha Omega Engineering, Nazareth, Israel) (Slavin and Burchiel, 2002) under local anesthesia using two exploration electrodes (Neuroprobe 366-000024, Alpha Omega Engineering, Nazareth, Israel) that were steered by rigid guide tubes (ACS-7905/200-5, DIXI Microtechniques, Besançon, France): one for the planned track (named the central track) and one placed 2 mm in parallel, usually posterior or posterolateral to the central one. MER was acquired in millimeter steps using the micro contact of the electrode which was retracted before starting stimulation tests in order to avoid tissue damage. Gradual stimulation tests were performed at the same locations through a macro contact to assess clinical benefit and adverse effects and to identify the optimal target. For each stimulation test, the surgical team identified and noted the maximum change in the patient's tremor relative to the initial state of the patient (baseline), and the corresponding stimulation amplitude as well as the occurrence of side effects. MER and stimulation tests were in general performed in a range starting some millimeters in front of the target point and going slightly below depending on the anatomical location. In addition to this routine assessment of tremor, wrist accelerometer measurements and video recordings were performed. Following the stimulation tests, a quadripolar DBS-lead (Lead 3389, Medtronic Inc., USA) was implanted at the optimal stimulation spot for chronic stimulation.

#### Acceleration Measurements

To perform intraoperative acceleration measurements, a 3-axis accelerometer, placed inside an in-house developed plastic case, was tied to the patient's wrist on the opposite side of the stimulated hemisphere. Via a USB cable, the device was connected to a laptop based data recorder using homemade software LemurDBS (Java 1.6, Oracle, USA) (Shah et al., 2013). Synchronization between acceleration data and test stimulation amplitudes was assured by a pulse sent from the laptop to the stimulating equipment. The sensor was always attached at the same position on the wrist of the patient, and at each position a baseline recording was acquired before initiation of each stimulation sequence.

In order to quantify the clinical improvement for each stimulation amplitude, a previously developed analysis method in Matlab (R2014b) was used (Shah et al., 2016b). As a first step, movements other than tremor were removed offline by using the smoothness priors method (Tarvainen et al., 2002) and thereafter a second order Butterworth low pass-filter was applied at 10 Hz in order to suppress noise. Statistical features (standard deviation, signal energy and the spectral amplitude of the dominant frequency, defined as the frequency of the signal with maximum spectral power) were extracted by moving a 2 second-window over the data. These features were then normalized to the feature set representing the most intense tremor at baseline, i.e., during an initial off-stimulation period at the same position. For each stimulation amplitude the mean of the obtained quantitative clinical improvement was retained as a percentage value. An example is presented in **Figure 3A** where a typical acceleration signal in one position is presented together with the clinical improvement based on the calculated features. In the presented example, it can be seen that an improvement of 98.1% was obtained at a stimulation amplitude of 1.0 mA.

(PU); nucleus anterolateral (AL); nucleus ventro-oral (VO); field of Forel (FF); nucleus centromedian (CM); the nucleus ventrocaudal medial (VCM) and the pre-lemniscal radiations (PLR) within the prerubral field of Forel are not visible. Central and posterior left trajectories are visible (brown lines) and marked as dashed lines if inside the nuclei.

# Electric Field Simulations

In order to simulate the EF spatial distribution within the brain, a 3D FEM model of the exploration electrode with the surrounding brain tissue was built (Comsol Multiphysics, Version 4.4 Comsol AB, Sweden) for adapting an already established patient-specific modeling technique for DBS leads (Åström et al., 2010; Wårdell et al., 2015).

#### Brain Tissue Model

The axial preoperative T1 MRI was registered and resampled to the stereotactic preoperative axial CT dataset. In a next step, it was imported into the in-house developed software (Matlab R2013) (Wårdell et al., 2012) modified for the creation of the brain tissue models. A separate filtered axial T1 image batch with enhanced region of interest was used to segment cerebrospinal fluid (CSF), gray matter and white matter (Alonso et al., 2016). The segmented image voxels were assigned with the corresponding electrical conductivities (σ) (Gabriel et al., 1996) 1 . CSF and blood were set to 2.0 Siemens/meter (S/m) and 0.7 S/m, respectively. Considering the frequency (130 Hz) and pulse length (60 µs) of the stimulation (Wårdell et al., 2013), to gray matter was assigned 0.123 S/m and to white matter 0.075 S/m. Interpolation was done for conductivity values in-between the thresholds used. In order to reduce the simulation time, a region of interest (a cuboid of approximately 100 mm per side) covering the thalamus and its surroundings was selected from the brain tissue model.

#### Exploration Electrode and Guide Tube Model

A model of the stimulating contact of the exploration electrode and the guide tube was developed. **Figure 4A** presents the outer and inner dimensions of the exploration electrode and the guide tube, **Figure 4B** the corresponding model. The end of the guide tube was fixed 12 mm above the chosen target point, i.e., above the a priori optimal anatomic spot. A second exploration electrode and guide tube model was positioned in parallel at a distance of 2 mm. The distance between the guide tube and the center of the stimulating contact decreased or increased when the simulation site was ahead or beyond the target point, respectively. The center of the stimulating contact was placed at the different planned stimulation positions. The micro contact was not considered as it was retracted during the stimulation tests.

#### Simulations

The EF was calculated by using the equation of continuity for steady current according to:

$$\nabla \cdot \bar{J} = \nabla \cdot (\sigma \nabla V) = 0$$

where J is the current density, σ a matrix containing the electrical conductivity values for the region of interest (thalamus and neighborhood) and V the electric potential. A monopolar configuration was conducted using the guide tube as the reference electrode setting it to ground, and the active electrode set to the same current as used during the stimulation tests. The non-active contact of the parallel lead was set to floating potential (Schmidt et al., 2013). The exterior boundaries of the tissue model were set to electrical insulation. The mesh density (consisting of about 250,000 tetrahedral elements) was defined by the built in physics-controlled mesh generator, where the smallest elements (0.204 mm) were located nearby the stimulating contacts in order to capture the strong EF gradients. The Cartesian coordinates of the points describing the surface of the simulated EF volume (**Figure 3C**) were exported for further analysis. In this study an EF isolevel of 0.2 V/mm was used in order to be able to perform relative comparisons between the simulations and to comply with approximate axon diameters in the thalamus (Kuncel et al., 2008; Åström et al., 2015; Alonso et al., 2016).

<sup>1</sup>Andreuccetti, D., R. Fossi and C. Petrucci, Florence, Italy. (2005). "Dielectric properties of the tissue." from http://niremf.ifac.cnr.it/tissprop/.

superimposed EF isosurface (green). (F) Close up view of (E) with the structure volumes inside the EF isosurface indicated in orange.

# Thalamic Brain Maps and Electric Field Visualization

The thalamic structures (**Figures 2** and **3B**) initially outlined on the WAIR weighted sequence in the iPlan software were exported in form of slices parallel to the stereotactic CT data set via an interface based on VVLink and VTK (VTK 5.2.0, Kitware Inc., Clifton Park, NY, USA). Target and trajectory coordinates were also exported in CT image coordinates by the same software interface. The CT data set was chosen as it provides a higher resolution and no distortion of the stereotactic reference system compared to MR sequences. With the exported data a 3D thalamic brain map with trajectories was generated in Matlab (R2014b) (**Figure 3D**). For each stimulation test position and amplitude, the EF isosurface generated through FEM simulations was imported, superimposed to the 3D thalamic brain map and color-coded depending on the induced, quantitatively evaluated improvement (**Figure 3E**).

#### Volumetric Analysis

An in-house algorithm developed by FHNW in Matlab (R2014b) was applied to detect and calculate the volume of the anatomical structures inside each EF isosurface. To reduce the computational time, a list of candidate structures (e.g., VIM, VO, and CM) was identified from the entire structure group by excluding the structures outside the coordinate's ranges of the EF volume. For each candidate structure, the points of the EF isosurface inside the structures' volume were detected by considering them as a concave or convex hull according to their shape. The obtained volume based on the selected point cloud was then calculated and associated with the respective clinical improvement. The algorithms then generated a list of the thalamic structures lying partially or completely inside the 0.2 V/mm EF isosurface, their volumes as well as the volume covered by the EF surface and the associated improvement value (**Figure 3F**).

# Clinical Application

The above presented protocol was applied to five ET patients (three male and two female) undergoing bilateral DBS electrode implantation in the VIM region and successively to both hemispheres. They gave their written informed consent to participate in the study (Ref: 2011-A00774-37/AU905, Comité de Protection des Personnes Sud-Est 6, Clermont-Ferrand, France). No alterations were made to the routine surgical procedure. In all patients a central and a posterior trajectory were chosen per hemisphere for MER and stimulation tests (stimulation parameters: amplitude = 0.2 to 3.0 mA in steps of 0.2 mA, pulse width = 60 µs, frequency = 130 Hz). At each stimulation position stimulation lasted 1 to 3 min depending on the response of the patient and on side effect occurrence or not. Between all stimulation tests, a non-stimulation period was maintained to leave time to the symptoms to come back. The duration of this period depended on patient symptoms (minimum 2 min). Acceleration measurements were performed in parallel to the

test stimulation in 31, 22, 30, 28, and 32 positions for Patient 1 to 5, respectively, mostly from 5 mm above the target point down to 4 mm below depending on the individual anatomical locations. The final electrode implantation site was based on clinical subjective evaluations.

Electric field simulations were performed for all stimulation test positions in both hemispheres of the five patients. At each position, up to four tested stimulation amplitudes were chosen for simulations using the following criteria based on the quantitatively evaluated symptom improvements (Iacc): (1) The highest amplitude not resulting in any improvement in tremor compared to baseline; (2) the lowest amplitude at which a first improvement in tremor was measured; (3) The lowest amplitude resulting in at least 50% improvement in tremor; (4) The lowest amplitude resulting in at least 75% improvement. When the first improvement in tremor was more than 75%, criteria (2– 4) gave the same amplitude. When the first improvement was identified already between 0.2 and 0.6 mA, no simulations were performed for the criterion 1. The extracted patient-specific 3D brain maps of the thalamus were superimposed with the four trajectories of each patient and with the simulated patient-specific EF isosurfaces. To make the data comparable between patients, the volume inside the isosurface was normalized to the size of the structure resulting in the percentage of the structure covered by the EF (**Figure 5**). For example, if the volume of VIM was 10 mm<sup>3</sup> and only 2 mm<sup>3</sup> of it was encompassed by an EF isosurface, the covered volume of VIM for that EF would have been (2/10)<sup>∗</sup> 100 = 20%.

In order to identify structures responsible for the reduction in tremor, the results of all patients together were classified following the quantity of improvement detected by accelerometry (Iacc). Data were divided into two groups considering no/low improvements (Iacc ≤ 50%) and intermediate/high improvements (Iacc > 50%), respectively. The resulting data are presented in two different ways for comparison of these two improvement groups. Firstly, for each thalamic structure, the relative number of occurrences (the structure is at least partially covered by the EF) was determined: the absolute number of occurrences of each structure in the considered improvement range was normalized to the total number of occurrences of all structures in this range. Second, the percentage volume of each structure covered by a specific EF isosurface was analyzed to see for example if the covered volume of some structures increases for higher improvements. These percentage volumes were graphically represented and visually analyzed together with the induced clinical improvement for all simulations. Furthermore, mean values and the standard error of the mean (SEM) were determined for each structure for the two improvement groups. The results for each structure in the two improvement ranges were statistically compared applying the Mann–Whitney U test. Mean stimulation amplitudes for 50% or less improvement and more than 50% improvement were determined.

# RESULTS

#### Simulations

The proposed concept has been successfully applied to the five patients, resulting in 272 simulations at 143 different stimulation test positions. The detailed numbers of simulations for each patient and different improvement ranges are presented in **Table 1**.

#### Visualization

**Figure 6** shows an example of visualization for Patient 4 with three simulated EF isosurfaces in the left hemisphere along the central tract. Each isolevel is superimposed to the extracted anatomical structures (seen in brown) and the patientspecific MRI (**Figure 6A**). At the target position or the a priori optimal anatomic spot for the left central trajectory an improvement of 90% was reached with a stimulation amplitude of 3 mA (**Figure 6B**). 3 mm below the target no improvement in tremor could be observed. The corresponding EF in red overlays the EF of 90% of improvement as can be seen at cross section II through the stimulation electrode as presented in **Figure 6C**.

#### Involvement of Anatomical Structures

The relative occurrences of the different thalamic structures within the isosurfaces for improvements above and below 50% are presented in **Figure 7**. It shows that the percentage



of occurrences of the different structures was always inferior or equal to 30%. The relative occurrence of InL, VO and especially VIM decreases for higher improvements. This means that their appearance does not as much increase as for CM, VCM, VCL and especially for FF/PLR. For all four structures the relative occurrence increases for higher improvements. Mean stimulation amplitudes for improvements Iacc ≤ 50% and Iacc > 50% were 0.9 ± 1.1 mA and 1.5 ± 1.2 mA, respectively.

# Relation of Structure Occurrences, Clinical Improvement, and Volumes Covered by the Isosurfaces

A comparison between the clinical improvement and the volume of the structures included in the isosurface of the corresponding simulation is presented in **Figure 8**. While **Figure 8A** shows all available data of the five patients, **Figures 8B,C** summarize the data in form of mean and SEM for improvements ≤50% and >50%. All SEM values remain below 3% except for the VO and the InL for the range Iacc ≤ 50% and the FF/PLR for both improvement ranges. A closer analysis of the volume of the different structures covered by the EF isosurface shows that the percentage volumes of the target structure VIM, of the InL and of the FF/PLR increase with significant clinical improvements. The difference for the VIM was statistically significant (p < 0001). Only small volumes of CM and VCL are covered by the isosurface in both improvement ranges. Nevertheless the difference for the CM could be shown to be significant (p < 0.01). The neighboring nuclei VIM and VCM appear together for nearly all simulations. FF/PLR and VO occur mostly in combination with VIM and VCM (same horizontal line) (**Figure 8A**).

# DISCUSSION

In the present study, a methodology is described that has the potential to give new insights into the efficacy of different anatomical structures in DBS. It consists in the combined analysis of intraoperatively acquired accelerometry data, patient-specific EF simulations for intraoperative stimulation tests and patientspecific anatomy. The method was successfully applied to five patients with ET and included more than 250 EF simulations. An exemplary way of analysis and preliminary results have been presented for the identification of the therapeutically effective anatomical region.

# Quantitative Symptom Evaluation

In order to overcome the limits of existing routinely used clinical rating scales, i.e., the inter- and intra-observer variability (Post et al., 2005; Palmer et al., 2010), the discrete evaluation levels and the high dependence on the experience of the evaluating neurologist (Griffiths et al., 2012), we have used accelerometrybased, quantitative tremor evaluations during intraoperative stimulation tests.

Tremor quantification outside the OR has been proposed since a long time by various authors (Mansur et al., 2007), many of whom have concluded that a quantitative evaluation method is more sensitive than the visually performed clinical evaluation. Birdno et al. (2008) used an acceleration sensor to study the effects of temporal variations of the stimulation pulse during the replacement of the implantable pulse-generator. Journee et al. (2007) and Papapetropoulos et al. (2008) used quantitative tremor evaluation after the DBS lead was implanted, in order to compare the effects of stimulation through different contacts. But those systems were not designed to be used in different clinical centers or during stimulation tests performed through an exploration electrode. In a previous study, we have demonstrated the use of our system in 15 DBS surgeries in two different clinical centers, the possibility to visualize and revisit recorded data during surgery and the possible influences of quantitative evaluations on the choice of the final implant position of the lead for chronic stimulation (Shah et al., 2016b).

# Determination of the Therapeutically Implicated Structures

In the present clinical study, structures individually outlined by the neurosurgeon were available and could be used as anatomical reference. The use of the patient-specific MR-WAIR sequence together with a 4.7 T in-house atlas as reference and stereotactic books make an approximate identification of the structures possible (Zerroug et al., 2016). Other groups have proposed various approaches (Caire et al., 2013) among them projecting the position of the active contact(s) directly onto anatomical

in consequence be found on the same horizontal level. As an example, the horizontal line at an improvement of approximately 18% indicates that VIM, VCM, VCL, and FF/PLR were present inside a same EF isosurface. (B) Summary of the mean percentage volume included in the EF isosurfaces and the standard error of the mean for the different structures in the improvement range >50% and (C) in the improvement range ≤50%. ∗∗: Statistically significant difference between the results of the two improvement groups for a specific structure with p < 0.001. <sup>∗</sup> : Statistically significant difference with p < 0.01.

images (Vayssiere et al., 2004), onto anatomical (Saint-Cyr et al., 2002; Sarnthein et al., 2013) or probabilistic functional atlases (Lalys et al., 2013), or linking them to MER results (Zonenshayn et al., 2004) sometimes combined with imaging data (Weise et al., 2013) and white matter tracking (Coenen et al., 2012). To analyze the relationship between the anatomical location of stimulating contacts and the clinical effectiveness of stimulation, we have decided to take into account the extent of stimulation by using EF simulations (Åström et al., 2012) as discussed in detail in the next paragraph. Other published approaches consider either the anatomical position of the center of the contact (Starr et al., 2002; Voges et al., 2009; Garcia-Garcia et al., 2016) or of the whole contact taking into account its dimensions (Saint-Cyr et al., 2002; Zonenshayn et al., 2004; Herzog et al., 2007; Hemm et al., 2008).

#### Electric Field Simulations

Finite element method models are commonly used to simulate and visualize the EF distribution around DBS electrodes and the

EF is one of the electrical entities that may be used to represent the stimulation field. In comparison with the electric potential or the second derivative of the electric potential with respect to the distance (activating function), EF has been shown to be the most stable and unchanged entity for different stimulation parameters (amplitude and pulse width) (Åström et al., 2015).

Today, FEM models have progressed from non-specific (McIntyre et al., 2004; Hemm et al., 2005; Åström et al., 2006) to patient-specific taking into account the individual's data (Åström et al., 2009; Chaturvedi et al., 2010; Wårdell et al., 2015). There is no consensus of the degree of complexity of the model to accurately simulate the neural response, however, many groups (Chaturvedi et al., 2010; Schmidt et al., 2013; Alonso et al., 2016; Howell and McIntyre, 2016) have shown that the inclusion of the heterogeneity and anisotropy of the brain tissue increases the model accuracy and prediction capability. For instance, Chaturvedi et al. (2010) and Åström et al. (2012) observed an overestimation of neural activation for homogeneous models. The present study relies on a brain model built upon the segmentation of the gray matter, white matter, CSF and blood from the patients' MRI and in consequence takes into consideration the inhomogeneity of brain tissue. An even more realistic model may be based on DTI which provides more anisotropic information, however, its resolution is lower than the one of MRI and may introduce other errors (Åström et al., 2012). The simulations in this study were performed for constant current while the dispersive components of the brain tissue have been considered by adjusting the conductivity values for gray and white matter to the particular stimulation frequency and pulse width (Wårdell et al., 2013).

According to previous studies where neuron activation distances were calculated using neuronal models (Åström et al., 2015), an isolevel of 0.2 V/mm represents an equivalent activation distance for neurons within 3–4 µm of diameter (Alonso et al., 2016) and thus seems to comply with axon diameters in the thalamus as previously calculated by Åström et al. (Figure 6, 2015) based on Kuncel et al. (2008). The selection of a fixed EF isolevel allows then to compare the volume recruited for different amplitude settings and different positions.

# Transferability

The described methodology has been presented for an institution-specific surgical protocol but can be transferred to other clinical centers. The approach can be adapted to any kind of anatomical information. Instead of using manually outlined structures, it is possible to combine the generated data with anatomical atlases – with the limitations inherent to such an approach (Vayssiere et al., 2002; Wodarg et al., 2012; Anthofer et al., 2014) – or with fiber tracking data in order to analyze the implication of different fibers in the mechanism of action of DBS (Coenen et al., 2012). The MR image data (T1, T2) that are needed for the EF simulations are generally acquired in every institution for the surgical planning procedure. A modification of the developed model to the institution-specific stimulation test protocol in awake patients might be necessary: the characteristics of the stimulating electrode as well as the position of the guide tube during stimulation have to be adapted. The acceleration data recording can relatively easily be added in the intraoperative phase without any changes in the surgical protocol, without lengthening surgery and most importantly, without any discomfort for the patient. Nevertheless, the correlation of the simulation results can be performed as well based on subjective visual evaluations.

# Clinical Application

The results of the present paper are described as relative occurrences and percentage volumes of the different anatomical structures covered by the EF isosurface. Even if the number of patients presenting ET in our clinical study was low and thus the confidence concerning the analysis of the mechanism of action of VIM-DBS is limited, we can present preliminary results thanks to the high number of stimulation test positions and EF simulations per patient. First results of these EF simulations in **Figure 7** show that the percentage of occurrence of VCM, CM, and FF/PLR increases for higher improvements while the percentage of VIM occurrences decreases. This can be explained by the fact that in 60 out of 143 measurement sites the center of the stimulation contact was already within the VIM. Furthermore, as shown, there is a tendency that higher improvements are linked to higher stimulation amplitudes leading in general to a larger distribution of EF for a same tissue type. When looking at the percentage of volume of the VIM covered by the isosurface between the two improvement groups (**Figures 8B,C**), a statically significant difference exists. Nevertheless, the size of the individual volumes varies inside each group. This result can be interpreted in two ways: either (I) a specific part of the VIM, for example the efferent fibers, has to be stimulated or (II) other structures than the VIM might at least partially be responsible for the therapeutic effect. Following our preliminary results such structures could be the InL or especially the fiber tracks FF/PLR. This hypothesis would confirm previously published data: parts of the InL have been earlier mentioned for tremor reduction (Hirai and Jones, 1989) and several authors (Spiegelmann et al., 2006; Vassal et al., 2012) have reported that chronic stimulation of PLR works very well. Some authors (Caparros-Lefebvre et al., 1999; Vassal et al., 2012) already suggested that parts of the VO or of the zona incerta (Fytagoridis et al., 2012) could be appropriate targets as well. Recently Groppa et al. (2014) proposed the dentatothalamic tract as key therapeutic DBS target structure.

Following **Figure 7** our data suggest that parts of VCL and CM might be stimulated in some cases. However, **Figure 8** shows that the structure volumes included in the isosurface are below 5% in both improvement ranges. In order to avoid misinterpretation, either patient-specific improvement maps should be used for presentation or thresholds should be introduced to exclude insignificant volumes.

An optimal stimulation position and statistically significant clinical conclusions can only be provided after the analysis of more intraoperative data, the identification of occurring structure combinations and especially the side effect occurrences, which have a major influence on the choice of the final implant position of the chronic DBS lead.

#### Limitations and Future Work

fnhum-10-00577 November 23, 2016 Time: 17:24 # 12

The suggested methodology allows a detailed interpretation of intraoperatively acquired data but one has to be aware of certain limitations. First of all, the substantial caveats of non-stimulation factors influencing tremor are undeniable and unfortunately inherent in the operating room conditions. Nevertheless, we have employed various signal analysis techniques to minimize the effect of such non-stimulation factors on the evaluation of tremor using accelerometer. Furthermore, the method was defined in a way trying to limit transformation and fusion errors as much as possible (Zrinzo, 2010). Nevertheless due to the available data, WAIR and T1 MRI data sets containing the anatomic information had to be fused to the stereotactic preoperative CT data set providing the reference for the targeting procedure. Concerning the position of the stimulating contact in relation to the structures, we assumed that the microelectrode was positioned exactly as planned. This seems to be a reasonable approach as the microelectrode was the first entering the brain, and it has been observed that brain shifts in the final electrode position and trajectory can appear when the exploration electrode is replaced by the DBS lead (unpublished data).

As the anatomical information is based on the structures manually outlined on the preoperative image data set, the approach does not consider the movement of the tissue due to the electrodes' insertion or brain shift between implantation sides. On the other hand, the use of these preoperative image data sets is common in analysis and simulation methods. The limitations are acceptable as postoperative image data sets present disturbing artifacts around the implanted DBS leads, in the region of interest. To increase the power of the statistic test performed in the present study, more data should be acquired from further patients and included in the analysis. A further limitation, specific to the anatomical information, concerns the availability of only some anatomical structures and the FF/PLR and that always part of the volume of the EF isosurface is outside any manually defined anatomical structure. In consequence, information from white matter fiber tracking would be helpful to define the region anterior to the VIM and the InL and for further investigating possible activation of fiber tracks (Coenen et al., 2012).

The method could in a next step also be complemented with the available MER data at the different positions including the analysis of time patterns describing the network dynamics as proposed by Andres et al. (2015).

The data analysis approach proposed in the present paper considers the percentage of the structure volume covered by the simulated EF isosurface and not which parts of the structure. Further data interpretation could consider the 3D position as well and should generate improvement maps taking into account stimulation positions of amplitudes as well as the occurrence of side effects.

combining objective intraoperative tremor evaluation with patient-specific EF simulations on manually outlined anatomical structures has been defined and applied to five patients with ET undergoing DBS-implantation. This new approach is combined with an algorithm for detection of the volume of the anatomical structures involved during intraoperative microelectrode stimulation. It can be adapted to further surgical protocols, intraoperative set-ups and to other anatomical data. Its application will allow the analysis of intraoperative data obtained in clinical routine and will support the identification of anatomical structures, parts of them or white matter fibers responsible for the therapeutic effect. The analysis of more data and inclusion of occurrence of side effects are necessary to draw any final conclusions of the most efficient brain targets. The first results, however, indicate agreement with published data hypothesizing that the stimulation of structures other than the VIM might be responsible for good clinical effect in ET.

# AUTHOR CONTRIBUTIONS

All authors contributed in writing the manuscript and critically reviewed the last version. SH: Idea and conception of whole approach; design of protocols and clinical study, especially acceleration measurements; set-up of whole method; development of brain map extraction; participation in data analysis and interpretation; main drafting of manuscript. DP: Technical set-up and implementation of the defined workflow; final data analysis of all available multimodal data; participation in data interpretation; main drafting of manuscript. FA: Set-up of the EF simulation with its patient-specific models; performing all the simulations; data analysis. AS: Realization of acceleration measurements and their analysis; choice of parameters for simulation. JC: Set-up of clinical study and intraoperative realization of study; stimulation data acquisition. J-JL: Set-up of clinical study; patient selection and operation; data interpretation. KW: Set-up of initial e-field simulations; conception of work from simulation point of view; support and critical review of whole concept and especially patient-specific simulations.

#### FUNDING

This research was financially supported by the Swiss National Science Foundation (CR3212\_153370), the Germaine de Staël program of the Swiss Academy of Engineering Sciences, the French Ministry of Health (2011-A00774-37), the Swedish Research Council (621-2013-6078) and the Parkinson Foundation at Linköping University.

#### ACKNOWLEDGMENT

A new concept for the analysis of data acquired during DBS surgery has been proposed. A workflow and methodology The authors acknowledge the contribution of Dr. Miguel Ulla for the neurological evaluation of the patients.

CONCLUSION

# REFERENCES

fnhum-10-00577 November 23, 2016 Time: 17:24 # 13



**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2016 Hemm, Pison, Alonso, Shah, Coste, Lemaire and Wårdell. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Changes in Motor-Related Cortical Activity Following Deep Brain Stimulation for Parkinson's Disease Detected by Functional Near Infrared Spectroscopy: A Pilot Study

Takashi Morishita<sup>1</sup> \*, Masa-aki Higuchi <sup>2</sup> , Kazuya Saita1,3 , Yoshio Tsuboi <sup>2</sup> , Hiroshi Abe<sup>1</sup> and Tooru Inoue<sup>1</sup>

<sup>1</sup> Department of Neurosurgery, Faculty of Medicine, Fukuoka University, Fukuoka, Japan, <sup>2</sup> Department of Neurology, Faculty of Medicine, Fukuoka University, Fukuoka, Japan, <sup>3</sup> Department of Rehabilitation, Faculty of Medicine, Fukuoka University, Fukuoka, Japan

It remains unclear how deep brain stimulation (DBS) modulates the global neuronal network involving cortical activity. We aimed to evaluate changes in cortical activity in six (two men; four women) patients with Parkinson's disease (PD) who underwent unilateral globus pallidus interna (GPI) DBS surgery using a multi-channel near infrared spectroscopy (NIRS) system. As five of the patients were right-handed, DBS was performed on the left in these five cases. The mean age was 66.8 ± 4.0 years. The unified Parkinson's disease rating scale (UPDRS) motor scores were evaluated at baseline and 1- and 6-month follow-up. Task-related NIRS experiments applying the block design were performed at baseline and 1-month follow-up. The mean of the total UPDRS motor score was 48.5 ± 11.1 in the off-medication state preoperatively. Postoperatively, total UPDRS motor scores improved to 26.8 ± 16.6 (p < 0.05) and 22.2 ± 8.6 (p < 0.05) at 1- and 6-month follow-up, respectively. A task-related NIRS experiment showed a postoperative increase in the cortical activity of the prefrontal cortex comparable to the preoperative state. To our knowledge, this is the first study to use a multi-channel NIRS system for PD patients treated with DBS. In this pilot study, we showed changes in motor-associated cortical activities following DBS surgery. Therapeutic DBS was concluded to have promoted the underlying neuronal network remodeling.

Keywords: Parkinson's disease, near infrared spectroscopy, deep brain stimulation, neuronal circuit, neuroplasticity

#### Edited by:

Daniela S. Andres, ETH Zurich, Switzerland

Reviewed by: Hasan Ayaz, Drexel University, USA Stephane Perrey, University of Montpellier, France

> \*Correspondence: Takashi Morishita tmorishita@fukuoka-u.ac.jp

Received: 22 August 2016 Accepted: 24 November 2016 Published: 12 December 2016

#### Citation:

Morishita T, Higuchi M, Saita K, Tsuboi Y, Abe H and Inoue T (2016) Changes in Motor-Related Cortical Activity Following Deep Brain Stimulation for Parkinson's Disease Detected by Functional Near Infrared Spectroscopy: A Pilot Study. Front. Hum. Neurosci. 10:629. doi: 10.3389/fnhum.2016.00629

**Abbreviations:** DBS, deep brain stimulation; fNIRS, functional near infrared spectroscopy; fMRI, functional magnetic resonance imaging; GPI, globus pallidus interna; HbO, oxy-hemoglobin; HHb, doxy-hemoglobin; MRI, magnetic resonance imaging; NIRS, near infrared spectroscopy; PD, Parkinson's disease; UPDRS, Unified Parkinson's Disease Rating Scale.

# INTRODUCTION

There have been numerous neuroimaging studies concerning the pathophysiology of Parkinson's disease (PD; Weingarten et al., 2015; Al-Radaideh and Rababah, 2016), and various aspects of PD came to be investigated with the development of neuroimaging technologies, especially magnetic resonance imaging (MRI). These MRI approaches include task-related functional MRI (fMRI), resting-state fMRI, and diffusion tensor imaging. In addition to these neuroimaging technologies, deep brain stimulation (DBS) surgery has been used to investigate the pathophysiology of PD. Although the literature includes a variety of MRI studies of patients with PD, due to metal artifacts and safety issues, MRI should be performed in patients who have undergone DBS surgery with caution. Newer generation DBS systems are supposedly compatible with MRI; however, disastrous complications have been reported (Henderson et al., 2005; Zrinzo et al., 2011).

Near-infrared spectroscopy (NIRS) has been increasingly being applied in studies of stroke, epilepsy and neurorehabilitation (Obrig, 2014). The NIRS system measures the oxygenation levels of the target area by detecting the characteristic absorption spectra of hemoglobin through near-infrared light probes, and this hemoglobin response has been considered to reflect the neuronal activity in the cerebral cortex based on the neurovascular coupling mechanism. In general, neuronal activation increases the oxy-hemoglobin (HbO) and decreases deoxy-hemoglobin (HHb) levels (Devor et al., 2012). The NIRS system cannot measure activity in deep brain areas such as the basal ganglia; however, there are advantages of functional near infrared spectroscopy (fNIRS) over fMRI, such as the portability and safety of the system. Recent studies using multi-channel NIRS have revealed hemodynamic responses to various motor tasks such as the postural control task (Mihara et al., 2012) and pursuit rotor task (Hatakenaka et al., 2007). These studies reveal the changes in the cortical activities following rehabilitation and/or motor skill learning, and multi-channel NIRS has the potential to measure cortical activity changes following intervention. Of particular relevance to the present study, it should be noted that the multi-channel NIRS system enables the safe measurement of cortical activity in DBS patients.

Several studies have suggested that DBS modulates the global neuronal network of cortical activity, and recent studies show that DBS may disconnect the abnormal coupling of the oscillation activities between the basal ganglia and the motor cortex (de Hemptinne et al., 2015). We hypothesized that the abnormal activity in the neuronal circuits ultimately affects the neurovascular reactivity in the cerebral cortex of PD patients. Therefore, therapeutic DBS may address the pathologically altered cortical activity (neurovascular reactivity) especially in the primary motor cortex, and NIRS may safely detect changes in the cortical activity between pre- and post-DBS therapy. In this pilot study, we aimed to evaluate changes in cortical activity following DBS surgery using a multi-channel NIRS system.

# MATERIALS AND METHODS

#### Study Design

Data were collected from six consecutive patients with PD (two men; four women), recruited between April and November 2015, who underwent DBS surgery at our institution. This study was carried out in accordance with the permission from our institutional review board (IRB) of Fukuoka University Hospital, and written informed consent was obtained from all study participants. DBS candidates were first admitted to the Department of Neurology for detailed assessment. Preoperatively, we performed cognitive and neuropsychological evaluations, and patients with cognitive or psychological issues were excluded from the study.

The mean age of our cohort was 66.8 ± 4.0 years, and the mean disease duration was 12.2 ± 7.6 years at baseline. DBS surgery was performed on the left side for five right-handed patients. The mean number of preoperative levodopa equivalent doses (LEDs) was 1017.3 ± 306.0. One patient underwent right-sided DBS despite right-handedness (case 5), and another patient underwent left-sided DBS despite left-handedness (case 6). All patients were cognitively intact, as the mean Mini Mental State Examination (MMSE) score was 28.0 ± 2.1. These demographic data are summarized in **Table 1**.

Unified Parkinson's Disease Rating Scale (UPDRS) motor scale (Part III) scores were evaluated in the off- and on-medication conditions to predict the clinical efficacy of DBS surgery. UPDRS motor assessment in the offmedication condition was performed following at least 12 h of levodopa and dopamine agonist washout, and patients took carbidopa/levodopa (25 mg/100 mg) for evaluation in the onmedication state. Each case was discussed at the interdisciplinary team meeting for DBS candidacy. In this study, all patients underwent unilateral DBS implantation in the globus pallidus interna (GPI), and the DBS side was selected based on multiple factors, such as handedness and patient-specific symptoms.

In patients with PD who underwent DBS surgery, UPDRS Part III scores were evaluated with on-DBS conditions in the off-medication state 1 and 6 months postoperatively, and all adverse events were prospectively recorded. The UPDRS Part III scores were compared between pre- and postoperative states using the Wilcoxon signed rank test. An NIRS experiment was performed pre- and 1-month-post-DBS surgery. The details of this NIRS experiment are described below.

#### Surgical Procedure

A high-resolution volumetric MRI study was performed for stereotactic planning, and the obtained MRI sequences included T1-weighted images with contrast and fast gray matter acquisition T1 inversion recovery (Sudhyadhom et al., 2009). Stereotactic planning was performed 1 day prior to DBS implantation using commercial software (iPlan Stereotaxy, BrainLab, Germany). After anchoring the Cartesian coordinate system determined by the anterior commissure, the posterior commissure, and the midline plane, we planned the trajectory of microelectrodes such that they passed through the ventrolateral


TABLE 1 | Demographic data.

MMSE, Mini Mental State Examination; LED, levodopa equivalent dose; SD, standard deviation.

area of the GPI, terminated in the optic tract, and avoided the blood vessels, lateral ventricle and sulci.

All antiparkinsonian medications were stopped so that patients were in the off-medication state during the procedure. On the morning of surgery, a Leksell frame was applied, and the patient was transferred to a computed tomography scan site for stereotactic imaging. The computed tomography images were merged with the MRI images, and the preoperative DBS planning was finalized.

Patients were awake during the intracranial DBS lead implantation. In the operating room, the head was fixed to the operating table. A 5.5 cm linear skin incision was made, and a burr-hole was fashioned in the skull. Once the dura was opened, a microelectrode recording procedure was performed to map out the GPI, and then a DBS lead (Model 3387, Medtronic Inc., USA) was implanted for the macrostimulation procedure. A macrostimulation procedure was performed to confirm the clinical effects and threshold levels of stimulationinduced side effects. An implantable pulse generator (Activa SC, Medtronic Inc., Fridley, MN, USA) was implanted on the same day following the intracranial lead implantation. DBS programming was meticulously performed for a few weeks in a hospital setting, and continued monthly for 6 months at the neurology clinic.

# Functional Imaging Experiment

We used a continuous-wave NIRS system (FOIRE-3000, Shimadzu, Japan) for functional imaging. The wavelengths of the near-infrared light were 780, 805 and 830 nm, and the sampling rate was set at 7.8 Hz (time resolution: 130 ms). We attached 32 optodes (16 light sources and 16 detectors) to a head cap bilaterally over the frontal and parietal areas, forming a total of 48 channels (24 channels for each hemisphere). The interoptode distance and the inter-channel distance were 3.0 cm and 2.1 cm, respectively. Regions of interest primarily included the primary motor cortex. Prior to starting the experiment, we took pictures of the patient's head with the head cap and the three-dimensional digitizer from 15 perspectives using a high-resolution digital camera for spatial registration (**Figure 1**).

A block design was applied for data acquisition. While in a sitting position, each patient was instructed to open and close the hand contralateral to the operative side for 15 s at comfortable speed, and rest for 30 s between this motor task for each cycle of 45 s. This motor task was repeated seven times in each session. The NIRS system acquired values of HbO and HHb levels following changes in its cortical concentration based on the modified Beer-Lambert law.

For individual analyses, we used the software pre-loaded on the NIRS machine. Optode location was superimposed on the three-dimensional brain image from the volumetric T1-weighted image used for surgical planning. Spatial registration was performed using 15 pictures covering all probes and points of references, including the nasion and bilateral tragi (**Figure 1**). Subsequently, changes in HbO and HHb levels were colormapped onto each brain image. Prior to the analysis, baseline correction was performed for individual analysis by averaging the starting point data of seven blocks and correcting them to zero. A temporal low-pass cut-off frequency was set at 0.1 Hz to remove high-frequency noise from fNIRS detectors associated with systemic oscillations from cardiac signal and respiration. The relative changes in the hemoglobin signal were denoted in arbitrary units of mM × mm. The color maps of our cohort were shown in **Figures 2**, **3**, **4**.

A group analysis of fNIRS data was performed using the NIRS-SPM software (KAIST, Korea) operated in the MATLAB environment (Mathworks, Natick, MA, USA; Ye et al., 2009). This software enables an analysis similar to the statistical parametric mapping of fMRI. In the analyzing process, signal distortion due to breathing or movement of the subject was first corrected by hemodynamic response function and the wavelet minimum description length detrending algorithm incorporated in the NIRS-SPM software. A predicted model of HbO and HHb response was then generated based on the general linear model using data from the experimental condition. Using the same spatial information as that in the spatial registration for the individual analysis, the NIRS-SPM aligned the mean optode positions of six participants according to the Montreal Neurological Institute's standardized brain coordinate system (**Figure 1F**). The activation map based on the Hemoglobin level changes was then computed for the standardized brain. SPM t-statistic maps were produced, and HbO levels were considered significant at the uncorrected threshold of p < 0.01. Left/right information was flipped in the right-sided DBS cases for group analysis.

### Statistical Analysis of Clinical Outcomes

We performed a Wilcoxon signed rank test to compare the pre- and post-DBS UPDRS off medication scores. For the

FIGURE 1 | Probe position on the head and estimated channel position on the individual and model brains. (A) Anterior view of the head with the cap. The arrow indicates the nasion where a landmark is for spatial registration. (B) Lateral view from the left. Probes were attached to 16 holes in the highlighted area. The arrow indicates the left tragus. (C) Lateral view from the right. Probes were attached to 16 holes in the highlighted area. The arrow indicates the right tragus. (D) Schema of the probe positioning. L, light source; D, detector; Ch, channel. (E) Estimated channel position on the 3-D model of the patient's brain created from a volumetric T1-weighted image. Red and blue dots indicate the light sources and detectors, respectively. (F) Estimated channel position (yellow squares) on the model brain based on the Montreal Neurological Institute coordinate system.

FIGURE 2 | Comparison between pre- and post-deep brain stimulation (DBS) cortical activity in a representative case. The color map representing cortical activity superimposed onto the three-dimensional brain model was produced from the patient's T1-weighted magnetic resonance imaging (MRI) image. Red and blue dots indicate the light sources and detectors, respectively. Yellow dots are channels formed by the combinations of the light sources and receivers. These images demonstrate the dynamic oxy-hemoglobin (HbO) response to motor task at 0, 5, 10 and 15 s. At 0 s, the levels of the measured areas were all corrected to zero. Cortical activity in the wide areas of operated-upon hemisphere increased following DBS surgery.

statistical analysis, we used SPSS version 21.0 (IBM Corp., Armonk, NY, USA).

#### RESULTS

#### Clinical Outcomes

Preoperative motor evaluation showed 58.8 ± 11.1% improvement on the levodopa challenge test, as total scores on the UPDRS motor scale were 48.5 ± 11.1 and 19.7 ± 5.6 in the off- and on-medication states, respectively. Postoperatively, total UPDRS motor scale scores had improved to 26.8 ± 16.6 (p < 0.05) and 22.2 ± 8.6 (p < 0.05) at 1- and 6-month follow-up, respectively. These clinical outcomes and DBS programs at the point of the fNIRS experiment are summarized in **Table 2**. As for adverse events, only one patient (case 5) had wound dehiscence and underwent debridement. Otherwise, no serious adverse events were reported.

# Representative Case (Patient 3)

Patient 3 was a woman of 65 years of age when she underwent left unilateral GPI DBS. Her baseline UPDRS motor scores were 17 and 55 in the on- and off-medication states, respectively. Preoperatively, her main problems were festination in the off-medication state, on/off motor fluctuation, and leg tremor worst on the right side. Her motor symptoms dramatically improved following DBS surgery, and her postoperative UPDRS motor scores were 5 and 18 with stimulation in the off-medication state at 1- and 6-month follow-up sessions, respectively. In the fNIRS experiment, motor performance in this case was changed from 2 (moderately impaired) to 1 (mildly impaired) as shown in the UPDRS subscore (item 24). NIRS showed that her HbO levels were higher in the operated-upon left hemisphere, postoperative changes seemed to occur in tandem with the improvement of motor function (**Figure 2**).

#### Individual Analysis

The motor task performance was scored by UPDRS item 24 (hand movement score). The task performance score ranges from 0 to 4 with four being the worst symptom, and the results are summarized in **Table 2**. The task performance was improved in all cases except for case 6. Based on the HbO levels, there seemed to be a tendency that the cortical activity in the operated hemisphere was more activated compared to pre-operative state (**Figure 3**). On the other hand, color maps of HHb levels failed to show as dramatic changes as those of HbO levels (**Figure 4**). Even though the motor tasks were different, changes in the HHb levels

#### TABLE 2 | Clinical outcomes.


The numbers in the parentheses at baseline and 1 month are item 24 (hand movement subscore) of the UPDRS on the treated side of the body. The subscore ranges 0–4 (0-Normal; 1-Mild slowing and/or reduction in amplitude; 2-Moderately impaired. May have occasional arrests in movement; 3-Severely impaired. Frequent hesitation in initiating movements or arrests in ongoing movement; 4-Can barely perform task). "C" represents "case" for monopolar stimulation. <sup>∗</sup>These postoperative scores were significantly improved compared to baseline off-medication score. A Wilcoxon signed rank test was performed for the statistical analysis.

in our experiment seemed to be consistent with previous studies using a similar NIRS system that have not shown significant changes in HHb levels in association with the motor task (Mihara et al., 2012; Fujimoto et al., 2014).

#### Group Analysis

The group analysis using NIRS-SPM showed that, cortical activity based on the HbO level changes was shown to be of almost equal intensity throughout the investigated areas prior to DBS surgery. However, the cortical activity associated with movement of the hand was more focused in the corresponding motor area following surgery. The motor-related activity was more focused in the motor cortex, postoperatively (**Figure 5**). T-statistics of HHb level changes failed to show significant voxel on the map.

#### DISCUSSION

We investigated the cortical activity changes in both hemispheres of patients with PD that were received unilateral therapeutic

FIGURE 5 | Results of near infrared spectroscopy (NIRS)-SPM group analysis. Cortical activity, as determined by NIRS-SPM, was relatively increased in the primary motor area in the operated-upon hemisphere postoperatively. Higher t-values represent relatively higher cortical activity than areas with lower t-values in this figure.

GPI DBS, based on a 6-month follow-up of clinical outcomes. Previous studies have shown that remote areas with functional connectivity to the subthalamic nucleus or GPI could be modulated using electrical stimulation (Hashimoto et al., 2003; de Hemptinne et al., 2015). Recent neurophysiological studies have shown that there is an abnormal synchronization of the beta-band oscillation between the subthalamic nucleus and the motor cortex in PD (de Hemptinne et al., 2015; Kondylis et al., 2016), and that DBS may modulate the global network of the brain. Most of these studies have investigated patients with PD who received bilateral DBS of the subthalamic nucleus. Our data also suggests that DBS may modulate the global networks involved in motor activities. As a novelty of this study, we believe that investigated the effects of unilateral DBS, and this may be preferable as a means of revealing the influence of each DBS site on the global network.

The combined use of NIRS and DBS has been investigated in only one other study. However, while this previous study by Sakatani et al. (1999) showed increased levels in the prefrontal areas with increased electrical intensity of DBS from 0 to 10 volts in subjects with PD, merely increasing stimulation intensity is no longer considered to be of therapeutic benefit. Additionally, this previous study used a single-channel NIRS system. Our study aimed to exceed these standards by detecting task-related cortical activities with therapeutic DBS conditions using the multi-channel NIRS system, yielding clinically-relevant imaging results of superior spatial resolution.

Previous task-related fMRI studies showed decreased activity in the supplementary motor area and relative hyperactivity in the primary sensorimotor and premotor areas of subjects with PD compared to healthy controls (Sabatini et al., 2000; Haslinger et al., 2001). Hypoactivation of the sensorimotor area was considered to be related to the festination to initiate voluntary movements. Our data show a relative increase in the activity of the primary motor areas of the ipsi-DBS side with clinical improvement as shown by the changes in HbO levels. Moreover, our results are consistent with previous fMRI studies.

One noteworthy advantage of fNIRS in clinical use is its ability to provide patients with real-time visual feedback of cortical activity, although this feature was not used in the present study. Several studies have investigated the potential of a real-time neurofeedback system for neurorehabilitation (Holper and Wolf, 2010; Fazli et al., 2012; Mihara et al., 2013). The real-time visualization of cortical activity may be useful for intraoperative monitoring. Recent studies emphasize the benefits of a closed-loop DBS system, but the best biomarker to detect changes in brain activity remains to be determined (Johnson et al., 2016; Rossi et al., 2016). However, a multi-channel NIRS system might spur the development of a new generation of DBS systems. Our data demonstrated the potential of fNIRS to detect cortical activity, as we hypothesized. Further studies are warranted to develop the system for real-time neurofeedback during DBS surgery.

In the present study, we showed the potential of the NIRS system for investigating patients with PD who have undergone DBS surgery. Nonetheless, there are important limitations of our study. First, current NIRS systems are unable to investigate the activity of subcortical areas such as the basal ganglia. Second, several studies have investigated the effects of a microlesion following DBS surgery (Mann et al., 2009; Morishita et al., 2010; Okun et al., 2012), and a few studies determined that the microlesion effect may postoperatively continue for up to 6 months (Mann et al., 2009; Morishita et al., 2010). The NIRS data reported here was obtained at a 1-month follow-up, thus we cannot exclude the possibility that the NIRS findings that we observed may represent changes in cortical activity that are attributable to the microlesion effect. Third, it should be noted that our NIRS findings may represent an improvement in the produced movement itself, and this question should be addressed in a future study. Such an improvement would indicate that the fNIRS system allows for a more objective evaluation of the clinical effect of therapeutic DBS compared to other clinical scales. Fourth, we did not perform sophisticated statistical analysis to address false discovery rate, Bonferonni correction and controlling type I error in this pilot study (Tak and Ye, 2014). Finally, as only six PD cases were included in our study, and we have not related the performance status with the fNIRS data. Further studies with larger samples and more sophisticated study design are warranted to confirm the reproducibility of our experiment.

#### CONCLUSION

To our knowledge, our study is the first to use a multichannel NIRS system to measure the changes in cortical activity following unilateral GPI DBS. In this pilot study, we

#### REFERENCES


show the changes in the motor-associated cortical activity following DBS surgery. Moreover, we conclude that therapeutic DBS promoted the remodeling of neuronal networks and changes in cortical activity in association with symptomatic improvements.

#### AUTHOR CONTRIBUTIONS

TM contributed to conception of the study and data collections, and wrote the manuscript. MH collected the data, and reviewed the manuscript. KS collected and analyzed the data. YT reviewed the manuscript. HA reviewed the manuscript. TI supervised the study and reviewed the manuscript.

#### ACKNOWLEDGMENTS

This study was partly supported by the Japan Society for the Promotion of Science Grant-in-Aid for young scientists (B) 15K19984, the Takeda Science Foundation, the Uehara Memorial Foundation, and the Central Research Institute of Fukuoka University (No. 161042). We also appreciate Dr. Hisatomi Arima for supervising statistical analysis and Ms. Asuka Ikezaki for the statistical analysis.


Zrinzo, L., Yoshida, F., Hariz, M. I., Thornton, J., Foltynie, T., Yousry, T. A., et al. (2011). Clinical safety of brain magnetic resonance imaging with implanted deep brain stimulation hardware: large case series and review of the literature. World Neurosurg. 76, 164–172; discussion 169–173. doi: 10.1016/j.wneu.2011. 02.029

**Conflict of Interest Statement**: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2016 Morishita, Higuchi, Saita, Tsuboi, Abe and Inoue. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution and reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Probing the Role of Medication, DBS Electrode Position, and Antidromic Activation on Impulsivity Using a Computational Model of Basal Ganglia

#### Alekhya Mandali and V. Srinivasa Chakravarthy \*

*Computational Neuroscience Lab, Department of Biotechnology, Indian Institute of Technology Madras, Chennai, India*

Everyday, we encounter situations where available choices are nearly equally rewarding (high conflict) calling for some tough decision making. Experimental recordings showed that the activity of Sub Thalamic Nucleus (STN) increases during such situations providing the extra time needed to make the right decision, teasing apart the most rewarding choice from the runner up closely trailing behind. This prolonged deliberation necessary for decision making under high conflict was absent in Parkinson's disease (PD) patients who underwent Deep Brain Stimulation (DBS) surgery of STN. In an attempt to understand the underlying cause of such adverse response, we built a 2D spiking network model (50×50 lattice) of Basal ganglia incorporating the key nuclei. Using the model we studied the Probabilistic learning task (PLT) in untreated, treated (L-Dopa and Dopamine Agonist) and STN-DBS PD conditions. Based on the experimental observation that dopaminergic activity is analogous to temporal difference (TD) and induces cortico-striatal plasticity, we introduced learning in the cortico-striatal weights. The results show that healthy and untreated conditions of PD model were able to more or less equally select (avoid) the rewarding (punitive) choice, a behavior that was absent in treated PD condition. The time taken to select a choice in high conflict trials was high in normal condition, which is in agreement with experimental results. The treated PD (Dopamine Agonist) patients made impulsive decisions (small reaction time) which in turn led to poor performance. The underlying cause of the observed impulsivity in DBS patients was studied in the model by (1) varying the electrode position within STN, (2) causing antidromic activation of GPe neurons. The effect of electrode position on reaction time was analyzed by studying the activity of STN neurons where, a decrease in STN neural activity was observed for certain electrode positions. We also observed that a higher antidromic activation of GPe neurons does not impact the learning ability but decreases reaction time as reported in DBS patients. These results suggest a probable role of electrode and antidromic activation in modulating the STN activity and eventually affecting the patient's performance on PLT.

Keywords: deep brain stimulation, parkinson's disease, subthalamic nucleus, impulsivity, electrode position, spiking neuron model, reinforcement learning

#### Edited by:

*Daniela S. Andres, ETH Zurich, Switzerland*

#### Reviewed by:

*Maria Herrojo Ruiz, Goldsmiths, University of London, UK Fabiano Ferrari, Universidade Federal dos Vales do Jequitinhonha e Mucuri, Brazil*

> \*Correspondence: *V. Srinivasa Chakravarthy schakra@iitm.ac.in*

Received: *16 March 2016* Accepted: *25 August 2016* Published: *12 September 2016*

#### Citation:

*Mandali A and Chakravarthy VS (2016) Probing the Role of Medication, DBS Electrode Position, and Antidromic Activation on Impulsivity Using a Computational Model of Basal Ganglia. Front. Hum. Neurosci. 10:450. doi: 10.3389/fnhum.2016.00450*

# INTRODUCTION

Parkinson's disease (PD) is a neurodegenerative disorder known to be caused due to the death of dopaminergic neurons in the mid-brain structure called Substantia Nigra pars compacta (SNc) (Obeso et al., 2008) of Basal Ganglia (BG). Apart from the visible motor symptoms such as bradykinesia, rigidity and tremor (Xia and Mao, 2012), cognitive functions of PD patients are also affected (Lees and Smith, 1983; Levin and Katzen, 1994). As an initial treatment, pharmacological medication in the form of dopamine (DA) precursor (L-DOPA) and/or Dopamine agonists (DAA) are prescribed to PD patients (Connolly and Lang, 2014). But it has been observed that the "ON" time (where the medication is effective in relieving the symptoms) decreases as the disease progresses and 80% of the patients develop L-DOPA induced dyskinesias as a side effect (Schrag and Quinn, 2000). Under these circumstances, surgical intervention through Deep Brain Stimulation (DBS) is advised as an alternative treatment wherein an electrode is implanted and external stimulation is given to one or more nuclei of the brain. Though stimulation to the Sub Thalamic Nucleus (STN) of BG is widely followed as the gold standard for PD (Garcia et al., 2005) due to its effectiveness in alleviating the motor symptoms, various experimental studies show a controversial effect of DBS on cognition (Jahanshahi et al., 2000) particularly on impulsivity (Frank et al., 2007; Smeding et al., 2009; Brittain et al., 2012).

Among various experimental paradigms used to study the cognitive ability of PD patients, probabilistic learning task (PLT) (Frank et al., 2004, 2007) captures decision-making ability as well as the impulsivity features. PLT tests the learning capability of the performer not only in choosing rewarding choices but also in avoiding punishing ones. Experimental results show that the performance of normals and PD OFF subjects during PLT is similar in terms of choosing rewarding and avoiding punishing choice (Frank et al., 2007). Contrastingly, the results from the same research group showed a bias toward punishment learning, i.e., the PD OFF subjects learnt better to avoid punitive choice than to choose rewarding choice (Frank et al., 2004) during PLT. The performance of PD ON subjects was opposite to that of PD OFF with a preference toward the rewarding choice, which was accounted by the presence of excess DA levels in the striatum due to medication. This excess DA (due to medication) prevents the PD subjects to learn from punishments. Another critical feature captured by PLT is the reaction time (RT). It has been observed that normal subjects take more time when presented with multiple equally rewarding stimuli (high conflict) and are expected to choose one among them (Frank et al., 2007). Frank et al. (2007) hypothesized that STN increases its activity and buys the extra time needed ("holding the horses") during such situations. This was further shown by Zaghloul et al. (2012), where an increase in STN activity in PD patients during high conflict conditions was observed (Zaghloul et al., 2012). Experiments conducted by Frank et al. (2007) showed that the performance of DBS subjects on PLT was not significantly different in terms of learning ability but showed impulsive behavior in terms of RT.

Various clinical and experimental studies suggest that the stimulation of STN neurons could lead to a decline in cognitive functions of PD patients (Saint-Cyr et al., 2000; Smeding et al., 2006; Temel et al., 2006; Smeding et al., 2009). Stimulation parameters such as electrode position, pulse frequency and current amplitude seem to play a critical role in altering behavior (Hershey et al., 2004). Using a computational model of subjects/patients performing the Iowa Gambling Task, we earlier showed that the performance of the model in PD with DBS condition showed impulsive behavior, which was further dependent on the position of the electrode and amplitude of the stimulating current (Mandali and Chakravarthy, 2015). Using the same computational BG model (Mandali et al., 2015), we now study the effect of DBS parameters on performance in PLT in terms of accuracy and RT. PLT was simulated using reinforcement learning (RL) framework (Sutton and Barto, 1998), where the temporal difference error term (δ) is hypothesized to resemble the phasic DA released by dopaminergic cells in the midbrain (Schultz et al., 1997).

The aim of this study is two-fold, first to show that the spiking BG model is able to replicate the performance of normal, PD OFF, PD ON (L-DOPA) conditions as in experimental studies (Frank et al., 2007) and secondly to hypothesize the effect of DAA, DBS electrode and antidromic activation on learning, impulsivity and behavior.

# MATERIALS AND METHODS

We used the spiking neuron model of the Basal Ganglia (BG) (Mandali et al., 2015) in normals, PD OFF, PD ON (L-DOPA and DAA), and DBS (electrode position and antidromic) conditions to simulate PLT (Frank et al., 2004, 2007). The various performance measures used to validate the model results are also introduced in this section.

# Spiking Neuron Model of Basal Ganglia

The network model of BG (Mandali et al., 2015) (**Figure 1**) was built using 2-variable Izhikevich spiking neurons (Izhikevich, 2003) where each nucleus was modeled as a 2D array of neurons. Parameters for each of the nuclei [STN, Globus Pallidus externa (GPe), and interna (GPi) were chosen; (Mandali and Chakravarthy, 2015; Mandali et al., 2015)] to resemble their biological counterparts. STN and GPe neurons are bidirectionally connected (Plenz and Kital, 1999) in one-to-one fashion where GPe (STN) projections are inhibitory (excitatory). The striatum which receives input from the cortex (Tritsch and Sabatini, 2012; Silberberg and Bolam, 2015) consists of both D1R-expressing and D2-R expressing medium spiny neurons (MSNs) and was segregated based on the classical anatomical classification of direct and indirect pathways (Gerfen and Surmeier, 2011) and was modeled as Poisson spike trains modulated by DA levels. Each GPi neuron receives both glutamatergic projection from STN and GABAergic projection from D1 MSN. Similarly, each GPe neuron receives GABAergic input from D2 MSN and glutamatergic from STN neuron. The full set of equations related to the Izhikevich spiking

neuron model are described in Appendix A and Table A.I (Supplementary Materials). The input from the cortex to STN, also known as the hyper-direct pathway (Nambu, 2015), and the GABAergic projection from GPe to GPi were not included in the model as the functional significance of these connections is not fully understood.

dv<sup>x</sup> ij dt <sup>=</sup> 0.04(<sup>v</sup> x ij) <sup>2</sup> + 5v x ij − u x ij + 140 + I x ij + I syn ij + I DBS ij (1) du<sup>x</sup>

$$\frac{d\boldsymbol{u}\_{ij}}{dt} = \boldsymbol{a}(b\boldsymbol{\nu}\_{ij}^{\boldsymbol{\times}} - \boldsymbol{u}\_{ij}^{\boldsymbol{\times}}) \tag{2}$$

$$\text{if } \nu\_{ij}^{\chi} \succeq \nu\_{peak} \left\{ \begin{aligned} \nu\_{ij}^{\chi} &\leftarrow c \\ \mu\_{ij}^{\chi} &\leftarrow \mu\_{ij}^{\chi} + d \end{aligned} \right\} \tag{3}$$

where, v x ij = membrane potential, u x ij = membrane recovery variable, I Syn ij = total synaptic current received, I x ij = external current applied to neuron x at location (i, j), vpeak = maximum voltage set to neuron (+30 mv) with x is a generalized notation denoting either STN, GPe or GPi neuron and I DBS ij is the stimulation current applied only for STN neurons (defined in the next Section Simulating Various Conditions in the Model).

#### Simulating Various Conditions in the Model

The methods and parameters used in the model to simulate the task in normal, PD OFF, PD ON (L-DOPA and DAA), and DBS conditions are explained in this section.

#### Normals

To simulate the normal condition, the direct pathway (D1 MSN to GPi) weight in spiking BG model was kept high (wStr→D1 = 4) and the weight from STN to GPi low (wSTN→GPi = 1.5). The lateral weights within STN and GPe were kept at wsg = 0.91 and wgs = 18 based on the experimental evidence that there is high amount of inhibition from GPe to STN in normal conditions (Wilson and Bevan, 2011). The radius of neighborhood of connectivity in STN (r<sup>s</sup> = 1.4) and GPe (r<sup>g</sup> = 1.6) which controls the level of synchrony in the nuclei, is chosen such that the STN-GPe system exhibits desynchronized dynamics as observed in the normal healthy condition (Bergman et al., 1998; Wilson and Bevan, 2011).

#### Parkinsonian Condition in "OFF" and "ON" State **PD OFF state**

Bearing in mind that the TD error (δ) is similar to the DA activity (O'doherty et al., 2003; Schultz, 2007; Rolls et al., 2008) and there is loss of DA neurons in PD, we simulated it by clamping the "δ" value to a low limit (δlim = −0.1) representing a loss of dopaminergic neurons in PD condition (Equation 4).

$$
\delta\_{\rm lim} = \min(\delta, DA\_{ceil}) \tag{4}
$$

where <sup>z</sup> = min(y, <sup>a</sup>)is defined as <sup>z</sup> <sup>=</sup> y if y <sup>&</sup>lt; <sup>a</sup> a if y > a

We also decreased the direct pathway weight (wStr→D<sup>1</sup> = 3) which represents a decreased inhibitory output from D1 striatum to GPi and increased STN to GPi weight (wSTN→GPi = 2) representing an increase in the excitatory input from STN. The remaining parameters were not varied.

#### **PD ON state**

PD "ON" medication clinically involves external intake of dopamine precursors such as L-DOPA. A simple way to simulate oral medication is to add a "δmed" term to the δlim term (Equation 4) (Magdoom et al., 2011; Muralidharan et al., 2013; Mandali and Chakravarthy, 2015).

$$
\delta\_{\text{new}} = \delta\_{\text{lim}} + \delta\_{\text{med}} \tag{5}
$$

Another class of medication that is prescribed to PD patients is Dopamine Agonists (DAAs), which could be receptor specific. Here we simulated DAA action in such a way that it precisely effects D2 class of receptors, which are also linked to impulsivity (Macmahon and Macphee, 2008). Therefore, in case of DAA, the parameter δnew in Equation (5) will be used to update only D2 cortico-striatal weight (wD2, Equation 8) unlike for L-DOPA where both wD1 and wD2 were updated. All the parameter values of the model are kept the same as in PD OFF state, except the weight parameters which are reverted to normal. This is one among many approaches used to simulate the effect of dopaminergic medication. The medication is added to the model as described in Equation (5) where (δmed = 2) is added to the clamped delta (δlim = −0.1). The two types of dopaminergic medications (L-DOPA and DAA) differ only in terms of weight update as described in next Section (Simulating PLT Using Spiking BG Model).

#### DBS Stimulation

The effect of DBS on the STN neurons was modeled by giving an external stimulation current (Equation 6). The parameters (frequency, pulse duration, and amplitude) of the stimulation current are chosen such that they are comparable to that used in a clinical setting (Garcia et al., 2005). The stimulation current is given to the entire/part of STN module (50 × 50 neurons) in the form of Gaussian distribution (Hauptmann and Tass, 2007; Foutz and McIntyre, 2010; Mandali and Chakravarthy, 2015). The mean of the Gaussian coincides with the lattice position (i<sup>c</sup> , jc) which is assumed to be the center of the electrode and the extent of the current spread is controlled by the Gaussian width (σ).

$$I\_{ij}^{DBS} = A^\* e^{\frac{-((i-i\_c)^2 + (j-j\_c)^2)}{\sigma^2}} \tag{6}$$

where I DBS ij is the DBS current received by the STN neuron at position (i, j), A is the amplitude of the current (pA), σ controls the spread of the current, and (i<sup>c</sup> , jc) is the mean/center point of the electrode. The effect of electrode position (i<sup>c</sup> , jc) and stimulation parameters A and σ on STN activity and on decision making behavior is simulated.

All the synaptic weight values are kept similar to that of PD OFF condition and external current (IDBS) as described in Equation (6) was added to the STN neurons. The current was applied at a frequency of 130 Hz, mostly monophasic mode with pulse duration = 100µS, the spread of the current σ = 5 and amplitude of the current around 220 pA with the electrode center at the lattice point (25, 25).

#### **Electrode position**

Experimental results show that change in the electrode position alters behavior (Hershey et al., 2004, 2010) and this can be attributed to the difference in pattern and volume of STN activation due to the electrode position (Miocinovic et al., 2006). Also, the final action or choice selection depends on the activity of GPi neurons which receive weighted input from STN and D1R-expressing MSNs. Bearing these points in mind, we chose three electrode positions where the lattice point indicates the center of the electrode, i.e., Pos 1 in the upper half of the STN nucleus at lattice point (13, 13), Pos 2 with electrode contact center at the lattice point (25, 25), and Pos 3 in the lower half of the STN nucleus at lattice point (38, 38). Each module (StrD1, StrD2, GPe, and STN) in the model is divided into four quadrants corresponding respectively to the four panels in the PLT. This is a modeling assumption that has to be made in the absence of experimental data about how the four action choices might be represented in the basal ganglia nuclei. The electrode position that we study in the model is also described with reference to such representations. Thus, the four quadrants in the modules do not correspond to the well-known basal ganglia loops like sensorimotor, associative, limbic etc.

#### **Antidromic activation**

Based on theories that stimulation of STN could result in antidromic activation of GPe, GPi, or cortical neurons (Hauptmann and Tass, 2007; Montgomery and Gale, 2008), we studied the effect of antidromic activation of GPe neurons during the task. This effect was modeled by adding a percentage of DBS current (given to STN neurons) directly to GPe neurons. The antidromic activation of GPe neurons in the network was simulated by providing certain percentage of DBS current (Equation 6) to the GPe neurons. For example, 25% antidromic activation would have 25% of the DBS current (I DBS ij ) added to the membrane potential equation (Equation 1) of GPe neurons and the remaining 75% to STN neurons.

#### Probabilistic Learning Task (PLT)

The experiment consists of two stages, training and testing (**Figure 2**). During the training stage, the model was presented only with three pairs of stimuli (AB/CD/EF) one at a time in a random fashion. Each of the six choices (A/B/C/D/E/F) was associated with a reward with a priori probability. For example, selection of choice "A" leads to reward (= +1) 80% of the time whereas choice "B" leads to a reward only 20% of the time. Similarly, choice "C" ("E") gives reward with a probability of 70% (60%) and choice "D" ("F") leads to reward only 30% (40%) of the time and punishment (= −1) for rest of the trials. The model was expected to learn these reward probabilities by the end of training.

During the testing stage, the model was tested with 15 novel combinations (e.g., AC, CE, DE) which were not presented during the training stage. No feedback was provided for the response made after each stimulus. The model was tested for its learning ability based on whether it chose (avoided) a rewarding (punishing) choice from the presented combination pair. For example, if a novel combination of choice "A" with another choice was presented; the model was expected to choose "A" as the probability of obtaining a reward was the highest for "A." Similarly, when the stimuli with combination of "B" with other choices were presented, the model is expected to avoid selecting "B" as its reward probability was the lowest. Apart from testing for the learning ability, the model was also tested for performance

during High conflict (HC) and Low conflict (LC) situations. For example, the stimulus combination "AC" falls under the category of HC as both choice "A" (80%) and "C" (70%) have high reward probabilities but stimulus combination "BC" comes under the category LC as reward probabilities ("B" = 20% and "C" = 70%) are significantly different. The reaction time was measured for each of the conditions (HC/LC).

During this stage, the model was tested for the following conditions:


#### Simulating PLT Using Spiking BG model

During training and testing phase, a pair of choices was presented out of the six choices in each trial, so the input to the BG model was also given as a set of two inputs. Every nucleus in the model was divided equally into two parts which receive the corresponding input. The expected reward probability of the corresponding input was learnt in the cortico-striatal weights. Based on the experimental evidence that striatal neural spiking activity is irregular (Reti, 2015), the input to GPe and GPi (i.e., the output of D2R- and D1R-expressing striatal MSNs) was modeled as Poisson spike trains whose frequency is proportional to the cortico-striatal weight (w D1 i,k ,w D2 i,k ) of the corresponding stimulus pair (i) and trial (k). Since the rate of firing of the striatal neurons was observed to be between 2 and 40 Hz (Kravitz et al., 2010), the cortico-striatal weight of the individual card was normalized to fall in the above range. Since release of DA is known to modulate plasticity (Surmeier et al., 2007) in cortico-striatal connections, in the present model, the temporal difference error term "δ" was used to update the corticostriatal synapses (Reynolds and Wickens, 2002; Surmeier et al., 2007). The synaptic weights between all other nuclei were not plastic and were changed only depending on the physiological condition.

#### **Cortico striatal weight update and temporal difference error**

Each choice (A/B/C/D/E/F) was associated with 2 weights (w D1 i,0 ,w D2 i,0 ) which were initialized with random values selected from a uniform distribution over (0, 1). The two weights represent the cortico-striatal weights of D1R and D2R-expressing striatal MSNs and are trained as,

$$
\Delta \boldsymbol{w}\_{i,k}^{D1} = \eta \boldsymbol{\delta}\_k \boldsymbol{x}\_{i,k}^{inp} \tag{7}
$$

$$
\Delta \boldsymbol{\omega}\_{i,k}^{D2} = -\eta \boldsymbol{\delta}\_k \boldsymbol{\omega}\_{i,k}^{imp} \tag{8}
$$

The expected value (V<sup>k</sup> ) for kth trial, which is expressed in terms of the activity of D1R-expressing MSNs (Chakravarthy et al., 2010; Muralidharan et al., 2013; Mandali and Chakravarthy, 2015; Mandali et al., 2015), is calculated as

$$\mathbb{V}\_{\mathbb{k}} = \sum\_{i=1}^{6} \mathbb{w}\_{i,k}^{D1} \ast \mathbb{x}\_{i,k}^{sel} \tag{9}$$

The gain or reward (Re<sup>k</sup> ) for kth trial is calculated as

$$\text{Re}\_{\mathbf{k}} = \sum\_{i=1}^{6} r\_{i,\mathbf{k}} \bowtie\_{i,\mathbf{k}} \mathbf{x}\_{i,\mathbf{k}}^{sel} \tag{10}$$

The error (δ) for kth trial is defined as

$$
\delta\_k = \mathcal{R}e\_k - V\_k \tag{11}
$$

where,

w D1 i,k are the cortico-striatal weights of D1 striatum for ith card in kth trial, w D2 i,k are the cortico-striatal weights of D2 striatum for ith card for kth trial,

Card "i" represents one of the six cards (A/B/C/D/E/F)

ri,<sup>k</sup> is the reward obtained for the selected ith card in kth trial x inp is the binary input vector representing the choices presented to the model each time, e.g., if the stimulus presented is CD, xinp = [0 0 1 1 0 0] and xsel is the binary vector representing the choice that got selected; if "C" is selected then xsel = [0 0 1 0 0 0]; η (=0.1) is the learning rate of the cortico-striatal synapses of D1 and D2 MSNs; V<sup>k</sup> is the expected value for the selected card for kth trial.

#### Performance Measures

In this section, we explain all the performance measures used in this study to quantify and validate the results obtained from the model for all the conditions.

#### Learning

The model was trained for 120 trials [=40 per combination (AB/CD/EF)] and the learning ability of the model was checked during the training stage in terms of training accuracy where the probability of selecting the correct choice was plotted as the training progressed (trials were divided into five equal bins). The performance of the model was compared with the results (Figure 2A from Zaghloul et al., 2012).

#### Testing Accuracy and Difference in Reward Expectation (DRE)

#### **Difference in reward expectation (DRE)**

After training, the a priori choice selection probability was calculated based on the number of times the corresponding choice was presented and selected. We then calculated the Difference in Reward Expectation (DRE), which is the difference between the 2 apriori choice probabilities for that particular presented stimulus. DRE captures the amount of conflict between the presented choices, the higher (lower) the DRE for that stimulus the lower (higher) is the conflict. For example, if stimulus "BC" was presented then DREBC, which is the difference between P(B) and P(C), would be low, thereby reducing the probability of choice "B" getting selected.

#### **Testing accuracy**

Once the training phase is completed, the model was tested by presenting 15 novel combinations. The objective was to calculate the probability with which the first choice in the presented stimulus was selected. For example, if stimulus "AC" was presented for 20 times and choice "A" was selected for 16 times, then the testing accuracy for choice "A" would be 0.8 (=16/20).

The learning ability of a system to select the most rewarding choices while avoiding the punitive ones can be obtained by just evaluating the relationship between DRE and testing accuracy. For example, the testing accuracy (of choice "A") for the stimulus "AF" (whose DRE > 0) would be expected to be high because the reward probability associated with choice A is also high. So for an optimally trained system, one can expect a linear relationship between testing accuracy and DRE.

#### Choice/Avoidance Accuracy

This quantity measures the ability of the model to select the most rewarding option "A" and avoid the punitive choice "B" when presented with novel combinations not used during training.

#### Reaction Time

The final action selection was done at the level of thalamus which was simulated using the "race model" with mutual inhibition (Bogacz et al., 2006) where an action is selected when temporally integrated neuronal activity of the output neurons crosses a threshold (Frank, 2006; Frank et al., 2007; Humphries et al., 2012).

The dynamics of the thalamic neurons is as follows,

$$\begin{array}{l} \frac{dz\_1(t)}{dt} = -z\_1\left(t\right) + f\_{\mathbb{G}\bar{p}i1}(t) - z\_2\left(t\right) \\ \frac{dz\_2(t)}{dt} = -z\_2\left(t\right) + f\_{\mathbb{G}\bar{p}i2}(t) - z\_1\left(t\right) \end{array} \tag{12}$$

$$\begin{aligned} f\_{Gpik} &= \frac{1}{(N \ast N)/k} \sum\_{t=1}^{T} \left( \sum\_{i=1}^{N} \sum\_{j=1}^{N/k} S\_{ij}^{GPik} \{t\} \right) \\ f\_{GPik} &= \frac{f\_{GPi}^{\max} - f\_{Gpik}^{'}}{f\_{GPi}^{\max}} \end{aligned} \tag{13}$$

Mandali and Chakravarthy Modeling Impulsivity in Parkinson's Disease

where, z1(t), z2(t) = integrating variable for 1st and 2nd choice, fGPi1(t) and fGPi2(t) = normalized and reversed average firing frequency of GPi neurons receiving 1st and 2nd choice from striatum, f max GPi = highest firing rate among the GPi neurons, S GPik ij = neuronal spikes of GPi neurons receiving kth stimulus, N = number of neurons in a single row/column of GPi array (=50), T = duration of simulation.

The first neuron (z<sup>k</sup> ) among k stimuli to cross the threshold (=0.25) represents the action selected and "t" is the time instant when the action gets selected which is nothing but the RT. All the variables representing neuronal activity are reset immediately after each action selection.

#### RESULTS

The simulation study was performed to study various aspects of behavior in Normals, PD OFF, PD ON (L-DOPA and DAA), and DBS conditions. **Table 1** presents list of simulation (from the model) and experimental measures for various conditions.

#### Learning

As explained in Section Learning, we evaluated the training ability of the model for Normals, PD OFF, PD ON (L-DOPA and DAA), and DBS conditions. The training ability in PD OFF condition was compared with experimental results (Figure 2A of Zaghloul et al., 2012). As shown in **Figure 3**, the training accuracy levels (probability of choosing the particular choice during training) for choices "A," "C," "E" reach their actual reward probabilities (0.8/0.7/0.6) as the training progresses. **Table 2** shows the accuracy levels of choosing "A," "C," and "E" for all the above conditions.

By the end, the training accuracy for "A," "C," and "E" in normal, PD OFF and PD ON (L-DOPA) reached their reward probabilities (**Figures 3B–D**) showing the learning ability of the model in that condition. But PD ON (DAA) and DBS conditions (**Figures 3E,F**) showed lower training accuracy compared to other conditions.

# Testing Accuracy and Difference in Reward Expectation (DRE)

The model was then tested with 15 novel stimuli consisting of all the combinations of the choices (A–F). Linear regression was used to fit the testing accuracy as a function of DRE for most of the above conditions. The results of PD OFF obtained from the model (**Figure 4B**) were compared with results (**Figure 2C**) from that of Zaghloul et al. (2012) (**Figure 4A**). The fit of the regression line for experiment (=0.81) and that obtained from simulation (=0.87). The testing accuracy obtained from experiment and simulation were compared using t-test and found not to be significantly different (p = 0.16). We also studied the same for PD medication "ON" conditions (both L-DOPA and DAA). As one can observe, the testing probability is confined to top left corner in PD ON (L-DOPA) condition (**Figure 4C**). The PD-ON (L-DOPA) condition did not show a linear relationship between testing accuracy and DRE. The DRE obtained for all the novel combinations did not cross 0.24 and the testing accuracy is high even in negative DRE conditions (**Figure 4D**). A good fit was not



\**Experimental literature not available for these conditions.*

*<sup>a</sup>Zaghloul et al., 2012.*

*<sup>b</sup>Frank et al., 2007.*

obtained for the L-DOPA condition, the better polynomial (with order 2) is reported below. This suggests that L-DOPA interferes with training leading to wrong estimation of the reward. We also checked the same for normal condition (**Figure 4E**) and the results showed a low testing accuracy for low DRE and vice versa with a regression of (=0.94). The testing accuracy for DBS condition (**Figure 4F**) also showed a better correlation only for a polynomial fit of order 5 (=0.47). **Table 3** lists the goodness of fit obtained from the model for testing accuracies and DRE for all the conditions.

Apart from studying the relationship between testing accuracy and DRE (in terms of regression fit), we wanted to know if there was fundamental difference in absolute values of the testing accuracies obtained from each of these conditions. So we conducted one way ANOVA and found the testing accuracies to be significantly different at p = 0.05 level with [F(2, 40) = 3.62, p = 0.03].

The training and testing accuracy results from the model show its ability to capture the behavior of PD patients. The testing accuracy results (with DRE) suggest how medication and DBS could affect the decision making ability.

#### Choice/Avoidance Accuracy

To this end, the model was presented with novel combinations of choices "A" and "B" similar to that mentioned in the earlier Section Choice/Avoidance Accuracy. The test was implemented

TABLE 2 | Shows the training accuracy levels for choosing "A" [P (A)], choosing "C" [P(C)], and choosing "E" [P(E)] in normal, PD OFF, PD ON (L-DOPA and DAA), and DBS conditions.


on normals, PD OFF, PD ON (L-DOPA and DAA) (**Figures 5**, **6**), and DBS conditions (**Figure 7**).

#### Normal and PD Conditions

The results [normals, PD OFF, PD ON (L-DOPA)] obtained from the model were compared with that of from Frank et al. (2007). The results obtained from experiment (**Figure 5A**) and simulations (**Figure 5B**) were found to be similar. The mean accuracy (in terms of choosing "A" and avoiding "B") with standard error (SE) obtained from the computational model in normal condition for choosing A (avoiding B) was 82.98 ± 2.68 (78.29 ± 5.08), PD OFF was 82.42 ± 2.03 (81.25 ± 7.39), PD ON (L-DOPA) was 73.29 ± 3.13 (51 ± 2.56), and PD ON-DAA was 56.1 ± 8 (57.5 ± 8.05). The mean performance calculated in selecting choice "A" among the four conditions were found to be significantly different at p = 0.05 level at [F(4, 23) = 5.53, P = 0.003]. Post-hoc analysis using Bonferroni method showed a significant difference between PD ON-DAA condition and normals (P = 0.004) and PD OFF (P = 0.04). Similarly, a significant difference was observed among the 4 conditions for avoiding choice "B" at P = 0.05 level at [F(4, 23) = 4.49, P = 0.01]. Post-hoc analysis using Bonferroni method showed a significant difference only between normals and PD ON (L-DOPA) condition (P = 0.03).

The PLT study conducted by Frank and group on 2 sets of PD OFF patient groups showed opposite results in terms of avoiding "B" choice. One group's performance was similar to normals (Frank et al., 2007) but the other patient pool showed a bias toward punishment learning (Frank et al., 2004), i.e., they learnt to avoid the punitive choice "B" better than to select the rewarding choice "A." It was quite intriguing to observe that a similar patient pool exhibited two contrasting behaviors. In order to analyse this, we checked the effect of various model parameters on avoiding "B" performance. We observed that the lateral connectivity parameter within STN (rs) and GPe (r<sup>g</sup> ) neurons played a critical role in altering the behavior. We used 2 set of values for lateral strengths of STN and GPe, i.e., [Condt1 = (r<sup>s</sup> = 3.3) and (r<sup>g</sup> = 0.7)], [Condt2 = (r<sup>s</sup> = 1.43) and (r<sup>g</sup> = 1.7)]. We observed that the performance obtained using Condt1 values was similar to patient behavior that showed no bias to reward and punishment whereas Condt2 values showed punishment biased behavior (**Figure 6**).

#### STN Stimulation

Various experimental and clinical studies reported impulsivity in PD patients after stimulation of STN (Hershey et al., 2004; Smeding et al., 2006, 2009; Frank et al., 2007; Ballanger et al., 2009; Wylie et al., 2010) which was soon contradicted (Castrioto et al., 2015). Keeping this in mind we studied the effect of electrode position and antidromic activation on reward and punishment learning. Based on our earlier results that position of the electrode could be a potential factor for impulsivity in DBS subjects (Hershey et al., 2010; Mandali and Chakravarthy, 2015), we varied the position of the electrode and changed the percentage of antidromic activation keeping all other stimulation parameters constant.

#### **Electrode position**

As explained in Section Electrode position, three positions have been chosen (**Figure 7A**) and accuracy levels (in terms of choosing "A" and avoiding "B") have been calculated. The mean accuracy levels for Pos 1 in choosing A (Avoiding B) was 0 (100), for Pos 2 it was 53.125 ± 10.72 (55.85 ± 8.8), and for Pos 3 it was 100(0). As it can be observed from the plot (**Figure 7B**), the model performance is biased to either reward-based (Pos 3) or punishment-based learning (Pos 1) based on the position of the electrode. As the final choice selection is dependent on GPi activity which is partly controlled by STN, the stimulation current's ability to vary the STN activity influenced the final action selection.

#### **Antidromic activation effect**

We varied the percentage (10, 50, and 75%) of antidromic activation for a fixed position of electrode (Pos 2, **Figure 7A**), frequency (=130 Hz) and amplitude (200 pA) and observed the result in terms of the accuracy. The mean accuracy level in choosing A (avoiding B) for 10% was 66.4 ± 7.8 (32.81 ± 8.02), 50% was 53.9 ± 10.72 (56.09 ± 8.8), and for 75%, it was 58.5 ± 8.46 (55 ± 7.5) (**Figure 8**). We observed that for lower values of GPe activation, the model behaved similar to medication (L-DOPA), i.e., a bias toward reward learning but on further increase the model accuracy turned out to be similar for both reward and punishment learning.

The above results demonstrate the learning ability and performance (reward and punishment) under physiological and pathological condition. The simulation results suggest that change in STN dynamics, arising due to a change in STN lateral connection strength, seems to be a key contributing factor to altered behavior among PD patients. We also observed that stimulation parameters such as electrode position and antidromic activation are critical and influence reward and punishment learning.

#### Reaction Time and Impulsivity

As explained in Section Reaction Time, we calculated the RT for each of the five conditions [Normals/PD OFF/PD ON (L-DOPA and DAA) and DBS] for correct and error trials.

#### Normal and PD Conditions

The mean RT's (in milliseconds) with standard deviation obtained for correct trials in LC (HC) condition (**Figure 9A**) for normals is 1620 ± 0 (1703 ± 91.59); PD OFF is 2381.4 ± 190.91 (2250 ± 0); PD ON (L-DOPA) is 1890 ± 3.07 (1890 ± 0); and

PD ON(DAA) is 1710 ± 0 (1539 ± 540.75). The RT for PD-DAA condition was the lowest for HC among all the other cases, suggesting impulsive behavior known to be present in dopamine agonist treated subjects (Voon et al., 2007). The mean RTs with SD obtained for error trials in LC (HC) condition (**Figure 9B**) for normals is 1735.7 ± 134.64 (1780 ± 35.4), PD OFF is 1125 ± 1591.1 (2430 ± 254.5), PD ON (L-DOPA) is 1926 ± 80.49 (1890 ± 0), and PD ON(DAA) is 1368 ± 720.9 (1539 ± 540.75). On performing ANOVA on reaction times in all the conditions, we observed a significant difference between correct (P < 0.0001) and error trials (P = 0.004).

Based on the theory that STN-GPe chaotic dynamics are responsible for the generation of noise that is crucial for exploration (Chakravarthy et al., 2010; Kalva et al., 2012; Chakravarthy, 2013; Mandali et al., 2015) and which is no longer produced in PD condition, we can safely assume the presence of bursting and synchronous activity in STN. This pathological bursting activity leads to two outcomes; (1) Increase in the firing rate of GPi neurons leading to longer RT and (2) Regularized bursting STN activity that lead to a deterministic activation of the GPi neurons without any noise eventually leading to non-variable reaction time (no standard deviation). Similar could be the case for PD ON condition where the bursting activity of STN in PD OFF condition modulated by medication is changed to a more regular spiking eventually leading to RT values with low variance. From **Figure 9B** of PD OFF condition, we can observe that model's RT is low for error trials. It is clear that due to faster/impulsive response the model performed poorly in those particular trials.

#### STN-DBS

We then checked for the effect of electrode position and antidromic activation on RT, as described earlier.

#### **Electrode position**

The electrode was shifted between the three positions keeping all other stimulation parameters constant and the RT was measured in LC and HC trials (**Figure 10**). We observed that the RT decreased for HC condition, decreased for a specific electrode position (=Pos 3) (for both correct and error trials as plotted in **Figure 10**).

TABLE 3 | Table explains the goodness of fit for the simulation results obtained from testing accuracies and DRE for all the conditions.


*\*Indicates the significance level at p* = *0.05.*

To further analyse the above obtained result (**Figure 10**), we observed the STN activity in HC conditions for the two electrode (Pos 2 and Pos 3) positions. This was to observe how the stimulation current affected the activity of STN neurons in HC condition. We observed that the STN activity for Pos 3 (which corresponds to decreased RT) was significantly lower (t-test at p = 0.05) than Pos 2 (**Figure 11**).

#### **Antidromic activation effect on RT**

We also checked this antidromic effect on reaction time in both correct and error trials. We did not observe any difference in RT between LC and HC conditions in correct trials for any of the percentages. But RT's in error trials were low for HC trials for 10% and 75% of GPe activation. Only for 50% the model's RT for HC trial was higher than LC trial (**Figures 12A,B**). These results

FIGURE 6 | Shows the Normalized performance of the model (with respect to maximum performance in each condition) in PD OFF state for two conditions. Condt 1 and Condt 2 results are similar to the two contrasting behaviors of the subjects observed in Frank et al. (2007, Expt. 1) and Frank et al. (2004, Expt. 2) redrawn. Condt 1 showed no bias to punishment learning which was observed in Condt 2.

FIGURE 5 | Shows the testing accuracy of the model in for HC, PD Off, PD ON (L-DOPA and DAA). (A) Shows the experiment results redrawn from Frank et al. (2007) (B) simulation results obtained from the spiking BG model.

suggest a probable role of antidromic activation in controlling the STN activity.

This set of results study the impulsivity characteristics in various conditions of the model in terms of RT. The DBS electrode position not only modulates learning ability but also influences the RT by decreasing the STN activity in certain electrode positions. An increased activation of GPe neurons by DBS current also reduced the RT.

#### DISCUSSION

PD patients suffer not only from motor abnormalities, but also show signs of cognitive impairment in terms of working memory, learning and executive functions (Owen et al., 1992; Dubois and Pillon, 1996; Chaudhuri et al., 2006; Kehagia et al., 2010). Although therapeutic methods, such as medication as well as stimulation, relieve motor symptoms, they often cause side effects such as impulsivity, learning deficits (Frank et al., 2004, 2007; Voon et al., 2007). L-DOPA has been observed to interfere with learning and DAA are linked to impulse control disorders. Similarly, various experimental results drew attention to STN stimulation effects on cognitive aspects also (Hershey et al., 2004; Smeding et al., 2006; Temel et al., 2006; Frank et al., 2007). Experimental and modeling studies suggest the role of electrode parameters (position and current) on the behavioral outcome of the PD patients (Hershey et al., 2010; Mandali and Chakravarthy, 2015). We start the discussion by explaining the effect of medication and stimulation parameters on learning ability and then on impulsivity.

#### Learning

We first studied the learning and performance aspects in normals, PD OFF, PD ON (L-DOPA and DAA), and DBS conditions. The learning curve for each of the rewarding choices (A/C/E) obtained from the model (**Figure 3**) in PD OFF condition was compared with the results from Zaghloul et al. (2012). By the end of training the model's training accuracy in PD OFF reached the actual reward probabilities of the choices, confirming the learning ability of the spiking model. In normals and PD ON (L-DOPA) cases, the model was able to reach the expected reward probability value, which did not occur in PD ON (DAA) and DBS conditions.

#### Testing Accuracy and Choice/Avoidance Accuracy

The model's ability to differentiate between a high rewarding and low rewarding choice in each of the physiological and pathological conditions [PD OFF, PD medicated conditions (L-DOPA and DAA), and stimulation] was tested by comparing DRE and testing accuracy.

#### Normal and PD Conditions

The results from PD OFF (**Figure 4B**) showed a linear relationship between DRE and testing accuracy which indicated the model's ability to choose a particular choice with high DRE and avoid it otherwise. Another way of explaining the same result is by analysing the performance in terms of Choice/Avoidance accuracy, where the accuracy levels were not significantly different for both choosing A and avoiding B cases. This trend in Normals and PD OFF conditions can also be noticed in **Figures 5A,B**. But this behavior was absent in both of the medicated conditions (L-DOPA and DAA). Even for stimuli where one of the choices was punitive in nature, the testing accuracies (i.e., selecting the particular choice) obtained after training were high (**Figures 4C,D**). This suggests that the model in PD ON (L-DOPA) condition could not learn from punishments and continued to select the lower rewarding choice. This can be further verified from the post-hoc analysis where only the accuracy of PD ON (L-DOPA) was significantly different from all other conditions in avoiding "B." This behavior was also experimentally observed where PD patients under medication tend to learn more from rewards than punishments (Frank et al., 2004, 2007). This can be accounted by the medication term (δmed = 2) (Equation 5) which prevents the model to learn from punishments. The higher amounts of DA due to medication prevented the dip even on the selection of punitive choices. The model's performance in DAA condition did not yield good accuracy in reward learning but performed better than L-DOPA condition in punishment learning (**Figure 5**). This could be observed in the DRE vs. accuracy plot (**Figure 4D**) where the testing accuracy in DAA condition was low for DRE values (<0) when compared to that obtained in L-DOPA (**Figure 4C**) condition.

To address how two groups of PD OFF subjects can show contrasting behavior in punishment learning (Frank et al., 2004, 2007), we modified the connection parameters within STN and GPe in the model and studied the performance (**Figure 6**). Our simulation results show that the lateral connection strength and the level of synchrony in STN and GPe neurons can influence the behavior. Earlier studies by Rubchinsky and colleagues suggested the presence of intermittent synchrony in PD patients (Park et al., 2011). The final action selection at the level of GPi was influenced by the STN-GPe oscillations through indirect pathway. Therefore, a difference in lateral connection radius within these excitatory-inhibitory neurons led to subtle changes in their synchrony level which eventually reflected at the level of decision making.

#### STN-DBS

Another important aspect of our study is to observe if there is any effect of stimulation parameters such as electrode position

FIGURE 10 | Shows the RT for LC and HC conditions when the position of the electrode was changed. Pos 1, Pos 2, Pos 3 are also described in Figure 7. (A) Correct, (B) Error trials.

and increased antidromic activation of DBS on learning. As the position of electrode was changed (**Figure 7**) the model switched from reward-based to punishment-based learning. The presence of parallel BG loops with different functions has been well known anatomically (Alexander et al., 1986); the points of intersection of these loops with STN have also been topographically mapped (Hamani et al., 2004). It is possible that variation in the electrode position physiologically could be related to an activation of different areas which are known to modulate reward and punishment learning differently (Wächter et al., 2009). For a specific position (**Figure 7**- Pos 2), the accuracy level for choosing A and avoiding B was same but reduced compared to normals. These results show that electrode might be playing an important role in the cognitive function of the subject. Apart from electrode position, we also studied the effect of antidromic activation of GPe neurons due to stimulation in STN neurons (**Figure 8**). For 10% of the stimulating current affecting the GPe neurons, the behavior in terms of accuracy was quite similar to PD-ON (L-DOPA) results. For higher percentages (i.e., 50 and 75%), the behavior was similar to experimental DBS results.

#### Reaction Time and Impulsivity

Based on the experimental evidence that an increase in STN activity was observed during HC conditions, we analyzed the reaction time for each of the conditions in LC and HC cases in each of the five conditions.

#### Normal and PD Conditions

We observed that the model in normal condition took more time to make a choice during HC case compared to that in LC in both correct and error trials (**Figures 9A,B**). The impulsivity behavior observed clinically due to DAA medication (Voon et al., 2007; Ondo and Lai, 2008) was captured by the model where we observed a lower RT for HC case. PD-ON DAA condition showed the lowest RT compared to other conditions and PD-OFF the highest as shown in Frank et al. (2007) and Hauptmann and Tass (2007). High STN activity in untreated PD condition could make the model take longer to reach the threshold thus leading to a higher reaction time. DAAs which selectively affect D2 receptors could decrease the STN activity making the system to respond faster leading to incorrect and impulsive decisions. The model in PD OFF condition also showed an increase in RT for HC case during error trials but a decrease during correct ones.

#### STN-DBS

The concept of impulsivity due to DBS was studied by varying parameters such as electrode contact position within STN nucleus and inducing antidromic activation of GPe neurons. We observed that the RTs were different for different electrode

positions and a lower RT was obtained for HC case during both correct and error trials for a specific electrode position (Pos 3). To further analyse why such behavior was observed, we checked the STN activity for each of the positions (Pos 2, Pos 3) in HC conditions (**Figure 10**). We observed a significant (P < 0.05) decrease in STN activity in Pos 3 condition compared to Pos 2 during the first 600 ms of stimulation (**Figure 11**). We hypothesize that it is due to this initial difference in the STN activity that a reduction in RT was observed.

Based on the theory which considers the possibility for antidromic activation of GPe neurons during STN stimulation (Hauptmann and Tass, 2007; Montgomery and Gale, 2008), we studied the RT for various levels of GPe neuronal activation (**Figures 12A,B**). For a fixed position of DBS electrode at the lattice position (25, 25), there was no significant difference in the RT for correct trials. But for error trials, the percentage of GPe activation affected the RT especially in HC case. Only for 50% of the cases, the RT in HC case was higher than that obtained in LC. A higher activation of GPe neurons (=75%) though shows normal learning behavior gives a decreased RT in error trials as observed in experiment (Frank et al., 2007). These results show that higher antidromic activation of GPe neurons could be a probable reason for the observed impulsivity in DBS patients.

We also studied the effect of the inhibitory connection from GPe to GPi on accuracy and RT in normal. Preliminary results from the model showed no significant difference in accuracy or RT when simulated with and without this connection (results not included). But further analysis has to be performed to fully understand the functional significance of this anatomical connection.

#### Conclusions, Limitations, and Future Work

The symptom profiles for various patient types are diverse and so are their responses to either medication and/or stimulation.

#### REFERENCES

Alexander, G. E., Delong, M. R., and Strick, P. L. (1986). Parallel organization of functionally segregated circuits linking basal ganglia and cortex. Annu. Rev. Neurosci. 9, 357–381. doi: 10.1146/annurev.ne.09.030186.002041

Our spiking BG model gives an insight into the patient's response to each of these therapies (learning and RT) which might help to suggest alternative protocols. We also emphasize the importance of synchrony in STN critically modulated by the lateral connections within STN, and how STN influences the final behavior in patients, which is not accounted by many other computational BG models.

In future, we would like to address our model limitations in terms of explaining the lower RT observed in PD OFF patients. We believe that incorporating risk based approach (Balasubramani et al., 2015) in to our current model would capture the result of lower RT in PD OFF condition. Also, we would like to include the inhibitory GPe→GPi connection and hyper direct pathway connection in to the network. Another limitation which we would like to address is the spatial definition and boundaries in STN based on the functional cortico-BG loops. We are yet to fully understand the physiological effects of DBS on STN in terms of behavior and different frequency bands and would expand the model with more realistic connectivity and integration of other BG nuclei.

#### AUTHOR CONTRIBUTIONS

AM developed the computational model, simulated the tasks, analyzed the results, and prepared the manuscript. VSC developed the computational model, simulated the tasks, analyzed the results, and prepared the manuscript.

#### SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: http://journal.frontiersin.org/article/10.3389/fnhum. 2016.00450

Balasubramani, P. P., Chakravarthy, V. S., Ali, M., Ravindran, B., and Moustafa, A. A. (2015). Identifying the basal ganglia network model markers for medication-induced impulsivity in Parkinson's Disease patients. PLoS ONE 10:e0127542. doi: 10.1371/journal.pone. 0127542


function in Parkinson's disease. Brain 123, 1142–1154. doi: 10.1093/brain/123. 6.1142


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2016 Mandali and Chakravarthy. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.