# CLOSED-LOOP SYSTEMS FOR NEXT-GENERATION NEUROPROSTHESES

EDITED BY: Timothée Levi, Paolo Bonifazi, Paolo Massobrio and Michela Chiappalone PUBLISHED IN: Frontiers in Neuroscience and Frontiers in Neurology

#### *Frontiers Copyright Statement*

*© Copyright 2007-2018 Frontiers Media SA. All rights reserved. All content included on this site, such as text, graphics, logos, button icons, images, video/audio clips, downloads, data compilations and software, is the property of or is licensed to Frontiers Media SA ("Frontiers") or its licensees and/or subcontractors. The copyright in the text of individual articles is the property of their respective authors, subject to a license granted to Frontiers.*

*The compilation of articles constituting this e-book, wherever published, as well as the compilation of all other content on this site, is the exclusive property of Frontiers. For the conditions for downloading and copying of e-books from Frontiers' website, please see the Terms for Website Use. If purchasing Frontiers e-books from other websites or sources, the conditions of the website concerned apply.*

*Images and graphics not forming part of user-contributed materials may not be downloaded or copied without permission.*

*Individual articles may be downloaded and reproduced in accordance with the principles of the CC-BY licence subject to any copyright or other notices. They may not be re-sold as an e-book.*

*As author or other contributor you grant a CC-BY licence to others to reproduce your articles, including any graphics and third-party materials supplied by you, in accordance with the Conditions for Website Use and subject to any copyright notices which you include in connection with your articles and materials.*

> *All copyright, and all rights therein, are protected by national and international copyright laws.*

> *The above represents a summary only. For the full conditions see the Conditions for Authors and the Conditions for Website Use.*

ISSN 1664-8714 ISBN 978-2-88945-466-2 DOI 10.3389/978-2-88945-466-2

#### About Frontiers

Frontiers is more than just an open-access publisher of scholarly articles: it is a pioneering approach to the world of academia, radically improving the way scholarly research is managed. The grand vision of Frontiers is a world where all people have an equal opportunity to seek, share and generate knowledge. Frontiers provides immediate and permanent online open access to all its publications, but this alone is not enough to realize our grand goals.

#### Frontiers Journal Series

The Frontiers Journal Series is a multi-tier and interdisciplinary set of open-access, online journals, promising a paradigm shift from the current review, selection and dissemination processes in academic publishing. All Frontiers journals are driven by researchers for researchers; therefore, they constitute a service to the scholarly community. At the same time, the Frontiers Journal Series operates on a revolutionary invention, the tiered publishing system, initially addressing specific communities of scholars, and gradually climbing up to broader public understanding, thus serving the interests of the lay society, too.

#### Dedication to Quality

Each Frontiers article is a landmark of the highest quality, thanks to genuinely collaborative interactions between authors and review editors, who include some of the world's best academicians. Research must be certified by peers before entering a stream of knowledge that may eventually reach the public - and shape society; therefore, Frontiers only applies the most rigorous and unbiased reviews.

Frontiers revolutionizes research publishing by freely delivering the most outstanding research, evaluated with no bias from both the academic and social point of view. By applying the most advanced information technologies, Frontiers is catapulting scholarly publishing into a new generation.

#### What are Frontiers Research Topics?

Frontiers Research Topics are very popular trademarks of the Frontiers Journals Series: they are collections of at least ten articles, all centered on a particular subject. With their unique mix of varied contributions from Original Research to Review Articles, Frontiers Research Topics unify the most influential researchers, the latest key findings and historical advances in a hot research area! Find out more on how to host your own Frontiers Research Topic or contribute to one as an author by contacting the Frontiers Editorial Office: researchtopics@frontiersin.org

# **CLOSED-LOOP SYSTEMS FOR NEXT-GENERATION NEUROPROSTHESES**

Topic Editors: **Timothée Levi,** University of Bordeaux, France **Paolo Bonifazi**, Biocruces Health Research Institute, Spain **Paolo Massobrio,** Universita di Genova, Italy **Michela Chiappalone,** Istituto Italiano di Tecnologia Genova, Italy

Cloud of the keywords used by the authors contributing to this e-book and logo of the FET European project BrainBow, which depicts a futuristic closed-loop neuroprosthesis for the brain. Copyright: T. Levi, P. Bonifazi, P. Massobrio, M. Chiappalone, S. Chiappalone.

Cover image: Naeblys/Shutterstock.com

Millions of people worldwide are affected by neurological disorders which disrupt the connections within the brain and between brain and body causing impairments of primary functions and paralysis. Such a number is likely to increase in the next years and current assistive technology is yet limited. A possible response to such disabilities, offered by the neuroscience community, is given by Brain-Machine Interfaces (BMIs) and neuroprostheses.

The latter field of research is highly multidisciplinary, since it involves very different and disperse scientific communities, making it fundamental to create connections and to join research efforts. Indeed, the design and development of neuroprosthetic devices span/involve different research topics such as: interfacing of neural systems at different levels of architectural complexity (from in vitro neuronal ensembles to human brain), bio-artificial interfaces for stimulation (e.g. micro-stimulation, DBS: Deep Brain Stimulation) and recording (e.g. EMG: Electromyography, EEG: Electroencephalography, LFP: Local Field Potential), innovative signal processing tools for coding and decoding of neural activity, biomimetic artificial Spiking Neural Networks (SNN) and neural network modeling. In order to develop functional communication with the nervous system and to create a new generation of neuroprostheses, the study of closed-loop systems is mandatory. It has been widely recognized that closed-loop neuroprosthetic systems achieve more favorable outcomes for users then equivalent open-loop devices. Improvements in task performance, usability, and embodiment have all been reported in systems utilizing some form of feedback. The bi-directional communication between living neurons and artificial devices is the main final goal of those studies. However, closed-loop systems are still uncommon in the literature, mostly due to requirement of multidisciplinary effort. Therefore, through eBook on closed-loop systems for next-generation neuroprostheses, we encourage an active discussion among neurobiologists, electrophysiologists, bioengineers, computational neuroscientists and neuromorphic engineers.

This eBook aims to facilitate this process by ordering the 25 contributions of this research in which we highlighted in three different parts: (A) Optimization of different blocks composing the closed-loop system, (B) Systems for neuromodulation based on DBS, EMG and SNN and (C) Closed-loop BMIs for rehabilitation.

**Citation:** Levi, T., Bonifazi, P., Massobrio, P., Chiappalone, M., eds. (2018). Closed-Loop Systems for Next-Generation Neuroprostheses. Lausanne: Frontiers Media. doi: 10.3389/978-2-88945-466-2

# Table of Contents

*07 Editorial: Closed-Loop Systems for Next-Generation Neuroprostheses* Timothée Levi, Paolo Bonifazi, Paolo Massobrio and Michela Chiappalone

### **Optimization of Different Blocks Composing The Closed-Loop System**


Ilknur Telkes, Joohi Jimenez-Shahed, Ashwin Viswanathan, Aviva Abosch and Nuri F. Ince

*51 EEG-Based Quantification of Cortical Current Density and Dynamic Causal Connectivity Generalized across Subjects Performing BCI-Monitored Cognitive Tasks*

Hristos Courellis, Tim Mullen, Howard Poizner, Gert Cauwenberghs and John R. Iversen

*68 An Improved Unscented Kalman Filter Based Decoder for Cortical Brain-Machine Interfaces* Simin Li, Jie Li and Zheng Li

*87 An FPGA Platform for Real-Time Simulation of Spiking Neuronal Networks* Danilo Pani, Paolo Meloni, Giuseppe Tuveri, Francesca Palumbo, Paolo Massobrio and Luigi Raffo

*100 A Bidirectional Brain-Machine Interface Featuring a Neuromorphic Hardware Decoder*

Fabio Boi, Timoleon Moraitis, Vito De Feo, Francesco Diotalevi, Chiara Bartolozzi, Giacomo Indiveri and Alessandro Vato


Fatemeh Karimi, Jonathan Kofman, Natalie Mrachacz-Kersting, Dario Farina and Ning Jiang

#### *141 Factors of Influence on the Performance of a Short-Latency Non-Invasive Brain Switch: Evidence in Healthy Individuals and Implication for Motor Function Rehabilitation*

Ren Xu, Ning Jiang, Natalie Mrachacz-Kersting, Kim Dremstrup and Dario Farina

### **Systems for Neuromodulation Based on DBS, EMG and SNN**

#### *150 Proceedings of the Third Annual Deep Brain Stimulation Think Tank: A Review of Emerging Issues and Technologies*

P. Justin Rossi, Aysegul Gunduz, Jack Judy, Linda Wilson, Andre Machado, James J. Giordano, W. Jeff Elias, Marvin A. Rossi, Christopher L. Butson, Michael D. Fox, Cameron C. McIntyre, Nader Pouratian, Nicole C. Swann, Coralie de Hemptinne, Robert E. Gross, Howard J. Chizeck, Michele Tagliati, Andres M. Lozano, Wayne Goodman, Jean-Philippe Langevin, Ron L. Alterman, Umer Akbar, Greg A. Gerhardt, Warren M. Grill, Mark Hallett, Todd Herrington, Jeffrey Herron, Craig van Horne, Brian H. Kopell, Anthony E. Lang, Codrin Lungu, Daniel Martinez-Ramirez, Alon Y. Mogilner, Rene Molina, Enrico Opri, Kevin J. Otto, Karim G. Oweiss, Yagna Pathak, Aparna Shukla, Jonathan Shute, Sameer A. Sheth, Ludy C. Shih, G. Karl Steinke, Alexander I. Tröster, Nora Vanegas, Kareem A. Zaghloul, Leopoldo Cendejas-Zaragoza, Leonard Verhagen, Kelly D. Foote and Michael S. Okun

#### *165 Detecting a Cortical Fingerprint of Parkinson's Disease for Closed-Loop Neuromodulation*

Kevin Kern, Georgios Naros, Christoph Braun, Daniel Weiss and Alireza Gharabaghi

*174 A Rodent Model of Dynamic Facial Reanimation Using Functional Electrical Stimulation*

Mark A. Attiah, Julius de Vries, Andrew G. Richardson and Timothy H. Lucas


Adeline Zbrzeski, Yannick Bornat, Brian Hillen, Ricardo Siu, James Abbas, Ranu Jung and Sylvie Renaud

### **Closed-Loop Bmis for Rehabilitation**


Soraya Pérez-Nombela, Antonio J. del-Ama, Joan Aranda, Javier Minguez, Angel Gil-Agudo and Luis Montesano

*240 A Personalized Multi-Channel FES Controller Based on Muscle Synergies to Support Gait Rehabilitation after Stroke*

Simona Ferrante, Noelia Chia Bejarano, Emilia Ambrosini, Antonio Nardone, Anna M. Turcato, Marco Monticone, Giancarlo Ferrigno and Alessandra Pedrocchi *252 Low Latency Estimation of Motor Intentions to Assist Reaching Movements along Multiple Sessions in Chronic Stroke Patients: A Feasibility Study*

Jaime Ibáñez, Esther Monge-Pereira, Francisco Molina-Rueda, J. I. Serrano, Maria D. del Castillo, Alicia Cuesta-Gómez, María Carratalá-Tejada, Roberto Cano-de-la-Cuerda, Isabel M. Alguacil-Diego, Juan C. Miangolarra-Page and Jose L. Pons

*264 Corrigendum: Low Latency Estimation of Motor Intentions to Assist Reaching Movements along Multiple Sessions in Chronic Stroke Patients: A Feasibility Study*

Jaime Ibáñez, Esther Monge-Pereira, Francisco Molina-Rueda, J. Ignacio Serrano, Maria D. del Castillo, Alicia Cuesta-Gómez, María Carratalá-Tejada, Roberto Cano-de-la-Cuerda, Isabel M. Alguacil-Diego, Juan C. Miangolarra-Page and Jose L. Pons

*265 Closed-Loop Neuroprosthesis for Reach-to-Grasp Assistance: Combining Adaptive Multi-channel Neuromuscular Stimulation with a Multi-joint Arm Exoskeleton*

Florian Grimm and Alireza Gharabaghi


Nuray Yozbatiran, Amy A. Blank, James French, Christof Karmonik, Robert G. Grossman, Marcia K. O'Malley, Gerard E. Francisco and Jose L. Contreras-Vidal

*319 What Turns Assistive into Restorative Brain-Machine Interfaces?* Alireza Gharabaghi

# Editorial: Closed-Loop Systems for Next-Generation Neuroprostheses

Timothée Levi <sup>1</sup> \*, Paolo Bonifazi 2,3, Paolo Massobrio<sup>4</sup> and Michela Chiappalone<sup>5</sup>

 Laboratoire IMS, UMR Centre National de la Recherche Scientifique 5218, University of Bordeaux, Bordeaux, France, Biocruces Health Research Institute, Bilbao, Spain, <sup>3</sup> School of Physics and Astronomy, Tel Aviv University, Tel Aviv, Israel, Department of Informatics, Bioengineering, Robotics and System Engineering, University of Genova, Genova, Italy, <sup>5</sup> Rehab Technologies, Istituto Italiano di Tecnologia, Genova, Italy

#### Keywords: neuroprostheses, neuromodulation, stimulation, closed-loop experiments, neuronal circuits, artificial spiking neural network

**Editorial on the Research Topic**

#### **Closed-Loop Systems for Next-Generation Neuroprostheses**

Millions of people worldwide are affected by neurological disorders which disrupt the connections within the brain and between brain and body causing impairments of primary functions and paralysis. Such a number is likely to increase in the next years and current assistive technology is yet limited. A possible response to such disabilities, offered by the neuroscience community, is given by Brain-Machine Interfaces (BMIs) and neuroprosthetic research.

The latter field of research is highly multidisciplinary, since it involves very different and disperse scientific communities, making it fundamental to create connections and to join research efforts. Indeed, the design and development of neuroprostheses involve different research topics such as: interfacing to nervous systems at different levels of architectural complexity (from in vitro neuronal ensembles to human brain), bio-electronic interfaces for stimulation (e.g., micro-stimulation, DBS: Deep Brain Stimulation) and recording (e.g., EMG, Electromyography; EEG, Electroencephalography; LFP, Local Field Potential), innovative signal processing tools for coding and decoding of neural activity, biomimetic artificial Spiking Neural Networks (SNN) and neural network modeling (Indiveri et al., 2001; Bonifazi et al., 2013). In order to develop functional communication with the nervous system and to create a new generation of neuroprostheses, the study of closed-loop systems is mandatory. It has been widely recognized that closed-loop neuroprosthetic systems achieve more favorable outcomes than open-loop devices. Improvements in task performance, usability, and embodiment have all been reported in systems utilizing some form of feedback. The bi-directional communication between living neurons and artificial devices is the main final goal of those studies. However, closed-loop systems not only based on visual feedback are still uncommon, mostly due to requirement of multidisciplinary effort. Only few examples in this direction can be cited from the literature, such as O'Doherty et al. (2011) and Capogrosso et al. (2016). Therefore, through this research topic on closed-loop systems for next-generation neuroprostheses, we encourage an active discussion among neurobiologists, electrophysiologists, bioengineers, computational neuroscientists, and neuromorphic engineers.

This Editorial aims to facilitate this process by ordering the 25 contributions of this research in which we highlighted in three different parts: (A) Optimization of different blocks composing the closed-loop system, (B) Systems for neuromodulation based on DBS, EMG, and SNN, and (C) Closed-loop BMIs for rehabilitation.

### (A) OPTIMIZING THE DIFFERENT BLOCKS COMPOSING A CLOSED-LOOP SYSTEM

To design closed-loop neuroprostheses, the three main blocks which require optimization are recording, signal processing, and stimulation.

Edited and reviewed by: Mikhail Lebedev, Duke University, United States

\*Correspondence: Timothée Levi timothee.levi@ims-bordeaux.fr

#### Specialty section:

This article was submitted to Neuroprosthetics, a section of the journal Frontiers in Neuroscience

Received: 25 December 2017 Accepted: 12 January 2018 Published: 12 February 2018

#### Citation:

Levi T, Bonifazi P, Massobrio P and Chiappalone M (2018) Editorial: Closed-Loop Systems for Next-Generation Neuroprostheses. Front. Neurosci. 12:26. doi: 10.3389/fnins.2018.00026

To target specific structures with micro-stimulation, localization methods should be developed. For that, Telkes et al. focuses on multiple spectral features extracted from microelectrode-recorded LFPs which could be used to automate the identification of the optimal track and the SubThalamic Nucleus (STN) localization. These results establish the initial evidence that LFPs can be strategically fused with computational intelligence in the operating room for STN localization and the selection of the track for chronic Deep Brain Stimulation (DBS) electrode implantation.

After recording electrical activities of the brain or specific neural network, the coding part follows. For example a decoder translates recorded neural activity into motor commands while an encoder delivers sensory information collected from the environment directly to the brain creating a closed-loop system. Yang et al. define a novel decoding algorithm regardless of the number of neurons generating the recorded signals. Gailey et al. describe a proof of concept for online EMG-based decoding of hand postures and Individual digit forces for prosthetic hand control. Courellis et al. propose an algorithmic and computational framework for identifying cortical networks across subjects in which dynamic causal connectivity is modeled among user-selected cortical regions of interest. Li et al. aim to improve accuracy of signal processing by designing a better encoding model of primary motor cortical activity during hand movements and combining this with decoder engineering refinements, resulting in a new unscented Kalman filter-based decoder.

Another important element to consider in closed-loop systems is the use of artificial SNNs to replace lost neural networks, or to implement artificial intelligence in the signal processing decoder. Pani et al. present a modular and efficient FPGA design of an in silico SNN exploiting the Izhikevich model. The proposed system is able to simulate a fully connected network counting up to 1,440 neurons, in real-time, at a sampling rate of 10 kHz, which is reasonable for small to medium scale extra-cellular closed-loop experiments. Boi et al. create a bidirectional BMI which establish a two-way direct communication link between the brain and the external world. As a first step toward this goal, they developed a modular bidirectional BMI setup that uses a compact neuromorphic processor as a decoder. On this chip, a network of SNNs is built using its ultra-low-power mixed-signal analog/digital circuits. Kohno et al. review different SNNs and propose qualitative neuron models for designing more biomimetic SNN.

For BMI system, optimization of Movement Related Cortical Potential (MRCP) recordings is important to improve control of a neural prosthesis. MRCP, a slow cortical potential from the scalp EEG, has been used in real-time brain-computerinterface (BCI) systems designed for neurorehabilitation. Karimi et al. propose a new MRCP detection method based on constrained independent component analysis (cICA). Xu et al. investigate single trial EEG traces during motor imagery on healthy individuals, and provided a comprehensive analysis of the performance of a short-latency brain switch. The morphological investigation showed a cross-subject consistency of a prolonged negative phase in MRCP, and a delayed beta rebound in sensory-motor rhythms during repetitive tasks.

### (B) SYSTEMS FOR NEUROMODULATION BASED ON DBS, EMG, AND SNN

Electrical stimulation and neuromodulation are techniques used as therapeutic solutions for neurorehabilitation. We distinguish here three approaches: DBS, EMG-based electrical stimulation, and SNN for biomimetic micro-stimulations.

Rossi et al. review the proceedings of the 3rd Annual Deep Brain Stimulation Think Tank which discussed the most contemporary clinical, electrophysiological, imaging, and computational work on DBS for the treatment of neurological and neuropsychiatric disease. Recent evidence suggests that DBS of the STN in Parkinson's disease mediates its clinical effects by modulating cortical oscillatory activity, presumably via a direct cortico-subthalamic connection. This observation might pave the way for novel closed-loop approaches comprising a cortical sensor. Kern et al. follow the same direction and provide preliminary evidence for detecting a cortical fingerprint of Parkinson's disease for closed-loop neuromodulation.

By following a different approach, Attiah et al. perform closed-loop experiments for reanimating paralyzed facial muscles in a rodent model. The EMG signal of the intact side was used to trigger Functional Electrical Stimulation (FES) on the paralyzed side to restore symmetric whisking. The results demonstrate a novel in vivo platform for developing control strategies for neuromuscular facial prostheses. Time-variant muscle responses under ES are often problematic for all the applications of neuroprosthetic muscle control. Hayashibe overviews the background of this topic and highlights important points to be aware of when using ES to induce the desired muscle activation regardless of the time-variance. He also demonstrates how to deal with the common critical problem of ES to move toward robust neuroprosthetic muscle control with the evoked electromyographically controlled electrical stimulation paradigm.

Neural prostheses based on electrical micro-stimulation offer promising perspectives to restore lost functions following lesions of the Central Nervous System (CNS). A challenging perspective is to control micro-stimulation by SNN hybridized to the living tissue. Joucla et al. design an artificial Central Pattern Generator (CPG) based on real-time SNN to generate alternating activity. This system is hybridized to living spinal cord to drive electrical micro-stimulation. These results are a first step toward hybrid artificial/biological solutions based on electrical micro-stimulation for the restoration of lost function in the injured CNS. Spinal cord injury can disrupt connections between the brain respiratory network and the respiratory muscles which can lead to partial or complete loss of ventilatory control and require ventilatory assistance. Zbrzeski et al. present an original bio-inspired technology for real-time ventilation assistance, implemented in a digital circuit. The bio-inspired controller, which is a SNN inspired by the medullary respiratory network, is as robust as a classical controller while exhibiting a flexible, low-power and low-cost hardware design.

### (C) CLOSED-LOOP SYSTEMS WITH BMI FOR REHABILITATION

In this section, we focus on closed-loop systems for rehabilitation. BMIs may support motor impaired patients during activities of daily living by controlling external devices such as prostheses (assistive BMI). Wright et al. review control strategies in existing experimental, investigational and clinical neuroprosthetic systems in order to establish a baseline and promote a common understanding of different feedback modalities and closed-loop controllers.

The closed-loop control of rehabilitative technologies by neural commands has shown a great potential to improve motor recovery in patients suffering from paralysis. BMIs can be used as a natural control method for such technologies. Lopez-Larraz et al. present a proof-of-concept study to validate the feasibility of a BMI to control an ambulatory exoskeleton by patients with incomplete paraplegia. Ferrante et al. design a personalized multi-channel FES controller for gait training, integrating three novel aspects: (1) the FES strategy was based on healthy muscle synergies in order to mimic the neural solutions adopted by the CNS to generate locomotion; (2) the FES strategy was personalized according to an initial locomotion assessment of the patient and was designed to specifically activate the impaired biomechanical functions; (3) the FES strategy was mapped accurately on the altered gait kinematics providing a maximal synchronization between patient's volitional gait and stimulation patterns.

Different studies on rehabilitation of stroke patients are presented in this research topic. (Ibanez et al.; Ibanez et al.) explore the feasibility of using BMIs providing low-latency support to upper-limb reaching movements in patients with stroke as a reliable and usable solution for motor rehabilitation interventions with potential functional benefits.

Stroke patients with severe motor deficits cannot execute task-oriented rehabilitation exercises with their affected upper extremity. Advanced rehabilitation technology may support them in performing such reach-to-grasp movements. The challenge is, however, to provide assistance as needed, while

### REFERENCES


maintaining the participants' commitment during the exercises. In a feasibility study, Grimm and Gharabaghi introduce a closed-loop neuroprosthesis for reach-to-grasp assistance which combines adaptive multi-channel neuromuscular stimulation with a multi-joint arm exoskeleton. Grimm et al. also combine a hybrid BMI with neuromuscular stimulation and antigravity assistance which augments upper limb function and brain activity during rehabilitation exercises and may thus provide a novel restorative framework for severely affected stroke patients. Combining gravity-compensating assistance with adaptive closed-loop feedback in virtual reality provides customized rehabilitation environments for severely affected stroke patients. Grimm et al. develop this approach to simplify motor learning by progressively challenging the subject in accordance with the individual capacity for functional restoration. Bhagat et al. demonstrate the feasibility of detecting motor intent from brain activity of chronic stroke patients using an asynchronous EEG-based BMI. Intent was inferred from movement related cortical potentials (MRCPs) measured over an optimized set of EEG electrodes. These findings provide evidence that closed-loop EEG-based BMI for stroke patients can be designed and optimized to successfully perform across multiple days without system recalibration.

To conclude this part, Gharabaghi wrote a perspective article discussing the necessary features and prerequisites of restorative BMI for stroke rehabilitation.

### AUTHOR CONTRIBUTIONS

TL, PB, PM, and MC: Prepared and discussed about this research topic, invited authors, revised their manuscripts, and handled their revisions.

### ACKNOWLEDGMENTS

This work is supported by the generous contribution of the European Commission ICT-FET FP7 (FET Young Explorers scheme) BRAIN BOW (www.brainbowproject.eu), granted to the authors of this research topic. PB acknowledges financial support from Ikerbasque (The Basque Foundation for Science) and from the Ministerio Economia, Industria y Competitividad of Spain (grant SAF2015-69484-792R).

silicon neuron circuits. Front. Neurosci. 5:73. doi: 10.3389/fnins.2011. 00073

**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Levi, Bonifazi, Massobrio and Chiappalone. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# A Sliced Inverse Regression (SIR) Decoding the Forelimb Movement from Neuronal Spikes in the Rat Motor Cortex

Shih-Hung Yang1 †, You-Yin Chen2 †, Sheng-Huang Lin3, <sup>4</sup> , Lun-De Liao5, 6 , Henry Horng-Shing Lu<sup>7</sup> , Ching-Fu Wang<sup>2</sup> , Po-Chuan Chen<sup>2</sup> , Yu-Chun Lo<sup>8</sup> , Thanh Dat Phan<sup>1</sup> , Hsiang-Ya Chao<sup>9</sup> , Hui-Ching Lin<sup>10</sup>, Hsin-Yi Lai <sup>11</sup> \* and Wei-Chen Huang<sup>12</sup>

<sup>1</sup> Department of Mechanical and Computer Aided Engineering, Feng Chia University, Taichung, Taiwan, <sup>2</sup> Department of Biomedical Engineering, National Yang Ming University, Taipei, Taiwan, <sup>3</sup> Institute of Biomedical Engineering, College of Medicine, National Taiwan University, Taipei, Taiwan, <sup>4</sup> Department of Neurology, Tzu Chi General Hospital, Tzu Chi University, Hualien, Taiwan, <sup>5</sup> Institute of Biomedical Engineering and Nanomedicine, National Health Research Institutes, Zhunan Township, Taiwan, <sup>6</sup> Singapore Institute for Neurotechnology, National University of Singapore, Singapore, Singapore, 7 Institute of Statistics, National Chiao Tung University, Hsinchu, Taiwan, <sup>8</sup> The Ph.D. Program for Neural Regenerative Medicine, College of Medical Science and Technology, Taipei Medical University, Taipei, Taiwan, <sup>9</sup> Department of Electrical Engineering, National Taiwan University, Taipei, Taiwan, <sup>10</sup> Department and Institute of Physiology, School of Medicine, National Yang Ming University, Taipei, Taiwan, <sup>11</sup> Interdisciplinary Institute of Neuroscience and Technology, Qiushi Academy for Advanced Studies, Zhejiang University, Hangzhou, China, <sup>12</sup> Department of Materials Science and Engineering, Carnegie Mellon University, Pittsburgh, PA, USA

#### Edited by:

Timothée Levi, University of Bordeaux 1, France

#### Reviewed by:

Alireza Mousavi, Brunel University London, UK Inaki Iturrate, École Polytechnique Fédérale de Lausanne, Switzerland

\*Correspondence:

Hsin-Yi Lai laihy@zju.edu.cn These authors have contributed

†

equally to this work. Specialty section:

This article was submitted to Neuroprosthetics, a section of the journal Frontiers in Neuroscience

Received: 10 July 2016 Accepted: 21 November 2016 Published: 09 December 2016

#### Citation:

Yang S-H, Chen Y-Y, Lin S-H, Liao L-D, Lu HH-S, Wang C-F, Chen P-C, Lo Y-C, Phan TD, Chao H-Y, Lin H-C, Lai H-Y and Huang W-C (2016) A Sliced Inverse Regression (SIR) Decoding the Forelimb Movement from Neuronal Spikes in the Rat Motor Cortex. Front. Neurosci. 10:556. doi: 10.3389/fnins.2016.00556 Several neural decoding algorithms have successfully converted brain signals into commands to control a computer cursor and prosthetic devices. A majority of decoding methods, such as population vector algorithms (PVA), optimal linear estimators (OLE), and neural networks (NN), are effective in predicting movement kinematics, including movement direction, speed and trajectory but usually require a large number of neurons to achieve desirable performance. This study proposed a novel decoding algorithm even with signals obtained from a smaller numbers of neurons. We adopted sliced inverse regression (SIR) to predict forelimb movement from single-unit activities recorded in the rat primary motor (M1) cortex in a water-reward lever-pressing task. SIR performed weighted principal component analysis (PCA) to achieve effective dimension reduction for nonlinear regression. To demonstrate the decoding performance, SIR was compared to PVA, OLE, and NN. Furthermore, PCA and sequential feature selection (SFS) which are popular feature selection techniques were implemented for comparison of feature selection effectiveness. Among SIR, PVA, OLE, PCA, SFS, and NN decoding methods, the trajectories predicted by SIR (with a root mean square error, RMSE, of 8.47 ± 1.32 mm) was closer to the actual trajectories compared with those predicted by PVA (30.41 ± 11.73 mm), OLE (20.17 ± 6.43 mm), PCA (19.13 ± 0.75 mm), SFS (22.75 ± 2.01 mm), and NN (16.75 ± 2.02 mm). The superiority of SIR was most obvious when the sample size of neurons was small. We concluded that SIR sorted the input data to obtain the effective transform matrices for movement prediction, making it a robust decoding method for conditions with sparse neuronal information.

Keywords: sliced inverse regression (SIR), neural decoding, forelimb movement prediction, neural networks (NN), principle component analysis (PCA)

## INTRODUCTION

In order to improve daily life activities for paralyzed patients, the establishment of a non-muscular communication interface between brain neurons and machines has rapidly developed over the last two decades (Schwartz, 1993, 1994; Donoghue, 2002; Schwartz et al., 2006; Velliste et al., 2008). With assistance from stable generated brain-derived control signals incorporated with prosthetic devices and motor functions, paralyzed patients now possibly regain their ability to move a computer cursor (Kennedy et al., 2000; Hochberg et al., 2006; Gilja et al., 2015), control an anthropomorphic prosthetic arm (Wodlinger et al., 2015), and drive a prosthetic device (Hochberg et al., 2012; Collinger et al., 2013) through a brain-machine interface (BMI). One important challenge to BMI is how to design an appropriate neural decoder (Pohlmeyer et al., 2014). To address the challenge, previous studies have carefully utilized training paradigms that have been designed for a BMI decoder and controller. For brain-derived control signals, neural decoding is an indispensable technique that translates neuronal activities to physical states, such as the position of a foraging rat (Brown et al., 1998), arm movement (Ashe and Georgopoulos, 1994), movement speed (Moran and Schwartz, 1999), hand position (Paninski et al., 2004), and joint angular velocity (Reina et al., 2001).

A population vector algorithm (PVA), one method for decoding motor cortical activity, assumed that a neuron's firing rate is related to the velocity vector of movement. PVA categorizes each neuron's contribution into directional and distance information of the movement by a directional tuning function under uniform variance conditions (Georgopoulos et al., 1988). A previous study showed that PVA decoding could expose the visuomotor coordinate transformations between visual and motor information by processing masses of neuronal activities recorded from relative brain regions (Takeda and Funahashi, 2004; Watanabe et al., 2009). PVA presented superior performance in predicting hand path throughout reaching tasks (Schwartz, 1994). However, a uniformity constraint is usually not the case for real experiments, and the equality of the tuning function is variable because of the small amount of unit recordings in realistic applications (Schwartz et al., 2001). To compensate for the non-uniform preferred directions in the population of recorded neurons, an optimal linear estimator (OLE) was proposed to define the preferred direction of each neuron using the center of mass of the tuning function (Salinas and Abbott, 1994). Requiring large numbers of neurons with a temporal solution of 10–100 ms, PVA and OLE studies successfully predicted the kinematic parameters of a primate arm movement (Schwartz et al., 2001; Takeda and Funahashi, 2004; Watanabe et al., 2009).

A Bayesian decoder, a probabilistic decoding technique, could achieve accurate offline trajectory reconstructions by combing simple trajectory models (Yu et al., 2007). However, off-line reconstruction may not be suitable for online prosthesis control because the essential features of a real prosthesis are not acquired, and the system dynamics may vary because the user is in a closed loop. Furthermore, offline and online approaches resulted in different parameter choices for decoding algorithms (Cunningham et al., 2011). Therefore, the neural decoders and the motor prosthesis must be tested online even though online control experiments are more expensive both in terms of physical resources and time (Gilja et al., 2011). A recursive Bayesian decoder, i.e., a Kalman filter, was developed to decode the neural data recorded in the monkey motor and premotor cortex in response to goal-directed reaching movements (Shenoy and Carmena, 2014). It yielded high decoding performance and accurate trajectory prediction when the probability modeling assumptions were satisfied. For online purposes, a modified Kalman filter that transforms the acquired neural signals into a controller input was further designed for online cursor-control tasks and resulted in high performance in rhesus monkeys (Gilja et al., 2012). To adapt decoders to the dynamics of a prosthetic device and its environment, a likelihood gradient ascent and a self-recalibrating classifier were proposed to update decoder parameters during closed-loop BMI operation and normal use (Dangi et al., 2013; Bishop et al., 2014). Additionally, neural networks developed from probabilistic aspects were designed in an online setting (Sussillo et al., 2012) and in a real-time setting (Dethier et al., 2013).

A selection of cortical neurons, instead of all available neurons, used in the encoding process could improve the control performance of the neuroprosthetic system, such as robotic arms (Wahnoun et al., 2006). However, neural coding mechanisms evolve with time, individual experience, and the learning process (Nicolelis, 2001), i.e., the contribution from individual neurons may vary considerably from day to day (Carmena et al., 2003). It has been observed that neuronal activity for monkey was not as stable from day to day (Sadtler et al., 2014). The decoding algorithms which require previous day's observation of neural activity, such as PVA and OLE, may be affected by neuron's stability. For this reason, the selection of cortical neurons becomes an essential issue for the decoding processes, especially after continuous practice and learning. Furthermore, long-term inflammatory responses lead to a gradual decrease of recording quality and the eventual breakdown of the electrode's recording ability (Polikov et al., 2005; Schwartz et al., 2006). Losing neural signals over time will result in chronic coding failure. Thus, a decoder that has the capability to process recorded signals from a small number of neurons will become more important.

Sliced inverse regression (SIR) is a data-analytic tool that can effectively perform nonlinear regression based on a small number of inputs (Li, 1991). It divides the range of output variable into several intervals and partitions the input data into several slices according to the corresponding output value. Each slice consists of data with a similar contribution to output estimation. SIR then applies a weighted principal component analysis (PCA) to theses slice means of data to find effective dimension reduction directions for general and flexible setups. Each slice gains weight according to its contribution to output estimation. With a simple inverse regression model, SIR requires a low computational cost and retains reliably stable subspaces to extract primary information from noisy data with effective dimension reduction directions. Because of the good performance in dimension reduction and data de-noising, SIR has been widely applied to data-intensive marketing environments (Naik et al., 2000), data classification (Dai et al., 2006; Wu, 2008), and medical images (Wu and Lu, 2007; Lu, 2008; Tu et al., 2015).

It has been known that the number of recorded neurons in rodents is less than primates in BMI applications. Owing to the abilities of processing small number of input variables and slicing data for inverse regression model, SIR was considered as a neural decoding algorithm to provide realistic behavior interpretation for brain-derived control tasks in this study. A surrogate driving task with lever-pressing was designed for evaluating the efficacy of the proposed decoding algorithm in predicting motor functions of rats. The signals of lever-pressing and the spike trains related to the intended forelimb movement trajectories were simultaneously acquired during the task. This study adopts SIR to predict intended forelimb movement trajectories according to the recorded neurons from primary motor (M1) cortex. We presented experimental validation of the proposed decoding system using the recorded neurons to predict forelimb movement. We demonstrated that the proposed SIR decoding algorithm can not only extract primary features from the small number of neurons but perform more accurate prediction of intended forelimb movement trajectories than PVA and OLE which usually require hundreds of neurons in primates.

## MATERIALS AND METHODS

#### Animal Preparation

Four male Wistar rats weighing 250–300 g (BioLASCO Taiwan Co., Ltd., Taiwan) were used in this study. All rats were individually housed in a 12 h light-dark cycle (light from 7.00 to 19.00 h) at room temperature (22 ± 1 ◦C) with access to food and water ad libitum in the experimental animal center of National Yang Ming University. All experiments were conducted according to standards established in the Guide for the Care and Use of Laboratory Animals, which has been approved by the Institutional Animal Care and Use Committee at the National Yang Ming University.

### Animal Training and Behavioral Tasks

The rats were trained to use their right forelimb to press a lever to obtain the water reward for a week before electrode implantation, as shown in **Figure 1**. The rat was placed in the lab-designed Plexiglas testing box with a 15-cm tall lever above the floor on the left side (**Figure 1A**) and a water-feeder with a flow rate of 1 ml/time on the right side as shown in **Figure 1B** (Lin et al., 2016). **Figure 1C** shows the experimental setup where a rat was pressing a lever for the water reward. Before achieving the successful leverpressing training, all rats underwent water deprivation for 8 h per day. In this study, we have defined the criterion for the successful training was to consecutively repeat the lever-pressing and waterdrinking for five times during daily 5-h sessions (9:00–14:00), for 4 days at the most. Once the rats learned the association, they always kept the skilled concept (Lin et al., 2016).

## Surgery and Electrophysiological Mapping

Animals were anesthetized with pentobarbital (50 mg/Kg, i.p.) and were placed on a stereotaxic apparatus (Model 900, Kopf Instruments, Tujunga, CA, USA). A 16-channel stainless microwire electrode array (diameter of 0.002 ft., California Fine Wire Co., Ltd, Grover Beach, CA, USA) was inserted vertically and was implanted into layer V of the M1 cortex (2–4 mm anterior and 2–4 mm left-lateral to bregma; 1.7 mm ventral to the cortical surface) by referring to a previous work . Here, a standard intracortical microstimulation (ICMS) technique was conducted to deduce maps of rat forelimb movement representations in the M1 cortex, which could help assess the functional integrity of M1 cortex and activate pyramidal cell fibers. ICMS delivered a 40 ms stimulus train with 0.2 ms square-wave monophasic cathodal pulses at 350 Hz to the electrodes (impedance: 200–400 k at 1 kHz) by an isolated pulse stimulator at a rate of 1/s (Model 2100, A-M Systems Inc., Sequim, WA, USA). Because the intensity of the stimulating current depends on the distance between the neuron and the stimulating electrode, the threshold current intensity can be estimated by a strength-distance relationship as follows:

$$I = kr^2 + I\_m \tag{1}$$

where k = 1292µA/mm<sup>2</sup> for direct activation, r is the distance, and I<sup>m</sup> = 1µA. An implantation location of the electrode site was defined as valid when the rat forelimb was activated by ICMS with a current intensity less than 60µA. Then, a stainless steel screw was secured to the skull over the cerebellum as a reference electrode. Finally, the microwire electrode array was secured in the skull using dental acrylic (Type 1 Class 1, Hygenic Corp., Akron, OH, USA) and was covered with a small amount of 2% agar. For a better recovery, all rats were given an analgesia (Buprenorphine/Buprenex, 0.05 mg/kg s.c.; Reckitt Benckiset Healthcare Ltd, Hull, UK) every 8–12 h for 3 days and antibiotic treatment (Ampicillin, 100 mg/kg s.c. twice daily; Bristol Myers Squibb, New York, NY, USA) for 7 days after surgery. Following a 1-week post-surgery recovery period, all implanted rats received the behavioral task to use their forelimb to press a lever for water reward. The forelimb movements in the rat were captured by a charge coupled device (CCD) camera (DFK 21F04, Imaging Source, Bremen, Germany) and the neuronal signals were recorded by a Multi-channel Acquisition Processor (MAP, Plexon Inc., Dallas, TX, USA) through a 16 channel stainless microwire electrode array implanted in the rat M1 cortex. The detailed data recordings for forelimb movement and neuronal signals are described in the Supplementary Note 1.

### Trajectory Prediction Model

This study assumed that lever-pressing forelimb movement, which was considered to be a stereotyped movement, was performed at a nearly constant distance from the CCD in each trial, i.e., the distance did not vary dramatically. The recorded trajectory might consist of major forelimb movement and minor whole body movement which led to the coupling mechanism of two-dimensional forelimb movement (see Supplementary Note 1). The coupling mechanism resulted in a nonlinear relationship between neural activity and forelimb movement, and thus caused general linear regression to fail at forelimb movement prediction. SIR performs as a nonlinear regression since it can recover the most severe nonlinearity of the data by estimating effective dimensional reduction (e.d.r.) space (Li, 1991). Therefore, SIR

FIGURE 1 | Experimental setup and protocol. A perspective drawing (A) and vertical view (B) of the Plexiglas testing box. A lever is on the left side of the barrier and a water supply is on the right side. (C) A rat is using his right forelimb to press the lever to obtain a water reward. Simultaneously, his forelimb movement trajectory is videotaped by a camcorder approximately 25 cm away from the box, and the neuronal activities are recorded by the implanted electrode.

was adopted to predict the forelimb movement according to the neural activity in this study.

The two-dimensional trajectory movement vectors v<sup>x</sup> and v<sup>y</sup> in Cartesian coordinates are transformed into polar coordinates as follows:

$$\nu\_r = \sqrt{\nu\_x^2 + \nu\_\mathcal{y}^2} \tag{2}$$

$$\nu\_{\theta} = \tan^{-1} \frac{\nu\_{\circ}}{\nu\_{\times}} \tag{3}$$

where v<sup>r</sup> and v<sup>θ</sup> are the magnitude and direction, respectively.

The movement response g is assumed to be given by a deterministic function f with additive noise ε, so that

$$g = f\left(\beta\_1 z, \dots, \beta\_K z, \varepsilon\right) \tag{4}$$

where g is v<sup>r</sup> or v<sup>θ</sup> , and z is firing rate data in **R** p . Here, β's are unknown linearly independent projection row vectors, and K is the sufficient number of β's. The p-dimensional variable z is projected onto the K-dimensional space by functional relationship f , where p ≥ K. The combinations of β's are called the e.d.r. direction and the linear space produced by the β's are called e.d.r. space. The present study assumes the movement response g was predictable from K projected variables. To train the functional relationship f , a set of training data consisting of N training samples was prepared. According to the model assumptions, the centered inverse regression curve E z g − E (z) is included in the linear subspace, which is spanned by βk6zz k = 1, . . . , K , where 6zz represents the covariance matrix of z. SIR sorts and divides the whole data z into H intervals (slices) according to the g value. Each slice has almost equally number of observations. SIR then performs an eigenvalue decomposition of the weighted covariance matrix 6E(z|<sup>g</sup> ) with respect to 6zz. The weighted covariance matrix 6E(z|<sup>g</sup> ) is constructed as:

$$\Sigma\_{E(z|\lg)} = \frac{\sum\_{h=1}^{H} m\_h(\bar{z}\_h - \bar{z}) \ (\bar{z}\_h - \bar{z})'}{(N-1)} \tag{5}$$

where m<sup>h</sup> denotes the size of each slice, z¯ is the sample mean of z, and z¯<sup>h</sup> is the sample mean of the hth slice. The e.d.r. directions could be estimated by solving the generalized eigen-problem:

$$
\Sigma\_{E(z \mid \mathcal{g})} \beta\_{\dot{\jmath}} = \lambda\_{\dot{\jmath}} \Sigma\_{zz} \beta\_{\dot{\jmath}} \tag{6}
$$

where j = 1, . . . , p and λ<sup>1</sup> ≥ λ<sup>2</sup> ≥ · · · ≥ λp. Then, z was further projected onto the e.d.r. space by the first K e.d.r. directions as follows:

$$\mathbf{w} = [\beta\_1 z, \dots, \beta\_K z] \tag{7}$$

Then, a linear combination of w was performed to predict the forelimb movement. Although a linear combination approach was adopted, SIR was considered as a nonlinear regression since there is no linearity constraint on the prediction rules. Note that the user-specified parameters of SIR are only the number of slices H and the number of components K. It has been known that SIR can provide root n consistent estimates regardless of the choice of H. A previous study has demonstrated that the performance of SIR is less sensitive to the selection of H when H was set 5, 10, and 20 (Li, 1991). Furthermore, it has been found that the first component (K = 1) is close to the e.d.r. space. Therefore, H and K were set to 10 and one, respectively, in this study.

#### Time-Lags and Temporal Orders

In fact, the physical relationship between the neuronal signal and the forelimb movement may imply time-lags in the neuronal signal. Previous work indicates that a model that assumes that all cells exhibit the same time-lags is computationally simple (Wu et al., 2004). Then, the optimal time-lags could be found with an empirical setting for further improvement of the decoding task. A number of time-lags (0–5 time bins, at levels corresponding from 33 to 165 ms) were evaluated for trajectory prediction by SIR.

In addition to the time-lags, the temporal order of the input is another interesting factor for the decoding issue. The information at the n th time bin may have a relationship with that at the (n − 1) th time bin. Hence, both current and previous neuronal activities are important and are considered as the inputs for prediction. Therefore, a tapped delay line model of neuronal activities is adopted in this study where a third-order model would consider the n th , (n − 1) th, and (n − 2) th time bins as the input.

### Performance Evaluation and Statistical Analysis

This study computed the root mean square error (RMSE) between true and predicted forelimb movements from movement start to endpoint in order to examine the performance of proposed decoding algorithm (Srinivasan and da Silva, 2011). The experimental trails were randomly split 70/30% into training and testing sets for each rat. Therefore, the performance of the proposed decoding algorithm on the testing set could be evaluated on each rat individually. A 10-fold cross validation was applied to avoid capitalization on chance (Efron and Tibshirani, 1994).

For statistical analysis, the predicted performance (RMSE) from 4 testing sets (145 trails; rat 74: 24 trials, rat 102: 18 trails, rat 106: 78 trails and rat 129: 25 trails) were represented as the mean ± standard error of mean (SEM). Two-way ANOVA was calculated using effects of time-lag (bin number 0, 1, 2, 3, 4, and 5) and temporal order (1-oder, 2-order, and 3-oder of time bins) as the independent variables in order to determine if there were any differences in the decoding ability based on which parameters was employed. Post-hoc comparisons were conducted using a Tukey HSD post-hoc test and the significance level was corrected to ∗P < 0.002 using a Bonferroni correction for the comparison of six time-lags and three temporal order. MATLAB (MathWorks, Natick, MA, USA) was used for all statistical analyses.

#### RESULTS

To evaluate the decoding performance of the proposed algorithm for trajectory prediction, the majority of decoding methods, PVA, OLE, and NN, were implemented for comparison. Furthermore, two popular feature selection techniques, PCA (Wold et al., 1987) and sequential feature selection (SFS) (Aha and Bankert, 1996), were implemented for comparison of feature selection effectiveness. Although PCA was developed as dimension reduction of feature space, it could be considered as a way to select features from principle components. Then a linear regression approach was adopted to perform regression whose inputs were the features provided by PCA and outputs were forelimb movements. The number of principal components was selected according to the variance of the reconstruction error (Valle et al., 1999). The same regression procedure was applied to SFS. A set of time-lags (0–5 time-lags) was carried out to observe the effect of different delays between the neuronal activity and the forelimb movement in each method. Furthermore, a set of experiments was conducted to evaluate the effect of various temporal orders (1–3 temporal orders) used in each decoding method. The experimental data were recorded from four rats where the number of trials and number of neurons per trial for each rat are shown in **Table 1**. The average number of successful trials was 45.9 ± 4.3, and the average number of recorded neurons was 21.2 ± 2.6 units. The period of each trial was 0.4– 0.7 s before lever-pressing and 0.2–0.4 s after the lever-pressing.

### Neuronal Signal Pattern during Behavior Task

The task timeline in **Figure 2** presents the sequential images of the lever-pressing (**Figure 2A**) and the corresponding spike trains (**Figure 2B**). The spike trains were acquired from five neurons related to right forelimb movement and were represented as the neuronal activity histogram with 33 ms time bins. The neuronal activities distinctly increased approximately 0.4 s before lever-pressing (**Figure 2B**), corresponding to the second image at 01:56.332 s (**Figure 2A**). The maximum value in the histogram appears approximately 0.1 s before the leverpressing. The neuronal activity has a substantial reduction after the rat completes the lever-pressing and then it re-strengthens gradually because of the redundant movement off of the lever.

### Effects of Different Time-Lags and Temporal Orders

Results of a particular test trial (Rat 106, 29 units, and 52th trial) were shown in **Figure 3** where the trajectories were reconstructed by six decoding methods with delay activity of four time-lags. The actual trajectory (blue solid line) was compared with the decoded trajectories by SIR (black dashed line), OLE (red dashed line), PVA (green dashed line), PCA (magenta dashed line), SFS (cyan dashed line), and NN (yellow dashed line) in **Figure 3**. Furthermore, the results of one and three temporal orders were conducted to demonstrate the advantage of SIR with the requisite amount of input data as shown in **Figures 3A,B**, respectively. In the one temporal order experiment, the trajectories reconstructed by PVA and OLE obviously deviated from the actual trajectory more than that reconstructed by SIR. PCA and SFS, which perform feature selection, could achieve accurate prediction in the previous time steps but did not predict the latter time steps well. Similarly, a NN, which has learning ability for nonlinear regression, could predict the trajectory well in the first few time steps, but it did not have robust prediction performance because of the random initialization of weights that leads to prediction error. As the temporal order increased to three, all methods had more accurate prediction compared to those of one temporal order. Overall, SIR shows the best performance among the other methods, especially when the decoding methods used less neuronal activity information.


**Figure 4** presents the effects of various time-lags and temporal orders in each method. **Figure 4A** shows the results of SIR where the smallest RMSE (8.47 ± 1.32 mm) was obtained by using four time-lags (132 ms) and one temporal order. It can be seen that SIR with four time-lags could achieve a significantly smaller RMSE than those with various time-lags [F(5, 54) = 4.22, <sup>∗</sup>P < 0.002 with Two-way ANOVA with Bonferroni correction, N = 145]. Furthermore, the RMSE of SIR had no conspicuous variations among the three different temporal orders. As shown in **Figure 4B**, OLE achieved the smallest RMSE (17.22 ± 3.80 mm) by using four time-lags and three temporal orders. However, there was no significant enhancement of the prediction performance using OLE decoding with different time-lags and temporal orders. **Figure 4C** shows the results of PVA where the RMSE decreased as the number of temporal orders increases. PVA resulted in an average RMSE of 21.76 ± 8.11 mm when using one time-lag and three temporal orders. **Figures 4D,E** showed the results of PCA and SFS, respectively, where the features were selected via these two algorithms. PCA achieved a decreasing RMSE as the number of time-lags increased and obtained the smallest RMSE (19.13 ± 0.75 mm) when using four time-lags and three temporal orders. The results of SFS did not present a decreasing RMSE as the number of time-lags increased. SFS achieved the smallest RMSE (22.75 ± 2.01 mm) when using five time-lags and three temporal orders. **Figure 4F** shows the results of NN where the smallest RMSEs (16.75 ± 2.02 mm) were achieved by using three time-lags and two temporal orders. The RMSEs of NN did not present a regular trend as the number of time-lags increased. The forelimb movement predictions using OLE, PVA, PCA, SFS, and NN were not affected by either timelag or temporal order. No significant interaction of time-lag and temporal order was found in the decoding methods of OLE, PVA, PCA, SFS, and NN in comparison to SIR. These results indicated that SIR outperformed other methods for trajectory prediction.

## DISCUSSION

The main finding of this study is that a rat's forelimb movement could be successfully predicted and reconstructed using relatively few motor cortical neurons. In comparison with competing neural decoding algorithms including PVA, OLE, PCA, SFS, and NN, SIR presented an extremely superior RMSE in distance deviation between the reconstructed and real forelimb movement trajectories.

Previous studies indicated that neuronal activity discharged before the onset of the desired movement, such as the motor preparation period, and encoded behaviors (Chapin et al., 1999; Churchland et al., 2006). The kinematic parameters therefore were decoded and reconstructed with high accuracy using the neuronal activity before the onset of the movement. Hence, in this study, SIR and the competing algorithms decoded the neuronal activities during the motor preparation period for reconstruction of introduced upcoming lever-pressing. The results showed that SIR, OLE, and PCA achieved optimal efficiency when using the neuronal activities that led to the onset of forelimb movement for 132 ms. PVA, SFS and NN each

presses down the lever with the right-forelimb. Moreover, our results showed that neuronal firing rates highly correlated with forelimb movement; >71% (41/57) neurons exhibited specific firing changes during movement used to discriminate directional pairs.

FIGURE 3 | Reconstructed trajectories of the test trial with the use of delayed activity with four time-lags (132 ms). The actual trajectory (blue solid line) and the trajectories predicted by SIR (black dashed line), OLE (red dashed line), PVA (green dashed line), PCA (magenta dashed line), SFS (cyan dashed line), and NN (yellow dashed line) are shown for an example trial using (A) one- and (B) three- temporal orders. The trajectory reconstructed by SIR is more accurate than the other methods.

achieved optimal efficiency by using the neuronal activities that led to the onset of forelimb movement for 33, 165, and 99 ms, respectively. In **Figure 2B**, the peak of the spike train occurred at four time-lags, prior to the onset of lever-pressing. The neuronal firing rate then declined for 0.2 s as a result of the completion of motor command transmission. Because the rat performed an

unexpected forelimb swing, the neuronal firing rate increased again approximately 0.2 s after to the onset of lever-pressing.

The performance of cortical neural decoding hinged on the exploited information in chronically-recorded neuronal activities. Previous studies using PVA (Schwartz, 1994) and OLE (Salinas and Abbott, 1994) show that cortical neurons with known motor associations were chronically sampled and as many as possible were recorded. Because of the lack of a precise technique to target the modulated neurons that largely contributed to goal-directed behavior, PVA and OLE summed the weighted vectors across all neurons, performing a neuronal vote, to predict the kinematic parameters (Salinas and Abbott, 1994; Schwartz, 1994). A large number of electrodes and sample neurons (usually up to hundreds) was required forreconstruction of kinematic parameters with a high degree of accuracy (Chapin et al., 1999; Wessberg et al., 2000; Serruya et al., 2002; Taylor et al., 2002). However, the neuronal activity was not as stable from day to day (Sadtler et al., 2014). PVA and OLE may be affected by neuron's stability since they extract movement information from the selected cortical population.

In this study, we recorded only tens of neurons from rat M1 where the amount of recorded neurons was insufficient for PVA and OLE, which usually require hundreds of neurons to provide a robust neural decoding process (Takeda and Funahashi, 2004; Wahnoun et al., 2006). Compared to PVA and OLE, SIR can effectively achieve nonlinear regression from a small number of inputs (Li, 1991). SIR adopted a sliced regression framework with a sorting procedure to divide the neuronal dataset into several slices according to the sorted output variable value. Each slice contained neurons with a similar contribution to the introduced lever-pressing and was then modified by a weight. Slices containing neurons with tiny or even a null contribution to the lever-pressing may gain zero weight and can be removed from the decoding model. Multiplied by a proper weight according to the weighted PCA, a slice containing a few neurons with a high contribution presented a comparable influence on the prediction and reconstruction of the introduced lever-pressing to the neuronal vote from hundreds of neurons. Hence, SIR is able to perform forelimb prediction through a small number of neurons. On the other hand, dimensionality reduction technique factor analysis is usually adopted to describe population activity using low-dimensional set of factors and highlight feature of interest in data from a large number of recorded neurons (Sadtler et al., 2014). Although SIR could perform dimensionality reduction through weighted PCA, it preserved all recorded neurons and assigned weights to the slices according to their contribution. It learned forelimb movement prediction from whole neuronal activities regardless of neuron's stability across days. Thus, SIR was robust to uncertain variation of movements and neuronal activities across days due to the success of inverse regression and effective dimension reduction. Furthermore, using this SIR, the size of the neural decoding model topology was significantly reduced, burdened with data storage and reduced computational loading, indicating that efficiency in neural decoding in comparison to PVA and OLE was attainable. Compared to PCA and SFS, which perform feature selection and dimensional reduction, PCA could further project the data onto another space, which could lead to a better reconstruction than SFS. However, SIR outperformed PCA and SFS because SIR clusters data into each slice according

to the output values. NN performed better prediction than PCA and SFS because of its nonlinearity and learning ability. Nevertheless, NN did not result in a robust reconstruction because of the mechanism of random initialization and the existence of many local optima. These comparisons indicate that the neural decoding based on SIR with one temporal order presents a smaller RMSE in reconstructing limb movement than those based on PVA or OLE with three temporal orders and those based on feature selection and learning ability. It indicates that SIR can be a more suitable solution than the commonly used linear progression methods using the neuronal ensemble inputs to predict and reconstruct the introduced limb movement.

### CONCLUSIONS

Neural decoding models that require hundreds of input variables, such as PVA and OLE, not only require considerable computation but also have detrimental effects in the decoding process because of errors in assigning neuronal spikes or nonstationary noise, especially for non-adaptive models. Reducing the neurons that may cause model over-fitting emerges as a significant neural decoding issue. However, with the help of the proposed approached based on SIR, researchers can predict and reconstruct the limb movement of interest with high accuracy using only tens of neurons in a single setting. Furthermore, SIR outperformed other feature selection methods, such as PCA and SFS because of its clustering ability. SIR further achieved more robust performance than NN because there is no random initialization and local optimization problems in SIR. By indexing the contribution of multiple cortical areas with different sizes, it has become feasible to ascertain the importance of selected areas for the motor commands. This will provide valuable insights for follow-up studies in the future.

#### REFERENCES


#### ETHICS STATEMENT

The animal use protocol listed below has been reviewed and approved by the Institutional Animal Care and Use Committee (IACUC) at the National Yang Ming University. Protocol Title: Peripheral Nerve Prostheses: A Paradigm Shift in Restoring Dexterous Limb Function. IACUC Approval No: 1050622. Period of Protocol: Valid From: 08/01/2016. To: 07/31/2020. There is no endangered animal species involved in this study.

### AUTHOR CONTRIBUTIONS

SY, HYL, and YC designed the project, organized the entire research. SL, LL conceived the experiments. CW, PC, TP, HCL, and WH conducted the experiments. HC, HHSL, and YL analyzed the results. SY, HYL, and YC wrote the manuscript. All authors discussed the results and reviewed on the manuscript.

#### ACKNOWLEDGMENTS

This research is financially supported by the Ministry of Science and Technology of the Republic of China, Taiwan under Contract numbers of MOST 105-2221-E-010-014-MY2, 103-2320-B-010- 014-MY2, 103-2321-B-010-016, 102-2221-E-010-011-MY3, 103- 2515-S-035-002 and National Health Research Institutes grant of BN-105-PP-15 and the Zhenjiang University, China under the Fund number of 181110- 193544B01/007.

### SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: http://journal.frontiersin.org/article/10.3389/fnins. 2016.00556/full#supplementary-material


Coding of the direction of movement by a neuronal population. J. Neurosci. 8, 2928–2937.


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2016 Yang, Chen, Lin, Liao, Lu, Wang, Chen, Lo, Phan, Chao, Lin, Lai and Huang. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Proof of concept of an Online eMg-Based Decoding of hand Postures and individual Digit Forces for Prosthetic hand control

#### *Alycia Gailey1 , Panagiotis Artemiadis2 and Marco Santello1 \**

*1School of Biological and Health Systems Engineering, Tempe, AZ, USA, 2School for Engineering of Matter, Transport, and Energy, Arizona State University, Tempe, AZ, USA*

#### *Edited by:*

*Marc Slutzky, Northwestern University, USA*

#### *Reviewed by:*

*Adenike Adewuyi, Northwestern University, USA Matthew R. Williams, U.S. Department of Veterans Affairs, USA*

> *\*Correspondence: Marco Santello marco.santello@asu.edu*

#### *Specialty section:*

*This article was submitted to Neuroprosthetics, a section of the journal Frontiers in Neurology*

*Received: 30 July 2016 Accepted: 06 January 2017 Published: 01 February 2017*

#### *Citation:*

*Gailey A, Artemiadis P and Santello M (2017) Proof of Concept of an Online EMG-Based Decoding of Hand Postures and Individual Digit Forces for Prosthetic Hand Control. Front. Neurol. 8:7. doi: 10.3389/fneur.2017.00007*

Keywords: myoelectric hand, neuroprosthesis, machine learning applied to neuroscience, neurorobotics, brain– machine interface

## INTRODUCTION

A significant challenge faced in modern medicine is in replacing a lost hand for upper limb amputees. The human hand performs many complex functions in the activities of daily living. One area of challenge is in the tradeoff between functionality and ease of use. A prosthetic hand that performs more functions will generally require more cognitive effort from the user. In contrast, a prosthetic hand that is simpler to control generally has more limited functionality.

According to an epidemiology study by Dillingham and colleagues (1), from 1988 to 1996 about 134,000 Americans underwent upper limb amputations from trauma and another 29,400 Americans lost their upper limbs to dysvascular disease. Though there are already a number of different upper limb prostheses available, the rate at which individuals with upper limb loss stop using the prostheses is significant. A meta-analysis by Biddiss and Chau (2) showed abandonment rates of 26% in body-powered prosthesis users and 23% in electrically powered prosthesis users. Major reasons for these abandonment rates include heavy weight, lack of functionality and durability, discomfort, poor cosmetic appearance, and finally too much effort required to control the prosthesis.

One of the main remaining challenges for prosthetic hand developers is in allowing the user to reliably control many different hand movements without too much cognitive effort. Bodypowered systems are reliable, but their harness system can result in fatigue and strain (2). Furthermore, body-powered prostheses are limited in their functionality. Control systems based on electroencephalographic (EEG) signals can be used to control prosthetic hands for above-elbow amputees and paralyzed individuals (3, 4). However, the implementation of these systems tends to be challenging because EEG signals are associated with many other behaviors besides hand motion, such as proximal musculature involved in hand transport, trunk movement, and so forth. Other methods are being developed to extract signals from within the brain or peripheral nerve tissue, but such methods are invasive and expensive (5).

Myoelectric systems are based on electromyographic (EMG) activity of residual muscles following an amputation and offer several advantages relative to the above-described systems. Specifically, EMG-controlled systems are non-invasive, and they take advantage of signals recorded from residual muscle activity that is specifically involved in the task. Numerous systems have been developed for recording surface EMG signals from the upper limb and extracting features to predict in real-time grasp postures and/or forces, or for individuals with upper limb loss, predicting the user's intended hand movement. Castellini et al. (6) demonstrated the use of EMG in predicting hand postures in healthy able-bodied individuals. Castellini et al. (7) further demonstrated that EMG signals from the residual muscles of amputees can predict five different imagined grasp poses to accuracies around 79–95% and grip force with accuracies between 7 and 17% normalized root mean square error (NRMSE) (7). The grasp poses included open hand, closed fist, 2-digit pinch, tripod (3-digit) grip, and index finger pointing.

In the same year, Yang et al. (8) demonstrated the use of EMG in the forearm to predict one of 18 possible combinations of finger flexion/extension for the thumb, index, and middle finger. It was found that while the classification accuracy was quite high when training and testing on data collected within the same session, training on data in an earlier data collection session and testing on a later session yielded much lower classification accuracies (50–60%). Subsequently, Castellini and Kõiva (9) demonstrated the use of a myoelectric control system that allowed 12 able-bodied subjects to modulate individual finger forces when the hand lay flat on a surface with each finger placed on top of force sensors. Although this system is a good proof of principle of myoelectric control of finger forces, this type of control was not demonstrated in a grasping task when the subject's hand is not laid flat on a surface, but rather is in a fist. In addition, this work does not account for wrist rotation during grasping tasks, although future work should focus on testing the robustness of our decoder across wrist postures that are commonly found in activities of daily living. Later work by Cipriani et al. (10) examined real-time myoelectric control of grasp types by individuals with upper limb loss with nine EMG electrodes placed along either side of the residual forearm muscles. Predicted gestures included open hand, closed fist, thumbs up, index finger pointing with extended thumb, flexed thumb with four extended fingers, and 3-digit grip. Average online control classification accuracy was 79% for transradial amputees and 89% for able-bodied subjects. However, this work did not examine online control of digit forces.

The present work attempts to expand upon the work of Castellini et al. (7) by implementing online control of hand postures—as opposed to offline cross-validation—and online control of individual finger forces, which allow subjects to modulate the distribution of force across the fingers of the prosthetic hand. The system is programed in Matlab, and therefore does not implement true real-time control. However, the delay between a change in EMG signals and a change in desired hand motion is only 0.3 s. For a proofof-concept demonstration of our approach, we used the commercially available i-limb hand (Touch Bionics) because it has a separate motor for each digit and each digit can exert seven different levels of grip forces, thus allowing for some degree of force modulation by individual digits. We examined several grasping tasks, including lifting the object using a chosen grasp type (2-digit, 3-digit, or whole-hand grasp), alternating between grasp types while holding the object, and modulating the grip force during the object holding phase. Because the i-limb hand has certain limitations (limited speed of motion, limited ability to exert a discrete set of forces per digit), we tested this system mostly by using a computer program to give online feedback of finger forces and grasp type for each loop iteration, while simultaneously instructing the subject to perform different hand poses. The system's performance is demonstrated on able-bodied individuals before future testing on individuals with upper limb loss.

### MATERIALS AND METHODS

#### Subjects

Eight right-handed subjects (age: 23.5 years, SD: ±3.42, five males, three females) participated in the study. We recruited subjects who identified themselves as right handed. Subjects had no history or record of neurological disorders and had never performed tasks involving myoelectric control of an external device. Subjects gave informed written consent to participate in the experiments. The experiments were approved by the Institutional Review Board at Arizona State University and were in accordance with the Declaration of Helsinki. Each experimental session (one session/subject) lasted approximately 1.5–2 h.

### Experimental Protocols

We asked subjects to perform two sets of tasks. In the first set of tasks, subjects were asked to perform a series of grasping and finger pointing tasks (*Task 1: hand postures*; **Figure 1A**). In the second set of tasks, subjects were asked to vary the distribution of normal forces among the fingers in a closed fist (*Task 2: digit forces*). For both tasks, we recorded EMG signals from five surface

EMG electrodes and extracted features from these signals to train a one-against-one support vector machine (SVM). This SVM was used to distinguish hand poses, and a random forest regression (RFR) was used to predict each of the five digit forces. Subjects performed both tasks in the same experimental session. Below we describe the procedures for both training and testing of our EMG decoder system.

### EMG Recording, Signal Processing, and Feature Extraction

We placed five surface EMG electrodes (Delsys) around the circumference of the forearm just below the elbow (**Figure 1B**). The electrodes were roughly equally spaced apart. At least one EMG electrode was placed over the m. extensor digitorum superficialis (finger extensor) on the dorsal surface of the forearm. The other electrodes did not target specific muscles. A reference EMG electrode was placed directly over the lateral epycondyle of the humerus. Prior to EMG electrode placement, the area was cleaned with rubbing alcohol pads. We did not target specific muscles because this approach is not always feasible when using surface EMG from residual muscles in individuals with upper limb loss. Specifically, availability of specific target muscle depends on the extent and state of residual muscle fibers following amputation which, in turn, may affect the EMG signal quality and the extent to which it can be used for hand posture or grasp force decoding. Thus, we aimed at using a muscle-independent EMG decoding approach to resemble a more realistic scenario of extracting features from non-specific forearm muscles.

Our preliminary online and offline testing showed that increasing the number of electrodes did not improve the hand pose classification accuracy. We had also performed preliminary offline testing on a high-resolution electrode array (90 channels) and found that hand pose classification was more difficult due to lower signal quality. Furthermore, work by Castellini et al. (7) showed that five electrodes placed around the circumference of the forearm allowed for prediction of desired hand pose in three amputees. This work, as well as our study, suggested that the quality of the EMG signal might be more important for hand pose and finger force decoding than the number of EMG channels.

All EMG signals during training and testing were analyzed in individual 50-ms non-overlapping time windows for the purpose of enabling online control. Although time windows longer than 50 ms may improve prediction accuracy, we chose a 50-ms time window length based on the need to reduce control delays due to data processing (see below). However, we note that pilot testing revealed that longer time windows would not have significantly improved prediction accuracy. Furthermore, Castellini et al. (7) reported that a time window of 50 ms was sufficiently long to make predictions of desired hand motions in amputees. Control was quasi real time as the processing delay was about 0.3 s. For each EMG signal-recording interval, the computed signal average was subtracted to center the signal amplitude about 0. Next, all points in the time interval were normalized to a range spanning −1 to +1 where a value of ±1 represents the EMG magnitude recorded during maximum voluntary contraction (MVC). MVC EMG was recorded by asking subjects to perform six maximum isometric force contractions:

isometric finger flexion, finger extension, wrist flexion, wrist extension, wrist abduction, and wrist adduction. For each EMG channel, the root mean square (RMS) was computed for each 50-ms time window throughout the entire MVC trial. The maximum RMS value computed for each channel during the MVC recording was used as the EMG magnitude representative of MVC for that channel. All signals collected in that channel in subsequent recordings were centered at 0 and then divided by the MVC magnitude.

Electromyographic signals were amplified in hardware (gain: 1,000; Delsys Bagnoli-8 EMG System) before being digitized and analyzed in software. After subtraction of the mean and normalization of the EMG signal, 60 Hz line noise was filtered by passing the signal through a 60-Hz notch filter with 3-dB cutoff and bandwidth set to 20 Hz. Next, features were extracted from each filtered 50-ms time window. We explored various EMG signal features presented in previous work [e.g., Zecca et al. (11) and Khushaba et al. (12)] including mean absolute value, variance, Willison amplitudes, mean of amplitudes, auto-regressive coefficients, and other novel features presented by Khushaba et al. (12). Of these features, three were found to be most informative for predicting hand posture. The first chosen feature is the RMS, computed according to Eq. 1:

$$\text{RMS} = \sqrt{\frac{1}{N} \sum\_{t=t\_i}^{t\_i} \mathbf{x}^2(t)}\tag{1}$$

where the term *x*(*t*) denotes the EMG signal value at time *t*.

The next two features were derived by Khushaba et al. (12). These features were computed by the zero-order moment (same as RMS above), *m*0, the second-order moment, *m*2, and the fourthorder moment, *m*4. These moments were computed according to Eqs 2–4:

$$m\_0 = \int\_{t-T}^{T} \mathbf{x}^2(t) \, dt \tag{2}$$

$$m\_2 = \int\_{t-T}^{t} \left(\frac{dx}{dt}\right)^2 dt\tag{3}$$

$$m\_4 = \int\_{t-T}^{t} \left(\frac{d^2x}{dt^2}\right)^2 dt\tag{4}$$

The variable *T* is the length of time for one time window. After computing the moments, the irregularity factor, IF, and the wavelength variable, WL, were computed according to Eqs 5 and 6:

$$IF = \sqrt{\frac{m\_2^2}{m\_0 m\_4}}\tag{5}$$

$$\text{WL} = \int\_{t-T}^{T} \left| \frac{d\mathbf{x}}{dt} \right| dt \tag{6}$$

One of the features (*f*) used was dependent on the zero-order moment or RMS, computed according to Eq. 7:

$$f\_2 = \log\binom{m\_o}{\nearrow}\_N \tag{7}$$

The variable, *N*, is the number of samples within each time window. The third feature type was computed according to Eq. 8:

$$f\_{\mathfrak{s}} = \log\left(\overset{\text{IF}}{\underset{\text{WL}}{\overset{\text{(IF}}{\underset{\text{ML}}{\rightleftharpoons}}}}\right) \tag{8}$$

Each type of feature was computed not only for each individual EMG channel but also for each combination of channel pairs where the feature value of one channel would be subtracted from the feature value of the other channel, thus yielding 15 features of each feature type. To identify non-redundant features, we performed linear correlation between feature pairs. As many features were highly correlated with each other, we did not use all features for hand posture and digit force prediction. Therefore, we chose the 15 features characterized by the weakest correlation with each other. For each pair of features that were highly correlated, we chose the feature that changed the most between grasp types. We found that the features that changed the most between grasp types also were highly correlated with each other, and were therefore redundant. Selecting features least correlated with each other yielded less redundancy, and therefore more information. As such, the same set of features was used in distinguishing all pairs of hand poses.

#### Force Data Recording and Processing

Normal forces exerted by each digit were measured by five force/ torque (F/T) sensors (ATI Industrial Automation) mounted on a grip device grasped by the left hand. Four F/T sensors (Nano-17) were mounted on the finger side of the grip device, and an F/T sensor (Nano-25) was mounted on the thumb side. During recording sessions, the subject grasped the force-sensing device with one digit on each force sensor, and force data were collected synchronously with EMG data to form a mapping between EMG and each digit force.

#### EMG and Force Data Processing

Electromyographic and force data were acquired at 1 kHz by 12-bit analog-to-digital converter boards (NIDAQ PCI-6225, National Instruments; sampling frequency: 1 kHz). EMG data recording was performed through Matlab (Mathworks), which forces data recording through LabVIEW (version 8.0, National Instruments). EMG data and force data were synchronized by a trigger pulse sent by the LabVIEW program to the NIDAQ board at the start of the recording. This pulse appeared in the EMG recording in Matlab.

#### Training Data Collection

After recording MVC EMG, the subject was prompted to relax the hand for 1 min while EMG signals were recorded. RMS was computed for each 50-ms time window of the resting period and the resting threshold was computed as 1.5 times the average magnitude of the EMG signal in each channel. When the RMS of all five EMG channels was below their resting threshold, the computer program predicted that the hand was at rest.

The EMG decoder predicts one of six possible hand poses. The hand poses are shown in **Figure 1A**. We chose these poses to capture basic grasp types [2- and 3-digit precision grasps where the thumb contacts the fingertips; closed fist to approximate a "power" grasp (13)], as well as non-prehensile hand postures (rest, open hand, and index finger pointing). To form a mapping between EMG signals and hand pose/digit forces, data were collected from 12 training trials lasting 30 s each. For trials 1 and 2, the subject extended all fingers. About 15 s after the start of the recording, the subject was told to co-contract the hand muscles while keeping fingers extended for about 7–8 s. For trials 3 and 4, the subject shaped the hand into a 2-digit grasping pose where the thumb and index finger pressed against each other at the tips and the other three fingers were extended (**Figure 1A**). Fifteen seconds after the start of the recording, the subject was told to increase the pinch force for about 7–8 s. For trials 5 and 6, the subject did an index finger pointing (**Figure 1A**). Subsequently, the subject was told to increase and then decrease the muscle co-contraction. For trials 7 and 8, the subject did a 3-digit grasp (index finger and middle finger pressing against the tip of the thumb while ring finger and little finger were extended; **Figure 1A**). Fifteen seconds after the start of the recording, the subject was told to increase the pinch force.

For trials 9–12, subjects made a fist with the ipsilateral hand and grasped the force-sensing object with the contralateral hand (**Figure 1B**). The subject was asked to keep flexion forces approximately the same for both hands. For trial 9, the subject began by making a fist using a small grip force, and then the subject shifted most of the force onto the index finger with the ipsilateral hand while still maintaining a closed fist. With the contralateral hand, the subject exerted most or all of the pressure on the force sensors with the thumb and index finger to teach the RFR that most of the force was concentrated on the index finger and thumb. The same procedure was repeated for the middle finger. For trial 10, the process was repeated for the ring finger and little finger. For trials 11 and 12, the subject started the trial with the ipsilateral hand in a relaxed fist and the contralateral hand exerting minimal force on all five force sensors. The subject was then prompted to ramp up the flexion force across all digits on both hands to a moderately high value, maintain the higher grip force for about 5 s, and then ramp the force back down.

The rationale for having subjects make a fist with the right ipsilateral hand during training while grasping the force sensors with the contralateral hand was that this approach is more feasible for training individuals with unilateral (ipsilateral) upper limb loss. Specifically, these individuals can use the intact (contralateral) hand to grasp force-sensing object while sending similar motor commands to the ipsilateral hand to trying to match forces across the two limbs.

One of the goals of the system in the current study is to allow the user to grasp the object with the prosthetic hand by simply making a fist with the real hand, and then pressing the fingers into the palm in order to modulate the distribution of force across the fingers. EMG signals can change depending on the size of the object being grasped. Therefore, if EMG signals are recorded while the subject is grasping a force-sensing object of a certain size, then the subject may need to grasp an object of the same size during online control in order to generate the desired finger forces.

### Training the Machine Learning Classifiers and Regressions

**Figure 2** shows a block diagram of the training portion and testing portion of our system. 15 The training portion inputs data from the subject to form a mapping between EMG signal and 16 hand pose, and a mapping between EMG signal and finger forces. The testing portion of the 17 system inputs EMG signals and makes online predictions of hand pose and finger forces.

To distinguish among the five hand postures, we used a oneagainst-one radial basis kernel SVM. Such a system consists of one SVM binary classifier to distinguish each pair of hand poses. When one hand pose is guessed in favor of the other one, the hand pose receives a vote. The hand pose receiving the most votes across the different SVM classifiers gets selected. The hyperparameters were determined by offline testing on preliminary pilot data. Such testing would train on some data and test on the other data. The classification errors were computed for each set of hyperparameters, and the hyperparameters yielding the lowest classification errors were selected. For more information on SVMs and the one-against-one multiclass classification methods, the reader is referred to Hsu and Lin (14). To distinguish each pair of poses, 15 features were selected with the lowest correlation with each other. For digit force predictions, the 15 RMS features were used to train a RFR, which used 50 trees and all 15 features/tree. For a description of RFR algorithm, see Breiman (15). A separate RFR was trained for each digit force.

### Online Myoelectric Control: Finger Force Prediction through a Computer Program

Regulation of force is difficult with the i-limb hand because (1) the i-limb hand commands only consist of integer values from −7 to +7, thereby limiting the resolution of force exertion and (2) if force is too high, it is difficult to reduce the force on a given finger by a specific amount unless extensive calibration is employed to determine the duration of the extension command necessary to decrease the force by a specific amount without losing contact. Furthermore, (3) the wireless Bluetooth-based communication required to receive commands tended to be unstable in our setup. Therefore, most of the online testing of the quality of myoelectric control was performed using a custom-made computer program. This program printed to the screen the predicted flexion forces for each digit during each loop iteration (~3 loop iterations/second). The computer program prompted the subject either to perform grasping tasks or to make a fist and vary the force distribution among the fingers.

For simulated grasping tasks using the computer program, 50 ms of EMG data were recorded for each loop/iteration. EMG signal features were extracted and input into the SVM classifiers for hand posture prediction, and RFRs for digit force prediction. Flexion and extension forces for each digit were printed to the screen at the end of the loop iteration. A force value greater or lower than 0 indicated predicted flexion or extension, respectively. A value equal to 0 indicated a predicted resting state. If all printed digit forces were negative, then the predicted hand pose was open hand. If all printed digit forces were positive, then the predicted hand pose was closed fist. If only the top 2-digit forces (index finger and thumb) were positive, then the predicted hand pose was a 2-digit pinch grip. If only the index finger force was negative and all other forces were positive, then the predicted hand pose was an index finger point (which could be performed while grasping an object).

Initially, the subject was given time to familiarize with the computer program used to display predicted finger forces by learning to associate the printed forces on the screen with specific hand poses. If digit force control was not sufficient, then training would be repeated for specific hand poses that were found to be more difficult to predict. To test the quality of myoelectric control, commands were posted on the screen to the subject about what hand posture to adopt and what target force to reach during grasping.

The computer program would summate the normal forces of the digits in contact and print the total grip force in the space below the individual digit forces. When the subject was asked to grasp, a target grasping force was printed to the screen in the area below the predicted total grip force. We chose submaximal target grasping forces (ranging from 6 to 30 N for a 2-digit and power grasp, respectively) as these are typically associated with activities of daily living. An example of force feedback (printed to the screen once per loop iteration) is shown below:

#### INDEX-THUMB GRASP

Thumb: 5.62 Index finger: 4.32 Middle finger: 0.05 Ring finger: −1 Little finger: −1 9.944 N Target Force: 6 N

Each grasp task consisted of 100 iterations, at roughly 300 ms/ loop iteration. Below is the list of commands given to the subject in chronological order:


To factor out response time of subjects to the change in command, the program paused for 4 s whenever there was a change in the command given to the subject.

For the second set of exercises, subjects made a fist the entire time and then varied the magnitude and distribution of flexion force across the digits. For the first 20% of iterations, subjects began with a relaxed fist (minimal flexion forces) and when prompted, made a tight fist. Every 20 iterations, subjects transitioned from tight to relaxed fist, or from relaxed to tight fist. For the second 20% of iterations, subjects were prompted to shift the force to the little finger. For the third 20% of iterations, subjects were prompted to shift the force to the ring finger. For the fourth 20% of iterations, subjects shifted the force to the middle finger. For the fifth 20% of iterations, subjects shifted the force to the index finger. Both sets of trials were performed five times. Subjects alternated between grasping task rounds and force-shifting task rounds, making a total of 10 rounds. We accepted all attempts at achieving the target force. The measure of how close the subject attained the target force was captured the NRMSE, which was computed as the square root of the difference between the target and predicted force divided by the target force.

#### Testing with the i-Limb Hand

For practical demonstration purposes, the myoelectric control system was tested on a commercially available prosthetic hand, the i-limb (Touch Bionics). For each loop iteration, commands were wirelessly sent to the i-limb hand where the integer value of the flexion command was proportional to the predicted flexion force. The subject was asked to grasp the 5-digit force-sensing object (one force sensor/digit) using a 5-digit grasp, 2-digit pinch grip, and a 3-digit tripod grip. The subject was only required to complete one successful trial for each grasp type. Finally, the subject was prompted to alternate grasp types during object hold. For this trial, the subject used myoelectric control to grasp the object with five digits. Upon command, the subject released the index finger to do an index finger point, and then upon command, the subject returned to the 5-digit grasp. Next, the subject was prompted to release 2 or 3 of the fingers and transition to a 3-digit or 2-digit grasp without losing contact. Finally, the subject released the object. For all grasping tasks with the i-limb hand, a successful trial was defined as a trial during which the subject could hold a grasp for about 5 s without unwanted hand opening, and then release the grasp on verbal command.

To quantify the level of difficulty in completing simple grasping tasks with the i-limb hand, subjects were asked to rate each i-limb task level of difficulty on a scale from 1 to 5. On the scale, 1 = "easy," 2 = "able to do the task," 3 = "able to do the task, but with effort," 4 = "moderately hard," and 5 = "very hard or could not do."

#### Data Analysis: Predicted Finger Forces and Force Distributions

For trials concerned with grasping tasks, variables of interest included the NRMSE of the grip force across grasp types, and confusion matrix that relates predicted hand posture to desired hand posture. The error of the grip force was computed as the difference between the total predicted grip force and the target force. Total grip force was computed as the summation of normal forces across all digits that were supposed to be flexed. For hand pose predictions, the overall percent accuracy was computed as the percent of the loop iterations during which the predicted hand pose was correct out of the total of 500 loop iterations. Another variable of interest is the number of times that an open hand is predicted when the correct hand pose is a grasp. Such a situation would indicate unwanted loss of contact.

For trials concerned with force distribution across the fingers, the finger forces were summated for each loop/iteration, and then the fraction of the total finger force contributed by each finger was computed. For loop iterations 101–200, the little finger was expected to contribute the highest fraction of total finger force out of all the other fingers. Similarly, for loop iterations 201–300, the ring finger was expected to contribute the highest fraction of total finger force. For loop iterations 301–400, the middle finger was expected to contribute the highest fraction of total finger force. For loop iterations 401–500, the index finger was expected to contribute the highest fraction of total finger force. Data analyses examined the extent to which the force distribution across fingers changed as the subject was prompted to shift the flexion force from one finger to the next. Force distributions were measured as the percent of total finger force that was contributed by each finger during each time epoch. Force distribution index was used as an additional measure of force distribution, with a value of −1 for all force concentrated on the little finger, a value of +1 for all force concentrated on the index finger and a value of 0 for evenly distributed forces.

When force distribution changed across fingers, the myoelectric control system may predict an incorrect grasp type. To investigate this issue, the variable of interest was the percentage of the time that the 5-digit (correct) hand posture was predicted.

#### Statistical Analysis

The first statistical analysis focused on performance during the in-hand force-shifting task. A repeated-measures analysis of variance (ANOVA) was performed on the force distribution index (range: −1 to 1) with three within-subjects factors: round of trials (*Round*; five levels), time epoch within each round (*Time epoch*; five levels), and first half versus second half of each time epoch (*Sample*; 2 levels). Subjects performed five rounds of the force-shifting tasks. Changes in force distribution across rounds indicate whether there was significant change in performance with practice. Changes in force distribution across time epochs indicate whether there was a significant change in force distribution across the fingers when subjects were prompted to change their force distribution from one finger to the next. The purpose of analyzing the factor *Sample* was to examine if performance was dependent on the length of time that subjects were asked to maintain a force distribution concentrated on one particular finger. Because each loop/iteration lasted approximately 300 ms, each time epoch was about 30 s long.

A second ANOVA was performed on overall hand pose prediction accuracy for the grasping tasks using one within-subject factor (*Round*; five levels). Results of this analysis would indicate whether overall prediction accuracy improved with practice.

### RESULTS

The EMG control system design presented here is for demonstrating the use of machine learning techniques to decode five surface EMG signals from the forearm to predict desired hand motion. A machine learning-based mapping was created between EMG signal features and individual finger movements, allowing online control of individual finger movements in a robot hand. The end goal is to allow subjects to open the hand, grasp an object using a chosen grasp type, and execute an index finger pointing for gesturing or typing. When grasping, subjects were told to use the EMG decoder system to modulate

not only the amount of grip force but also the distribution of grip force across the fingers. To test system performance, subjects were prompted to perform grasping tasks and finger pointing tasks with a virtual hand in an *ad hoc* computer program where finger and thumb forces were printed to the computer screen at the end of each loop iteration.

#### Grasping Task Performance

**Figure 3** illustrates task performance during the third round of virtual grasping tasks for a representative subject. Each of the five grasping task rounds consisted of 500 loop iterations, and each grasping task within a round consisted of 100 loop iterations (~300 ms/loop iteration). For the first 100 loop iterations, the subject was prompted to open the hand, do a 2-digit grasp, and then open the hand again. During the next 100 loop iterations, the subject was prompted to repeat with a 3-digit grasp. During loop iterations 200–300, the subject was prompted to perform a 5-digit grasp. During loop iterations 300–400, the subject was prompted to perform a 5-digit grasp, and then transition back and forth between a 5-digit grasp and an index finger point. During loop iterations 400–500, the subject was prompted to perform a 5-digit grasp, and then transition back and forth twice between a 5-digit grasp and a 2-digit grasp. The subjects were instructed to hold each hand pose for a fairly long duration to assess not only subjects' ability to achieve a given hand pose but also to maintain it.

#### Confusion Matrices

We created confusion matrices to illustrate the performance of the EMG decoder in predicting hand poses (**Table 1**). A confusion matrix not only indicates the overall accuracy of hand pose prediction but also specifies areas of confusion in predicting hand pose. Each entry of the confusion matrices is the median ± SEM across subjects. Entries in each row of the confusion matrix represents the hand pose that the subjects were instructed to adopt. Entries in each column represent the percentage of time that each hand pose was predicted. For example, the confusion matrix for Round 1 shows that when the subject was prompted to open the hand, an open hand was only predicted about 50% of the time (median across subjects). For 14% of the time, a 2-digit grasp was predicted instead. For 1% of the time, a 3-digit grasp was predicted. For 2 ± 6% of the time, a closed fist was predicted. An unclassified hand pose was predicted 5 ± 3% of the time.

For an ideal discrimination of hand postures, a confusion matrix will have a value of 100% along the diagonal and a value of 0% off the diagonal. Non-zero percentages that are off of the diagonal represent areas of confusion between pairs of hand poses. Predictions of an unclassified hand pose occurred when transitioning from one classified hand pose to the next. Some unclassified poses included a closed fist with extension of the little finger, extension of the little finger and index finger, extension of the middle finger and ring finger, and flexion of only the thumb and middle finger. These poses occurred mostly when the subject was attempting an open hand, 2-digit grasp, or 3-digit grasp.

The confusion matrices for each round show an improvement in performance across rounds. However, it should be noted that for some subjects, the EMG decoder was retrained on one of the hand poses only after Round 1, thereby partially explaining the increase in performance from Round 1 to Round 2. From Round 2 to Round 5 however, there was some improvement in the ability to perform a 3-digit grasp and an index finger point that can be attributable only to practice.

Analysis of variance on the overall hand pose prediction accuracy revealed that prediction accuracy varied significantly across grasping tasks (*Time epoch*; *p*< 0.05) with no main effect of *Round* (practice) or interaction between *Round* and *Time epoch*. Although 5- and 2-digit grasping had the highest and lowest prediction accuracy, we found that no significant difference was found between these two grip types. However, we found a main significant effect of experimental session (*p* < 0.001).

#### Achieving Target Forces

During each grasp, subjects were told to reach a target force. The total predicted grip force across the fingers in contact was summated for each loop/iteration. The subject was prompted to attain the predicted total grip force to match the target force as closely as possible. The task was challenging because of the large variation in predicted grip force from one loop iteration to the next. Although the predicted grip force was computed as the average of the previous three loop iterations, there still was significant variability in force predictions across consecutive loop iterations.

Quality of grip force control was measured by the NRMSE. **Table 2** shows the NRMSE for each round of grasping tasks. The first row represents Round 1 and the bottom row represents Round 5 after subjects have undergone practice. The first column is the average NRMSE computed across all grasp types ± SEM. The other columns show the NRMSE for specific grasp types. The NRMSE shows little improvement with practice, but finer control of grip force during the 2-digit grasp. We found a marginally significant effect of number of digits when comparing force production with 2, 3, and 5 digits (*p* = 0.057), with NRMSE being lowest in the 2-digit case. This was most likely due to the fact that the 2-digit grasp only involves control of 2 of the digit flexion forces rather than all 5, leaving less room for variability in total grip force. The overall ANOVA revealed NRMSE from the 2-digit condition to be lower than 3- and 5-digit condition, whereas 3 and 5-digit conditions were not significantly different from each other. We found no interaction or effect of session on NRMSE.

### Force Distribution across Fingers

**Figure 4** shows how the force distribution across the fingers changes across time epochs. For the first time epoch, subjects were told to make a fist and periodically vary the total force every 20 iterations. For the second time epoch, subjects were told to shift the grip force to the little finger. For the third time epoch, subjects were told to shift the force to the ring finger. For the fourth time epoch, subjects were told to shift the force to the middle finger. For the fifth time epoch, subjects were told to shift the force to the index finger.


*Numbers in bold represent the percent of the time that each hand pose was 5 correctly predicted.*

TABLE 2 | Normalized root mean square error (NRMSE) of grip force control **±** SEM.


FIGURE 4 | Electromyographic decoder output for each finger flexion force (one subject). The subject was instructed to vary the force distribution across the fingers. The instruction given to the subject is shown on top for each time epoch. Time epochs are separated by vertical dashed lines. Data from each digit are shown on each row. The instructed digit for each task is denoted by a red trace. The little finger is expected to show an increased predicted force during loop iteration 100–200. During loop iteration 200–300, the ring finger is expected to show an increased predicted force. During loop iteration 300–400, the middle finger is expected to show an increased predicted force. During loop iteration 400–500, the index finger is expected to show an increased predicted force.

**Figure 5** shows how the percent of total grip force on each finger varied across time epochs (horizontal axis). In time epoch 1, subjects were told to flex all of the fingers. Naturally, more force will be on the index finger and middle finger as these fingers can produce more force than other fingers. In time epoch 2, subjects were told to shift the force to the little finger. It can be seen that a shift in the force distribution toward the little finger occurred. In time epoch 3, subjects were told to shift the force to the ring finger. A substantial increase in predicted ring finger force is observed with a drop in predicted little finger force from time epoch 2. In time epoch 4, subjects were told to shift the force to the middle finger. A drop in predicted ring finger force is shown along with an increase in middle finger force and often an increase in index finger force. In time epoch 5, subjects were told to shift the force to the index finger. An increase in predicted index finger force is observed along with a decrease in middle finger force. The upper left plot represents data taken across subjects from Round 1. The lower left plot shows force distribution across time epochs in Round 2 after subjects have benefited from some practice and after retraining was done on one of the subjects for force predictions. The lower right plot represents data taken across subjects from Round 5 after subjects have benefited from more practice without any additional retraining. In Round 5, subjects are better at shifting more force selectively to the pinky and ring finger in time epochs 2 and 3. In both Rounds 1 and 5, subjects can easily shift force to the index finger and less on the little and ring finger.

Another way of illustrating grip force distribution across the fingers is the force distribution index, which is equal to −1 when all force across the fingers is concentrated on the pinky and equal to +1 when all force is concentrated on the index finger. When force is evenly distributed across the fingers, the force distribution index is 0. Force distribution index is computed as follows:

$$F\_{\text{dist}} = F\_{\text{index}}\,\,\,\,\mathbf{1} + F\_{\text{mid}\,\text{dllc}}\,\,\,\,\mathbf{0}.\,\,\mathbf{5} - F\_{\text{ring}}\,\,\,\,\mathbf{4}\,\,\,\mathbf{5} - F\_{\text{mid}\,\text{kdy}}\,\,\,\,\mathbf{1} \tag{9}$$

When the subject was told to make a fist during the first time epoch, we expected the force distribution index to be greater than 0 because naturally the index finger and middle finger each have a higher grip force than the little finger. **Table 3** shows force distribution indices across time epochs (by row) and subjects (by column). Data from across rounds show more negative values when subjects were told to shift the force to the little finger, and greater positive values are found when subjects were told to shift the force to the index finger. Compared to when subjects were told to make a regular fist, the force distribution index was lower when subjects were told to shift the force to the ring or little finger, and higher when subjects were told to shift the force to the middle or index finger.

We found that subjects were able to shift force distribution across the fingers from one time epoch to the next (significant main effect of *Time epoch*; *p* < 0.002). However, subjects did not improve with practice, and furthermore, we found that the duration of the time epoch did not affect how well subjects could maintain a given force distribution across fingers (no main effect of *Round*, *Sample*, or significant interaction; *p* > 0.1).

#### Ability to Perform Grasping Tasks without Dropping the Object

A major factor influencing whether a grasping task can be performed successfully with an artificial hand is whether there is an unwanted opening of the hand as this can cause dropping of the object. In the trials involving grasping tasks, an opening of the hand when the hand was not supposed to be open rarely occurred (4.4 ± 1.4% of loop iterations), and there was little change in the incidence of unwanted hand openings across

rounds. Unwanted hand opening occurred most often for the index-thumb grasp, as illustrated in the confusion matrices (**Table 1**). In trials involving shifts in force distribution across the fingers, unwanted hand opening also occurred very rarely (2.7 ± 1.2% of the time).

To further assess the ability to performing grasping tasks, subjects were allowed to myoelectrically control the i-limb hand. For these trials, subjects were instructed to do a few basic grasping tasks. The first task involved simply opening and closing the hand. The second and third tasks involved performing 2- and 3-digit grasps, respectively. The fourth task involved grasping with a 5-digit grasp, transitioning to index finger pointing, transitioning back to a 5-digit grasp, then transitioning to a 2- or 3-digit grasp before opening the hand. All subjects who attempted the opening and closing of the hand succeeded in performing these movements, although some subjects did so more easily than others. All subjects were able to do 2- and 3-digit grasps, but often with difficulty. All but one subject were able to perform the index finger pointing and transition from a 5- to a 3-digit grasp without losing contact. Subject 7 did not attempt the grasping tasks because of insufficient time availability.

Finally, we also asked subjects to report the level of difficulty they experienced while performing online control of each online i-limb task. 1 = "easy," 2 = "able to do the task," 3 = "able to do the task, but with effort," 4 = "moderately hard," and 5 = "very hard or could not do." **Table 4** shows the rating for each subject and task. More than half of the ratings (19/35) were below 3, i.e., subjects found the task to be easy or at least doable. Some of the ratings (9/35) were equal or greater than 3, indicating that subjects found the task moderately or very hard. These tasks were usually the finger pointing or the transition from 5- to 3-digit grasp. Although all subjects were able to transition from a 5- to a 3-digit grasp, some had trouble transitioning from a 5- to a 2-digit grasp without accidentally opening the hand. A 2-digit grasp involves flexing of 3 fingers, which could have caused the hand pose to be confused with the open hand pose, which involves flexing of all digits.

#### DISCUSSION

This work has demonstrated a proof of principle for a system that decodes EMG signals from the upper limb of able-bodied subjects for online prediction of individual digit forces. Subjects


#### TABLE 4 | Subjective rating of difficulty level of i-limb grasping task.


were able to perform a variety of grasping tasks using an *ad hoc* computer program. The system is to eventually be used on transradial amputees for enabling them to perform grasping tasks and hand gesturing with myoelectric control of a prosthetic hand such as the i-limb hand.

Clearly, predicting individual digit forces alone is insufficient for online prediction of hand motion because the EMG-to-force mapping on a given digit changes depending on which of the other digits is flexed. For example, the EMG-to-force mapping for the middle finger is going to be different for a thumb-middle finger precision grasp versus a closed fist. By incorporating an SVM classifier that distinguishes between hand postures, myoelectric control of hand motion, and individual digit forces for everyday activities becomes more feasible.

As with any myoelectric control system, this system is vulnerable to changes in the EMG signal over time. This issue is particularly relevant for myoelectric control systems that predict multiple hand motions. As the proposed system has not been tested for its functionality over time periods of hours or days, future work will examine the sensitivity of our myoelectric control system to prolonged usage. Nevertheless, the impact of changes in EMG signals on myoelectric controllers has been extensively studied and the insight provided by this previous work could potentially be integrated with the proposed approach. Below we discuss previous work on myoelectric prosthesis controller algorithms and the contributions of the present work.

#### Previous Work on Myoelectric Decoders

Previous work has also explored numerous techniques for using EMG signals to predict desired total grasp force and hand postures. With regard to grasp force prediction, Gijsberts and colleagues (16) demonstrated the use of a supervised non-linear incremental learning method (Incremental Ridge Regression) that makes occasional updates with small batches of training data each time. This approach led to a reduction in normalized mean square error and an increase in the correlation between desired and predicted grip forces.

With regard to hand kinematics, Anam and Al-Jumaily (17) used an online sequential learning method that used small chunks of finger movement data collected online as additional training data. These additional chunks of data were used to update the weights of the trained model without retraining the entire model. The online retraining allowed for a model prediction accuracy to be maintained at 85% day-to-day, whereas the system without online retraining could not. This system could distinguish among 10 movement classes, i.e., 5 individual finger movements and 5 combined finger movements where more than one finger flexed simultaneously. Another group (18) used supervised adaptationbased linear discriminant analysis methods to adapt to drifts in the EMG signal. Offline analysis showed improvements in classification accuracy from 75% without adaptation to 92% with adaptation. For online control, the accuracy increased by 25%. The system distinguished between a mix of hand and wrist movements, which included wrist pronation/supination, wrist flexion/extension, hand open, 2-digit pinch grip, key grip, and no movement. Al-Timemy and colleagues (19) demonstrated classification of up at 15 finger movements with 98% accuracy using only 6 EMG channels in intact subjects and 90% accuracy using 11 EMG electrodes in amputees. Interestingly, the system showed better performance using Orthogonal Fuzzy Neighborhood Discriminant Analysis (OFNDA) for feature selection than principal component analysis (PCA). Unlike PCA, OFNDA takes into account maximum separation of feature values between classes. PCA only takes into consideration maximum variability among feature values in feature space.

Continuous morphing between hand postures has been another attempt at making prosthetic hand movements more natural. Segil and Weir (20) created a mapping between the EMG principal component and joint angle domains for allowing realtime control by able-bodied subjects of 15 joints of a virtual hand displayed on a computer screen. The authors reported accurate control of 13 out of 15 joints in the best-case scenario. In addition, EMG control and joystick control were found to be comparable in controlling joint motions. By using SVMs, Khushaba et al. (21) distinguished 10 classes of individual and combined finger movements, including a hand close movement, with 90.3% accuracy in real-time control experiments on eight able-bodied subjects.

Finally, ultrasound imaging has also been studied as a means to overcome the limitations of surface EMG. Akhlaghi and colleagues (22) implemented a real-time control system that classifies hand motions by sensing mechanical deformations in forearm muscle compartments. Although the results were encouraging, the system is not feasible for eventual integration with hand prosthesis controllers.

#### Contributions to Previous Work

It should be noted that the above-cited work on myoelectric algorithms has focused mostly on EMG-based prediction of total grasping force. To the best of our knowledge, the use of myoelectric signals to predict individual finger forces has not been investigated. This is a major gap, as successful prediction of flexion force of *individual* fingers from EMG signals could be useful for performing in-hand object manipulation using a prosthetic hand. Another gap in the literature on myoelectric controller algorithms is that there has been no demonstration of smooth transitions between grasp types without losing contact with the object. Although previous work has explored myoelectric control of smooth transitions between hand postures (20), such work did not address how easily such transitions between grasp types could be made without accidental loss of contact with the object.

In the present work, we have demonstrated proof of principle of a unique system that provides online intuitive control of individual finger forces. Subjects simply press their fingers into the palm of their hands, and the force exerted by each finger into the palm is approximately correlated with the flexing force command delivered to the corresponding finger of the robot hand. The training process involves grasping a force-sensing object only with the contralateral hand while pressing with approximately identical finger flexion forces on the ipsilateral hand. In this way, the system could be trained on amputees who have no ipsilateral hand with which to grasp the force-sensing object.

Although the RFR that predicted each finger force was trained only when the subject was making a fist, finger force prediction was transferable to other grasp types such as the 2-digit pinch grip. As long as the subject was given feedback of the digit forces, the subject was able to modulate his/her EMG signals to adjust the total grasping force accordingly. For grasping tasks performed using a computer program to display predicted finger forces, confusion matrices show that most of the improvement in performance across rounds was from Round 1 to Round 2, with smaller improvements shown in subsequent rounds (**Table 1**). In Round 5, our algorithm attained success rates ranging from 60% (index finger pointing) to 94 and 99% for closed fist and resting state, respectively. Some of the improvement from Round 1 to Round 2 was due to retraining on 1–2 hand poses on some of the subjects. Some improvement could thus be attributed to practice and familiarization to the task. Overall, our results show that subjects have voluntary control over the hand pose that the system outputs, although there is room for improvement in the system's prediction accuracy.

**Figures 4** and **5** demonstrate the feasibility of EMG control of not only total grip force but also grip force distribution across the fingers during a grasping task. By selectively pressing the fingers into the palm of the hand, subjects can control which fingers have the largest normal force at the system output. This feature of the system allows for more dexterous control of individual finger motions that is intuitive for the user. However, due to high noise in predictions of total grip force, NRMSE is higher than it could have been. Finally, we demonstrate that subjects can use myoelectric control to transition between grasp types without losing contact with the object, as shown in **Figure 3**.

#### Methodological Considerations

One way of improving the system is incorporation of a Kalman filter for hand posture predictions. A Kalman filter can take into account the degree of uncertainty in each hand posture prediction, and it gives higher weight to predictions with higher levels of certainty. A one-against-one SVM selects the hand posture with the most votes. Therefore, the first possible measure of uncertainty in the SVM is the number of votes in favor of the chosen hand posture relative to the number of votes in favor of the other hand postures. If there are two hand postures that both have the largest number of votes, then there is uncertainty in the hand pose prediction. The second possible measure of uncertainty is in the distance of a data point from the dividing hyperplane in feature space. For distinguishing each pair of hand poses in an SVM, there is a dividing hyperplane in feature space that assigns a specific hand posture to data points on one side of the hyperplane and another hand posture to data points on the other side of the hyperplane. The shorter the distance between a data point and the dividing hyperplane, the larger the degree of uncertainty, and therefore the lower the weight that would be given to that prediction. A third measure of uncertainty would be whether the prediction of hand posture is different from the previous few predictions. When the system has a high enough processing speed to make 20 predictions/second, each prediction can be weighed differently depending on its level of uncertainty.

Even though the trials performed using a computer program to display predicted finger forces were characterized by an unwanted open hand in a small percentage of the trial duration, when using the i-limb it only takes one unwanted hand opening of the hand to classify the grasping task as unsuccessful. As described above, there are several methods that can be implemented for preventing unwanted opening of a prosthetic hand during a real grasping task. One such method is a Kalman filter, which weighs different hand pose predictions depending on their level of uncertainty.

With regard to our delay in hand posture estimation, Farrell and Weir (23) estimated 100–125 ms as optimal delays for fast and slower prehensors, respectively. The optimality of these delays is based on compromising between allowing for sufficient time for EMG decoding and maximizing the responsiveness of the prosthesis. Although our 300-ms delay is more than twice the optimal delay identified by Farrell and Weir (23), it could be significantly improved by at least 10-fold by using a different software platform (i.e., C++ instead of Matlab) to enable a larger number of hand pose predictions to be made per second. Using Kalman filters (see above) would allow for the weighing of different hand pose predictions based on the level of certainty in each prediction.

### Potential Applications for Individuals with Bilateral Upper Limb Loss

Some individuals may have both hands amputated, in which case they have no hand with which to grasp a force-sensing device. In such cases, the system can make assumptions about the finger flexion forces at certain time points. For example, when the amputee is asked to concentrate the force on a specific finger, the system can assume an arbitrarily higher flexion force for that finger and a minimal force on the other fingers during that time frame. Amputees also can be prompted to vary the total grip force of a grasp where in some time frames the amputee can be prompted to exert a minimal total grasp force, and in other time frames the amputee can be prompted to exert a high grasping force. For each time frame, the system can make an assumption about the total grasp force and assume how the force would be distributed across the fingers during a grasp such as a regular closed fist depending on previous data from able-bodied individuals. For each case, the opposing thumb force can be assumed to be approximately equal to the summation of the four finger forces. An advantage of this approach is that it requires the same effort across subjects for exerting specific grasping forces because the assumption made of the grasping force for each time frame is independent of the magnitude of EMG signal (because of normalization) and of the overall muscular strength of the subject.

## CONCLUSION

We have demonstrated proof of principle in the use of five EMG electrodes for predicting hand pose and individual finger forces using a one-against-one SVM and RFR, respectively. The present system has potential for myoelectric control of dexterous hand

### REFERENCES


prostheses. Future work should explore additional methods of feature selection, signal filtering, machine learning classification, Kalman filtering, and training. New myoelectric control systems should be adjustable with small amounts of new training data without the need to retrain the entire system so that drifts in the EMG signal over time do not decrease classification accuracy.

### ETHICS STATEMENT

Subjects gave informed written consent to participate in the experiments. The experiments were approved by the Institutional. Review Board at Arizona State University (Protocol: #1201007252) and were in accordance with the Declaration of Helsinki. Prior the experiment, participants were given time to read a consent form that described the experimental procedure. After reading the form, participants were asked if they had any questions. After any questions were answered, participants were asked to sign and date the consent form. Next, the experimenter signed and dated the consent form. No participants from vulnerable populations were used in this study.

### AUTHOR CONTRIBUTIONS

AG designed the prosthetic control system, designed most of the experimental protocol, carried out the experimental protocol, collected data, analyzed data, and made data plots. PA provided expertise in the area of myoelectric control of robotic devices, thereby playing a role in design of the prosthetic control system and experimental protocol design. MS provided expertise in motor control and neuroscience. He contributed to the experimental design and data analysis methods. All authors contributed to manuscript preparation.

### ACKNOWLEDGMENTS

The authors thank Dr. Justin Fine for assistance with statistical analysis.

### FUNDING

Research reported in this publication was supported by the Eunice Kennedy Shriver National Institute of Child Health and Human Development of the National Institutes of Health under Award Number R21HD081938 and the Grainger Foundation.


**Discalimer:** The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

The reviewer AA and handling editor declared their shared affiliation and the handling editor states that the process nevertheless met the standards of a fair and objective review.

*Copyright © 2017 Gailey, Artemiadis and Santello. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Prediction of STN-DBS Electrode Implantation Track in Parkinson's Disease by Using Local Field Potentials

Ilknur Telkes <sup>1</sup> , Joohi Jimenez-Shahed<sup>2</sup> , Ashwin Viswanathan<sup>3</sup> , Aviva Abosch<sup>4</sup> and Nuri F. Ince<sup>1</sup> \*

*<sup>1</sup> Clinical Neural Engineering Lab., Biomedical Engineering Department, University of Houston, Houston, TX, USA, <sup>2</sup> Department of Neurology, Baylor College of Medicine, Houston, TX, USA, <sup>3</sup> Department of Neurosurgery, Baylor College of Medicine, Houston, TX, USA, <sup>4</sup> Department of Neurosurgery, University of Colorado, Aurora, CO, USA*

Optimal electrophysiological placement of the DBS electrode may lead to better long term clinical outcomes. Inter-subject anatomical variability and limitations in stereotaxic neuroimaging increase the complexity of physiological mapping performed in the operating room. Microelectrode single unit neuronal recording remains the most common intraoperative mapping technique, but requires significant expertise and is fraught by potential technical difficulties including robust measurement of the signal. In contrast, local field potentials (LFPs), owing to their oscillatory and robust nature and being more correlated with the disease symptoms, can overcome these technical issues. Therefore, we hypothesized that multiple spectral features extracted from microelectrode-recorded LFPs could be used to automate the identification of the optimal track and the STN localization. In this regard, we recorded LFPs from microelectrodes in three tracks from 22 patients during DBS electrode implantation surgery at different depths and aimed to predict the track selected by the neurosurgeon based on the interpretation of single unit recordings. A least mean square (LMS) algorithm was used to de-correlate LFPs in each track, in order to remove common activity between channels and increase their spatial specificity. Subband power in the beta band (11–32 Hz) and high frequency range (200–450 Hz) were extracted from the de-correlated LFP data and used as features. A linear discriminant analysis (LDA) method was applied both for the localization of the dorsal border of STN and the prediction of the optimal track. By fusing the information from these low and high frequency bands, the dorsal border of STN was localized with a root mean square (RMS) error of 1.22 mm. The prediction accuracy for the optimal track was 80%. Individual beta band (11–32 Hz) and the range of high frequency oscillations (200–450 Hz) provided prediction accuracies of 72 and 68% respectively. The best prediction result obtained with monopolar LFP data was 68%. These results establish the initial evidence that LFPs can be strategically fused with computational intelligence in the operating room for STN localization and the selection of the track for chronic DBS electrode implantation.

Keywords: local field potentials, subthalamic nucleus, microelectrode recordings, least mean square algorithm, LDA classification

#### Edited by:

*Paolo Bonifazi, Tel Aviv University, Israel*

#### Reviewed by:

*Alessandro Stefani, University of Rome, Italy J. Luis Lujan, Mayo Clinic, USA*

> \*Correspondence: *Nuri F. Ince nfince@uh.edu*

#### Specialty section:

*This article was submitted to Neuroprosthetics, a section of the journal Frontiers in Neuroscience*

Received: *29 December 2015* Accepted: *21 April 2016* Published: *09 May 2016*

#### Citation:

*Telkes I, Jimenez-Shahed J, Viswanathan A, Abosch A and Ince NF (2016) Prediction of STN-DBS Electrode Implantation Track in Parkinson's Disease by Using Local Field Potentials. Front. Neurosci. 10:198. doi: 10.3389/fnins.2016.00198*

## INTRODUCTION

Deep brain stimulation (DBS) of the subthalamic nucleus (STN) is an effective therapy for the treatment of the motor symptoms of Parkinson's disease (PD) (Herzog et al., 2004; Hariz, 2012). DBS surgery involves localization of the motor territory of the STN, for permanent implantation of a DBS electrode at this site. Although the exact mechanism of DBS remains to be elucidated, STN stimulation is well-tolerated and improves all of the cardinal symptoms of PD (Levy et al., 2002). However, STN stimulation can result in side effects arising from the spread of stimulation to structures surrounding the STN (Richardson et al., 2009). Moreover, sub-optimal positioning of DBS electrodes accounts for up to 40% of cases of inadequate efficacy of stimulation postoperatively (Okun et al., 2005). Thus, developing quantitative electrophysiological methods to define the optimal site of stimulation may help optimize DBS outcomes.

The task of the neurosurgeon is to place the DBS electrode within the motor territory of the STN, and well within the STN borders such that current does not spread to the surrounding structures, thereby resulting in stimulation-limiting side effects (Richardson et al., 2009). Although the surgical procedure varies somewhat between medical centers, targeting of the STN during DBS surgery generally includes preoperative stereotactic imaging (MRI), used in conjunction with stereotactic atlases. This step is followed by intraoperative electrophysiological techniques consisting of the conversion of neural activity, in the form of single-unit neuronal activity (SUA) recorded at the microelectrode tip, into audio and visual signals. This procedure is experience-based and depends critically on the neurosurgeon's and neurophysiologist's ability to recognize entry into the STN, based on a variety of cues.

In order to obtain a three-dimensional map of the STN and surrounding structures, multiple microelectrode recording tracks (typically up to five) (Benabid et al., 2009) are carried out, either sequentially or simultaneously. Determination of the optimal track for DBS implantation is a key component to successful therapeutic outcome. Optimal track selection is primarily based on microelectrode recording of single unit activity (MER/SUA), which is used to identify cells with firing characteristics consistent with STN neurons and response characteristics confirming the motor sub-territory of the STN (Falkenberg et al., 2006). Despite the common usage of MER/SUA during stereotactic surgery for PD, limitations of this technique include difficulties interpreting complex signal patterns to localize the anatomical borders of the STN, highly overlapping spiking characteristics of single neurons around the target structure, recording SUA from a very small region, sensitivity of SUA to noise, susceptibility of SUA to small amounts of blood or edema within the microelectrode track, and the binary nature of SUA (unlike local field potentials; LFP), all of which may affect the accuracy of STN localization in PD (Chen et al., 2006; Gross et al., 2006; Novak et al., 2011). The caliber of single-unit recordings can be easily diminished due to drift of the recorded unit away from the electrode tip, as a consequence of transmitted pulsations of the brain and other environmental conditions (Sanghera and Grossman, 2004).

Interpretation of SUA recordings with computational intelligence was proposed as a new approach to help clinical decision making in the operating room (Wong et al., 2009). However, such approaches are still susceptible to the challenges of isolating single neurons in the operating room. LFPs represent the aggregate activity of neuronal populations, and are particularly sensitive to synchronous and oscillatory firing patterns (Priori et al., 2004; Gross et al., 2006). Recent studies indicate that LFPs in PD correlate with both motor and nonmotor symptoms of the disease, and their signals are more robust than SUA (Priori et al., 2013; Thompson et al., 2014). Importantly, LFPs are an objective and quantitative metric while MER/SUA is more qualitative and subject to inter-practitioner variability. Although, the functional role of LFPs during DBS surgery is not fully established, we propose that they can be used to contribute to target localization in PD. In the present study, for the purpose of assisting with clinical decision making, we aimed to develop an automated approach by processing LFPs from multiple tracks to localize the dorsal border of STN and predict the macroelectrode implantation track identified by the neurosurgeon based on SUA interpretation. In the next sections first we describe our data collection methods and then detail our signal processing and classification techniques. We study the role of LFP sub-bands in prediction of the location of the dorsal border of STN. Moreover, we explore different decision criteria fused with LFP sub-band features toward prediction of optimal track selected by the neurosurgeons. We show experimental results obtained from 22 patients and discuss our results and demonstrate that LFPs can be used effectively in the operating room for clinical decision making.

## METHODS

### Patients and Surgical Procedure

This is a multicenter study in which patients were recruited in either University of Minnesota Medical Center or Baylor St. Luke's Medical Center. The experimental protocol was approved by the Institutional Review Boards of the University of Minnesota and Baylor College of Medicine. All patients provided written informed consent to participate in the study. Intraoperative LFPs were recorded from 22 patients (14 men, 8 women), who were diagnosed with idiopathic PD, and exhibited typical motor symptoms which were tremor, rigidity, and bradykinesia. Disease duration ranged between 1 and 20 years, with a mean of 10.55 years (standard deviation of 4.7 years) (**Table 1**). All patients discontinued short-acting Parkinson's medications at 12 h prior to surgery, and long-acting medications at least 24 h prior to surgery. As per standard clinical protocol, target coordinates and trajectory to the STN, were identified by preoperative stereotactic MRI, which was fused to a stereotactic computed tomography (CT), on a neuro-navigational platform (StealthStation, Medtronic Corp, MN). Then, again based on standard clinical protocol, three simultaneous tracks were performed in each subject (**Figure 1A**). The superior and inferior borders of STN, along with the optimal depth for positioning the DBS electrode, were determined by the clinical team via electrophysiological mapping using MER/SUA, and the DBS


*† UPDRS* = *Unified Parkinson's Disease Rating Scale.*

*‡ Pre-Operative Medication OFF-to-ON UPDRS Scores: Total Improvement.*

electrode was implanted by the neurosurgeon based on these spatial data, followed by macro stimulation to confirm electrode location based on benefit and side effect profile—i.e., location within motor territory of STN, but not so close to border with adjacent internal capsule or medial lemniscus, that lowthreshold stimulation- induced side effects were detected followed by confirmatory intra-operative imaging modalities. Surgeries were performed in awake patients, under the benefits of local anesthesia. In 3 of 22 patients, microelectrode mapping of right and left STN occurred on different surgical dates, as the surgical procedures were staged for clinical reasons. Therefore, these recordings were counted as separate, enabling 25 individual STN microelectrode recordings for LFP-based optimal track prediction.

#### Intra-Operative Recordings and Track Selection

Following standard stereotactic techniques, and insertion of three brain cannulas and microelectrodes (Abosch et al., 2002), MER/SUA recording was carried out using a Microguide system (AlphaOmega Inc., USA) at 12 kHz. Simultaneous LFPs were recorded with an XLTEK-EMU128FS system (Natus, San Carlos, California) at 2 kHz with 16 bit A/D resolution or gHIAmp (gTec Inc., Graz, Austria), 38.4 kHz with 24 bit A/D resolution. The LFP recordings were obtained from a 1 mm wide stainless steel contact which is 3 mm (NeuroProbe, AlphaOmega Inc., USA) or 1 mm (MicroTargetingTM, FHC Inc., USA) above the SUA recording tip and referenced to the cannula (**Figure 1B**). All microelectrodes were advanced toward the estimated target using a NeuroDrive (AlphaOmega Inc., USA) with micrometer resolution. In order to synchronize the SUA and LFP recordings, the digital depth information of the NeuroDrive is transmitted from the MicroGuide system to LFP recording system using a TCP/IP connection. Initial recordings began 20 mm above the intended final location of the electrode tip ("target") as determined by direct targeting methods and proceeded until the electrode reached 3 mm below the MER-determined target, within the substantia nigra. Electrodes were lowered in 1 mm steps until 10 mm above "target," and then in 0.5 mm steps. Duration of recordings at each depth was 15–30 s.

At each depth, the subjects sequentially rested and after a certain depth (<10 mm) executed limb movements for 10–15 s period. The neurosurgeon used standard clinical techniques for localizing the STN, via real-time auditory and visual analysis of the recorded SUA. The dorsal, ventral, and posterior borders of the STN were identified by noting increased background noise and cell firing rate, and the STN neurons were examined for movement-responsive receptive fields. In particular, the superior border of STN was determined when the background activity increased and border cells were first observed among one of the tracks in MER-SUA. This position was used as the target value in STN border identification. Among three tracks, the track with the longest span of bursting cell firing and movement responsive fields was selected for the chronic DBS electrode implantation. This track was labeled as the "optimal track" and used as the target variable in LFP based track number prediction. The neurosurgeons were blinded to the LFP recordings in the operating room and the identification of STN dorsal border and the selection of optimal track for chronic DBS electrode implantation was not influenced by the LFP recordings. The STN border and the track selected by the neurosurgeon are predicted with the LFP data using the signal processing methods detailed below.

#### Signal Processing

All data were analyzed offline. Before any processing, all recorded signals were visualized with a custom in-house developed software and annotated to distinguish artifact and/or epochs of resting, active, and passive movements. Based on these annotations, resting state data were extracted into MATLAB (Mathworks, Natick, Massachusetts) and the data recorded by XLTEK system and gHIAmp system were downsampled to 1 and 1.2 kHz, respectively, for analysis.

A schematic diagram of our signal processing pipeline is given in **Figure 2**. As an initial step the raw signals were visualized and it was observed that tracks were difficult to distinguish, due to a high amount of common activity masking spatially localized activity and/or artifacts resulting from abrupt movements of the patient and other environmental factors. In order to eliminate the common activity among tracks, but still preserve the trackspecific neural activity, the LFP data on each track were decorrelated using a least mean square (LMS) algorithm with a steepest descent update. The general formula for the decorrelation method is as follows:

Initialization: w(0) = 0.5, n = 0, 1, 2, . . .

$$\mathbf{y}^{\prime}(n) = \mathbf{w}^{T}(n)\mathbf{x}^{\prime}(n) \tag{1}$$

$$e\left(n\right) = d\left(n\right) - \left.\chi\left(n\right)\right|\tag{2}$$

$$\hat{e}\left(\eta\right) = \begin{cases} \text{sign}\left(e\left(\eta\right)\right) \* \text{ 20} & \text{if } |e\left(\eta\right)| > \text{20} \\ e\left(\eta\right) & \text{otherwise} \end{cases} \tag{3}$$

$$
\omega(n+1) = \omega(n) + \mu \hat{e}(n) \ge (n) \tag{4}
$$

where y(n) is the filter output, eˆ(n) is the residual which is the de-correlated signal, d(n) is the desired signal, µ is the step size, and w(n) is time varying filter coefficient (Hayes, 1996). In the current method, each channel, d(n), was predicted by using a

linear weighted combination of other two channels, x(n). LFP activity from three tracks were recorded continuously during the entire surgery while the microelectrodes were traveling to the estimated target. Consequently, the signal characteristic varied over depths. Since in each depth the signal was recorded for 15– 30 s, temporal variability exists in the signal. Therefore, the filter coefficients, w<sup>T</sup> (n), were updated on a sample by sample basis recursively to make the system to adapt to depth and time varying signal properties. At each iteration, the error, e(n), was calculated and this residual was used as the de-correlated LFP data in future steps for feature extraction and visualization. At 20 mm above the estimated target, all three tracks showed very similar signal characteristics indicating that they were in the white matter. Therefore, the initial filter coefficients were selected as the average of two channels with equal weights with the initialization of the filter coefficients w (n) = 0.5. By using this adaptive approach, we aimed to eliminate the common activity across tracks and suppress localized artifacts caused by patient movements and environmental factors. In order to prevent the system from being affected by high amplitude artifacts and to preserve the robustness, the error was saturated by using a 20µV threshold (Equation 3). This threshold was determined experimentally and we observed that the system recovered from localized artifacts pretty fast even if the artifact amplitude was too large.

Due to differences in spatial correlation of low and high frequency bands, the monopolar data were, first, decomposed into two frequency bands which were 8–200 and 200–450 Hz by using a 2nd order Butterworth IIR filter (**Figure 2**). The LMS algorithm was individually applied to these subbands with step size of µ = 0.0002. Each track was de-correlated by using LFPs on the other two tracks. The algorithm was applied to each depth by transferring filter coefficients to the next depth. In this way, filter coefficients were not required to start from 0.5 at each new depth so that the algorithm would adapt faster and can use both temporal and spatial information of the past. Decomposed and de-correlated data were re-merged and spectral analysis was performed. In this regard, a modified Welch periodogram method with a robust statistics was used (Telkes et al., 2014). Specifically, a fast Fourier transform (FFT) was computed with a 1024 samples long Hanning window and the window was shifted with 50% overlap. Since some artifacts destroy or dominate the power spectrum estimate obtained with mean operator, the median of the spectra of all sliding windows was calculated to eliminate localized artifacts in the spectrum. The method was repeated for each depth and all spectra were combined to visualize depth-varying power spectrum of LFPs on multi tracks. Generated depth-frequency maps (DFMs) were resampled with a 0.25 mm depth resolution and linearly interpolated to obtain equidistance depth values. The maps were smoothed using a Gaussian kernel filter to suppress noise and to reveal beta and high frequency band oscillations (HFOs). Then, DFMs were normalized with the average baseline of three tracks and transformed into log scale using the Equations (5) and (6). The tracks were not normalized by their own baseline but by the mean of all three tracks in order to compare the signal power between them. The baseline used for normalization was selected as the highest depths which assumed to be in the white matter. Therefore, the baseline was determined as top 5 depths (20– 15 mm above the estimated target) in 22 recordings. However, in rest of the three recordings, since the analysis started from lower than 20 mm (such that 18 mm) due to artifacts, the baseline segment was kept shorter and selected as top 3 depths. The purpose of using higher depths was to avoid from including any thalamic activity in normalization segment. The baseline normalization formula is noted below:

$$
\overline{b}\_{\text{avg}} = \frac{\left(\overline{b}\_1 + \overline{b}\_2 + \overline{b}\_3\right)}{3} \tag{5}
$$

$$
\overline{n}\_{\rm dfm} = \ 20 \times \log\_{10} \left( \frac{\overline{r}\_{\rm dfm}}{\overline{b}\_{\rm avg} + \Phi(f)} \right) \tag{6}
$$

where b1, b2, b<sup>3</sup> are the baseline spectrum of each 3 track, bavg is the average baseline power, rdfm indicates the DFM, 8(f)

is a small regularization parameter which is applied for each frequency f and ndfm is the normalized DFM.

In order to observe the depth-varying frequency content of LFPs, DFMs of the patients were visualized. We noted that when the electrodes reached the STN border identified by the neurosurgeon, generally there was also an excessive activity in the beta and HFO range. In order to identify the most beneficial track along with the dorsal border of the STN, the sub-band power was extracted from all tracks and normalized by using a subject-specific average baseline. Based on the distribution of neural activity on the tracks, the sub-band frequencies were designated 11–32 Hz for beta band and 200–450 Hz for HFOs. The distribution of power in the STN among all tracks and the distribution of power only on the selected track inside and outside of STN (above the dorsal border of STN) were investigated by box and whisker plots. Student's t-test with twosample was used to check if the distributions were significantly different or not.

#### Classification

After sub-band power features were normalized between zero and one with a Max-Min normalization method for inter-subject comparison, a linear discriminant analysis (LDA) was used for classification. The principle of LDA is to maximize the separation of classes while keeping the within class densities small by using linear combination of features, −→<sup>v</sup> · −→<sup>z</sup> (Alpaydin, 2010). The linear discriminant function:

$$\begin{aligned} g\_i \left( z \middle| \nu\_i, \ \nu\_{i0} \right) &= \nu\_i^T z + \nu\_{i0} \\ &= \sum\_{j=1}^d \nu\_{ij} z\_j + \nu\_{i0} \end{aligned} \tag{7}$$

where g<sup>i</sup> (z) is the discriminant function for the input features z<sup>j</sup> with sum of the weights v<sup>j</sup> and threshold values vi0.

#### Localization of the Dorsal Border of STN

In the present study, the dorsal border of STN identified by clinical team is predicted from the depth varying LFP data by using the decision distance of a linear classifier as shown in **Figure 3A**. First an LDA classifier was trained by contrasting the LFP sub-band features coming from inside and outside of STN (above the dorsal border of STN). This classifier was evaluated at each depth and the returned decision distance was used as a measure of confidence. The depth with the highest confidence for IN-STN decisions was marked. Then we traced the decision distances above this depth and found the location where the LDA classifier voted for OUT-STN. This point where the classifier made IN vs. OUT decision transition was finally chosen as the predicted dorsal border of STN. The difference between prediction and the STN border identified by MER-SUA was calculated in each patient and the root mean square (RMS) of the prediction errors was used to quantify the performance of the classifier. Further, statistical analysis by using Student's t-test and F-test was conducted in order to compare the mean and the variance of predictions obtained by different subband features, respectively.

#### Prediction of the Optimal Track

The optimal track selection among three tracks is done by the neurosurgeons through interpreting the excessive single cell firings within the STN. Consequently, for the prediction of the optimal track using LFP data, an LDA classifier was trained by contrasting the LFP subband features of selected track vs. unselected tracks below the dorsal border of STN. This classifier was evaluated at all depths as in the STN border prediction and the returned decision distance was used as a measure of confidence. The distance returned by the linear classifier

optimal track. (A) Decision strategy in the prediction of dorsal border of STN. The classifier trained by contrasting the LFP sub-band features coming from inside and outside of STN gives a decision at each depth from top to bottom. These decision distances were used as a measure of confidence. Depth with the highest confidence for IN-STN decisions was marked. Then decisions were traced back where the LDA classifier voted for OUT-STN. This point where the classifier made IN vs. OUT decision transition was chosen as the predicted dorsal border of STN. (B) Decision strategy in the optimal track prediction. Classifier trained by contrasting the LFP subband features of selected track vs. un-selected tracks below the dorsal border of STN returned decision distances at all depths. The decision distances were used in three different scenarios: (1) Below the SUA-based STN border, (2) below the one standard deviation from the average of STN border, and (3) below the LFP-based STN border obtained from prediction of dorsal border of STN. The decision distance with the longest span was used as a decision criterion to predict the optimal track.

was used in three different scenarios for final decision making (See **Figure 3B**). In the first scenario, the optimal track was predicted below the STN border provided by the neurosurgeon for that specific test subject based on the SUA interpretation. This represents the setup in which we fuse SUA- and LFPbased information. In the second scenario, no SUA information about the STN border of the test subject was used, and the decisions were given below the one standard deviation from the average of STN border estimated from all training subjects. In the third scenario, the optimal track decisions were made below the STN border which was derived solely from the LFP data. Specifically, here we explored whether or not the LFP could predict the optimal track without any SUA-based interpretation. We studied the classification performance below and above the STN border in these three different scenarios. A schematic diagram related to this process is given in **Figure 3B**. Depthvarying LFP subband features of each track were classified using the trained LDA and a label and related decision distance were generated by the classifier for each depth. We classified one of the tracks as the optimal one based on the longest span of decision distances voting for optimal track within the STN. Note that the longest span is a common approach used intraoperatively by neurosurgeons for MER-SUA-based optimal track selection. Also note that, if the track selected by neurosurgeon in the operating room did not match with the decision of the algorithm, the decision was counted as a misclassification.

The prediction of optimal trajectory was investigated using individual sub-band powers, beta and HFO, and their combination. To explore the benefit of the LMS algorithm over monopolar signals (raw signals), the same classification procedure was carried out with the raw data.

Finally, in order to assess the efficiency and reproducibility of the classification, a leave-one-subject-out method was used. In each step, one subject was used for testing, whereas the other subjects were used for training the LDA classifiers for STN dorsal border and optimal track prediction. The procedure was repeated until the whole sample was classified. In addition, this procedure was performed separately for individual beta and HFO subbands of LFP and their combinations to examine their efficacy in classification performance.

In order to explore a relationship between classification results and post-operative simulation parameters used for the initial programming 6 months after the surgery were compared in correctly classified and misclassified groups. The distribution of stimulation amplitude was investigated by box and whisker plots. Student's t-test with two-sample was used to check if the distribution of simulation amplitude, frequency, and pulse width were significantly different or not.

## RESULTS

### De-Correlation of LFP Data from Multiple Tracks

We analyzed LFP data derived from 75 MER tracks in patients with PD who were undergoing STN DBS electrode placement. Typical raw SUA and LFP data coming from various depths were shown in **Figure 4**. The red dashed line indicates the dorsal border of STN. The correlation between the two modalities is clearly seen. When the electrode enters the STN, both single cell firing and oscillatory activity increase.

In **Figures 5A,B**, the mean correlation matrices of raw LFP data filtered in beta band (11–32 Hz) and HFO band (200– 450 Hz) were shown. The correlation matrix in the beta band (**Figure 5A**) explicitly shows that the spatial correlation between tracks is high whereas the correlation between tracks in HFO range is small (**Figure 5B**). The small amount of correlation at HFO band in raw data also shows that oscillations at higher frequencies are more localized than the oscillations at lower frequencies. For these reasons, the LFP data were de-correlated with LMS separately in these frequency bands. It was found that the correlation between tracks is reduced after the LMS-based preprocessing step (**Figure 5C**) which helped more to distinguish the tracks.

**Figure 6** demonstrates the effect of LMS algorithm by comparing it to the raw LFP signals. As it can be seen in **Figures 6A–C**, the raw LFP data have a high amount of common activity across all tracks at various depths which masks the spatially and temporally distinguishing patterns during targeting. In **Figure 6B**, DFMs indicate that the high-energy low band activity among tracks masks other oscillations. The common activity across three tracks and the high energy low band oscillations can be also seen in the power spectrum shown in **Figure 6C** which was generated from the LFP data below the dorsal border of STN. On the other hand, target specific oscillations are clearly seen on de-correlated LFP data (**Figure 6D**). In particular, the energy in the first track is much higher than the other two tracks and it is easier to observe the track differences and the estimated STN border depth for the target localization. The DFMs of these tracks shown in **Figure 6E** demonstrate that the first track contains LFPs with higher energy in low and high frequency bands below the dorsal border of the STN which is marked with a white dashed line. Furthermore, the power spectrum shown in **Figure 6F** demonstrates that not all three tracks show excessive beta activity. There is an increase in the gamma band (35–55 Hz) and great enhancement in HFO range (200–400 Hz) in the first track compared to other tracks. The LMS algorithm not only reveals the pathological beta oscillations but also the HFOs having lower energy.

#### Spatio-Spectral Patterns of Multitrack LFP

In order to provide a sense of the depth-varying frequency content of multitrack LFPs, we demonstrated representative normalized DFMs of de-correlated LFP data of all three tracks from four patients in **Figure 7**. In each map, the dorsal STN border is marked with a white dashed line. The excessive beta oscillations can be clearly seen in the first subject dominantly in the center track and localized to certain depths (**Figure 7A**). The power of beta oscillations in the posterior and medial track is weak, yet it can be still observed. Furthermore, there is a strong and track-specific HFO around 350 Hz which is well aligned with the low band activity. On the other hand, in the subject presented in **Figure 7B**, beta oscillations are observed in all tracks along with HFOs. Although the excessive LFP activity occurs below the STN dorsal border as for the patient presented in the **Figure 7A**, the excessive depth varying spectral patterns are pretty track and region specific. The HFO in the center track sits at 350 Hz while it is located at 250 Hz in the posterior track. The lateral track shows wider but weaker oscillations. DFMs in **Figure 7C** demonstrate similar LFP characteristics to the first subject (**Figure 7A**) with dominant beta oscillations and HFOs in the center track. Similarly, these oscillations are well aligned below the dorsal border and highly stronger than the beta oscillations in the other two tracks. Distinctly, strong oscillations at higher depths are observed above the dorsal border in the posterior track which might be related to thalamic activity. Note that we observed high frequency activity localized at higher depths above the dorsal border of STN in at least one of the un-selected tracks in 56% of recordings. The number of the un-selected tracks with the observed oscillations were: 8 posterior, 5 center, and 1 medial. Similar to the HFO activity seen in **Figure 7C**, these oscillations were noted from 11.5 ±

specific oscillations are visible on de-correlated LFP data. The red dashed line shows the dorsal border of STN determined by MER-SUA. (E) DFM of de-correlated LFPs in each microelectrode track. The high energy oscillations in distinct low and high frequency bands inside the STN (below the white dashed line) can be clearly seen in the first track. (F) Power spectrum of de-correlated LFPs. Tracks do not show excessive and correlated beta activity anymore. There is an increase in the gamma band (35–55 Hz) and great enhancement in HFO range (200–400 Hz) in the first track compared to other tracks.

2.6 to 5.7 ± 2.4 mm (average values) above the estimated final location of the electrode tip. The tracks having higher-depth HFOs do not include strong beta activity. These oscillations have a longer spatial span with lower power. It is likely that these oscillations rise from thalamic structures (Hutchison et al., 1998; Falkenberg et al., 2006), and given their spatial distribution in relation to the beta band activity, they might be used as markers for STN localization. The fourth representative subject shown in **Figure 7D** introduces a different LFP characteristic compared to others. None of the tracks are associated with strong, long span of beta oscillations. Specified border is not aligned with weak beta oscillations but the short lasting excessive one in the lateral track. All tracks demonstrate spatially different weak-to-minor HFOs. The overlap in the LFP activity between tracks, the weak activity across tracks and thalamic oscillations are some of the factors contributing to the challenges to the prediction of dorsal border of STN and selection of optimal track.

#### Intra-Track and Inter-Track Differences of LFP Spectra

For the neurosurgeon selected track, the distribution of beta and HFO subband powers above and below the dorsal border of STN are given in **Figure 8A**. The analysis shows that there is a significant difference between the power inside and outside the STN region (above the STN dorsal border) (t = 44.72, p < 0.001; t = 34.89, p < 0.001) in the selected track. As seen from the box-plot in **Figure 8A**, the sub-band power is much higher

inside the STN. When the subband power was compared between the optimal and other tracks (**Figure 8B**), the distributions were found significant as well (t = 16.47, p < 0.001; t = 15.17, p < 0.001). The significance is consistent at the beta band and HFO band in both distributions. The variance of HFO power in the un-selected tracks is higher than the variance in the selected track. Based on our previously mentioned findings, we postulate that thalamic activity in un-selected tracks might contribute to increased variance of HFO power when the distribution includes entire track.

#### Localization of the Dorsal Border of STN

The progression of prediction of STN dorsal border for representative subjects and the average results estimated from the entire patient population is shown in **Figure 9**. In **Figures 9A,B**, decision distances returned by the classifier voting either for IN-STN or OUT-STN are shown for two representative subjects. The decision distance curves were obtained from the fused beta and HFO features. Note that the predicted STN border is shown with an arrow corresponds to the position where we find the maximum confidence point associated with IN-STN and trace back to the depth crossing zero. The dorsal border of STN provided by the neurosurgeon based on SUA interpretation is shown with a dashed vertical line. **Figure 9A** shows a late prediction of the dorsal border (e = −0.75 mm) while **Figure 9B** indicates an early border prediction (e = +1 mm). **Figure 9C** demonstrates the average border decisions with the standard deviation coming from all test subjects by using individual subband powers, beta and HFO, and fused features. The overall localization error of the dorsal border of STN was quantified by calculating RMS of the error between target values and LFP predictions across all subjects. The red and blue lines show the decisions obtained with beta and HFO band features indicating an RMS error of 1.98 mm and 1.18 mm, respectively. The mean value of prediction error for beta band features was 0.83 ± 1.84 mm while the mean of error for HFO band features was −0.23 ± 1.18 mm. The decisions obtained through the fused beta and HFO band features had an RMS error of 1.22 mm with mean of 0.24 ± 1.22 mm. In **Figure 9D**, the distribution of prediction errors are shown for each studied subband and their fusion. Student's two sample t-test analysis indicated that the difference between mean values of beta-based prediction error and HFO-based prediction error was significantly different (t = 2.22, p = 0.0322) while no statistically significant difference

FIGURE 9 | Localization of dorsal border of STN. Panels (A,B) show the decision distances returned by the classifier voting either for IN-STN or OUT-STN for two representative subjects. Predicted STN border is shown with an arrow corresponds to the position where we find the maximum confidence point voted for IN-STN and trace back to the depth crossing zero. The black dashed lines indicate the dorsal border of STN obtained by MER/SUA. The predicted border is 0.75 mm lower than the SUA-based border in the first subject (A) and 1 mm higher in the second (B). Panel (C) demonstrates the average border decisions with the standard deviation coming from all test subjects by using individual sub-band powers, beta (red) and HFO (blue), and fused features (green). (D) Comparison of prediction errors. The error (RMS) of the predictions obtained with beta and HFO band features are 1.98 and 1.18 mm. The RMS of the error in predictions obtained from fused features is 1.22 mm. The variance of RMS values were compared by *F-test*. The difference between beta-based and HFO-based prediction error is marginally significant (*p* = 0.054) with α = 0.05. <sup>+</sup>indicates the outlier error value.

was found neither between beta-based prediction error and the error of fused features (t = 1.23, p = 0.2244) nor HFObased prediction error and the error of fused features (t = −1.25, p = 0.2185). When the variances of these distributions were compared by using an F-test, the analysis showed that the difference between beta-based and HFO-based border prediction was only marginally significant [F(1,2) = 2.42, p = 0.054] while there was no statistically significant difference between the variance of individual sub-band powers (beta and HFO) and fused power [F(1,3) = 2.26, p = 0.075; F(2,3) = 0.94, p = 0.88, respectively].

#### Prediction of the Optimal Track

We studied the optimal track classification in three different scenarios using the LFP data: (i) below the SUA-based STN border, (ii) below one standard deviation from the average STN border obtained from training data, and (iii) the LFP-based STN border. We trained the LDA classifier using individual subband powers and their combination. Our results toward the prediction of optimal track from LFP data is given in **Table 2**. We note that the best results were obtained from the combined subband power features and consistently in all these scenarios, the optimal track prediction accuracy was 80% (shown in bold type) indicating that the classifier can predict the track targeted to the STN in 20/25 recordings. These results show that prediction of optimal track can be performed independently from single unit recordings. When the beta and HFO subband features were used individually, the classification accuracy dropped to 72 and 68% respectively. When the procedure was repeated with the raw data, the prediction rate was poor. In particular, the classification accuracy was 64% for beta band power and 68% for the HFO and fused features which supports the observation that HFOs obtained in monopolar configuration are already highly de-correlated among different tracks.

Despite the spatially localized thalamic oscillations, the classification results obtained above the STN border were quite poor. The prediction accuracy was found to be 40% when the classification was computed above SUA-based or LFP-based STN border. Decision accuracy with average STN border was even lower at 36% by using fused sub-band power. The results indicate that the LDA classifier trained with the LFP features above the STN does cannot predict the optimal track with a reasonable accuracy and was close to chance level.

The progression of classification over depths for three representative subjects are shown in **Figures 10A–C**. In each plot, the STN border location provided by the neurosurgeon based on the SUA interpretation is also represented with a vertical dashed line. The decision distances in both selected and un-selected tracks returned by the classifier are close to each other down to the dorsal border of STN. Since the spectral characteristics of LFPs change inside the STN compared to higher depths, we observe a sudden change between the decision distances as well. If only one of the tracks deviates from the others and reaching the highest confidence level, it is easily determined as the optimal track by the classifier. If more than one track are voted for the optimal track (below the zero line in **Figure 10**) with high confidence levels, algorithm gives the optimal track decision by computing the longest span of the selected track votes. The progression of the classification for a misclassified subject is given in **Figure 10C**. The average decisions for the optimal track of all subjects with the associated standard deviation are given in **Figure 10D**. A clear separation is observed in decision distances between selected and un-selected tracks indicating a high percentage of correct prediction among the subjects.

#### Distribution of Selected Tracks

**Table 3** shows the frequency of selected tracks based on MER-SUA interpretation and LFP processing. As per standard clinical protocol, the initial trajectory to target STN is determined by preoperative stereotactic imaging. Three tracks are selected by the neurosurgeon based on the initial planning for microelectrode recordings. The initial expectation is that the center track will hit the STN while other tracks account for possible targeting



#### TABLE 3 | Frequency of track selection in the present study.

#### Comparison of SUA-based and LFP-based track selection


*The selection frequency based on LFPs are shown in bold type.*

error. Then based on the MER-SUA recordings the optimal track is selected among these three trajectories. Although the image based planning aims to hit the STN with the center track, **Table 3** demonstrates that intraoperative MER-SUA-based decisions among 25 recordings is not biased toward the center track. We note that the selection frequency is higher in anterior track based on MER-SUA mapping. In addition, the posterior track is not selected at all. When the selection frequency based on LFPs is studied, it can be seen that both MER-SUA and LFP decisions (shown in bold) match with a high percentage but LFP based prediction was more in favor of the center track. Overall, our results indicate that stereotactic planning does not perfectly correlate with intraoperative electrophysiology based track selection and highlight the variance in track selection.

#### Post-Operative Programming Parameters

We explored whether there exist any difference in programming parameters between the correctly classified and misclassified patients. In particular we investigated the post-operative simulation parameters such as voltage, frequency and pulse width which were selected during the programming 6 months after the surgery. The distribution of selected stimulation voltages are presented in **Figure 11**. We note that the average stimulation voltage used in correctly classified group is 1.72 ± 0.63 V while it is 2.12 ± 0.69 V in misclassified group. Student's two sample t-test analysis indicates that the difference in voltages between two groups is not statistically significant (t = −1.16, p = 0.2595). However, one of the misclassified subjects we observed has high level beta and HFO activity in both selected and un-selected track and this patient is stimulated with 1 V. This indicates that both tracks could be viable. When this subject is removed from the misclassified group, we note that the mean post-operative stimulation voltage increases to 2.4 V ± −0.46 for the misclassified population. The difference in post-operative stimulation voltages between correctly classified and misclassified groups without this outlier becomes marginally significant (t = −1.92 p = 0.0685). No significant difference is found either in the frequency (183.1 ± 5.8 Hz) or in the pulse width (90 ±

the un-selected track-2 is passing zero line and reaching the highest confidence level, it is determined as the optimal track by the algorithm. Panel (D) shows the average optimal track decisions with the standard deviation coming from all test subjects. It shows a clear separation in decision distances between selected (red) and un-selected tracks (blue) indicating a high percentage of correct prediction among the subjects.

26.5µs) between groups (t = −0.74, p = 0.4692 and t = 0.96, p = 0.3477, respectively).

## DISCUSSION

Significant variability exists in the axial and coronal orientation of the STN in humans (Patel et al., 2008), and the motor territory of the STN is small, measuring ∼4–6 mm extent from dorsal to ventral. These factors combined with brain shift between preoperative stereotactic imaging and intraoperative electrode brain penetration can lead to targeting errors in the operating room. Pre- and intra-operative clinical imaging methods alone are suboptimal for accurate placement of a DBS electrode; they are subject to distortion, and visualization of a clear differentiation between the STN and surrounding structures can be difficult. In this regard, our study also indicates a considerable amount of variance in track selection. Under the assumption of hitting the STN through center track by image based planning, track selection was not found to be biased toward the center track in the intraoperative MER/SUAbased decisions indicating that stereotactic planning does not perfectly correlate with intraoperative electrophysiology based track selection.

Accurate localization of STN motor territory through intraoperative electrophysiology is a crucial step for DBS electrode implantation (Zonenshayn et al., 2000; Gross et al., 2006). As recently as 2013, an international survey of highvolume DBS implanting sites revealed that 83% of centers use microelectrode recording indicating that the most commonly used electrophysiological mapping method remains MER-SUA recordings (Abosch et al., 2013). However, the method has several limitations in practice as subjective interpretation of complex signal patterns to localize the anatomical borders of the STN, being less stable and more "susceptible to technical (e.g., impedance) and physiological (e.g., cerebrospinal fluid and blood) fluctuations" (Thompson et al., 2014). As Gross et al. indicated, the number of groups using solely macroelectrode/DBS mapping to target the STN without any microelectrode recording is high (Chen et al., 2006; Gross et al., 2006; Telkes et al., 2014). Although there are advantages of using macroelectrode/DBS technique alone in STN targeting like carrying less amount of risk for intracranial hemorrhage since there is no multiple trajectories and due to the blunt-tip of the macroelectrode (Xiaowu et al., 2010), the drawbacks like microlesion effect which might limit the clinician's ability to test or therapeutic effectiveness in the operating room and poor spatial resolution should be taken into consideration (Rezai et al., 2006; Wang et al., 2014). Although asleep, MRI-based non-MER-guided surgery is gaining sway (Starr et al., 2014), the possibility of clinically testing a DBS electrode prior to permanent

implantation does not exist as yet in the context of such a procedure.

Earlier investigations have documented that excessive beta oscillations in certain basal ganglia structures, especially the STN, represent a pathophysiological feature of PD (Weinberger et al., 2006; Kane et al., 2009; Lopez-Azcarate et al., 2010; Oswal et al., 2013). Excessive beta band (8–30 Hz) activity is detected when the electrodes enter into the STN (Levy et al., 2002; Kühn et al., 2008; Brittain and Brown, 2014). Similarly, excessive oscillations at very high frequency ranging from 200–400 Hz are also observed (Priori et al., 2004; Lopez-Azcarate et al., 2010; Özkurt et al., 2011). Even though these high frequency oscillations are considered to represent a pro-kinetic state, and appear with dopaminergic medication and/or induced movement (Foffani et al., 2003, 2006; Trottenberg et al., 2006), others have demonstrated that HFOs (>200 Hz) can still be observed in the STN during the medication OFF state or at rest (Lopez-Azcarate et al., 2010). In our study, all patients discontinued with their short and long acting medication before the surgery and were in OFF state. As others, we observed HFOs in the resting state and increased band power along with entry into the STN. Existence of excessive beta band and HFOs within the STN in PD can be used in target localization. However, the variability and patientspecific characteristics of spatial distribution of excessive beta and HFOs should be taken into consideration (Chen et al., 2006; Weinberger et al., 2006; Wang et al., 2014).

Despite a few publications using intraoperative microelectrode LFPs for STN localization (Michmizos et al., 2008; Holdefer et al., 2010; Wang et al., 2014), to our knowledge, no studies exist on the functional use of LFPs recorded from multiple microelectrodes for the selection of the optimal trajectory targeting the STN in PD. The present study demonstrates that using spectral features of LFP to identify the optimal track without any decorrelation technique provide suboptimal results due to widely distributed neural signals and/or artifacts masking the spatially and temporally distinguishing patterns during targeting. Therefore, the LMS algorithm is used as an efficient technique to decorrelate the tracks by keeping localized activities in each. The adaptive LMS algorithm is widely used in the biosignal processing field since early 80s for signal enhancement due to its efficiency and low complexity (Widrow et al., 1975; Ferrara and Widrow, 1982; Chen et al., 1990). Since the decorrelation is being done recursively without violating the causality constraint, where each channel is predicted by the current samples of other data channels, the algorithm can be easily executed on standard PC architectures in real-time. Since it is an adaptive technique, the time and depth varying parameters allows tracking time and depth varying LFP activity and does not suffer from the cross talk as much as in the common average based derivation. It should be noted that since it estimates current signal by using a linear combination of other signals, the LMS algorithm cannot fully eliminate the high amplitude artifacts if they are not distributed among the tracks, which constitutes a major drawback of the algorithm. One way of reducing the effect of large artifacts and keep the system stable is to use an error threshold with upper and lower boundaries. Another important factor influencing the benefit of the algorithm is the learning parameter µ. It should be investigated by considering signal properties such that adaptation of the system should be neither very slow nor very fast.

Spectral analysis showed that beta oscillations are getting stronger as the electrodes approach the STN. Not only beta oscillations but also strong HFOs can be observed in the STN area well aligned with beta oscillations. This strong relation is noted in the tracks selected by neurosurgeons in 17 recordings out of 25. In rest of the eight recordings, HFO was either weak or fully absent or they were noted only in one of the un-selected tracks. The energy changes above and in the STN were used to localize the dorsal border. The RMS error of prediction for the dorsal border of STN is obtained from 1.18 to 1.98 mm when the different features are used. The minimum prediction error is found with the power of HFOs (1.18 mm) indicating that, despite the unknown functional role of these high frequency components, they can still be eligible in STN targeting. The LFP is a continuous process and does not suffer from SUA isolation challenge while the target variable is a SUA driven information which is also prone to interpretation error and isolation challenge. Considering the dorsa-ventral size of STN, 1.22 mm prediction error in depth with the fused features may represent 11% in DBS electrode 3387 or 16% in 3389 difference which can be easily compensated with the multiple contacts of the DBS electrode.

The features computed above the STN border provided poor results in prediction of optimal track. We note that the optimal track can be predicted with higher accuracy with the features obtained below the dorsal border of STN. Analysis manifest that 14 recordings out of 25 (56% of entire dataset) indicate spatially distinct HFOs together with beta activity above the dorsal border of STN (see **Figures 7C,D**) in at least one of the un-selected tracks. It can be assumed that these relatively weak oscillations located away from the dorsal border of STN are recorded from thalamic structures. To our knowledge, this considerable amount of thalamic oscillations in PD are not well studied phenomena. These findings presented here might be used as spatial markers in STN localization and might form the basis of further investigation into PD pathophysiology from a spatio-spectral perspective.

It should be noted that our classification technique could not predict the selected track in 20% of the subjects. We did not observe a gender difference between these five misclassified subjects. Specifically, three of them were men and two of them were women. None of the misclassified patients were tremor dominant. Three of these patients were typical PD and the other two were bradykinetic/rigid. The mean age and disease duration were 62.2 ± 13.6 and 12 ± 4.6, respectively and were not significantly different from other correctly classified patients (62.1 ± 8.3 and 10.1 ± 4.6, respectively). The misclassification in the 20% of the patients occurred due to many different factors in LFP signal including weak activity or similar activity between tracks. During these recordings, we did not use any sedation. Therefore, weak activity cannot be related to anesthesia. In one patient with typical PD phenotype, the LFP signal was weak across all tracks. We noted that the beta and HFO activity started to develop toward the bottom border of the planned target deeper than the other patients. We believe that in this particular patient the weak activity across all tracks can be described with the electrode positions. Our observations indicate that the three tracks just started to enter the STN and did not fully went through it. In another misclassified case, the LFP activity was quite strong and similar in two out of three tracks. Therefore, the classifier output was very close for these two tracks. In the other three patients, the HFO activity in the SUA selected track was weak compared to LFP selected track. Studies hypothesize that maximum beta band (13–32 Hz) and gamma band (48–220 Hz) power is highly correlated with stimulation programming parameters in DBS chronic electrode (Ince et al., 2010). When a particular contact pair on the electrode shows strong beta and gamma oscillations, it's assumed that the electrode is closer to the source so that lower stimulation would provide better symptom improvement and less side effects. The present study supports these results. We observed higher stimulation voltages in those patients where the LFPs did not correlate with MER/SUA selected tracks. Despite the lack of statistical significance, the stimulation voltages in the 6-mo-programming of implantable pulse generator (IPG) indicate lower values in the patient group having stronger LFPs in beta and/or HFO bands. A study with larger sample size would be needed to test the validity of this observation.

#### REFERENCES

Abosch, A., Hutchison, W. D., Saint-Cyr, J. A., Dostrovsky, J. O., and Lozano, A. M. (2002). Movement-related neurons of the subthalamic nucleus in patients with Parkinson disease. J. Neurosurg. 97, 1167–1172. doi: 10.3171/jns.2002.97.5.1167

#### CONCLUSION

The present study describes an automated approach for electrophysiological localization of STN, using microelectroderecorded LFPs acquired during DBS surgery simultaneous to MER. This work is novel, in that it is the first study to combine different sub-band features derived from beta (11– 32 Hz) and HFOs (150–450 Hz) of LFPs in order to (1) estimate the optimal track for DBS implantation, and (2) identify the dorsal STN border, with high accuracy. This work also contributes to knowledge about the neurophysiology of PD by describing the spatial localization of HFOs. Because recording LFPs simultaneous with MER/SUA does not prolong the total duration of surgery, using this technique online in the operating room would increase the chance of optimal placement of the DBS macroelectrode within the motor subterritory of the STN, without an appreciable downside. Fused with existing mapping techniques, automated online LFP analysis may increase the accuracy of the DBS macroelectrode placement. This might contribute to the efficacy of DBS by reducing the stimulation voltage and associated side effects. Since the electrode placement is guided by LFP activity, the current technique could also be useful to monitor the LFP events which are capable to fine tune the future DBS settings in a closed loop paradigm (Ince et al., 2010; Rouse et al., 2011; Priori et al., 2013).

Further prospective investigations regarding the clinical outcomes using this technique of optimal track selection are warranted.

### AUTHOR CONTRIBUTIONS

NI and IT designed the study, collected the data, wrote the manuscript, and conducted the analyses. AA and AV performed the surgeries, contributed to data collection, and interpretation of the results. JJ performed the behavioral tests during surgery, evaluated the condition of subjects, and contributed to interpretation of the results. All authors reviewed the manuscript and approved the final version of the manuscript.

#### ACKNOWLEDGMENTS

We wish to thank all the patients who participated in this study. We are grateful to Akshay Gupte for his help with clinical data collection. We are also thankful to Ibrahim Onaran, Anish Sen, and Jiang Tianxiao for their help with data collection and visualization. This research was supported by National Science Foundation, award CBET-1343548.

Abosch, A., Timmermann, L., Bartley, S., Rietkerk, H. G., Whiting, D., Connolly, P. J., et al. (2013). An international survey of deep brain stimulation procedural steps. Stereotact. Funct. Neurosurg. 91, 1–11. doi: 10.1159/000343207

Alpaydin, E. (2010). Introduction to Machine Learning, 2nd Edn. Cambridge, MA: The MIT Press.


mechanism in Parkinson's disease. J. Neurosci. 30, 6667–6677. doi: 10.1523/JNEUROSCI.5459-09.2010


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2016 Telkes, Jimenez-Shahed, Viswanathan, Abosch and Ince. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# EEG-Based Quantification of Cortical Current Density and Dynamic Causal Connectivity Generalized across Subjects Performing BCI-Monitored Cognitive Tasks

Hristos Courellis 1, 2 \*, Tim Mullen<sup>1</sup> , Howard Poizner <sup>3</sup> , Gert Cauwenberghs 2, 3 and John R. Iversen<sup>1</sup>

<sup>1</sup> Swartz Center for Computational Neuroscience, University of California, San Diego, San Diego, CA, United States, <sup>2</sup> Department of Bioengineering, University of California, San Diego, San Diego, CA, United States, <sup>3</sup> Institute for Neural Computation, University of California, San Diego, San Diego, CA, United States

#### Edited by:

Timothée Levi, University of Bordeaux 1, France

#### Reviewed by:

David Thomas Bundy, Kansas University of Medical Center Research Institute, United States Alireza Mousavi, Brunel University London, United Kingdom

> \*Correspondence: Hristos Courellis hcourellis@gmail.com

#### Specialty section:

This article was submitted to Neuroprosthetics, a section of the journal Frontiers in Neuroscience

Received: 10 January 2017 Accepted: 20 March 2017 Published: 17 May 2017

#### Citation:

Courellis H, Mullen T, Poizner H, Cauwenberghs G and Iversen JR (2017) EEG-Based Quantification of Cortical Current Density and Dynamic Causal Connectivity Generalized across Subjects Performing BCI-Monitored Cognitive Tasks. Front. Neurosci. 11:180. doi: 10.3389/fnins.2017.00180 Quantification of dynamic causal interactions among brain regions constitutes an important component of conducting research and developing applications in experimental and translational neuroscience. Furthermore, cortical networks with dynamic causal connectivity in brain-computer interface (BCI) applications offer a more comprehensive view of brain states implicated in behavior than do individual brain regions. However, models of cortical network dynamics are difficult to generalize across subjects because current electroencephalography (EEG) signal analysis techniques are limited in their ability to reliably localize sources across subjects. We propose an algorithmic and computational framework for identifying cortical networks across subjects in which dynamic causal connectivity is modeled among user-selected cortical regions of interest (ROIs). We demonstrate the strength of the proposed framework using a "reach/saccade to spatial target" cognitive task performed by 10 right-handed individuals. Modeling of causal cortical interactions was accomplished through measurement of cortical activity using (EEG), application of independent component clustering to identify cortical ROIs as network nodes, estimation of cortical current density using cortically constrained low resolution electromagnetic brain tomography (cLORETA), multivariate autoregressive (MVAR) modeling of representative cortical activity signals from each ROI, and quantification of the dynamic causal interaction among the identified ROIs using the Short-time direct Directed Transfer function (SdDTF). The resulting cortical network and the computed causal dynamics among its nodes exhibited physiologically plausible behavior, consistent with past results reported in the literature. This physiological plausibility of the results strengthens the framework's applicability in reliably capturing complex brain functionality, which is required by applications, such as diagnostics and BCI.

Keywords: brain-computer interface (BCI), electroencephalography (EEG), causality analysis, source localization, LORETA, independent component analysis, motor activity, spatial reach and saccade

## INTRODUCTION

Dynamic intra-cortical interactions offer a wealth of information that can be used to develop an understanding of brain functionality and quantify the effects of brain pathologies on brain functionality. Traditional static functional characteristics of individual brain regions, and dynamic interactions within a network of brain regions (Moeller et al., 2015) offer limited insight into brain function as they typically do not examine the causality among these interactions. The advent of Grangercausal analysis provided a framework to quantify the asymmetric causal interactions between regions of the cortex through specific frequency bands, which hold functional significance for those regions (Barnett and Seth, 2014; Seth et al., 2015), and has shed light on a large number of network-related disorders, such as Alzheimer's and Parkinson's Disease (Pievani et al., 2014). However, this connectivity information is not limited in use to diagnostics, as an increasing number of brain-computer interface (BCI) applications are utilizing connectivity features for intent discrimination when conducting BCI-related tasks (Zhang et al., 2012; Kabbara et al., 2016).

While a potential wealth of information is available in intra-cortex interactions, a systematic algorithmic approach that allows for reliable generalization of cortical network dynamics across test subjects in a subject group is not readily available. Many sophisticated tools exist (Linder et al., 2011; Niso et al., 2013; Barnett and Seth, 2014), which allow for both parametric and non-parametric estimation of dynamic causal interactions among neural activity signals within an individual. However, these tools exhibit various limitations, such as strict channel-space applicability or lack of accounting of common sources, thus, giving rise to spurious causal interactions within the chosen cortical network (Trongnetrpunya et al., 2015). Additional limitations exist in the source-localization algorithms applied to reconstruct cortical-source activity from the measured channel space data. Many popular algorithms stemming from Independent Component Analysis (ICA) are not able to achieve reproducible source localization outcomes across test subjects (Bell and Sejnowski, 1995). The inability to control where these algorithms localize the sources for decomposition on separate subjects presents a large problem since the sources represent nodes in a cortical network, and an ICA-based algorithm could localize a different network for every subject in a group, making group-level network comparisons difficult. Independent Component (IC) ordering is a highly non-trivial task (Hyvärinen and Oja, 2000) for which very few automated approaches exist barring manual selection, a prohibitively tedious and time consuming task for increasingly large datasets.

In this paper, we develop an algorithmic and computational framework that addresses the aforementioned issues through a multi-step, open-source pipeline of analytic tools (Delorme et al., 2011; Iversen et al., 2014). The pipeline combines the deconvolution power of ICA; the systematic modeling of the geometry of gray matter, thereby creating a common source space across subjects with the use of individually warped boundary element models (BEMs); the selectivity afforded by Cortical Low Resolution Electromagnetic Tomography (cLORETA)—a source-space based, distributed source localization algorithm; and the robustness of the Shorttime direct Directed Transfer Function (SdDTF)—a causality measure, computed from Multivariate Autoregressive (MVAR) model coefficients, that addresses the spurious connectivity affecting Granger Causal methods (Korzeniewska et al., 2003).

A similar approach to source localization and connectivity analysis was proposed by Mullen et al. (2015) as a means of demonstrating that the connectivity features considered in this framework can be applied with great success in real-time brain state identification. In order to demonstrate the power afforded by this framework in group-level analysis, a BCIrelated task originally introduced by Park et al. (2014) was employed, requiring subjects to reach and saccade to presented visual targets while their cortical activity was recorded using electroencephalography (EEG) as the neuroimaging modality. The approach utilized for BCI-related task is a passive-BCI approach, also described as an implicit human-computer interaction, wherein passive EEG equipment that senses the user is used to evaluate cognitive states of the user (Schmidt, 2000; George and Lecuyer, 2010). While, in this case, the closedloop feedback elements of the task simply involve auditory and visual confirmation of reach/saccade task completion, the applicability of this approach to more active real-time closedloop feedback BCI systems is also discussed. Although the motor activity present in the human cortex during simple planned motor actions has been carefully documented with respect to the activity of individual cortical regions (Johnson et al., 1996; Filimon, 2010; Heider et al., 2010; Glover et al., 2012), the dynamic interactions among these cortical regions during planning and execution of the task have not been documented as extensively. Park et al. were able to differentiate between spatial movement of the arm and eye by employing channel-space empirical mode decomposition, but did not characterize connectivity between cortical regions underlying these processes. The algorithmic approach implemented in this paper was first described by Iversen et al. (2014), where the method was applied to a smaller subject group. In this paper, we introduce modifications to the original pipeline in Iversen et al. (2014) including a new maximum-power heuristic that captured previously undetected connectivity involving the Supplementary Motor Area and Precuneus, regions expected to exhibit connectivity given the motor-related BCI task, and non-parametric statistical thresholding of the computed causal connectivity values. Through application of our new pipeline, described fully in the Methods section that follows, we demonstrate in the Results section the strength and specificity of the inference that can be made regarding the dynamic causal interactions present among regions of the cortex during reach/saccade planning and execution. The Discussion section illustrates the physiological plausibility of the findings reported in the Results section with reference to previous findings reported in the literature. The Conclusion section summarizes the features, the scalability, extensibility, and the applicability of the proposed algorithmic and computational framework.

#### METHODS

#### Overview

Electrophysiological data were collected from human participants conducting a BCI-monitored task, and processed according to the methodology and computational tools outlined in **Figure 1**, using MATLAB as the computational platform. The workflow in **Figure 1** was employed to extract cortical source information from a cortical network defined for all of the participants, and, subsequently, identify statistically significant spectro-temporal interactions among the nodes (sources) of this network.

### Participants and Task

Ten healthy, right-handed individuals (age: 20.8 ± 2.6 years) participated in this study. The study was approved by the

FIGURE 1 | General outline for cross-subject quantification of cortical connectivity. Initial data processing and artifact rejection is conducted in EEGLAB (Delorme and Makeig, 2004) (A,B). ROIs are selected through IC clustering, also conducted in EEGLAB (C). Head model construction and Electric Lead Field calculation are conducted in MoBILAB (Ojeda et al., 2014) (D). Source localization is conducted in BCILAB (Delorme et al., 2011) through an implementation of cLORETA that iteratively estimates regularization hyperparameters (E). Connectivity estimation is conducted using SIFT, an extension of EEGLAB (Delorme et al., 2011) (F,G). All of the above software and their internal dependencies are run in MATLAB.

Human Subjects Institutional Review Board of the University of California, San Diego. Written informed consent was obtained from all subjects prior to the start of the experiment.

The task involved a time-constrained eye movement and reach using the subject's dominant hand (right) from a target in the center of a touch-screen to a laterally-offset target as depicted in **Figure 2** (Park et al., 2014). Each trial was either a control condition (lift), where the participant moved the stylus off of the central fixation dock after a brief target onset period (500–700 ms), or a test condition (reach/saccade), where the participant shifted their visual focus on and reached with the stylus to a new target that appeared in the upper right or lower left of the screen. Trials were defined as a combination of the planning period, 0 to 500 ms after the end of the target onset window, during which the subject planned either the reach/saccade or lift, and subsequent execution from 500 to 1000 ms after the target onset window, during which the subject moved their hand and reached for the new target. Each subject completed 256 trials, 96 of which were control condition trials, with the remaining 192 being test condition trials. The position of the new target randomly varied by a small angular amount (± 5 degrees) about a 30 degree direction centered at the position of the initial target—the center of the screen. Detailed description of the cognitive task performed by the participants can be found in Park et al. (2014).

#### EEG Recording

Scalp electroencephalographic (EEG) activity was recorded with a sampling rate of 512 Hz from 64 electrodes positioned on the scalps of the participants in an extended International 10–20 system configuration. The electrode array was grounded/referenced on a standard Driven Right Leg/Common Mode Sense (DRL/CMS) reference (Biosemi Inc., Amsterdam, The Netherlands), and each electrode was individually impedance-tested to ensure low impedance across the entire electrode array (below 5 k for each electrode). The 3D position of each electrode was digitized in order to construct individualized head models for each participant (see Park et al., 2014 for details on the digitization system).

#### Preprocessing and Artifact Rejection

Scalp electroencephalographic (EEG) channel space data preprocessing was conducted using EEGLAB (Delorme and Makeig, 2004) through a procedure outlined in **Figure 1A**. EEG data from each of the 10 participants were band-pass filtered,

was monitored by 64-channel EEG. Participants held stylus over a white fixation point in the center of the screen (Top Left) and were prompted for each trial to either lift and retouch the stylus (Control Condition) or to reach and saccade (Test Condition) to a randomly generated green target (Bottom Left). Epoch structure (Bottom Right) was identical for control and test conditions, with planning conducted while the participants remained on the fixation point with the green target present, and execution beginning when the fixation point was removed. Analysis window includes both planning and execution periods of reach/saccade or lift. Direction of the reach (upper right vs. lower left) was not discriminated since our focus was on the general causal dynamics involved in reaching rather than the encoding of spatial directionality.

retaining frequencies from 1 to 55 Hz, and re-referenced to the channel average since the data was recorded reference-free. The data were epoched, extracting the time period of interest, which encompassed both the planning phase and the execution phase of every trial. No distinction was made between reach directions since the emphasis for this analysis was placed on the general process of reach/saccade planning and execution rather than directional discrimination and, as such, both upward and downward angled reach trials were pooled and analyzed together. The control trials (Lift) and test trials (Reach/Saccade) were analyzed separately, using a cortical network defined over the same ROIs. Differences between the control and test conditions were examined by considering differences in network dynamics between the control and test conditions during the planning and execution phases. Noise and artifact laden trials and channels were identified and removed in a semi-automated manner. The kurtosis of each channel or trial was used as the identification metric, with channels or trials exhibiting a kurtosis Z-score > +5 or < −5 being flagged and subsequently visually inspected before removal.

The epoched channel space data were decomposed into sets of maximally independent components, where independence was achieved by minimizing mutual information between components using Infomax ICA (Bell and Sejnowski, 1995) (process outlined in **Figure 1B**). These components represented a combination of putative effective cortical sources, muscular artifacts, ocular artifacts, and electrical activity from the heart, among other noise sources. Through a semi-automated procedure, cortical sources were defined by employing independent components that both visually, by identifying power spectra and topographical maps with dipolar cortical source character, and computationally, through dipole-fitted residual variance, corresponded to dipoles (each fitted to a dipole with low residual variance). This procedure is particularly well suited for EEG artifact removal since the electrophysiological activity of interest present in the cortex manifests detectably in the form of an electrical dipole. The linear mixing of the electric fields generated by these dipoles, and corresponding unmixing through ICA allows for very precise retention of cortical data and rejection of non-cortical artifacts. Using the weight matrix computed during ICA, the retained cortical independent components were re-projected to channel space, creating cleaned channel space electrical signals that were further downsampled to 128 Hz for use in subsequent processing.

#### ROI Selection Methods-Clustering

Cortical regions of interest (ROIs) for source localization were determined by spatially and functionally clustering the ICs aggregated across all datasets/participants into a common cortical volume, as outlined in **Figure 1C**. The clustering feature vector for each IC was constructed by combining information from: the topographical scalp map of the IC, the event related potential (ERP) associated with the IC and its power spectrum, the spectral power content of a rolling window Fourier Transform (FT) across the IC time-series data before ERP-type averaging, known as the Event-Related Spectral Perturbation (ERSP) of the IC, the associated phases from this rolling FT window that give the Inter-Trial Coherence (ITC) of the IC, the ERP-image constructed from concatenation of all trials for the epoched event, and the three dimensional location of the equivalent current dipole fitted to the IC (Makeig, 1993). These features were selected to ensure that the IC clusters were both spatially tight and functionally homogeneous, thus promoting the creation of clusters from the same cortical region that demonstrated similar spectro-temporal dynamics during the conducted task. PCA was used to reduce both the dimensionality of each of the above features, as well as the final feature vector for each IC to 10 dimensions corresponding to the 10 highesteigenvalues, and, subsequent, K-means clustering of the ICs was conducted with K = 14 groups, whose clustering outcome yielded more spatially tight and functionally homogeneous clusters than the outcomes for K = 9 and K = 16 groups. The centroid of each cluster was computed and anatomically labeled using the Talairach atlas (Lancaster et al., 2000).

#### Head Model Construction

The (BEMs) that defined the geometry of the cortical source space for current localization were constructed using MoBILAB (Ojeda et al., 2014), as outlined in **Figure 1D**. The BEM approximation for the cortex of each participant included a standardized head model—the MRI MNI Colin27 three-layer BEM comprised of scalp, skull, and cortical layers. The cortical layer, where the source localization n occurs, was discretized into a triangular mesh containing 4825 vertices, which were automatically aggregated into anatomical regions based on the MNI standard gray matter atlas, used for ROI selection. A customized head model was constructed for each of the 10 participants based on the digitized electrode positions on EEG cap conforming to the participant's scalp during the experiment. Using standard electrical conductance values for the three BEM layers (0.33 S/m Scalp, 0.022 S/m Skull, 0.33 S/m Cortex Oostendorp et al., 2000), an electrical lead field matrix was derived, relating the conduction of electric fields generated by current dipoles from the cortical mesh (bottom layer) to the top of the skin (outer layer) where the field potential is recorded by scalp electrodes. This lead field matrix, along with the discrete surface Laplacian of the cortex layer of the BEM, were computed using the OpenMEEG toolbox (Gramfort et al., 2010) which was also used later for computing the current density during source localization.

### Source Localization-Cloreta

Cortically Constrained Low Resolution Electromagnetic Tomography (cLORETA) (Pascual-Marqui et al., 1994) was used for source localization due to the fixed source-space it required for current localization, allowing for cross-participant ROI consistency. The source-space was the cortical BEM constructed in the previous section that accounted for the conduction of electric fields generated in the cortex and projected to the surface of the scalp where they were detected by EEG. The cortical constraint placed on the BEM was also necessary due to the inability of EEG to reliably resolve electrophysiological activity in subcortical regions of the brain. Source localization (procedure outlined in **Figure 1E**) was conducted independently for every epoch in each dataset in order to prevent regularization parameters and current activations from being computed by concatenating temporally non-contiguous epochs. The solution to the minimum-norm Lagrange multiplier problem for cLORETA is the Tikhonov regularized pseudoinverse solution for cortical current density expressed as follows:

$$J = T\Phi\tag{1}$$

$$T = \left(K^T K + \lambda^2 H^T H\right)^{-1} K^T \tag{2}$$

where J is the current density on the cortex, Φ is the channel space data, **T** is the optimal transformation matrix from channel space to current density space, **K** is the discrete electrical lead field matrix, and **H** is the 2D spatial Laplacian of the cortical head model, introduced to spatially regularize the localized current on the head model grid points. λ is a regularization parameter initialized on a log-space, using generalized cross validation (GCV), and subsequently inferred using an iterative EM formulation derived from the Bayesian structure of the original current localization problem (Trujillo-Barreto et al., 2004). The cLORETA implementation in BCILAB (Delorme et al., 2010) was used for source localization.

Since the regularization parameter λ was recomputed for every epoch, a new transformation matrix (**T**) was also computed, leading to dynamic increases and decreases of spatial regularization in the final current solution as needed. This prevented both over-regularization in cases where current density ramped quickly between grid points, and underregularization when channel space, and therefore current density magnitudes, were lower. A representative signal was extracted from each ROI by computing the signal power of the current localized at each gridpoint within a given ROI and selecting the gridpoint that exhibited the maximum power.

Based on this max-power selection heuristic, representative current signals were generated for the ROIs—a process which was applied to each of the 10 subjects individually, greatly simplifying network-level comparisons across test subjects since the nodes of the cortical network (the ROIs) were localized in the same location every time. This approach differs from traditional ROI determination methods involving ICs because the latter rely heavily on the common spatial localization of ICA outcomes, which is not guaranteed for multiple separate runs of ICA on different data, and, thus, may lead to inconsistencies for group level connectivity analysis (Thompson et al., 2011).

#### Connectivity Analysis–Sift

Application of the maximum power heuristic on the results of cLORETA created a single time series that represented cortical current dynamics for each ROI selected through clustering. These time series representing ROIs (ROI signals) were used to construct linear (MVAR) models from which spectro-temporal causal interactions between ROIs were computed using the Source Information Flow Toolbox (SIFT) (procedure outlined in **Figure 1F**) (Delorme et al., 2011). Specifically, for each participant, the ROI signals were preprocessed by employing detrending and normalization across time and ensemble. The MVAR models were constructed by applying the Vieira-Morf lattice algorithm to the ROI signals, using a sliding window with a length of 550 ms and a window step size of 10 ms, leveraging the window size to decrease the number of the estimated MVAR coefficients (ratio of data points to parameters > 10:1)—a necessary provision arising from the lack of regularization in the Vieira-Morf algorithm. The optimal order for the MVAR model was determined automatically by averaging the optimal orders estimated by six different information criteria: Akaike Information Criterion, corrected Akaike Information Criterion, Hannan-Quinn Criterion, Schwarz-Bayes Criterion, Rissanen Prediction Error, and logarithm of Akaike's Final Prediction Error (Rissanen, 1978; Tu and Xu, 2012). Each of these information criteria was computed for the range of possible model orders between 1 and 30, and the optimal model order for each participant was automatically determined by identifying the minimum value for each of the six generated curves, computing the average, and rounding down to the nearest integer. The model order for the Lift and Reach/Saccade conditions were determined independently. After model order determination, tests of stability and consistency were performed for each MVAR model in order to validate the ability of the model to capture the dynamics of the ROI signal. The autocorrelation function and three different portmanteau tests were used to determine the whiteness of the model residuals. The percent consistency of the model was evaluated for every sliding window within the epoch, indicating, as a percentage, the relative amount of correlational structure within the data that had been captured by the model. A stability test was conducted on every window within the epoch the MVAR coefficient matrix was evaluated.

Pairwise causal relationships between ROIs were computed from the validated MVAR models by employing the (SdDTF) (Korzeniewska et al., 2008). The SdDTF connectivity η 2 ij f , t at a given spectro-temporal point f , t between ROI-i and ROI-j is computed as follows:

$$\eta\_{j \to i}^2 \left( f, t \right) = \frac{\left| H\_{ij} \left( f, t \right) \right| \left| P\_{ij} \left( f, t \right) \right|}{\sum\_f \sum\_{kl} \left| H\_{kl} \left( f, t \right) \right|^2 \left| P\_{kl} \left( f, t \right) \right|^2} \tag{3}$$

where Hmn (f , t) is the MVAR transfer matrix between network nodes (ROIs) m and n, and Pmn (f , t) is the partial coherence of the signals for nodes m and n. The SdDTF connectivity metric is similar to conditional spectral Granger Causality, but only reports direct causal relationship between signals, removing spurious or indirect relationships caused by upstream network topology, particularly common inputs. Unlike Granger Causality, which utilizes AR model residuals to determine causal influence, SdDTF utilizes the AR transfer functions themselves (see Equation 3) allowing for straightforward generalization of the metric to multivariate systems, in this case a network with more than two cortical nodes. To control for the potentially large and unregularized AR transfer functions, DTF normalization is conducted across incoming and outgoing signals, and across frequencies. This normalized term is weighted by the partial coherence in order to incorporate the direct relationships between source nodes, thus mitigating the phantom source problem present in Granger causal metrics and yielding the full form of the direct Directed Transfer Function. This metric is further developed by evaluating it over a short sliding window across the epoch, an additional feature which serves to promote local stationarity of the ROI source signals and observe changes in connectivity dynamics between signals as the epoch progresses. The SdDTF measure was computed separately across epochs and frequencies of interest in this way, yielding a spectrotemporal asymmetrical causal connectivity matrix of SdDTFs with causal source nodes (FROM) defining the columns and sink nodes (TO) defining the rows of the connectivity matrix. Note that the diagonal elements of this matrix do not have relevant physical meaning for the purposes of this network analysis, and were, therefore, omitted.

A permutation-bootstrap approach was employed to determine the statistical significance of each SdDTF value within the asymmetric connectivity matrix (outlined in **Figure 1G**). The permutations were created by randomly mixing ROIcurrent density epochs from the control and test conditions with equal representation, generating an epoch ensemble on which connectivity analysis was conducted. The resulting ROI connectivity matrices from each permutation were accumulated to create a surrogate distribution. The values of each spectrotemporal position in this distribution were, then, sorted in ascending order, allowing for the associated p-value of the tested connectivity value to be computed in a non-parametric manner through its place in the ranked connectivity values at that spectro-temporal position of the surrogate distribution. This process was applied to both the Lift and Reach/Saccade conditions, where the null hypothesis (H0) was the lack of significant increase or decrease in connectivity between the evaluated condition and the case of randomly mixed ROIs. The alternative hypothesis (H1) was that the evaluated condition individually exhibited connectivity structure that was significantly different from the structure acquired by randomly intermixing the two conditions. The p-value for each spectrotemporal point was determined by using an averaged surrogate distribution computed across all 10 participants. A surrogate distribution for each of the 10 participants was computed by conducting connectivity analysis on 300 permutations of the original data, yielding 300 SdDTF values for each spectrotemporal position. The average surrogate distribution was computed by applying spectro-temporal position-wise averages across all 10 surrogate distributions for each of the 300 values and, subsequently, sorting in ascending order the resulting average values at each spectro-temporal position. With 300 permutations computed for each subject, there was sufficient pvalue resolution to establish which time-frequency points passed the α = 0.05 threshold. This threshold was automatically adjusted since the statistical significance of each spectro-temporal point was evaluated independently by applying the False Detection Rate (FDR) control method as a means of accounting for the multiple comparisons problem (Benjamini and Hochberg, 1995).

The difference between statistically significant values of average connectivity matrices corresponding to the Lift condition and Reach/Saccade condition was computed in order to determine the statistically significant changes in the cortical network between the two conditions.

### RESULTS

### IC Clustering Outcome

The ROIs selected for source localization based on the cluster centroids were: bilateral (left/right) inferior occipital (Occ), superior parietal (Par), and precentral gyrus motor (Mot) cortices, as well as, anterior cingulate cortex (ACC), supplementary motor area (SMA), and precuneus, whose left and right hemisphere grid point regions were combined and treated as a single anatomical region during source localization and ROI collapse.

**Figure 3** depicts the clustering outcome for the reach/saccade condition, showing the dipole cluster and average scalp projection for the eight clusters from which ROIs were determined. The 3D dipole maps plotted on a standard MNI MRI volume demonstrate the spatial tightness of the clusters selected as candidates for ROIs, while the average composite topographical maps indicate that there is consistency in both the orientation and di-polarity of the ICs in each cluster. All of the selected clusters feature a relatively tight positive spike in electric potential over the cortical area where the equivalent dipoles are placed; suggesting that the collection of dipoles within the cluster originate from similarly oriented pyramidal cells in gray matter, generating electrical dipoles whose resultant field produces the positive potential present in the scalp maps. Were the IC clusters not as spatially tight, or comprised of dipoles oriented in different directions, the average scalp maps would have electric potential much lower in magnitude and dispersed over a much wider area of the brain. The spatial tightness of the dipoles and the scalp maps suggests that a robust clustering outcome was achieved, and the associated ROIs were reliably utilized with this data in subsequent analysis.

The determination of ROIs from ICs is not necessarily a one-to-one mapping. A single cluster can provide evidence for inclusion of multiple ROIs, as is the case with cluster 5, for example, whose dipole locations span across both hemispheres. So, when identified with the Talairach atlas, the ROIs fall in bilateral regions of the Occipital Cortex (Inferior gyri) that are too far from each other to be grouped across the medial longitudinal fissure, as was the case with all of the other midline ROIs.

All nine ROIs selected for the cortical network are displayed in **Figure 4**, with color-coded grid-points associated with each ROI as follows: midline ROIs are colored Red (ACC, SMA, Precuneus), and bilaterally (left and right) situated ROIs away from the midline (Mot, Par, and Occ) are dark blue, light blue, and yellow respectively.

## MVAR Model Construction and Validation

Model order selection was conducted with an average MVAR parameter-to-datapoint ratio of 0.096 for the Lift condition and 0.034 for the Reach/Saccade condition. The window size was selected so that, in all cases, this ratio never exceeded 0.1, as is recommended for a well-posed MVAR model computed via the Vieira-Morf Lattice Algorithm. For the Lift condition, the average model order across all subjects was 7.4 with a standard deviation of 1.35, and for the Reach/Saccade condition, the model order

average was 7.2 with a standard deviation of 1.03. **Figure 5** shows the output of the model order selection criteria pertaining to the MVAR model order of the Lift condition for a single participant. The top graph shows the average (across epochs and sliding windows) value of each criterion over model orders from 1 to 30. The histograms that follow show the order distribution using each of the criteria.

The MVAR model validation results averaged across all subjects and both conditions are shown in **Figure 6**. The average model performance was highly representative of the individual

performance of each subject's MVAR model since, in all cases, the MVAR model was stable across the entirety of the epoch, as indicated by the natural logarithm of the largest eigenvalue being negative (**Figure 6** Stability Panel). The percent consistency for both Lift and Reach/Saccade conditions was above 90% and the standard deviation in the percent consistency was <1.0% (0.877% for Lift and 0.712% for Reach/Saccade), indicating that the high consistency of the MVAR model with respect to the data was maintained for the duration of the epoch (all 43 sliding windows within each epoch). None of the computed MVAR models passed the residual-whiteness tests at any point during the epoch, suggesting that there was statistically significant correlational structure exhibited by the MVAR model residuals, and that the MVAR modeling did not succeed in capturing the entirety of the temporal dynamics exhibited by the network nodes during the epoch.

#### Spectro-Temporal Connectivity

The connectivity computed between the ROIs in the defined cortical network is displayed in **Figure 7** for the lift condition and **Figure 8** for the reach/saccade condition, with statistically significant spectro-temporal connectivity (p < 0.05) thresholded for each matrix respectively.

The difference between the p-value thresholded connectivity matrices for Lift and Reach/Saccade, shown in **Figures 7**, **8** respectively, was computed and shown in **Figure 9**. Since the lift (control) condition was subtracted from the reach/saccade (test) condition, the warmer (brighter red) regions of the matrix indicate that connectivity was stronger during the reach/saccade and the cooler (brighter blue) regions indicate that connectivity was stronger during the lift.

The majority of lift-dominant causal interactions originate from the supplementary motor area, parietal areas and the Precuneus. During Lift, the SMA exhibits notable outflow to the Left Parietal Area (L Par) and the Precuneus. The SMA drives activity in the L Par over a constant frequency range (8–15 Hz) during both planning and execution (**Figure 9A**). However, a dichotomy between planning and execution exists in the connection form the SMA to the Precuneus, with sporadic low-frequency connectivity (3–14 Hz) present during planning that narrows in frequency band (8–14 Hz) and becomes more consistent during execution (**Figure 9B**). From L Par, the only significant outflow of information occurs to the Precuneus, dominating lower frequencies (3–14 Hz) during part of the early planning phase and the entirety of the execution phase of the lift (**Figure 9C**). From R Par, information flows to the ACC across low frequencies (3–14 Hz) during the planning phase and for some of the execution phase, though the causal connection is limited to a narrow frequency band (10–14 Hz) by the end of the epoch (**Figure 9D**). Similar connectivity is present with the SMA, although in the case of information flow from R Par to SMA, the connectivity does not subside as late in the execution phase as in the case with the ACC (**Figure 9E**). The R Par exhibits another connectivity motif, providing strong causal influence on both the Left Parietal Area and the Precuneus during the planning phase of the lift over a low frequency band (3–10 Hz), and with

relatively higher power compared to its other causal influences (**Figure 9F**). The Precuneus exhibits the greatest amount of causal outflow during lift, acting as an information source for the majority of the other ROIs in the network. The Precuneus has relatively lower-power (lighter blue shading) influence on both precentral gyri, during both planning and execution, with the band of active connectivity narrowing when transitioning from planning to execution (**Figure 9G**). The Precuneus sends information with a similar spatiotemporal structure to the ACC, but with slightly greater causal power, hence the brighter shading of the connectivity during the planning period (**Figure 9H**). The influence the Precuneus has on the SMA is stronger still (even darker shade of blue), and now occupies the entire duration of the epoch and maintains signal bandwidth (3–14 Hz) (**Figure 9H**). The Precuneus influences both parietal areas very strongly during lift, with R Par receiving information only during the planning phase across a low frequency band (3–10 Hz) (**Figure 9I**), and L Par receiving information across a slightly larger band (3–14 Hz) and for both planning and execution. During execution, however, the connectivity is limited to the higher end of that frequency band (10–14 Hz) (**Figure 9J**).

Within reach/saccade-dominant activity regions, which are defined by the red spectro-temporal regions of **Figure 9**, greater connectivity is present in the frontal regions of the cortex. Specifically, both the left and right precentral gyri exhibit stronger connectivity to the left parietal composite area during the planning phase of the epoch across lower frequencies (3–14 Hz), with this connectivity decreasing in power relative to lift during the execution phase (**Figure 9K**). Additionally, the anterior cingulate cortex (ACC) exhibits increased outflow of information across lower frequencies (3–14 Hz) to both precentral gyri again during the planning phase (**Figure 9L**). The ACC also appears to feed information to the left and right visual cortices across a narrower frequency band (8–14 Hz) during the planning phase (**Figure 9M**), with the bandwidth of information flow encompassing lower frequencies from ACC to R Occ as the epoch progresses from planning to execution. For the reach/saccade condition, the SMA exhibits a greater amount of causal influence on the ACC than any other cortical node, with a strong 14–40 Hz communication band present during the planning phase that dips down to lower frequencies (3–8 Hz) as the subject transitions from planning to execution (**Figure 9N**). The ACC reciprocates some of this activity to the SMA, with higher frequency (15–40 Hz) communication during the planning phase that dies down during the execution phase (**Figure 9N**). The SMA also exhibits some information outflow to the visual cortices, with the L Occ flow subsiding by the end of the planning period, and the R Occ flow remaining constant in both frequency band and relative amplitude over the entire epoch (**Figure 9O**). The right parietal composite area (R Par) exhibits causal influence on the Precuneus in higher frequencies (20–25 Hz) during the planning phase, and significantly stronger lower frequency (3–13 Hz) influence during reach/saccade execution in comparison to the lift execution (**Figure 9F**). The

last of the significant communication motif that is strong during the reach/saccade conditions originates from the Precuneus. The influence of Prec on L/R Occ is most prevalent during the execution phase (**Figure 9P**). In both cases, however, the precuneus predominantly drives lower frequency activity (3–14 Hz), with a significant decrease in connectivity magnitude above 14 Hz. There is also evidence of temporally localized higher power bursts of connectivity across low frequency from the Precuneus to the right parietal area, beginning early in the planning period in the 14–20 Hz band and terminating at the end of the epoch after drifting down to 8–13 Hz (**Figure 9I**).

### DISCUSSION

#### Neurophysiological Connectivity

A cortical network associated with the Lift and the Reach/Saccade conditions was identified across 10 subjects, involving distinct ROIs, and the dynamic causal interactions between pairs of ROIs were computed for each condition. The cortical network, the ROIs, and the dynamic causal connections between pairs of ROIs exhibited physiological plausibility consistent with previous results reported in the literature, although some important differences were noted as outlined further below. The bilateral Inferior Occipital Gyri coupled with adjacent Occipital regions, known to form the Visual ROIs, represent primary processing of the visual stimuli present in the reaching task (Blumberg and Kreiman, 2010) . The various motor regions, including bilateral Precentral Gyri (housing the Left and Right Motor Cortices) and the bilaterally grouped Supplementary Motor Area, have been implicated in the planning and execution of the physical movement, e.g., in this case of the participant's arm to reach the presented target as well as the saccade of the eyes to focus on the target (Johnson et al., 1996; Nachev et al., 2008; Wang et al., 2010). Parietal activity is largely implicated by existing literature in guiding action directed at visual targets (Blangero et al., 2009; Filimon, 2010). The Anterior Cingulate Cortex, a grouped ROI consisting of the Left and Right cortices merged across the longitudinal fissure, is thought to be involved in higher level decision making and, of particular interest in this case, the translation of intentions to actions (Kennerley et al., 2006). The Precuneus, also bilaterally grouped across the longitudinal fissure, has been shown to be directly involved in both visual and non-visual reaching tasks in both macaque monkeys and humans, being responsible for the processing of both visual and proprioceptive sensory information during the task (Filimon et al., 2009).

Though entirely possible to treat the three grouped ROIs (SMA, Prec, and ACC) as distinct, the close proximity of the bilateral ROIs to each other within the longitudinal fissure and the spatially-regularized source localization can lead to significant overlap of posterior current density between the ROIs, generating spurious causal interhemispheric coupling, which is avoided by considering those three ROIs to be trans-hemispheric.

Inspection of the connectivity during the lift and reach/saccade condition, shown in **Figure 9**, reveals that the cortical network exhibits relatively lower strength, but

widespread effective connectivity during the reach/saccade condition, and more focal, higher-strength connectivity during the lift condition. In the case of the reach/saccade condition, the ACC exemplifies this widespread, low-power connectivity profile, driving activity in both Left and Right Precentral Gyri during the planning period, and driving activity in both visual cortices for both planning and execution of the reach. The connection from the ACC to L/R Mot (**Figure 9L**) suggests a causal influence of a decision making center in the frontal cortex onto regions directly involved in the planning and execution of a reach and saccade, a connectivity motif that has been observed before using other neuroimaging modalities (Asemi et al., 2015). Furthermore, this causal influence is shown only to exceed lift-connectivity magnitude during the planning period, indicated by the lack of ACC to L/R Motor interaction after 0.5 s, suggesting that after the movement is planned, execution no longer requires direct influence of the ACC onto the Motor cortices. On the other hand, the ACC to L/R Visual connections (**Figure 9M**) remain elevated for the entirety of the epoch during the reach/saccade condition, indicating a possible involvement of the cognitive-control and error detection functions of the ACC (Stevens et al., 2011) in visual processing during execution. The task-positive connections formed by the ACC exhibit higher frequency connectivity (**Figures 9 L–N**), with the lower end of these connectivity bands lying in the α (8–13 Hz) wave range, which is expected considering the motor-relevance of the α-band to indicate inactivation of motor regions (Brinkman et al., 2014), and the higher end of these connectivity bands stretching into the 30–50 Hz range, consistent with the generally higher-frequency content of cognitive cortical processes (Buzsáki and Silva, 2012).

However, this does not seem to be the case for the information flow from the SMA to the ACC (**Figure 9N**), wherein a high α/low β band connection (10–20 Hz) present during the planning period migrates down to lower frequencies (3–8 Hz), occupying the θ band by the end of execution. A possible explanation for this exception stems from the suggested involvement of the SMA in the real-time control of actions during execution, by which it might provide feedback to the ACC regarding the accuracy of the conducted reach (Nachev et al., 2008). The connections from L and R Mot to L Par (**Figure 9K**) exhibit a connectivity profile similar to the ACC → L/R Mot interaction, with lowerpower, reach/saccade-dominant connectivity during planning that is absent during execution. While both the composite parietal regions and the precentral gyri have been implicated in visuo-spatial processing and reaching to identified targets, the analysis indicates that their interaction is limited during the planning phase of a reach, and is not required in order for the reach to be executed after planning. A strong, sharply divided connection is also observed from the Right Composite Parietal Region to the Precuneus (**Figure 9F**)—a connection which is facilitated during lift-planning and reach/saccade-execution, and suppressed during reach/saccade-planning (for low frequencies) and lift-execution. The connectivity profile suggests that during reach/saccade planning, R Par predominantly communicates with the Precuneus through higher frequencies (14–30 Hz), which is of great significance to the Precuneus since it normally communicates with other nodes in the Default Mode Network through this frequency band (Neuner et al., 2014). The transition to lower frequency connectivity (3–14 Hz) during Reach/Saccade execution may simply be a feature of task demands, since similar communication was seen in **Figures 9H,I,M,N,P**, suggesting a generalized tendency for visuo-spatial and motor related cortical regions to communicate through lower frequency bands when the reach/saccade is being executed. The reach/saccadedominant dynamics of the connection from the Precuneus to R Par (**Figure 9I**) also exhibit characteristics similar to the other motor regions, where the high-frequency planning connectivity gives way to lower frequency execution connectivity. However, during the epoch, the frequency range of the connectivity varies periodically, dipping to lower frequencies and, subsequently, returning to higher frequencies (**Figure 9I**, red profile) at the rate of 5 Hz. This behavior suggests that the Precuneus dynamically alters the frequencies over which it communicates with R Par in a periodic manner. The persistence of this behavior through both planning and execution suggests that the behavior is associated with generalized motor-reach information processing, though the decrease (**Figure 9I**, red profile) at the transition between planning and execution at 0.5 s supports the involvement of this periodicity in the motor task being executed by the subject.

Connectivity present in the cortical network is summarized in **Figure 10**, where the Reach/Saccade condition is shown in a 3D (BrainMovie3D) rendering of a cortical volume (Delorme et al., 2011). The ROIs are represented by labeled spheres of varying size and color, with the magnitude of causal information inflow indicated by color and the magnitude of causal information out-flow indicated by size. Larger size spheres indicate larger information out-flow and warmer colored spheres (brighter red) indicate larger information in-flow. Connections between ROIs are represented by tapered cylinders with varying color and diameter. Cylinder tapering indicates the direction of information flow, while cylinder width indicates flow magnitude, and cylinder color represents the dominant frequency of communication, with cooler colors for lower frequencies and warmer colors for higher frequencies. Causal information flow is integrated across all computed frequencies (2–50 Hz), and is shown here (**Figure 10**) for t = 250 ms, that is, during the planning period. The 3D rendering demonstrates the dominant role of the Precuneus as an information source, indicated by its large size, the higher power/frequency coupling of frontal regions (ACC, SMA, L/R Mot), indicated by warmer colored and larger connecting cylinders, and the higher frequency communication present in the network during reach planning.

#### Framework Limitations

The above discussion of neurophysiologically relevant findings serves to demonstrate the nature of the conclusions that can be drawn through the application of this algorithmic framework. Application of this method yields findings that are both consistent with existing literature, and reveal connectivity information in the cortex with an immense degree of detail, allowing for the documentation of source-space spectrotemporal information flow. There are, however, limitations with the current approach which must be considered.

The source orientation assumptions applied to the cortical BEM for the source space held that all cortical dipoles were oriented normally to the surface of the BEM, a canonical assumption that has been previously employed for BEM construction (Fuchs et al., 2002), and is motivated by the perpendicular orientation of cortical pyramidal cells to the cortical surface. Applying this assumption simplifies solving the inverse problem by reducing the number of variables to be estimated to N<sup>g</sup> (number of BEM grid-points) from 3 ∗ N<sup>g</sup> , as is the case when accounting for the x, y, and z components of the current at each grid-point. However, this assumption greatly restricts the form of the computed current density, and does not allow for any data-driven flexibility with the computed dipoles. Utilizing the position-free BEM and associated Lead Field Matrix would allow for much greater flexibility, allowing for cortical dipole orientation to be computed either jointly with the current density, or after source localization using the current density, and thus identifying orientations through observed data instead of through a prior model.

Another potential drawback of the current source-localization algorithm is the Laplacian regularization utilized by cLORETA. The smoothing features of the discrete Laplacian when used as a Tikhonov matrix cause the estimated current density to be blurred over large regions of the cortical surface. This blurring can cause current density from a single source to spill over into multiple adjacent ROIs and potentially skew causal interactions between these ROIs in a manner that is not corrected by using the SdDTF connectivity metric. Preliminary investigations using this data suggest that this spill-over does not have a significant effect on causal analysis in this proposed framework, but further investigation would be prudent to ensure that the computed connectivity dynamics are not skewed by the choice of cLORETA as the source localization algorithm.

The validity of the presented connectivity dynamics is supported by the statistical analysis of the SdDTF connectivity matrix and the validation conducted on the fitted MVAR models. Though a large amount of the computed connectivity was statistically significant, the residuals of the computed MVAR models failed the whiteness tests because not all correlational structure of the localized current was captured by the models, suggesting the existence of non-linear dynamics. These nonlinear dynamics are readily visible in the model prediction error and suggest that non-linear AR modeling is required to effectively capture these dynamics in the MVAR coefficients. However, all models exhibited a large Percent Consistency in the order of 90%, indicating that a very large percentage of the correlational structure of the localized current in each node was captured by the MVAR. The high percent consistency of the model indicates that these nonlinearities constitute a small portion of the localized current, and the computed MVAR models are able to account for the vast majority of the observed dynamics.

### Extensions to Real-Time BCI Applications

In spite of the potential limitations discussed in the previous section, the proposed algorithmic framework has great potential as a means for extracting connectivity features in real time that can be used for brain state identification and BCI control. While in its current form, the framework is designed for offline processing, Mullen et. al have demonstrated in Mullen et al. (2015) that elements of this framework can be applied in a real-time BCI setting and, with high accuracy, decode brain states using the spectro-temporal connectivity features computed through the use of this framework. The linear nature of the inverse problem formulated through cLORETA allows for rapid localization of current density in the cortex by application of the closed-form solution given in equation 2. Furthermore, the lattice algorithm used for MVAR coefficient estimation, while well suited for off-line analysis where all data is available for fitting, can be replaced by alternative estimation algorithms which take advantage of the convexity of MVAR model-fitting to compute robust models quickly and with relatively few data-points. In addition, the development of online recursive ICA (Hsu et al., 2014) allows for robust online artifact rejection, source identification, and lends to the overall feasibility of this approach in real-time BCI applications. There are other elements of the pipeline, such as the data-driven identification of ROI's for connectivity analysis, which must be adapted to account for the lack of all data present, or meet the computational demands in real-time analysis. Nevertheless, there is great potential use for this framework in both a clinical and research settings for real-time BCI control.

### CONCLUSION

The presented algorithmic and computational framework successfully identified a cortical network, across several subjects, associated with a cognitive task, using EEG signals recorded from individuals performing the task with the aid of a BCI. It also computed statistically significant causal dynamic connectivity profiles among the ROIs comprising the nodes of the identified cortical network. The modular nature of the presented framework allows for flexibility and scalability. The algorithm computing the cortical network can easily scale up to include a greater number of participants without compromising the reliability or accuracy. The ease of incorporating large amounts of data comes as a consequence of utilizing a distributed source localization method with a BEM-defined source space. The flexible-yet-fixed source space geometry allows for warping head models to align with a realistic 3D representation of each participant's cortex by way of the digitized electrode positions, while retaining the identity of anatomical ROIs and their associated gray matter grid points in the head model. Should the user deem it necessary, the modular nature of this framework allows for substitutions of preprocessing algorithms, source localization methods, and connectivity analysis tools as is seen fit. As long as a source-space based localization algorithm is utilized to maintain consistent anatomical regions across subjects, the user is free to create a customized pipeline that best suits their needs, and is not strictly limited to usage of the algorithmic pipeline with the proposed software and algorithms. The proposed analytical approach allows for extraction of information rich connectivity features which account for spectro-temporal connectivity dynamics during the performance of a cognitive task. This feature extraction is made possible by coupling the segmentation-MVAR modeling of cortical signals with the SdDTF—a connectivity metric that is evaluated independently for different frequencies, leading to the asymmetric spectro-temporal connectivity dynamics displayed

### REFERENCES


in **Figures 7**–**9**. The presented framework also includes a nonparametric statistical test to assess the statistical significance of the extracted features—the spectro-temporal connectivity dynamics. The physiological plausibility of the cortical network and the causal dynamic features characterizing interaction among the nodes of the network computed by employing this framework strengthens the framework's applicability in reliably capturing complex brain functionality, which is required by applications, such as diagnostics and BCI.

### ETHICS STATEMENT

This study was carried out in accordance with the recommendations of the Human Subjects Institutional Review Board of the University of California, San Diego with written informed consent from all subjects. All subjects gave written informed consent in accordance with the Declaration of Helsinki. The protocol was approved by the Human Subjects Institutional Review Board of the University of California, San Diego.

### AUTHOR CONTRIBUTIONS

HC introduced improvements to the pipeline, conducted reprocessing and analysis of data, and wrote the manuscript. TM assisted with data processing on the first iteration of the pipeline. HP and GC contributed in an advisory manner with strategy and direction of the paper and the underlying computational methods. HP also directed the original project in which the data was collected, and GC assisting with editing the manuscript. JI (senior author) conducted processing and analysis on the first iteration of the pipeline, contributed in an advisory manner more closely than GC and HP, and assisted with editing the manuscript.

### FUNDING

This work was supported in part by the National Science Foundation (NSF) under Grant ENG-1137279 (EFRI M3C), Grant SMA-1041755 and Grant BCS-1460885, and in part by the Office of Naval Research (ONR) Multidisciplinary University Initiative (MURI) under Award No. N00014-10-1-0072.


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2017 Courellis, Mullen, Poizner, Cauwenberghs and Iversen. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# An Improved Unscented Kalman Filter Based Decoder for Cortical Brain-Machine Interfaces

Simin Li 1, 2, Jie Li 1, 2 and Zheng Li 1, 2 \*

*<sup>1</sup> State Key Laboratory of Cognitive Neuroscience and Learning and IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, China, <sup>2</sup> Center for Collaboration and Innovation in Brain and Learning Sciences, Beijing Normal University, Beijing, China*

Brain-machine interfaces (BMIs) seek to connect brains with machines or computers directly, for application in areas such as prosthesis control. For this application, the accuracy of the decoding of movement intentions is crucial. We aim to improve accuracy by designing a better encoding model of primary motor cortical activity during hand movements and combining this with decoder engineering refinements, resulting in a new unscented Kalman filter based decoder, UKF2, which improves upon our previous unscented Kalman filter decoder, UKF1. The new encoding model includes novel acceleration magnitude, position-velocity interaction, and target-cursor-distance features (the decoder does not require target position as input, it is decoded). We add a novel probabilistic velocity threshold to better determine the user's intent to move. We combine these improvements with several other refinements suggested by others in the field. Data from two Rhesus monkeys indicate that the UKF2 generates offline reconstructions of hand movements (mean CC 0.851) significantly more accurately than the UKF1 (0.833) and the popular position-velocity Kalman filter (0.812). The encoding model of the UKF2 could predict the instantaneous firing rate of neurons (mean CC 0.210), given kinematic variables and past spiking, better than the encoding models of these two decoders (UKF1: 0.138, p-v Kalman: 0.098). In closed-loop experiments where each monkey controlled a computer cursor with each decoder in turn, the UKF2 facilitated faster task completion (mean 1.56 s vs. 2.05 s) and higher Fitts's Law bit rate (mean 0.738 bit/s vs. 0.584 bit/s) than the UKF1. These results suggest that the modeling and decoder engineering refinements of the UKF2 improve decoding performance. We believe they can be used to enhance other decoders as well.

Keywords: brain-machine interface, neural decoding, encoding model, unscented Kalman filter, brain-computer interface, neuroprosthetic

### INTRODUCTION

Brain-machine interfaces (BMIs) have the potential to improve the well-being of people with paralysis, locked-in syndrome, and other ailments, as well as change how humans interact with machines and each other. While there has been substantial progress (Baranauskas, 2014; Nuyujukian et al., 2015) in the accuracy or communication bandwidth of BMI, there is still room for improvement. In our previous work with the unscented Kalman filter based decoder

#### Edited by:

*Paolo Massobrio, University of Genoa, Italy*

#### Reviewed by:

*Robert D. Flint, Northwestern University, USA Jacopo Tessadori, Fondazione Istituto Italiano di Technologia, Italy*

> \*Correspondence: *Zheng Li lz@bnu.edu.cn*

#### Specialty section:

*This article was submitted to Neuroprosthetics, a section of the journal Frontiers in Neuroscience*

Received: *23 September 2016* Accepted: *07 December 2016* Published: *22 December 2016*

#### Citation:

*Li S, Li J and Li Z (2016) An Improved Unscented Kalman Filter Based Decoder for Cortical Brain-Machine Interfaces. Front. Neurosci. 10:587. doi: 10.3389/fnins.2016.00587* (Li et al., 2009), which we refer to as UKF1, we proposed a nonlinear model of neural tuning which modeled the relationship between spike counts and the position and velocity of a cursor. Since that study, much progress has been made in decoder engineering and motor cortical encoding models. We have collected several novel modeling and decoder engineering refinements, as well as incorporated work from others, to form an improved unscented Kalman filter based decoder, which we call UKF2.

The refinements to the encoding model can be summarized as adding neural tuning to hand acceleration, hand position and velocity in an interactive term, and target position and modeling neuron autocorrelation and cross-neuron correlation using spiking history. We include target position as a decoded variable, i.e., the UKF2 does not require knowledge of the true target position to operate. The refinements to decoder engineering are the use of a combination of position and velocity estimates to control the cursor, probabilistically thresholding velocity to determine when the user wishes to remain still, and using estimates of future intended movement to drive the cursor. We do not modify the unscented Kalman filter algorithm itself; rather, our improvements are in the design of the filter's observation model and post-processing of filter outputs.

Using data from two Rhesus monkeys, we compare UKF2 to UKF1, as well as the popular position-velocity Kalman filter in terms of offline reconstructions of hand-controlled cursor movement, encoding model predictive power, and closed-loop neural control of cursor. We also examine the contributions of each modeling refinement. Our results show that the UKF2 reconstructs hand-controlled cursor movement more accurately than the position-velocity Kalman filter and the UKF1. Our analysis suggest that the encoding model of the UKF2 encodes neural activity better, as evidenced by better predictions of firing rate given kinematic and past spiking information. Our analysis of the modeling refinements indicates that spiking history contributed the most to encoding accuracy, but hand acceleration and target position contributed most to decoding accuracy. Finally, experiments in which monkeys used the decoders in closed-loop neural control of the cursor showed that, using the UKF2, monkeys could complete a center-out task significantly faster and with higher Fitts's Law bit rate than using the UKF1, and UKF2 performance was comparable to the FIT Kalman filter (Fan et al., 2014).

Our results indicate that the enhancements of the UKF2 improved the functionality of the decoder. Some of the enhancements, such as modeling of hand acceleration, target position, as well as the probabilistic thresholding of velocity, can be readily used by the Kalman filter and similar decoding algorithms.

#### MATERIALS AND METHODS

#### Surgical Procedures

All surgical procedures were in compliance with the U.S. National Institutes of Health Guide for the Care and Use of Laboratory Animals and were approved by the Institutional Animal Care and Use Committee of Beijing Normal University. Two adult male (6 years, 11 kg; 4 years, 8 kg) Rhesus monkeys (Macaca mulatta) were implanted with silicon-based electrode arrays (Utah array, Blackrock Microsystems) in the left primary motor cortex under sterile conditions. We followed standard Utah array implantation procedures. In each animal, a Utah array was implanted approximately 4 mm anterior to the central sulcus, at approximately 15 mm lateral from the midline (**Figure 1A**), targeting arm and hand areas. Photos from the implantation surgeries for monkey B and monkey M are shown in **Figures 1B,C**, respectively.

#### Electrode Array

We used 96-channel Utah arrays (Blackrock Microsystems) with 1.0 mm long electrode shanks (monkey B) or 1.5 mm shanks (monkey M). The arrays were arranged in a 10 × 10 grid pattern with inter-electrode separation of 400 um. The shank material was silicon with platinum coating on the electrode tip and polyimide insulation (Jones et al., 1992). Electrode diameter tapered from 80 um to a fine point (Jones et al., 1992).

#### Signal Acquisition and Processing

Signals were recorded from the Utah array using a Plexon Omniplex recording system in an experiment room shielded from electromagnetic interference. Signals were amplified (up to 8000x), digitized (at 16 bit, 40 kHz), and processed in the Omniplex system. A desktop personal computer (Dell Precision T3500 with an Intel Xeon W3565 3.2 GHz processor and 8 GB RAM) received the processed signals and executed the Plexon PlexControl software, as well as our experimental control and decoding software.

Spikes were detected and sorted in real time using the Omniplex hardware. The spike detection threshold and sorting parameters were set by visual inspection by the experimenter using Plexon's software. Both well-isolated single units and multiunits were used for decoding and analysis, and subsequently referred to as units without distinction.

We aggressively spike sorted, that is, we preferred to differentiate waveforms into a larger number of units when the choice was not obvious. We often sorted several multiunits per channel. Our reasoning was that if we mistakenly split the waveforms from one neuron into two units, the model fitting should not be biased (though noise due to variance would increase during decoding). If we put two neurons in the same unit, barring a specially designed decoding algorithm such as the switching Kalman filter (Wu et al., 2004), we would lose information. We illustrate the sorted spike waveforms from all channels of monkey B and monkey M in **Figures 1D,E**, respectively.

After spike sorting, spikes were counted in 50 ms duration, non-overlapping bins to estimate the instantaneous firing rate of each unit. All decoders and parameter fitting used this spike count for input.

#### Experiment Control and Kinematics Measurement

A custom brain-machine interface software suite (BMI3, Nicolelis Lab, Duke University) performed experiment control,

model fitting, and real time decoding. This software suite communicated with the Plexon software via Plexon's C language application program interface.

An analog 3-axis potentiometer joystick (CH-400R-P3, Hangzhou Chuang Hong Electric Co.) was used to capture hand motion data. Only the x and y axes were used, and the rotation

axis was ignored in these experiments. The length of the joystick, including handle, was 6.5 cm, and the maximum deflection of the joystick was approximately 4 cm (**Figure 2A**). The joystick was self-centered by a weak spring. This joystick was smaller than that used in our previous work (Li et al., 2009); we observed that the monkey primarily moved its elbow and shoulder joints to control the joystick.

The joystick was connected to a PCI-DAS1002 analog-todigital recording card (Measurement Computing) mounted in a separate desktop personal computer (Dell Precision T3500). Custom software read the joystick measurements and sent them to BMI3 using a gigabit Ethernet local area network.

The position of the joystick was mapped to the position of the cursor on the screen in a one-to-one, piecewise-linear manner, with forward (anterior) joystick positions mapped to upper screen locations and backward (posterior) joystick positions mapped to lower screen locations. Joystick movements and onscreen cursor movements had a scaling ratio of approximately 1:4. Joystick measurements were recorded at 100 Hz. To match the 50 ms bin size of spike counts, the average of the five joystick position measurements within each bin was used.

#### Behavioral Tasks

During experiments, the monkey sat in a primate chair and a flat panel computer monitor was placed 55 cm in front of it (**Figure 2A**). Prior to experiments, all monkeys were trained on two behavioral tasks: center-out and pursuit. In the centerout task (**Figure 2B**), monkeys had to move a circular cursor (logically 0 cm diameter) into a circular target (5 cm diameter) which alternatively appeared in the screen center and the periphery. The peripheral locations were equidistant (8–10 cm) from the screen center at a random angle. Hold time was set to 500 ms. In the pursuit task, monkeys had to keep the cursor within a continuously moving target (6 cm diameter). The target moved according to a Lissajous curve (**Figure 2C**) or a smoothed point-to-point trajectory (**Figure 2D**). The center-out task was used for offline and closed-loop decoding. The pursuit task was only used for offline decoding. Details of the tasks can be found in the Supplementary Materials.

#### Algorithmic Overview

Our decoding method is based on the n-th order unscented Kalman filter decoder (Li et al., 2009), but with numerous enhancements, some novel and some based on prior work. The enhancements fall into two broad categories: those that modify the neural encoding model and those that modify the control mechanism during closed-loop neural control. Encoding model refinements help both offline reconstructions and closed-loop neural control, while control mechanism refinements are only applicable to closed-loop neural control.

In the neural encoding model category are four enhancements. (1) We use the Cartesian coordinates of hand acceleration and a novel acceleration magnitude. (2) We include a novel multiplicative hand position and hand velocity interaction term in the encoding model. (3) We include the target position and a novel target-cursor distance term in the encoding model. (4) We include the recent spiking history (i.e., spike count in the previous time bin) of the entire recorded population in the encoding model.

For the control mechanism, we have added three ideas: (1) we use a mixture of decoded position and decoded velocity to derive the new cursor position (Homer M. et al., 2013). (2) We use a novel probabilistic threshold for velocity, which detects when the user is trying to move in a more principled way than a simple threshold on velocity. (3) We use predicted future kinematics to drive the cursor, which is intended to improve the responsiveness of the decoder.

These refinements are specified below and discussed in the discussion section. Table S1 in the Supplementary Materials provides an overview of which refinements are used during offline reconstructions and closed-loop decoding.

#### State Variables and Neural Tuning Model

The filter's state variables are the variables to be decoded, but can also include other variables which may improve the accuracy of the neural encoding model. In the standard position-velocity Kalman filter, the desired position and velocity in the x and y axis are the state variables (for a 2D task). In the UKF1, the state has multiple taps of position and velocity. That is, at time t (discrete index which counts bins), the UKF1's state variables are the position and velocity at time t, t + 1, t + 2, t + 3, t + 4, t + 5, t − 1, t − 2, t − 3, and t − 4, for a total of 10 taps. These taps include estimates of "future" values of the kinematics as well as "past" values, relative to the current time t. By including future taps and past taps, the UKF1 is able to model neural tuning at multiple time offsets simultaneously. Since the task is twodimensional, the number of state variables is 2 (dimensions) × 2 (position, velocity) × 10 (taps) = 40.

In the UKF2 decoder, hand acceleration and target position are added to the state variables. The target position is a decoded variable and not required as input to the decoder. The number of taps is reduced to five. The number of state variables is thus 2 (dimensions) × 4 (cursor position, velocity, acceleration; target position) × 5 (taps) = 40.

The neural encoding model or tuning model is the observation model of the (unscented) Kalman filter when it is used as a decoder. It is a generative model that predicts the binned spike count (instantaneous firing rate) given the state variables' values. The neural encoding model works in one direction, while the decoder, which wraps around it, works in the other direction by "inverting" the encoding model. To decode a variable, it must be present as a feature in the encoding model and as a state variable in the Kalman filter; however, not all encoding model features are decoded by the decoder (e.g., spiking history), and not all state variables are output by the decoder to control the cursor. The encoding model of the position-velocity Kalman filter is:

$$f\_{i,t}^{\text{PVKalman}} \approx c \cdot p\mathbf{x}\_t + c \cdot p\mathbf{y}\_t + c \cdot \nu \mathbf{x}\_t + c \cdot \nu \mathbf{y}\_t,\tag{1}$$

where fri,<sup>t</sup> is the (mean subtracted) firing rate of unit i at time t, every instance of c is a different coefficient fitted from training data, px<sup>t</sup> and py<sup>t</sup> are the x-axis and y-axis positions at time t, respectively, and vx<sup>t</sup> and vy<sup>t</sup> are the x-axis and y-axis velocities at time t, respectively.

In the UKF1, the encoding model is:

$$\begin{split} \left| \boldsymbol{f}\_{i,t}^{\text{UKF1}} \approx \text{position} \left( t+5 \right) + \text{velocity} \left( t+5 \right) + \text{position} \left( t+4 \right) \right| \\ + \text{velocity} \left( t+4 \right) + \dots + \text{position} \left( t-4 \right) \\ + \text{velocity} \left( t-4 \right) \,. \end{split} \tag{2}$$

For clarity, we have broken down the model to contributions from different kinematic features:

$$\text{position} \left( k \right) = c \cdot p\chi\_k + c \cdot p\chi\_k + c\sqrt{p\chi\_k^2 + p\chi\_k^2},\tag{3}$$

$$\text{velocity}\left(k\right) = c \cdot \nu \mathbf{x}\_k + c \cdot \nu \mathbf{y}\_k + c \sqrt{\nu \mathbf{x}\_k^2 + \nu \mathbf{y}\_k^2}.\tag{4}$$

Again, to reduce notational clutter, every instance of c is a different coefficient (total 60).

Combining the model of UKF1 and the above outlined encoding model enhancements, and reducing the number of taps to five, we obtain the encoding model of the UKF2:

$$\begin{split} \mathcal{F}\_{i,t}^{\text{UKE}} &\approx \text{position}\left(t+2\right) + \text{velocity}\left(t+2\right) + \text{acceleration}\left(t+2\right) \\ &+ \text{interaction}\left(t+2\right) + \text{target}\left(t+2\right) + \dots \\ &+ \text{position}\left(t-2\right) + \text{velocity}\left(t-2\right) + \text{acceleration}\left(t-2\right) \\ &+ \text{interaction}\left(t-2\right) + \text{target}\left(t-2\right) + \text{sjking}\left(t-1\right), \end{split} \tag{5}$$

$$\text{acceleration} \left(k\right) = \left. c \cdot a\chi\_k + c \cdot a\chi\_k + c\sqrt{{a\chi\_k^2 + a\chi\_k^2}} \tag{6}$$

$$\begin{aligned} \text{interaction} \left(k\right) &= \mathcal{c} \cdot \left(p\chi\_k \cdot \nu \chi\_k\right) + \mathcal{c} \cdot \left(p\chi\_k \cdot \nu \chi\_k\right), \\ \text{target} \left(k\right) &= \mathcal{c} \cdot t\chi\_k + \mathcal{c} \cdot t\chi\_k \end{aligned} \tag{7}$$

$$+c\sqrt{\left(t\mathbb{x}\_{k}-p\mathbb{x}\_{k}\right)^{2}+\left(t\mathbb{y}\_{k}-p\mathbb{y}\_{k}\right)^{2}},\qquad(8)$$

$$\text{spiking}\left(k\right) = \left. c \cdot f r\_{1,k}^{\text{actual}} + c \cdot f r\_{2,k}^{\text{actual}} + \dots + c \cdot f r\_{n,k}^{\text{actual}}, \text{(9)}\right)$$

where ax<sup>t</sup> and ay<sup>t</sup> are the x-axis and y-axis hand accelerations at time t, respectively, tx<sup>t</sup> and ty<sup>t</sup> are the x-axis and y-axis target positions at time t, respectively, n is the number of units, and as before every instance of c is a different coefficient (total 70 + n). Note that our use of target position in the encoding model includes a target-to-cursor distance term. The five taps of kinematic features are followed by spiking history terms for the entire population of n units [in spiking(t − 1)]. This spiking history is the spike count in the previous bin for the entire population.

#### Mixing Position and Velocity Outputs

Now we describe the first of the control mechanism refinements. These refinements operate on the output of the unscented Kalman filter. The outputs, or "decoded" variables, are the means of the state variables of the unscented Kalman filter after it has performed its filtering operations. These decoded variables are processed further by the methods described below to get the final on-screen cursor position.

Instead of using only the decoded position to control the cursor as in UKF1, for the UKF2, both the decoded velocity and the decoded position are used to update the cursor during closedloop operation. The two inputs are mixed together using a mixing coefficient (Homer M. et al., 2013):

$$
\overline{\mathbf{x}}\_{t} = \overline{\mathbf{x}}\_{t-1} + c\_m \cdot \mathbf{dt} \cdot \overline{\mathbf{v}}\_{t} + (1 - c\_m) \operatorname{dt} \frac{\|\overline{\mathbf{v}}\_{t}\|}{\|\overline{\mathbf{e}}\_{t}\|} \overline{\mathbf{e}}\_{t}, \tag{10}
$$

$$
\overline{e}\_t = \overline{p}\_t - \overline{x}\_{t-1},
\tag{11}
$$

where x<sup>t</sup> is the vector of on-screen cursor position at time t, c<sup>m</sup> is the mixing coefficient (0.5), dt is the delta time between time steps (50 ms), v<sup>t</sup> is the decoded velocity vector at time t [i.e., v<sup>t</sup> = (vx<sup>t</sup> <sup>+</sup> <sup>2</sup>, vy<sup>t</sup> <sup>+</sup> <sup>2</sup> ), the +2 is due to use of future predictions, described below], e<sup>t</sup> is an intermediate variable representing the difference between the decoded position and previous on-screen cursor location, and p<sup>t</sup> is the decoded position vector at time t [i.e., p<sup>t</sup> = (px<sup>t</sup> <sup>+</sup> <sup>2</sup> , py<sup>t</sup> <sup>+</sup> <sup>2</sup> )]. One way to interpret these equations is that the decoded velocity magnitude acts as a gate or limit for the amount that the decoded position can affect the cursor. We did not use this feature in offline reconstructions and analysis. Note this control mechanism refinement is orthogonal to the position-velocity interaction refinement in the encoding model.

#### Movement Thresholding

We add a probability-based mechanism to help the user stop the cursor during closed-loop control (only) which applies a threshold on the decoded velocity. Since the Kalman filter provides a covariance estimate for the velocity state variable, we can perform a check using the decoded mean and covariance values that allows us to place a threshold in terms of false positive rate. Since the distribution of the state variables is assumed to be multivariate normal, the x-axis and y-axis velocity values together form a vector that has a two-dimensional multivariate normal distribution. Given the mean vector (v) and covariance matrix (Cv) of this distribution, we can compute the statistic:

$$X = \overline{\nu}^{\mathrm{T}} \mathcal{C}\_{\mathrm{v}}^{-1} \overline{\text{v}}, \quad X \sim \mathcal{X}^2 \,(2) \tag{12}$$

X has a chi-squared distribution with 2 degrees of freedom. We can consult the cumulative distribution function of the chisquared distribution to test if the velocity is significantly different from zero with a given α-value.

During decoding, the decoded velocity outputted by the UKF2 is tested using this method. If the null hypothesis is rejected, i.e., the user is deemed to want to move the cursor, unscented Kalman filtering proceeds as normal and the decoded position and velocity are mixed as described in the previous section. Otherwise, the cursor is not moved (skipping mixing of position and velocity) and in the next iteration, instead of adding the velocity to the position in the execution of the transition model, we do not modify the position. Note that this procedure does not set the velocity variable in the state to zero. If we were to do that, accelerating from zero velocity might be difficult: if the velocity increases from zero to a small value, but does not pass the threshold, it is then set to zero again. By not editing the velocity in the state, we allow it to build up over time to exceed the probabilistic threshold. In our experience, a p-value of around 0.3 worked well, and we used values in the range 0.1–0.5 in closed-loop experiments, varying with session. We did not use this enhancement in offline reconstructions and analysis.

#### Use of Future Predictions

During closed-loop neural control, we used filter predictions of future intentions to drive the cursor for the UKF2. The UKF1 included multiple taps in its state. This allowed the filter's observation model to capture tuning relationships between kinematic variables and neural firing rates at multiple time offsets simultaneously. A concrete example is that the x-axis velocity at time bins t − 4, t − 3... t + 5 are all included in the function that models the spike count at time bin t. However, for the UKF1, the position in the filter state corresponding to time t is used to control the cursor when the filter is processing neural activity at time t, i.e., there is no temporal offset.

We keep five taps of kinematic variables in the state of the UKF2, t − 2, t − 1, t, t + 1, t + 2, where t is the time of the bin of neural activity the filter is currently processing. We use the estimated kinematics at the t + 2 tap to control the cursor during closed-loop neural control of the cursor. With our bin size of 50 ms, this amounts to a temporal offset of 100 ms. This offset is in the causal direction, i.e., compatible with the notion that neural activity encodes for movement occurring 100 ms later. For offline reconstructions and analysis, the UKF2 used the zero offset temporal tap (t), otherwise we would introduce error due to temporal misalignment.

#### Decoder Comparison

We compared the improved unscented Kalman filter based decoder (UKF2) with the previously published unscented Kalman filter based decoder (UKF1). The settings for UKF1 differ from Li et al. (2009) in several ways. First, the bin size was 50 ms instead of 100 ms, so as to be easily comparable to UKF2 using the same training data. Second, in offline reconstructions the UKF1 transition model used one time tap of kinematics to estimate the future-most tap of kinematics, whereas, in our previous work, the UKF1 estimate was based on all (10) taps in the state. This change improved filter stability: when all 10 taps are used, forming a 10th order autoregressive model, the fitted transition model was more likely to cause filter instability. In closed-loop experiments, the transition model was pre-designed around physical laws of motion, as described in the Model Fitting section. This contrasts to our previous work, where the transition model was always fit to data. We made this change so that all three decoders would use transition models based on physical laws of motion, since transition model design is not the focus of this study.

We also compare with a Kalman filter which includes position and velocity in its state space (position-velocity Kalman filter). During closed-loop decoding, this Kalman filter used two refinements developed by other researchers: modeling position as a feedback signal (Gilja et al., 2012) and using intention estimates to fit observation model parameters (Gilja et al., 2012; Fan et al., 2014), which makes it equivalent to the FIT Kalman filter (Fan et al., 2014). We do not use the re-training paradigm of the ReFIT Kalman filter (Gilja et al., 2012), because we needed to keep the training data for all of the tested decoders the same to achieve a fair comparison. Moreover, adding a re-training phase to the experiment protocol would allow more time to practice using the Kalman filter decoder, and inject additional variation in animal behavior. Since Fan et al. (2014) reported that intention estimation applied to initial training data had comparable benefits as retraining with intention estimation, we opted to use intention estimation on initial training data in our experimental protocol. See the Supplementary Materials for the implementation of the two refinements.

A brief description of the Kalman and unscented Kalman filters and a table summarizing the compared decoders can be found in the Supplementary Materials.

#### Model Fitting

We fitted the encoding models of all decoders using the same training portion of each session. This data consisted of population binned spike counts and simultaneously recorded cursor positions (equivalent to transformed hand positions), velocities, accelerations, and target positions, if applicable. The encoding models included terms which were non-linear in the state variables, but the coefficients for them can be fitted in a linear regression since the non-linear terms can be pre-calculated as features. We used Tikhonov regularized linear regression (ridge regression) to fit the coefficients of the models, with automatic finding of the best ridge parameter. The parameter fitting procedure is very similar to the one in Li et al. (2009), except for the details of the ridge parameter selection scheme (see Supplementary Materials). We used this parameter fitting procedure to fit the coefficients of the encoding models for all analysis.

In offline reconstructions, the transition model of all three decoders were fitted to training data in the same way as Li et al. (2009), except that we used the newer scheme described in the Supplementary Materials for choosing the ridge regression parameter. Note that this means the target position is decoded, but otherwise does not directly affect the other variables.

In closed-loop neural control, the transition models of all three decoders were set to be similar to the equations describing physical laws. For the Kalman filter:

$$
\overline{p}\_{t+1} = \overline{p}\_t + \overline{\nu}\_t \cdot \mathrm{d}t,\tag{13}
$$

$$
\overline{\nu}\_{t+1} = c\_\mathbf{v} \cdot \overline{\nu}\_t + \overline{\epsilon}\_\mathbf{v}, \tag{14}
$$

where p<sup>t</sup> and v<sup>t</sup> are the position and velocity vectors at time t, respectively, c<sup>v</sup> is the coefficient representing friction (0.85), dt is delta time between filter iterations, i.e., the bin width (50 ms), and ǫ<sup>v</sup> is the random noise on the velocity (details below for fitting procedure). The position lacks a noise term since we used the position-as-feedback scheme. The friction term gives the cursor a virtual mass, which makes it easier to control.

The transition model for the UKF1 is slightly different since it has multiple taps in the state space. We set the leading tap similar to above:

$$
\overline{p}\_{t+6} = \overline{p}\_{t+5} + \overline{\nu}\_{t+5} \cdot \mathrm{d}t + \overline{\epsilon}\_{\mathbb{P}}, \tag{15}
$$

$$
\overline{\nu}\_{t+6} = \mathcal{c}\_{\mathbf{v}} \cdot \overline{\nu}\_{t+5} + \overline{\epsilon}\_{\mathbf{v}}.\tag{16}
$$

Note that position also has a random noise term. For the other taps, values are propagated through time without change.

The UKF2 model includes acceleration, thus we change our equation for velocity and include an equation for acceleration. Acceleration is constant except for a decay coefficient and random noise. Furthermore, target position is included as a constant value modified by random noise. Target position attracts the cursor by affecting acceleration during closed loop control. The UKF2 transition model for the leading tap is:

$$
\overline{p}\_{t+3} = \overline{p}\_{t+2} + \overline{\nu}\_{t+2} \cdot \mathrm{d}t + \overline{\epsilon}\_{\mathbb{P}^\*} \tag{17}
$$

$$
\overline{\nu}\_{t+3} = \mathcal{c}\_{\mathbf{v}} \cdot \overline{\nu}\_{t+2} + \overline{a}\_{t+2} \cdot \mathrm{d}t + \overline{\epsilon}\_{\mathbf{v}}, \tag{18}
$$

$$
\overline{a}\_{t+3} = c\_{\text{a}} \cdot \overline{a}\_{t+2} + c\_{\text{g}} (\overline{g}\_{t+2} - \overline{p}\_{t+2}) / \text{dt}^2 + \overline{\epsilon}\_{\text{a}}, \tag{19}
$$

$$
\overline{\mathcal{g}}\_{t+3} = \overline{\mathcal{g}}\_{t+2} + \overline{\epsilon}\_{\mathfrak{g}}.\tag{20}
$$

where a<sup>t</sup> and g<sup>t</sup> are the vectors of acceleration and target position at time t, respectively, c<sup>a</sup> is the coefficient representing acceleration decay (0.75), and c<sup>g</sup> is the gain on the attraction effect of the target position (0.01). We add acceleration decay to prevent an error in acceleration decoding from affecting decoded output for an unlimited duration. For the other taps, values are propagated through time without change. The values of the constants were picked to be similar to typical values seen when fitting transition models to training data in our preliminary analysis. We did this so that the values would be similar to fitted values, but do not change per closed-loop recording session.

The transition model noise covariance matrices, which describe the joint distribution of the noise terms (ǫp, ǫv, ǫa, and ǫg) were fitted under the above specified models. That is, the above physics-based models were used to predict the variables (using one time step as the input and having the next time step be the desired output), and the sample covariance matrix of the prediction residuals was used as the transition model noise covariance matrix. Note that, for the FIT Kalman filter, we set the positional noise's variance and covariance entries to zero to achieve the position-as-feedback enhancement of Gilja et al. (2012).

#### Experiment Procedure

We first compared the performance of decoders in making offline reconstructions. For this, we used portions of sessions where the monkey controlled the cursor with its hand. We reconstructed the cursor movements with each decoder and measured the accuracy of the reconstructions vs. the actual cursor movements. The data was divided into training and testing portions. We ignored the first 30 s of data to avoid transient problems, used the subsequent 10 min for the training portion, and set aside the remainder as the testing portion. If the session had less than 12.5 min of hand control data, we used 5 min of data for training (the shortest session had 8 min).

We measured accuracy of reconstructions by computing the correlation coefficient (CC) and signal to noise ratio (SNR, see below for equation). CC or SNR for each Cartesian axis' position and velocity were computed separately and combined by averaging. For SNR, the arithmetic mean in decibels was calculated. We did not use mixing of position and velocity (Homer M. et al., 2013) or the position-as-feedback scheme (Gilja et al., 2012) in reconstructions, as those are designed for closed-loop control.

We compared the encoding accuracy of the UKF2 encoding model with the encoding model of the UKF1 and the positionvelocity linear model of the Kalman filter decoder. To do this, we again used portions of sessions where the monkey controlled the cursor with its hand. Since the number of parameters differs substantially between models, a comparison of model fit may be biased toward the more complex model. Therefore, we compared the ability of the models to predict binned spike counts on testing data that was not used to fit model parameters. Model predictions on separate testing data are unbiased toward more complex models, since prediction accuracy reflects generalization accuracy.

We split the data into training and testing portions using a two-fold cross-validation procedure. We used the training portions to fit the parameters of the encoding models. Then we tested the encoding models by providing kinematics data (and past spike counts, if applicable) and then predicting spike counts. We compared the predicted spike counts with the actual spike counts in the testing portion and calculated accuracy using the correlation coefficient or the signal-to-noise ratio (SNR). We repeated this procedure, switching training and testing data, and averaged results between the two repetitions. For model predictions, we only used one tap of kinematics for all models, since our previous work has already shown the advantage of using multiple taps (Li et al., 2009) and that is not the focus of this study.

Finally, we compared the ability of monkeys to use the decoders, in turn, to control a cursor in closed-loop neural control. All the decoders' parameters were fitted on the same initial training data (10 min.), collected at the beginning of each session when the monkey used its hand to control the cursor. The order of use of the decoders was shuffled across sessions to average out order effects. In each session (day), each decoder was used for 10 min, with the first 5 min for familiarization and the last 5 min used for accuracy calculation. During neural control of cursor, the monkey continued to manipulate the joystick, even though it was disconnected (i.e., brain control with hand movements).

When analyzing the closed-loop performance data, we calculated fraction of targets acquired, movement time, and Fitts's Law bit rate (Gilja et al., 2012). Shorter movement durations meant the monkey could move and hold the cursor in the target faster, which reflects better controllability. Since we kept target sizes and reach distances constant, the Fitts's Law bit rate was monotonic with the movement time. For fraction of targets acquired, we included acquisition of the center target as well as peripheral targets. We only considered movements from the center to the periphery for movement time and Fitts's Law bit rate. Since the monkey sometimes paused during performance of the task due to lack of motivation or distraction, failure in the task may occur due to inactivity. We observed that, when the monkey was actively participating, the percentage of successful trials was high (>90%). Thus, to eliminate failures due to inactivity from the time and rate calculations, which are confounds not related to decoder performance, we only analyzed the movement time and Fitts's Law bit rate of successful center to peripheral trials which followed successful acquisition of the center target.

The SNR, in decibels, was calculated by:

$$\text{SNR}\_{dB} = 10 \cdot \log\_{10} \left( \frac{Var\_s}{MSE} \right),\tag{21}$$

where Var<sup>s</sup> is the variance of the desired signal, e.g., recorded position during reconstructions or measured spike counts during encoding model analysis, and MSE is the mean squared error between the desired signal and decoded value, e.g., reconstructed kinematics or predicted spike counts. The SNR can be seen as a normalized, inverted, and log-transformed mean squared error. Unlike the CC, SNR does not saturate. It also detects scale and offset errors which CC cannot. We believe it is better than the mean squared error because it is normalized and thus more comparable across experimental setups, does not saturate, and naturally increases with quality.

For statistical analysis, we used two-factor analysis of variance (decoder × session, or model × unit) with single replication, and we focus on the decoder and model differences. Post-hoc multiple comparison testing was conducted with two-tailed paired t-tests with p-values corrected by the Holm-Bonferroni method. All testing used a significance level of α = 0.05.

#### RESULTS

#### Offline Reconstructions

We compared the ability of the decoders to reconstruct handcontrolled cursor trajectories. We analyzed 16 sessions from monkey B, recorded 24–97 days post-implant, and 16 sessions from monkey M, recorded 17–162 days post-implant. Handcontrolled portions of these sessions ranged from 8 to 73 min in length, with mean 27.2 min. For each session, reconstructions were performed by using up to 10 min to fit parameters of models and then reconstructing the remainder of the hand-controlled portion of the session (see Experiment Procedure). The results are summarized in **Table 1** and graphed in **Figure 3**.

We calculated the SNR of the trajectory reconstructions for each decoder (**Figure 3A**). ANOVA found a main effect of decoder [monkey B: F(2, 15) = 27.03, p = 1.94 × 10−<sup>7</sup> , monkey M: F(2, 15) = 42, p = 2.01 × 10−<sup>9</sup> ] and post-hoc testing showed that all three decoders were significantly different from each other (monkey B: all corrected p < 0.0024, monkey M: all corrected p < 0.00007). We also quantified accuracy in terms of correlation coefficient (CC) (**Figure 3B**). ANOVA found a main effect of decoder [monkey B: F(2, 15) = 15.59, p = 2.28 × 10−<sup>5</sup> , monkey M: F(2, 15) = 53.86, p = 1.18 × 10−10] and post-hoc testing showed that all three decoders were significantly different from each other (monkey B: all corrected p < 0.015, monkey M: all corrected p < 0.00002). The UKF2 reconstructed most accurately among the three decoders.

We pooled data from both monkeys and analyzed the contributions of each of the different encoding model enhancements used in the UKF2 decoder, in terms of CC (**Figure 3C**, white bars). To do this, we added to the UKF1 model each of the model enhancements in turn: acceleration (+A), position-velocity interactions (+PVI), target position (+T), which includes the target-to-cursor distance term, and spiking history of the population (+SH). For the pooled data, ANOVA found a main effect of decoder [F(6, 31) = 29.32, p < 10−10] and post-hoc comparisons showed that UKF2 was significantly more accurate than KF (corrected p < 10−<sup>6</sup> ) and UKF1 (corrected p = 6 × 10−<sup>6</sup> ) and UKF1 was significantly more accurate than KF (corrected p = 0.000014). In terms of feature contributions, UKF1+A was significantly more accurate than UKF1 alone (corrected p = 0.00006). UKF1+PVI was significantly less accurate than UKF1 alone (corrected p = 0.0035). UKF1+T was significantly more accurate than UKF1 alone (corrected p = 0.00092). UKF1+SH was not significantly different from UKF1 (corrected p = 0.39). UKF2 was significantly more accurate than UKF1 augmented with PVI (corrected p = 3 × 10−<sup>6</sup> ) and SH (corrected p = 0.000014). Significance testing results when using SNR values were similar [main effect of decoder, F(6, 31) = 33.49, p < 10−10], except that UKF2 was also significantly better than UKF1+A (corrected p = 0.0019) and UKF1+T (corrected p = 0.00081). Comparing the contribution of individual features in terms of CC, UKF1+A was better than UKF1+PVI (corrected p = 0.000013) and UKF1+SH (corrected p = 0.0011), and UKF1+T was better than UKF1+PVI (corrected p = 0.00023) and UKF1+SH (corrected p = 0.0017). These comparisons indicate acceleration and target tuning contributed the most to reconstruction accuracy.

It was concerning that the UKF1+PVI reconstructed less accurately than UKF1 alone. When we examined behavioral tasks separately, we found that for the pursuit task (both variants combined), UKF1+PVI (0.831 ± 0.026, mean CC±SEM) was nominally higher than UKF1 alone (0.829 ± 0.025), though the difference was not significant (two-tailed paired t-test, uncorrected, n = 6, p = 0.103). We think this is due to the more


*A, acceleration; PVI, position-velocity interaction; T, target; SH, spiking history of population; Pooled, combining data from two monkeys; Merged units, undoing spike sorting by merging units in each channel.*

thorough sampling of the space of possible position/velocity combinations seen in the pursuit task data.

Accuracy for monkey M was generally poorer, and upon examining fitted model parameters, we suspected that the spiking history features were not fitted as well for monkey M. Thus, we examined the contribution of spiking history per monkey. For monkey B, in terms of CC, UKF1+SH was significantly more accurate than UKF1 alone (two-tailed paired t-test, uncorrected, n = 16, p = 0.04995), but for monkey M, UKF1+SH was significantly less accurate than UKF1 alone (two-tailed paired ttest, uncorrected, n = 16, p = 0.0001). The significance testing results were the same for SNR.

As described in the methods, we spike sorted aggressively, which resulted in a large number of multiunits. We wondered if our aggressive spike sorting affected the observed trends. Thus, we tried performing reconstructions using unsorted spiking data. We merged all units on each channel, undoing the process of spike sorting. Note that this is different from using all threshold crossings, since waveforms which crossed the threshold but did not match any unit's template are excluded. The reconstruction results from merged units are shown by the gray bars in **Figure 3C**. The mean accuracy when using merged units was significantly and substantially worse than the mean accuracy when using sorted units under every decoder variant (two-tailed paired t-tests, all corrected p < 0.000035). ANOVA (merged decoders x session) found a main effect of decoder [F(6, 31) = 34,31, p < 10−10]. The trends in accuracy among UKF2, KF, UKF1 and when enhancements are individually added to UKF1 for merged units were similar to the trends from sorted units (except that UKF2 was significantly better than UKF1+T, corrected p = 0.0028), confirming that our aggressive spike sorting did not influence the trends we observed.

We illustrate sample reconstructions of the UKF2, positionvelocity Kalman filter, and UKF1 in **Figure 3D**. In this panel, we show the 30 s of reconstructed x-axis velocity vs. time from one center-out session of monkey M. We can see that the UKF2 reconstruction follows the cursor velocity better at several peaks and valleys, though there are also instances where UKF2's reconstruction is farthest from the true value.

### Encoding Model Predictions

We next compared the encoding model of the UKF2 with that of UKF1 and the position-velocity Kalman filter. The purpose of this is to better understand what motor cortical neurons encode and to explain the improvements in decoding accuracy—it would be unsatisfying if the source of decoding improvements were unknown—and not as evidence for better decoding accuracy. We analyzed the same 16 sessions from monkey B and 16 sessions from monkey M, which together had a total of 8582 single units and multiunits. These units represent a far smaller number of distinct neurons, since electrodes usually record the same neurons between sessions. Due to this, we also conducted analysis on a single session from each monkey, so as to obtain unique units for significance testing. The results are summarized in **Table 2** and graphed in **Figure 4**.

We first plotted the distribution of encoding model prediction accuracy (in CC), among all units in all sessions, as a histogram in **Figure 4A**. In this figure, the dashed vertical lines represent the mean correlation values for each model. We see that the UKF2 model predicted spike counts more accurately for many unit instances (which include repeated observations of the same units over different sessions), with substantially higher mean prediction accuracy (UKF2: 0.210, p-v Kalman: 0.098, UKF1: 0.138). These values are substantially lower than decoding accuracy values, which is not surprising given our limited understanding of what individual neurons are doing and the intrinsic noise in spiking. The fact that we can achieve higher decoding accuracies despite this is because the decoding algorithm is aggregating information from hundreds of units.

We wanted to know how each new enhancement of the UKF2 model contributed to the improvement in encoding accuracy. In addition to the four improvements, we also tested intention estimation (Gilja et al., 2012; Fan et al., 2014), applying it to both training and testing data, to see if this improves encoding accuracy for the Kalman filter's position-velocity linear model. We plot the resulting mean correlation coefficient and SNR in **Figure 4B**. We see that adding spiking history to the UKF1 model resulted in the largest increase in accuracy. Adding the other features resulted in smaller increases in accuracy. Intention estimation did not improve the mean encoding accuracy of the position-velocity linear model.

We wanted to know if these improvements were significant. However, the prediction accuracy of different units is not independent in this analysis since units from different sessions may be the same neuron. Thus, we chose one session from each monkey to perform significance testing. We chose a relatively long center-out session for each monkey with a large number of units sorted. We show the mean prediction accuracy for one session from each monkey in **Figure 4C**. As measured by correlation, ANOVA found a main effect on decoding method [monkey B: F(10, 410) = 716.58, p < 10−10, monkey M: F(10, 218) = 96.68, p < 10−10]. Post-hoc comparisons showed that the mean prediction accuracy of the UKF2 model was significantly higher than that of the position-velocity Kalman and UKF1 models for both monkeys (all corrected p < 10−<sup>6</sup> ). The differences in mean prediction accuracies were all significant (all corrected p < 10−<sup>6</sup> ) between UKF1 and UKF1 augmented with each enhancement, as well as between UKF2 and UKF1 augmented with each enhancement. The UKF1 model also had significantly higher (corrected p < 10−<sup>6</sup> ) mean prediction accuracy than the position-velocity linear model of the Kalman filter decoder. Comparing the contribution of different enhancements when added to UKF1, all pair-wise tests were significant (all corrected p < 10−<sup>6</sup> ), except UKF1+A vs. UKF1+PVI for monkey M


*Kalman, linear encoding model of the position-velocity Kalman filter decoder; Kalman intention estimation, training and testing hand velocity data were modified using the intention estimation scheme; A, acceleration; PVI, position-velocity interaction; T, target; SH, spiking history of population; Self history, using past spiking of neuron to predict its future spiking; Others history, using past spiking of other neurons in population to predict future spiking; Population history, using past spiking of all neurons in population to predict future spiking. For details, see text. Pooled: combining data from two monkeys. Top 10 percentile units: using only units with mean spike height (peak to trough) in the top 10 percentile of all units in the session.*

(corrected p = 0.25). For both monkeys, the linear positionvelocity model with intention estimation had lower mean correlation than without intention estimation (corrected p ≤ 1.7 × 10−<sup>5</sup> ).

We wondered whether the trends in encoding model prediction accuracies would be the same when we only consider units which are more likely to be single units. Thus, we looked at units whose spike height (peak to trough) was in the top 10 percentile of units in their respective sessions. These are the units most likely to be well-isolated single units, and the encoding model prediction accuracies for them were higher under all models. We plot the mean prediction accuracy for top 10 percentile spike height units in **Figure 4D**. ANOVA found a main effect on decoding method [monkey B: F(10, 40) = 53.96, p < 10−10, monkey M: F(10, 20) = 4.96, p < 10−10]. Some posthoc comparisons between features were no longer significant, particularly for monkey M, since the amount of data was less (monkey B: n = 41, monkey M: n = 21), but the trends remained the same.

We were curious how much spiking history alone could predict firing rates. Thus, we compared three simple encoding models that did not use any kinematics or target position, only firing rate history. The self history model uses one bin of firing rate history of neuron i to predict neuron i's instantaneous firing rate:

$$fr\_{i,t}^{\text{self history}} \approx c \cdot fr\_{i,t-1}^{\text{actual}} \tag{22}$$

where c is a fitted coefficient. The others history model uses one bin of the firing rate history of the entire population, except the neuron we are trying to model:

$$\begin{split} \left| \boldsymbol{f} \right|\_{i,t}^{\text{others history}} &\approx \boldsymbol{c} \cdot \boldsymbol{f} \boldsymbol{r}\_{1,t-1}^{\text{actual}} + \cdots + \boldsymbol{c} \cdot \boldsymbol{f} \boldsymbol{r}\_{i-1,t-1}^{\text{actual}} + \boldsymbol{c} \cdot \boldsymbol{f} \boldsymbol{r}\_{i+1,t-1}^{\text{actual}} \\ &+ \cdots + \boldsymbol{c} \cdot \boldsymbol{f} \boldsymbol{r}\_{n,t-1}^{\text{actual}}, \end{split} \tag{23}$$

where cs are n-1 different fitted coefficients, and n is the size of the population. The population history model uses one bin of the entire population, including the neuron we are trying to model:

$$f r\_{i,t}^{\text{population history}} \approx c \cdot f r\_{1,t-1}^{\text{actual}} + \dots + c \cdot f r\_{i,t-1}^{\text{actual}} + \dots + c \cdot f r\_{n,t-1}^{\text{actual}},\tag{24}$$

where cs are n different fitted coefficients. The mean model prediction correlations are shown in **Figures 4C,D** in the rightmost bars.

When considering all units (**Figure 4C**), self history had the lowest accuracy. Others history was substantially higher, and population history was only a small amount higher than others history. Pair-wise differences between the three models were all significant (all corrected p ≤ 2.0 × 10−<sup>6</sup> ). Notably, for monkey B, all three models were significantly more accurate than the position-velocity model with and without intention estimation (all corrected p < 10−<sup>6</sup> ), and the others history and population history models were significantly more accurate than the UKF1 model (all corrected p < 10−<sup>6</sup> ). For monkey M, the others history and population history models were significantly more accurate than the position-velocity model with and without intention estimation (all corrected p < 10−<sup>6</sup> ) and UKF1 (p < 0.00076). The UKF2 and UKF1+SH models were significantly more accurate than all three history models for both monkeys (all corrected p < 10−<sup>6</sup> ), which is expected since they include the history models.

When considering top 10 percentile spike height units (**Figure 4D**), the trends are similar, but fewer comparisons were significant due to less data. Population history was significantly more accurate than the self history model for both monkeys (all corrected p ≤ 0.0033). For monkey B, all three history models were significantly more accurate than the position-velocity model with and without intention estimation and the UKF1 model (all corrected p ≤ 0.0013). For monkey M, differences between the three history models vs. the position-velocity model (with and without intention estimation) and vs. the UKF1 model were not significant. The UKF2 and UKF1+SH models were significantly more accurate than all three history models for both monkeys (all corrected p ≤ 0.00086), as expected.

To illustrate tuning to position-velocity interactions, we depict an example single unit from monkey B with relatively strong position-velocity interaction in **Figure 5**. **Figure 5B** shows representative spike shapes from this single-unit. **Figure 5A** consists of nine panels, where each panel shows tuning in a different portion of the work space. For example, the lower right panel shows velocity tuning when the cursor is near the lower right (hand is near the right and posterior) portion of the work space. Within each panel, the axes represent cursor velocity, with the center of the panel representing zero velocity. Firing rate is indicated by the shading. For example, the shading in the lower right of a panel is the firing rate of the single unit when the hand is moving toward the right and posterior. The visualization was created by performing Gaussian kernel smoothing on a 7 by 7 grid (per panel). All kinematic variables were normalized to be unit-variance and the smoothing kernel width was 3.

We can see differences in the velocity tuning at different positions in the work space. For example, the firing rate was higher for lower-right velocities in the lower-right position (4.26

Hz at the red dot) than in the upper-left position (2.55 Hz at the red dot).This suggests a multiplicative interaction between position and velocity. This figure illustrates why, for 349 out of 411 (monkey B) and 153 out of 219 (monkey M) units, UKF1+PVI predicted spiking rates better than UKF1, and why PVI is a tuning phenomenon which next-generation neural encoding models should probably take into account.

#### Closed-Loop Neural Control Experiments

We compared the ability of monkeys to complete a centerout task using the decoders in closed-loop neural control. We recorded 7 sessions with monkey B, 153–184 days post-implant, and 30 sessions with monkey M, 15–70 days post-implant. Each session was recorded on a separate day and contains one 10 min block of each condition (UKF2, KF, UKF1), and the last 5 min of each 10-min block were analyzed for performance. We recorded fewer sessions with monkey B because the difference between decoders was larger, and monkey B was tasked with other experiments. The results of closed-loop comparisons are summarized in **Table 3** and graphed in **Figure 6**.

We calculated the mean (across sessions) of the fraction of targets acquired (**Figure 6A**), time to move to a peripheral target (**Figure 6B**), and Fitts's Law bit rate (**Figure 6C**) for each decoder, as well as hand control of the cursor via joystick.

The fractions of targets acquired were not significantly different between conditions for monkey B [ANOVA F(3, 6) = 1.13, p = 0.36]. For monkey M, ANOVA found a main effect on mode of control [F(3, 29) = 5.4474, p = 0.0018], and post-hoc tests showed that fractions of targets acquired for the decoders were not significantly different, but all decoders had significantly lower fractions than hand control (all corrected p < 0.034). The high variance in fraction of targets acquired by monkey B (session MS = 0.02718, compared to control condition MS = 0.02309, while other closed-loop control


*Pooled, combining data from two monkeys.*

metrics had control condition MS which were 4–20 times larger than session MS) was due to poor parameter fits in some experimental sessions leading to relatively poorer control, which led to monkey non-participation during the evaluation period, as the monkey had grown used to very good neural control.

In terms of time to move to a peripheral target, for monkey B, ANOVA found a main effect on mode of control [F(3, 6) = 9.9, p = 0.0004]. Post-hoc tests showed that UKF2 times were significantly shorter than UKF1 times (corrected p = 0.049), other decoder comparisons were not significant. Hand control had significantly shorter times than every decoder (all corrected p < 0.040). For monkey M, ANOVA found a main effect on mode of control [F(3, 29) = 36.8127, p = 1.87 × 10−15], and post-hoc tests showed that UKF2 and FIT Kalman filter times were significantly shorter than UKF1 times (corrected p = 0.0012, p = 0.020, respectively), the UKF2 and FIT Kalman comparison was not significant. Hand control had significantly shorter times than every decoder (all corrected p <2 × 10−<sup>6</sup> ).

In terms of Fitts's Law bit rates, for monkey B, ANOVA found a main effect on mode of control [F(3, 6) = 90.09, p < 10−10]. Posthoc tests showed that UKF2 bit rates were significantly higher than UKF1 bit rates (corrected p = 0.00061), and other decoder comparisons were not significant. Hand control had significantly higher bit rate than every decoder (all corrected p < 0.00057). For monkey M, ANOVA found a main effect on mode of control [F(3, 29) = 155.2043, p = 8.2 × 10−35]. Post-hoc tests showed that UKF2 and FIT Kalman filter bit rates were significantly higher than UKF1 bit rates (corrected p = 0.0030, p = 0.012, respectively). UKF2 bit rates and FIT Kalman bit rates were not significantly different, and hand control had significantly higher bit rate than every decoder (all corrected p < 10−<sup>6</sup> ).

When we pooled the data between the two monkeys, the mean movement times were significantly different among the three decoders and hand control [ANOVA F(3, 36) = 34.94, p = 7.3 × 10−<sup>16</sup> , post-hoc UKF2 < FIT Kalman, corrected p = 0.045; UKF2 < UKF1, corrected p = 0.00024; FIT Kalman < UKF1, corrected p = 0.013]. Mean Fitts's law bit rates were also significantly different among the three decoders and hand control [ANOVA F(3, 36) = 80.19, p = 2.3 × 10−<sup>27</sup> , post-hoc UKF2 > FIT Kalman corrected p = 0.046; UKF2 > UKF1 corrected p = 0.00006; FIT Kalman > UKF1 corrected p = 0.0039]. The hand movement times (all corrected p < 10−<sup>6</sup> ), and bit rates (all corrected p < 10−<sup>6</sup> ) were significantly better than that for every decoder. For fraction of targets acquired, there was a main effect on mode of control [ANOVA F(3, 36) = 3.81, p = 0.012], but post-hoc testing did not find significant differences among the decoders. The hand control fraction correct was better than that for the FIT Kalman (corrected p = 0.036) and UKF1 (corrected p = 0.040).

**Figures 6D–F** show all center to peripheral movement trajectories generated during UKF2 control, FIT Kalman filter control, and UKF1 control, respectively, for one session from monkey B. For clarity of visualization, the trajectories have been rotated so that the peripheral target locations are aligned at the top. Thus, all trajectories start from the lower, red circle and end in the upper, green circle. We can see that the trajectories generated during UKF2 control start movement in the wrong direction and overshoot the target the least among the three decoders. From the movement time and bit rate comparisons and the trajectory illustrations, we can see that UKF2 allows the monkey to perform center-out movements more quickly and accurately than the UKF1 and comparably to the FIT Kalman filter.

One may ask why the plot of trajectories for UKF1 (**Figure 6F**) looks darker within the targets. This is because the decoded cursor positions are more "jumpy" during the hold period. This is due to the use of position as the signal to control the cursor, and noisy neural activity causes the estimated position to jump. We conjecture that the position-velocity mixing scheme (Homer M. et al., 2013) and our probabilistic velocity thresholding refinement solve this problem for the UKF2.

### DISCUSSION

The UKF2 reconstructed kinematics offline more accurately than the position-velocity Kalman filter and UKF1. Examining why the UKF2 performed better, analysis of the encoding model found that the UKF2's encoding model made more accurate predictions of neural activity. In closed-loop neural control experiments, the UKF2 allowed better task performance than the UKF1, but comparisons with the FIT Kalman filter were not significant on a per-monkey basis, though they were significant when data from two monkeys were combined.

The differences between monkeys were quite large. Monkey B was more proficient at the center-out task under hand control, with lower movement times and higher bit rate. Offline reconstructions were generally more accurate with monkey B. The differences between decoders in closed-loop control were larger for monkey B. A particularly pronounced area of disparity was encoding model predictions. Monkey B and monkey M had similar encoding model prediction accuracy for the Kalman filter, but quite different accuracy for the UKF1 and UKF2. For monkey B, adding spiking history to UKF1 resulted in the largest improvement, with target tuning also large. For monkey M, improvements from these two features were substantially smaller. The trends suggest that spiking history is less beneficial for modeling the activity of monkey M's units, and even harmful for decoding accuracy, which hints at some qualitative differences in the populations recorded from these two monkeys. Overall, the differences between monkeys may be due to: electrode length (B: 1.0 mm, M: 1.5 mm), age (B: 6, M: 4), amount of practice with the center-out task (B: 4 months, M: 2 months), and spike sorting performed by different experimenters.

During the design of the UKF2 algorithm, we used pursuit task data from another monkey, which we cannot publish here. We froze the design of our algorithm, as much as possible, before starting closed-loop experiments and data collection for reconstructions. Thus, we included refinements such as PVI and SH which were not universally beneficial.

Our results with CC and SNR are, in some cases, quite different in absolute terms. This is understandable since they are very different measures, with SNR including a logarithm to avoid saturation.

All decoders we tested, including the FIT Kalman filter, had significantly worse performance, in terms of movement time and Fitts's Law bit rate, than hand control via the joystick. These differences were large, for example, hand control had 1.9 times the bit rate of UKF2 and 2.1 times the bit rate of the FIT Kalman filter. This indicates there is still much room for improvement in decoding and signal acquisition methodology.

In terms of the contributions of the different refinements, our closed-loop results are limited in granularity; future work may investigate the individual contributions in closed-loop, which would greatly assist practitioners in optimizing their decoder design.

### Acceleration

Acceleration of the hand has been decoded in past studies (Ashe and Georgopoulos, 1994; Gao et al., 2003), and force has long been known to contribute to encoding (Evarts, 1968 and many others). In our previous work, we considered using acceleration as a feature in the encoding model, but did not detect a substantial benefit. After UKF1's publication, we continued to look for ways to improve the neural encoding model using new kinematic features. Our recent analysis included temporal smoothing (sliding window moving average) of kinematics before parameter fitting for the encoding model (see Supplementary Materials). Doing this, the quickly-changing acceleration signal is smoothed out, and neural encoding strength for acceleration is increased, resulting in worthwhile decoding accuracy increases. Similar to position and velocity, upon visualization of acceleration tuning, we found spatial patterns consistent with the existence of a relationship between the acceleration magnitude (in any direction) and the firing rate of many neurons. Thus, we have also included the magnitude of the acceleration vector as a novel feature of the UKF2 encoding model. Even though we do not use the acceleration in the filter state to directly control the cursor, the acceleration interacts with the other variables in the state space via the state transition model (movement model).

### Position-Velocity Interaction

In our quest for a better encoding model, we visualized neural tuning patterns in various ways (for example, **Figure 5A**). We found that the encoding of velocity changes with the position of the hand in a systematic way. Thus, we surmised that there is an interaction between position and velocity tuning. Differences in preferred directions at different limb postures have previously been found (Caminiti et al., 1990; Sergio and Kalaska, 2003), and gain-field encoding has been suggested for limb position and velocity (Hwang et al., 2003). To capture this interaction, we added a multiplicative feature to our encoding model which is a simple multiplication of the position and velocity for each dimension separately. This encoding model refinement was used both in offline and closed-loop decoding. We found that by adding this novel encoding feature, significantly more accurate predictions of firing rate could be achieved (**Figure 4C**).

However, in terms of offline reconstructions, using positionvelocity interaction was actually detrimental when considering both center-out and pursuit tasks, and slightly helpful (though not significant, since n = 6) when considering the pursuit task only. In the center-out task, position (with respect to center of workspace) and velocity are either very correlated (outward movement), very anti-correlated (inward movement), or independent (hold), with few instances of other relationships. This is not true for the pursuit task, which samples the possible space of position and velocity values much more thoroughly. We believe this difference accounts for the results we found. For neural control of a prosthetic, where movements throughout the space of possible position and velocity values need to be supported, the position-velocity interaction term will likely help.

### Target

Inspired by studies which included the target of reaches in the trajectory decoding process (Shanechi et al., 2012; Shanechi M. et al., 2013; Shanechi M. M. et al., 2013; Shanechi and Carmena, 2013), we investigated adding information about the target into the encoding model. In preliminary analysis we found that some neurons show significant encoding of target position, which has been found in the past (Fu et al., 1995). Though using the neurons we recorded to decode target position alone provides very noisy results, the rough information that can be decoded is still valuable. We set the target position to weakly attract the cursor during closed-loop control. This, in effect, gives a small assistance to the cursor decoding by using the rough estimate of the target location. This is somewhat similar to the mechanism that Shanechi and Carmena (2013) used, where the target position is used in an optimal feedback controller, which can be understood as biasing decoded movement toward the target.

In addition to target position in Cartesian coordinates, our preliminary analysis showed that there was significant encoding of the distance between target position and cursor position, which is similar to the reach distance found by Fu et al. (1995). Thus, we included this novel feature in our encoding model as well.

### Spiking History

In multiple previous studies (Paninski et al., 2004b; Truccolo et al., 2005; Lawhern et al., 2010; Saleh et al., 2010, 2012; Truccolo et al., 2010; Park et al., 2014; Xu et al., 2014) the past spiking of a neuron as well as the past spiking of other neurons in the population have been used to better model the probability of spiking in a point process framework, sometimes leading to very accurate models (Truccolo et al., 2010). The past neural activity of all neurons in the population may capture correlations in firing due to functional connectivity or common inputs. Another advantage of this modeling approach is the ability to indirectly capture tuning to latent neural states. One disadvantage of this approach is the large number of additional parameters that must be fitted, with the accompanying increase in over-fitting risk.

We wanted to include spiking history in our encoding model to capture these benefits. However, instead of using complex temporal features as in the point process studies, we use a comparatively simple idea: we include only the spike counts in the previous bin for the entire population. In preliminary analysis we investigated using more than one previous bin and found the benefit to be small in comparison to the additional cost in number of parameters that needed to be fit.

During closed-loop decoding, the spike count of the previous time bin is available, and is directly provided to the neural encoding model as point values without uncertainty. In other words, unlike the filter state variables in the encoding model, the previous spike counts are not decoded; their benefit comes from improving the fit of the encoding model. This is similar to the role of the position state variables in the ReFIT Kalman filter. By including spiking history, we aim to remove more systematic variation (e.g., from autocorrelation) from the encoding model's residual, making the remaining residual closer to white noise, which better fits the theoretical assumptions of the Kalman filter framework. However, it is still beneficial to include improved kinematic features in the encoding model: these provide the "conduits" through which information flows from neural activity to kinematic variables during the operation of the unscented Kalman filter.

Our results show that adding spiking history improved encoding model predictions substantially. However, offline reconstruction accuracy only improved when adding spiking history for monkey B, with possible reasons discussed above.

### Mixing Position and Velocity Outputs

In our previous work with the unscented Kalman filter, we used the decoder's position output to directly drive the cursor. As the position is decoded from noisy firing rates, we observed that the decoded position jumped around in a small area at high frequency. One benefit of the position-as-feedback enhancement of the ReFIT Kalman filter is that by controlling position only through decoded velocity, this "jumpy" cursor phenomenon is avoided.

We believe that while some neurons in the motor cortex do encode for the current position of the limb in a feedback or mental representation sense, there also exist neurons which encode for the desired position in the immediate future (as opposed a distant future time, i.e., the ultimate target of a reach). If we do not make use of encoded desired position, we may be losing information potentially helpful for controlling a neuroprosthetic, particularly because UKF2's encoding model includes position-velocity interaction terms.

Thus, we chose to retain position as a directly decoded variable in the state space. However, in an effort to avoid the "jumpy" cursor of the UKF1, we adopted the method for mixing position and velocity proposed by Homer M. et al. (2013). In this scheme, if the decoded velocity is zero, the position decode cannot change the cursor. This mechanism assists the user in stopping the cursor, as well as reduces the jumpiness of the cursor. By partially controlling the cursor using the position output, we also stabilize the cursor, preventing velocity decode errors from accumulating, which was a problem we discovered in preliminary experiments when combining the position-velocity interaction enhancement and the position-as-feedback refinement of the ReFIT Kalman filter.

### Movement Thresholding

The ability to stop and hold the cursor (or prosthetic limb) is important for various tasks. Several studies have examined how to stop the cursor more accurately. Golub et al. (2014) used a refinement of the transition model which allows the user to perform a "hockey stop," that is, the user changes the movement direction quickly to slow the cursor to a stop. Another approach (Velliste et al., 2014) decoded a speed term, separately from the Cartesian velocity coordinates. This speed term is used to scale the position and velocity uncertainties in the transition model, effectively acting as a gate for movements. In the mixture method of Homer M. et al. (2013), decoded velocity also acted like a gate for the influence of decoded position on the cursor. Another related decoding engineering feature is the detection of idle states—when the user is not actively using the neuroprosthetic. Aggarwal et al. (2013) and Velliste et al. (2014) detected states using a linear discriminant analysis classifier, separate from the movement decoder. When an idle or hold state was detected, the decoder output is ignored and movement was set to zero. Recently, Sachs et al. (2016) detected posture vs. movement states using linear discriminant analysis and used Wiener filters with different coefficients during each.

To improve the user's ability to stop the cursor within the target during closed-loop control, we added our own mechanism to detect movement intention, a probabilistic threshold for movement which is computed using the uncertainty output of the unscented Kalman filter. Using a probabilistic threshold means we can set the threshold in terms of a false positive rate. This probabilistic threshold is similar to a significance test; the null hypothesis is that the user wants to remain still. We check if there is enough evidence to reject the null hypothesis under the specified false positive rate. This framework, while more complex than a simple threshold on the decoded velocity, allows one threshold to work under different amounts of uncertainty in the decoded output, e.g., for both fast and imprecise (more uncertain) movements and slower and more precise (more certain) movements.

In this study, we set the desired false positive rate by hand. Larger false positive rates mean the cursor is rarely stopped through this mechanism, and holding inside a target may be more difficult if control is poor. Smaller false positive rates may make the cursor too difficult to move. Future decoders with multiple modes of operation may find it advantageous to use a lower false positive rate for certain modes where unwanted movements are dangerous or highly undesirable, e.g., when the user is asleep. While the position and velocity mixing method (Homer M. et al., 2013) also helps stop the cursor, it is dependent on accurate decoding of velocity. Our probabilistic threshold complements this method by verifying that the velocity is not non-zero due to mere noise.

### Use of Future Predictions

Motor cortex neurons encode for movements that occur up to a few hundreds of milliseconds later (Ashe and Georgopoulos, 1994; Schwartz et al., 2004; Paninski et al., 2004a; Wu et al., 2006; Wang et al., 2007; Wang and Principe, 2010), making decoding of intentions at t + 100 ms to t + 300 ms possible given neural activity at time t. These "future predictions" are merely a reflection of the built-in delays in the natural motor system. Most previous work used the decoded kinematics for time t as the output at time t, in effect mimicking the delay of the natural motor system. In the UKF1, even though future intended movements are decoded, we did not choose to use future predictions to control the cursor.

Cunningham et al. (2011) found that reducing the bin width during closed-loop neural control with feedback improves performance, and Willett et al. (2013) found that using predictions of future intentions can compensate for delays in the Li et al. Improved Unscented Kalman BMI Decoder

BMI system. Inspired by these studies, we wondered whether using future predictions could improve neural control. Our temporal offset was 100 ms in size, i.e., we use the decoded kinematics for t + 100 at time t, which is reasonable considering the 75–100 ms average offset found in prior work (Ashe and Georgopoulos, 1994; Paninski et al., 2004a; Schwartz et al., 2004).

One may ask why include the other taps in the filter state, if they are not used to control the cursor. The answer is that they help model the firing rate of neurons, which may encode for movements in a temporally persistent manner or specifically encode for movements in the past. Similar to adding spiking history to the encoding model or the position-as-feedback enhancement, the other taps do not directly affect the decoder output, but may improve decoder accuracy by explicitly modeling what would otherwise be thrown in to the catch-all error term.

### Related Work

Reviews of research in decoding for BMIs can be found elsewhere (Homer M. L. et al., 2013; Andersen et al., 2014; Baranauskas, 2014; Bensmaia and Miller, 2014; Kao et al., 2014; Li, 2014). Here we discuss the decoders compared in the present study.

The improved unscented Kalman filter decoder proposed in this study is a development of our previous unscented Kalman filter decoder (Li et al., 2009). That filter, which we refer to here as UKF1, used an encoding model with non-linear dependence on kinematic variables which modeled tuning to the speed or velocity magnitude of movements. The UKF1 modeled tuning at multiple temporal offsets, using an n-th order hidden Markov model framework where n taps of kinematics (n = 10 was tested) are held in the state space. Encoding studies by Paninski et al. (2004a,b), Hatsopoulos et al. (2007), Hatsopoulos and Amit (2012) and Saleh et al. (2010) found tuning to position and velocity trajectories, called movement fragments or pathlets. The n-th order framework makes the encoding model of the UKF1 flexible enough to capture such tuning. Even though including taps of position also indirectly includes velocity, explicitly including taps of velocity reduces the amount of non-linearity needed in the neural encoding model, which helps improve the approximation accuracy of the UKF. On the basis of UKF1, we expand the neural encoding model and add decoder engineering improvements developed by ourselves and other groups to make the UKF2.

The ReFIT Kalman filter (Gilja et al., 2012) has demonstrated high communications bit rate by using two advances in decoder engineering. In closed-loop experiments, we compared the UKF2 with the FIT Kalman filter (Fan et al., 2014), which is similar to the ReFIT Kalman filter in using position-as-feedback and intention estimation, but does not have the online re-training component. The bin size in this study, 50 ms, was the same as the Gilja et al. study. Our Fitts's Law bit rate values for the FIT Kalman filter are lower than those reported by Gilja et al. for the ReFIT Kalman filter, likely due to a combination of factors. First, online re-training separates the FIT and ReFIT Kalman filters. In terms of experimental setup, Gilja et al. used video tracking of natural reaching movements, whereas we used a joystick during hand control of the cursor. The use of an unnatural joystick made our task more difficult: the mean movement time during hand control in our task was approximately double those reported by Gilja et al. we used a joystick due to the limitations of our experimental platform and to compare with our previous work (Li et al., 2009). While using the same Fitts's law bit rate measure, our task used circular targets, which have a smaller acceptance area for the same width compared to the square targets of Gilja et al. We used circular targets because they are more natural in terms of determining whether the cursor is within the target by using a distance criterion. We also spike sorted and did not include unsorted or "hash" threshold crossings, whereas Gilja et al. used threshold crossing counts.

### Latent Neural State

Models proposed by many previous studies have modeled latent neural states explicitly (Brockwell et al., 2007; Kulkarni and Paninski, 2007; Wu et al., 2009; Lawhern et al., 2010; Macke et al., 2011; Petreska et al., 2011; Aghagolzadeh and Truccolo, 2014, 2016; Deng et al., 2015; Kao et al., 2015; Lakshmanan et al., 2015). In Aghagolzadeh and Truccolo (2014) and Kao et al. (2015), the latent neural state comprises the entirety of the Kalman filter state, and kinematics are decoded from this latent neural state after it is decoded from the spike counts. When modeling latent states like this, some form of unsupervised learning is required to fit the observation model of the filter. The typical approach is Expectation-Maximization applied to linear dynamical systems (Shumway and Stoffer, 1982).

An alternative approach is to implicitly model latent states by adding spiking history to the observation model. By adding spiking history, one may (partially) capture latent shared variables if they have temporal autocorrelation. In other words, if a unobserved common input of many neurons is changing slowly, by using the past neural activity, which partially encodes this hidden input, to predict the current neural activity, one is including this hidden common input in the encoding model by proxy.

Two engineering advantages of explicitly modeling the latent variable using additional state variables are: (1) lower dimensionality; (2) ability to impose prior assumptions on the model structure, such as in the transition model or observation model (Aghagolzadeh and Truccolo, 2016). Additionally, investigating these latent states may yield neuroscientific insights. However, it is not obvious this approach is always better from a decoding point of view, since the unsupervised learning of latent variables cannot be checked against a gold standard, and, in practice, it is vulnerable to local optima. Some of the autocorrelation or cross-correlation captured by spiking history may not be due to low-dimensional latent variables, but are due to biophysics and actual neuronal connectivity. The effect of these phenomena may not be easily captured by low dimensional latent states.

## CONCLUSION

We have shown in offline analysis and closed-loop experiments with two Rhesus monkeys that our encoding model features and decoder engineering refinements improve encoding and decoding accuracy. Some of the enhancements used in this work, particularly the probabilistic velocity thresholding and the inclusion of hand acceleration and target position (without the non-linear terms) are compatible with the standard Kalman filter, are fairly easy to implement, and are likely to bring the largest benefits. We hope that these enhancements will be utilized by others, just as we have improved our decoder using innovations published by others.

### AUTHOR CONTRIBUTIONS

SL and ZL designed the algorithm. SL and JL collected the data. SL, JL, and ZL analyzed the results. ZL drafted the manuscript. SL, JL, and ZL revised the manuscript.

### ACKNOWLEDGMENTS

This work was supported by the National Natural Science Foundation of China (61431002), National Key Basic Research

#### REFERENCES


Program of China (2014CB846101), National Natural Science Foundation of China (61401028), and Fundamental Research Funds for the Central Universities. We thank Wu Li, Dajun Xing, Feng Wang, Xibin Xu, Shujie Wang, Xing Ma, Lüyan Zhong, Yin Yan, Minggui Chen, Rujia Chen, Zhihan Li, Yang Li, Tian Wang, Chen Yang, Guanzhong Yang, Yi Yang, Xiuli Chen, and Chenguang Zhang for technical assistance and assistance during animal surgeries. We also thank Joseph O'Doherty, Wu Li, and the reviewers for helpful comments on the manuscript. We thank Miguel Nicolelis for permission to use the BMI3 software suite.

#### SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: http://journal.frontiersin.org/article/10.3389/fnins. 2016.00587/full#supplementary-material


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2016 Li, Li and Li. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# An FPGA Platform for Real-Time Simulation of Spiking Neuronal Networks

Danilo Pani <sup>1</sup> \*, Paolo Meloni <sup>1</sup> , Giuseppe Tuveri <sup>1</sup> , Francesca Palumbo<sup>2</sup> , Paolo Massobrio<sup>3</sup> and Luigi Raffo<sup>1</sup>

*<sup>1</sup> EOLab - Microelectronics and Bioengineering Lab, Department of Electrical and Electronic Engineering, University of Cagliari, Cagliari, Italy, <sup>2</sup> Information Engineering Unit, PolComIng Department, University of Sassari, Sassari, Italy, <sup>3</sup> Neuroengineering and Bio-nano Technology Lab, Dibris, University of Genova, Genova, Italy*

In the last years, the idea to dynamically interface biological neurons with artificial ones has become more and more urgent. The reason is essentially due to the design of innovative neuroprostheses where biological cell assemblies of the brain can be substituted by artificial ones. For closed-loop experiments with biological neuronal networks interfaced with *in silico* modeled networks, several technological challenges need to be faced, from the low-level interfacing between the living tissue and the computational model to the implementation of the latter in a suitable form for real-time processing. Field programmable gate arrays (FPGAs) can improve flexibility when simple neuronal models are required, obtaining good accuracy, real-time performance, and the possibility to create a hybrid system without any custom hardware, just programming the hardware to achieve the required functionality. In this paper, this possibility is explored presenting a modular and efficient FPGA design of an *in silico* spiking neural network exploiting the Izhikevich model. The proposed system, prototypically implemented on a Xilinx Virtex 6 device, is able to simulate a fully connected network counting up to 1,440 neurons, in real-time, at a sampling rate of 10 kHz, which is reasonable for small to medium scale extra-cellular closed-loop experiments.

#### Edited by:

*Eberhard E. Fetz, University of Washington, USA*

#### Reviewed by:

*Stavros Zanos, University of Washington Tacoma, USA Francesco Leporati, University of Pavia, Italy*

> \*Correspondence: *Danilo Pani pani@diee.unica.it*

#### Specialty section:

*This article was submitted to Neuroprosthetics, a section of the journal Frontiers in Neuroscience*

Received: *09 August 2017* Accepted: *10 February 2017* Published: *28 February 2017*

#### Citation:

*Pani D, Meloni P, Tuveri G, Palumbo F, Massobrio P and Raffo L (2017) An FPGA Platform for Real-Time Simulation of Spiking Neuronal Networks. Front. Neurosci. 11:90. doi: 10.3389/fnins.2017.00090* Keywords: FPGA, fixed-point, neural simulator, closed-loop hardware accelerator, real-time

## 1. INTRODUCTION

In the past decades, spiking neuronal networks (SNN) progressively acquired relevance due to possibility to exploit them in several application scenarios. Typical artificial intelligence applications take advantage of the learning capabilities of SNN for classifiers and autonomous controls. Nevertheless, SNN represent a powerful instrument in neuroscience, allowing to simulate living neuronal assemblies trying to gather from the characteristics of a fitted artificial neuronal network clues on the properties of the living tissue (Bonifazi et al., 2013). From this perspective, it is interesting not only the accurate evaluation of the single neuron behavior but, primarily, the study of the emergent properties of the neuronal assembly dynamics (van Pelt et al., 2004). This can be studied through intracellular recordings (single cell models, by voltage clamp techniques) or extracellular recordings (in vitro cultures of neurons or cortical implants, by Micro-Electrode Arrays MEAs). In this case, the adoption of SNN software simulators (Brette et al., 2007) is widely accepted and the use of such tools is cumbersome only when it is required to simulate very large networks, because of the explosion of the computational cost and, in turn, of the processing time.

This limitation fostered the research toward the development of hardware accelerators able to carry out such simulations in a shorter time (Cheung et al., 2016), possibly comparable to the time scales of a real evolving neuronal network, in order to enable complex simulations otherwise impossible. With the advancements of the technology, it is possible to look at such hardware simulators as enabling tools for new neuroscience experiments. For instance, closed-loop electrophysiological systems are characterized by the tight interaction between the living neuronal tissue and a controlling electronic unit able to interact with it through sensing and stimulation (Le Masson et al., 2002). Compared to open-loop experiments, closed-loop ones enable the study of both the input and output side of the neuronal assembly at the same time (Rolston et al., 2010). The largest part of systems for closed-loop experiments aim to study this aspect by means of an integration between a sensing interface, simple and abstracted computational models and a stimulation interface. Examples of interfacing with muscles for real-time control have been already presented in the literature (Zbrzeski et al., 2016). One aspect, paving the way to the development of neuroprostheses, would be the direct interface of a population of living neurons to an artificial SNN in order to evaluate the capability of the living system to interact with the simulated one (Bonifazi et al., 2013). This poses severe constraints on the hardware implementation of the SNN, particularly for everything concerning the timing, which is no more a matter of performance improvement over a simple PC implementation, but an aspect connected to the feasibility of the neuroprosthetic approach. Examples of the integration of living neurons into artificial SNN can be already found in the literature (Nawrot et al., 2003).

The goal of this paper is to describe and validate a scalable and modular hardware architecture to simulate the dynamics generated by biologically-plausible synthetic neuronal assemblies in real-time. This architecture was completely manually coded and optimized in Verilog Hardware Description Language (HDL) for a single Field Programmable Gate Array (FPGA) chip, thus it can be easily adopted in any lab for small-to-medium SNN sizes. This, along with a fixed-point implementation of the Izhikevich (IZ) neuronal model (Izhikevich, 2003), confers to the architecture a considerable real-time performance up to 0.1 ms, joined to a programmable delay that can be reduced down to such bound. The architecture is parametric in the number of neurons that can be simulated, with limitations imposed by the hardware only in terms of the possibility of fitting the SNN in a commercial FPGA. The long-term goal of the so developed architecture is to bi-directionally interface the SNN with a biological one. Three different experimental scenarios can be envisioned:


stimulation experiments have been already performed (Wagenaar et al., 2005; Wallach et al., 2011), such works have the intrinsically drawback to use "artificial stimulation," i.e., stereotyped stimuli delivered by a controlled stimulator. By means of the proposed architecture, it becomes feasible to deliver stimuli modulated by the intrinsic dynamics.

• In the long term, the real-time SNN could be used as a tool to replace a damaged biological network. In fact, in the last years, researchers started to develop a new family of prostheses applied to the central nervous system (neuroprostheses). As an example, Berger et al. developed a hippocampal prosthesis improving memory function in behaving rats (Berger et al., 2011, 2012).

## 2. MATERIALS AND METHODS

SNNs are more realistic than the conventional neural networks for neuroscientific simulations, taking into consideration not only the neuronal and the synaptic state, but also the concept of time into their operating model. Artificial neurons' firing activity is determined by the evolution of their membrane potential, which follows the model equations. Not only different models produce different firing behaviors, but within the same model it is often possible to tune the parameters so that the artificial neuron is able to reproduce different firing patterns proper of specific cells.

The architecture is currently based on the implementation of the IZ neuronal model (Izhikevich, 2003), which is characterized by an excellent trade-off between computational complexity and biological accuracy, being able to reproduce several spiking patterns by simply tuning its parameters. The proposed version of the architecture implements the model in fixed-point arithmetic, in order to primarily reduce the memory requirements (known to be a limiting factor for hardware SNN) and the latency of the mathematical computations. The architecture is conceived to be a customizable framework, where the neuronal model can be easily replaced while preserving the global structure, and the same holds for the synapses. The main features pursued in the design phase were the low latency, with a real-time timing constraint of 0.1 ms and a programmable delay between firing activity and its reflection on the network activity. In turn, this means that the output sample rate is 10 ksample/s, which is adequate for closed-loop experimental systems.

### 2.1. Izhikevich Spiking Model

The simple model of spiking neurons proposed by Izhikevich (2003) is composed of a two-dimensional system of ordinary differential equations.

$$\frac{d\nu}{dt} = 0.04\nu^2 + 5\nu + 140 - \mu - I \tag{1}$$

$$\frac{du}{dt} = a(b\nu - u)\tag{2}$$

v is the membrane potential of the neuron and it is modeled according to Equation (1), whereas Equation (2) provides the dynamic of u that is the membrane recovery variable. The term I, in Equation (1), is meant to take into account the contribution of the connected nodes to the considered neuron, by means of the sum of the synaptic currents or injected dc-currents. When a spike is fired, meaning v has reached its threshold, the following resetting condition is applied:

$$\nu \ge \nu\_{th} \Rightarrow \begin{cases} \nu = c; \\ u = u + d \end{cases} \tag{3}$$

Both the membrane potential and the membrane recovery value are normally measured in mV. The IZ spiking model is capable to reproduce several different firing patterns, 20 in the original article (e.g., chattering, fast spiking, low-threshold spiking, etc.) but others are being studied, representing the known types of neo-cortical and thalamic neurons, by tuning the a, b, c, and d dimensionless parameters:


#### 2.2. Hardware Spiking Neural Network

In this work, a SNN hardware emulation platform, based on the IZ spiking neuron model, was developed and tested. **Figure 1** depicts an overview of the system. The entire neural network is subdivided in units, which are the building blocks of the SNN. Each unit produces, according to the IZ spiking neuron model, a sub-set of spikes whose occurrence is stored in the binary spike register, which keeps trace of the whole network spiking activity. It is composed of one bit per neuron under emulation. The bit corresponding to a neuron is set high when the neuron has fired a spike. The chosen simulation paradigm is synchronous (or clockdriven), meaning that all neurons are updated at every simulation step, regardless of the spiking activity (Brette et al., 2007). From a macroscopic point of view, each simulation cycle is composed as follows:


Additional details, regarding what happens within the units in each simulation step, are provided hereafter in Section 2.2.2. It is worth to notice now that the architecture gives the possibility to set at design time a delay for the spike propagation, exploiting a register chain strategy. The firing activity is propagated back to the units with a delay of several (up to ten) emulation cycles, to mimic a physiological delay of up to 1 ms between firing and its reflection on the network status.

**Figure 1** depicts an exemplary instance of the proposed platform, parameterized in order to emulate a network of 256 neurons only for the sake of clarity. The platform instance integrates four units, emulating 64 IZ neurons each. According to the signal activity of all the 256 neurons, each unit generates up to 64 spikes per simulation cycle. The four different 64-bit signals, representing each unit's neuronal activity, are concatenated to create a single 256-bit signal and fed back to the spike register.

#### 2.2.1. Units: Architectural Overview

The units are responsible for the implementation of Equations (1) and (2) and for estimating the firing activity of the subset of neurons assigned to each of them. Equations (1) and (2) represent two derivatives and they have been implemented in hardware exploiting the finite-difference method, which is a numerical method for solving differential equations by approximating them with finite differences, as specified hereafter:

$$\frac{\nu(k+1) - \nu(k)}{h} = 0.04\nu(k)^2 + 5\nu(k) + 140 - \mu(k) - I \tag{4}$$

$$\frac{u(k+1) - u(k)}{h} = a(b\nu(k) - u(k))\tag{5}$$

The smaller is h (i.e., the time interval between k and k + 1) the better Equations (4) and (5), respectively, approximate Equations (1) and (2). In the current implementation, h is fixed to 0.1 ms, which turned out to be an excellent compromise between the approximation quality and the overall processing time required to update the status of the entire SNN. Furthermore, such a delta-cycle is compatible with hybrid closed-loop experiments.

Each unit, as depicted on the right hand side of **Figure 1**, is composed of a Neuron Section and of a Synapses Section, which are, respectively, responsible for computing the overall neuronal activity of the considered unit and for determining, for each neuron, the synaptic current (I) that has to be subtracted in Equation (4).

**Figure 2** presents an overview of the Neuron Section of each single unit block. Equations (4) and (5) are physically computed by the Izhikevich block, which receives v(k) and u(k) values from the u-v RAM block, a, b, c, and d parameters from the parameter RAM block, and finally I from the Synapses Section. The interactions with the u-v RAM and parameter RAM blocks, and the synchronization with the Synapses Section are controlled by a dedicated Finite State Machine (FSM), the neuron FSM. The neuron FSM also masters the execution of the different phases of the Izhikevich block itself. In terms of library IP module, the latter includes only a multicycle multiplier, which is re-used for all the

multiplications in Equations (4) and (5); all the other modules are HDL-coded. In this way, it is possible to save as much hardware resources as possible (by re-using the same processing element rather than instantiating several of them in parallel) and to maintain the operating frequency high (by pipelining the operations and, in turn, breaking down the critical path). The Izhikevich block processes one neuron at a time; its output is written on the out\_spike\_reg module, which contains the spikes of all the neurons within the given unit. The output of the Neuron Section is the updated unit's contribution to the firing activity of the SNN, to be sent to the concatenation logic.

The Synapses Section is depicted in the block diagram in **Figure 3**. The figure shows how the synaptic current is computed on the basis of the overall firing activity (i.e., the content of the spike register). It computes the weighted sum of the contribution of the pre-synaptic neurons, connected to the processed one by means of the synapses. Only the synaptic weights associated to those neurons that fired (having a 1 in the corresponding bit of the spike register) are eventually accumulated. The synaptic weights are provided by the weight RAM block. The contribution of each connected neuron is computed by a set of synapses blocks, placed in parallel to speed-up the computation since all the incoming neuronal activity has to be scanned to determine the synaptic weights associated to the pre-synaptic neurons that fired and, accordingly, retrieve and accumulate them. As an example, the Synapse section of the platform instance in **Figure 1** integrates 2 synapse blocks per unit, which means that each synapse block calculates the contribution given by 128 neurons. The outputs of the of synapses blocks are summedup by an adder-tree module to produce I. The interactions with the weight RAM block and the synchronization with the Neuron Section block is controlled by a dedicated FSM, the synapses-FSM.

#### 2.2.2. Units: Execution Flow

The execution flow of a unit within each simulation cycle can be described as follows:


All these steps are iterated for all the neurons within the unit to evaluate and update the status of the SNN. The neuron\_FSM is responsible of verifying whether all the neurons have been processed or not and to notify that the content of the out\_spike\_reg module is the complete updated unit contribution to the neuronal activity that can be sent to the concatenation logic. Depending on the initialization data stored within u-v RAM, parameter RAM and weight RAM blocks, the platform is thus capable to fully emulate an arbitrary IZ-based SNN.

A fixed-point arithmetic was adopted in order to pursue a fast execution and a compact hardware implementation. A Q3.4 format for the weights (meaning that a Q format is adopted, with the decimal point virtually placed in order to leave 4 bits for the fractional portion over a 7-bit word) was chosen. Such a solution limits the dynamic ranges of the weights with minor effects, since the low values are usually swamped by the thalamic noise term (Thomas and Luk, 2009). A more conservative approach based on the worst-case design, which is unrealistic in this kind of applications, would limit the SNN weights in the [−1,1) range. Multiple iterations with NEST have been performed in order to define a format for the weights able to reduce the memory requirements while preserving an acceptable accuracy. In order to avoid intermediate overflows, the synapse accumulators use 8 bits for the input and 32 bits for the outputs, and they are connected to a 32-bit in 32-bit out adder tree. Both of them are implemented using DSP48 macros for Xilinx FPGAs. In order to be able to represent the IZ parameters with enough flexibility to accommodate typical models and novel ones, a Q10.22 was selected for them. The internal computations rely on pre-computed constants, full-precision adders and multi-cycle multipliers with 32-bit in and 32-bit out.

The communication between the emulation platform and the external environment is required, at start-up, to allow the pre-loading of neuron parameters and synaptic weights for the target experiment inside the system memory blocks. Moreover, emulation results must be sent to the external environment, respecting real-time constraints, during the whole emulation time. Communication interface can be implemented in different ways, exploiting the connectivity in modern FPGA boards and the support offered by design tools and programming environment. Two different communication methods were tested. Firstly, the exploitation of a host general-purpose processor, implemented as a Microblaze soft-core on the FPGA, was evaluated. The processor reads system memories as part of its memory map and communicates with the external environment using an Ethernet connection or a serial UART connection. This solution is easy to implement and provides comfortable coupling of the processor with the emulator, but presents the disadvantage to occupy resources on the reconfigurable device, which may be exploited to emulate more neurons. A second solution is available using FPGA families that embed hard-wired processing cores, such as the Xilinx Zynq family, which includes chips integrating programmable logic and an ARM dual-core processor. In this case, an interface between the emulator, implemented on the programmable logic, and the ARM sub-system, serving as a host processor and providing adequate connectivity with the external environment, was developed. A master IP that loads ad stores data on the DDR memory connected to the ARM when needed, exploiting a set of 4 AXI-based interfaces natively available in the system, was mapped on the FPGA. Each of the AXI interfaces provides a bandwidth of 64 bits/cycle and can be clocked at more than 100 MHz, thus is sufficient to sustain output of emulation results in real time.

However, it must be noticed that the best interface implementation depends on the target FPGA device and on the target use-case of the emulation infrastructure.

### 2.3. Platform Validation Approach

#### 2.3.1. Architectural Performance Evaluation Metrics

The main performance evaluation metrics for the FPGA-based hardware implementation are timing and resource utilization. In the proposed parametric architecture, both depend on the chosen configuration of the architectural parameters, i.e., the number of units, the number of synapse modules inside each unit and the precision of the weights. The main limiting factors for the proposed implementation are the DSP modules and the RAM modules. In particular:


Obviously, the amount of DSP48E1 and RAMB36E1 needed to implement a given configuration is dependent on the architectural parameters. Performance will be evaluated considering the maximum number of neurons that can be emulated in real-time by the platform. This metric is impacted by several factors:


In order to provide an overview of the involved dependencies, a model, described in Section 3.1.1, was developed to enable a prospective user to estimate the achievable performance level for different architectural configurations implemented on different FPGA devices.

#### 2.3.2. Accuracy Evaluation Tests

To evaluate the accuracy of the simulations performed using the hardware SNN, a sister-pool of simulations with NEST (Gewaltig and Diesmann, 2007) was performed. In this way, it was possible to compare the so generated dynamics of the in silico (implemented in NEST) and hardware SNN, by means of well-known statistics commonly used to analyze the electrophysiological activity of large-scale neuronal networks coupled to Micro-Electrode Arrays (MEAs). For this purpose, an heterogeneous 1,024-neuron SNN with 768 excitatory neurons and 256 inhibitory ones was simulated. Compared to the maximum number of synthesizable neurons, which is 1,440, this number represents the highest power of two and was chosen in order to simplify the scripting operations required, for the time being, to load the parameters and analyze the results. The DC input currents are 4 pA and 2 pA for excitatory and inhibitory neurons respectively; neurons are randomly connected, according to the generation model in Izhikevich (2003); inhibitory neurons have stronger synaptic connections. In order to achieve heterogeneity (i.e., to model all exctitatory and inhibitory neurons), excitatory cells are generated by assigning (ai , bi) = (0.02, 0.2) and (c<sup>i</sup> , di) = (−65, 8) + (15, −6)r 2 i , where ri is a random variable uniformly distributed on the interval [0,1] and i is the neuron index; similarly, each inhibitory cell has (a1, bi) = (0.02, 0.25) + (0.08, −0.05)r 2 i and (c<sup>i</sup> , di) = (−65, 2).

To characterize the spiking activity, the mean firing rate (MFR) of the network and the inter spike interval distribution (ISI) of the excitatory and inhibitory neurons were evaluated. The bursting activity was characterized by means of the mean bursting rate (MBR), burst duration (BD), and inter-burst interval (IBI). Bursts have been detected by using the algorithm devised in Chiappalone et al. (2005). Detected bursts are sequences of spikes having an ISI smaller than a reference value (set at 100 ms in our simulations), and containing at least a minimum number of consecutive spikes (set at 4 spikes in our simulations). Briefly, MFR and MBR are the number of detected spikes per second and bursts per minute falling in a temporal window equal to the duration of the simulation. The ISI and IBI distributions are the probability density functions of time intervals between consecutive spikes and bursts, respectively (Dayan and Abbott, 2001). Finally, BD is the duration of the detected bursts.

In addition to the aforementioned statistics, the spike jitter between in silico and hardware simulations was also computed. Practically speaking, by considering as reference the spike timing of the NEST simulations, the temporal distance of the correspondent spikes generated by the hardware SNN was computed.

The evolution of the membrane potential of a single neuron in the two simulations were also compared. Five experiments were performed, in order to analyze the dynamics of the u and v potentials of a modeled neuron respectively belonging to three classes of excitatory neurons (regular spiking, intrinsically bursting and chattering), and two classes of inhibitory neurons (fast spiking and low-threshold spiking). The a, b, c and d parameters belonging to each of the emulated cortical cells are reported in **Table 1**; the DC input current, for all considered experiments, is 4 pA. The behavior of the emulated potentials with the potential evolution obtained by means of a NEST simulation were also compared.

#### TABLE 1 | Parameters of emulated cortical cells.


#### 3. RESULTS

In this section, the results have been organized in order to discuss at first those related to the architectural performance evaluation and then those related to the accuracy evaluation.

### 3.1. Architectural Performance Evaluation Results

As previously mentioned, the assessment of the quality of the proposed architecture has to consider two main factors: resource utilization and emulation performance. In the following, a description of the timing and utilization figures that can be obtained changing the architectural parameters is presented. At first, the timing characteristics of the architectures modules and their dependence on the selected architectural parameters have to be studied, within the real-time constraints imposed by the emulation. Then, the optimal parameter values have to be selected, in the range actually allowed by the target FPGA device. All the presented results were verified after synthesis and implementation.

#### 3.1.1. Timing Characteristics of the Architecture Modules

The real-time constraint to be considered defines how often the SNN output has to be evaluated. Such a metric, in emulation/simulation, is usually referred to as delta cycle. A delta cycle of 0.1 ms was chosen, aiming to be aligned with an acceptable sampling frequency in the scope of acquisition of signals from neuronal cultures with MEAs. The same time step was also used for the differential equations discretization, hereafter called Tsample.

As mentioned, hardware structures are reused to emulate more neurons, in the considered interval. Emulation of one neuron occupies the set of hardware resources in a unit for a determined number of cycles, that will be indicated in the following as Temu. In order to relate Temu with the realtime constraint, actual clock period, which will be indicated as Tclock in the following, used as synchronization reference within the architecture, shall be considered. In synchronous digital systems, the minimum clock period that can be chosen by the designer is related to the propagation delay of gates implementing combinational paths through the design. In the design of the proposed architecture, a pipeline strategy that allows the minimum clock period to be independent from the architectural configuration was adopted. After the implementation on the FPGA device, the optimal value of Tclock

can be evaluated to be 10 ns, corresponding to a maximum working frequency of 100 MHz. It was proven that this frequency can be substained for all the configurations that may fit in midto-high end FPGA devices, confirming the scalability of the proposed architecture.

Thus, the number of cycles available for emulation in a sampling interval is Tsample/Tclock = 10, 000 cycles. Then, each unit can emulate Nneu neurons in one sampling interval, where

$$N\_{neu} = 10,000/T\_{emu} \tag{6}$$

Temu is a function of the number of synapse modules Nsyn and units Nunits in the system. The number of cycles needed to emulate one neuron is the sum of two contributions:

$$T\_{emu} = \log\_2(N\_{\rm syn}) + \frac{N\_{\rm neu} \ast N\_{\rm units}}{2 \ast N\_{\rm sym}}\tag{7}$$

Equation (7) was constructed on the basis of the architectural details of the hardware modules and its validity was verified in HDL-level simulation and after actual implementation. The first contribution is an offset related to the pipeline stages in the adder tree connecting the output of the synapse modules. The second contribution is the actual time needed to perform all the accumulation routine that calculates the synaptic current. In the second term the numerator represents the total number of neurons constituting the emulated network, the denominator takes into account that the workload related with the accumulation is divided between the synapse modules in the unit, each one performing two add operations per cycle.

Combining Equations (6) and (7), Nneu can be calculated solving a quadratic equation, on the basis of the architectural parameters Nsyn and units Nunits. Eventually, the total number of neurons that can be emulated is Nunits × Nneu.

**Figure 4** shows how such numbers changes varying Nunits for two different Nsyn values. The selection of the values for such parameters is obviously bounded by the amount of resources available on the target FPGA device, as it will be described in more detail in the next section.

Even though the proposed architecture was not conceived as an hardware accelerator but, rather, a real-time simulation platform enabling closed-loop neurophysiological experiments, a rough performance comparison against NEST can be presented. When running on a PC platform (Ubuntu 14.04 LTS, kernel 3.13.0-96-generic, CPU Intel(R) Core(TM) i7-2600 CPU @ 3.40 GHz, 16 GB RAM), the emulation of the SNN with 1,440 neurons requires 6.3 s per second of simulation, which is far from the real-time achieved with the proposed architecture. Nevertheless, caution should be used when considering such numbers for a fair comparison. In fact, on one hand, the architecture is clocked in order to provide the real-time performance at 10 kHz, so the actual processing time is masked by the (higher) wall time, which is 0.1 ms. The same does not hold for a PC-based emulation, not constrained by such a wall time. On the other end, different programming styles, programming language and

processor architectures can lead to very different results, so the performance of PC-based solutions could seem unfairly poor.

#### 3.1.2. Hardware Resources Utilization

In order to select the correct architectural parameters, it is mandatory to understand their impact on the utilization of hardware resources in the target device.

When the architectural configuration is known, it is very easy to estimate the number of required DSP48E1 macros. Each unit uses 3 macros for each synapse module and 3 macros to implement the hardware emulating the neuronal dynamic. The amount of RAM macros depends on the total number of neurons to be emulated. The architecture should embed enough storage to contain all the weights determining the contribution of each pre-synaptic neuron to the post-synaptic ones. Such neurons are prospectively disjointed, thus the utilization of resources roughly has a quadratic dependence on the size of the emulated network. Some memory resources are also needed to store neurons' parameters and evolving values of u and v.

Considering the number of resources in commercial FPGA devices, for typical network configurations, the limiting factor is very often the availability of Block RAM (BRAM). This can sometimes limit the maximum number of neurons under emulation with respect to the possibilities offered by a given parameterization of the unit hardware modules.

In order to provide an estimation of the architecture configurations that may be implemented on mid-range commercial devices, the hardware-related features of a configuration featuring 8 units, each one embedding 16 synapse modules, realized on a XC6VLX240T Xilinx device, are presented. **Table 2** represents the device utilization summary.

This configuration occupies 94% of the BRAM resources and 53% of the DSP48 macros, and it is capable of emulating 1,440 neurons in real time.

Although power consumption analysis has been considered as a secondary development objective in this work, it is worth to provide some hints about the power-related features of the proposed architecture. Obviously, the actual power dissipation

TABLE 2 | Summary of the resource utilization.


*Target FPGA platform is a XC6VLX240T Xilinx device. The implemented architecture features 8 units with 16 synapses, capable of emulating 1,440 neurons.*

depends on the specific FPGA platform selected as target. However, for every FPGA device considered in the developed experiments, a significant part of the power consumption is related to the usually called quiescent power, that is the power dissipation of the idle FPGA, before its actual programming. Moreover, recent all-programmable FPGA-based SoCs as Xilinx Zynq devices, present an additional contribution to the power consumption due to the host processing system implemented on the chip. Finally, in general FPGA chips are mounted on a development board including several peripherals, that add a further contribution unrelated with the emulation. Thus, the power consumption of the overall emulation platform is weakly dependent on the number of emulated neurons. As an example, in the presented experiments, a Xilinx ZC706 evaluation board featuring a XC7Z045 FPGA chip dissipates 6.8 W in the idle state and 8.5 W when executing emulation with a 100 MHz clock on the FPGA.

#### 3.2. Accuracy Evaluation Results

#### 3.2.1. Single Neuron Membrane Potential Evolution

From the single neuron simulation, it is possible to find a good adherence between the membrane potential evolution as obtained with the proposed architecture and with NEST. This is clearly visible in **Figures 5**, **6** respectively for low-threshold spiking and fast spiking neurons.

#### 3.2.2. Fully-Connected Network of N Neurons

By using the NEST simulations as reference, the electrophysiological patterns of activity generated by the hardware implementation of the network model were compared. The raster plot of **Figure 7A** shows the spike timing of the 1024 neurons of the network. The first neurons (id: from 1 to 768) are excitatory whereas the others inhibitory (id: from 769 to 1024). Blue circles and red crosses are representative for the two simulation approaches (i.e., software, blue circles, and hardware red crosses, respectively). The zoom of **Figure 7A** shows a good overlap of the spike timing. In order to quantify such a jitter, the histogram (bin width equal to 0.3 ms) of the occurrences relative to the excitatory (**Figure 7B**) and inhibitory neurons (**Figure 7C**) was plotted. Both the neuronal populations display significant jitters less than 2.0 ms in 95% of the occurrences, indicating good performances of the hardware implementation of the network model.

In terms of number of spikes, both the software and hardware models present the same number of spikes, as the plot of the MFR displays (**Figure 8**). MFR values are 1.05 ± 0.48 spikes/s

in the NEST implementation of the model and 1.05 ± 0.49 spikes/s in the hardware one. Finally, the ISI distributions (**Figure 8B**, relative to the software, and **Figure 8C**, relative to the hardware model) by splitting the contribution of the excitatory (red line) and inhibitory (black line) populations, were evaluated. This analysis shows a good agreement of the two model implementations too. Qualitatively, the shape of the ISI distributions is similar, as well as the temporal position of the peaks of the curves (hardware: 2.54 ms vs. software: 2.50 ms).

The validation of the hardware network model was carried out by comparing the bursting dynamics (**Figure 9**). **Figures 9A–C** compare the values of MBR, BD, and IBI of the software and hardware network models, respectively. Although slight differences can be appreciated, such differences are not

FIGURE 7 | Network dynamics characterization: comparison between in silico and hardware simulations. (A) 2 s of electrophysiological activity. Blue circles and red crosses are representative for software and hardware approaches, respectively. (B,C) histograms of the spike jitter between software and hardware simulations evaluated for excitatory (B) and inhibitory (C) neurons of the network.

statistically significant (p >0.05, Mann-Whitney, non-parametric test). The Mann-Whitney U-test was chosen since the analysed data do not follow a normal distribution, as revealed by means of the Kolmogorov-Smirnov normality test applied to them. The chosen p-value is assumed to be adequate for the considered problem. Similar considerations can be done for the IBI distributions (**Figures 9B,C**).

### 4. DISCUSSION

Several SNN hardware accelerators have been proposed so far in the scientific literature (Maguire et al., 2007). They usually try to overcome the performance limitations of purely software simulators (Brette et al., 2007) such as NEURON (Carnevale and Hines, 2006), NEST (Gewaltig and Diesmann, 2007), BRIAN (Goodman and Brette, 2009), etc., widely accepted as research tools in the community of computational neuroscience. However, unless an explicitly parallel processing is pursued on large multiprocessors high performance computing platforms, such simulators suffer the intrinsic scalability limits of the underlying object of the simulation, becoming soon too slow for largescale networks of biologically plausible neuronal models (Rast et al., 2010). In this section, a discussion about some relevant architectures for SNN simulations in hardware is presented, along with a comparison with the proposed approach. Among the different architectures cited hereafter, those aimed to simulate a large number of neurons have been grouped in **Table 3**, reporting the main relevant data gathered from the cited sources. Without being exhaustive, this table enables a quick overview of the present state of the art.

Despite the purely software solutions present the aforementioned limitations, it is obvious that they usually pursue simplicity (in the creation and simulation of the model), precision (typically double) and flexibility (the

TABLE 3 | Summary of some relevant state-of-the-art architectures for SNN hardware simulation.


*For some architectures flexible enough to implement several neuronal models, only one published result is reported.*

possibility to change topology, parameters, synaptic model, neuron model, etc.). For this reason, when moving toward the hardware simulation systems, it is obvious the success of architectures exploiting efficient signal processing cores, such as ParSPIKE (Wolff et al., 1999), which is based on the Analog Devices ADSP21060 Digital Signal Processor. In fact, Digital Signal Processors revealed better performance than highend mainstream processors in several biomedical and signal processing applications, with a power consumption that could be even two orders of magnitude lower (Pani et al., 2013, 2014) and they are currently being used for studies in neuroprosthetics (Pani et al., 2011, 2016).

A very successful implementation of a hardware architecture for SNN based on general purpose (embedded) processors is SpiNNaker (Furber et al., 2014). It is a multilevel tiled architecture, i.e., an architecture composed, at different levels, of a regular mesh of computing elements called tiles, mixing the flexibility of a software implementation of the neuronal model with the performance of a custom architecture (at macroscale). The smallest tile is a node, i.e., a custom chip consisting of multiple (up to 18 in the latest versions) ARM968 processors clocked at 200 MHz, without embedded floating-point units and exploiting a network-on-chip infrastructure for the communications. These nodes are assembled in boards comprising 48 of them, and exploiting highly customized selftimed connections. These boards communicate each other through custom serial links implemented on FPGA. This architecture is being used in the human brain project (www. humanbrainproject.eu). Despite the impressive work behind this platform, it is neither suited for a neuroengineering lab with limited budget nor for closed-loop applications. Furthermore, compared to our design, each core is able to model up to a few hundred neurons (LIF or IZ) with about one thousand input synapses each. The time resolution scales down to 1 ms only, which is inadequate for interfacing with the living tissue. Recent completely asynchronous extensions of the software framework on the SpiNNaker platform allowed performing simulations with networks implementing sub-millisecond tasks, such as sound localization, (Lagorce et al., 2015). However, some specific conditions apply, as the use of even-driven LIF neurons, the adoption of a dendritic delay core for every particular delay value (leveraging the large number of cores available), etc.

The same holds for the NeuroFlow architecture, which for sure represents the state of the art in the field of FPGA architectures for SNN (Cheung et al., 2016). Compared to it, the proposed architecture targets a real-time performance of 0.1 ms, which can be ever reduced by changing the size of the network or speculating on the connectivity (assumed to be full in our tests). In particular, the time step of 0.1 ms is one order of magnitude less then that of the NeuroFlow architecture. Furthermore, the impressive numbers of simulated neurons provided by NeuroFlow (up to 600,000 units) can be reached with 6 FPGAs, with a toroidal network configuration, limiting the number of synapses to 1,000–10,000, when the connection probability follows a Gaussian probability of the synaptic distance with standard deviation ranging from 32 to 512. This unfortunately makes a point comparison hard and potentially unfair. Overall, NeuroFlow targets larger networks than the proposed architecture, with a different aim which is closer to that of SpiNNaker. This is reflected by the choice of a large off-chip Dynamic Random Access Memory (DRAM) compared to the BRAM used in our and other designs, and by the higher flexibility in the simulation setup.

On the other end of the flexibility axis, it is possible to find neural architectures based on application-specific integrated circuits (ASIC). They usually range from the analog neuromorphic chips (Hofstoetter et al., 2005), which exploit the possibility to make the transistors work on the current flow as the ion channels do on the ions flow, to the custom VLSI digital neurocomputers (Van Sickle and Abdel-Aty-Zohdy, 2009). Typically, the former are more complex to design but achieve better performance than the latter, both in terms of silicon area and power consumption (Joubert et al., 2012). In order to take the best in terms of performance while preserving the flexibility of software solutions, programmable hardware progressively gained interest in the computational neuroscience community. If field programmable analog arrays are still not completely convincing in terms of performance, even though some reconfigurable analog VLSI neuromorphic chips exist (Yu et al., 2010), on the digital side there is a growing interest toward the use of FPGA for these purposes (Maguire et al., 2007). FPGAs, providing the user the possibility to reconfigure the device by full or partial reload of the configuration bitstream, present the advantages of a custom architectures (as for the ASIC) and a flexibility approaching that of software implementations. Furthermore, the presence of IP cores enables the creation of multiprocessors systems on chip even on FPGA (Glackin et al., 2005).

In the largest part of cases, the effort toward the development of very fast architectures had a negative impact on the biological plausibility of the adopted neuronal model. In fact, the neuronal models that have been presented in the scientific literature so far are characterized by different biological plausibility and computational complexity (Paugam-Moisy and Bohte, 2012). Despite even the most complex Hodgkin-Huxley model was implemented in hardware (Graas et al., 2004), not all of them are suited for medium to large scale SNNs on digital hardware, because they should be computationally simple and at the same time capable of representing the wide variety of firing patterns exhibited by the different biological neurons. For this reason, some of the architectures use simplified custom neuronal models (Upegui et al., 2003, 2005), much more use integrate-and-fire (I&F) (Glackin et al., 2005) or leaky-integrateand-fire (LIF) (Cassidy et al., 2007; Pearson et al., 2007). A recent interesting investigation on the limits of computer based approaches, FPGAs and Graphics Processing Units (GPUs) on highly complex biologically-plausible models of neurons belonging to the cerebellar cortex was reported in Florimbi et al. (2016). This work, in the main framework of the Human Brain Project, remarks how such complex neuronal models require huge hardware resources so that single-chip FPGAs cannot be an effective platform for cell networks, even though ASICs could, whereas GPUs can provide speedups that are still far from the real-time bound.

The proposed architecture exploits the very efficient IZ model (Izhikevich, 2003). Other works at the state of the art implemented the same neuronal model. Usually, for computational complexity reasons, the fixed-point processing is preferred. Despite this approach obviously limits the precision of the operations, compared to a floating-point solution, it has been shown to be adequate for several practical applications. First thing to notice is that several papers in the past described the implementation of the IZ model on FPGA without embedding it into a SNN. For instance, in Cassidy and Andreou (2008) a hardware implementation on the neuronal model alone is described (the model equations were changed in order to exploit power-of-two arithmetic, leading to less precision). The absence of the synapses is a remarkable limitation because of the quadratic dependence from the number of neurons of the synaptic weights, which is the main issue for scalability.

Other works, such as Rice et al. (2009), even though introducing the synapses, present topologies such that synapses connectivity is a minor issue. In that case, for instance, the number of neurons in very high (about 96 × 96, as much as the pixels of the input images) but the synapses are not as much as the square of such a value but rather only 48 times it. In that case, a Q4.12 format was used for the parameters, with the weights represented in Q4.12 format. The format depends on a trial and error procedure. SpiNNaker, for instance, uses different scaling factors for different parameters and values, showing that the best results can be achieved with a Q8.8 format for u, v, c, and d, whereas the Q0.16 format was chosen for parameter a and ab (since b alone is not used in that implementation), limiting such parameters to be <1 (Jin et al., 2008). With such an approach, the architecture is able to simulate up to 1000 IZ neurons on a single fascicle in a network with a low connectivity level (10%). Nevertheless, such a low connectivity is unrealistic: in Thomas and Luk (2009) a connectivity with 1,000 synapses per neuron, claimed to be a common-sense choice, is simulated. In this case, the fixed point representation used for the weight and the arithmetic of the adder tree is used, due to the limits imposed by the memory limits of the chosen platform, and weights are limited to 9 bits. Our architecture, with its 7 bits for the weights with a Q3.4 format, follows a similar approach, considering that on a physical platform it is acceptable to fix limits to the range of such parameters (Thomas and Luk, 2009). Spikes accumulation is performed at 32 bits to preserve as much as possible the precision. In Ambroise et al. (2013), the authors present a similar approach, that is capable of emulating up to 167 neurons (it is worth to notice that these results have been achieved on a smaller device: no data is provided on large FPGAs). Compared to the proposed

one, such a work uses a higher number of resources due to higher data precision and to a fairly more complex processing of the synaptic current. It also considers a reduced sampling frequency (1 kHz) with respect to our work. In Glackin et al. (2005), the I&F model is implemented in fixed point with a Q8.10 precision for the membrane voltage and 12-bit precision for the synaptic conductance (no further details on the data size), using powers of 2 for the scaling parameters in the model, to avoid multipliers and dividers, with some precision loss.

In the proposed work, precision of the computation is demonstrated by the achieved results. The architecture is capable to obtain the same firing patterns of NEST, with a real-time performance that reaches 0.1 ms. The slight differences cannot be considered a limiting factor for the exploitation of the architecture in real-time closed-loop experiments, since the global firing patterns are respected. Even though the proposed architecture is not highly optimized, because of the need to ensure flexibility at this development stage, it is possible to pursue energy/performance efficiency by means of fine applicationdriven customization of the hardware architecture, that requires adequate support by advanced design tools (Jozwiak et al., 2012, 2013). The proposed approach can evolve in such a direction for closed-loop implantable implementations.

In perspective, the possibility to have a valid and efficient hardware tool to simulate and generate in real-time realistic spiking dynamics could pave the way to the design of new devices to interface synthetic neuronal assemblies to biological excitable tissues. Indeed, the so developed architecture could be used to generate realistic signals (in terms of time and spatial constants) to stimulate biological networks (open-loop application) as well as to realize closed-loop systems in which, in a bi-directional way, biological and hardware networks are mutually stimulated. In such scenarios, similar to state-of-the-art

REFERENCES


closed-loop experiments (Wagenaar et al., 2005; Wallach et al., 2011), the availability of an embedded system implementing in hardware (e.g., FPGA) a biologically plausible SNN would be the only enabling technology. In fact, purely software simulations could not be used to interface in silico neuronal models with living beings.

In the meanwhile, the real-time performance of an FPGA platform as the one proposed in this work, overcoming the limitations of the software simulators, can be exploited to study the fundamentals of the interaction between living neuronal assemblies and synthetic ones, in closed-loop, opening to hitherto unexplored neurophysiological experiments.

#### AUTHOR CONTRIBUTIONS

All the authors substantially contributed to the manuscript. In particular, PMa, DP, and LR conceived the study, PMa and DP defined the methods and analyzed the performance results, PMe and FP defined the architectural features and coding approach, GT performed the hardware parameterization, debug and cosimulations HDL-NEST. All the authors drafted and revised the manuscript and agree with the current version.

#### FUNDING

The research leading to these results has received funding by the Region of Sardinia in the ELoRA project (Fundamental Research Programme, LR 7/2007, grant agreement CRP-60544).

#### ACKNOWLEDGMENTS

The authors wish to thank Mr. R. Montano for his precious work in coding a preliminary version of the architecture.


system," in Proceedings of the 7th ACM International Conference on Computing Frontiers (Bertinoro: ACM), 21–30.


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

The reviewer SZ and handling Editor declared their shared affiliation, and the handling Editor states that the process nevertheless met the standards of a fair and objective review.

Copyright © 2017 Pani, Meloni, Tuveri, Palumbo, Massobrio and Raffo. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# A Bidirectional Brain-Machine Interface Featuring a Neuromorphic Hardware Decoder

Fabio Boi <sup>1</sup> \* † , Timoleon Moraitis 2 †, Vito De Feo1 †, Francesco Diotalevi <sup>3</sup> , Chiara Bartolozzi <sup>4</sup> , Giacomo Indiveri <sup>2</sup> and Alessandro Vato<sup>1</sup>

<sup>1</sup> Neural Computation Laboratory, Istituto Italiano di Tecnologia, Rovereto, Italy, <sup>2</sup> Institute of Neuroinformatics, University of Zurich and ETH Zurich, Zurich, Switzerland, <sup>3</sup> Robotics, Brain and Cognitive Sciences, Istituto Italiano di Tecnologia, Genova, Italy, <sup>4</sup> iCub Facility, Istituto Italiano di Tecnologia, Genova, Italy

Bidirectional brain-machine interfaces (BMIs) establish a two-way direct communication link between the brain and the external world. A decoder translates recorded neural activity into motor commands and an encoder delivers sensory information collected from the environment directly to the brain creating a closed-loop system. These two modules are typically integrated in bulky external devices. However, the clinical support of patients with severe motor and sensory deficits requires compact, low-power, and fully implantable systems that can decode neural signals to control external devices. As a first step toward this goal, we developed a modular bidirectional BMI setup that uses a compact neuromorphic processor as a decoder. On this chip we implemented a network of spiking neurons built using its ultra-low-power mixed-signal analog/digital circuits. On-chip on-line spike-timing-dependent plasticity synapse circuits enabled the network to learn to decode neural signals recorded from the brain into motor outputs controlling the movements of an external device. The modularity of the BMI allowed us to tune the individual components of the setup without modifying the whole system. In this paper, we present the features of this modular BMI and describe how we configured the network of spiking neuron circuits to implement the decoder and to coordinate it with the encoder in an experimental BMI paradigm that connects bidirectionally the brain of an anesthetized rat with an external object. We show that the chip learned the decoding task correctly, allowing the interfaced brain to control the object's trajectories robustly. Based on our demonstration, we propose that neuromorphic technology is mature enough for the development of BMI modules that are sufficiently low-power and compact, while being highly computationally powerful and adaptive.

Keywords: bidirectional BMI, neuromorphic decoder, on-line learning, modular system, spiking neural network

### 1. INTRODUCTION

The possibility of controlling a prosthetic device through a direct interface with the central nervous system represents a promising solution for restoring sensory-motor functionalities in patients with limb amputations or peripheral and neurological deficits due to spinal cord injury, amyotrophic lateral sclerosis, or stroke. In the last two decades, a fast-growing worldwide scientific community has developed several brain-machine or brain-computer interfaces (respectively, BMIs or BCIs)

#### Edited by:

Timothée Levi, University of Bordeaux 1, France

#### Reviewed by:

Christoph Richter, Technische Universität München, Germany Mikhail Lebedev, Duke University, USA

#### \*Correspondence:

Fabio Boi fabio.boi@iit.it † These authors have contributed equally to this work.

#### Specialty section:

This article was submitted to Neuroprosthetics, a section of the journal Frontiers in Neuroscience

Received: 24 June 2016 Accepted: 22 November 2016 Published: 09 December 2016

#### Citation:

Boi F, Moraitis T, De Feo V, Diotalevi F, Bartolozzi C, Indiveri G and Vato A (2016) A Bidirectional Brain-Machine Interface Featuring a Neuromorphic Hardware Decoder. Front. Neurosci. 10:563. doi: 10.3389/fnins.2016.00563 toward the clinical application of these devices. Such interfaces offer also a powerful tool for exploring the sensory-motor mechanisms of control, adaptation, and learning that are employed by the central nervous system. This research has been assisted both by progress in our understanding of the underlying neural processes that take place in the brain, and by technological advances that have dramatically improved the quality of the signals recorded from the brain and the possibility of managing and processing large amount of data in real-time (Wolpaw et al., 2000; Lebedev and Nicolelis, 2006; Wander and Rao, 2014). Encouraging results have been recently obtained in controlling a robotic arm by using motor neural activity in tetraplegic patients (Hochberg et al., 2012) and by restoring cortical control of movement in humans with quadriplegia (Bouton et al., 2016) but these setups still have limitations that prevent their clinical use on a large scale (Baranauskas, 2014).

The development of a BMI system aiming for large clinical application requires crucial improvements of the hardware and software components. The hardware components need to be (a) fully implantable for long term use and therefore miniaturizable; (b) able to reliably process neural signals with a limited power budget; (c) powerful enough to implement non-trivial computational tasks involved in a BMI system. Additionally, the decoding algorithms need to be (d) sufficiently flexible to be implemented with different types of hardware components, and (e) able to dynamically adapt to changes in the neural activity due to the interaction with the artificial device (Dangi et al., 2011; Orsborn et al., 2014).

Neuromorphic devices comprise compact, energy-efficient, and adaptive circuits that have been demonstrated to be optimal for tasks that involve learning from real-world observations in an on-line fashion (Chicca et al., 2014). They achieve this by employing silicon emulations of biological neurons and synapses that can be physically configured to implement algorithms inspired by the asynchronous massively parallel computations performed in biological neural networks. Additionally, input to and output from neuromorphic chips is provided with asynchronous digital pulses that encode information in their analog timing, similarly to action potentials of biological neurons. Because of these features, neuromorphic processing chips are very promising candidates for implementing reliable and energy-efficient decoding of neural activity, that could ultimately be evolved to be portable, implantable, and directly interfaced with neural tissue.

For this reason we directed our efforts toward the development of a fully implantable BMI by prototyping a neuromorphic processor chip (Qiao et al., 2015) integrated in a bidirectional brain-machine interface, trained to decode neural signals recorded on-line, and to provide suitable outputs useful for controlling actuators and end effectors. In order to assess the performance of this system, we took the following steps: first we developed suitable spike-based decoding methods that could be implemented by the neuromorphic processor chip, then we configured the chip to implement these methods in real-time and adapted the bidirectional BMI designed and tested in our lab (Vato et al., 2012) to include in the processing chain this neuromorphic component. Finally, we tested this neuromorphic bidirectional BMI in a closed-loop real-time experimental setup that involved the control of the motion of an external device by the decoded neural signals recorded from the brain of an anesthetized rat. Here, we describe in detail the properties of the neuromorphic processor, and the network of spiking neurons that was implemented by the chip to carry out the decoding task. We present the main hardware and software modules that we developed to interface the chip with the other components of the BMI, and describe the experimental paradigm that we used to test the system.

Our approach differs from those of currently-developed BMIs, which are ad hoc ensembles of hardware and software elements designed to perform specific tasks, and which are difficult to replicate, generalize, or modify for use in other tasks or different environments (Leuthardt et al., 2006). As these are limitations that hinder collaborations between laboratories we chose to emphasize a modular approach in designing our BMI by developing a system that is compatible with a wide range of different hardware and software standards, and which is composed of a main control core module and multiple possible recording, stimulating, decoding, and encoding modules. We argue that the combination of this modular bidirectional BMI setup with the use of neuromorphic hardware modules can give a crucial contribution to the development of the next generation of brain-machine interfaces for large-scale clinical applications.

## 2. MATERIALS AND METHODS

We begin by describing the general scheme of this novel bidirectional BMI in Section 2.1 and the experimental procedure used to test the performance of the neuromorphic decoder in Section 2.2. In Section 2.3, we describe in details the main modules comprising the system and finally we present the hardware and the software implementation of the neuromorphic chip, respectively, in Sections 2.4 and 2.5.

### 2.1. General Scheme of the Modular Bidirectional BMI

We extended the Dynamic Neural Interface described in Szymanski et al. (2011) and Vato et al. (2012, 2014) with the inclusion of a neuromorphic decoder module. This system uses the neural signals collected from a rat's brain to control the movement of an external object by means of a sensory and motor interface. In designing it we took inspiration from earlier studies in frogs (Bizzi et al., 1991), rats (Tresch and Bizzi, 1999), and cats (Lemay and Grill, 2004) by emulating the functioning of the spinal cord that combines sensory information with brain instructions and organizes the movement of the limbs along dynamically stable trajectories. We set up a decoding and an encoding interface which generate a dynamic control policy in the form of a force field and robustly drive the movement of the controlled object. The neural signals are recorded from the motor cortex of the anesthetized rat by means of a recording multielectrode array. These signals are transformed by the decoder into a force vector to be applied to a device that can control the motion of the object. After receiving this external input, the device moves the object, according to its dynamics, for a predefined amount of time. An encoder maps each position of the object in the workspace to a pattern of intracortical microstimulation (ICMS) delivered to the somatosensory cortex of the rat. This is achieved by means of a stimulating multielectrode array which provides the brain with information about the position of the controlled object. A calibration procedure of the interface establishes a control policy based on an approximation of a radial force field with the aim of driving the controlled object toward a target location defined by the central equilibrium point of the field. In the implementation described here we use four different patterns of intracortical stimulation and, consequently, the workspace is divided into four different contiguous sensory regions. The four stimulation patterns differ from each other only in the combination of the electrodes chosen to deliver the stimulation. Each stimulation pattern consists of a train of 10 biphasic pulses (100 µA, 100 µs/phase, cathodic first) delivered at 333 Hz (Butovas and Schwarz, 2007; Semprini et al., 2012). After each stimulation, the decoder considers the first 256 ms of the evoked motor neural signal to produce the driving force for the external device. In **Figure 1**, we report the post-stimulus time course of the time-dependent firing rate (mean ± sem over 50 trials) of the evoked neural activity recorded from all the electrodes of the array. The raster plots represent the time occurrences of at least one spike recorded from all the electrodes of the multielectrode array.

The calibration force corresponding to each region was defined by a vector pointing from the region's centroid to the target (colored thick arrows depicted in **Figure 8**). The task of the decoder consists in extracting from each evoked neural response a resulting force, calculated as a weighted sum of the four calibration forces defining the force field. In particular, the decoder needs to extract the four coefficients corresponding to the contribution of each of the four calibration forces to the decoded force.

#### 2.2. Experimental Procedure

Neural data were collected from male Long-Evans rats (300– 400 g) anesthetized for the entire duration of the experimental sessions by means of Xylazine (5 mg/kg) and a mixture of Tiletamine and Zolazepam (30 mg/kg). Two craniotomies were

FIGURE 1 | Post-stimulus time course of the time-dependent firing rate (mean ± SEM across trials) and raster plot of the recorded neural activity evoked by four different stimulation patterns. Each short vertical line in the raster plots represents the occurrence of at least one spike recorded from all the electrodes on the recording array in a 1 ms time bin. In the inset, we report the neural activity recorded from each electrode of the microwire array during a single trial. performed above the somatosensory (S1) and the motor (M1) cortex representing the whiskers on the same hemisphere. The stimulation microwire array (Tucker Davis Technologies—TDT) was lowered perpendicular to the somatosensory cortex 300– 500 µm under the surface (AP −3.5 mm, LM +4 mm with respect to the most posterior medial electrode of the array). The recording array was placed at depth 900–1100 µm below the pia (AP −1.5 mm, LM +0.5 mm with respect to the most posterior medial electrode of the array) using a hydraulic microdrive. These locations have been chosen for the presence of several cortico-cortical connections between the two regions(Mao et al., 2011). Both arrays are composed of 16 microelecrodes (2 rows of 8 electrodes, 50 µm diameter) each one separated from the neighboring ones by 250 and 375 µm along and across the rows, respectively. All the experiments have been performed in accordance with DL116/92 of the Italian legal code and approved by the institutional review board of the University of Ferrara and by the Italian Ministry of Health (73/2008-B).

#### 2.3. Main Modules of the BMI System

The modular bidirectional BMI was designed around a core unit named Managing Unit (MU) that can be connected to satellite modules, each dedicated to specific tasks as decoding the neural signal, controlling the movement of an external device, and encoding the information collected from the external environment to provide sensory feedback. The MU does not require any information about the specific implementation of each module, which can be a software running on general purpose processing units, a dedicated programmable hardware such as Field Programmable Gate Arrays (FPGA) or a neuromorphic chip. This modularity ensures a fast and flexible prototyping phase required during research and development, whereby different software modules can allow testing the algorithms to be implemented on custom low-power, miniaturized implantable hardware.

In this implementation, we connected five different satellite modules to the MU realizing the functionalities required by a bidirectional BMI: Acquisition Unit, Stimulation Unit, Decoder, Encoder and Dynamical System, as shown in **Figure 2** that have been described in details in Boi et al. (2015a). The Dynamical System (see Boi et al., 2015b) consists of a small mobile cart connected to a water/pellet dispenser mounted on a vertical wall in a custom-made behavioral box for rodents and controlled by two servomotors spanning an area of 38 × 38 cm. The cart is protected by a transparent acrylic glass sheet with a slot that allows the rat to grab the food if the cart is positioned in the desired position. The Dynamical System was designed, developed, and tested in this way to be used in future experimental sessions with behaving subjects.

The main algorithm running on the MU named mbBMI algorithm is in charge of reading the spiking neural data coming from the Acquisition System module and communicating them to the decoder. Once the decoder generates an output signal, the algorithm transforms it into motor commands usable by

the Dynamical System. To close the loop on the brain, the algorithm acquires the current position reached from the external device and communicates it to the encoder that returns the next stimulus to be communicated to the Stimulation System module.

#### 2.3.1. Managing Unit

We implemented the Managing Unit by using the development board ZedBoardTM equipped with a Xilinx Zynq <sup>R</sup> -7000 All Programmable System On Chip (SoC). The Zynq <sup>R</sup> -7000 family integrates a feature-rich dual-core ARM CortexTM-A9 based processing system (PS) and 28 nm Xilinx programmable logic (PL) in a single device. In our implementation, the PL runs a custom module that can interface with neuromorphic chips and implements two software modules named NeuElab and Dynamical System Controller. The NeuElab module acquires the pre-processed brain signals from the mbBMI algorithm and routes them to the decoder and vice versa, via its hardware interface (Zynq2Neuro described in Section 2.3.1).

The MU stores the temporal offset of each recorded action potential with respect to the last delivered stimulation, as a list of time-stamps associated with the identity (or address) of the emitting electrode. At the end of each recording period, spike trains are generated from the recorded spike timestamps according to the decoder's requirements (Section 2.5 and **Figure 5**) and then forwarded to the neuromorphic chip. The MU communicates with the decoder using the native neuromorphic asynchronous communication protocol, known as Address Event Representation (AER) protocol (Mortara, 1998), where the information is encoded in the implicit timing between digital pulses (or spikes) and in the identity (or address) of the neuron that has emitted the pulse. The decoder's output AER spikes are acquired by the MU and forwarded to its Dynamical System Controller part.

When acquired on the MU clocked system, the implicit temporal information in the AER spike sequence is explicitly paired with the address of the spike by the TimeStamp block of the NeuElab part of the MU. NeuElab is composed of two different FIFOs that drive the data flow from/to the neuromorphic chip. The TX FIFO is filled with the address of the neuron that shall receive the spike and the time relative to the other spikes, by associating a delay time value by the TimeStamp block. NeuElab reads the TX FIFO and sends a spike to the neuromorphic chip at the time specified by the delay, the address associated to the spike allows the receiving chip to rout the spike to the corresponding neuron. The RX FIFO is filled with the spikes from the neurons of the neuromorphic chip. The received pairs of address and relative time-stamp are then sent to the BMI algorithm that translates the recorded neural activity into commands for the Dynamical System.

Besides managing the AER communication with the neuromorphic chip, the NeuElab interface is critical for the chip's configuration, through digital configuration bits and a number of tunable analog voltages or currents (biases) that set the operating point of the analog circuits. NeuElab can be used, in principle, for interfacing the BMI with any neuromorphic chip that uses the AER communication protocol. In this implementation, the output spiking activity of the neuromorphic chip is translated into a bidimensional force applied to the Dynamical System by means of a pair of Pulse Width Modulated (PWM) analog signals generated by the ZedBoard that drive the external object.

## 2.4. Hardware Aspects of the Neuromorphic Decoder

The decoder that transforms the recorded brain activity into motor commands is implemented on a neuromorphic chip. In the following, we describe the chip and the printed circuit board (PCB) that we developed to connect the chip with the rest of the system.

#### 2.4.1. The ROLLS Neuromorphic Processor

The Reconfigurable On-line Learning Spiking (ROLLS) Neuromorphic Processor is a general-purpose spiking neural network chip (Qiao et al., 2015). **Figure 3** shows the chip micrograph. It was fabricated using a standard 6-metal 180 nm CMOS process, occupies an area of 51.4 mm<sup>2</sup> and has approximately 12.2 million transistors. It comprises 256 adaptive exponential integrate-and-fire neurons implemented in a mixed signal analog/digital circuit design.

There are 128 K synapses, of which 64 K that can implement a Hebbian plasticity rule (Brader et al., 2007; Mitra et al., 2009) [Long-Term Plasticity (LTP) synapses] (Mostafa et al., 2014). The rest 64 K synapses can exhibit short term depression and short-term facilitation dynamics [Short-Term Plasticity (STP) synapses], and have two possible programmable weights resolution, in addition to the possibility to configure them as either excitatory or inhibitory. These two synaptic matrices (LTP and STP) allow arbitrary on-chip connectivity thanks to a crossbar structure. In principle all-to-all connections are possible through the programmable logic state of the synapses. Additional circuits next to the neurons' array represent the calcium concentration at the post-synaptic side, needed to implement the spike-based LTP weight update algorithm (Brader et al., 2007). We refer the readerto Qiao et al. (2015) for a detailed description of the circuits.

Both the neural network architecture and the parameters of the neuromorphic core are fully programmable via a high-level Python framework (Stefanini et al., 2014). The combination of reconfigurable hardware with the Python-based configuration framework supports the exploration of a wide range of spiking neural network architectures, and their real-time emulation in closed-loop setups. Here, these enabled us to configure a hardware implementation of a spiking neural network that learns on-line to decode patterns of recorded spike sequences.

#### 2.4.2. The Zynq2Neuro (Z2N)

With the aim to manage, program, and interface neuromorphic chips with the Managing Unit, we designed and developed the Zynq2Neuro (Z2N) PCB that can host up to two daughterboards (DTB) that mount neuromorphic chips. The Z2N connects the neuromorphic chips to the FNC connector of the ZedBoard, supplies power to the chips and supports the AER communication and the chip configuration signals. Analog biases that configure the parameters of the silicon neural and synaptic models on the neuromorphic chip can be set either

by means of external digital to analog converters (DAC), or by on-chip programmable bias generators (BG) (Delbruck and Lichtsteiner, 2006). NeuElab, together with the Zynq2Neuro board, can drive both systems, the Zynq2Neuro board hosts 64 DACs that can be programmed through an SPI interface and also hosts the necessary signals for programming different types of BGs, managed by NeuElab, hence supporting a large library of neuromorphic chips. The Z2N board is already configured to support future chip functionalities by means of I/O expanders and I2C protocol. The AER addressing space can be expanded up to 30 bits (configurable as inputs or outputs). The Z2N (**Figure 4**) can support logic levels, power supply and biases from Digital to Analog Converters of 3.3 or 1.8 V, as selected from the first DTB. This means that the two DTBs need to host chips that are homogeneous for the logical levels. In general, the Z2N can support chips fabricated on the 350 nm (3.3 V) and 180 nm process of the latest generation (1.8 V and mixed 1.8/3.3 V). To optimize the design, AER address lines, some bits of the Bias Generator programming, I2C and I/O expander are shared among the two DTBs. The sharing of the AER address lines is based on the assumption that they are in tri-state when the chip is not sending or receiving an event. This is guaranteed by the SCX protocol (Mortara, 1998), but can be supported also for the P2P protocol (Boahen, 2000), by adding buffers on the DTB driven by the handshake signals (ACK) from the ZedBoard. The correct addressing of the event to/from the chip is guaranteed by the reserved handshake signals (REQ/ACK and Bias LATCH) that target only one of the two chips. The Z2N specifically targets compatibility with neuromorphic chips such as the ROLLS (Qiao et al., 2015), but is a more general tool for most of existing neuromorphic chips based on parallel (or

word-serial Boahen, 2004) AER protocols, on Bias Generators externally configurable by means of SPI-like serial interfaces, or on external voltage tuning. Some examples of supported chips are the Dynamic Vision Sensor (Lichtsteiner et al., 2008), the AER EAR (Chan et al., 2007), the Selective Attention Chip (Bartolozzi and Indiveri, 2009), the spiking Winner-Take-All chip (Chicca et al., 2014), and the Asynchronous Time-Based Image Sensor (Posch et al., 2010).

### 2.5. Algorithmic Aspects of the Neuromorphic Decoder

We approached the neuromorphic decoding task by combining the constraints of a multi-class classification task with those of spiking neural networks with limited resolution synaptic weights, and with the BMI-specific requirements related to the simultaneous contribution of all four classes to each decoded force (see Section 2.1).

#### 2.5.1. The Silicon Spiking Neural Network

We configured the ROLLS chip to implement a feed-forward spiking neural network that exploits the spike-timing dependent plasticity of the chip's LTP synapses to learn how to extract the pattern of four calibration forces that should result in the net desired force, from the recorded neural activity. Each of the output neurons of the network was trained to act as a binary classifier by re-weighting the features of its input that were distributed across its synapses, so as to ultimately yield, via its activation function, a higher output spike rate for one, positive class of input compared to the other three, negative classes. Neurons were grouped into four ensembles, each corresponding to one of the four stimuli. The spike counts output by the

four ensembles during the presentation of the recordings to the network were directly used as the coefficients that weight the contributions of the four component forces acting on the BMI's end effector.

#### 2.5.2. Mapping the Neural Recordings to the ROLLS Neuromorphic Processor

The spike-based learning algorithm implemented on the chip is based on the model proposed in Brader et al. (2007). Using this model, feed-forward neural networks can learn to classify patterns based on their mean rates. However, in the neural data we recorded, the principal feature that distinguishes one class from the others is the precise timing of the recorded spikes, aligned to the offset of the sensory micro-stimulation (**Figure 1**). Therefore a transformation of the input spike sequence into an array of firing rates is required before it reaches the output layer. Furthermore, the number of non-redundant features in the data needs to be sufficiently high to support robust discrimination across all classes, but the recorded activity was very similar across all recording channels (see **Figure 1**, inset). Therefore it is likely impossible to find a single-layer feed-forward network configuration that can classify the recordings based on features corresponding directly to the recording channels.

To reconcile the characteristics of the data with the network requirements we mapped uncorrelated sub-samples of the spike sequence to different synapses of the classifier neuron, using a mean-rate encoding. Specifically, we binned the recorded spike trains in time bins of 1 ms (**Figure 5B**) and associated each bin with one input synapse of each neuron of the network (**Figure 5C**). We provided a 400 ms high mean-rate (100 Hz) Poisson spike train to the learning synapses for time bins that contained recorded spikes, and no input to the rest of the synapses (**Figure 5C**).

Under the constraint of a finite number (256) of available synapses per neuron, there was a trade-off among the number of recording channels, the duration of the recording patterns, and the temporal precision desired. The first 200–300 ms of each recorded pattern included significant differences across the four different classes (**Figure 1**), that would potentially be sufficient for the classifier to discriminate between them. Based on this, together with the observation that the distributions of spike timings were very similar across different recording channels, we merged the 15 recording channels into a single spike train, and we used the first 256 ms of the recordings, thus acquiring a temporal precision of one ms per time bin. Longer recording duration with a two-millisecond or lower precision was found to diminish decoding performance.

#### 2.5.3. The Neural Network's Task

The aim of the BMI is to best approximate the desired force field over the duration of the experimental session, through weighting the four force components. To achieve this aim, there are two criteria based on which the decoder has to simultaneously optimize its learning. Firstly, it needs to learn to classify the patterns, i.e., to correctly output the single class to which each presented recording truly belongs, as expressed by the "winning" (i.e., the most firing) ensemble of output neurons. Secondly, the decoder also needs to prevent the other three "losing" ensembles from biasing the force field toward particular directions on average over the trajectory of the end effector. That is, it

needs to classify the recordings under the constraint of learning to equalize the average outputs of "losing" ensembles. Thus, despite the similarities to a classifier, classification of individual recordings is only partly the decoder's task.

#### 2.5.4. Biased Similarities and Differences between Classes of Recordings: Addressing Them with Heterosynaptic Competition

The decoder had to address certain additional characteristics of the recordings to achieve its goal of approximating the desired force field over the experiment's course. Specifically, different classes of recordings differed in number of recorded spikes on average, and this difference in the input energy could be reflected as a bias in the chip's output and consequently in the direction of the decoded force in each trial. Moreover, even though spike timing was the principal difference between recordings of different classes, some spike timings were common between classes. This increased the difficulty in distinguishing between different classes. That is, the different classes had a certain level of overlap between their features, which could increase classification errors. Additionally, this overlap was not of the same extent for all pairs of classes, i.e., some classes were more similar to some than to others in terms of common spike timings (**Figure 1**). This asymmetry could result in additional biases in the weighting of the force components by the decoder, thus misshaping the resulting force field in certain parts of the working space.

To address these points, we used the "stop learning" feature of the ROLLS chip learning circuits (see Brader et al., 2007) which prohibits potentiation of synapses when the post-synaptic firing rate exceeds a threshold. When a certain number of synapses that correspond to a neuron's positive class are potentiated, the increased excitation from the input causes the neuron to stop learning. This introduces heterosynaptic competition (Royer and Paré, 2003) to the chip's output neurons, which serves (a) to normalize the network's output in response to different classes, (b) to make potentiated synapses a scarce resource hence biasing potentiation toward non-overlapping features, and (c) to equalize the output of "losing" ensembles. In addition, combined with device mismatch on the neuromorphic circuits, it biases different members of each ensemble to learn a slightly different decision boundary. This is similar to boosting techniques employed in machine learning and improves the classification performance by allowing for non-linear decision boundaries for the ensemble through the aggregation of the multiple linear boundaries defined by the ensemble's member neurons.

#### 2.5.5. Training the Neuromorphic Decoder

To train the neuromorphic decoder, we used an experimental session composed of 40 repetitions of each stimulation pattern (i.e., 160 evoked recordings). During the training procedure were randomly interleaved (**Figure 5A**) and presented to the ROLLS processor the 160 training trials, according to the method described in Section 2.5.2 (**Figures 5B,C**), along with a teacher signal representing the label of the presented example, i.e., the type of sensory microstimulation that produced the recorded neural response. Sixty-three output neurons were assigned to each class (**Figure 5C**, right). The teacher signal biased each neuron to be tuned to one class, by causing it to fire with a rate that maximized the probability that the neurons synapses got potentiated when an example of that class was presented, and depressed when an example of the other classes was presented. The mean rate of the Poisson spike train that would act as a teacher signal with these properties, as well as the analog parameters of the silicon neurons and synapses of the ROLLS processor were configured to match the characteristics of the input data with the requirements of the learning and of the decoding task.

#### 2.6. Assessing the BMI's Performance

Once the decoding and encoding interfaces were properly calibrated, in order to test the system we ran the BMI by decoding from each neural trial a bidimensional force and by encoding each position of the controlled object through an ICMS pattern. We used a test dataset of neural recordings acquired by 10 repetitions of each of the four stimulation patterns (i.e., 40 evoked recordings), which were unseen by the BMI during its training. We selected eight different equispaced and equidistant positions as starting points in which the dynamical system was initialized and we ran the BMI 100 times starting from each initial position by obtaining 800 trajectories. We tested the system under two conditions: under normal operation (encoder-ON condition), each test recording was selected according to the dynamical system's current position. An alternate condition (encoder-OFF) was used to test the bidirectionality of the BMI and the learned coordination between the encoder and decoder modules. In the encoder-OFF condition, each test trial was randomly selected among all 40 test recordings.

To assess the repeatability, the speed and the optimality of the generated trajectories we measured the number of steps required to converge to the target and the mean within-trajectory variance (abbreviated to wtv). In particular, each trajectory's wtv is defined as q C2 <sup>x</sup> + C<sup>2</sup> y , where C<sup>x</sup> and C<sup>y</sup> is the covariance of the distribution of the per-step displacement along the x and the y axis, respectively. We obtained the mean wtv by averaging the wtv computed for each set of trajectories that started from one initial position.

#### 3. RESULTS

#### 3.1. Decoding Performance

To assess the decoder we used test datasets, which were previously unseen by the decoder, as described in Section 2.6. For each decoded pattern, the output spikes produced by each neuronal ensemble (**Figure 6A**) were counted. Given a stimulus, the average spike count of the ensemble of silicon neurons corresponding to that stimulus was higher than the other three (**Figure 6B**).

In addition, as a result of the introduction of "stop learning" to the silicon neurons average spike counts were relatively uniform across the other three ensembles despite the biases in pairwise similarities between input classes (see Section 2.5.4). The chip learned to suppress this bias, and, consequently, decoded resultant forces for each stimulus were, as originally intended, most similar to one of the four forces used during the calibration phase (colored thick arrows shown in **Figure 8B**).

While the task of the decoder was not a pure classification task and it was not optimized to perform as a classifier, we also evaluated its performance in correctly classifying the recordings, as expressed by the maximally firing ensemble of neurons. For 20 different random splits between the training and the test sets, the classification performance on the test set ranged between 50 and 70% correct, with the chance performance level being at 25%.

#### 3.2. BMI Performance

In order to assess the BMI performance, we performed two different testing sessions: during the first session we set the maximum number of steps to 100 as stopping rule for the obtained trajectories (**Figure 7**). The BMI moved the object freely according to the sequence of forces that the closed-loop set-up applied and we placed the target as the origin of the axes. In each trial, the controlled object was initialized at one of eight starting positions and the BMI generated one trajectory of 100 encoding + decoding steps. We marked and plotted in the figure the point that was closest to the origin of the axes considered as the target point (**Figure 7A**). For each starting position we repeated the experiment 100 times, yielding 800 points in each of the two conditions (blue points for "Encoder ON" and red points for "Encoder OFF"). In condition ON, when a stimulus was provided to the sensory cortex, it was according to the current position of the object. In condition OFF, the stimulus was selected randomly among the four possible stimuli, thus not encoding the current position of the object. The distributions of the two sets of points (**Figure 7B**) are statistically different (independent samplest-test, p < 0.001) showing a decrease of 99% in the distance from the target and demonstrating that closing the loop in the proposed BMI is crucial in order to correctly drive the dynamical system toward a target.

In the second testing session, we simulated a real experiment in order to generate motor commands that drive a mobile cart from predefined initial positions toward a target position represented by a slot in the glass that allows the rat to get the reward (Boi et al., 2015b). In this session to distinguish between convergent and non-convergent trajectories, we defined the target as a circular region with radius set to 3.6 cm placed in the center of the workspace. A trial was considered successful as soon as the generated trajectory reached the borders of this area. When this happened the BMI was disconnected and the cart was automatically positioned in the center of the slot to allow the subject to receive the reward.

FIGURE 6 | Output of the trained decoder. (A) Raster plot of the output spikes of the trained ROLLS chip during presentation of four example test recordings each resulting from a different type of stimulus. The length of the bars on top shows the 400 ms long presentation of the input. During presentation of the four examples, the most active ensemble of output neurons corresponds to the true stimulus that caused the input recording. The spike count of the output each of the four neuronal ensembles was directly used to weight each of the four components of the force field to result in the motor command, i.e., force, that acted on the controlled object. The chip's neurons maintained some activity till shortly after the input stopped, mainly due to excitatory current leaking between the firing neuronal electronic circuits. (B) Average output spike count for each ensemble of neurons, for each type of stimulus that caused the decoded recording. For each stimulus, its corresponding ensemble fires on average more than the other three, demonstrating the classification aspect of the decoder's task. In addition, the decoder learned for each stimulus to partially equalize the response amplitudes of the three non-corresponding ensembles, compared to the extent of the differences between input classes (cf. Figure 1 and see Section 3.1).

points to the workspace axis origin with the encoder switched OFF while blue dots represent the same points for the trajectories generated with the encoder ON. Data were collected by running the BMI 100 times for each of the eight predefined initial positions (i.e., numbered circles) both with the encoder turned ON and OFF. (B) Box plots of the trajectories closest points distributions with the encoder ON and OFF. Two-sample t-test, \*\*\*p < 0.001.

**Figure 8A** shows the mean trajectories (blue lines) and the covariance (light blue area) generated during this experimental session with the encoder turned ON. Two distinct behaviors are distinguishable (see Figure S1C): if the pathway from the starting position to the target region lies inside the same sensory regions, we obtained an almost straight trajectory. On the other hand, when the controlled device crosses the border of one region, the systems oscillates along the border of the two adjacent regions.

FIGURE 8 | Testing of BMI performance with target-region stopping rule. (A) Mean trajectories plot. Starting from each starting point depicted with a numbered circle, the blue lines represents the mean trajectories and light blue areas represent the covariance of the trajectories along 100 trials. The workspace is subdivided into four sensory regions, one per each stimulation, highlighted by four different colors. We defined a target region centered in the origin of the axes and whenever the mobile cart reaches its edge the BMI considers the task accomplished. (B) Black arrows represent the decoded forces computed during the BMI test phase. Colored thick arrows represent the four calibration forces associated to the sensory regions. Forces were grouped on the basis of the stimulus that generates them. (C) Mean within-trajectory variance (wtv) ± SEM of all the 800 trajectories recorded both with the encoder turned ON (blue bar) and OFF (red bar). (D) Mean number of steps to convergence ± SEM. The red bar, obtained with the encoder turned OFF, is quite close to the maximum step allowed (100 steps) while when the encoder is active the steps number necessary to reach the target region is significantly lower. (E) Mean DT component magnitude ± SEM. Each decoded force has been split into Directed to the target - DT (magnitude of the force that points toward the target) and Orthogonal to the target - OT one (part of the force perpendicular to the directive component). The mean magnitude of the DT component obtained from forces generated with the encoder turned off (red bar)is much higher than when the encoder activated (blue bar). Two sample t-test, \*\*\*p < 0.001.

This particular behavior does not represent a decoding error but rather reflects the limitation of having only four different stimulation patterns encoding the information about the region in which the device is, disregarding the precise position inside it (Tehovnik, 1996; Romo et al., 1998). The BMI converges to the target region with a 100% success, and it does so in a very stable and straight path because the decoded forces obtained in response to the same stimulation pattern are very similar to each other, both in terms of direction and magnitude. This is demonstrated in the compass plots in **Figure 8B** showing that the forces decoded from the neural activity evoked from each stimulation pattern and used during the testing phase (i.e., black arrows) are almost overlapping. In order to further assess the neuromorphic decoding capabilities we also report the forces used to calibrate the BMI motor interface (colored thick arrows in **Figure 8B)** that, especially in terms of direction, are almost equal to most of the related forces decoded during the BMI run. In the encoder-ON case the mean wtv and the steps needed to reach the target region significantly decrease (respectively, 92 and 80%) with respect to the encoder-OFF case (**Figures 8C,D**).

Finally, for each force produced by the decoding process, we measured the magnitude of two components: the component of the force that points toward the target point, named Directed to the target—DT, and the component orthogonal to it, named Orthogonal to the target—OT. The mean of the DT-component is strongly positive (directed to the target) in the case of encoder-ON and slightly negative (divergent from the target) when the encoder is turned OFF (**Figure 8E** shows an increase of 69%). In both conditions (ON and OFF), the mean OT-components are almost null compared to the mean DT obtained with the encoder-ON (respectively, 90 and 97% less). In the OFF condition, this can be attributed to the randomness of the motion. In the ON condition, combined with the increased DT force, this is an indication of successful decoding.

Figures S1A,B show the complete set of trajectories collected without using the target-region stopping rule, respectively, with the encoder switched ON and OFF. Figures S1C,D shows the set of trajectories used to build the different panels of **Figure 8**.

#### 4. DISCUSSION

In this paper, we showed the applicability of neuromorphic hardware in a brain-machine interface system, in the first demonstration of this kind. In particular, the decoder module of the BMI was implemented by a spiking neural network on a mixed-signal analog/digital neuromorphic processor, the ROLLS, that learned to perform on-line the decoding of the neural recordings into commands that addressed the brain-controlled device.

The analog neuromorphic circuits of the ROLLS neuromorphic processor emulate functions of biological neurons and synapses by replacing biophysical properties with analogous properties of the sub-threshold physics of transistors. The resulting spiking neural networks operate on a power-efficient and compact system for applications of pattern recognition such as a BMI decoder's task. On the other hand, because of these underlying principles of operation, analog neuromorphic circuits like the ones found on the ROLLS are imprecise and variable, similar to biological neural elements, in sharp contrast to simulations of spiking neurons and synapses on digital neuromorphic or general-purpose processors. The neuromorphic decoding task was further complicated by the variability in the recorded data, and by the overlap in spike-timings between the to-be-discriminated classes.

Further difficulty arose by the fact that the decoder's task was not a standard classification task, as the BMI required the decoder to output a contribution of all potential classes of recorded activity simultaneously, while preventing the average chip output from being biased toward any pair of classes, even though the pair-wise similarities between classes were biased.

Despite these particularities, the spiking network we designed successfully learned the decoding task, enabling the BMI to perform at similar levels of a previous non-neuromorphic version of the bidirectional BMI. This was achieved by exploiting two key characteristics of the ROLLS chip: variability between silicon synapses and neurons deployed into an ensemble learning technique that aggregated multiple weak classifiers into a powerful one, and the heterosynaptic competition through the "stop-learning" feature of synapses on the ROLLS chip, which enabled the network to focus on the discriminative features of the input, thus both improving classification performance and reducing the reflection of biased similarities in the input onto the output of the trained network. A key feature of the decoder is that the spiking output of the neuromorphic chip is directly used to compute the force controlling the end-effector. The components of the force were weighted by the spike counts of the chip's output, an important step toward using neuromorphic hardware not only as a decoder, but also as prosthetic controller.

### 4.1. Features of the Proposed Neuromorphic Decoder

The set-up we propose has been designed as an initial proof of concept prototype to evaluate the potential of neuromorphic hardware computing in BMIs, and to determine its limitations; within this context, this work shows that, even at this level, integration of neuromorphic hardware in set-ups characterized by the intricacy of a bidirectional BMI is technically possible. Our results show that, despite the low precision, low resolution, and noisy (but compact and low-power) analog electronic circuits in the neuromorphic chip, the system built in this way can recognize multi-dimensional input patterns. In particular, the results demonstrate how this neuromorphic hardware can be configured to produce the correct average forces over the controlled object's trajectory (**Figure 8A**), despite the fact that the forces decoded from individual recordings could strongly deviate from the target (**Figure 8B**) due to the contributions of all four force components combined with unbalanced inputs (**Figure 1**). A unique aspect of the specific neuromorphic hardware used is represented by its ability to learn these computationally demanding tasks, with on-chip real-time spike-based plasticity circuits, as opposed to learning the network parameters off-line and configuring them at run-time. The flexibility provided by the digital event-based communication infrastructure, and the digital registers embedded in the chip, next to the subthreshold analog neuromorphic circuits, allow this system to be used in a variety of tasks that require real-time decoding or classification of sensory inputs, or real-time encoding of desired outputs. Although, the analog circuits have time constants of the order of milliseconds (in order to provide biological realism, and importantly, to minimize power consumption), the real-time response properties of the chip at network level have latencies that are extremely small (e.g., below tens of microseconds). This allows the chip to decode the neural activity on line in the BMI's loop, within one time step of the dynamical system's operation, whose bottleneck is determined not by the decoder, but by the inter-stimulus interval. The average power consumption of the chip, which has been measured to be approximately 4 mW, is competitive with stateof-the-art DSPs and much lower of general purpose low-power computing units that could be used to run the pattern recognition software. It is worth noting however, that since in the current set-up the neuromorphic chip is interfaced to additional devices mainly used for prototyping and debugging, the overall system requires additional relatively high power and area.

### 4.2. Limitations of the System and Proposed Future Additions

The simplicity of the single-layer feed-forward network of only 252 neurons that was employed for this particular application demonstrates the limitations and computational power of physical instantiations of spiking neural networks and suggests that further development of analog neuromorphic hardware and spike-based algorithms may yield a computationally powerful, yet low-power consuming alternative to software and conventional processors for a broad spectrum of tasks. With respect to the neuromorphic BMI decoder in particular, further work could enable two specific improvements and additions.

Firstly, the present implementation addresses the complex temporal dynamics of the recordings with a processing step introduced between the neural recording and the output layer of the neural network, and performed off-chip, which transforms the temporal dynamics of the recordings to a spatial pattern input to the chip. While the method proposed is suitable for the presented system, we have been investigating alternative algorithms and spiking neural network architectures that can potentially decode and recognize these types of spatio-temporal patterns entirely on the chip. In this way, the chip could directly receive the recorded spike train, and operate on it with no need for an intermediate off-chip storage step. This would be possible because of the ROLLS' real-time operation, with time constants that match those of real neurons. To this direction, Corradi and Indiveri (2015) perform a binary classification task on spatio-temporal recordings from the zebra finch, using reservoir computing on the ROLLS' silicon neurons, which demonstrates that future development of these types of methods can permit their application on a BMI.

On a separate but related note, here the BMI operated in discrete time steps. This permitted us to insert the processing step that inputs the recorded spike timings as rate-coded patterns into the ROLLS chip, without loss the system's continuity. Nonetheless, this will be a crucial obstacle for the decoding module's integration in future continuously operating BMIs. On the other hand, the limitation does not originate in the ROLLS chip itself. The chip does not have an internal clock that must be synchronized with the chosen time points. It rather recognizes inputs in which time represents itself in the spike train's statistics. This implies that removing any off-chip transformation that intermediates the input would also enable the on-line use of the chip in continuous-time BMI set-ups.

As a further future improvement, the fact that the network learns on line could be used to allow the decoder to adapt to changes in the neural responses with time. Specifically, in the current implementation, the decoder updates itself incrementally after the presentation of each training pattern. Training inputs are combined with a teacher signal that biases different neurons to strengthen or weaken their connections to different features of the input, through imposing different levels of output firing during the presentation of different input classes. After training, we use the chip to decode new recordings of brain activity. The on-line learning feature is not crucial for demonstrating the performance of the BMI in its current instantiation, but can become useful in future chronically implanted setups, that have to adapt to continuous slow changes in the nature of the signals being recorded. In such a future implementation, learning could continue during the chip's use as a trained decoder. As the trained silicon neurons respond with high firing rates to their corresponding input classes, and with lower rates to the other classes, the neurons could bias themselves to continue correctly adapting their synapses to the input patterns in the absence of an externally provided teacher signal. This would be made possible after tuning the parameters of STDP synaptic dynamics of the ROLLS to enable potentiation and depression in the ranges of firing rate that the trained neurons output when decoding the input.

#### 4.3. BMI Modularity

As technological and scientific progress accelerates, it brings new opportunities for improving the quality of life of millions of people. The interdisciplinary field of brain-machine interfaces largely relies on the rapid evolution in the diverse fields that are involved (Nicolas-Alonso and Gomez-Gil, 2012). Nevertheless, the complexity of BMI systems, the interdependence of their components cause them to be very difficult to manage, test, modify, and upgrade. Our work suggests a possible solution to this issue by proposing a new modular implementation that allows to modify or update each module without changing the entire system.

The modularity allows to develop different parts of the BMI in different labs and assemble the complete system by plugging in these parts as modules. This structure makes easier and more reliable both the implementation of the single module and its integration in the complete system. Parallel development of components could also accelerate the ultimate realization of a device compact and powerful enough to be used as clinical tool able to transfer data between the brain and external devices wirelessly through an implanted interface (Azin et al., 2011; Fan et al., 2011; Borton et al., 2013; Angotzi et al., 2014). In this work, we also demonstrated that the modular architecture does not affect BMI performances, showing results comparable with the ones achieved in Vato et al. (2012); this result suggests that BMI systems developed in other labs could also be re-implemented in a modular manner. To help the interested scientist in doing this, most of the material used in this project is freely available on Si-Code website : http://www.sicode.eu/results/software.

#### 5. CONCLUSIONS

The relevance of neuromorphic technology in the design of brain-machine interfaces is demonstrated by the flourishing work in this domain (see Dethier et al., 2013; Barsakcioglu et al., 2014; Hogri et al., 2015, as non-exhaustive examples). The main features of neuromorphic implementations are low power consumption, real-time operation, adaptability, and compactness. Simulations show that hardware Spiking Neural Networks can successfully decode the activity of neurons for closed-loop cortical implants (Dethier et al., 2013) and an ad-hoc working prototype is able to substitute a cerebellar learning function in the rat (Hogri et al., 2015). Our work extends this approach in proposing a modular and reconfigurable scheme whereby the neuromorphic chip can be exploited for implementing different algorithms and BMI functions; in particular, we demonstrated this approach by using the chip as neural decoder. We also explored the impact of using a neuromorphic decoder in such a closed-loop system by comparing its performance with the one previously developed in our lab.

As in Vato et al. (2012), we closed the loop with the brain by decoding the neural activity evoked by different patterns of intracortical micro-stimulation selected by the encoder. Even if we are not decoding from the anesthetized subjects any volitional input, this system, establishing a bidirectional interaction between the brain and an external device, needs to be considered the first necessary step toward the design of future experiments involving behaving subjects controlling the movements of a small mobile cart connected to a water or food dispenser (Boi et al., 2015b). The unique characteristics of the neuromorphic decoder will allow our

#### REFERENCES


modular bidirectional BMI to integrate the volitional component of brain activity in the decoding scheme and to explore the integration of the volitional input with the automatic brain response in controlling the movement of the external device.

#### AUTHOR CONTRIBUTIONS

FB, VD, FD, and CB designed, built, and debugged the hardware and software infrastructure for the mbBMI. TM and GI implemented the spike-based learning and decoding algorithm on the ROLLS neuromorphic processor. FB and AV performed the experiments and collected the neural data. FB, VD, and TM performed all the analysis presented in the paper. All the authors contributed in writing and editing the manuscript.

#### ACKNOWLEDGMENTS

Research supported by the EU Future & Emerging Technologies (FET) program under Grant "SI-CODE" (284553), the EU ERC Grant neuroP (257219), and the Samsung Advanced Institute of Technology (SAIT)s Global Research Outreach (GRO) Program.

#### SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: http://journal.frontiersin.org/article/10.3389/fnins. 2016.00563/full#supplementary-material

system," in 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) (Milan: IEEE), 51–54.


processor comprising 256 neurons and 128k synapses. Front. Neurosci. 9:141. doi: 10.3389/fnins.2015.00141


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2016 Boi, Moraitis, De Feo, Diotalevi, Bartolozzi, Indiveri and Vato. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Qualitative-Modeling-Based Silicon Neurons and Their Networks

#### Takashi Kohno<sup>1</sup> \*, Munehisa Sekikawa<sup>2</sup> , Jing Li <sup>3</sup> , Takuya Nanami <sup>4</sup> and Kazuyuki Aihara<sup>1</sup>

1 Institute of Industrial Science, University of Tokyo, Tokyo, Japan, <sup>2</sup> Department of Mechanical and Intelligent Engineering, Utsunomiya University, Utsunomiya, Japan, <sup>3</sup> College of Electronic Engineering, Xi'an Shiyou University, Xi'an, China, <sup>4</sup> Department of Electrical Engineering and Information Systems, University of Tokyo, Tokyo, Japan

The ionic conductance models of neuronal cells can finely reproduce a wide variety of complex neuronal activities. However, the complexity of these models has prompted the development of qualitative neuron models. They are described by differential equations with a reduced number of variables and their low-dimensional polynomials, which retain the core mathematical structures. Such simple models form the foundation of a bottom-up approach in computational and theoretical neuroscience. We proposed a qualitative-modeling-based approach for designing silicon neuron circuits, in which the mathematical structures in the polynomial-based qualitative models are reproduced by differential equations with silicon-native expressions. This approach can realize low-power-consuming circuits that can be configured to realize various classes of neuronal cells. In this article, our qualitative-modeling-based silicon neuron circuits for analog and digital implementations are quickly reviewed. One of our CMOS analog silicon neuron circuits can realize a variety of neuronal activities with a power consumption less than 72 nW. The square-wave bursting mode of this circuit is explained. Another circuit can realize Class I and II neuronal activities with about 3 nW. Our digital silicon neuron circuit can also realize these classes. An auto-associative memory realized on an all-to-all connected network of these silicon neurons is also reviewed, in which the neuron class plays important roles in its performance.

#### Edited by:

Paolo Bonifazi, Tel Aviv University, Israel

#### Reviewed by:

Alberto Mazzoni, Scuola Superiore Sant'Anna, Italy Maurizio Mattia, Istituto Superiore di Sanità, Italy

> \*Correspondence: Takashi Kohno kohno@sat.t.u-tokyo.ac.jp

#### Specialty section:

This article was submitted to Neuroprosthetics, a section of the journal Frontiers in Neuroscience

Received: 08 February 2016 Accepted: 31 May 2016 Published: 15 June 2016

#### Citation:

Kohno T, Sekikawa M, Li J, Nanami T and Aihara K (2016) Qualitative-Modeling-Based Silicon Neurons and Their Networks. Front. Neurosci. 10:273. doi: 10.3389/fnins.2016.00273

Keywords: qualitative modeling, silicon neuron, non-linear dynamics, low-power circuit, neuronal network emulation

### 1. INTRODUCTION

The nervous system allows individual animals and their populations to survive in severe environments by analyzing a huge amount of information from sensory organs and promptly generating adequate control signals for motor organs. This complex and intelligent information processing ability is autonomously obtained and adaptively maintained on its genetically developed physical basis, the network of neuronal cells. The nervous system consumes a sufficiently low power to allow for operation within the power supply limit of an animals' body; for example, the human brain consumes about 20 W (Clarke and Sokoloff, 1999), which is a lower power than mainstream CPUs. Because it is a network of neuronal cells with a wide variety of complex activities, the mechanisms of its information processing function are still poorly understood. It is attracting increased attention from biological, medical, and engineering fields.

A silicon neuronal network is a network of silicon neurons (SNs) connected via silicon synapses (SSs), which are electronic circuits that reproduce the electrophysiological activity of neuronal cells and synapses, respectively. Unlike neuro-inspired artificial neural networks, it is an approach to neuromimetic systems that realize intelligent, autonomous, robust, and power-efficient information processing via an architecture comparable to the nervous system (Arthur and Boahen, 2011; Brink et al., 2013b; Cassidy et al., 2013; Kohno et al., 2014b; Qiao et al., 2015; Giulioni et al., 2016). Because it is a bottom-up approach with cell-level granularity and reproduces neuronal spiking activities, it is also applicable to biohybrid systems including neuroprosthetic devices that replace damaged nerve tissues (Ambroise et al., 2013). Generally, SN circuits are required to have the capability of reproducing complex neuronal activities, have a low power consumption, and be compact and highly integratable.

In fields where the reproducibility is important, such as the biohybrid systems and high-speed simulators, SN circuits have been designed to solve ionic conductance neuronal models (Simoni and DeWeerth, 2006; Schemmel et al., 2010; Grassia et al., 2011; Saïghi et al., 2011). These models describe the dynamics of ionic currents that generate the dynamical behavior of the membrane potential by charging and discharging the membrane capacitance. They can precisely reproduce neuronal activities, but their equations are described by high-dimensional non-linear differential equations (DEs). It was demonstrated that their circuit implementations (conductance-based SNs) can well reproduce the neuronal activities of their target cells but require a relatively large amount of hardware resources and consume a relatively high power in the range of micro- to milliwatts. The ionic conductance models share a common structure, namely the Hodgkin–Huxley formalism, which allows their circuit implementation to mimic a variety of neuronal cells after fabrication by applying appropriate parameter voltages (Grassia et al., 2011; Saïghi et al., 2011).

In fields where low power consumption and integratability are important, SN circuits that solve integrate-and-fire (I&F) models are widely used. These models describe the neuronal activities with simple DEs by treating a spike as an event and focusing on the timing of spike generation. Their analog and digital circuit implementations (I&F-based SNs) have been developed (Thomas and Luk, 2009; Arthur and Boahen, 2011; Cassidy et al., 2013; Merolla et al., 2014; Mayr et al., 2015; Qiao et al., 2015; Giulioni et al., 2016). Analog I&F-based SNs achieve ultralow power consumption down to several nanowatts and several hundreds of them were integrated on a chip with thousands of SS circuits. Although their digital implementations consume more power, they are more portable, easy-to-operate, and highly integratable. A milestone work is the TrueNorth chip (Merolla et al., 2014) that integrates 1 million SNs and 256 million SSs on an application-specific integrated circuit chip and consumes less than 70 mW. Silicon neuronal networks implemented on field-programmable gate array (FPGA) chips achieve far less integration (about 1000 SNs) and consume higher power, but their low cost and reconfigurability have attracted many researchers. Sophisticated I&F-based models such as the Izhikevich (IZH) (Izhikevich, 2004) and adaptive exponential I&F (Brette and Gerstner, 2005) models incorporate the viewpoint of qualitative neuronal modeling described below, which allows them to reproduce a variety of neuronal activities. In principle, however, they cannot reproduce some neuronal properties related to the variability of spikes, which are reported experimentally and indicated theoretically. For example, it was reported that the amplitude of spikes at an axon terminal in the hippocampus is gradedly dependent on the stimulus intensity (Alle and Geiger, 2006), and a mathematical analysis of neuronal models indicated that a class of neurons, Class II in Hodgkin's classification (Hodgkin, 1948), can generate spikes in the same manner (Rinzel and Ermentrout, 1998). In addition, the parameter setting of the I&F-based models requires careful tuning; for example, we pointed out that the phase response curve (PRC) of the IZH model in the Class II setting is discontinuous at θ = 0, which causes a severe reduction in the retrieval ability of an auto-associative memory in all-to-all connected networks (Osawa and Kohno, 2015). This problem can be solved by increasing the parameter d of the model, which distorts the waveforms of the membrane potential by producing a huge hyperpolarization after each spike. Another example is that the spiking patterns of the IZH model in the intrinsic bursting (IB) setting have different characteristics from IB cells when a strong stimulus is applied (Nanami and Kohno, 2016). These facts suggest the possibility that a network of I&F-based silicon neurons may have limited ability to reproduce particular information processing in the nervous system.

In the field of qualitative neuronal modeling, the mathematical techniques of non-linear dynamics have been effectively applied to ionic-conductance models to produce lowdimensional and simple polynomial equations that qualitatively capture their dynamical properties (Rinzel and Ermentrout, 1998; Izhikevich, 2007). In contrast to the I&F approach, they do not ignore specific phenomena including the spike generation mechanism. The most well-known qualitative model is the FitzHugh–Nagumo (F-N) model (FitzHugh, 1961) that reproduces the dynamical structure in the Hodgkin–Huxley (H-H) model (Hodgkin and Huxley, 1952). The H-H model is described by four-variable non-linear DEs, whereas the F-N model is two-variable and its only non-linear term is cubic. The F-N model is Class II and can produce the graded spike response to a pulse stimulus. The first silicon nerve membrane circuit, the Nagumo circuit (Nagumo et al., 1962), implemented this model using the tunnel diode. Later, several SNs have implemented the F-N and other qualitative models using recent analog and digital circuit technologies (Linares-Barranco et al., 1991; Cosp et al., 2004; Weinstein and Lee, 2006).

In our previous works (Kohno and Aihara, 2005, 2007, 2008a; Sekikawa et al., 2008; Kohno and Aihara, 2010; Li et al., 2012; Kohno and Aihara, 2014a; Kohno et al., 2014b), we proposed a qualitative-modeling-based design approach for SNs. In this approach, a qualitative neuronal model that reproduces the dynamical structure in a target neuronal model is constructed by combining the formulae for the characteristic curves of favorable elemental circuit blocks instead of polynomials. The elemental circuit blocks are selected according to the required features of the SN; for example, subthreshold metal–oxide– semiconductor field-effect transistor (MOSFET) circuit blocks may be used for low-power SNs. Such a model is expected to be implemented efficiently in comparison to the direct implementation of polynomial-based qualitative models. In addition, a model that supports the mathematical structures in different classes of neurons can be designed, and one of them is invoked by appropriately selecting the model parameters. We developed a configurable low-power analog SN circuit (Kohno and Aihara, 2008a,b; Sekikawa et al., 2008; Kohno and Aihara, 2010) and a configurable simple digital SN circuit (Kohno and Aihara, 2007; Li et al., 2012, 2013). Our analog SN supports five classes of neuronal activities, Class I and II in the Hodgkin's classification, regular spiking (RS), square-wave bursting, and elliptic bursting (Wang and Rinzel, 2003) by appropriately setting the parameter voltages, and our digital SN supports Class I and II and Class I<sup>∗</sup> (Tadokoro et al., 2011) neuronal activities. Basu and Hasler (2010) developed two ultralow-power analog SNs that consume several nanowatts (Brink et al., 2013b) on the basis of a similar approach; one of them is dedicated to Class I and another to Class II. We are developing an analog SN that supports both classes and consumes a low amount of power that is comparable to their work (Kohno and Aihara, 2014a).

Most silicon neuronal networks incorporate the SS circuits that mimic the signal transmission in chemical synapses. Their three important features are the synaptic efficacy, plasticity, and summation (Destexhe et al., 1998; Song et al., 2000; Dan and Poo, 2004). A large (small) amount of synaptic current is injected into the postsynaptic neuronal cell if the synaptic efficacy is high (low). The synaptic efficacy is modulated on the basis of some factors including the neuronal spikes generated by the preand postsynaptic neuronal cells (the synaptic plasticity). It is called the spike-timing-dependent plasticity (STDP) if its rule (a learning rule) is based on the timing of neuronal spikes in the preand postsynaptic neuronal cells (Song et al., 2000; Dan and Poo, 2004). The synaptic summation allows a bursting spike input to enhance the effect of synaptic transmission. It was shown that this feature can play a critical role in spike timing recognition (Gütig and Sompolinsky, 2006). Note that the information of an input spike's magnitude can be transmitted via the time period of neurotransmitter release. The compactness and low-power consumption of SS circuits are also an important issue because the number of SSs in a silicon neuronal network is generally larger than that of SNs. In Merolla et al. (2014), the integration of a huge number of digital SSs was realized by limiting the functionality of the SS to the synaptic efficacy. Their synaptic weights have to be calculated by a off-chip system, but this is not a limitation in engineering applications in which "ready-trained" discriminators are required. They reported that this circuit could realize a multiobject detection and classification system. Only the synaptic efficacy was supported also in early FPGA-based silicon neuronal networks (Rice et al., 2009; Thomas and Luk, 2009), but in recent works, the synaptic summation is supported in Ambroise et al. (2013) and all of the three features are supported in Li et al. (2013) and Cassidy et al. (2013). The analog SS circuit in Giulioni et al. (2016, 2015) implements the synaptic efficacy and the plasticity. Their silicon neuronal network chip integrates 128 leaky I&F SNs and 16384 SSs whose synaptic efficacy is stored in a bistable memory and controlled by a Hebbian-type STDP rule (Fusi et al., 2000). They realized an autoassociative visual memory (Giulioni et al., 2015) and motion detectors (Giulioni et al., 2016). The analog SS circuit in Qiao et al. (2015) implements all of the three features of synapses. The synaptic summation is realized by a low-power currentmode integrator circuit called a differential-pair integrator (DPI). To reduce the circuit size, a DPI circuit is shared by multiple synapses by exploiting its linearity. The synaptic efficacy is stored in a bistable memory and controlled by an STDP-based algorithm (Brader et al., 2007). This chip integrates 256 adaptive exponential I&F SNs with more than 128,000 SSs and was applied to image classification tasks. Another full-featured analog SS in Brink et al. (2013b) stores the synaptic efficacy in an analog nonvolatile memory based on a floating-gate device and supports an asymmetrical STDP learning rule. This chip integrates 100 Class II SNs with 30000 SSs and realized a winner-take-all network and a rhythm generator (Brink et al., 2013a).

In this article, we briefly review our SN circuits designed by a qualitative-modeling-based approach. The next section summarizes the mathematical methods of qualitative neuronal modeling that are applied to SN design. Section 3 explains our analog and digital SNs and Section 4 concludes this review.

### 2. QUALITATIVE NEURONAL MODELING

In spiking neuronal cells, fast ionic currents such as the fast sodium and rectifying potassium currents are responsible for spike generation. Slower ionic currents such as the calcium currents and the potassium currents that are controlled by the intracellular calcium concentration modify the dynamics of the spike generation system. Various types of neuronal cells are known and each of them has its own combination of expressed ionic channels, which leads to a variety of neuronal activities. The mechanisms of these dynamical activities have been considerably elucidated from the perspective of non-linear dynamics (Rinzel and Ermentrout, 1998; Wang and Rinzel, 2003; Izhikevich, 2007).

#### 2.1. Spike Generation Systems

It is known that many spike generation systems can be projected onto two-variable systems without critically distorting their dynamics. Typically, the equations of the projected system are in the following form:

$$\frac{d\nu}{dt} = f\_{\nu}(\nu, n) + I\_{\text{stim}},\tag{1}$$

$$\frac{dn}{dt} = f\_n(\nu, n),\tag{2}$$

where v is the membrane potential and n is a variable that abstractly represents the activity of ionic channels. A stimulus current is represented by Istim. **Figures 1A,C** illustrate the phase plane of two typical spike generation systems. The v-nullcline, a set of points on which dv dt = 0, is N-shaped in both systems, which intersects the n-nullcline at three points in (**Figure 1A**) and at one point in (**Figure 1C**). In both cases, dv dt is negative (positive) above (below) the v-nullcline, and dn dt is negative (positive) above (below) the n-nullcline.

In **Figure 1A**, the leftmost intersection (equilibrium) (S) is a stable point, the middle (T) is a saddle point, and the rightmost (U) is an unstable point. Without any perturbation, the state point stays at (S), which corresponds to the resting state. If an excitatory instantaneous pulse stimulus is applied to the system, the system state is displaced horizontally rightward because v is the differential voltage of the membrane capacitance to which the stimulus current is directly injected. If the displacement is sufficiently large and the state point crosses the stable manifold of (T), it goes back to (S) along the longer branch of the unstable manifold of (T) by moving upward to the right and then leftward. This temporal increase in v is the mechanism of spike generation. Because the trajectory (the orbit of the state point) is attracted to the unstable manifold of (T), the shape of the spikes is not strongly dependent on the stimulus intensity. The threshold voltage for spike generation is determined by the stable manifold of (T) because if the state point does not cross it, the point goes back to (S) along the shorter branch of the unstable manifold by just moving leftward.

The v-nullcline is displaced upward by the application of an excitatory sustained stimulus Istim. Through this transition, (S) and (T) move toward each other, coalesce, and then disappear, which produces a stable limit cycle from the longer branch of the unstable manifold of (T). While the state point stays on the limit cycle, its v coordinate repeatedly increases and decreases, which is the mechanism of repetitive spiking. This process, the disappearance of two equilibria and the appearance of a limit cycle, is called a saddle-node on invariant circle bifurcation, and the critical value of Istim is a bifurcation point. **Figure 1D** shows a bifurcation diagram illustrating an overview of this transition. The horizontal axis is for Istim (the bifurcation parameter), and the dynamical structure of the phase plane for each value of Istim is projected onto the 1-d space of the vertical axis, v. Here, the limit cycle is represented by its maximum and minimum values of v. As is illustrated in **Figure 1B**, just above the bifurcation point, the limit cycle passes through a region near both of the v- and n-nullclines. Because both dv dt and dn dt are small in this region, the state point takes a long time to pass, which extends the period time of the limit cycle. As Istim is closer to the bifurcation point, this effect is stronger. It extends the period time, which diverges to infinity when Istim reaches the bifurcation point. This mechanism accounts for the Class I property in Hodgkin's classification.

If n is sufficiently faster than v, a stable limit cycle is produced via a homoclinic-loop bifurcation before (S) and (T) coalesce (**Figure 1E**). As the system is closer to the homoclinic-loop bifurcation point, the period of the limit cycle is extended to infinity by the extended passing time of a region near (T). Because the limit cycle appears before (S) disappears, the system is bistable in the range of Istim between the homoclinic-loop and saddle-node bifurcation points.

In **Figure 1C**, the unique equilibrium (S) is stable, which corresponds to the resting state. If the state point is displaced beyond the rising part of the v-nullcline by an excitatory instantaneous pulse stimulus, it starts moving rightward because dv dt > 0 below the v-nullcline. It then turns to left when it crosses the v-nullcline again. This is the mechanism of spike generation in this type of system. The magnitude of the spike, which is the maximum value of v on the spike's trajectory, is determined by its intersection with the v-nullcline, which is dependent on the starting point to which the state point is displaced by the stimulus. Thus, the spike shape is dependent on the stimulus intensity, which is referred to as graded response.

When Istim is a positive sustained stimulus, the v-nullcline is displaced upward, by which (S) is transferred upward to the right. At a critical value of Istim, the stability of this point is reversed via a subcritical Hopf bifurcation. In the bifurcation diagram (**Figure 1F**), the appearance of a set of stable and unstable limit cycles via another bifurcation, a fold bifurcation, is seen at a smaller value of Istim. Once (S) loses stability, the state point jumps to the stable limit cycle, and the system starts to spike repetitively. Because there is no dynamical structure that suppresses the velocity of the state point on the stable limit cycle down close to zero, the spiking frequency is always much higher than 0. This accounts for the Class II property in Hodgkin's classification. This system also has bistability composed of the resting state and the stable limit cycle.

#### 2.2. System with Slow Dynamics

Slow hyperpolarizing ionic currents activated by depolarization provide a negative feedback to the spike generation system, which is a most basic mechanism that maintains the spiking behavior "convergent." These currents play a role as inhibitory stimuli to the spike generation system that modify its dynamical structures. In a case that their time constants are similar, they can be projected even onto a single-variable system. It was elucidated that a simple system composed of a two-variable spike generation subsystem and a single-variable slow subsystem can explain the dynamics of several classes of neuronal activities including RS (**Figure 2A**), square-wave bursting (**Figure 2B**), elliptic bursting (**Figure 2C**), and low-threshold spiking (LTS). In this section, the dynamical structures of the first three classes are explained. Here, a slow subsystem is merged into the spike generation system in the previous section as follows:

$$\frac{d\nu}{dt} = f\_{\nu}(\nu, n, q) + I\_{\text{stim}},\tag{3}$$

$$\frac{dn}{dt} = f\_n(\nu, n),\tag{4}$$

$$\frac{dq}{dt} = f\_q(\nu, q). \tag{5}$$

**Figure 2D** illustrates the q–v plane of a system with a Class I spike generation subsystem. Because q works as an inhibitory stimulus to the spike generation subsystem, the dynamical structure is dependent on q in the opposite manner to Istim. Thus, the dynamical structure drawn in this figure is similar to a horizontal flip of the bifurcation diagram in **Figure 1D**. The q-nullcline illustrates the characteristics of the slow subsystem; above (below) it, dq dt <sup>&</sup>lt; 0 ( dq dt > 0). It has an intersection R with (S), which is a stable equilibrium point and corresponds to the resting state. In response to an appropriate step stimulus, this system produces RS, which is repetitive spiking whose frequency is initially high and then gradually decreases to a lower value (**Figure 2A**). Because the dynamical structure of the spike generation subsystem is displaced rightward by an excitatory sustained stimulus (**Figure 2E**), the state point is released at R at the onset of the step stimulus. It is attracted by the stable limit cycle, which is a repetitive spiking state. The slow variable q slowly increases because most parts of the limit cycle are above the q-nullcline. It converges to a value at which the increase and decrease in the portion of the limit cycle above and below the q-nullcline balance. The spiking frequency decreases as q increases because the spike generation subsystem is Class I, which produces spikes with a lower frequency in response to a stimulus closer to the bifurcation point. Some types of excitatory cells in the neocortex produce this type of activity. In contrast, some types of inhibitory cells generate faster spikes with a weaker frequency adaptation (fast spiking) (Harris and Shepherd, 2015). Such activity can be modeled by the same model with a weaker adaptation of the spike frequency or a Class II spike generation system.

**Figure 2F** illustrates the q–v plane of a system with the spike generation subsystem in **Figure 1E**. The dynamical structure of the spike generation subsystem is similar to its horizontal flip and the q-nullcline has no intersection with any stable states. If the state point is near (S) at some moment, it is attracted to (S). Because (S) is below the q-nullcline, q slowly decreases, and the state point moves leftward until (S) disappears via the saddlenode bifurcation. The system does not generate any spikes in this phase. The state point is then attracted to the stable limit cycle, which is the only stable state. Because the limit cycle is above the q-nullcline, q slowly increases, and the state point moves rightward until the limit cycle disappears via the saddleloop bifurcation. Repetitive firing is produced on the limit cycle in this phase. Then the state point is attracted to (S) again. The system repeats the alternation between the tonic firing and silent phases without any stimuli. This is the mechanism of squarewave bursting (**Figure 2B**). This class of neuronal activities is involved in life-supporting rhythm generation networks such as a respiratory rhythm generator and a heartbeat rhythm generator (Hill et al., 2001; Negro et al., 2001).

**Figure 2G** illustrates the q–v plane of a system with the spike generation subsystem in **Figure 1F**. As in the previous case, the state point is attracted to (S) if the state point is near to it. Because (S) is below the q-nullcline, the state point slowly moves leftward along (S) until it loses stability via the Hopf bifurcation. No spike is generated in this phase. Then the state point is attracted to the stable limit cycle. On the limit cycle, if the increase in q above the q-nullcline exceeds the decrease below, the state point slowly moves rightward, repetitively generating spikes. After the limit cycle disappears by the fold bifurcation, the state point is attracted to (S) again. The repeated alternation between these two phases is the mechanism of elliptic bursting (**Figure 2C**). This class of neuronal activities is observed in sleep spindles (Destexhe et al., 1993), which is a characteristic spiking pattern appearing in the thalamus during non-REM sleep.

#### 3. SILICON NEURON CIRCUITS

The core idea of our qualitative-modeling-based approach is to design an ideal silicon neuronal model that reproduces the dynamical structure of a target neuronal class by combination of "device-native" formulae (Kohno and Aihara, 2008b). For lowpower analog circuit implementation, the formulae of the V–I characteristic curves for compact and simple low-power analog circuit blocks can be selected. For digital circuit implementation, polynomials with the lowest order are appropriate because the multiplier is the circuit with the highest cost.

#### 3.1. Low-Power Analog Silicon Neuron

We developed a low-power analog SN circuit that can realize the Class I and II neuronal activities in Hodgkin's classification, RS, square-wave bursting, and elliptic bursting. The ideal model of this circuit was designed for implementation by subthreshold MOSFET circuits, which are typically chosen for low-power SN circuits. Because this circuit was intended to be a proof-ofconcept for the application of our qualitative-modeling-based approach to integrated circuits, elemental circuits were selected by attaching importance to stability and configurability instead of low-power consumption. The equations of the ideal model are constructed by combining the formulae of the sigmoidal V– I characteristic curves of differential-pair-based circuits and an integration operation with a leak that can be implemented by the Tau-cell circuit (van Schaik and Jin, 2003). The equations are

$$C\_\nu \frac{d\nu}{dt} = -g(\nu) + f\_m(\nu) - n - q + I\_a + I\_{\text{stim}},\tag{6}$$

$$\frac{dn}{dt} = \frac{f\_n(\nu) - n}{\pi\_n},\tag{7}$$

$$\frac{dq}{dt} = \frac{f\_q(\nu) - q}{\mathbf{r}\_q},\tag{8}$$

where v, n, and q represent the membrane potential, the abstracted activity of fast ionic channels, and the abstracted activity of slow ionic channels, respectively. The first two variables compose a fast subsystem, namely the spike generation system, and q provides a slow negative feedback to it. Parameters Cv, Ia, τn, and τ<sup>q</sup> are the membrane capacitance, a constant current, and the time constants of n and q, respectively. Functions fx(v) (x = m, n, q) and g(v) are the formulae of the idealized V–I characteristic curves of the differential pair and transconductance amplifier (**Figures 3A,B**) as follows:

$$f\_{\mathbf{x}}(\nu) = M\_{\mathbf{x}} \frac{1}{1 + \exp\left(-\frac{\kappa}{U\_T}(\nu - \delta\_{\mathbf{x}})\right)},\tag{9}$$

$$g(\nu) = S \frac{1 - \exp\left(-\frac{\kappa}{U\_T}(\nu - \theta\_\nu)/(1 + 1/\kappa)\right)}{1 + \exp\left(-\frac{\kappa}{U\_T}(\nu - \theta\_\nu)/(1 + 1/\kappa)\right)},\tag{10}$$

where U<sup>T</sup> and κ are the thermal voltage (approximately 26 mV at room temperature) and the capacitive-coupling ratio

block diagram is used to measure the nullclines experimentally. It is a transconductance amplifier that provides a negative feedback loop to the active terminal of Cv. When SWv is closed, it locks v near Vc, which is an input voltage of the amplifier. If SWq and SWn are open, its output current Iv compensates the current generated by the fm, gv, and Ia circuits. By scanning Vc in an appropriate range and measuring Iv, In, and Iq, the v-, n-, and q-nullclines are measured, respectively. The copied outputs of those currents are available for this purpose. This circuit is exploited to find the appropriate parameter voltages that replicate the dynamical structure in the ideal model, which are altered from their ideal values by fabrication mismatch and the transistors' second effects. Reprinted with modification from Kohno et al. (2014b).

(approximately 0.7 in our environment), respectively. Parameters Mx, δx, S, and θ are controlled by the externally applied voltages VM<sup>x</sup> , Vδ<sup>x</sup> , VS, and Vθ<sup>v</sup> in the figures. The v-nullcline of this model is given by n = fm(v) − g(v) + I<sup>a</sup> + Istim. Because both fm(v) and g(v) are sigmoidal curves and the latter is shallower than the former, it can be N-shaped by an appropriate choice of the parameters.

As is drawn in the block diagram of the overall circuit that solves the system equations (**Figure 3D**), Equation (6) is integrated by C<sup>v</sup> whose differential voltage corresponds to v and Equations (7) and (8) are solved using the Tau-cell (**Figure 3C**) whose ideal equation is

$$\frac{dI\_{\rm out}}{dt} = \frac{I\_{\rm in} - I\_{\rm out}}{C U\_T / I\_{\rm}},\tag{11}$$

where C is the capacitance in the circuit, and Iτ is a parameter current by which the time constant is arranged. The solutions of Equations (7) and (8) are given by the output current of the Tau-cell circuits (blue boxes in the block diagram) whose input terminals are connected to the outputs of the fx(v) circuits (x = n, q). The output current of the lower (upper) Tau-cell, I<sup>n</sup> (Iq), represents n (q). Parameter currents I<sup>τ</sup> <sup>x</sup> (x = n, q) and I<sup>a</sup> are generated by integrated V–I converters that are controlled by the externally applied voltages, Vτn, Vτq, and Va, respectively.

In this review, we focus on the square-wave bursting mode. In this mode, the parameters of the model are selected so that the dynamical structures of the fast subsystem resemble those in **Figure 1A**. **Figure 4A** illustrates an example of the v–n phase plane on which the v- and the n-nullclines are configured for this mode. The reversed N-shape of the v-nullcline is produced by a combination of a rising sigmoidal curve, fm(v), and a shallower falling sigmoidal curve, −g(v). Thus, M<sup>m</sup> is increased to make its rising phase steeper and S is increased to make its falling phases steeper. These factors control the deepness of its U-shaped and reversed U-shaped regions. The threshold voltage for spike generation depends on the former, which controls the distance between (S) and (T). Because the magnitude of the spikes is suppressed by the latter, the former is generally coordinated to be deeper than the latter to obtain sufficiently high spikes in comparison to the threshold, which can be realized by selecting a smaller value for θ<sup>v</sup> than δm. The spike height is also boosted by increasing the time constant of n, which slightly increases the spike period as well. The actual spike height can be estimated by drawing the unstable manifolds of (T) whose maximum v gives the minimum height. In **Figure 4A**, the spike height is estimated to be at least 20 mV. In this figure, the longer branch of the unstable manifold of (T) is pulled back to (T) because a relatively small value is selected for the time constant of n. In this situation, as described in the previous section, the system undergoes a saddle-loop bifurcation instead of a saddle-node on invariant circle bifurcation in response to the increase in the stimulus current (**Figure 1E**). Because the slow variable q is an inhibitory stimulus current to the v–n subsystem, this bifurcation structure appears on the v–q plane in a horizontally flipped manner (**Figure 4B**).

The appropriate selection of M<sup>q</sup> and δ<sup>q</sup> places the q-nullcline so that it separates the stable limit cycle and the stable equilibrium (S), which reproduces the dynamical structure in the square-wave bursting illustrated in **Figure 2F**. The position of the q-nullcline

of the v–n phase plane of the fast subsystem when q is fixed at 60 pA. (B) Bifurcation diagram of the fast subsystem whose bifurcation parameter is q. (C,D) Activities of the membrane potential v. Reprinted with modification from Kohno and Aihara (2010) (some parameters are modified).

Kohno et al. Qualitative-Modeling-Based SNs

relative to the stable states of the fast subsystem is a factor that controls the number of spikes in a burst. This dependence was theoretically elucidated in Wang (1993) using the Hindmarsh– Rose model, which is a qualitative square-wave bursting model. When the slow variable's nullcline is close to the limit cycle, tonic spiking is generated. As it departs from the limit cycle, the tonic spiking becomes chaotic via a period doubling bifurcation cascade. Then regular bursting appears and the number of spikes in a burst decreases one by one down to single-spike bursting. At those transitions, chaotic bursting can be observed if the number of spikes is relatively large. This transition was also observed in an ionic-conductance model of the pancreas β cell (Chay and Rinzel, 1985), in which the maximum conductance of a slow current was swept as the bifurcation parameter. The same transition appears in our circuit model when M<sup>q</sup> is swept. This parameter abstractly represents the activity of the slow ionic current which corresponds to the bifurcation parameter in Chay and Rinzel (1985). Examples of regular bursting and chaotic bursting are shown in **Figures 4C,D**.

We fabricated this circuit using a Taiwan Semiconductor Manufacturing Company (TSMC) 0.35 µm mixed-signal CMOS process (**Figure 5A**). The parameter voltages were tuned following the procedure explained in the caption for **Figure 5** on the basis of the dynamical structures in **Figure 4**. In **Figure 5C**, a typical bursting activity observed in the circuit experiments is shown. Its activity was always unstable and could not be stabilized by tuning parameters. In Kohno and Aihara (2011, 2013), we pointed out that this fluctuated behavior arises from the intrinsic dynamical structure of square-wave bursting, i.e., the sensitivity to the initial conditions near the saddle-loop bifurcation. By extending the time constant of q (decreasing Iτq), we could obtain a bursting pattern with a longer period that is similar to the activity of autonomous bursting cells in the pre-Bötzinger complex that generate the respiratory rhythm (Negro et al., 2001). In this case, the bursting activity is more stable than that in (c) because the extended time constant of q makes the trajectory of the state point pass closer to the stable equilibrium (S) and its sojourn time near the saddle-loop bifurcation point shorter. In addition to square-wave bursting, we could also realize Class I and II, RS, and elliptic bursting. For any settings, the power consumption of this circuit including the bias-voltage generators for the Tau-cell circuits did not exceed 72 nW.

#### 3.2. Ultralow-Power Analog Silicon Nerve Membrane

The power consumption of the above circuit is one order of magnitude higher than low-power-oriented leading-edge circuits (Basu and Hasler, 2010; Brink et al., 2013b; Qiao et al., 2015). We developed SN circuitry to attain a lower power consumption that is comparable to these works. A twovariable model that supports the Class I and II neuronal activities was designed on the basis of this circuitry to evaluate its practicality (Kohno and Aihara, 2014a). Its ideal model is given by

$$C\_{\nu}\frac{d\nu}{dt} = f\_{\nu}(\nu) - g\_{\nu}(\nu) + I\_{a\nu} - r(n) + I\_{\text{stim}},\tag{12}$$

and equipped with a copied output of Istim, this bifurcation diagram can be translated to the q–v plane. The parameter voltages related to the q-nullcline are selected

so that the total q-v plane resembles that of the ideal model. Finally, Vτq is tuned to obtain bursting activity with the appropriate period.

$$\mathcal{C}\_n \frac{dn}{dt} = f\_n(\nu) - \mathcal{g}\_n(\nu) + I\_{an} - r(n),\tag{13}$$

where v and n represent the membrane potential and the abstracted activity of fast ionic channels, respectively. Parameters C<sup>v</sup> and C<sup>n</sup> are the capacitances that are determined at circuit fabrication, Iav and Ian are parameter currents, and Istim is the stimulus current. Functions fx(v), gx(v), and r(n) (x = v, n) are monotonic increasing sigmoidal functions that correspond to the idealized V–I characteristic curves of the elemental circuits shown in **Figures 6A–C**. Their equations are

$$f\_{\mathbf{x}}(\nu) = \frac{M\_{\mathbf{x}}}{1 + \exp\left(-\frac{\kappa}{U\_{\Gamma}}(\nu - \delta\_{\mathbf{x}})\right)},\tag{14}$$

$$\lg\_{\mathbf{x}}(\nu) = \, I\_{\mathrm{0P}} \sqrt{\frac{R\_{\mathbf{x}20} \exp\left(\frac{\kappa}{U\_{\mathrm{T}}} \theta\_{\mathbf{x}}\right)}{1 + R\_{\mathbf{x}21} \exp\left(-\frac{\kappa}{U\_{\mathrm{T}}} (\nu - \theta\_{\mathbf{x}})\right)}},\tag{15}$$

$$r(n) = I\_{0\text{P}} \sqrt{\frac{\exp\left(\frac{\kappa}{U\_T}\theta\_r\right)}{1 + \exp\left(-\frac{\kappa}{U\_T}(\nu - \theta\_r)\right)}}.\tag{16}$$

The parameters in these equations are explained in the caption for **Figure 6**. A block diagram of a circuit that solves this model is shown in **Figure 6D**. In this circuit, v and n are coded by the voltage difference between Vdd and the voltage across capacitors C<sup>v</sup> and Cn, respectively. Blocks fx(v) and gx(v) (x = v, n) correspond to the circuits in (a) and (b), respectively. For gv(v), the transistors in the shadowed region are omitted to simplify the circuit. The two blocks of r(n) correspond to a single circuit of (c).

Equations (12) and (13) are transformed as follows by defining the variable n˜ ≡ r(n):

$$\mathcal{C}\_{\nu}\frac{d\nu}{dt} = f\_{\nu}(\nu) - g\_{\nu}(\nu) + I\_{a\nu} - \tilde{n} + I\_{\text{stim}},\tag{17}$$

$$\mathcal{C}\_n \frac{d\tilde{n}}{dt} = r'(n)(f\_n(\nu) - g\_n(\nu) + I\_{an} - \tilde{n}).\tag{18}$$

By using the two voltage clamp amplifiers, VAV and VAN, in the block diagram, the v- and n˜-nullclines are measured and r ′ (n) is evaluated (see the caption for **Figure 6**).

The major improvement of this circuitry from the previous SN is a reduction in the static current consumption. In the fx(v) circuit, M7, M8, M10, and M11 are used to extend the output current range without increasing the tail current I<sup>s</sup> . The current consumption of the cascode circuitry in gx(v) is equal to the output current, whereas the transconductance amplifier in the previous gx(v) circuit constantly consumes its maximum current. The integration of n is performed by a capacitor instead of the Tau-cell. The Tau-cell is an easy-to-use current-mode integrator with a constant time constant, which supports a wide range of input and output currents. However, it requires additional circuits that generate I<sup>τ</sup> , 2I<sup>τ</sup> , and Vofst. The currents required to drive these circuits are cut off by the direct integration of the currents into a capacitor. In this case, the acceptable range of the variables is limited by the range of rn(n), on which the time constant of n is dependent. The above nullcline-drawing function helps to find the appropriate parameter values under these limitations; once rn(n) is specified by the time constant requirement of n, the appropriate dynamical structure for the target neuronal class can be constructed within the acceptable range of the variables by tuning the other parameters utilizing the nullcline drawing function.

By a similar parameter tuning procedure to that for the fast subsystem in the previous SN circuit, we found the parameter values for the Class I and II modes. **Figure 7** shows the simulation results obtained using the Spectre circuit simulator with the TSMC 0.25 µm mixed-signal CMOS process development kit. Capacitances C<sup>v</sup> and C<sup>n</sup> are implemented by metal–insulator– metal capacitors (MIMCAPs) with capacitances of 1.5 and 2.0 pF, respectively. In the pulse stimulus responses (**Figure 7A**), the height of spike is dependent on the intensity of the stimulus (graded response) in the Class II mode (lower plot), whereas the dependence is weak in the Class I mode (upper plot). In the sustained stimulus response (**Figures 7B,C**), the spike frequency can be reduced close to 0 by applying a sufficiently weak stimulus in the Class I mode [(B)], whereas spike generation is terminated before the spike frequency reaches close to 0 in the Class II mode [(C)]. In both settings, the power consumption increases with the spike frequency and is less than 3.2 nW when the spike frequency is less than 70 Hz.

#### 3.3. A Silicon Neuronal Network by Digital Arithmetic Circuits

Generally, the power consumption of a dynamical digital circuit is higher than that of the subthreshold analog circuits used in the previous sections. However, the continuous evolution of the fabrication process is lowering the power consumption. In Merolla et al. (2014), a combination of ultrafine processes and technologies such as asynchronous and near-threshold logic realized low-power silicon neuronal networks whose power consumption per neuron is only one order of magnitude higher than the lowest-power analog silicon neurons. Reduced power consumption these days is facilitating a fascination with the scalability and stability of digital circuits.

In Li et al. (2012, 2013), we developed a silicon neuronal network in an FPGA based on our qualitative-modeling-based silicon neuronal model for digital arithmetic circuits (Kohno and Aihara, 2007). **Figure 8A** is a block diagram of its basic unit, the silicon neuronal network module (SNNM), which executes a calculation related to 16 SNs including spike-timing-dependent learning. Larger-scale networks are constructed by connecting more than one of these modules in parallel.

The digital spiking silicon neuron (DSSN) Unit calculates the silicon neuronal model designed with the same principle as above so that it can be implemented with reduced hardware resources by using the minimum number of multipliers. Its equations are given by

$$\frac{d\nu}{dt} = \frac{\phi}{\tau} \left( f(\nu) - n + I\_0 + I\_{\text{stim}} \right), \tag{19}$$

$$\frac{dn}{dt} = \frac{\mathcal{g}(\nu) - n}{\tau},\tag{20}$$

FIGURE 6 | Circuits of our ultralow-power analog silicon neuron. (A–C) Elemental circuits and (D) block diagram. In Equations (14)–(16), I0P is the current-scaling parameter of common-sized PMOS transistors [M1 in (A), M1 to M3 in (B), and all of the transistors in (C)]. Parameter Mx is the drain current of M1 in (A), which is controlled by Vb. Parameters δx (x = v, n) and θx (x = v, n, r) correspond to Vdd - Vdlt and Vdd - Vm (Vdd = 1.0 V), respectively. Parameter Rx20 is 1 (0.5) when M6 is on (off) and M7 is off (on) in panel (B). Parameter Rx21 is 1 (2) when M8 is on (off) and M9 is off (on). These two parameters are used to shift the curve of gx(v) horizontally. The two voltage clamp amplifiers, VAV and VAN, in (D) are for drawing the v- and n˜-nullclines and evaluating r ′ (n) in Equations (17) and (18). The output of these transconductance amplifiers is 0 when the two input voltages are the same and positive (negative) when the "+" input voltage is higher (lower) than that of the "−" input. When they are activated and the rn(n) block is deactivated, their outputs are equal to the sums of fx(v), gx(v), and Iax (x = v, n). By scanning Vcv and maintaining Vcn constant, the nullclines are measured. Similarly, when the rn(n) block is activated and the other blocks are deactivated, the output of VAN is equivalent to rn(n). By scanning Vcn, the dependence of rn(n) on n is measured, from which r ′ (n) can be evaluated. Reprinted with modification from Kohno and Aihara (2014a).

FIGURE 7 | Circuit simulation results of our ultralow-power analog silicon neuron. (A) Responses to a pulse stimulus in the Class I mode (upper) and the Class II mode (lower). (B,C) Spiking frequency and power consumption in response to a sustained stimulus. The horizontal axes represent the intensity of the stimulus. (B) is for the Class I mode and (C) is for the Class II mode. Reprinted with modification from Kohno and Aihara (2014a).

$$f(\nu) = \begin{cases} a\_{fn} (\nu - b\_{fn})^2 + c\_{fn} \text{ ( $\nu < 0$ )}, \\ a\_{\not\mathbb{p}} (\nu - b\_{\not\mathbb{p}})^2 + c\_{\not\mathbb{p}} \text{ ( $\nu \ge 0$ )}, \end{cases} \tag{21}$$

curve utilized in the Hebbian-type learning in our digital silicon neuronal network. Reprinted with modification from Li et al. (2013).

$$g(\nu) = \begin{cases} a\_{\mathfrak{g}n} (\nu - b\_{\mathfrak{g}n})^2 + c\_{\mathfrak{g}n} \ (\nu < r\_{\mathfrak{g}}), \\ a\_{\mathfrak{g}p} (\nu - b\_{\mathfrak{g}p})^2 + c\_{\mathfrak{g}p} \ (\nu \ge r\_{\mathfrak{g}}), \end{cases} \tag{22}$$

where v and n are variables with no unit that represent the abstracted membrane potential and ionic current activity, respectively. Parameters φ and τ configure the time scale of the model's activity. A stimulus input is represented by Istim. The other parameters determine the shape of the nullclines on the v–n phase plane; n = f(v)+I<sup>0</sup> for the v-nullcline and n = g(v) for the n-nullcline. The N-shaped v-nullcline is realized by the piecewise quadratic function f(v) instead of a cubic function, which reduces the number of multiplications between variables. Multiplication between a constant and a variable can be implemented by small numbers of shifters and adders if the number of active bits in the constant's binary expression is small. By the parameter tuning procedure similar to our analog SNs, parameter sets that realize the Class I and II activities were found with which this model is solved by Euler's method with fixed-point operations.

The Synapse Unit calculates the following silicon synapse model by Euler's method with fixed-point operations.

$$\frac{dI\_s}{dt} = \begin{cases} \alpha \left( 1 - I\_s \right) \text{ when } \nu \ge 0, \\ -\beta \left| I\_s \right. \quad \text{when } \nu < 0, \end{cases} \tag{23}$$

where I<sup>s</sup> is the post-synaptic stimulus received by other SNs. Parameters α and β determine the time constants of I<sup>s</sup> in the rising and falling phases, respectively. This model was developed on the basis of the kinetic models of chemical synapses (Destexhe et al., 1998) so that it can transmit the analog information of the graded responses in Class II neurons.

The Accumulator Unit calculates the sum of I<sup>s</sup> given by other SNNMs as follows:

$$I\_{\rm ss}^{\dot{l}} = \mathcal{c} \sum\_{i} \mathcal{W}\_{\dot{\rm j}i} I\_{\rm s}^{\dot{l}},\tag{24}$$

where c is a scaling parameter, Wji is the synaptic weight from SN i to SN j, and I i s is I<sup>s</sup> generated by the Synapse Unit for neuron i. The sum of I j ext and I j ss is given to the SN's stimulus input, Istim.

The Learning Unit supports spike-time-dependent learning with exponential-based rules. In Li et al. (2013), we implemented the following Hebbian-type rule:

$$
\Delta W\_{\vec{\mu}} = A\_+ \cdot \exp\left(\frac{-|\Delta t\_{\vec{\mu}}|}{\mathfrak{r}\_+}\right) - A\_- \cdot \exp\left(\frac{-|\Delta t\_{\vec{\mu}}|}{\mathfrak{r}\_-}\right), \tag{25}
$$

where 1Wji is the modification applied to Wji, and 1tji is the time between the two nearest spikes of neuron j and neuron i. The time of a spike here is defined as the time when v exceeds 0. Parameters A+ and A− configure the amplitude of the learning curve and τ+ and τ− specify its time scale. Mexicanhat-type learning curves can be realized by selecting appropriate parameter values (**Figure 8B**).

By implementing 16 SNNMs in an FPGA chip, we constructed an all-to-all connected silicon neuronal network composed of 256 SNs. To verify its functionality, we executed associative memory tasks in which the four patterns shown in **Figure 9A** are stored. A pattern comprises 256 (16 × 16) pixels, each of which has a value of 1 or −1. In the figure, a black (white) pixel has value of 1 (−1). Firstly, these patterns were stored by correlation learning without using the Learning Unit as follows:

$$\boldsymbol{W}\_{ij} = \begin{cases} \frac{1}{4} \sum\_{\mu=1}^{4} \boldsymbol{\pi}\_i^{\mu} \boldsymbol{\pi}\_j^{\mu} & \text{when } i \neq j, \\ 0 & \text{when } i = j, \end{cases} \tag{26}$$

where x u i represents the value of the ith pixel in pattern u. In the retrieval process, all SNs are repetitively spiking owing to the application of an appropriate sustained stimulus I i ext for all i. Their initial spiking phases are arranged by a short positive external input applied before the sustained stimulus only to the SNs that correspond to the pixel with a value of 1 in the input pattern. The input patterns were generated by flipping the values of randomly selected pixels in a stored pattern. **Figure 9B** plots examples of observed retrieval processes, in which the time step for numerical integration is 375 µs. In the left column, Mu, an index that reflects the correlation between the current spiking pattern and the uth pattern, is plotted. This value is 0 when the spiking pattern has no relation with the uth pattern and approaches 1 as the pattern matches to it. In the right column, the phase synchronization index (PSI) that reflects the degree of synchronization is plotted. It is 0 when the SNs are spiking fully asynchronously and approaches 1 as their spike timing is synchronized. In the upper row, 10% of the pixels in pattern (1) are flipped and applied as inputs. In this case, M<sup>1</sup> quickly increases and remains near 1, which indicates the successful retrieval of pattern (1). In the lower row, 40% of the pixels are flipped. Then, none of the values of M<sup>u</sup> remain close to 1, which indicates that no pattern was retrieved. The PSI plotted in the right column stayed near 1 when a correct pattern was retrieved but not when no pattern was retrieved. We executed 100 retrieval processes: 10 different levels of flipped (error) pixels from 5 to 50% in 5% increments and 10 patterns for each error level. The red and blue plots in **Figure 9C** show the rate of successful retrieval when the SNs are in the Class I and II modes, respectively. The network could retrieve a correct pattern from a

FIGURE 9 | Auto-associative memory tasks executed in our all-to-all digital silicon neuronal networks. (A) Stored patterns. Each pattern is composed of 256 (16 × 16) pixels with a value of 1 or −1. Black (white) pixels have a value of 1 (−1). Reprinted with modification from Li et al. (2012). (B) Examples of the transition of Mu and PSI when the input pattern has 10 and 40% flipped pixels. Reprinted with modification from Li et al. (2012). (C) Error recovery performance when the patterns are stored by correlation learning and the SNs are in the Class I mode (red) and in the Class II mode (blue). The yellow plot is for the case where the patterns are stored by Hebbian-type spike-timing-dependent learning and the SNs are in the Class I mode. The horizontal axis is the ratio of the flipped pixels to the total number of pixels (the error level). The vertical axis represents the rate of successful retrieval rate. For each error level, 10 trials were executed. Reprinted with modification from Li et al. (2013).

larger number of errors when the SNs are in the Class II mode than when they are in the Class I mode. This indicates that the spiking dynamics may play important roles in auto-associative memory tasks.

Second, the four patterns were stored by activating the Learning Unit as follows. The stored patterns were applied to the network in the sequence of (1), (1), (1), (1), .. (2), (2), .. (3), (3), .. (4), (4), .. (1), (1), .., where (n) means the reversed pattern of (n). A couple of a stored pattern and its reversed pattern was repeated eight times in a block. This sequence was applied until one of the values of Wij reached 1 or -1 by the modification in Equation (26). In the retrieval process, the Learning Unit was deactivated and input patterns with errors were applied in the same way as above. The yellow plot in **Figure 9C** shows the rate of successful retrieval when the SNs are in the Class I mode. The error recovery performance exceeded both results with correlation learning. In our preliminary results with SNs in the Class II mode, this performance was further boosted (not shown in the figure). The spiking dynamics may also be important in auto-associative memory with spike-timingdependent learning rules.

#### 4. DISCUSSION

As reviewed above, our silicon neuron circuits can realize different classes of neuronal activities by selecting appropriate parameter values and their characteristics can be modified by finely tuning the parameters as shown in **Figure 5D**. This high configurability is advantageous not only for bio-silico hybrid systems but also for constructing " field-programmable" silicon neuronal networks in which each SN can be reconfigured after fabrication or each SN autonomously obtains appropriate dynamical properties on the basis of the history of stimulus inputs as in the brain. This high configurability arises from the fact that the activity of many neuronal classes can be explained using common dynamical structures that are reproduced in our models by a combination of implementation-efficient formulae. In contrast, the circuitry is simplified by supporting only one neuronal class in the non-I&F-based SN circuits developed by a similar approach (Basu and Hasler, 2010; Brink et al., 2013b). These circuits realize ultralow power consumption down to several nanowatts at the expense of configurability. In their SN network systems, the configurability is supplemented by accommodating a sufficiently large SN circuit pool, in which the appropriate SNs for a desired network are activated. Our circuit in Section 3.2 supports both Class I and II neuronal activities and consumes a similar power; however, it has the drawback of high configurability. The circuit has to be configured appropriately by tuning a number of parameter values, and additional circuits are required for storing parameter values. The complexity of the configuration process is solved by parameter tuning procedures that utilize the nullcline drawing circuits as explained in detail in Section 3.1. This procedure is still not straightforward, but all of the students who worked on our circuit learned to be able to finish the tuning procedure within several tens of minutes. For a large-scale silicon neuronal network, this procedure has to be automated. It may be done by metaheuristic approaches similar to those utilized in Grassia et al. (2011). The power consumption and area occupied by additional circuits for storing parameter values may be reduced by evolving non-volatile memory technologies such as memristors.

In digital silicon neuronal networks, the accumulation of synaptic inputs consumes a considerably larger amount of hardware resources than SN circuits. Thus, the compactness of the SN circuit is not a major issue. The advantage of our circuit is that its model is non-I&F-based and thus can mimic the spikegeneration-related properties in neuronal activities more finely than I&F-based circuits. One of these properties is the graded response in Class II neurons. Because the graded response is found in the brain, as mentioned in the introduction, there is possibility that it plays some roles in information processing in the brain. Our silicon neuronal network model intends to provide a platform in which a wide variety of neuronal activities including the dynamics of spike generation is qualitatively reproduced without a major increase in hardware resource consumption. For this goal, our SN model is being expanded so that it can realize more classes of neurons including RS, LTS, and IB as well as autonomous bursting supported by our analog SN. It has four variables (two original and two additional slow variables) but still can be solved by one multiplication per a numerical integration step. The details of this model is explained in Nanami and Kohno (2016).

A goal of our analog silicon neuronal circuits is to establish an ultralow-power general-purpose silicon neuronal network platform that will be applicable to neuromimetic computing when the mechanism of information processing in the nervous system is elucidated. We expect that it has an advantage also in the application to large-scale neuronal network simulators (Schemmel et al., 2010; Stromatias et al., 2013) and brain-prosthetic devices such as an artificial hippocampus (Berger et al., 2012; Hampson et al., 2013; Song et al., 2015), an artificial cerebellum (Hogri et al., 2015), and an artificial prefrontal cortex (Hampson et al., 2012) because our circuits meet their requirements of a low power consumption and the ability to mimic various complex neuronal activities finely. Construction of such systems may contribute to the elucidation of the brain's mechanisms by the "analysis by synthesis" approach. Our digital silicon neuronal network platform is also applicable to neuromimetic computing and large-scale neural network simulation. It consumes more power than analog circuits but has advantage in scalability.

### AUTHOR CONTRIBUTIONS

MS performed circuit experiments on analog silicon neurons. JL implemented, simulated, and performed experiments on digital silicon neuronal networks. TN expanded the digital silicon neuron model. KA checked the mathematical correctness of the research. TK did all other things.

### FUNDING

The works reviewed in this article were partially supported by the JST PRESTO program, the JST CREST program, and a JSPS Grant-in-Aid for scientific Exploratory Research Number 25240045.

#### REFERENCES


#### ACKNOWLEDGMENTS

The works reviewed in this article were supported by the VLSI Design and Education Center (VDEC), The University of Tokyo with the collaboration with Cadence Design Systems, Inc. and Mentor Graphics, Inc.


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2016 Kohno, Sekikawa, Li, Nanami and Aihara. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Detection of Movement Related Cortical Potentials from EEG Using Constrained ICA for Brain-Computer Interface Applications

Fatemeh Karimi <sup>1</sup> , Jonathan Kofman<sup>1</sup> , Natalie Mrachacz-Kersting<sup>2</sup> , Dario Farina<sup>3</sup> and Ning Jiang<sup>1</sup> \*

<sup>1</sup> Department of Systems Design Engineering, Faculty of Engineering, University of Waterloo, Waterloo, ON, Canada, <sup>2</sup> Department of Health Science and Technology, Aalborg University, Aalborg, Denmark, <sup>3</sup> Neurorehabilitation Engineering Department of Bioengineering, Imperial College London, London, United Kingdom

The movement related cortical potential (MRCP), a slow cortical potential from the scalp electroencephalogram (EEG), has been used in real-time brain-computer-interface (BCI) systems designed for neurorehabilitation. Detecting MPCPs in real time with high accuracy and low latency is essential in these applications. In this study, we propose a new MRCP detection method based on constrained independent component analysis (cICA). The method was tested for MRCP detection during executed and imagined ankle dorsiflexion of 24 healthy participants, and compared with four commonly used spatial filters for MRCP detection in an offline experiment. The effect of cICA and the compared spatial filters on the morphology of the extracted MRCP was evaluated by two indices quantifying the signal-to-noise ratio and variability of the extracted MRCP. The performance of the filters for detection was then directly compared for accuracy and latency. The latency obtained with cICA (−34 ± 29 ms motor execution (ME) and 28 ± 16 ms for motor imagery (MI) dataset) was significantly smaller than with all other spatial filters. Moreover, cICA resulted in greater true positive rates (87.11 ± 11.73 for ME and 86.66 ± 6.96 for MI dataset) and lower false positive rates (20.69 ± 13.68 for ME and 19.31 ± 12.60 for MI dataset) compared to the other methods. These results confirm the superiority of cICA in MRCP detection with respect to previously proposed EEG filtering approaches.

Keywords: brain-computer interface (BCI), movement related cortical potential (MRCP), constrained independent component analysis (cICA), electroencephalogram (EEG), spatial filters

## INTRODUCTION

The movement-related cortical potential (MRCP) is a low frequency (0–5 Hz) negative shift in the electroencephalogram (EEG) signal, which has recently been used as an EEG modality for real-time brain computer interface (BCI) applications, particularly in neuromodulation systems (Mrachacz-Kersting et al., 2016). The ability to detect MRCPs with high accuracy and short latency (usually shorter than 300 ms) on a single trial basis is crucial for these applications. Specifically, the high demand on temporal precision has been shown to be fundamental in efficiently inducing plasticity in neurorehabilitation applications (Mrachacz-Kersting et al., 2012). Improvement in accuracy and

#### Edited by:

Paolo Bonifazi, Tel Aviv University, Israel

#### Reviewed by:

Ricardo Chavarriaga, École Polytechnique Fédérale de Lausanne, Switzerland Jianjun Meng, University of Minnesota, United States

> \*Correspondence: Ning Jiang ning.jiang@uwaterloo.ca

#### Specialty section:

This article was submitted to Neuroprosthetics, a section of the journal Frontiers in Neuroscience

Received: 13 October 2016 Accepted: 07 June 2017 Published: 30 June 2017

#### Citation:

Karimi F, Kofman J, Mrachacz-Kersting N, Farina D and Jiang N (2017) Detection of Movement Related Cortical Potentials from EEG Using Constrained ICA for Brain-Computer Interface Applications. Front. Neurosci. 11:356. doi: 10.3389/fnins.2017.00356 latency of single-trial MRCP detection is therefore a relevant challenge. The amplitude of the MRCP is typically between 5 and 30 µV and therefore easily masked by other brain activities (Wright et al., 2011). Moreover, low frequency motion artifacts and the electrooculogram (EOG) have frequency bandwidths similar to the MRCP, but with much greater magnitudes. Thus, extracting a single trial MRCP from an EEG signal with high accuracy and minimal latency in real-time is a challenging task.

Spatial filtering is one of the most commonly used EEG signal processing approaches for artifact removal and improving the detection accuracy of cortical potentials. The MRCP has a welldefined spatial distribution, being located directly over the scalp area of the corresponding primary motor cortex region. For example, the MRCP accompanying an ankle dorsiflexion task is most pronounced over the apex (Cz of 10–20 montage). The most common spatial filters used in EEG-based BCI systems are the Common Spatial Pattern (CSP) (Blankertz et al., 2008), Laplacian spatial filter (LAP) (McFarland et al., 1997; Xu et al., 2014a,b), and Independent Component Analysis (ICA) (Bell and Sejnowski, 1995; Cardoso, 1999). CSP decomposes multi-channel EEG signals into distinct spatial patterns by solving a generalized eigenvalue problem. This method has been widely used to extract motor imagery-based BCIs, particularly in sensory-motor rhythm (SMR) (Ramoser et al., 2000; Blankertz et al., 2008) and has also been tested preliminarily in MRCP detection (Niazi et al., 2011). However, the performance of CSP is very sensitive to outliers, which are inevitable in real-time BCI applications (Blankertz et al., 2008). LAP calculates the second derivative of the instantaneous spatial voltage distribution for each electrode location, and thereby emphasizes the activity originating in radial sources immediately below the electrode (McFarland et al., 1997). LAP has been applied in MRCP detection (Xu et al., 2014a,b). ICA-based spatial filters have been also successfully used in a variety of EEG signal processing applications, such as artifact reduction and source localization (Xu et al., 2004; Jiang et al., 2015). However, there are limitations associated with the implementation of ICA, especially for real-time applications, as it requires manual selection of the desired components from the estimated sources.

The constrained ICA (cICA), also known as one-unit ICA (Zhang, 2008), is a recent approach introduced to overcome the manual intervention limitation of ICA. cICA is a spatial filter extended from ICA that uses a reference signal to automatically extract only the desired source, without requiring the manual selection procedure of traditional ICA-based methods. cICA has recently been applied for EEG signal processing applications (James and Gibson, 2003; Joshua and Rajapakse, 2005) and has been shown to be successful in extracting event-related cortical potentials (ERP), such as the P300 (Spyrou and Sanei, 2006; Lee et al., 2013), as well as removing ocular artifacts (Huang et al., 2011); however, cICA has not been used previously for the detection of MRCPs. In this paper, we present for the first time, the application of cICA for MRCP detection, including a systematic investigation of the efficacy of cICA in singletrial MRCP detection, and comparison of cICA performance with the previously proposed CSP, LAP, Infomax (Bell and Sejnowski, 1995), and JADE (Cardoso, 1999). The performance of these filters was evaluated both with metrics based on the morphology of the MRCP and on the detection accuracy. For quantifying detection accuracy, the filtered EEG was classified with the previously proposed Locality Preserved Projection (LPP) followed by Linear Discriminator Analysis (LDA) (Xu et al., 2014a).

## MATERIALS AND METHODS

## Data Acquisition

#### Participants

The data used in the current study are part of the dataset previously reported in Jochumsen et al. (2015). In the following, the experimental protocol is briefly described for clarity. The full details of the experimental procedure can be found in Jochumsen et al. (2015). Twenty-four healthy participants (7 female and 17 male 27 ± 4 years old) without any prior BCI experience participated in the experiment. All procedures were approved by the local ethics committee (N-20130081), and the participants gave their written informed consent before the experiment.

#### Experimental Procedures

The participants were seated in a chair, relaxed and with their foot fixed to a pedal. During the experimental session, the participants were instructed to perform ankle dorsiflexion following a visual cue display on a computer screen that was located at a distance of 1.5 m in front of them. The cue was presented with a custom-made program (Knud Larsen, SMI, Aalborg University) which provides the instructions by displaying Ready, Focus, and Task commands in 8–10 s intervals. The 24 participants were divided into two groups. The first 12 participants (Group 1) were asked to perform actual dorsiflexion (motor execution, ME), while the remaining 12 participants (Group 2) were asked to perform only motor imagery (MI) of the movement. Four contraction types were performed: fast contraction targeted at 20% maximum voluntary contraction (MVC), fast contraction targeted at 60%, slow contraction targeted at 20%, and slow contraction targeted at 60% MVC. In the visual cue, a moving cursor showed when and how fast the subject should perform the task. For each of the four contraction types, each participant performed approximately 50 trials of the ankle dorsiflexion task (ME or MI). The order of contraction types was randomized for both ME and MI sessions. The motor tasks were separated randomly between 8 to 10 s. For the purpose of this study, we only analyzed and report the results using the trials of fast 20% MVC, for both ME and MI tasks. For this particular task, the instruction shown on the screen for Ready, Focus, and Task commands lasted between 4–6, 3, and 1 s, respectively. The subjects focused for 3 s, followed by the execution phase 0.5 s to reach 20% MVC, and the contraction was maintained for 0.5 s, after which a rest period was given (between 4 to 6 s).

#### EEG Recording

A multichannel EEG electrode system (32 Channel Quick-Cap, Neuroscan) and an EEG Amplifier (Numaps Express, Neuroscan) were used according to the international 10–20 system to obtain EEG signals. Ten electrodes placed at standard 10–20 positions FP1, F3, Fz, F4, C3, Cz, C4, P3, Pz, and P4 were used to collect EEG data at a sampling rate of 500 Hz. The reference electrode was located on the right ear lobe. All analyses presented below were performed offline.

#### Data Processing

Since zero-phase non-causal IIR filters have been shown to perform well on Slow Cortical Potentials (SCPs) related to anticipatory behavior (Garipelli et al., 2011), the EEG data in the current paper were non-causally bandpass filtered between 0.05 to 3 Hz using a zero-phase second-order Butterworth bandpass filter prior to further processing. The choice of the filter was consistent with prior studies that used MRCP for real-time detection of motor intentions (Xu et al., 2014a) and similar to the recommendations of (Garipelli et al., 2011). All data were analyzed without rejecting segments with artifacts.

#### cICA for MRCP Detection

The cICA approach is briefly explained in the following.

Suppose that a N-dimensional observed sensor signal x(t) = [x1(t), x2(t), . . . , xN(t)]<sup>T</sup> can be expressed as:

$$\alpha(t) = As(t),\tag{1}$$

where s(t) = [s1(t),s2(t), . . . ,sM(t)]<sup>T</sup> is a M-dimensional mutually-independent latent source vector, and A is an unknown non-singular mixing matrix. The objective of cICA is to find a separating or de-mixing vector w without knowing the source vector and mixing matrix, such that:

$$\mathbf{y}(t) = \boldsymbol{\w}^T \mathbf{x}(t) = \boldsymbol{\w}^T \mathbf{A} \mathbf{s}(t),\tag{2}$$

where y(t) is the desired independent component (desired source signal). To determine this de-mixing vector, the cICA algorithm consists of the following steps. First, a linear whitening transformation is applied to the time series so that each column of z(t) has unit variance and the columns are uncorrelated, i.e., the covariance matrix of z(t) becomes the identity matrix:

$$z(t) = Vx(t),\tag{3}$$

where V is a whitening matrix (Zhang, 2008). Next, according to the negentropy maximum criterion (Hyvärinen et al., 2001), the objective function of the next step is defined by:

$$J(\boldsymbol{\chi}) \approx \gamma [E\{G(\boldsymbol{\chi}(t))\} - E\{G(\boldsymbol{\nu})\}]^2,\tag{4}$$

where E{} indicates expectation of the signal and y(t) = w T z(t) is the output of the algorithm, γ is a positive constant, ν is a Gaussian variable having zero mean and unit variance, and G (·) can be any non-quadratic function. For traditional ICA methods, which have several independent components at the output, all columns of the output will be independent of each other by maximizing (4). To obtain one specific source signal, a priori information about the particular desired source needs to be incorporated into the cost function. In order to achieve this goal, the cICA problem is formulated as:

$$J(\boldsymbol{\omega}) \approx \gamma [E\{G(\boldsymbol{\omega}^T \boldsymbol{z})\} - E\{G(\boldsymbol{\nu})\}]^2$$

$$\text{Subject to } : \boldsymbol{g}(\boldsymbol{\nu}) = \boldsymbol{\varepsilon}(\boldsymbol{\nu}, \boldsymbol{r}) - \boldsymbol{\xi}, \ h(\boldsymbol{\nu}) = E\{\boldsymbol{\nu}^2\} - 1 = 0, \tag{5}$$

where ε(y,r) is the similarity measure between the independent component y and the reference signal r, and ξ is a the similarity threshold. Therefore, g(ω) is the similarity constraint for the ICA optimization criterion, and h(ω) constrains y to have unit variance. Assuming that the desired IC is the one and only one closest to the reference r, one can get the following inequality relationship:

$$\varepsilon(\boldsymbol{w}^{\*T}\boldsymbol{z},\boldsymbol{r}) \prec \varepsilon(\boldsymbol{w}\_1\boldsymbol{1}^T\boldsymbol{z},\boldsymbol{r}) \prec \dots \varepsilon(\boldsymbol{w}\_{N-1}\boldsymbol{1}^T\boldsymbol{z},\boldsymbol{r}),\tag{6}$$

where the optimum vector ω ∗ is the optimum demixing vector corresponding to the desired IC, and wi(i = 1, . . . , N − 1) corresponds to other unwanted ICs. The value of the similarity threshold lies in [ε(w ∗T z,r), ε(w<sup>1</sup> T z,r)]. The Lagrange multipliers method is used to solve the optimization problem of (5) (Lu and Rajapakse, 2005, 2006; Zhang, 2008):

$$\begin{aligned} \boldsymbol{w}\_{t+1} &= \boldsymbol{w}\_t - \eta \boldsymbol{R}\_z^{-1} \boldsymbol{\Gamma}\_1 / \boldsymbol{\Gamma}\_2 \\ \boldsymbol{\Gamma}\_1 &= \bar{\boldsymbol{\gamma}} E \{ \boldsymbol{z} \boldsymbol{G}\_{\boldsymbol{\gamma}}' (\boldsymbol{\gamma}) \} - 1 / 2 \mu E \{ \mathbf{g}\_{\boldsymbol{\gamma}}' (\boldsymbol{\gamma}) \} - \lambda E \{ \boldsymbol{z} \boldsymbol{\gamma} \} \\ \boldsymbol{\Gamma}\_2 &= \bar{\boldsymbol{\gamma}} E \{ \boldsymbol{z} \boldsymbol{G}\_{\boldsymbol{\gamma}^2}'' (\boldsymbol{\gamma}) \} - 1 / 2 \mu E \{ \boldsymbol{g}\_{\boldsymbol{\gamma}^2}'' (\boldsymbol{\gamma}) \} - \lambda, \end{aligned} \tag{7}$$

where t represents the iteration number. R<sup>z</sup> = E{zz<sup>T</sup> }, γ¯ = γ · sign(E{G(y)} − E{G(v)}); and G ′ y (y), g ′ y (y), G ′′ y 2 (y), g ′′ y 2 (y), are respectively, the first and second derivatives of G(y), g(y) with respect to y. The optimum multipliers µ and λ are found by iteratively updating them based on a gradient-ascent method:

$$
\mu\_t = \text{Max}\{0, \mu\_{t-1} + \eta g(\omega\_{t-1})\}
$$

$$
\lambda\_t = \lambda\_{t-1} + \gamma\_{t-1} h(\omega\_{t-1})\tag{8}
$$

Designing the reference signal plays a crucial role in cICA. The reference signal should be closely related to the desired source signal in terms of shape and phase (Zhang and Zhang, 2006; Zhang, 2008). For example, it is possible to use one of the observed channels as a reference signal (Mi, 2014). We propose the use of the average MRCP from Cz (for dorsiflexion) over all trials of a training set to build a subject-specific reference signal. Details of the training sets and construction of the reference signal using the training sets are discussed below.

#### Movement Detection Analysis

"Go" epochs and "No-go" epochs were extracted from the recorded signals according to the onset of the performed dorsiflexion task. Go epochs were the time intervals containing the MRCP whereas No-go epochs contained only noise. The effect of the filters on the MRCP morphology was quantified by two indices: the Signal to Noise Ratio (SNR) and the Go epoch variability (ρ). Moreover, three additional indices were calculated from the dataset of each subject to evaluate the performance of spatial filters in MRCP detection: True Positive Rate (TPR), False Positive Rate (FPR), and Detection Latency (DL). This was done using an offline evaluation framework, as described next. Following the extraction of Go epochs and No-go epochs, cross validation was implemented, and in each fold of the cross-validation, two thirds of the Go epochs and No-go epochs were randomly selected as the training set, and the remaining third of the Go and No-go epochs formed the testing set. Cross validation was performed whereby two thirds of the trials from the entire data set were randomly selected as a training set and the remaining one third as the testing set, and this was repeated ten times. The training set was used to generate the weights for spatial filters, and by assuming that the characteristics of the MRCP signals did not change across sessions, the demixing vector obtained from the training phase was applied to the test data. This offline evaluation over a number of folds allows a systematic evaluation of each method's performance by obtaining the receiver operating characteristics (ROC) curve of each method through cross-validation.

The SNR was calculated for each subject by extracting Go and No-go epochs, respectively, from [−2, 2] s and [2, 6] s with respect to the task onset (the turning point of the cue, see (Jochumsen et al., 2015)). Denoting the lth Go epoch and No-go epoch by x l S (t) and x l N (t), respectively, each containing T samples, the SNR can be expressed as:

$$\text{SNR} = \frac{\sum\_{l=1}^{L} \sum\_{t=0}^{T} \left[ \boldsymbol{\chi}\_{\text{S}}^{l}(t) \right]^2}{\sum\_{l=1}^{L} \sum\_{t=0}^{T} \left[ \boldsymbol{\chi}\_{\text{N}}^{l}(t) \right]^2}. \tag{9}$$

The Go epoch variability ρ was defined as:

$$\rho = \frac{\frac{1}{LT} \sum\_{l=1}^{L} \sum\_{t=0}^{T} \left| \mathbf{x}\_{\mathcal{S}}^{l}(t) - \overline{\mathbf{x}\_{\mathcal{S}}(t)} \right|}{\max \left[ \overline{\mathbf{x}\_{\mathcal{S}}(t)} \right] - \min \left[ \overline{\mathbf{x}\_{\mathcal{S}}(t)} \right]},\tag{10}$$

where xS(t) is the average of the L Go epochs. The lower the value of ρ, the more consistent the Go epochs are. It should be noted that the two indices are calculated for all spatial filter outputs.

TPR, FPR, and DL were calculated on Go and No-go epochs, respectively extracted from [−3, 1] s and [2, 6] s with respect to the task onset, for each subject. TPR and FPR for each fold of the testing set were defined as:

$$TPR = \frac{\text{Total number of correctly detected Go epochs}}{\text{Total number of Go epochs}}, \quad (11)$$

and

$$FPR = \frac{\text{Total number of incorrectly detected No-go epochs}}{\text{Total number of No-go epochs}}$$

The Go epoch interval used to calculate the measures of detection performance was chosen to be different from the Go-epochs used for SNR calculation because, considering the length of the moving window, the time interval [−2 2] s, which perfectly covers all MRCP components, cannot be used if one would expect negative detection latencies where detection happens before the movement execution (t = 0). It should be noted that since the time interval [−3, 1] covers most parts of MRCP, this choice does not affect the TPR values.

To train Infomax and JADE, the training sets were built by concatenating all Go epochs and all No-go epochs of the training set. This means that all concatenated Go epochs (randomly selected) formed the first half of the training set signals; and the corresponding second half of the training set signals was formed by the concatenation of randomly selected No-go epochs in each channel. This approach was chosen as it provided a consistent training process for each method, and furthermore, it enabled us to perform the cross validation process. A similar approach was used for cICA, with an additional reference signal for the EEG signals. The reference signal for cICA was constructed using two steps: first, a subject-specific MRCP template was generated by averaging all Go-epochs of the Cz epochs in the training set ([−2, 2] s with respect to the task onset). Next, considering that the training sets were concatenated Go and No-go epochs for the other methods (Infomax and JADE), the reference signal of cICA was built by repeating the MRCP template corresponding to the signal epochs and using zero for the No-go epochs. By knowing the actual occurrence time of the executed or imagined movements, this approach could be implementable in the training phase of an online application as well. To train CSP, No-Go epochs and Go epochs were provided to the algorithm in two different matrices built by placing Go epochs in the rows of the signal matrix and each No-Go epoch in the rows of the noise matrix. LAP is not a supervised method; therefore, no training was required.

A LPP-LDA classifier was used for classification of the Go and No-go epochs (Xu et al., 2014a). A sliding window with length 2 s and 50 ms shift was applied to each Go and No-go epoch. A detection occurred when n consecutive sliding windows resulted in detection at the output of the LPP-LDA classifier. The choice for n determines the sensitivity of the overall system. Therefore, by varying n from 1 to 10, the average (over subjects) ROC curve was derived through cross-validation on the testing dataset of all subjects. TPR is defined as the ratio of the number of correctly detected Go epochs to the total number of Go epochs in the testing set. Similarly, FPR is defined as the ratio of the number of false detections of No-go epochs to the total number of No-go epochs in the testing set. The detection latency is defined as the time difference between detection and movement onset for the executed movements, and between detection and task onset for the imagined movements, in each Go epoch.

#### Statistical Analysis

To investigate the effect of the spatial filtering method on SNR and ρ, Friedman's Two-way ANOVA was performed, where the factor was Methods with five levels (LAP, CSP, Infomax, JADE, and cICA). When a significant difference was observed, a multiple comparison (Bonferroni) was carried out to identify which methods were significantly different. The significance level

of all tests was set at p < 0.05. Furthermore, in order to investigate the effect of the five methods on MRCP detection, two-way repeated measure ANOVA was performed on the ME and MI datasets, with fixed factor the spatial filtering algorithms (LAP, CSP, Infomax, JADE, and cICA) and random factor the subject (SUB, 12 levels). The main null hypothesis was that Methods was not a significant factor on TPR, FPR, and DL. When the null hypothesis was rejected, a multiple comparison (Tukey with Bonferroni correction) followed.

### RESULTS

The boxplot for the average values of SNR and ρ for the output of the spatial filters over folds from the testing sets, and for all subjects are presented in **Figure 1**. Direct observation indicates that, in this offline study, Infomax is able to suppress the noise better than other methods (highest SNR) in both the ME and MI datasets. In contrast, cICA had the lowest SNR values compared to other methods. However, in both the ME and MI datasets, cICA resulted in the lowest values for ρ among all methods. For the ME dataset, results from the Friedman's Two-way ANOVA showed that Methods had a significant effect on ρ and SNR (p < 0.001). The multiple comparison tests found that the SNR was smaller for cICA than LAP, Infomax, and JADE. Moreover, cICA, LAP, and CSP led to significantly lower variability compared to JADE and Infomax. For the MI dataset, the factor Methods again had a significant effect on ρ and SNR (p < 0.001). Post-hoc comparisons showed that SNR for cICA was significantly lower than Infomax, and JADE; and Infomax had significantly greater SNR values than LAP. For ρ, similar to the ME dataset, cICA, LAP, and CSP led to significantly lower variability than JADE and Infomax.

**Figure 2** represents the algorithm used to calculate the detection latency when 5 consecutive windows result in detection at the output of the LPP-LDA classifier (n = 5). The average of the ROC curves of MRCP detection over all subjects for both ME and MI (testing) datasets is provided in **Figure 3** for all spatial filters and 10 decision thresholds (n = 1, 2, . . . , 10). The area under the ROC curves is provided in **Table 1**. For both datasets, the area under the ROC curve of cICA has the highest value confirming that for each n, cICA provides the best combination of TPRs and FPRs (high TPR and low FPR). Therefore, the accuracy of cICA is superior compared to other spatial filters. As seen from the ROC curves, five decision windows are located at the midpoint of the convex part of the ROC curve, meaning that five consecutive detections could be a good balance between TPR and FPR for all filters. Therefore, the results presented next were calculated for five as the decision threshold.

The detection performance is presented in **Table 2** for both ME and MI datasets. The highest TPRs and lowest FPRs and DLs were obtained for cICA for both datasets. The detection latency for cICA (−34 ± 29 ms for ME and 28 ± 16 ms for MI dataset) was significantly smaller than for the other spatial filters.

For the ME dataset, the ANOVA test showed that Methods has a significant effect on TPR, FPR, and DL (p < 0.001). Multiple comparisons found that TPR for cICA (87.11 ± 11.73) was significantly higher than with all other methods. LAP (74.65 ± 13.13) had significantly greater TPRs than CSP (67.14 ± 13.99) and Infomax (67.27 ± 7.69). FPR for cICA (20.69 ± 13.68) was significantly lower than for Infomax (31.70 ± 9.94) and JADE (30.44 ± 10.26); and FPR for Infomax (31.70 ± 9.94) was significantly higher than for CSP (24.55

± 11.31) and cICA (20.69 ± 13.68). Regarding the detection latencies, the statistical analysis showed that cICA (−34 ± 29 ms) had significantly lower detection latencies compared with all other methods. In contrast, the detection latencies with CSP (295 ± 13 ms) were significantly greater than for Infomax (245 ± 9 ms), LAP (197 ± 15 ms), and cICA (−34 ± 29 ms).

Results for the MI dataset were similar to those for the ME dataset. Methods influenced significantly TPR, FPR, and DL (p = 0.00 for TPR and DL, and p = 0.02 for FPR). Multiple comparisons indicated that TPR from cICA (86.66 ± 6.96) was significantly greater than for all other methods, and TPR for LAP (75.06 ± 12.94) was significantly higher than for CSP (66.87 ± 10.13) and Infomax (64.69 ± 9.42). FPR of cICA (19.31 ± 12.60) was significantly lower than for LAP (25.99 ± 17.04), Infomax (26.19 ± 7.78), and JADE (26.12 ± 10.25), but not significantly different from CSP (23.02 ± 10.56). The detection latency obtained with cICA (28 ± 16) was significantly lower than for all other methods.

The average TPR, FPR, and DL over the 10 folds are reported for each subject from both datasets in **Figure 4**. For 11 of the 12 subjects, cICA has the highest TPR and lowest FPR and DL among all spatial filters.

#### DISCUSSION

The MRCP has recently been implemented as a control signal in a variety of BCI applications (Xu et al., 2014a,b; Jiang et al., 2015; Mrachacz-Kersting et al., 2016). The reliable and efficient detection of MRCPs enables the design of accurate and fast brain switches. Depending on the application of BCI systems, the importance of accuracy and latency of the system may vary. To be more specific, while large DL may not be ideal for BCI applications developed to induce brain plasticity, slightly lower TPR may not greatly affect the performance of the BCI system. On the other hand, high TPR are required for the control of exoskeletons for replacement rather than restoration of function, and for this application, a low DL is not so imperative. Accuracy and latency of detection of the MRCP highly relies on the signal processing method used to extract features from raw EEG. Spatial filters are one of the most efficient and successful feature extraction methods in EEG signal processing due to the spatial distribution of the signal features. In this study, the performance of cICA, a newly introduced ICAbased spatial filter, was compared with four other spatial filters in an offline experiment for MRCP detection from multi-channel EEG recordings, during execution and imaginary dorsiflexion of healthy subjects.

The performance of each spatial filtering algorithm in the detection of MRCPs was initially evaluated based on clarity and consistency of the extracted MRCP, quantified by SNR and ρ, respectively. Moreover, TPR, FPR, and DL were investigated through cross-validation in an offline experiment. The reported TPRs in this study are in agreement with the previous similar

TABLE 1 | Average of the ROC curves of movement detection for ME and MI datasets.


TABLE 2 | Average TPR, FPR, and DL for movement detection for ME and MI datasets.

studies (Xu et al., 2014a,b). However, since, in this study, it was intended to evaluate the performance of the detector and determine the optimum parameters for movement detection using ROC, the values of FPR were calculated with a different measure than previous similar studies. In the previous studies, FPR was defined as the number of false detections per minute. Such approach for calculating FPRs caused the values of FPRs to be biased by the experiment protocol and inconsistent with TPRs. In this paper, the approach used to calculate FPR values makes the values independent of the experimental protocol, in which parameters such as the refractory period of the MI/ME can affect the accuracy of the definition of FPR used in previous studies (Niazi et al., 2011, 2013): false positive per unit time. Also, this approach is consistent with the approach used to calculate TPRs, enabling us to obtain ROC curves for the detector. The calculation of DL in this study is also in agreement with previous studies. It should be noted that a non-causal filter was used in the current study. In a real online experiment, a causal filter should be used. In order to investigate the effect of type of the bandpass filtering method (causal vs. non-causal), we performed an additional analysis to compare the performance of a causal second-order Butterworth bandpass filter with the bandwidth of 0.05–3 Hz with the same non-causal filter. The average signal of all causally and non-causally filtered Go-epochs (MRCPs) from the Cz channel for Subject 1 are provided in **Figure 5**. The observations indicate that there is a smaller amplitude in the negative peak of MRCP when the causal filter is used. We also compared the detection performance for causally and non-causally filtered signals for all subjects in the ME group. The causal filtering resulted in slightly higher FPR and lower TPRs compared to using non-causally filtered data, and the change was consistent in overall detection accuracy for all spatial filters investigated (the change of the averaged TPR values from causally to non-causally filtered signals was: 0.87, −0.19, −3.87, −5.89, −4; and the corresponding change of the averaged FPR values was: −7.69, 9.95, 7.46, 10.86, 13.47 for LAP, CSP, Infomax, JADE, and cICA respectively). This consistent change in overall detection accuracy is expected given the results shown in the figure, as the causal filter resulted in a less pronounced MRCP. However, causal filtering had no significant effect on the detection latencies (the difference between the averaged DL values for causally and non-causally filtered signals was: −0.07, −0.03, −0.06, −0.02, 0.00 s for LAP, CSP, Infomax, JADE, and


The results are presented (mean ± standard deviation across subjects) for each spatial filter.

cICA, respectively). Therefore, it is highly possible that the choice of causal or non-causal filtering has a slight effect on the overall detection accuracy, but there was no effect on DL values. However, this needs to be verified in a subsequent dedicated online study, which is beyond the scope of the current study with the objective of introducing cICA for MRCP detection.

The cICA requires the choice of a threshold that weights the relative importance of similarity with the reference signal in the optimization (Zhang, 2008). The suitable value of threshold depends on both the designed reference signal and the similarity measure. An effective way to determine the threshold given a reference signal, which was also used in this paper, is to use a small threshold initially, and then gradually increase the threshold (Lu and Rajapakse, 2006). For the reference signal based on the average of the Go epochs of the Cz channel, the value of the threshold was set to 0.9. On the other hand, as mentioned earlier, the shape of the designed reference signal plays an important role in the performance of cICA. Therefore, investigation of the effect of other types of reference signals such as the common rectangular pulse, smoothed MRCPs (Garipelli et al., 2013), and discriminative-based reference signal (Lee et al., 2016) will be done in the future in attempt to improve detection performance.

With the selected parameters, the area under the ROC curve for cICA was greater than for the other methods and cICA outperformed all the other filters for TPR and DL. Moreover, FPR was lower for cICA than for three of the other investigated methods. Overall, this indicates an improved performance of cICA with respect to previously proposed filtering methods. Considering that the detection of MRCP can be affected by hyperparameters such as the overlap of the sliding windows and the number of detections required, further investigation will be done in the future to optimize the cICA algorithm based on these and other aspects. The averaged SNR values for the tested methods were not well associated with the detection performance. Indeed, cICA provided high TPRs and low FPRs compared to other methods but resulted in the lowest SNR values. One reason for the low SNR of cICA may be the optimization criteria of the method and the way SNR values were calculated in this study. The reference signal for cICA requires the algorithm to optimize the weights such that the desired signal can be obtained. As a result, the trial-by-trial consistency of the signal was improved by cICA. On the other hand, the results for the average ρ-values were more consistent with those obtained for TPRs. This is one of the findings of the current study: SNR does not necessarily correlate very well with detection performance, and the consistency of the Go epochs is equally (if not more) important for achieving a high detection performance. This likely stems from the fact that MRCP is a rather deterministic waveform, compared to other motor imagery BCI signal modalities, such as ERD/ERS. It can be concluded that, considering the shape of the reference signal applied in the current study, cICA seems to allow a more accurate modeling of the class of the Go epochs, and consequently a more pronounced effect on the sensitivity of the detector. This is because the choice of the reference signal can affect the ability of the cICA in modeling each class and separability of the classes. Therefore, cICA in the current study has limited effect on the specificity of the detector due to the choice of reference signal. It is possible that other types of reference signals can tune the algorithm to focus on other aspects of performance, such as specificity, which will be explored in future studies.

Regression analysis and template matching are also methods that have been used to extract desired EEG features and for EEG artifact removal (Wallstrom et al., 2004; Niazi et al., 2013; Urigüen and Garcia-Zapirain, 2015). Regression algorithms estimate the influence of the reference signal on the desired signal either in the frequency or time domain. Linear regression assumes that each EEG channel is the sum of the non-noisy source signal and a fraction of the source artifact that is available through a reference channel. Then, the goal of regression is to estimate the optimal value for the factor that represents such a fraction. Regression approaches need a reference channel to be able to operate automatically. In comparison, cICA is more flexible because although it uses a reference signal to extract features of the EEG signal or artifacts, the reference signal does not have to be a good estimation of the source(s). In fact, the reference signal can be very general, as long as it provides some reasonable constraint to ICA. For example, in Lee et al. (2016), a rectangular reference signal, which was not similar to the underlying source, was successfully implemented. In addition,

#### REFERENCES


since the regression methods are based on the time and frequency characteristics of the signals, they do not take into account the spatial information of the sources. Template matching techniques such as matched filter, which uses a template to maximize the SNR of the extracted signal, are also methods used for MRCP extraction (Niazi et al., 2013). Similar to regression, such methods only depend on the temporal features of the template and do not consider the spatial distribution of different sources. Also, matched filters are only optimal with additive Gaussian noise, so they are sensitive to other types of noise and artifacts.

In the current manuscript, we only used data from one of the four tasks for the purpose of introducing cICA for the first time in MRCP detection. Subsequent studies will be performed to investigate the generalizability of cICA when presented with data from different types of tasks.

### CONCLUSION

We have proposed a new spatial filter for MRCP detection. The proposed cICA extracts the desired signal by utilizing additional prior (spatial) information with respect to classic ICA, while exploiting higher order statistical structures as the CSP does. The results indicated that cICA did not enhance the extracted MRCP from multi-channel EEG significantly better than several commonly used spatial filters, including CSP, LAP, and ICA. However, cICA significantly outperformed these spatial filters in single-trial MRCP detection, with higher TPRs, lower FPRs, and shorter latency, both for ME and MI tasks. These results indicate that cICA is a promising new algorithm for detecting MRCP from multi-channel EEG. Following the promising results of the current study, we will conduct online experiments in a future study, in which cICA will be compared with LAP and CSP.

#### AUTHOR CONTRIBUTIONS

Inception of ideas for the manuscript was by FK, NJ, JK, DF, and NM; conceptualization, methodology, validation, investigation, writing (review and editing) by FK, NJ, JK, DF, and NM; data acquisition/curation by NJ, DF, and NM; formal data analysis by FK and NJ; manuscript first draft by FK; manuscript revisions by FK, NJ, JK, NM, and DF; resources, supervision, administration, funding acquisition by NJ, JK, DF, and NM.

#### FUNDING

This work was supported by the Natural Sciences and Engineering Research Council of Canada; Ontario Ministry of Training, Colleges and Universities; and University of Waterloo.


Cardoso, J. F. (1999). High-order contrasts for independent component analysis. Neural Comput. 11, 157–192. doi: 10.1162/089976699300016863

potentials. J. Neural Eng. 10:036014. doi: 10.1088/1741-2560/10/3/0 36014


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2017 Karimi, Kofman, Mrachacz-Kersting, Farina and Jiang. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Factors of Influence on the Performance of a Short-Latency Non-Invasive Brain Switch: Evidence in Healthy Individuals and Implication for Motor Function Rehabilitation

Ren Xu1, 2, Ning Jiang<sup>3</sup> , Natalie Mrachacz-Kersting<sup>4</sup> , Kim Dremstrup<sup>4</sup> and Dario Farina<sup>1</sup> \*

<sup>1</sup> Department of Neurorehabilitation Engineering, Bernstein Center for Computational Neuroscience, University Medical Center, Göttingen, Germany, <sup>2</sup> Institute of Computer Science, Faculty of Mathematics and Computer Secience, Georg-August University, Göttingen, Germany, <sup>3</sup> Department of Systems Design Engineering, University of Waterloo, Waterloo, ON, Canada, <sup>4</sup> Center for Sensory-Motor Interaction, Department of Health Science and Technology, Aalborg University, Aalborg, Denmark

#### Edited by:

Michela Chiappalone, Istituto Italiano di Tecnologia, Italy

#### Reviewed by:

Jose Luis Contreras-Vidal, University of Houston, USA Dingguo Zhang, Shanghai Jiao Tong University, China

\*Correspondence: Dario Farina dario.farina@bccn.uni-goettingen.de

#### Specialty section:

This article was submitted to Neuroprosthetics, a section of the journal Frontiers in Neuroscience

Received: 30 September 2015 Accepted: 30 December 2015 Published: 21 January 2016

#### Citation:

Xu R, Jiang N, Mrachacz-Kersting N, Dremstrup K and Farina D (2016) Factors of Influence on the Performance of a Short-Latency Non-Invasive Brain Switch: Evidence in Healthy Individuals and Implication for Motor Function Rehabilitation. Front. Neurosci. 9:527. doi: 10.3389/fnins.2015.00527 Brain-computer interfacing (BCI) has recently been applied as a rehabilitation approach for patients with motor disorders, such as stroke. In these closed-loop applications, a brain switch detects the motor intention from brain signals, e.g., scalp EEG, and triggers a neuroprosthetic device, either to deliver sensory feedback or to mimic real movements, thus re-establishing the compromised sensory-motor control loop and promoting neural plasticity. In this context, single trial detection of motor intention with short latency is a prerequisite. The performance of the event detection from EEG recordings is mainly determined by three factors: the type of motor imagery (e.g., repetitive, ballistic), the frequency band (or signal modality) used for discrimination (e.g., alpha, beta, gamma, and MRCP, i.e., movement-related cortical potential), and the processing technique (e.g., time-series analysis, sub-band power estimation). In this study, we investigated single trial EEG traces during movement imagination on healthy individuals, and provided a comprehensive analysis of the performance of a short-latency brain switch when varying these three factors. The morphological investigation showed a cross-subject consistency of a prolonged negative phase in MRCP, and a delayed beta rebound in sensory-motor rhythms during repetitive tasks. The detection performance had the greatest accuracy when using ballistic MRCP with time-series analysis. In this case, the true positive rate (TPR) was ∼70% for a detection latency of ∼200 ms. The results presented here are of practical relevance for designing BCI systems for motor function rehabilitation.

Keywords: brain-computer interface, motor intention detection, ballistic and repetitive task, movement-related cortical potential, sensory-motor rhythm

## INTRODUCTION

In the past decade, non-invasive brain-computer interfacing (BCI) based assistive technology has been proposed as a novel rehabilitation tool for people suffering of motor disorders (Daly and Wolpaw, 2008; Shih et al., 2012), such as stroke (Ramos-Murguialday et al., 2013) and spinal cord injury (Enzinger et al., 2008). In BCI systems for neurorehabilitation, the volition of subjects is detected from brain signals. Such a brain switch is used to control a neuroprosthetic device, such as an electrical stimulator (Niazi et al., 2012; King et al., 2014) or a robotic system (Xu et al., 2014b), to close the sensory-motor loop for either restoring motor function or modulating neural pathways (Shih et al., 2012).

As the first step of these closed-loop systems, accurate online detection of motor intention is a crucial and challenging task for non-invasive neural recordings such as scalp electroencephalography (EEG), mainly due to its low spatial resolution and poor signal-to-noise ratio. For neuroprosthesis control in general, the acceptable delay between intention and action is ∼200 ms (Lauer et al., 2000). In particular, the detection latency is crucial in inducing Hebbian associative neural plasticity for rehabilitation purposes (Hebb, 1949). The efficiency of plasticity induction would be extremely slow, if at all possible, when the artificially afferent triggered by the brain switch arrived at the cortical level either too early or too late relative to motor intention (Mrachacz-Kersting et al., 2012).

Several signal processing approaches have been proposed for motor intention detection from EEG (Venkatakrishnan et al., 2014). Among these methods, two main EEG signal modalities have been explored for the purpose of motor rehabilitation: sensory motor rhythms (SMRs; Yuan and He, 2014) and movement related cortical potentials (MRCPs; Xu et al., 2014a), a type of slow cortical potential.

SMR corresponds to an increase or decrease in power at various subbands of EEG signals recorded over the sensory motor cortex (e.g., Cz for foot movements), prior to, during and after movement, or movement imagination (Yuan and He, 2014). The increase of subband power implies that the neurons in the corresponding cortical area discharge more synchronously than at baseline and therefore it is referred to as event-related synchronization (ERS; Pfurtscheller and Lopes Da Silva, 1999). Conversely, the decrease of subband power corresponds to less synchronous neural activity, termed eventrelated desynchronization (ERD; Pfurtscheller and Lopes Da Silva, 1999). Immediately after movement imagination, an ERS is usually observed in the beta-band (∼20 Hz; Pfurtscheller and Solis-Escalante, 2009). This is also referred to as beta-rebound. In most SMR-based BCI studies for neurorehabilitation, the movement imagery that the subjects were instructed to perform was repetitive movement, such as foot tapping, with a few exceptions (Pfurtscheller and Solis-Escalante, 2009).

MRCP is another EEG signal modality observable on the sensory motor cortex prior to, during and after movement or movement imagery. It is characterized by a slow negative deflection of the near-DC component in the EEG signal before movement or movement imagery, reaching its peak of negativity near the onset of movement or movement imagination, and followed by a positive rebound before the signal returns back to its reference level (Jahanshahi and Hallett, 2003). MRCP is characterized by a time-series change at a very narrow low frequency content (0.05–3 Hz; Jahanshahi and Hallett, 2003). In MRCP-based BCI studies, the movement or imagined movement is usually executed once, often as a brisk or ballistic task, as opposed to the repetitive movement used in SMR-based studies. More recently, a combined approach of subband SMR and timeseries MRCP has been proposed (Ibáñez et al., 2014). This fusion approach yielded improved performance for the detection of ballistic reaching movement (Ibáñez et al., 2014).

Due to the low signal-to-noise ratio of EEG, spatial filtering is usually used as a pre-processing step to enhance the desired feature. Among them, common spatial pattern (CSP) has been very successful in processing SMRs (Ang et al., 2008; Blankertz et al., 2008), particularly when the channels are more than 20. When the channels are less (typically <10), the Laplacian filter has been widely used for both signal modalities (Müller-Putz and Kaiser, 2010; Niazi et al., 2011).

The above survey indicates that a motor imagery based short-latency brain switch is predominantly influenced by three factors: the type of motor task (ballistic or repetitive), the frequency band (e.g., MRCP, alpha or beta band) of EEG, and the corresponding processing technique (subband power estimation or time-series analysis). In previous studies, the effect of some of these factors was partly investigated, e.g., SMR in brief and sustained movement (Cassim et al., 2000; Alegre et al., 2003) or MRCP vs. SMR in real movements (Toro et al., 1994; Babiloni et al., 1999; Filipovic et al., 2001). More recently, there have been studies on the analyses of the frequency band in motor intention detection (Garipelli et al., 2013; Ibáñez et al., 2014; López-Larraz et al., 2014). However, to date, there has been no direct comparison of advantages and disadvantages of all the above factors in the context of a short-latency brain switch for rehabilitation purposes. In this study on healthy subjects, these factors are directly compared in their influence on the low latency detection of movement intention, in an attempt to provide a guideline for BCI researchers working toward closed-loop neuroprosthetic applications.

## METHODS

### Subjects

Ten healthy volunteers (seven male and three female, age: 26.5 ± 4.6 years) participated in the study. Informed consent was obtained from all participants, and ethical approval was provided by the local ethics committee in accordance with the Declaration of Helsinki.

## Experimental Setup

#### • EEG

Nine channels of EEG were acquired with an active electrode system (ActiCap, Brain Products, Germany) and 16-channel EEG amplifier (g.USBamp, gTec GmbH, Austria). The electrodes were placed in the standard 10–20 locations at Cz, Fz, F3, F4, C3, C4, P3, P4, and Pz. The ground and reference located on AFz and the left earlobe, respectively. Sampling frequency was 1200 Hz, with no hardware filter. During all experiments, the impedances of all channels were monitored regularly to ensure that they were below the recommended values indicated by the manufacturers of the active electrode system.

#### • EMG

Surface electromyography (EMG) signals were collected from the tibialis anterior (TA) muscle of the right foot with disposable electrodes (Neuroline 720, Ambu). EMG was acquired by the last channel of the g.USBamp amplifier, with separate ground and reference electrode from the EEG channels. A monopolar electrode was placed on the mid-belly of the right TA muscle while the reference and ground electrodes were placed on the bony surface of the right knee and right ankle, respectively.

#### Experimental Procedure

During an experimental session, the subject was comfortably seated in a chair, ∼1 m from a computer screen. Participants were instructed to look at the center of the screen and to follow the cue presented, minimizing eye movements. During the experiment, the cue on the screen indicated four states (**Figure 1**): idle, focus, preparation, and task. Each trial started from the 5-s idle phase, during which the subjects could adjust their position as they wished. In the second phase, the subjects were asked to focus on the screen without moving. This was followed by the preparation phase, where the subjects were instructed to follow the 3-s backwards counting presented on the screen, and to start imagining the movement immediately when it turned to the task phase, which lasted for 4 s. One trial ended with the next idle phase before the next trial commenced.

Each experimental session was divided into six runs, which consisted of three ballistic and three repetitive runs. The ballistic and repetitive runs were identical, except in the task phase. For ballistic runs, the subjects were instructed to imagine performing ballistic dorsiflexions at the beginning of the 4-s task phase; in repetitive runs, they were continuously repeating motor imagery for the whole 4-s task phase. The subjects were instructed to perform the repetitive task at a moderate speed, i.e., around once per second. The TA muscle activity was monitored through the EMG recording, and those trials with visible EMG signal were not included in further analysis. Each run comprised approximately 20 trials of ballistic or repetitive imaginary movements. The duration of each run was ∼6 min. The order of ballistic and repetitive runs was randomized.

#### EEG Processing Algorithm MRCP and SMR Morphology Analysis

The nine channels of EEG were band-pass filtered (2nd order Butterworth) at 0.05–3 Hz for MRCP analysis, (Xu et al., 2014a) and 4–40 Hz for SMR analysis (Planelles et al., 2014). Then a large Laplacian spatial filter centered at Cz (see Equation 1) was used to enhance the signal-to-noise ratio of the "virtual" Cz channel, which was then processed in subsequent steps.

$$\text{virtual\\_Cz} = \text{Cz} - \sum\_{i} \text{CH}\_i / 8 \tag{1}$$

Where CH<sup>i</sup> stands for the eight channels around Cz.

In subsequent data segmentation, the data from t = −3 to t = 6 s, w.r.t. the task onset, of the filtered virtual Cz were extracted for each trial. For both SMR and MRCP, the reference interval, from which the baseline value was calculated, was −3 to −2 s (3 to 2 s before motor imagery onset).

#### 1. MRCP morphology analysis

For each subject and each movement type, a statistical comparison was performed on the characteristics of the morphorlogy of the MRCP (see details in the Section of Statistical Methods below).

2. SMR morphology analysis

The power spectral density (PSD) of each trial was calculated over 1 s windows overlapped for 0.5 s using Hamming windows (Matlab function pwelch). For each subject and each movement type, a Bootstrap test was performed between the PSD of the SMR at each time-frequency point and the reference PSD of the SMR in a baseline window. The time-frequency SMR characterization was quantified as follows (Pfurtscheller and Lopes Da Silva, 1999):

$$\text{SMR}\_{f,t} \%= \frac{A\_{f,t} - R\_f}{R\_f} \tag{2}$$

where SMRf,t% is the relative power of the SMR at time t and frequency f, Af,<sup>t</sup> is the absolute power of the SMR at the same time-frequency point, and R<sup>f</sup> is the power of the reference interval for the frequency f. A positive SMRf,t% value indicates an ERS, while a negative value of SMR indicates an ERD.

#### Time-Series Feature Extraction and Motor Imagery Detection

In order to analyze the information content in EEG for motor intention detection, six types of time-series features were extracted with band-pass filters at MRCP (0.05–3 Hz), Theta (4– 7 Hz), Alpha (8–15 Hz), Beta (16–30 Hz), lower Gamma (31–40), and the full frequency band (0.05–40 Hz), followed by a large Laplacian filter centered at Cz. In order to evaluate the BCI performance, a three-fold cross-validation was used, in which two runs of the virtual Cz (either ballistic or repetitive) were taken

as the training set, while the remaining ballistic or repetitive run was used as the testing set. For the training set, the virtual Cz was segmented into portions of 2-s segments with 0.1-s increments. The segments between −1 to 1 s with respect to the task onset were labeled as "signal" portion, while all the remaining segments were labeled as "noise" portion. The testing set was treated in a pseudo-online way, mimicking the real online processing, where the data arrives continuously with a 2 s-length window and a refresh step of 0.1 s.

For the time-series feature, a manifold-learning based method called Locality Preserving Projection (LPP; He and Niyogi, 2003), followed by a Linear Discriminant Analysis (LDA) classifier was trained and used for detection. LPP-LDA has previously been implemented for MRCP detection with good performance (Xu et al., 2014a). The procedure is described briefly here. The "signal" and "noise" portions were first projected to a lower dimension using LPP, which maintained the intrinsic structure in the original manifold with high dimension (He and Niyogi, 2003). A LDA classifier was trained using the LPP projected training data. Once the LPP-LDA classifier was trained, the testing segments were projected by the obtained LPP projection, and the trained LDA was used to classify the testing data into either "signal" or "noise." A detection of motor intention would be registered when a number of continuous windows [referred to window number (WN)] were classified as "signal." One detection would be determined either as a true or false detection according to the detection latency (DL), i.e., the time difference between the detection and the task cue. If the DL was between -1 and 1 s, the detection was considered a true detection, otherwise a false detection. It should be emphasized here that the target of this study is a short-latency brain switch. Therefore, we only considered these detections within a few hundreds microseconds, even though signals outside this range may improve the accuracy for modalities such as Beta rebound. The true positive rate (TPR), false positive (FP) per minute, and DL were calculated to quantify the BCI performance. Compared with false positive rate (or specificity) which is generally used for evaluating binary classification (Hashimoto and Ushiba, 2013; Jochumsen et al., 2013) FP/min is more suitable for quantifying the performance of continuous detection in a (pseudo-) online paradigm, as was done in Niazi et al. (2011) and Xu et al. (2014a). As any detector, there is a trade-off between TPR and FP. Both TPR and FP are constrained by WN, whose increment would make the detection stricter (more difficult), leading to lower TPR and smaller FP. In order to objectively compare the BCI performance, we chose the WN value for which the FP was smaller than or equal to 8/min for all comparisons. Thus, only the TPR and DL were statistically compared.

#### Subband Power Estimation and Motor Imagery Detection

The subband power was also used as a direct feature for classification. For this purpose, the powers of the virtual Cz channel at the six frequency bands were estimated using the Welch periodogram with a resolution of 1 Hz. As for the timeseries features, the window duration was of 2 s, with increments of 0.1 s. "Signal" and "noise" portions were the same as for the time-series features, and the same three-fold cross-validation method was used to test performance.

Since the dimensions of the subband power features are small, no dimension reduction method was used. The "signal" and "noise" portions were directly used to train a LDA classifier, which was used for detection of motor intention from the testing set. All BCI performance criteria were calculated with the same steps as in the time-domain processing, and analyzed using the statistical methods described in the following.

### Statistical Methods

A paired t-test was performed for MRCP morphology analysis. The comparison was between the amplitude of the MRCP for the reference interval (i.e., mean value of −3 to −2 s w.r.t. task onset) and the magnitude of each 0.1 s-length segments of the virtual Cz outside the reference interval. A Holm-Bonferroni correction was performed for this multiple comparison, and the significance level was set to 0.05.

Three-way repeated ANOVA was used to investigate the effect of the three factors on BCI performance. The independent variables were TPR and DL. The three main factors were motor task (ballistic and repetitive imagination), frequency band (MRCP, Theta, Alpha, Beta, Gamma, and full frequency bands), and processing technique (time-series analysis and subband power estimation). A full model ANOVA with all interaction terms was performed first and, when significant interactions were detected, post-hoc tests (Tukey simultaneous test with significance level of 0.05) were performed.

## RESULTS

## Signal Morphology

#### MRCP Morphology

The MRCPs of a typical subject performing the two types of motor imageries are shown in **Figures 2A,B**. For the ballistic task (**Figure 2A**), the MRCP started to decrease approximately 2 s prior to the task onset, reached the negative peak around t = 0 s, and returned to the baseline in approximately 2 s after the task onset. In the repetitive task, the MRCP (**Figure 2B**) had a similar shape to that of the ballistic case before t = 0 s but the rebound phase was much longer in duration. Before task onset, both the ballistic and repetitive MRCP showed significant differences to the baseline starting from −1 s. However, the characteristics of the two motor tasks were different for the rebound part. No significant difference was found between the ballistic MRCP and the baseline from the time of 1 s, indicating it already returned back to the baseline. On the other hand, the repetitive MRCP still showed significance as late as 4 s after the task onset.

The above difference between ballistic and repetitive tasks was consistent across all subjects, as shown in **Figure 2C**. These results indicate a strong predictive power of MRCPs in detection of movement intention of the subjects for both types of motor imageries.

#### SMR Mapping

The SMR maps for three representative subjects for the two types of movement imageries are shown in **Figure 3**. For subject

FIGURE 2 | MRCP corresponding to (A) ballistic (BAL) and (B) repetitive (REP) motor imagery from a typical subject. The dashed lines indicate the logarithmic p-value of the paired t-test between the MRCP (either ballistic or repetitive) and its reference between −3 and −2 s., while the solid horizontal line indicates significance level. (C): The average MRCP over the 10 subjects. The black line corresponds to ballistic imagery and the red line to the repetitive motor imagery.

A, there was an evident ERD starting slightly earlier than the task onset, between the Beta and the lower Gamma band (above ∼20 Hz) for the ballistic imagery, while it corresponded to a larger bandwidth for repetitive imagery. There was also an evident ERS in the Alpha and Beta bands for both tasks, but the repetitive ERS occurred much later than the ballistic one. However, the SMR landscape was very different for subject B, whose ERD and ERS mainly appeared in the Alpha and lower Gamma band, respectively. Subject C showed still other characteristics. The ERD occurred over almost the full band for the ballistic imagery, whereas it did not present a clear pattern in the repetitive imagery. Moreover, for all subjects, both the ERD

and ERS occurred earlier in case of ballistic imagery with respect to repetitive imagery. ERS appeared before imagery onset in both ballistic and repetitive tasks of subject B, and also in the repetitive task of subject C.

In summary, the SMR mapping differed substantially among the subjects, thus a general average across the subjects would not be meaningful and therefore is not reported.

#### BCI Performance

The BCI performance in detection of motor imagery is summarized in **Table 1**. The WN is shown in **Figure 4**. The ballistic MRCP with time-series analysis reached the highest TPR

#### TABLE 1 | BCI performance.


BAL and REP stand for ballistic and repetitive imagery, respectively. True positive rate (TPR) is the ratio between true detection and the total number of trials in the testing set. Detection latency (DL) is the timing difference between the detection point and the task onset. The TPR and DL were chosen where false positive ≤8 min−1. The positive DL indicates that the detection happened after the task onset. The best accuracy is indicated in bold.

(70 ± 20%), followed by ballistic Beta band with subband power estimation (57 ± 19%). SMRs with time-series analysis yielded shorter DL (<100 ms), however the corresponding detection accuracy was extremely low (∼30%). For each frequency band of either motor task (except the full band of the ballistic task), the time-domain technique resulted in shorter DL than the frequency-domain technique.

A representative segment of detecting ballistic MRCP with time-serial analysis was shown in **Figure 5**. Three black stars stand for the onsets of three consecutive tasks. For the first two, the detector successfully identifies them, which were labeled as Green stars. Moreover, this method demonstrated its robustness again moderate variation which appeared after the second trial (∼25 s). In spite of this, there is still a false detection (labeled as a red star) when huge noise was introduced during the third trial.

#### Analysis of True Positive Rate

For a meaningful comparison between methods, the TPR is reported in all cases for the same level of false positives (≤8 min−1). The Three-way ANOVA on TPR found no threeway interaction (p = 0.074). Neither was the interaction between motor task and processing technique (p = 0.283). However, there was a significant interaction between motor task and frequency band (p = 0.015), as well as between processing technique and frequency band (p < 0.001). Therefore, we performed post-hoc tests on the significant interactions.

Focusing on the interaction between motor task and frequency band, the post-hoc comparison revealed that the MRCP of the ballistic task yielded the highest TPR (60 ± 22%). This was comparable to the full band of the same task, and significantly better than all other combinations (no significance was found among them). In addition, for both the MRCP and the full band, the ballistic task significantly outperformed the repetitive task.

For the other significant interaction between processing technique and frequency band, the difference depended on each factor. MRCP with time-series analysis provided the highest TPR (57 ± 22%), which was significantly better than all other frequency bands with the same processing technique. For the other processing technique, i.e., subband power analysis, there was no significant difference among frequency bands. Furthermore, it was observed that, for Alpha and Beta band, subband power analysis outperformed time-series analysis.

#### Analysis of Detection Delay

There was no three-factor interaction (p = 0.451), nor two-factor interactions (p = 0.197, 0.532, 0.081). Both the processing technique and frequency band were significantly different (p < 0.001 and p = 0.004, respectively). The posthoc comparison revealed that the time-domain technique had lower DL than the frequency-domain technique (117 ± 169 vs. 252 ± 194 ms), while the MRCP resulted in longer DL than Alpha, Beta, and Gamma band (265 ± 185 vs. 145 ± 205, 129 ± 193, and 125 ± 159 ms, respectively).

#### Summary

Based on the above statistical analysis of TPR and DL, we summarize the influence of the three factors here. The ballistic task, is preferable over the repetitive task, as it yielded higher TPR for both MRCP and full band. Even though MRCP's DL

was slightly larger than for some SMRs, MRCP was still the best choice among all six frequency bands, given its highest TPR for both tasks and for the time-series processing technique. Timeseries analysis outperformed subband power analysis, mainly due to a significantly shorter DL.

In summary, this comprehensive comparison on motor intention detection with two motor tasks (ballistic and repetitive), six frequency bands (MRCP, Theta, Alpha, Beta, Gamma, and full band), and two processing techniques (timeseries analysis and subband power estimation) showed that the combination of ballistic, MRCP and time-series analysis significantly is preferred among all the considered options.

### DISCUSSION

As a crucial aspect of closed-loop rehabilitation systems, the detection of the motor intention from scalp EEG is a central challenge in motor imagery based BCI. In recent years, there has been an increasing number of publications of clinical studies using SMR-based BCI (Ramos-Murguialday et al., 2013; Ang et al., 2014a,b; Li et al., 2014; Mukaino et al., 2014; Ono et al., 2014). On the other hand, MRCPs have also been proven as a promising signal type, particularly for neuromodulation purposes due to its short detection latency (Mrachacz-Kersting et al., 2012; Niazi et al., 2012; Xu et al., 2014b). Preliminary studies of MRCP-based BCI applied to chronic stroke patients have also been reported (Mrachacz-Kersting et al., 2015).

According to the Hebbian principle of associative plasticity (Hebb, 1949), neuroplasticity would only be induced when the motor intention and the task specific afferent feedback, e.g., passive movement delivered by an orthosis, occur synchronously in a cause-and-effect fashion. Therefore, an effective neuromodulation system requires not only accurate algorithms, but also algorithms that present short detection latencies, ideally shorter than 300 ms.

In the past decades, SMR has been the main signal modality used for detection purposes. TPRs above 80% were reported with Beta ERS (Müller-Putz and Kaiser, 2010; Wang et al., 2012), while the performance of ERD was also demonstrated to be above 70% (Planelles et al., 2014; Yang et al., 2014). However, the issue of detection latency was largely overlooked in these studies, with only one exception which reported a latency in the range of seconds (Hashimoto and Ushiba, 2013). In this study, the best TPR using SMR was 57%, for the beta band in the ballistic task using subband power estimation. Compared with previous studies, the performance here decreased obviously, mainly due to the limited range of latency. In these previous studies using SMR, detection latency was rarely reported. Its average latency can be more than one second if we take into consideration those detections which occur several seconds after the task onset (Hashimoto and Ushiba, 2013). On the contrary, in the current study, short latency detection is essential, as it is mandatory for the purpose of plasticity induction (Mrachacz-Kersting et al., 2012). As such, those detections occurring after 1 s were counted as false detections rather than true ones, resulting in a latency of several hundred milliseconds as shown in **Table 1**. Taking later windows into consideration would likely improve TPR, but would lead to a long-latency brain switch, which would be out of the focus of this study.

On the other hand, slow cortical potentials, such as MRCP, have been investigated for movement intention detection in recent years (Qian et al., 2010; Bai et al., 2011; Niazi et al., 2011; Lew et al., 2012; Bulea et al., 2013, 2014; Bhagat et al., 2014; Xu et al., 2014a). In the current study, the average TPR and DL was 70% and <300 ms for motor imagination, consistent with the results reported in previous studies (Niazi et al., 2011, 2012; Xu et al., 2014a,b). The relatively good performance in TPR (>70%) is essential for the high efficiency of MRCPbased BCI system. In particular, the detection latency of a few hundred milliseconds was demonstrated to be crucial for plasticity induction (Mrachacz-Kersting et al., 2012).

In this study, we demonstrated that ballistic motor imagery task, frequency band of MRCP, and time-series analysis is the optimal combination in terms of detection performance. Other options investigated are sub-optimal, mainly due to a trade-off between TPR and DL.

We also observed that repetitive SMR with subband power estimation was significantly better in accuracy than the ballistic one. This is in accordance with the discussion by Pfurtscheller & Solis-Escalante that SMR in repetitive task is easier to classify (Pfurtscheller and Solis-Escalante, 2009).

The different EEG features between the ballistic and repetitive task may be attributed to the difference in their afferent input (Cassim et al., 2000). As explained by Bear (Bear et al., 2007), compared to the repetitive task which has sufficient time for the sensory-motor loop to feedback, the ballistic movement, once initialized, is too fast to adjust. In addition, this difference may also be explain by the fact that there is an inhibitory process immediate following the ballistic task (Alegre et al., 2003), while this is not the case for the repetitive task.

The morphological difference between ballistic and repetitive tasks may mainly contribute to the differences in the detection performance of the two motor tasks. The prolonged negative phase in repetitive MRCP (in **Figure 2**) makes its rebound feature after task onset not as distinct as the ballistic one. That is why the accuracy of repetitive MRCP with time-series analysis is much lower than the ballistic one. On the other hand, in the case of subband power analysis, the frequency band did not have a significant influence on accuracy. This is likely explained due to the observed variability in the optimal frequency bands among individual subjects (see **Figure 3**).

#### Limitations

The above comparison was performed only on healthy subjects. Previous findings mostly support the similarity in slow EEG waves (e.g., MRCP) between healthy and stroke or spinal cord injured patients (Castro et al., 2007; Mattia et al., 2009; Niazi et al., 2011; Xu et al., 2014c), despite their non-negligibly distinct features such as onset and amplitude (Yilmaz et al., 2014), whereas SMRs trend to have greater difference on patients with central neural injury (Tran et al., 2004; Gourab and Schmit, 2010; Müller-Putz et al., 2014). Other different factors such as medication and mental status in patients could be challenging for clinical measurements. Therefore, further investigation on the target patient population is necessary.

In addition, the combination of several features with different processing techniques, such as those presented in Ibáñez et al. (2014) and López-Larraz et al. (2014), was not investigated in

#### REFERENCES


the current study. This study focused exclusively on a general comparison without subject-specific optimization, the combined features, e.g., SMR and MRCP, would be worthy to investigate in future work.

### CONCLUSION

In this study, we performed a comprehensive comparison of motor task, frequency band, and processing technique, to investigate their influence on the performance of a short-latency brain switch. The morphological investigation found crosssubject consistency in MRCP, which supports its advantage for a subject-independent use. The BCI detection performance was maximized by using the ballistic imagery task, the DC bandwidth (MRCP), and the time-series analysis.

### AUTHOR CONTRIBUTIONS

Conceived and designed the experiments: RX, NJ, NM, KD, DF. Performed the experiments: RX, NJ. Analyzed the data: RX, NJ, DF. Wrote the paper: RX, NJ, NM, DF.

#### ACKNOWLEDGMENTS

This work was supported financially by the China Scholarship Council (Contract Nr. 201204910155).

analysis. Signal Process. Mag. IEEE 25, 41–56. doi: 10.1109/MSP.2008.4 408441


from severe hemiplegia due to chronic stroke. Front. Neuroeng. 7:19. doi: 10.3389/fneng.2014.00019


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2016 Xu, Jiang, Mrachacz-Kersting, Dremstrup and Farina. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

## Proceedings of the Third Annual Deep Brain Stimulation Think Tank: A Review of Emerging Issues and Technologies

#### Edited by:

Paolo Massobrio, University of Genoa, Italy

#### Reviewed by:

Fritz A. Henn, Mt. Sinai Medical Center, USA Daniela S. Andres, ETH Zurich & University of Zurich, Switzerland

\*Correspondence:

P. Justin Rossi pjrossi@ufl.edu

#### Specialty section:

This article was submitted to Neuroprosthetics, a section of the journal Frontiers in Neuroscience

Received: 11 November 2015 Accepted: 11 March 2016 Published: 06 April 2016

#### Citation:

Rossi PJ, Gunduz A, Judy J, Wilson L, Machado A, Giordano JJ, Elias WJ, Rossi MA, Butson CL, Fox MD, McIntyre CC, Pouratian N, Swann NC, de Hemptinne C, Gross RE, Chizeck HJ, Tagliati M, Lozano AM, Goodman W, Langevin J-P, Alterman RL, Akbar U, Gerhardt GA, Grill WM, Hallett M, Herrington T, Herron J, van Horne C, Kopell BH, Lang AE, Lungu C, Martinez-Ramirez D, Mogilner AY, Molina R, Opri E, Otto KJ, Oweiss KG, Pathak Y, Shukla A, Shute J, Sheth SA, Shih LC, Steinke GK, Tröster AI, Vanegas N, Zaghloul KA, Cendejas-Zaragoza L, Verhagen L, Foote KD and Okun MS (2016) Proceedings of the Third Annual Deep Brain Stimulation Think Tank: A Review of Emerging Issues and Technologies. Front. Neurosci. 10:119. doi: 10.3389/fnins.2016.00119 P. Justin Rossi <sup>1</sup> \*, Aysegul Gunduz <sup>1</sup> , Jack Judy <sup>1</sup> , Linda Wilson<sup>2</sup> , Andre Machado<sup>3</sup> , James J. Giordano<sup>4</sup> , W. Jeff Elias <sup>5</sup> , Marvin A. Rossi <sup>6</sup> , Christopher L. Butson<sup>7</sup> , Michael D. Fox <sup>8</sup> , Cameron C. McIntyre<sup>9</sup> , Nader Pouratian<sup>10</sup>, Nicole C. Swann<sup>11</sup> , Coralie de Hemptinne<sup>11</sup>, Robert E. Gross <sup>12</sup>, Howard J. Chizeck <sup>13</sup>, Michele Tagliati <sup>14</sup> , Andres M. Lozano<sup>15</sup>, Wayne Goodman<sup>16</sup>, Jean-Philippe Langevin<sup>17</sup>, Ron L. Alterman<sup>8</sup> , Umer Akbar <sup>18</sup>, Greg A. Gerhardt <sup>19</sup>, Warren M. Grill <sup>20</sup>, Mark Hallett <sup>21</sup>, Todd Herrington<sup>22</sup> , Jeffrey Herron<sup>13</sup>, Craig van Horne<sup>19</sup>, Brian H. Kopell <sup>16</sup>, Anthony E. Lang<sup>15</sup>, Codrin Lungu<sup>21</sup> , Daniel Martinez-Ramirez <sup>1</sup> , Alon Y. Mogilner <sup>23</sup>, Rene Molina<sup>1</sup> , Enrico Opri <sup>1</sup> , Kevin J. Otto<sup>1</sup> , Karim G. Oweiss <sup>1</sup> , Yagna Pathak <sup>24</sup>, Aparna Shukla<sup>1</sup> , Jonathan Shute<sup>1</sup> , Sameer A. Sheth<sup>24</sup> , Ludy C. Shih<sup>8</sup> , G. Karl Steinke<sup>25</sup>, Alexander I. Tröster <sup>26</sup>, Nora Vanegas <sup>24</sup> , Kareem A. Zaghloul <sup>21</sup>, Leopoldo Cendejas-Zaragoza<sup>6</sup> , Leonard Verhagen<sup>6</sup> , Kelly D. Foote<sup>1</sup> and Michael S. Okun<sup>1</sup>

<sup>1</sup> Department of Neuroscience, Center for Movement Disorders and Neurorestoration, University of Florida, Gainesville, FL, USA, <sup>2</sup> Formerly affiliated with the International Technology Roadmap for Semiconductors (ITRS), Washington, USA, <sup>3</sup> Neurological Institute Cleveland Clinic, Cleveland, OH, USA, <sup>4</sup> Neuroethics Studies Program, Department of Neurology, Georgetown University Medical Center, Washington, DC, USA, <sup>5</sup> Neurological Surgery and Neurology, Stereotactic and Functional Neurosurgery, Department of Neurosurgery, University of Virginia Health Science Center, Charlottesville, VA, USA, <sup>6</sup> Department of Neurology, Rush University Medical Center, Chicago, IL, USA, <sup>7</sup> Scientific Computing and Imaging Institute, University of Utah, Salt Lake City, UT, USA, <sup>8</sup> Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA, USA, <sup>9</sup> Department of Biomedical Engineering, School of Medicine, Case Western Reserve University, Cleveland, OH, USA, <sup>10</sup> Department of Neurosurgery, University of California, Los Angeles, Los Angeles, CA, USA, <sup>11</sup> University of California, San Francisco, San Francisco, CA, USA, <sup>12</sup> The Emory Clinic, Emory University, Atlanta, GA, USA, <sup>13</sup> Department of Electrical Engineering, University of Washington, Seattle, WA, USA, <sup>14</sup> Movement Disorders Program, Department of Neurology, Cedars-Sinai Medical Center, Los Angeles, CA, USA, <sup>15</sup> Department of Neurosurgery, University of Toronto, Toronto, ON, Canada, <sup>16</sup> The Icahn School of Medicine at Mount Sinai, New York, NY, USA, <sup>17</sup> Greater Los Angeles Veterans Affairs Healthcare System, Los Angeles, CA, USA, <sup>18</sup> Department of Neurology, Alpert Medical School, Brown University, Providence, RI, USA, <sup>19</sup> University of Kentucky Medical Center, Lexington, KY, USA, <sup>20</sup> Department of Biomedical Engineering, Duke University, Durham, NC, USA, <sup>21</sup> National Institute of Neurological Disorders and Stroke, National Institutes of Health, Bethesda, MD, USA, <sup>22</sup> Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA, <sup>23</sup> Department of Neurosurgery-Center for Neuromodulation, NYU Langone Medical Center, New York, NY, USA, <sup>24</sup> Neurological Institute, Columbia University Medical Center, New York, NY, USA, <sup>25</sup> Boston Scientific Neuromodulation, Valencia, CA, USA, <sup>26</sup> Department of Clinical Neuropsychology, Barrow Neurological Institute, Phoenix, AZ, USA

The proceedings of the 3rd Annual Deep Brain Stimulation Think Tank summarize the most contemporary clinical, electrophysiological, imaging, and computational work on DBS for the treatment of neurological and neuropsychiatric disease. Significant innovations of the past year are emphasized. The Think Tank's contributors represent a unique multidisciplinary ensemble of expert neurologists, neurosurgeons, neuropsychologists, psychiatrists, scientists, engineers, and members of industry. Presentations and discussions covered a broad range of topics, including policy and advocacy considerations for the future of DBS, connectomic approaches to DBS targeting, developments in electrophysiology and related strides toward responsive DBS systems, and recent developments in sensor and device technologies.

Keywords: deep brain stimulation, local field potentials, neuromodulation, closed-loop, electrodes

## INTRODUCTION

The Third Annual Deep Brain Stimulation (DBS) Think Tank convened at the University of Florida's Research and Academic Center in Orlando, FL, on March 18-20, 2015. This report provides a summary of the conference sessions, which addressed the most current research, clinical, ethical and policy work on DBS for the treatment of neurological and psychiatric disease. DBS research and its clinical translation incur wide ranging, complex issues that necessitate ongoing frank discourse and exchange of ideas among the multi-disciplinary group of neurologists, neurosurgeons, neuropsychologists, psychiatrists, scientists, engineers, and ethicists developing and engaging DBS in research and clinical practice. The DBS Think Tank aims to provide an annual forum where contemporary issues, innovations, and challenges of the research and use of DBS are shared, discussed, and debated. Presentations and discussions addressed policy and advocacy considerations for the continued advancement of DBS, connectomic approaches to DBS targeting, developments in electrophysiology and related progress in responsive DBS systems, and recent innovation in sensor- and stimulation-device technologies.

The field continues to advance at an impressive pace. Our hope is that this meeting promotes awareness among stakeholders in DBS of currently unresolved and newly emerging issues, so as to ultimately strengthen the field and better serve patients. As in previous years, the meeting was conducted in a "think tank" style; speakers presented analyses of critical issues to foster dialog in subsequent discussions. The nature of this think tank format implies that this is not an evidence-based overview of developments in DBS; rather, it is a report of ongoing developments that have been advancing this dynamic field and discussion of obstacles hindering further advancement and potential solutions. This summary includes key points of both the presentations and the follow-up discussions.

### POLICY AND ADVOCACY FOR THE FUTURE OF DBS

### Viability of a DBS Industry Roadmap and Consortium

An industry roadmap process, organization, success factors, and typical and expected outcomes were discussed, using the semiconductor technology roadmap as a prior example of successful effort (Spencer and Seidel, 1995; Schaller, 2001). Industry roadmaps provide a dynamic and evolving collaborative technology management process for determining critical needs and drivers, identifying technology and manufacturing targets, and assessing and modeling potential solutions to focus an industry community. They can also provide direction toward consensus-based resolution of needs and problems within specific timeframes (Rathore, 2009; Cartreine et al., 2010; Qattan et al., 2012; Finch et al., 2014). Such "road mapping" has existed within corporations and organizations for decades. Industry-wide roadmaps are versions of the corporate process that can effectively be used to identify gaps in solutions for common precompetitive challenges; suggest methods and programs to resolve those gaps, and address lead-time issues by indicating timeframes of opportunities for facilities, materials, and equipment development within a supply chain community (Garcia and Bray, 1997). Implementation can occur within organizations in the competitive space, thereby contributing to the growth of industry, by generating positive outcomes inclusive of strategic and tactical partnerships throughout the industry.

DBS is being used to mitigate signs and symptoms of an increasing range of neuropsychiatric disorders. In this way, DBS has attained considerable success in treating a greater number of patients, and in turn, fostered increased public awareness, receptivity and demand for this technology.

The investment and success of DBS has fortified the viability of key technological, commercial, and clinical aspects of neurotechnology and are contributing to the continued expansion, development, and success of the field of neurotechnology in general and the use of DBS in medical practice in particular. As a result, there is substantial energy and investment to broaden the applications of DBS and to increase the capability and complexity of DBS systems. The question of whether and how the DBS industry would be best served by a technology roadmap and/or consortium may reflect the nature and extent of the common challenges presently impeding technology advancement and deployment. The field is laden with numerous issues, including increasing system complexity (e.g., increasing interfaces and channel count, telemetry bandwidth, recording and stimulation capability, etc.), variable biocompatibility, packaging challenges (i.e., demands for smaller size units with greater power efficiency and battery capacity), a proliferation of potential brain targets and indications, and these are reflected in—and foster—increasing regulatory requirements. **Figure 1** presents an overview of expert perceptions of the state of maturity of DBS relative to other neurotechnologies, solicited from participants at the think tank in an anonymous poll.

Implementation of industry-wide standards can evoke both positive and negative effects for different stakeholders. For example, on one hand growing mandated regulatory standards can result in longer development times and higher development costs. On the other hand, such standards can fortify the integrity, efficacy, and safety of DBS technology in use, particularly now as the applications of DBS are expanding tremendously. In addition, the use of standards could actually lower barriers to market entry as certain components of DBS systems become more common commodities (e.g., the implanted pulse generator,

**Abbreviations:** DBS, deep brain stimulation; FDA, Food and Drug Administration; IIR, investigator-initiated research; IDE, investigational device exemption; NIH, National Institutes of Health; RoR, right of reference; SEEG, stereoencephalography; MRI, magnetic resonance imaging; DTI, diffusion tensor imaging; FA, fractional anisotropy; MCT, modulated circuit tractography; SAS, subtraction activated SPECT; fcMRI, functional connectivity magnetic resonance imaging; TAM, tractography activation models; PD, Parkinson's disease; IPG, implantable pulse generator; MICC, multiple independent current control; HIFU, high intensity focused ultrasound; LFP, local field potential; ECoG, electrocorticography; STN, subthalamic nucleus; ViM, ventralis intermedius; TS, Tourette's syndrome; CM-PF, centromedian parafascicular nucleus of the thalamus; SVM, support vector machine; SC, subcallosal cingulate gyrus; TRD, treatment resistant depression; PFG, prefrontal gyrus; EEG, electroencephalography; LHb, lateral habenula; ECT, electroconvulsive therapy; CSF, cerebrospinal fluid; PTSD, post-traumatic stress disorder; CAPS, clinican-administered PTSD scale; BLn, basolateral nucleus of the amygdala.

wireless telemetry). If regulatory processes at the Food and Drug Administration (FDA) related to DBS systems evolve to approve key target-agnostic DBS system blocks or components, rather than entire systems designed for specific anatomical targets, it could significantly increase the rate of neurotechnology innovation. Specifically, such new regulatory processes would support the efforts of smaller companies, with novel algorithmic, anatomical target, and neural-interface concepts, to more quickly deliver solutions to an increasing diversity of patient populations, including those that are smaller and thus less economically appealing to the existing medical-design manufacturers. More attention is required to assess the potential viability of industry-wide roadmaps and consortia to resolve these issues.

### Policy to Support Physician Initiated Research and Innovation

Physician investigator-initiated research (IIR) is generally regarded as conferring considerable advantages to DBS research compared to industry-sponsored studies. Physicians are more likely than industry to sponsor research focused on orphan and small disease populations, and increasing IIR in DBS would diversify and broaden ideas focusing upon the current challenges—and opportunities—in the field (Rossi P. J. et al., 2014). Moreover, as a group, physician researchers also have a longer time horizon for assessing outcomes and adding knowledge than most industry sponsors, which can lead to different and uniquely valuable types of studies. However, anecdotal evidence from physician researchers performing DBS trials suggests that significant regulatory burdens are slowing the pace of IIR research, and could be discouraging physicians from participating in such efforts (Rossi P. J. et al., 2014).

A case study providing an overview of the timeline and resources required to meet regulatory requirements for a DBS clinical trial was presented (Kelly et al., 2014). Financial costs for FDA-compliant data management were estimated at \$100,000 USD for a 10 patient pilot study. Appropriate data management is critically important for both enabling maximum use of any and all information, and for protecting the privacy of patients and the integrity of the research. However, the costs of FDA-compliant data management substantially reduced funds available for performing the study. Regulatory consultants and support staff to interface with the FDA and to insure that all necessary requirements for the investigational device exemption (IDE) were met also contributed to overall regulatory costs, which in the case study were ∼\$75,000 per year. It was noted that such costs are prohibitive to many "stand-alone" investigators without federal funding or institutional resource sharing.

While industry-sponsorship of a study is possible for certain indications, such support is unlikely for research involving orphan disease populations that represent a small market share of potential consumers. Time costs were also significant; in the case study presented, it took three (3) years from approval of the NIH funding to the time that the first patient was enrolled in the trial. Time costs to the investigator also include dedicated efforts preparing regulatory paperwork. For example, it was estimated that the amount of hours required to prepare documentation for submission of an IDE was equivalent to the time required to prepare three (3) NIH R-01 grant proposals.

Even given these barriers to physician-initiated DBS research, it was emphasized that physicians are not seeking "less regulation" but instead are calling for regulatory reform. In this light, it was recommended that practices such as data review, process auditing, and unannounced site visits would be welcomed and could be increased, while processes related to documentation could be streamlined. Specifically, it was recommended that a FDA-approved template for DBS-related IDEs could guide investigators to more efficiently dedicate time and resources to the components definitively required for this type of research. In addition, it was recommended that the right of reference (RoR) process be reformed. Currently, RoR requires that industry partners approve studies that use their devices; this approval is a means of protecting commercial intellectual property and corporate assets and providing protection from litigation. It was suggested that this process could be reformed by indemnifying companies from the consequences of an investigator's FDA-approved off-label use of corporate intellectual property. This reform would center responsibility for the scope and conduct of research upon the physician, rather than the industrial sponsor. In turn, this would enable physicians' control over clinical studies and, at the same time, incur benefits to industry by no longer holding industry partners accountable for reviewing (and ultimately deciding upon) research proposals. Still, this reform may not resolve device manufacturers' concerns about a potential loss of confidence in their brand should a poor outcome occur.

The DBS field is becoming increasingly competitive, with potentially sizeable prizes, and sizeable risks. Achieving a leftward shift in the time course for IIR will require, attention to the benefits and burdens incurred in biomedical, ethical, and legal domains. Toward this end, a multi-step paradigm for comprehensively addressing critical issues and, importantly, guide forward progress in DBS research and its clinical translation was described and recommended. First, an overall "6-R" stance was advocated, which encourages responsibility for: assessment of capabilities and limitations of DBS in treatment of particular neuropsychiatric conditions, research to evaluate DBS effects in practice, regulation, responsivity to incurred burdens and harms and revision of DBS technology and techniques, and regulatory process, as necessary. Meeting these responsibilities invokes a "6-W" set of questions that can be used to define the parameters of use, and "6-Cs" that must be addressed in order to establish ethical probity in use (Giordano, 2015). Details of the "6-R, 6-W, 6-C" model for the ethical development of DBS technology are presented in **Figure 2**.

While the need for continuity of care for patients involved in experimental DBS interventions is clear from ethical and clinical perspectives, the actual provision of such longitudinal care has proven to be challenging in practice (Rossi P. et al., 2014). One vexing recurrent issue is that insurance providers occasionally decline reimbursing costs for off-label DBS, despite granting "pre-approval" in a peer-to-peer review process with the insurance company's medical directors.

Given the increasing diversity of DBS approaches, cumulative data aggregation, sharing, assimilation and synthesis will be increasingly important to the iterative assessment and improvement of the field. Toward these ends recommendation was made to establish a common database for DBS research and clinical outcomes, although the question was posed how—and through which entity or institution—such a common database would be established, hosted and curated (Giordano, 2012, 2014). Modification of extant systems, and development of new information management frameworks will be required to collect and integrate, support and sustain the wide distribution of many types and levels of data. Such approaches should: establish a common data format, optimize harvesting, aggregation and synthesis, establish checking systems to assess and characterize the type and quality of data, maximize accessibility and ensure the source of data, and enable retraction of data that are inaccurate or in need of revision for currency. In light of the increasing number and internationality of IIR DBS studies, it will also be important to address issues and questions of intellectual property and proprietary use (Brindley and Giordano, 2014). To fortify IP, provenance, attribution (and relative indemnities), and data use and sharing agreements will need to be implemented to achieve a dynamic repository that supports the range of intended uses for these data (clinical care, training, and research).

#### Highlights


## INNOVATIVE TECHNIQUES AND TECHNOLOGIES IN DBS

### Functional Connectivity Tools to Guide Stimulation for Epilepsy

A critical step toward optimizing direct modulation of refractory focal-onset epilepsy is to effectively interface depth electrodes with complex epileptogenic brain circuits. Some novel approaches are currently being exploited to achieve this goal.

It was recently shown that a correlation-based measure of functional connectivity could be used to identify epileptogenic zones from intracranial stereoencephalography (SEEG) signals and that this information can be used to predict the outcome of lobectomy in intractable temporal lobe epilepsy (Antony et al., 2013). Indeed, patients with weakly connected, homogenous networks responded less favorably to temporal lobectomy. These findings suggest the value of such SEEG-based functional connectivity modeling in predicting the outcomes of depth electrode placement for epilepsy (Gonzalez-Martinez et al., 2013).

In addition, the FDA recently approved a depth electrode system as an adjunctive therapy for individuals with refractory focal-onset epilepsy with two epileptogenic sources. A novel

pre-implant depth electrode placement planning system has been shown to enable the propagation of therapeutic current to communicating distant epileptogenic sources. The preimplantation planning process consisted of several components: (1) structural magnetic resonance imaging (MRI) and diffusion tensor imaging (DTI) datasets; (2) computation of the induced electric potential surrounding the electrode contacts using a finite element method; (3) analysis of the effect of the electric fielddependent FA (fractional anisotropy) model on depolarizing axon bundles as identified by high-resolution DTI; and (4) predicting distant cortical activation by strategically placing the FA volume seeds to create a modulated circuit tractography (MCT) map. The pre-implant MCT map was then used as a targeting template for placing up to two depth leads intraoperatively. This planning system was validated via subtraction activated SPECT (SAS), which is a perfusion imaging technique that captures stimulation induced transient changes in cerebral blood flow. SAS was utilized post-implantation to validate in vivo, the maximal extent of epileptogenic regions influenced by stimulation therapy.

## Functional Connectivity Tools Enable Personalized DBS

The effect of focal brain stimulation is not limited to the region targeted and a DBS current can propagate through anatomical connections to influence distributed neural networks in the brain. Emerging techniques that can help DBS practitioners visualize these networks are likely to prove valuable for understanding and guiding brain stimulation. One imaging technique particularly well suited to visualizing brain networks is resting state functional connectivity magnetic resonance imaging (fcMRI) (Fox and Raichle, 2007). This technique has already been demonstrated to (1) identify thalamic DBS targets based on connectivity to brain regions implicated in tremor (Anderson et al., 2011), (2) link invasive and non-invasive brain stimulation sites across 14 different neurological and psychiatric diseases (Fox et al., 2014), and (3) be safely applied in patients implanted with DBS electrodes using special low-energy MRI sequences (Kahan et al., 2014).

Another method that has emerged with considerable promise is patient-specific tractography-activation models (TAMs), which can enable the identification and visualization of white matter pathways activated by brain stimulation. TAMs essentially predict action potential generation in specific pathways. They combine anatomical imaging data, probabilistic tractography from the brain region surrounding the implanted DBS electrode, models of the electrical fields generated by DBS parameter settings, and cable models of axons (Lujan et al., 2013). TAM may lead to improved personalized surgical targeting and stimulation parameter selection. TAM may also facilitate identification of new DBS targets (Downes and Pouratian, 2014), and a deeper understanding of the mechanisms underlying both the

therapeutic and off-target effects of DBS (Riva-Posse et al., 2014; Sweet et al., 2014).

Harnessing advances in neuroimaging techniques may also play a role in re-evaluating conventional thinking in the field. For example, with probabilistic diffusion tensor tractography, the four anatomical targets for DBS in cluster headache described as hypothalamic in literature were shown to be localized to the midbrain tegmentum posterior to the hypothalamus. Importantly, tractography also revealed common tracts across these targets, which included projections to the ipsilateral hypothalamus, reticular formation, and cerebellum (Clelland et al., 2014). Collectively, these results can motivate a shift from stimulation of specific brain targets to stimulation of specific brain networks.

#### Investigating Lead Placement Variability

There can be considerable variability in DBS outcomes, and some clinical trials have failed recently because of profound variability in response rate across the patient cohort. This is evident even in successful clinical trials. For example, in one of the larger DBS trials in PD the standard deviation in clinical outcome scores was roughly equal to the effect size (Deuschl et al., 2006). Over the past 10 years computational models have been used to characterize potential sources of variance in the way DBS is applied (Grill et al., 2004; Johnson and McIntyre, 2008; Dorval et al., 2009, 2010; Santaniello et al., 2011). In general, these studies attempted to characterize how and where stimulation was applied in each patient. With regard to the latter, one critical element for both surgical planning and population research has been coregistration of pre-operative patient MRI to a brain atlas. This is performed prior to surgery to permit indirect targeting of nuclei that have poor contrast on conventional imaging, and it is performed after surgery so that regions of activation for each patient can be expressed in a probabilistic atlas of outcomes (Butson et al., 2011). During this process it has been observed that lead locations often vary within and among surgical sites. This observation has led to questions about errors that could be introduced during atlas registration, and has motivated an evaluation of the accuracy of this registration process. The most important finding from this evaluation was that the observed variability in lead location cannot be attributed to errors introduced during atlas registration. In fact, three different registration algorithms yielded virtually the same results. This information supports the suggestion that the neuromodulation community could benefit from wider adoption and acceptance of open source registration algorithms, several of which have been rigorously developed and tested.

### Temporal Pattern of Stimulation as a New Dimension of Therapeutic Innovation

Although DBS is an established therapy for the treatment of movement disorders, debate persists about the mechanisms by which high frequency stimulation reduces symptoms. This probably results in a failure to achieve full optimization of the therapy with maximal benefits and minimal side effects. Thus, an improved understanding of therapeutic mechanisms will be important to enable further innovations in DBS technique and technology.

The cellular effects of DBS on neurons of the central nervous system include simultaneous inhibition of the cell body and activation of the axon (McIntyre et al., 2004). This finding motivated the "informational lesion" hypothesis positing that DBS masks pathological oscillatory activity by normalizing the activity of neurons within the stimulated nucleus. The striking parallel between the frequency-dependent effects of DBS on regularizing the activity of model neurons and the clinical effects on symptoms provides strong correlational evidence for this hypothesis (Grill et al., 2004). The informational lesion hypothesis has been tested in several recent experiments (Zimnik et al., 2015). The changes in representation of kinematic information in the firing patterns of neurons of the globus pallidus and thalamus that occurred during DBS indicate that effective DBS produces at least a partial disruption of neural information (Agnesi et al., 2013).

In contrast, a more recent study concluded that DBS does not disrupt information transmission in the basal ganglia (Zimnik et al., 2015). However, the currents used in this study were orders of magnitude smaller than those required for positive effects on symptoms, and therefore the effects of DBS on neuronal activity were substantially underestimated. Using a highly innovative paradigm to render a temporary direct connection to the brain lead during surgical replacement of the implantable pulse generator (IPG) enabled another test of this hypothesis (Swan et al., 2014). Random patterns of subthalamic nucleus DBS, even when delivered at a high average frequency (130 Hz), were not effective in relieving bradykinesia in patients with Parkinson's disease. These findings reinforced the potential importance of regularization (rather than complete disruption) of neuronal activity for the effectiveness of DBS (Dorval et al., 2010).

The finding that the effects of DBS were dependent on the temporal pattern of stimulation, in addition to the frequency of stimulation, inspired the design and testing of novel temporal patterns of DBS. Patterns were developed that treat the symptoms of Parkinson's disease (PD) more effectively than conventional regularly patterned DBS (Brocker et al., 2013). or alternatively enabled equivalent treatment of symptoms but with a substantial reduction in the required energy. This latter innovation is an important consideration for the size, recharge frequency, and battery life of implanted pulse generators. Collectively, the results demonstrated the utility of an entirely new dimension of neural stimulation parameters—the timing between stimulation pulses—to increase the efficacy and efficiency of stimulation.

### Advancements in Lead Design

Emerging DBS device technology will enable controlling of stimulation fields. Three novel lead designs—all with uniquely engineered approaches to achieving this objective were presented. Two of the presented leads feature electrodes segmented radially about the lead, in contrast with existing leads that are segmented only along the long axis of the lead. The third lead featured an extended span, and brings with it a new kind of current control to DBS. These new leads will allow the stimulating currents to be programmed in order to preferentially stimulate therapeutic targets and avoid stimulating areas prone to side effects. The importance of tuning and directing the stimulating field is based on the observation that electrodes are occasionally sub-optimally placed (Okun et al., 2005) and sometimes quite far from the intended target (Ellis et al., 2008). All three of the lead designs are currently being evaluated in various clinical trials (Contarino et al., 2014; Pollo et al., 2014; Vitek and Starr, 2015). The use of directional electrodes can improve DBS outcomes when electrodes are sub-optimally placed. Preliminary data presented at the Think Tank indicated that targeted stimulation with these approaches is promising, although the long-term benefits of the "directional" leads remain to be demonstrated, particularly in a chronic study. **Figure 3** provides a detailed comparison of these new lead designs.

One lead under investigation (Medtronic-Sapiens) possesses an advanced multiplexer unit that supports a total of 40 electrodes and a span of 7.41 mm. With 10 rows of 4 electrodes per row, and alternating rows offset by 45◦ , 8 radial electrode directions are possible. Stimulation can be further shaped and aimed radially by choosing various combinations of active electrodes and splitting the current between them; however, details are not yet public regarding how electrodes may be combined and programmed. Additionally, recording of local field potentials (LFPs) is possible from each of the 40 electrodes potentially yielding spatiotemporal information on pathologic neuronal activity. Preliminary intraoperative testing of this system suggests that it may be possible to utilize intraoperative LFP recordings to assess the effect of stimulation in different electrode combinations and current settings on pathological subthalamic electrical activity (Bour et al., 2015). This field shaping capability can possibly avoid stimulation of unwanted regions and enhance engagement of target areas (Barbe et al., 2014).

Another recently tested lead (Aleva) has a total of eight electrodes (span, 5.5 mm): two are the traditional ring electrodes and the remaining two rings are divided into three segments each, allowing for directional current delivery through each segment (Chase, 2014; Pollo et al., 2014). The electrode corners are rounded so as to avoid "hot spots." Clinical data suggest that this directional lead, when tested in the acute setting in either the subthalamic nucleus (STN) or the ventralis intermedius (VIM) thalamus, can improve the therapeutic index, i.e., enlarge the window between therapeutic effect and adverse effects, and also may possibly use less current to achieve the same therapeutic benefits (Chase, 2014; Pollo et al., 2014).

Another lead (Boston Scientific DB-2201; currently undergoing clinical evaluation in the United States) also possesses eight electrodes (15.5 mm span), current on each of which can be precisely controlled, as the IPG is capable of current steering. The aim of this design, also referred to as Multiple Independent Current Control (MICC), is to achieve more precise targeting of stimulation by enhancing control of the therapeutic electric field. As the IPG associated with this lead is a current control design, it should theoretically enable more stable stimulation. By reducing variability in impedance and permitting the effective use of lower pulse widths, it should also be possible to expand the therapeutic window of effective current amplitudes. Results of a European trial have shown clinical outcomes comparable to existing leads, and during the trial over 70% of programmers utilized the current steering feature (although motivations for this use have not yet been evaluated; Timmermann et al., 2015). The system also has cordless recharging, and the rechargeable IPG utilizes cero volt technology, which helps prevent substantial loss in battery capacity following frequent and/or full discharges. Future capabilities that could be co-deployed with this lead were discussed, such as computer-guided distribution of current across contacts for an optimized or informed programming.

### Non-DBS Technology Impacting the Field: Focused Ultrasound

In addition to addressing important improvements in DBS system technology, the Think Tank sought to identify "non-DBS" approaches that could impact the DBS field. In this regard, the use of transcranial high intensity focused ultrasound (HIFU) for treatment of movement disorders was reviewed (Dallapiazza et al., 2014). While effective, stereotactic lesioning of the brain for the treatment of movement disorders has been largely abandoned with the development of DBS (Dallapiazza et al., 2015). However, progress in transcranial MR-guided focused ultrasound technology has renewed an interest in stereotactic lesioning mainly because of the potential for continuous MRIguidance of an "incisionless" thalamotomy (Wang et al., 2015). Three pilot studies at different institutions have demonstrated significant improvements in hand tremor in patients with severe essential tremor following focused ultrasound thalamotomy (Elias et al., 2013; Wintermark et al., 2014a,b). These studies have suggested functional improvements in activities, disabilities, and quality of life with minimal morbidity. Furthermore, transcranial ultrasound at low intensities can be used to manipulate deep brain circuitry through non-invasive brain mapping prior to lesioning. Clinical trials of HIFU for mapping neural circuitry and treating in essential tremor and PD are currently ongoing.

Proponents of HIFU further highlight that an incision and burr hole are not required to perform the procedure, offering a "lower-cost, less invasive" alternative to DBS that eliminates both the risks of penetrating the brain and the inconvenience and costs imposed by implanted hardware (Lipsman et al., 2013). Dissenters believe this to be an oversimplification. Unlike DBS surgery, HIFU requires the head to be completely shaved; patients must remain awake during the procedure and must lie flat within the MRI scanner for a few hours while the target is localized. The MRI environment, while offering the potential for procedural monitoring, can be difficult to work in and some patients cannot tolerate these image-guided procedures. In comparison, DBS targeting scans are obtained in just a few minutes and patients are positioned more comfortably during surgery. Most importantly, HIFU is an ablative and irreversible lesion, and can result in adverse effects especially when used bilaterally.

Direct comparisons of traditional (non-HIFU) thalamotomy and thalamic DBS have already been performed (Tasker, 1998; Schuurman et al., 2000; Pahwa et al., 2001). Three studies, conducted at reputable centers in the U.S., Canada, and

Reference: St. Jude DBS Brochure 2010, St. Jude DBS Product Catalog 2011, Medtronic DBS 3387/3389 Lead Kit Manual.

Europe have all reached a similar conclusion. While initial tremor control was comparable for the two interventions, thalamic DBS was safer than thalamotomy, causing fewer neurological complications and reducing the need for reoperation in the event of tremor recurrence. Indeed, one member of the University of Virginia HIFU cohort suffered permanent dysesthesia (Elias et al., 2013), and a follow-up study from that group suggests that tremor control may lessen over time as the lesion shrinks in size (Wintermark et al., 2014a). However, it is worth noting that ET can also become refractory to or tolerant of DBS despite increasing currents (Favilla et al., 2012). Clearly, there is a subset of patients who will subjectively prefer HIFU to DBS; however, it remains unknown whether HIFU ablation is objectively better or cheaper than DBS of the thalamus or any other target. Moreover, there are concerns over limitations of this technology including the need for bilateral procedures, the safety profile, and the lack of programmability.

#### Highlights

1. SEEG-based functional connectivity modeling is helping to predict outcomes of depth electrode placement for epilepsy (Gonzalez-Martinez et al., 2013).


### CLOSING THE LOOP WITH LOCAL FIELD POTENTIALS

#### Local Field Potentials Provide Insight into Neuropsychiatric Disorders

DBS has shown promise as a therapy for neuropsychiatric disorders; however, advances have been hindered by relatively poor understanding of the neural networks involved in the pathophysiology of these conditions. To address this gap, an ongoing study by researchers at University of California San Francisco is investigating the mechanisms underlying depression and anxiety in PD patients undergoing awake DBS surgery.

This population of patients is ideal for studying the circuits involved in these depressive and anxious symptoms because they frequently co-occur in varying degrees of severity in Parkinson's disease (Gallagher and Schrag, 2012). These symptoms may be modulated by both DBS and dopamine replacement therapies (Tan, 2012; Storch et al., 2013). High spatial resolution recordings of neuronal activity are performed intraoperatively within brain structures that have been shown to be involved in emotion regulation and cognitive control. Field potentials are recorded from the DBS lead (containing 4 electrode contacts) implanted in the basal ganglia (either the subthalamic nucleus or globus pallidus) and from an additional subdural electrode (containing 28 contacts, spaced 2 mm apart) placed over the prefrontal cortex. Cortical areas targeted include the dorsolateral/medial prefrontal cortex, the orbitofrontal cortex, and the ventrolateral prefrontal cortex. Recordings are performed while patients rest or perform tasks that engage networks involved in emotion regulation and cognitive control. In addition, cortical and subcortical stimulation are used to modulate mood. To identify correlations between neural physiology and symptom severity, depression, mood, and anxiety are characterized both before the surgery using a variety of validated scales, and during the surgery using visual analog mood scales.

Preliminary results of these experiments in the prefrontal cortex suggest that, similar to what has been observed in the motor cortex (Shimamoto et al., 2013; Yang et al., 2014; de Hemptinne et al., 2015), the prefrontal cortex is dominated by beta oscillations that can be coupled to broadband gamma activity (Hammond et al., 2007). It was found that the magnitude of beta rhythm and phase-amplitude coupling varies between patients. In addition, in few patients, it was shown that stimulation delivered to the deepest contact of the DBS lead can induce anxiety, and this effect on mood is associated with a decrease in beta activity and an increase in broadband gamma. Although preliminary, these findings suggest that both the beta oscillations and broadband gamma activity might be relevant to psychiatric symptoms, and that the excessive synchronization observed in cortical-basal ganglia motor networks might also occur in cortical-basal ganglia networks involved in emotion and cognition.

### Long-Term Cortical and Subcortical Local Field Potentials (LFPs) in Parkinson's Disease

While DBS is an effective treatment for movement disorders such as PD there are several ways that it might be improved. The current approach is to deliver constant stimulation without adjusting the therapy or controlling for the patient's disease state, medication status, or side effects. A goal of advancing DBS therapy is to implement a "closed-loop" system where electrical signals from the patient's brain are used in real-time as feedback to customize stimulation delivery. Closing the loop can be used to reduce undesired side effects of stimulation and to extend battery life, as well as to improve stimulation effectiveness.

An important step in the development of closed-loop DBS for PD is the characterization of brain signals associated with the disease-relevant network. A recent on-going study aimed at identifying pathophysiological activity related to PD implanted five patients with novel devices capable of both stimulating and long-term recording and storing of LFP data. In this study, patients are implanted with a DBS electrode in the STN capable of both stimulation and sensing/recording electrical activity. In addition, 4-contact electrocorticography (ECoG) strip placed over the primary motor cortex (M1), which is used for sensing/recording only. Each patient is tested multiple times, both on and off medication, and on and off DBS. Preliminary results reveal that in most cases, medication is associated with a reduction in beta (13–30 Hz) power in the STN, while no consistent changes in beta power are recorded from M1. While these results support the importance of beta synchrony throughout basal ganglia-thalamo-cortical loops in PD, they also suggest that there is variability between patients, and that a closed loop signal may need to be optimized on a patient-bypatient basis, and/or that a combination of control signals may be needed.

In addition to identifying markers that are of relevance to hallmark PD symptoms, this study also aimed to assess adverse effects associated with dopaminergic therapy such as the dyskinesias. In two patients experiencing marked contralateral arm dyskinesia, a consistent and reproducible emergence of a narrow-band ∼70 Hz increase in cortical power was observed. There was also increased coherence in the same frequency range between STN and M1. Similar patterns of high frequency, narrow-band activity has been previously observed in a rodent model of dyskinesia (Halje et al., 2012), and these may be useful for closed loop approaches.

### Long-Term Cortical and Subcortical LFPs in Tourette's Syndrome

Tourette Syndrome (TS) is a paroxysmal neuropsychiatric disorder characterized by involuntary movements and/or vocal outbursts (i.e., - tics) typically preceded by a premonitory urge (Cheung et al., 2007; Kenney et al., 2008). DBS has been used to treat cases of severe and intractable TS (Almeida et al., 2015). It is estimated that ∼120 TS patients worldwide have been treated with DBS since 1999, and almost 48 published studies report some degree of motor tic reduction (Schrock et al., 2015). While initial trials have been promising, the mechanisms subserving the effectiveness of DBS in reducing TS signs and symptoms have yet to be identified. Current models of TS hypothesize that thalamocortical-basal ganglia dysfunction is a key network underlying many TS symptoms. Inhibitory input from basal ganglia structures affecting the activity of key thalamic nuclei likely plays a role in patterns of motor behaviors. It may be that inhibition of basal ganglia structures leads to disinhibition of thalamic nuclei, which ultimately evokes initiation of tics (Perlmutter and Mink, 2006). Previous research has demonstrated potential biomarkers of tics (Maling et al., 2012; Bour et al., 2014). Building upon this work, the validity, reliability and relative predictive value of these biomarkers, and the development of an algorithm that can be used in the early detection of tics was presented.

In the presented study, two patients with severe, medication refractory TS were implanted with bilateral DBS devices. Depth leads were placed in the centromedian-parafascicular nucleus of the thalamus (CM-PF) and electrocorticography (ECOG) strips were placed over the precentral gyrus. Experiments consisted of separate interleaved trials in which patients were instructed to (1) tic freely, (2) suppress tics (baseline), and (3) execute volitional movements (shaking hands rapidly, opening and closing hands, raising arms up, and down, talking). Data were recorded intraoperatively and post-operatively. Intraoperative recordings demonstrated that both significantly more low (1–10 Hz) and high (30–100 Hz) frequency CM-PF activity was present during tics but not during volitional movements. A support vector machine (SVM)- based detector (Temko et al., 2011; Wissel et al., 2013) was constructed to investigate the relationship between this activity and tics during each post-operative visit (for a period of 6 months). Three types of tics were recorded including simple, complex, and long complex tics. Long complex tics were shown to be concurrent with a consistently detectable thalamocortical signature. Short complex tics were more difficult to detect than long complex tics, and simple tics were the most difficult to detect. Acute trials of closed loop stimulation using the human tic detector are currently underway.

#### Local Field Potentials and Depression

A confluence of information prompted exploration of the subcallosal cingulate gyrus (SC) as a DBS target for treatment resistant depression (TRD) (Lozano et al., 2008). Attempts to reduce TRD with SC DBS have shown clinical benefit: a recent study involving 10 patients showed significant response and remission rates following SC DBS (Holtzheimer et al., 2012). However, numerous other targets for TRD, including the nucleus accumbens, dorsolateral prefrontal cortex, and the lateral habenula, have been proposed and subsequently explored in clinical studies, with many of these showing at least some evidence of clinical benefit (Rosa and Lisanby, 2012). The question then arises if and how these targets may be related. The conceptualization of depression as a network disorder suggests that neuromodulation at the purported origin as well as at "nodes" of the network can be beneficial (Mayberg, 2009). TAMs of patients who underwent SC DBS for depression demonstrate significant differences between those who responded to therapy and those who did not (Riva-Posse et al., 2014). Specifically, it was shown that responders had greater tract coverage in critical regions.

Recent work has focused on using this information to improve DBS targeting. Prospective work is currently underway to utilize DTI pre-operatively to plan lead placement. This approach has been associated with an increase in the 6-month response rate from 41 to 76%. An important next stage of research will be to identify other markers that can confirm if a lead has been placed in a location that would elicit maximal benefit. Ongoing studies aimed at identifying physiological markers useful in evaluating lead placement and predicting treatment response were presented. These studies focused on intraoperative LFPs from the DBS electrodes in conjunction with intraoperative ECoG and electroencephalograhy (EEG). Previous studies have identified changes in alpha rhythm in depression; the alpha band frequency is increased in the left frontal lobe in depression (Saletu et al., 2010) while alpha is decreased in the right prefrontal cortex (PFC) (Bruder et al., 2001). Preliminary studies of intraoperative LFPs from both ECoG and depth electrodes after stimulation at "effective" contacts (determined by imaging) have demonstrated decreases in the alpha and beta frequencies in the left PFC, as well as a decrease in alpha and beta bands in the subgenual cingulate.

### Local Field Potentials and Strides Toward a Closed-Loop DBS

DBS systems currently deployed in the clinic are "open-loop" and do not take into account the potentially intermittent nature of symptoms. By detect the neurophysiologic correlates of symptoms such as tremor, we can determine not only when stimulation may be necessary but also estimate the intensity of stimulation needed. Stimulating only when necessary can increase the battery life of the implanted devices and reduce a patient's exposure to unintended effects.

A novel mobile, wireless platform for investigating closedloop DBS applications in ambulatory patients was presented (Herron and Chizeck, 2014). The platform consisted of a set of body-worn sensors communicating wirelessly to a host application running on a smartphone or a personal computer. Taking advantage of movement data including inertial measurements, electromyography, and LFPs, these host applications are capable of performing digital signal processing and data fusion in order to make control decisions. These control decisions can include enabling or disabling stimulation or modifying individual stimulation parameters (voltage, pulse width, frequency) in response to changes in neurological symptoms (Herron et al., 2015). These control decisions are then sent wirelessly to an external receiver that then relays packets and control decisions to an implanted neurostimulator. This realtime command link to the implanted device has enabled the implementation of an integrated closed-loop DBS system.

This system confers several important benefits for both research and patient care. Currently, studies are underway to assess clinical performance of the system, and future studies are being planned which utilize the wealth of consistent, chronic data generated from the integrated system to investigate neurological movement disorder, particularly tremor.

#### Highlights


### INNOVATIVE TARGETS FOR NEW INDICATIONS

### Lateral Habenula as a Target for Depression

In 2007, a non-human primate study provided the first electrophysiological evidence that the lateral habenula (LHb) played a role in the brain's reward system. The study demonstrated that reward was associated with suppression of inhibitory input from the LHb and subsequent activation of dopamine neurons, while the reverse was observed for nonrewarding trials (Matsumoto and Hikosaka, 2007). That same year, Alex Sartorius and Fritz Henn advanced the hypothesis of over activation of the habenula in major depressive episodes and argued that DBS of the lateral habenula could be beneficial for TRD (Sartorius and Henn, 2007). More recent studies in humans have corroborated the notion of LHb involvement in the reward system (Salas et al., 2010), and, interestingly, two case reports have since been published (2010, 2013) showing remission of major depression under DBS of the LHb (Sartorius et al., 2010; Kiening and Sartorius, 2013; Schneider et al., 2013). Early results from an on-going open-label trial of six patients undergoing habenular DBS for TRD were presented. The hypothesis that patients who were responsive to electroconvulsive therapy (ECT)—even if for a short duration could be "better" candidates for habenular DBS was proposed, and is under investigation in the present study. As a DBS target, the LHb poses unique challenges for electrode implantation among these is the proximity of the target to the third ventricle and the resultant motion artifact observed in imaging studies due to the pulsatile movement of the cerebro-spinal fluid (CSF). Programming-related adverse events observed to date were discussed. Upper extremity paresthesias occurred commonly, although patients mostly habituated to this and the effect was modifiable by gradually increasing the current. Oculomotor abnormalities were also observed which limited current dosing, and it was proposed that current steering could be useful in limiting this side effect.

### DBS of the Basolateral Nucleus of the Amygdala for PTSD

Neuromodulation of the amygdala may prove beneficial in disease processes where symptoms arise from the aberrant assignment of an emotion to a specific event or context (Langevin, 2012). In post-traumatic stress disorder (PTSD), patients assign fear to benign situations, which may potentially lead to avoidance behavior. Several neuroimaging studies have demonstrated that the amygdalae of PTSD patients are metabolically overactive during symptomatic episodes (Etkin and Wager, 2007; Hughes and Shin, 2011). The level of activity within the amygdala correlates with the severity of the symptoms as measured by the clinician-administered PTSD scale (CAPS). An important study by Koenigs and colleagues showed that Vietnam veterans who suffered traumatic brain injury to the amygdala never developed PTSD (Koenigs et al., 2008). These results suggest that the amygdala plays a critical role in the production of PTSD symptoms. Focal interference of amygdala activity through DBS may improve PTSD. Although its mechanism is not fully understood, high frequency DBS is thought to functionally inactivate a specific, gray matter target. It was previously shown that DBS of the basolateral nucleus of the amygdala (BLn) reduced the behavior associated with PTSD in a rodent model (Langevin et al., 2010). A subsequent study in the same rodent model demonstrated that BLn DBS was superior to paroxetine one of the drugs approved by the FDA to treat PTSD (Stidd et al., 2013). These results led to the recent development of a clinical trial to evaluate the feasibility and the safety of this technique in PTSD patients (Koek et al., 2014). An overview of the trial design was presented, as well as an overview of the first surgical subject implanted. In addition, the targeting technique and intra-operative microelectrode recording findings were described. Consideration of the patient's neuroanatomy is critical because of the wide variation in size and shape of the mesiotemporal structures.

## DBS for Stroke

The concept of post-stroke neurostimulation does not focus on modulation of the area damaged by ischemia, rather it is intended to (1) augment the perilesional cortex or (2) modulate other areas whose connectivity has been disrupted by the stroke. Direct cortical stimulation as a means of enhancing excitability and plasticity has been investigated (Alonso-Alonso et al., 2007), but has failed to produce the intended benefits in clinical trials. One possible reason for the failure of cortical stimulation was the relationship between cortical axons and the source of stimulation (Manola et al., 2005a,b). In rodent models, axonal arrangement perpendicular to the cathode is predictable. However, in human brains, axonal arrangement is less predictable, and it is possible that axons were inhibited and excited in near equal proportions during cortical stimulation interventions and the net effect washed out. A novel approach to stroke therapy via DBS was discussed; the intention was to stimulate natural fiber pathways to the perilesional cortex in a way that mimics their native function. Preliminary results of this design in a rodent model were presented. Based on motor pathway fiber tracing by Dum and Strick, a cerebellar target (lateral cerebellar nucleus) was selected with the goal of modulating the dentatothalamocortical pathway (Dum and Strick, 2002; Machado and Baker, 2012). Stimulation of this target (particularly at 30 Hz) was shown to evoke cortical excitability. A follow-up study tested the effect of chronic 30 Hz DBS in this target on motor function in a rodent model of stroke (Machado et al., 2013). Animals in the stimulation group showed a significant improvement in motor function compared with post-ischemia baseline performance as well as in comparison with the non-stimulation group (Machado et al., 2013). Moreover, perilesional synaptic density testing showed that animals in the stimulation group had significantly greater numbers of perilesional synapses. Preliminary results of on-going studies of cytoarachitecture in these animals addressing whether these observations reflect neurogenesis in addition to synaptogenesis were also presented. Substantial discussion surrounded the topic of the appropriate level of pre-clinical evidence needed to justify early clinical translation studies.

#### Highlights


### CONCLUSION

These proceedings represent the deliberations of the third Annual Deep Brain Stimulation Think Tank. The group addressed critical issues affecting the progress of the DBS field. These issues span multiple domains, including regulatory and ethical issues as well as study design. There are also important barriers to advance electrophysiology and system engineering. In discussing these challenges, participants in the Think Tank proposed and discussed possible solutions.

#### REFERENCES


Scientific, clinical, and engineering advances that could transform the DBS field in the near future served as a primary focus of the Think Tank. The meeting focused on recent discoveries that may lead to transformative, not just incremental change in DBS therapy. Participants discussed the broad range of potential applications of the new knowledge, techniques, and technologies presented; they also discussed ways in which these advances could be fully exploited to rapidly advance to the next generation of DBS therapy. The future of DBS will depend heavily on building on these advances and on filling knowledge current gaps.

### AUTHOR CONTRIBUTIONS

All authors listed, have made substantial, direct and intellectual contribution to the work, and approved it for publication.

between responders and nonresponders to an SSRI antidepressant. Biol. Psychiatry 49, 416–425. doi: 10.1016/S0006-3223(00)01016-7


potential field, activating functions and responses of simple nerve fibre models. Med. Biol. Eng. Comput. 43, 335–343. doi: 10.1007/BF02345810


a review and updated recommendations. Mov. Disord. 30, 448–471. doi: 10.1002/mds.26094


**Conflict of Interest Statement:** PR: None; AG: None; UA: None; RA: consulting fees and research support from Medtronic, Inc.; HC: Donation support from Medtronic Inc., Research support from National Science Foundation (Engineering Research Center for Sensorimotor Neural Engineering); CB: consultant for Boston Scientific, St Jude Medical, NeuroPace, Intelect Medical, Advanced Bionics and Functional Neuromodulation. CB is a shareholder of Intelect Medical. CB has authored intellectual property related to deep brain stimulation; CD: Aspects of work described in this review were supported through DARPA SUBNETS, DARPA/ARO Contract # W911NF-14-2-0043; WE: research funding from: Insightec, Ltd. and Focused Ultrasound Foundation; GG: None; MF: supported from NIH grant K23NS083741 and the Sidney Baer Foundation, listed as an inventor on submitted or issued patents on guiding neurological interventions with fMRI; JG: grant support from the William H. and Ruth Crane Schaefer Endowment; Children's Hospital and Clinics Foundation; the Clark Foundation; and the United States Air Force Office of Scientific Research; RG: consultant for Medtronic, Inc. and St. Jude Medical Corp., receives compensation for these services; WMG: inventor on licensed patents on temporal patterns of deep brain stimulation and owns equity in Deep Brain Innovations, LLC.; JH: Donation support and consulting fees from Medtronic. TH: None; JJ: None; BK: consultant for Medtronic SNT, St Jude Neuromodulation, MRI interventions, NP: fellowship support from Medtronic, paid consultant for Second Sight Medical Products, Inc.; ALang: advisor for Abbvie, Allon Therapeutics, Avanir Pharmaceuticals, Biogen Idec, Boerhinger-Ingelheim, Bristol Myers Squibb, Ceregene, Cipla, Intekrin, Lilly, Medtronic, Merck, Novartis, NeuroPhage Pharmaceuticals, Teva and UCB, honoraria from Medtronic, Teva, UCB, AbbVie, grants from Brain Canada, Canadian Institutes of Health Research, Edmond J Safra Philanthropic Foundation, The Michael J. Fox Foundation, the Ontario Brain Institute, National Parkinson Foundation, Parkinson Society Canada, Physicians Services Incorporated (PSI), Tourette Syndrome Association, and W. Garfield Weston Foundation, publishing royalties from Saunders, Wiley-Blackwell, Johns Hopkins Press, and Cambridge University Press, has served as an expert witness in cases related to the welding industry; CL: None; ALozano: consultant for Aleva, St. Jude Medical, Boston Scientific, Medtronic, Functional Neuromodulation, Inc.; JL: None; AM: consultant for Spinal Modulation Functional Neuromodulation, St Jude Medical, Icahn School of Medicine; Distribution rights to: Enspire, ATI, Cardionomics; Fellowship Support from Medtronic, Research Support from National Institutes of Health.; DM: None; CM: paid consultant for Boston Scientific Neuromodulation and a shareholder in the following companies: Surgical Information Sciences, Inc.; Autonomic Technologies, Inc.; Cardionomic, Inc.; Enspire DBS, Inc.; Neuros Medical, Inc.; AYM: Medtronic: consulting and research support St. Jude: consulting and research support, Boston Scientific: research support; LVM: teaching honorarium from Medtronic, Inc; clinical trial agreement (site PI) with Medtronic, Inc., US WorldMeds LLC, AbbVie, through Quintiles, Avanir Pharmaceuticals Inc.; The Michael J. Fox Foundation, St. Jude Medical Inc.; fellowship grant from Medtronic, Inc.; consortium grant from NIH Research Grant R01 NS40902/P.I. Corcos; RM: None; KGO: None; EO: None; AP: None; AS: None; KGS: paid employee of Boston Scientific Neuromodulation; JS: None; NS: None; AT: research funding from The Michael J. Fox Foundation and Barrow Neurological Foundation, Consultant/Scientific Advisory Board member for Medtronic, Boston Scientific, Teva, St Jude, and Theravance, Royalties from Oxford University Press. MT: consulting fees and research support to my institution from Medtronic, St Jude, and Boston Scientific, and speaker honoraria from Medtronic.; LS: None; CV: None; NV received a Medtronic Fellow Education Scholarship 6/2014-6/2015"; LV: Clinical trial agreement with Medtronic, AbbVie, Avanir Pharmaceuticals, and St. Jude Medical. Teaching honorarium and fellowship grant from Medtronic. Consultant for St. Jude Medical and Depomed; KZ: None; KF: None; MO: consultant for the National Parkinson Foundation; research grants from NIH, NPF, the Michael J. Fox Foundation, the Parkinson Alliance, Smallwood Foundation, the Bachmann-Strauss Foundation, the Tourette Syndrome Association, and the UF Foundation; participated in CME activities on movement disorders sponsored by PeerView, Prime, and by Vanderbilt University.

The reviewer FH declared a past co-authorship with author GW and a shared affiliation with authors GW and BK to the handling Editor, who ensured that the process met the standards of a fair and objective review.

Copyright © 2016 Rossi, Gunduz, Judy, Wilson, Machado, Giordano, Elias, Rossi, Butson, Fox, McIntyre, Pouratian, Swann, de Hemptinne, Gross, Chizeck, Tagliati, Lozano, Goodman, Langevin, Alterman, Akbar, Gerhardt, Grill, Hallett, Herrington, Herron, van Horne, Kopell, Lang, Lungu, Martinez-Ramirez, Mogilner, Molina, Opri, Otto, Oweiss, Pathak, Shukla, Shute, Sheth, Shih, Steinke, Tröster, Vanegas, Zaghloul, Cendejas-Zaragoza, Verhagen, Foote and Okun. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Detecting a Cortical Fingerprint of Parkinson's Disease for Closed-Loop Neuromodulation

#### Kevin Kern<sup>1</sup> , Georgios Naros <sup>1</sup> , Christoph Braun2, 3, Daniel Weiss <sup>4</sup> and Alireza Gharabaghi <sup>1</sup> \*

*<sup>1</sup> Division of Functional and Restorative Neurosurgery and Centre for Integrative Neuroscience, Eberhard Karls University Tuebingen, Tuebingen, Germany, <sup>2</sup> Magnetoencephalography Center, Eberhard Karls University Tuebingen, Tuebingen, Germany, <sup>3</sup> Center for Mind/Brain Sciences (CIMeC), University of Trento, Italy, <sup>4</sup> Department for Neurodegenerative Diseases and Hertie Institute for Clinical Brain Research and German Centre of Neurodegenerative Diseases (DZNE), Eberhard Karls University Tuebingen, Tuebingen, Germany*

Recent evidence suggests that deep brain stimulation (DBS) of the subthalamic nucleus (STN) in Parkinson's disease (PD) mediates its clinical effects by modulating cortical oscillatory activity, presumably via a direct cortico-subthalamic connection. This observation might pave the way for novel closed-loop approaches comprising a cortical sensor. Enhanced beta oscillations (13-35 Hz) have been linked to the pathophysiology of PD and may serve as such a candidate marker to localize a cortical area reliably modulated by DBS. However, beta-oscillations are widely distributed over the cortical surface, necessitating an additional signal source for spotting the cortical area linked to the pathologically synchronized cortico-subcortical motor network. In this context, both cortico-subthalamic coherence and cortico-muscular coherence (CMC) have been studied in PD patients. Whereas, the former requires invasive recordings, the latter allows for non-invasive detection, but displays a rather distributed cortical synchronization pattern in motor tasks. This distributed cortical representation may conflict with the goal of detecting a cortical localization with robust biomarker properties which is detectable on a single subject basis. We propose that this limitation could be overcome when recording CMC at rest. We hypothesized that—unlike healthy subjects—PD would show CMC at rest owing to the enhanced beta oscillations observed in PD. By performing source space analysis of beta CMC recorded during resting-state magnetoencephalography, we provide preliminary evidence in one patient for a cortical *hot spot* that is modulated most strongly by subthalamic DBS. Such a spot would provide a prominent target region either for direct neuromodulation or for placing a potential sensor in closed-loop DBS approaches, a proposal that requires investigation in a larger cohort of PD patients.

Keywords: deep brain stimulation, Parkinson's disease, beta cortico-muscular coherence, source space, neurophysiological biomarker, cortical targeting, closed-loop stimulation

## INTRODUCTION

Cortico-subcortical networks of Parkinson's disease (PD) patients are characterized by pathological circuit dynamics such as dysfunctional synchronization in the beta-frequency (12–35 Hz) band (Little and Brown, 2014). Dopaminergic medication and electrical stimulation may rebalance and clinically improve these altered interactions both locally and on a network level

#### Edited by:

*Michela Chiappalone, Istituto Italiano di Tecnologia, Italy*

#### Reviewed by:

*Nader Pouratian, University of California, Los Angeles, USA Daniela S. Andres, University Zurich and Federal Institute of Technology (ETH Zurich), Switzerland*

\*Correspondence:

*Alireza Gharabaghi alireza.gharabaghi@uni-tuebingen.de*

#### Specialty section:

*This article was submitted to Neuroprosthetics, a section of the journal Frontiers in Neuroscience*

Received: *30 October 2015* Accepted: *07 March 2016* Published: *30 March 2016*

#### Citation:

*Kern K, Naros G, Braun C, Weiss D and Gharabaghi A (2016) Detecting a Cortical Fingerprint of Parkinson's Disease for Closed-Loop Neuromodulation. Front. Neurosci. 10:110. doi: 10.3389/fnins.2016.00110*

(Kühn et al., 2008; Litvak et al., 2011; Eusebio et al., 2012; Hirschmann et al., 2013b; Weiss et al., 2015). Even in PD patients with early motor complications, deep brain stimulation (DBS) of the subthalamic nucleus (STN) provides better relief (alleviating motor symptoms and improving life quality) than medication (Schuepbach et al., 2013).

Two lines of evidence nominate the cortex as a target for modulating the dysfunctional network dynamics in PD: In patients, cortical beta activity precedes pathological subcortical synchronization (Marreiros et al., 2013). Furthermore, findings in both animal models of DBS (Gradinaru et al., 2009; Li et al., 2012) and patients (de Hemptinne et al., 2015) suggest that anti-dromically activated responses in the motor cortex are involved in the beneficial effects of STN DBS. Although previous studies of electrical cortical stimulation (CS) in PD revealed lower efficacy compared to DBS (Cilia et al., 2007; Moro et al., 2011; Bentivoglio et al., 2012), recent technological developments of implantable devices for simultaneously sensing and stimulating (Afshar et al., 2012) have refueled the interest in CS by applying closed-loop devices (Rosin et al., 2011; Beuter et al., 2014). Simultaneous magnetoencephalography (MEG) and local field potential (LFP) recordings in the STN of PD patients (using implanted DBS electrodes with externalized leads) detected long-range functional connectivity between STN and the ipsilateral sensorimotor and premotor cortex in the beta frequency range, suggesting the involvement of the hyperdirect pathway (Hirschmann et al., 2011; Litvak et al., 2011). More recently, electroencephalography (EEG) has also been applied to capture cortical beta-gamma phase-amplitude coupling (PAC) as a functional marker in PD while demonstrating its modulation by levodopa (Swann et al., 2015). However, the elevation of the EEG PAC signal in PD patients compared to healthy controls was not demonstrable on a single subject basis, but only at the group level (Swann et al., 2015). We therefore still lack a non-invasive technique that could be applied before surgery on a single subject basis to functionally localize a cortical target area informative on pathological beta oscillatory characteristics for either direct CS or as a sensor for closed-loop DBS.

There is converging evidence, that enhanced cortical beta activity is a prominent feature in the pathophysiology of PD (Kühn et al., 2008; Eusebio et al., 2011; Airaksinen et al., 2012; Little and Brown, 2012). Moreover, the hyperdirect pathway, connecting the STN and the motor cortical area is believed to mediate a DBS-related decrease of this exaggerated beta activity in parallel to alleviation of bradykinesia-rigidity symptoms (Kühn et al., 2006; Whitmer et al., 2012). Therefore, the most intuitive candidate marker is expected within the beta frequency range in order to detect a reliable cortical biomarker for PD. Oscillatory activity in the beta range, however, is widely distributed over the cortical surface. Additional measures are warranted to spot the cortical area reliably modulated by DBS. Cortico-muscular coherence (CMC) might provide such an additional measure. During motor tasks, CMC generally displays a spatially distributed cortical representation (Crone et al., 1998a,b; Grosse-Wentrup et al., 2011; Hipp et al., 2011), thereby conflicting with the goal of detecting a cortical hot spot. There is also a large inter-subject variability of CMC in PD patients during a motor task (Kühn, 2004; Kühn et al., 2008; Tropini et al., 2011; Weiss et al., 2012; Selzler et al., 2013) most probably explaining the variable findings in previous motor task related CMC studies in PD (Airaksinen et al., 2015).

We propose that this limitation could be overcome when performing the CMC measurement at rest. We suggest that—unlike in healthy subjects—(I) CMC would be detectable in the PD condition even in the absence of movement due to the disease-specific enhanced beta oscillations. Furthermore, we hypothesize that (II) the cortical CMC spot would topographically converge with the cortical spot modulated most strongly by subthalamic DBS. To provide proof-of-concept evidence, we implemented a MEG set-up for resting-state CMC and DBS to perform source space analysis of related cortical activity.

### EXPERIMENTAL SET-UP

### Recording

The measurements were performed inside a magnetically shielded room (Vakuumschmelze, Hanau, Germany) with a whole-head 275-channel MEG (CTF, VSM Medtech, Port Coquitlam, Canada) at a sampling rate of 1172 Hz. Recordings included bipolar electromyography (EMG) from the right flexor carpi radialis muscle (grounded to elbow), bipolar electrooculography (EOG) and electrocardiography (ECG). The patient was seated comfortably and asked to remain motionless. Her gaze was focused on a fixation cross at eye level. The preprogrammed stimulation was switched off 30 min beforehand to obtain reliable wash-out of subthalamic DBS effects (Cooper et al., 2013; Weiss et al., 2013). We recorded 6 min (360 s) of resting-state MEG without stimulation (rest). This baseline measurement was followed by recordings during 10 stimulation trials, each lasting 2 min; 1 min stimulation-on followed by 1 min stimulation-off. For data analysis, we selected the first 40 s of the stimulation-off period of each trial. This resulted in an overall time-period of ∼360 s after artifact rejection. Time periods immediately following DBS were compared with the 360 s of baseline measurement at rest before the stimulation trials. This approach was chosen because: (i) methods for suppressing the stimulation artifact in MEG recordings during simultaneous monopolar DBS might not be sufficiently effective for interpreting stimulation results (Devos, 2004; Silberstein et al., 2005; Airaksinen et al., 2012). By analyzing the period immediately after stimulation, stimulation artifacts are avoided. (ii) Even short term stimulation has neuronal effects that last for several seconds or minutes after stimulation (Kühn et al., 2008; Bronte-Stewart et al., 2009; Whitmer et al., 2012). The time period immediately following stimulation might therefore provide information on stimulation-induced physiological effects. (iii) By comparing this data to the resting measurements before stimulation trials, any carry-over effects in the data used as baseline can be excluded. Source space analysis was performed using a T2-weighted individual MRI.

#### Pre-processing

Notwithstanding stimulation, the DBS hardware itself induced relevant MEG artifacts. These were removed using temporal signal space separation (tSSS; Taulu and Simola, 2006; Taulu and Hari, 2009), which was applied with a subspace correlation limit of 0.9 (Medvedovsky et al., 2009; Airaksinen et al., 2011). The Dynamic Imaging of Coherent Sources (DICS) beamformer has been shown to reliably suppress metal artifacts of implanted DBS electrode leads in the stimulation-off mode as well as with externalized extension cables, i.e., an impulse generator (IPG) outside the body (Litvak et al., 2010). However, it has not yet been shown that this approach also removes the artifacts induced by an implanted IPG during breathing or even during stimulation.

By contrast, tSSS has been shown to overcome these additional artifacts when applied in an Electa Neuromag. Together with the developers of tSSS, we have therefore implemented tSSS in our CTF MEG system. The data presented here was preceded by phantom and in vivo studies to optimize the device settings so as to reliably suppress artifacts. When signal quality did not allow a reliable rejection of artifacts, the algorithm refused the artifact rejection per se. In these pre-studies, we reliably localized movement-related activity and event-related potentials to their anticipated cortical localization during stimulation (unpublished data).

The patient's magnetic resonance images (MRI) were interpolated to a resolution of 256 × 256 × 256 voxels. Fiducial (nasion, left, and right preauricular) and anatomical landmarks (anterior and posterior commissure, mid-sagittal plane) were used to align the MRI to standard space of the Montreal Neurological Institute (MNI). Finally, the source space was defined by a single shell segmented model and lead fields with a resolution of 8 mm.

We applied a first order zero-phase lag finite impulse response bandpass filter (1–40 Hz) to the MEG time series, and highpass filters of 0.5 Hz to the ECG, 0.05 Hz to the EOG, and 2 Hz to the EMG, respectively, to preserve relevant spectral components in the signals. Thereafter, we applied an independent component analysis (ICA) and correlation analysis to identify and remove eye movement, blink, and cardiac-related artifacts. We decorrelated each trial of the MEG sensor time series into 40 independent components via fast independent component analysis. Both independent components, which maximally correlated with the ECG and EOG, were labeled artifacts and skipped during backprojection of the independent components to sensor level. An additional visual inspection, where we focused on the eye movement, blink, and cardiac-related artifacts in the time series of the components, verified the respective artifacts. The magnetic fields of the single sensors were recomputed using an interpolation toward average head position throughout the recordings. All trials and conditions were divided into 5-s epochs. Finally, z-value-transformed epoch-wise artifact rejection was performed using a cutoff-value of 20.

### Data Analysis

We applied a previously described approach (Litvak et al., 2011) to identify the frequency band with maximal corticomuscular coherence (CMC) and to calculate source space CMC. We rectified the EMG channel (Myers et al., 2003; Yao et al., 2007) and calculated cross-spectra with all MEG sensor time series from 2 to 40 Hz by tapering the 5-s epochs using slepian functions (Mitra and Pesaran, 1999) and 2 Hz spectral smoothing resulting in a frequency resolution of δf = 0.2 Hz (see **Figure 1A**). Source space CMC was then calculated by using these cross-spectra and Dynamic Imaging of Coherent Sources (DICS) beamforming (Gross et al., 2001). Pre-processing with tSSS has previously been shown not to restrict the usage of a subsequent beamformer analysis (Hillebrand et al., 2013). Common spatial DICS filter of both conditions, i.e., rest and stimulation, were calculated beforehand and used for source space CMC calculation. Calculating a common spatial filter for beamforming is usually applied when comparing different conditions after beamforming to avoid biased results due to different spatial filters of the conditions.

For the statistical comparison of the rest and stimulation condition for both CMC and power, a non-parametric clusterbased permutation approach was used to correct for multiple comparisons (Maris and Oostenveld, 2007; Oostenveld et al., 2011). We computed voxel-wise t-values between the rest and stimulation condition and clustered voxels exceeding p < 0.001 (uncorrected). The sum of the t-value within the clustered voxel area defined the cluster-level statistics. By randomly permuting the data between rest and stimulation condition for 1000 times, we obtained a reference null distribution of the maximum cluster-level statistics. In order to correct for multiple comparisons, the maximum cluster-level statistics that differed from the reference null distribution with p < 0.05 (corrected) were considered significant. All data analysis was performed offline with custom written scripts and FieldTrip toolbox (Oostenveld et al., 2011) in MATLAB <sup>R</sup> (R2011a, The MathWorks <sup>R</sup> Inc.,).

## EMPIRICAL DATA

An idiopathic akinetic-rigid PD patient (female; age: 69 years; disease duration: 12 years; UPDRS III (motor part) stimulation on/off: 15/20) participated after giving written informed consent. We examined the patient 5 months after DBS surgery with bilateral electrode implantation in the STN (Medtronic 3389 leads and Activa PC <sup>R</sup> stimulator, Minneapolis, USA). The physiological effects of unilateral DBS (left hemisphere) were studied using the same stimulation parameters as applied clinically (monopolar, contact 2-G+, 1.5 V, 60 µs, 125 Hz) while the patient was in medication-on (levodopa equivalent dose = 581 mg; Deuschl et al., 2006). The study was conducted with the patients' informed consent and in accordance with the guidelines approved by the local ethics committee of the University Hospital Tuebingen.

Spectral analysis of CMC at baseline, i.e., in rest and stimulation-off, revealed that the strongest connectivity occurred in the beta-band (**Figure 1A**). Scanning all sensors showed predominant involvement of the hemisphere contralateral to the EMG recording (**Figure 1B**) with a topography suggestive of one major source in MEG sensor space (**Figure 1C**). Source space CMC analysis showed the maximum of the beta-CMC within the motor cortex of the hemisphere contralateral to the muscle (**Figure 2A**) as opposed to a distributed pattern of cortical activation in the beta-band (**Figure 2B**). During STN-DBS, the same beta-CMC (**Figure 3A**) and beta power (**Figure 3B**) source-cluster revealed the strongest stimulation-induced changes, i.e., decreases of CMC and cortical power in the beta-band.

#### DISCUSSION

We performed source space analysis of beta CMC recorded during resting-state MEG and DBS and provide proof-of-concept evidence for a cortical hot spot that is most strongly modulated by subthalamic DBS. Since this study is carried out in only one subject, this proposal requires investigation in a larger cohort of PD patients to probe the robustness of this approach, e.g., for different disease states.

The present work suggests that a circumscribed cortical area involved in the pathologically synchronized cortico-subcorticalmuscular motor network can be detected non-invasively in PD on single subject level, which constitutes a mandatory prerequisite for future closed-loop neurostimulation strategies on the basis of a cortical sensor. Due to its high spatial resolution of ∼2 mm (Pizzella and Romani, 1990), MEG is as an excellent tool for non-invasive mapping prior to neurosurgical procedures. Up to date, MEG examinations have been successfully used for presurgical evaluation of epilepsy patients (Knowlton and Shih, 2004) and patients with tumors in eloquent areas (Rezai et al., 1996; Orrison, 1999). Integration of the MEG information in the stereotactic planning software has been used for electrode placement (Agirre-Arrizubieta et al., 2014) and to aid safe resection of tumors (Kelly, 1996; Rezai et al., 1996; Orrison, 1999). The knowledge acquired with the presented approach may thus be integrated in stereotactic planning for the placement of cortical implants for direct neuromodulation or for placing a potential sensor in closed-loop DBS approaches. We did not intend to describe a specific extent or location of such a cortical area, since these parameters may vary between patients. However, in accordance with previous literature on the hyperdirect cortico-subthalamic pathway recorded with cortico-subthalamic coherence (Hirschmann et al., 2011), the data presented here suggests that this cortical area covers a part of the medial motor und premotor cortex.

The involvement of the cortical areas associated with motor processing in Parkinson tremor is well-documented (Timmermann et al., 2003; Hirschmann et al., 2013a). However, source space topographic information about CMC in akineticrigid PD and in the absence of movement is rather sparse. As a conceptual novelty of this work, we captured the cortical spot of pathological resting-state CMC in the beta band. An implantable cortical sensor for closed-loop applications needs to be spatially restricted from a neurosurgical point of view. Therefore, the cortical target point would be defined as the area which addresses both networks, i.e., the rigidity/bradykinesia and the tremor network. The cortical area involved most strongly in the rigidity/bradykinesia network observed here seems to be more restricted than the tremor network reported by Timmermann and colleagues. Since still being part of the latter such an area would qualify as the common hotspot.

Our concept suggests capturing a cross spectrum for every patient to detect the individual peak frequency band of CMC (see **Figure 1A**). As expected, the presented case also showed the peak CMC in the beta-band followed by the alpha-band. This observation is in accordance with a large body of literature reporting relating enhanced beta oscillations both in the STN

and on the cortical level to the pathophysiology of PD (Kühn et al., 2008; Eusebio et al., 2011; Airaksinen et al., 2012; Little and Brown, 2012; Whitmer et al., 2012). Moreover, beta-band oscillations have repeatedly been shown to mediate corticomuscular communication (Salenius et al., 2002; Baker, 2007; Engel and Fries, 2010; Weiss et al., 2012; Airaksinen et al., 2015). In contrast to beta-CMC, alpha-CMC is not consistently found (Kilner et al., 2000; Budini et al., 2014) and may be confounded by coherence occuring at double tremor frequency (Timmermann et al., 2003). Therefore, it is not surprising that we did not find any significant source cluster for alpha-CMC in the resting state and in the absence of tremor.

STN stimulation is known to modulate spontaneous activity and somatosensory evoked responses over the sensorimotor cortex (Mäkelä et al., 2007; Airaksinen et al., 2011). Furthermore, excessive beta oscillatory activity is suppressed by STN-DBS in PD along with symptom alleviation (Kühn et al., 2008; Whitmer et al., 2012; Little and Brown, 2014). This correlation of therapeutic beta activity modulation and motor symptom alleviation is not surprising, given that oscillatory beta activity relates to motor processing, sensorimotor control, and corticoperipheral interactions (Brovelli et al., 2004; Schoffelen et al.,

2005; Baker, 2007; Chakarov et al., 2009; Engel and Fries, 2010). Cortical beta band activity may parallel the maintenance of the sensorimotor state (status quo; Engel and Fries, 2010). Therefore, decreased motor cortical beta power, as found in this study, may represent a release of cognitive resources, which restores the ability of motor self-control mediated by corticobasal ganglia-thalamo-cortical loops (Kringelbach et al., 2010; McIntyre and Hahn, 2010; Little and Brown, 2012). As already described previously (Whitmer et al., 2012; Weiss et al., 2015), STN stimulation suppresses cortical beta band activity in motorrelated areas. Thus, the findings reported here are in line with these previous results.

Cortico-peripheral interactions as measured by CMC in the beta frequency band may be increased by levodopa administration (Salenius et al., 2002). The effect of STN stimulation was more variable showing a slight beta band CMC increase on fine-motor integration (Weiss et al., 2012) and variable outcomes during joint movement (Airaksinen et al., 2015). This case study suggests that, in the resting state, this long-distance corticomuscular synchronization is decreased by STN-DBS. This tallies with the anticipated dysbalanced drive from cortex to muscle in PD (Salenius et al., 2002; Hirschmann et al., 2013b). The concept of maintaining the status quo via sensorimotor activity in beta band may therefore be extended to beta band CMC as well (Baker, 2007; Engel and Fries, 2010). Decreasing beta CMC in the resting state through STN-DBS may thus represent a reduction of an abnormal persistence of the status quo, i.e., of the pathological oscillatory drive to the muscle in PD (Brown et al., 2001; Marsden et al., 2001). It should be noted, however, that CMC in itself may have a functional role in the motor system apart from the cortical oscillatory activity (Baker and Baker, 2003), thus deserving a consideration independent from cortical beta band power modulation.

However, previous work performing resting-state MEG recordings in PD patients with DBS has neither captured CMC nor has it analyzed source level oscillatory activity (Cao et al., 2015). Only movement-related beta-CMC is currently being investigated as a physiological marker in PD patients undergoing DBS surgery (Hirschmann et al., 2011; Airaksinen et al., 2015). Although, these studies focused on the impact of levodopa medication and DBS on CMC as physiological predictors of clinical outcome, an application of this technique for delineating a cortical hot spot for surface stimulation remains elusive. Airaksinen and co-workers suggested that the cortical representation of beta-CMC and the cortical area modulated by DBS might even differ (Airaksinen et al., 2015). Based on our findings, we suggest that the cortical representation of beta-CMC during resting state might be more restricted.

We therefore suggest that resting-state CMC may eliminate the large inter-individual variability inherent to movements in PD patients in general (Kühn, 2004; Kühn et al., 2008; Tropini et al., 2011; Selzler et al., 2013) and of the related CMC in particular as observed by Airaksinen et al. (2015). Moreover, we demonstrate that source reconstruction—and co-location of both CMC (**Figure 3A**) and oscillatory power (**Figure 3B**) during DBS reveals the cortical convergence of pathological synchronization within a cortical hot spot.

Using simultaneous MEG and LFP recordings, motor cortical areas directly connecting STN and cortex, i.e., mediated via the hyperdirect pathway, were identified in source space thereby disentangling different sub-frequency bands within the betaband and attributing the upper beta-band to cortico-subcortical interactions (Hirschmann et al., 2011). In the present study, we did not capture cortico-subcortical coherence, which would have necessitated externalized leads. Future studies will therefore need to explore frequency-specific interactions between local beta activity and rather long-range interaction with the sensorimotor loop, i.e., for STN/cortex, muscles/cortex, and STN/muscles. Corroborating results for a direct connection between STN and cortex were however obtained by quantitative modeling of axonal fiber activation (Hartmann et al., 2015) or a combination of diffusion tensor imaging (DTI) and electrocorticographic (ECoG) electrodes (Whitmer et al., 2012).

More recently, ECoG (de Hemptinne et al., 2015) and EEG (Swann et al., 2015) have also been applied to capture cortical beta-gamma phase-amplitude coupling (PAC) as a functional marker, while demonstrating its modulation by DBS (de Hemptinne et al., 2015) and levodopa (Swann et al., 2015); PAC might provide an even more robust biomarker than beta oscillations in PD patients (de Hemptinne et al., 2015; Swann et al., 2015). Capturing PAC in MEG source space, however, poses methodological challenges. We are therefore currently applying EEG to compare different physiological markers in PD patients, e.g., for determining topographic and functional differences between PAC and beta oscillations. Future physiological studies will moreover need to compare the interactions in medication on/off and to capture clinical lateralization scores as well. Medication may modulate the very same networks that are modulated by DBS, and the observed DBS effects might even be more pronounced in the medication off state.

This proof-of-concept is however the first to show that beta-CMC delineates the very same cortical area as modulated by STN-DBS, and is meant to inspire and encourage other groups to challenge this testable hypothesis by examining their patients with resting-state CMC as well. These future larger scale studies will then have to address further questions, such as (1) can resting-state CMC be detected in all PD patients, (2) is restingstate CMC correlated with the severity of the disease, (3) does the cortical hot spot vary between patients, and (4) do peak frequencies vary between patients?

### CONCLUSION

We argue that the particular pathophysiology, i.e., the increased synchronization in the oscillatory beta-frequency band, in advanced Parkinson's disease (PD) would facilitate the noninvasive detection of a cortical hot spot since cortico-muscular communication is also mediated in the very same frequency band.

We demonstrated that both clusters projected to the same anatomically plausible area in the primary motor/ premotor area and that these overlapping source clusters revealed the strongest stimulation induced changes for CMC (**Figure 3A**) and cortical power (**Figure 3B**). The decreases of CMC and cortical power were both in the same beta-band. Only a circumscribed area of the whole extended region with increased cortical beta activity (**Figure 2B**) was significantly reduced by DBS (**Figure 3B**). All these complementary and consistent findings cannot be explained by random collocation and are therefore not compromised by the sample size of one. However, the inherent intersubject variability in signals will necessitate the study of further patients to draw definite conclusions.

The novelty of this concept is grounded in the new combination of methods rather than in a new methodology, i.e., by combining resting-state (instead of the usual movement related) cortico-muscular coherence (CMC) with source space analysis of magnetencephalography recordings. Future studies will need to reveal which cortical recording technique, MEG or EEG, is better suited to detect the cortical hot spot before surgery.

The underlying hypotheses are (a) that resting-state CMC would be detectable in PD (other than in healthy subjects), (b) that resting-state CMC would avoid the known variability of movement related CMC in PD, and (c) that pathologically synchronized loops would converge on the cortical level. Such a topographic convergence within a cortical-subcortical-muscular network would provide a prominent target region either for direct neuromodulation or for placing a potential sensor in closed-loop DBS approaches since DBS was recently shown to mediate its therapeutic effects also via remote cortical modulation.

#### AUTHOR CONTRIBUTIONS

KK designed and performed research, analyzed data, and wrote the paper. GN designed and performed research. CB designed research and edited the paper. DW designed research and edited

#### REFERENCES


the paper. AG designed research, analyzed data, and wrote the paper.

#### ACKNOWLEDGMENTS

KK was supported by the Graduate Training Centre of Neuroscience, International Max Planck Research School for Cognitive and Systems Neuroscience, Tuebingen, Germany, GN was supported by a grant from the German Research Council [DFG EC 307], and AG was supported by grants from the German Research Council [DFG EC 307], and from the Federal Ministry for Education and Research [BFNT 01GQ0761, BMBF 16SV3783, BMBF 0316064B, BMBF 16SV5824].

individuals during mechanically-induced tremor. PLoS ONE 9:e115012. doi: 10.1371/journal.pone.0115012


interactions in the human brain. Proc. Natl. Acad. Sci. U.S.A. 98, 694–699. doi: 10.1073/pnas.98.2.694


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2016 Kern, Naros, Braun, Weiss and Gharabaghi. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# A Rodent Model of Dynamic Facial Reanimation Using Functional Electrical Stimulation

#### Mark A. Attiah, Julius de Vries, Andrew G. Richardson and Timothy H. Lucas \*

*Department of Neurosurgery, Center for Neuroengineering and Therapeutics, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA*

Facial paralysis can be a devastating condition, causing disfiguring facial droop, slurred speech, eye dryness, scarring and blindness. This study investigated the utility of closed-loop functional electric stimulation (FES) for reanimating paralyzed facial muscles in a quantitative rodent model. The right buccal and marginal mandibular branches of the rat facial nerve were transected for selective, unilateral paralysis of whisker muscles. Microwire electrodes were implanted bilaterally into the facial musculature for FES and electromyographic (EMG) recording. With the rats awake and head-fixed, whisker trajectories were tracked bilaterally with optical micrometers. First, the relationship between EMG and volitional whisker movement was quantified on the intact side of the face. Second, the effect of FES on whisker trajectories was quantified on the paralyzed side. Third, closed-loop experiments were performed in which the EMG signal on the intact side triggered FES on the paralyzed side to restore symmetric whisking. The results demonstrate a novel *in vivo* platform for developing control strategies for neuromuscular facial prostheses.

#### Edited by:

*Paolo Bonifazi, Tel Aviv University, Israel*

#### Reviewed by:

*Andrew Joseph Fuglevand, University of Arizona, USA George C. McConnell, Stevens Institute of Technology, USA*

#### \*Correspondence:

*Timothy H. Lucas timothy.lucas@uphs.upenn.edu*

#### Specialty section:

*This article was submitted to Neuroprosthetics, a section of the journal Frontiers in Neuroscience*

Received: *10 January 2017* Accepted: *21 March 2017* Published: *05 April 2017*

#### Citation:

*Attiah MA, de Vries J, Richardson AG and Lucas TH (2017) A Rodent Model of Dynamic Facial Reanimation Using Functional Electrical Stimulation. Front. Neurosci. 11:193. doi: 10.3389/fnins.2017.00193* Keywords: paralysis, facial nerve, functional electrical stimulation, electromyography, rodent

## INTRODUCTION

Facial paralysis is a disfiguring condition affecting 127,000 individuals annually (Bleicher et al., 1996). Beyond cosmetic disfigurement, facial paralysis can lead to permanent disability. Blindness may occur if the cascade of corneal dryness, conjunctivitis, and ulceration develops (Otto et al., 1986). Nasal obstruction and mouth leakage result from an inability to keep nasal passages patent and mouth closed, respectively (May et al., 1977). Finally, dysarthria and distorted facial expressions hinder both spoken and nonverbal communication (Coulson et al., 2004), resulting in awkward and strained social interactions.

The ultimate goal of facial reanimation is to enable independent control of paralyzed facial muscles. This goal has proven elusive. Development of effective treatment options is confounded by the wide range of causes that result in a shared facial palsy phenotype, namely, trauma, infection, neoplasm, iatrogenic insults and idiopathic etiologies (Melvin and Limb, 2008). Current treatments palliate the condition by partially restoring muscle tone or by attempting to prevent catastrophic consequences, like blindness. These procedures all have significant drawbacks. Sacrifice of the hypoglossal nerve for nerve transfer results in ipsilateral tongue paralysis. Cross-facial nerve grafts route contralateral facial branches to the paralyzed side, but this procedure places the healthy nerves at risk (Hontanilla et al., 2013). Dynamic procedures routinely take 9–18 months to become effective and require rigorous rehabilitation (Spector et al., 1991; Robey and Snyder, 2011). Attiah et al. Rodent Model of Facial Reanimation

Spontaneous movement or emotional expression is rarely achieved and the functional status is often compromised by synkinesis. To improve outcomes researchers are developing several new treatments (Hadlock and Cheney, 2008), including the use of functional electrical stimulation (FES) to reanimate paralyzed facial muscles (Griffin and Kim, 2011).

The effectiveness of FES in activating paralyzed facial muscles, in particular the orbicularis oculi to restore eye blink, has been demonstrated in rabbits and dogs (Rothstein and Berlinger, 1986; Salerno et al., 1991; Otto, 1997; Somia et al., 2001; Sachs et al., 2007; Jie et al., 2016). Since many facial movements including eye blink are symmetric, a natural closed-loop FES control signal in unilateral facial paralysis can be derived from the intact side of the face. In particular, electromyographic (EMG) recordings from a facial muscle contralateral to the injury can be used to trigger FES of the homologous paralyzed muscle. A similar, bilateral, closedloop approach has been used to rehabilitate hand movements after stroke (Knutson et al., 2012, 2016). In the context of facial paralysis, this closed-loop strategy has been demonstrated in rabbits, dogs, and humans (Zealear and Dedo, 1977; Tobey and Sutton, 1978; Broniatowski et al., 1987, 1989, 1991; Cao et al., 2009; McDonnall et al., 2009; Kurita et al., 2010; Frigerio and Cavallari, 2012; Yi et al., 2013). Most of these studies focused on restoration of eye blink and utilized largely qualitative measures of FES performance.

In the present work, we sought to develop a new, quantitative animal model of FES-controlled facial reanimation. Quantitative tracking of bilateral facial movements is typically done with video cameras and feature recognition software (Sachs et al., 2007). But this strategy is error-prone and has low temporal resolution. An alternative strategy is to track a more conspicuous facial feature present on some animals: facial vibrissae (whiskers). Rodent whiskers are controlled by muscles innervated by the facial nerve (Berg and Kleinfeld, 2003), typically exhibit bilateral symmetric motion (Gao et al., 2001), and can be readily tracked with high spatiotemporal resolution (Bermejo et al., 1998). Thus, we hypothesized that a rodent model of unilateral facial paralysis could provide an improved assessment of FES performance for dynamic facial reanimation. A similar approach with rats was developed previously for the study of facial nerve function (Heaton et al., 2008). In the present study we quantified, for the first time, the effects of open- and closed-loop FES on whisker motion. The results suggest this new approach could be useful for designing improved FES control policies for a facial prosthesis.

### METHODS

All procedures were approved by the University of Pennsylvania Institutional Animal Care and Use Committee. Four female Sprague-Dawley rats were used for these experiments.

#### Training and Experimental Apparatus

Prior to surgery, all animals were habituated to the laboratory environment and the experimental apparatus. The apparatus (**Figure 1**) consisted of (1) a half-pipe with hook-and-loop strap to restrain the rat's body, (2) neck plates at the front

FIGURE 1 | Experimental setup. (A) Laser micrometers positioned on either side of the face were used to monitor whisker movement. (B) Polyimide tubes were placed on selected whiskers and micrometer sensitivity was adjusted such that only the motion of one pair of whiskers was detected. (C) Head restraint was achieved using magnetic implant and a repositionable rod. (D) Microwire electrodes implanted in facial muscles were accessed via a connector anchored to the skull.

of the pipe to restrain forward or backward head motion, (3) a metal rod with magnet that mated with a headmounted magnet to further restrain the head, and (4) two commercial laser micrometers in a V-configuration such that they approximated both sides of the face (see details below). The half pipe could be moved to adjust the position of the rat relative to the micrometers. A Faraday cage, which enclosed the entire apparatus, was assembled with copper mesh and red translucent acrylic in order to shield recordings from ambient electromagnetic interference and to provide an enclosed environment that minimized the stress of the animals. Rats were placed in the restraints each day until they were able to be calm for at least 1 h.

#### Implant

For each rat, a custom microwire array was constructed to interface with the facial musculature. Seventeen insulated stainless steel wires (50 µm, A-M Systems) were soldered to a printed circuit board (PCB, 6.68 × 7.05 × 1.59 mm) and routed to a miniature connector on the board (A79042-001, Omnetics Corp). Sixteen of the wires served as working electrodes in facial muscles (eight on each side) and one as a reference electrode. An uninsulated silver wire (127 µm, A-M Systems) was also soldered to the PCB to serve as the ground. Wires were impedance tested to ensure the integrity of the electrical connections. The PCB was covered with an epoxy to insulate and stabilize the solder connections and traces. The distal 2 mm of the stainless steel wires was stripped of insulation. Before surgery the implant was cold sterilized in chlorhexidine (0.1%).

### Surgical Procedure

Our development of the rodent model proceeded in two sequential phases: (1) establish the techniques for bilateral recording of facial EMG and whisking motion (animals 1 and 2) and (2) paralyze and implement open- and closed-loop FES (animals 3 and 4). As such, the facial nerve transection portion of the surgical procedure applied only to animals 3 and 4.

Each rat was anesthetized with intraperitoneal administration of ketamine (60 mg/kg) and dexmedetomidine (0.25 mg/kg). The animal was kept on a heating pad and the respiratory rate, palpebral reflex, and pedal pinch reflex were monitored periodically throughout the procedure to track depth of anesthesia. Once a surgical plane of anesthesia was reached, the head and right side of the face posterior to the whisker pad were shaved and cleaned with povidone-iodine. Lidocaine was administered subcutaneously at the incision sites for further analgesia.

The buccal and marginal mandibular branches of the facial nerve, the only branches supplying motor input to the whisker pad (Semba and Egger, 1986), were exposed through a single vertical skin incision on the right side of the face. Nerve identity was confirmed with bipolar stimulation, which elicited movement from the whisker pad. A 3 mm section of each nerve was transected. Functional denervation was confirmed again with bipolar stimulation to the proximal and distal stumps. The incision was closed with 6-0 nylon suture.

The animal was then placed in a stereotaxic frame. An incision was made on the midline of the scalp and skin, soft tissue, and periosteum were retracted laterally to expose the frontal and parietal bones. Six burr holes were made on the skull bilaterally and six screws (00-90 × 1/8′′) were inserted. The implant PCB was positioned in the middle of the skull screws and the silver ground wire was wrapped around the screws. To aid head fixation, a high-strength cylindrical neodymium magnet (D34- N52, K&J Magnetics Inc.) was placed rostral to the PCB. Dental acrylic was then poured over the PCB, screws, and around the magnet to secure them to the skull.

The stainless steel microwires were then inserted into the muscles of the whisker pad in pairs. The hooked ends of each pair of wires were placed in a 23-gauge hypodermic needle that was used to drive them into the targeted muscle (Carvell et al., 1991). The targets initially included both intrinsic and extrinsic muscles of the whisker pad, which protract and retract the whiskers, respectively (Dörfl, 1982). However, in preliminary studies we found that actively controlling the retractor muscles, nasolabialis and maxillolabialis, introduced significant complexity (see Section Discussion). Whisker retractions can occur passively, due to the viscoelastic properties of the facial connective tissue (Bermejo et al., 1998). Accordingly, the intrinsic muscles were targeted preferentially. Intramuscular placement was confirmed with bipolar stimulation that evoked whisker protraction. The stainless steel reference wire was placed subcutaneously on the dorsal aspect of the nasal bone. The midline incision was closed with 6-0 nylon sutures.

Anesthesia was reversed with atipamezole (5 mg/kg). For 3 days postoperatively, animals were maintained on an analgesic regimen of ketoprofen (5 mg/kg, once daily) and buprenorphine (0.05 mg/kg, twice daily), administered subcutaneously. To prevent infection, gentamicin sulfate drops (0.3%, once daily) were applied around the implant.

### Whisker Monitoring

After surgery, the rats were further acclimated to the experimental apparatus via training sessions in which their heads were held stationary by the magnetic attachment (**Figure 1**). Horizontal whisker motion was monitored bilaterally by two laser micrometers (IG-028, Keyence Corp) mounted in front of the body restraint apparatus. The micrometers had a measurement range of 28 mm, a spatial resolution of 5 µm, and a temporal resolution of 490 µs. One bilateral homologous pair of whiskers was selected for monitoring. Selection was based on (1) the ability to fully capture the whisking motion on the micrometer array and (2) the ability to actuate the whisker on the paralyzed side with FES. A small polyimide tube (0.36 mm diameter) was placed on the selected pair of whiskers to facilitate its detection by the micrometer arrays (Heaton et al., 2008). The spatial sensitivity settings of the micrometers were adjusted such that they only detected the whisker in the tube and not the other whiskers.

## Data Acquisition and FES

The laser micrometer output and EMG signals from the implanted microwires were simultaneously recorded at 3052 samples/s (PZ2 preamplifer, RZ2 processor, Tucker-Davis Technologies). EMG signals were recorded differentially with respect to the subcutaneous reference wire. Pairs of digitized EMG channels were then subtracted to yield the bipolar EMG signal used in offline analyses and online stimulus triggering. Electrical stimulation delivered to the microwires consisted of trains of charge-balanced, biphasic square pulses (IZ2H stimulator, Tucker-Davis Technologies). Pulse amplitude, pulse width, number of pulses, and pulse frequency were the adjustable stimulation parameters used to achieve different whisker motions. For closed-loop, EMG-triggered FES, the digital signal processors of the RZ2 were programmed for real-time EMG signal conditioning (bipolar subtraction, rectification, and 10– 1,000 Hz bandpass filtering). Stimuli were then triggered when the conditioned signal amplitude reached an experimentally defined threshold, determined by trial and error at the beginning of the closed-loop sessions. A trigger lockout period starting with the first pulse and ending 20 ms after the end of the pulse train was imposed to prevent stimulus artifacts from triggering the stimulator.

#### Data Analysis

In offline analyses, the EMG envelope was estimated by bandpass filtering (5–100 Hz passband), rectifying, and then lowpass filtering (10 Hz cutoff) the raw bipolar signal. Power spectra of both the whisking signal and EMG envelope were computed using Welch's averaging method on 1-s, Hamming-windowed data segments with 50% overlap. Time delays between local peaks in the EMG envelope and whisking were used to assess the lag between these signals. Pearson's correlation coefficient was used to assess the linear relationship between peak whisking and EMG envelope amplitudes. For comparison of FES effects across animals, we identified the stimulus intensity required to evoke a 2 ± 0.2 mm whisker protraction, where protraction amplitude was defined as the peak positive displacement after subtracting the baseline mean. Closed-loop stimulation effects were summarized by stimulus-triggered average displacement of both paralyzed and intact whiskers, with 95% confidence intervals based on the t-distribution. The onset and duration of the post-stimulus protraction were defined by the first and last occurrences of positive displacement (i.e., lower confidence bound >0) in these averages, averaged across animals.

#### RESULTS

We developed a rodent model for studying FES-based facial reanimation. Optical micrometers (**Figure 1A**) were used to track a pair of whiskers bilaterally (**Figure 1B**) in head-fixed rats (**Figure 1C**). The facial musculature of the rats was implanted with microwires to allow both EMG recording and FES of facial muscles (**Figure 1D**). This system allowed us to study, at high spatiotemporal resolution, facial movements resulting from either volitional or artificial (i.e., FES) commands.

#### Whisking and EMG activity

The head-fixed rats engaged in bouts of volitional, rhythmic whisking behavior (**Figure 2A**). This behavior could be produced spontaneously but often needed prompting by delivering scents near the nose. Rhythmic activity in the EMG of the whiskerprotracting muscles corresponded with rhythmic motion of the tracked whisker on the intact side of the face (**Figure 2A**). To quantify the EMG-whisking relationship, the envelope of the EMG signal was computed using a zero-phase low pass filter. The dominant frequency of whisking and of the EMG envelope oscillation was 6 Hz in all four rats (**Figure 2B**). The EMG envelope led the whisker motion by an average of 28.7, 24.9, 24.7,

Attiah et al. Rodent Model of Facial Reanimation

and 29.2 ms in each of these animals, respectively (**Figure 2C**). Thus, the timing of each of cycle of the whisker motion could be accurately determined from the EMG. In one case, amplitude of the whisker motion could also be inferred from the EMG. The best observed correlation between peak whisk amplitude and peak EMG envelope amplitude was 0.75 (**Figure 2D**). However, in the other animals the whisk amplitude estimate was less reliable, with whisk-EMG correlations of 0.35, 0.32, 0.24, likely due to suboptimal electrode placement.

A unilateral transection of the buccal and marginal mandibular branches of the facial nerve was performed in two of the four rats (**Figure 3A**). This procedure was effective in focally paralyzing the whisker pad. Rhythmic whisking was completely absent on the paralyzed side (**Figure 3B**). Only miscellaneous fibrillations up to 0.5 mm were present in the de-efferented whiskers. For over a month after the lesion, the rats maintained proper weight and eating habits and exhibited no self-injurious behavior.

#### FES of Intrinsic Muscles

Next, electrical stimuli were delivered to the paralyzed intrinsic muscles to reanimate the whiskers. Trains of current-controlled square pulses were delivered between the pair of electrodes that produced whisker motion at lowest current amplitude. Across four testing sessions, the current to produce a 2-mm protraction was 43 ± 6 µA (animal 3) and 98 ± 13 µA (animal 4), when using 0.3-ms pulse widths, 10 pulses, and 200-Hz pulse frequencies. We characterized how variations in these stimulus parameters changed the evoked whisker motion. Increases in whisker displacement could be produced by increasing pulse width (**Figure 4A**), number of pulses (**Figure 4B**), or pulse frequency (**Figure 4C**). However, only pulse width affected amplitude without also changing the shape or timing of the motion. Increasing the number of pulses increased the amplitude and duration of the motion (**Figure 4B**). For both animals the duration, measured as time to peak amplitude, increased linearly, with a mean slope of 2.7 ± 0.37 ms/pulse when stimulating at 200 Hz. Increasing pulse frequency increased the amplitude and the velocity of the protraction (**Figure 4C**). The relationship between pulse frequency and whisker velocity, measured as the slope of the initial protraction for each frequency, was steeper in one animal and more graded in the other (**Figure 4D**). The results demonstrate that the amplitude and shape of the whisker motion can be controlled by stimulus pulse trains.

#### Contralaterally-Triggered FES

Finally, in the unilaterally paralyzed rats, we used EMG activity recorded on the intact side to trigger stimulation of the paralyzed intrinsic muscles. The EMG was bandpass filtered and thresholded in real-time to generate stimulus triggers (**Figure 5A**). A train of stimulus pulses with fixed parameters was delivered immediately after each trigger. The stimulus parameters were chosen to achieve a whisker motion comparable to that observed on the intact side. A typical example of the EMGtriggered stimulation is shown in **Figure 5A**. The system was able to accurately deliver stimuli during each protraction of the intact whisker, as expected based on the preceding analyses. However,

peripheral branches of the facial nerve distal to the stylomastoid foramen. The two sites of transection are indicated by arrows. (B) Typical whisking behavior recorded simultaneously on the intact (top) and paralyzed (bottom) side. Whisker movements were recorded 10 days after transection of the buccal and marginal mandibular branches of the facial nerve on the paralyzed side.

the overall symmetry between the intact and paralyzed whiskers was limited by several factors, including the inability to actively control retraction. To summarize the results, we computed the stimulus-triggered average motion of the paralyzed and intact whiskers in both animals (**Figure 5B**). The mean evoked motion on the paralyzed side had a peak amplitude and timing that was very similar to the mean intact-side protraction. Average onset of the post-stimulus protraction was 27 ms (paralyzed) and 24 ms (intact). The duration of post-stimulus motion was 150 ms (paralyzed) and 156 ms (intact). The average whisker motion of both sides was triphasic, although the intact motion was more variable being drive volitionally (**Figure 5B**).

#### DISCUSSION

In this study, we developed and tested a rodent model of closedloop FES-based facial reanimation in the setting of unilateral facial paralysis. A primary strength of the model was the ability to track the relevant facial feature (i.e., whiskers) with high spatiotemporal resolution. The documented whisking behavior confirmed earlier findings. Prior studies have observed 5–6 Hz whisking frequencies in head-fixed rats (Gao et al., 2001). This is in contrast to the significantly higher, 6–12 Hz whisking frequencies seen in unrestrained rats (Hill et al., 2008). Also, the observed 24–29 ms lag between intrinsic muscle EMG and whisker protraction is nearly equivalent to a prior report (Berg and Kleinfeld, 2003). This lag can be attributed to the

viscoelastic properties of the whisker pad. Electrical stimulation of the intrinsic muscles has previously been shown to protract the whiskers (Hill et al., 2008). However, here for the first time we documented how protraction varied with stimulation parameters. The velocity, peak amplitude, and duration of the protraction could largely be controlled through the number of stimulus pulses and pulse frequency. Finally, we demonstrated a closed-loop, contralaterally-triggered FES strategy that has been proposed for dynamic facial reanimation in humans (Cao et al., 2008; Griffin and Kim, 2011).

A limitation of the animal model is the difficult in durable placement of electrodes in the intrinsic muscles. These small, sling-like muscles around the base of each whisker cannot be directly visualized during implantation. As a result the electrode placement, and thus the recording and stimulation effects, can vary. This likely explains variability in the results, include the relationship between EMG amplitude and protraction amplitude and the relationship between whisker velocity and stimulus pulse frequency. A second limitation, which impacted the ability to restore symmetric whisking, is not actively controlling retraction. Whisker retraction is controlled by the nasolabialis and maxillolabialis muscles (Berg and Kleinfeld, 2003). However, these muscles pull the whiskers out of the horizontal plane along dorsal-posterior and ventral-posterior trajectories, respectively (Hill et al., 2008). The vertical components of these motions could not be captured by our micrometers, but could be

under EMG). The series of stimulus trains (10 pulses/train delivered at 200 Hz with 0.3-ms pulse width and 45-µA (animal 3) or 95-µA (animal 4) pulse amplitude) produced whisker motion on the paralyzed side (middle) that approximated the motion seen on the intact side (bottom). (B) Stimulus-triggered average of whisker displacements on the paralyzed (top) and intact (bottom) side for animals 3 and 4 compiled from all 345 and 278 stimulus trains delivered during the closed-loop experiment, respectively. Ninety-five percent confidence intervals on the mean are shown in gray.

monitored if the setup was extended to including two orthogonal micrometers on each side of the face (Hill et al., 2008). Alternatively, simultaneous stimulation of both retracting muscles can produce a mostly in-plane, posterior movement (Hill et al., 2008). Initial experiments found it quite difficult to get the concurrent electrode placement in all four muscles bilaterally. Thus, we relied on the passive retraction that occurs following active protraction (Berg and Kleinfeld, 2003).

Another potential limitation involves the FES strategy itself. Most etiologies of unilateral facial paralysis leave facial sensation intact, as the trigeminal nerve is not affected. Therefore, FES has the potential to activate sensory axons associated with nociceptors, evoking painful percepts. However, several lines of evidence suggest this issue does not completely undermine the strategy. No signs of pain (flinching, blinking, or vocalizing) were observed in our experiments. A prior study observed that rabbits initially flinched in response to orbicularis oculi stimulation, but signs of pain diminished over time (Otto, 1997). There is evidence that FES paradigms including interferential stimulation (McDonnall et al., 2009) and lowintensity multichannel stimulation (Somia et al., 2001) render facial muscle stimulation functional but not painful. Finally, facial FES has been achieved in humans reporting only mild pain (McDonnall et al., 2009; Frigerio et al., 2015).

An issue that was not studied, but should be considered, in this model of chronic facial paralysis is the time-dependent effects of denervation and the modulation of these effects by FES. Denervation is associated with muscle atrophy, which in the rat can result in a 50% loss of muscle weight after 2 weeks (Ohira, 1989). Importantly for FES, denervation also causes a transient increase in excitability, due in part to increased sensitivity to acetylcholine, followed after a few days by decreased excitability due to Wallerian degeneration of the axons distal to the injury (Sunderland, 1978). Nevertheless, prior animal studies of facial FES have elicited functional movement (e.g., complete eyelid closure) for several months after paralysis (Salerno et al., 1991; Otto, 1997; Sachs et al., 2007). This may be explained in part by demonstrations that FES can prevent and even reverse the effects of denervation, both in facial muscles (Salerno et al., 1991) and non-facial muscles (Eberstein and Eberstein, 1996). Reinnervation, potentially from motor axons in surrounding, non-denervated muscles, may also play a role (Sachs et al., 2007). Our rodent model of facial paralysis provides another means to explicitly study the interaction of muscle denervation and FES, with the benefit of precise quantification of muscle activation through whisker monitoring.

Previous research on contralaterally-triggered facial FES has been done in larger animal models and humans over the course of several decades (Zealear and Dedo, 1977; Tobey and Sutton, 1978; Broniatowski et al., 1987, 1989, 1991; Cao et al., 2009;

#### REFERENCES


McDonnall et al., 2009; Kurita et al., 2010; Frigerio and Cavallari, 2012). However, the therapy has not advanced beyond proof of concept. Our motivation for developing a less expensive, more quantifiable animal model for this therapy was to move toward clinical translation by improving performance through advanced closed-loop controllers. In the present study, as in all previous work, the closed-loop controller was simple: deliver hand-tuned, fixed-parameter stimuli to the paralyzed muscle when triggered by thresholded EMG activity of the homologous intact muscle. Myriad strategies, including iterative learning control (Bristow et al., 2006), adaptive feedforward control (Abbas and Triolo, 1997), and supervised learning with a distal teacher (Jordan and Rumelhart, 1992), could be used to automatically tune a mapping between EMG activity and stimulation parameters based on measured facial asymmetries to yield superior performance. The animal model developed here provides an improved platform with which to test these advanced controllers. We believe this is a necessary step to develop a therapy that could improve the quality of life of thousands of patients with facial paralysis.

#### ETHICS STATEMENT

This study was carried out in accordance with the recommendations of the National Institutes of Health guidelines on the use of animals. The protocol was approved by the University of Pennsylvania Institutional Animal Care and Use Committee.

#### AUTHOR CONTRIBUTIONS

AR and TL designed the study. MA, Jd, and AR conducted the experiments and analyzed the results. All authors contributed to writing the manuscript.

#### ACKNOWLEDGMENTS

The authors thank Pauline Weigand, Erica Von Stein, and Nathan Kidambi for experimental assistance and helpful discussions.

balanced coupling of oral and ocular musculature from the intact side in the canine. Otolaryngol. Head Neck Surg. 105, 727–733.


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2017 Attiah, de Vries, Richardson and Lucas. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Evoked Electromyographically Controlled Electrical Stimulation

#### Mitsuhiro Hayashibe\*

Institut National de Recherche en Informatique et en Automatique (INRIA), University of Montpellier, Montpellier, France

Time-variant muscle responses under electrical stimulation (ES) are often problematic for all the applications of neuroprosthetic muscle control. This situation limits the range of ES usage in relevant areas, mainly due to muscle fatigue and also to changes in stimulation electrode contact conditions, especially in transcutaneous ES. Surface electrodes are still the most widely used in noninvasive applications. Electrical field variations caused by changes in the stimulation contact condition markedly affect the resulting total muscle activation levels. Fatigue phenomena under functional electrical stimulation (FES) are also well known source of time-varying characteristics coming from muscle response under ES. Therefore, it is essential to monitor the actual muscle state and assess the expected muscle response by ES so as to improve the current ES system in favor of adaptive muscle-response-aware FES control. To deal with this issue, we have been studying a novel control technique using evoked electromyography (eEMG) signals to compensate for these muscle time-variances under ES for stable neuroprosthetic muscle control. In this perspective article, I overview the background of this topic and highlight important points to be aware of when using ES to induce the desired muscle activation regardless of the time-variance. I also demonstrate how to deal with the common critical problem of ES to move toward robust neuroprosthetic muscle control with the Evoked Electromyographically Controlled Electrical Stimulation paradigm.

Keywords: electrical stimulation, evoked electromyography, personalized stimulation, muscle activation control, electrode effect cancelation

### 1. CHALLENGES IN TRANSCUTANEOUS (SURFACE) ELECTRICAL STIMULATION -AN INTRODUCTION

Electrical stimulation of the nervous system is a technique which is frequently used in physical therapy as it offers clinical diagnosis on neuromuscular activation, physiological investigation, and functional control of paralyzed extremities (Merletti et al., 1992). In diagnostic applications, it is used to ascertain the integrity of neuromuscular junctions and reflex loops, as well as the excitability of motor neuron pools, nerves, and muscle fibers (Merletti et al., 1992; DeLuca and Erim, 1994). In therapeutic treatments, continuous usage of electrical stimulation can help to maintain muscle volume and enhance blood circulation following lesions of the nervous system. It can prevent muscle atrophy due to non-use, which can readily occur in spinal cord injured patients, and also in stroke patients during the immobilization period. In orthotic treatments, electrical stimulation can be used to provide functional control of paralyzed muscles, which is called functional electrical stimulation (FES).

#### Edited by:

Timothée Levi, University of Bordeaux, France

#### Reviewed by:

Ping Zhou, University of Texas Health Science Center at Houston, USA Thierry Keller, Tecnalia Research & Innovation, Spain

#### \*Correspondence:

Mitsuhiro Hayashibe mitsuhiro.hayashibe@inria.fr

#### Specialty section:

This article was submitted to Neuroprosthetics, a section of the journal Frontiers in Neuroscience

Received: 21 January 2016 Accepted: 01 July 2016 Published: 14 July 2016

#### Citation:

Hayashibe M (2016) Evoked Electromyographically Controlled Electrical Stimulation. Front. Neurosci. 10:335. doi: 10.3389/fnins.2016.00335

FES and neuroprosthetic muscle control have been used to compensate motor functions or produce movements in patients with complete spinal cord injury (SCI; Kobetic et al., 1997), as well as in stroke patients with other sensory-motor deficiencies such as drop foot syndrome (Liberson et al., 1961). In the late 1990s, progress in microprocessor technology provided the means for computer-controlled FES systems (Donaldson et al., 1997; Kobetic et al., 1997; Guiraud et al., 2006). Electrical stimulation strategies are wide-ranging, from implanted stimulation (Triolo et al., 1996; Johnston et al., 2003; Guiraud et al., 2006), spinal stimulation (North, 2008) to transcutaneous surface stimulation (Mangold et al., 2005). In real-world applications, transcutaneous surface stimulation (TES) is the most frequently applied technique for muscle and nerve activation despite the significant efforts made for implantable technology developments (Keller and Kuhn, 2008), simply because TES is an easier solution in practice. Electrodes are placed on the skin at locations where the underlying tissue is intended to be activated. Electrical current is injected through a pair of bipolar electrodes and generates a potential gradient over the targeted area. This artificially generated gradient depolarizes excitable tissue beneath the electrodes that serve as cathodes, thus activating the underlying muscles.

TES has already been popularly utilized for muscular massage purpose. Portable electrical stimulators for home use are widely commercially available. Impulses are generated by a device and delivered through electrodes on the skin in direct proximity to the muscles to be stimulated. TES has also been receiving increasing attention in the last few years because of its potential to serve as a muscle strength training tool for healthy subjects and athletes and as a preventive tool for partially immobilized patients (Maffiuletti et al., 2011). Strength training promotes neural and muscular adaptations that are complementary to the well-known effects of voluntary resistance training. These types of TES devices for muscular massage and strength training are technically equivalent to TES systems which are applied by clinical specialists for therapeutic and functional purposes.

Following the wide availability of TES, many people now have experience on testing transcutaneous surface electrodes for stimulation in massage or training settings. If one has experienced to use surface stimulation electrodes, he would likely have noticed that the muscle status under TES is sometimes suddenly changed by the electrode contact condition to the skin. This is indeed a universal problem in all TES applications. A slight contact condition change can markedly affect the real muscle activation status, even if the same stimulation input is constantly applied. This is related to the correct electrode placement issue, which can cause reduced muscle contraction, especially in dynamic muscle contractions (Keller and Kuhn, 2008). There have been many FES studies of TES applications, it ranges from foot drop correction (O'Keeffe et al., 2003; Azevedo Coste et al., 2014) to upper limb muscle control (Chan et al., 2009). However, in these TES papers, variations in actual muscle activation due to the electrode condition is normally ignored in real applications, as it is hard to deal with the time-variant muscle response issue. As many FES researchers/users frequently encounter muscle activation changes due to electrode effects in TES, it is not negligible level of effect in terms of muscle activation changes. This intra-subject variability in muscle activation under TES is due to electrode dependency. There is also inter-subject variability coming from the muscle strength differences between different subjects. An able-bodied subject's muscle response could be more substantial than the weak muscle response in motorimpaired subjects. This factor also limits the controllability of muscles by ES, a problem that should be addressed via personalized modeling for FES. In this perspective article, I introduce a way to capture muscle activation changes through the concept of Evoked Electromyographically Controlled Electrical Stimulation, which was previously developed to compensate for muscle fatigue variations under FES. I address these issues below by outlining the research studies that have been carried out so far:


#### 2. PERSONALIZED ELECTRICAL STIMULATION THROUGH EVOKED EMG

The challenge in implementing the present FES system arises with the problem of how to process the high nonlinearity and complexity of the neuro-muscular system (Durfee, 1993; Riener, 1999). Another challenge concerns time-varying muscle dynamics due to physiological and biochemical factors (such as fatigue, reflexes), as these need to be compensated in order to augment FES applications. Subject-specific modeling and timevariance compensation are essential to improve the performance of motor neuroprosthetics beyond the current limited use. A use of a mathematical model can improve the development of neuroprosthetics by optimizing their functionality for individual patients (Riener, 1999).

To achieve a reliable stimulation pattern and compensate muscle property changes during FES, in Ferrarin et al. (2001) the authors suggested using model-based approaches involving a feedforward controller to improve the control performance. In Jezernik et al. (2004), a sliding model closed-loop control method was proposed to control shank movement. In another work (Ajoudani and Erfanian, 2009), classic sliding model control and a neural network were combined to control FES to track the desired knee joint trajectory. A Feedback Error Learning controller for FES was developed by applying an antireset windup scheme (Watanabe and Fukushima, 2010). In a recent work (Freeman, 2014), iterative learning control was applied to control joint angles via stimulation of an arbitrary set of muscles with a Hammerstein-type muscle-activation recruitment relation.

Most closed-loop FES systems addressed in previous works were established between the electrical stimulus and joint angle since it is more convenient to measure and process the joint angle than joint torque or muscle force. However, in order to take the immediate effect of muscle activity induced by FES control into account, introducing biofeedback into closed-loops should be considered for further FES development (Bruns et al., 2013; Hayashibe et al., 2015), thus contributing to individualized modeling and control to manage different muscle responses from each different subject.

As muscle contraction is induced by artificial stimulation in FES, the drawback of closed-loop control of joint motion is that the resultant motion may not only derive from stimulation but also from external forces (such as environmental contact and gravity). The motion is the result of both external forces and the muscle contractions activated by the stimulation. The motion itself may not be directly related to the stimulation inputs. A stimulation pattern based only on motion signals is likely to be unsafe and unreliable in this case. In addition, it should be kept in mind that the muscle is a very slow actuator in terms of motion control. Error feedback by position information thus can not be instantly compensated, contrary to electromagnetic motor control in robotics. The muscle response to the same stimulation input is also not very consistent due to the physiological time-variant muscle response. Thus, considering the usage of biosignal feedback such as EMG from the muscle itself is a natural way for taking those potential physiological changes into account. By taking advantage of EMG signals, an EMG-triggered FES control system was presented through a pattern recognition (Dutta et al., 2008). This addresses mainly the FES start timing issue by using remained voluntary contractions rather than adapting the stimulation pattern itself. Then it does not handle time-variant muscle responses under FES.

Evoked EMG (eEMG) offers a way to study the myoelectric features of neuromuscular activation associated with electrical stimulation. Motor units activated by electrical stimulation have synchronous activity, with the so-called M-wave present in the EMG signal. When processing FES-evoked EMG signals, stimulation artifacts that appear at the onset of each stimulation impulse, and which are much larger than Mwaves, must be dealt with. In order to retrieve the signal of interest, i.e., Mwaves, suitable techniques such as the blanking window method should be implemented to remove stimulation artifacts. The eEMG signal was found to be highly correlated with FESinduced muscular torque under various stimulation situations (Chesler and Durfee, 1997), and a similar phenomenon was also found in an implanted FES SCI subject (Hayashibe et al., 2011b). Moreover, M-waves extracted from the eEMG can be an effective detector for tracking potential muscle fatigue (Heasman et al., 2000). A pioneer work (Erfanian et al., 1998) proposed a predictive model of muscle force production under an isometric percutaneous continuous FES system. After comparing the performance of force prediction from stimulation and from EMG, the authors suggested using measured EMG signals instead of stimulation signals to predict muscle torque. This study was mainly carried out to predict joint torque as a kind of muscle force sensor in FES, and it has not yet been really used to achieve systematic FES closed-loop control based on evoked EMG. In our team, a new control strategy, i.e., EMG-Feedback Predictive Control (EFPC; Zhang et al., 2013), was proposed to adaptively control stimulation patterns to compensate time-varying muscle state changes. This facilitates the prediction of the muscle response and then the system can respond to time-variant muscle state changes toward muscle-response-aware FES control. It was further implemented combined with a wireless portable stimulator (Toussaint et al., 2010) to achieve real-time FES control (Li et al., 2015).

### 3. MUSCLE FATIGUE PREDICTION AND COMPENSATION IN FES

Muscle fatigue has been defined as a failure to maintain the required or expected force from a repeatedly activated muscle (Edwards, 1981). In voluntary contraction, a variety of biological and motivational factors contribute to muscle fatigue (Gandevia et al., 1995), such as reduced motor drive by the CNS, failure of peripheral electrical transmission, and failure of the muscular contractile mechanism. The rate of muscle fatigue during FES is much greater than that which occurs during natural contractions (Binder-Macleod and Snyder-Mackler, 1993). This fast fatigue phenomenon is complex and is not yet fully understood. Currently, it is understood as follows: (1) the inverse size principle, according to which artificial stimulation recruits the motor neurons from the largest to the smallest (Hamada et al., 2004), and the larger the motor neuron, the more fatigable the muscle fiber; (2) motor units are activated in a synchronized manner with artificial stimulation, which is different from asynchronous activation during natural contraction. This situation requires a much higher stimulation frequency in synchronous stimulation to achieve quasi-tetanic contraction. The high stimulation frequency causes fatigue; and (3) the constant order of recruitment, with fast fatigable motor units activated first, then slow fatigue-resistant motor units. Another factor related to fast fatigue with FES is that the fatigue resistance of paralyzed muscles decreases after injury (Pelletier and Hicks, 2011). Systematic fatigue monitoring is especially important in paraplegic patients suffering from a lack of sensory feedback from their paralyzed muscles, because it can be used to adjust stimulation to prevent failure. Second, force prediction is essential if the muscle force has to be used as feedback in closed-loop stimulation.

Various fatigue models have been drawn up, based on physiological and mathematical interpretation or fitting from experimental measurements. A biomechanical model was developed to predict the shank motion induced by FES (Riener et al., 1996). A five-element musculotendon model was developed to predict the force generation capacity of activated muscles, and a fatigue recovery function based on the metabolic profile was introduced (Mizrahi et al., 1997). In Cai et al. (2010), a Wiener-Hammerstein model was proposed to predict FESinduced muscle force in unfatigued and fatigued muscle, and the model was verified by stimulating Soleus in SCI patients.

Some researchers have attempted to predict force/torque variations with fatigue based on eEMG. An exponential function was proposed to predict the force of FES-activated quadriceps muscles from eEMG Peak-to-Peak (PTP; Mizrahi et al., 1994). PTP was suggested as a fatigue index at a constant cycling speed in (Chen and Yu, 1997). A close correlation between the EMG mean absolute value (MAV) and knee torque was found under continuous stimulation in paraplegic subjects (Erfanian et al., 1996). In their following work (Erfanian et al., 1998), they proposed a predictive model of muscle force production under an isometric percutaneous continuous FES system. A metabolic model was presented to predict force decline and recovery from EMG signals under intermittent condition (Levin and Mizrahi, 1999). In Ziai and Menon (2011), re-training the model was suggested to regain the high estimation quality lost as a result of degraded estimation accuracy over time. Therefore, the online estimation method proposed in this study is preferable to characterize muscle contraction dynamics for real-time FES control. However, the fatigue properties vary with different fatigue levels and recovery processes, which complicates the identification of the fatigue model. A fatigue model cannot work when the desired stimulation pattern is unknown in advance.

As in **Figure 1**, an online model estimation method is proposed for FES-induced torque prediction and muscle fatigue tracking through Kalman filter update (Zhang et al., 2011a,b). Muscle contraction dynamics modeled by a polynomial Hammerstein model (PHM) to represent joint torque from muscle activity based on evoked EMG, is updated systematically through Kalman filter with forgetting factor to correspond to the time-variant muscle contraction dynamics. This is further extended toward robust estimation with a nonlinear autoregressive with external input (NARX) model-based recurrent neural network (RNN) to predict torque with evoked EMG (Li et al., 2014b). The computational efficacy also makes it feasible for real-time implementation (Li et al., 2014a, 2016).

On the basis of the good predictive performance of the proposed estimation method, a new control strategy, i.e., EMG-Feedback Predictive Control, is proposed to explicitly control joint torque under FES (Hayashibe et al., 2011a; Zhang et al., 2013). Both muscle excitation and contraction dynamics are modeled by PHM. The eEMG signal is used for a dualpurpose: to correlate stimulation with the muscle electrical behavior in the muscle excitation process and to correlate the muscle electrical behavior with the muscle mechanical behavior in the muscle contraction process. Evoked EMG signal was used to feedback actual muscle states to track desired joint torque while considering the time-variant muscle dynamics. The EFPC control problem was resolved as a solution of two Nonlinear Predictive Control problems in series corresponding to activation and stimulation controller, respectively, as shown in **Figure 1**. The personalized models for excitation and contraction processes are developed for each subject for the model predictive controller to compute the inversed solutions. The activation controller solves the necessary muscle activation pattern to track the desired mechanical output reference. The stimulation controller solves the necessary stimulation sequence to achieve the necessary muscle activations. Once the torque deviates from the desired trajectory due to the effects of variations in muscle states such as fatigue, the controller adaptively generates the appropriate stimulation pattern in a systematic way to achieve the desired torque as long as it does not conflict with the stimulation constraints. This control framework provides satisfactory control accuracy and notable robustness in terms of joint torque control in FES (Zhang et al., 2013). This control strategy has a capacity to perform muscle fatigue compensation benefited from evoked EMG feedback.

## 4. MUSCLE ACTIVATION PREDICTIVE CONTROL AND CANCELATION OF THE STIMULATION ELECTRODE EFFECT

This section presents muscle activation closed-loop FES control through evoked EMG. It can be regarded as a simple version of EFPC regarding stimulation and muscle activation only.

This muscle activation control can present an advantage of Evoked Electromyographically Controlled Electrical Stimulation. In conventional FES, we normally specify the stimulation pattern and there is no way to check if the muscle is really responding as needed. As the electromagnetic motor is normally observed through the encoder, it is advantageous to monitor the reaction of muscles to electrical stimulation. Since evoked EMG can always be observable through the acquisition system, eEMG could be used for updating the stimulus-to-eEMG model. This could improve the modeling precision of the plant and guarantee the accuracy of the predictive model controller.

A real-time implementation of the predictive model controller for online control of muscle activation is as follows (Li et al., 2015):

1. The reference muscle activation trajectory is prepared before beginning estimation and control;


In this way, muscle activation can be fully specified in the framework of Evoked Electromyographically Controlled Electrical Stimulation, instead of specifying the stimulation parameters. This can be useful to compensate for time-variant processes in FES. **Figure 2** shows an example of cancelation of the stimulation electrode effect. At the time 65 s, part of the electrode is suddenly detached without informing it to the system, as indicated by the circle. Muscle activation can be easily influenced by this situation. However, as the change is observed with evoked EMG signal from the targeted muscle (tibialis anterior), the controller systematically modifies the stimulation input to be increased so as to track the desired reference pattern of muscle activation. It is pulse width control then it may not fully cancel

FIGURE 2 | Stimulation electrode effect cancelation in FES: The upper figure represents the experimental setup with typical foot drop correction montage for the stimulation electrode to stimulate the peroneal nerve for the dorsi flexion of the ankle. EMG is measured for the tibialis anterior muscle. At time 65 s, one electrode is detached, as indicated by the circle. The first plot shows the muscle activation obtained by evoked EMG. The second plot shows the stimulation pulse width. Y-axes are in normalized scale. Note that the stimulation is systematically modified to compensate for the effect of the electrode detachment. Muscle activation could thus still be maintained to minimize the effect of the stimulation field variation.

the effect but it could minimize the effect of sudden muscle activation changes by the detachment or the contact condition changes of the electrode. The stimulation electrodes are with typical montage for foot drop correction by FES.

#### 5. CONCLUSION

Transcutaneous electrical stimulation (TES) is the technique most frequently applied for muscle and nerve activation. It is known that it has high sensitivity to the electrode contact condition for electrical stimulation. To compensate the intersubject (muscle strength) and intrasubject variability (stimulation electrode contact change, muscle fatigue), it is essential to monitor the real muscle responses under FES. Evoked Electromyographically Controlled Electrical Stimulation could be one solution to deal with this important issue. This framework contributes to augmenting the FES system in several aspects. First, an appropriate personalized muscle response model under FES (stim-emg, emg-torque) could be quickly established. Prediction of joint torque affected by muscle fatigue could be performed based on evoked EMG. This allows us to

#### REFERENCES


systematically compute and control electrical stimulation so as to achieve the desired muscle activation even under time-variant disturbances such as muscle fatigue and contact changes of stimulation electrode. An example of stimulation electrode effect cancelation was demonstrated to show the promising performance of Evoked Electromyographically Controlled Electrical Stimulation, which enables us to induce stable muscle activation in TES.

#### AUTHOR CONTRIBUTIONS

MH wrote the perspective article along with previous research activities he has performed as a PI of this work.

#### ACKNOWLEDGMENTS

The author thanks Dr. Qin Zhang (Huazhong University of Science and Technology, China) and Dr. Zhan Li (University of Electronic Science and Technology of China, China) for their contributions during the PhD studies at our team, and Dr. David Guiraud (INRIA, France) for helpful discussions.


activation of paralyzed hand muscles of persons with tetraplegia. IEEE Trans. Neural Syst. Rehabil. Eng. 8, 286–296. doi: 10.1109/86.867870


**Conflict of Interest Statement:** The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2016 Hayashibe. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Generation of Locomotor-Like Activity in the Isolated Rat Spinal Cord Using Intraspinal Electrical Microstimulation Driven by a Digital Neuromorphic CPG

Sébastien Joucla1, 2, Matthieu Ambroise3, 4, Timothée Levi 3, 4, Thierry Lafon1, 2 , Philippe Chauvet 1, 2, Sylvain Saïghi 3, 4, Yannick Bornat 3, 5, Noëlle Lewis 3, 4, Sylvie Renaud3, 5 and Blaise Yvert 1, 2, 6, 7 \*

#### Edited by:

Mikhail Lebedev, Duke University, USA

#### Reviewed by:

Wolfgang Stein, Illinois State University, USA Johnny Padulo, eCampus University, Italy Shawn B. Frost, University of Kansas Medical Center, USA

> \*Correspondence: Blaise Yvert blaise.yvert@inserm.fr

#### Specialty section:

This article was submitted to Neuroprosthetics, a section of the journal Frontiers in Neuroscience

Received: 21 December 2015 Accepted: 15 February 2016 Published: 07 March 2016

#### Citation:

Joucla S, Ambroise M, Levi T, Lafon T, Chauvet P, Saïghi S, Bornat Y, Lewis N, Renaud S and Yvert B (2016) Generation of Locomotor-Like Activity in the Isolated Rat Spinal Cord Using Intraspinal Electrical Microstimulation Driven by a Digital Neuromorphic CPG. Front. Neurosci. 10:67. doi: 10.3389/fnins.2016.00067 <sup>1</sup> Centre National de la Recherche Scientifique, Institute for Cognitive and Integrative Neuroscience (INCIA), UMR 5287, Talence, France, <sup>2</sup> Institute for Cognitive and Integrative Neuroscience (INCIA), UMR 5287, University of Bordeaux, Talence, France, <sup>3</sup> Laboratoire de l'Intégration du Matériau au Système, UMR 5218, University of Bordeaux, Talence, France, <sup>4</sup> Centre National de la Recherche Scientifique, Laboratoire de l'Intégration du Matériau au Système, UMR 5218, Talence, France, <sup>5</sup> Bordeaux INP, Laboratoire de l'Intégration du Matériau au Système, UMR 5218, Talence, France, <sup>6</sup> Institut National de la Santé et de la Recherche Médicale, Clinatec-Lab, U1205, Grenoble, France, <sup>7</sup> Université Grenoble Alpes, Clinatec-Lab, U1205, Grenoble, France

Neural prostheses based on electrical microstimulation offer promising perspectives to restore functions following lesions of the central nervous system (CNS). They require the identification of appropriate stimulation sites and the coordination of their activation to achieve the restoration of functional activity. On the long term, a challenging perspective is to control microstimulation by artificial neural networks hybridized to the living tissue. Regarding the use of this strategy to restore locomotor activity in the spinal cord, to date, there has been no proof of principle of such hybrid approach driving intraspinal microstimulation (ISMS). Here, we address a first step toward this goal in the neonatal rat spinal cord isolated ex vivo, which can display locomotor-like activity while offering an easy access to intraspinal circuitry. Microelectrode arrays were inserted in the lumbar region to determine appropriate stimulation sites to elicit elementary bursting patterns on bilateral L2/L5 ventral roots. Two intraspinal sites were identified at L1 level, one on each side of the spinal cord laterally from the midline and approximately at a median position dorso-ventrally. An artificial CPG implemented on digital integrated circuit (FPGA) was built to generate alternating activity and was hybridized to the living spinal cord to drive electrical microstimulation on these two identified sites. Using this strategy, sustained left-right and flexor-extensor alternating activity on bilateral L2/L5 ventral roots could be generated in either whole or thoracically transected spinal cords. These results are a first step toward hybrid artificial/biological solutions based on electrical microstimulation for the restoration of lost function in the injured CNS.

Keywords: hybrid neural networks, neural prostheses, neuromorphic hardware, rehabilitation

## INTRODUCTION

Following spinal cord injury, neural prosthesis using electrical stimulation of the spinal circuitry below the lesion can be considered to restore functional locomotor capabilities (Giszter, 2008; Borton et al., 2013; Nishimura et al., 2013). These approaches rely on the ability of intraspinal central pattern generators (CPGs) to generate locomotor rhythms even when isolated from descending inputs, as first assessed ex vivo (Brown, 1911; Grillner et al., 1981; Grillner and Wallén, 1985; Cazalets et al., 1992, 1995; Kjaerulff and Kiehn, 1996), and then confirmed in vivo in animals and humans (Dimitrijevic et al., 1998; Ichiyama et al., 2005; Minassian et al., 2007). Based on this property, very promising rehabilitation results have been achieved in rodents using epidural stimulation of lumbar segments combined with pharmacological drug applications and rehabilitation training (Courtine et al., 2008, 2009; Musienko et al., 2009; van den Brand et al., 2012). Such approach also allowed a paraplegic patient to achieve assisted standing and stepping movements with full-weight bearing (Harkema et al., 2011).

Because epidural stimulations can activate large networks from the surface of the spinal cord including proprioceptive fibers, a higher degree of control in the activation of the spinal circuitry may be expected using intraspinal microstimulation (ISMS). It has been found in cats that different stereotyped hindlimb mouvements could be elicited by ISMS delivered on single microelectrodes, depending on the position of the microelectrode. Microstimulations delivered in dorsal L5- S1 segments generally elicit hindlimb flexions, while ventral stimulation mostly evokes hindlimb extension (Tai et al., 2003; Lemay and Grill, 2004). More rostral stimulations delivered in the dorsal half of L3-L5 segments also elicit ipsilateral flexion (Barthélemy et al., 2006). Combining ISMS delivered on the dorsal surface of L3-L7 segments with intraveneous injection of the noradrenergic agonist clonidine could further elicit bilateral locomotion (Barthélemy et al., 2006, 2007). Yet, producing sustained locomotion by intraspinal ISMS solely in absence of drugs remains unachieved. Toward this goal, tonic ISMS delivered at 20–50 Hz on microwires implanted in the lumbar ventral horn to target motoneurons of spinalized cats could elicit episodes of hindlimb flexion, extension, and even alternating sequences (Saigal et al., 2004; Guevremont et al., 2006; Lau et al., 2007).

These encouraging results open the way to the design of autonomous neural prosthesis, where coordinated sequences of ISMS control sustained locomotion. For this purpose, artificial neural networks can be used to drive coordinated ISMS sequences, a method recently used to control intramuscular stimulations to restore locomotor behavior (Vogelstein et al., 2008; Mazurek et al., 2012). Here we address this question in the neonatal rat spinal cord isolated ex-vivo and interfaced with a penetrating microelectrode array. This preparation can indeed generate locomotor-like activity under pharmacological activation while offering a direct access to intraspinal networks. Here, our goal was to achieve a proof of principle that an artificial neural network can control ISMS to drive locomotor-like activity in this preparation. Two intraspinal sites were identified at L1 level, the alternated stimulation of which generated locomotor-like activity on bilateral L2/L5 ventral roots. An artificial CPG was then implemented on FPGA to control ISMS on these two sites. This hybrid connection could successfully be used to generate lumbar locomotor-like rhythms in a whole and a transected spinal cord.

### METHODS

### Ethics Statement

All experimental protocols conformed to recommendations of the European Community Council Directive of November 24, 1986 (86/609/EEC) and local French legislation for care and use of laboratory animals. They were approved by the local ethical committee of Bordeaux under recommendation No A5012083.

#### Experimental Preparation and Recording

Whole spinal cord and medulla from newborn Sprague Dawley rats at postnatal stage between P1 and P3 (**Figure 1A**) were dissected in a cooled artificial CSF (aCSF) solution (pH 7.5) gassed with carbogen (95% O<sup>2</sup> and 5% CO2) and composed of (in mM): 113 NaCl, 4.5 KCl, 2 CaCl22H2O, 1 MgCl26H2O, 25 NaHCO3, 1 NaH2PO4H2O, and 11 D-Glucose. Bilateral L2 and L5 ventral roots were recorded using succion glass electrodes. Ventral root signals were amplified with a gain of 750, band-pass filtered between 0.08 Hz and 3 kHz, and then sampled at a rate of 20 kHz using the previously developed NeuroPXI system (Bonnet et al., 2012), which is an extended version of the former BioMEA system (Charvet et al., 2010). As shown in **Figure 1B**, this preparation exhibits locomotorlike activity under the application of 5-HT (5µM), NMA (10µM), and Dopamine (50µM), and is thus a good model to explore rehabilitation strategies in vitro with an easy access to intraspinal networks. The cords were superfused for 10 min with this cocktail to check that locomotor-like activity could be elicited pharmacology in each preparation prior to hybrid experiments. The drugs were rinsed for at least 60 min to let ventral root activity return to baseline before hybrid experiments started.

#### Data Processing

Raw ventral root signals were processed as follows to extract spiking activity (see also Heim et al., 2012): for each data sample, a moving average of the signal computed over a 10 ms window centered on this sample was first subtracted from the raw data (DC-removing), and then the obtained value was replaced by the average of the signal computed over a 1-ms time window centered on this sample (smoothing). Signals were then integrated to assess alternating patterns using polar representations and statistics. For this purpose, the signals were first blanked over a period ranging from 15 ms before the stimulation to 150 ms after, and then integrated with a time constant of 0.2 s (see gray traces superimposed on spiking activity in **Figure 1B**) and finally smoothed with a window of 1 s. The statistical significance of the phase relationship between bursting activity recorded on two different ventral roots was

assessed using a Rayleigh test on the delays between the peaks of both integrated signals normalized to 360◦ . **Figure 1C** shows the typical polar representation of a locomotor-like activity elicited pharmacologically, with antiphase locking between L2 and L5 ventral roots on each side and left-right L2 or L5 pairs.

#### Electrical Stimulation

As shown in **Figures 1D–F**, electrical microstimulations were delivered on single microelectrodes of Neuronexus neural probes made of four shanks separated by 400µm, each containing eight 30-µm-diameter microelectrodes separated by 100µm (probe model A4x8-5 mm-100-400-703-A32). Based on earlier studies showing the localization of rat CPGs (Cazalets et al., 1995; Antri et al., 2011), the neural probe was inserted at the L1 level of the spinal cord. Depending on the preparation, between 15 and 26 sites of the probe were individually tested for stimulation. Each stimulation consisted of a train of 10 biphasic pulses separated by 1 ms. Each pulse was chargebalanced with an initial cathodic phase of 500µs and an amplitude of typically 150–300µA, immediately followed by a 10 times longer anodic phase of 10 times smaller amplitude. Stimulations were monopolar with respect to a distant electrode in the bath.

#### Artificial CPG

An artificial CPG neural network inspired from Hill et al. (2001) was implemented onto a configurable digital integrated circuit (FPGA), supported by an electronic platform designed at IMS (**Figure 1G**). The implementation of this network is fully described in Ambroise et al. (2013). In brief, the core CPG network shown in **Figure 1H** consisted of two symmetric sub-networks of four regular spiking neurons (N1L-N4L and N1R-N4R in **Figure 1H**). These eight neurons have identical biomimetic dynamics following the Izhikevich model (Izhikevich, 2003) and are interconnected by reciprocal inhibitory GABAA-like synapses. Synaptic efficiency was governed by activity-dependent depression (Tabak et al., 2000). The dynamics of each neuron was thus governed by four parameters a, b, c, and d according to the following differential equations:

$$\frac{d\nu}{dt} = \frac{\nu^2}{32} + 4\nu + 109.375 - \mu + I\_{bias} + \sum\_{i=1}^{N\_{\text{N}} \text{sp}\_i} I\_{sym}^i$$

$$\frac{du}{dt} = a(b\nu - u)$$

with the after-spike resetting condition:

$$\text{if } \nu \ge 30mV \text{ then } \begin{cases} \nu \gets c \\ u \gets u+d \end{cases}$$

where the other parameter Ibias is a constant bias current and I isyn is the synaptic current received from the ith input neurons. For each input, the synaptic current Isyn is governed by the following equation:

$$\frac{dI\_{\rm sym}}{dt} = -\frac{I\_{\rm sym}}{\mathfrak{r}\_{\rm syn}} + (1 - \lambda\_{\rm syn}) \cdot \mathcal{W}\_{\rm syn} \cdot \delta(t - t\_{\rm spike}),$$

where τsyn is the synaptic current decay time constant, δ(t − tspike) = 1 if the input neuron produces a spike at time t and 0 otherwise, and λsyn is the synaptic efficiency. The synaptic efficiency was dynamic with a decay time constant τreg and synaptic depression modeled through the following equations:

$$\frac{d\lambda\_{\rm sym}}{dt} = -\frac{\lambda\_{\rm sym}}{\mathfrak{r}\_{\rm reg}} \cdot \left(1 - \delta(t - t\_{\rm speike})\right) + P \cdot \left(1 - \lambda\_{\rm syn}\right) \delta(t - t\_{\rm speke}),$$

where P is a dissipation percentage parameter decreasing synaptic efficiency after each spike.

The design was optimized to cost few digital resources on the FPGA while running in real-time at the ms resolution (Ambroise et al., 2013). Both sub-networks of the CPG produced alternating rhythmic bursting activity (**Figure 1I**). The resulting bursting activity was integrated for each subnetwork by a fifth spiking neuron (N5L and N5R in **Figure 1H**), producing only one spike per burst, and representing the CPG output neurons. In the current system version, each parameter change in the neural network model requires a new synthesis of the FPGA configuration, which is done by the implementation of a new configuration file (bit file) from the computer to the FPGA, a procedure taking several seconds. Further versions will allow dynamic reconfiguration of the FPGA, thus allowing online synaptic adaptation for example.

#### Hybridization of the Artificial CPG to the Spinal Cord

The CPG output neurons produced a rhythmic left-right alternating activity made of 1 spike per cycle on each network side. These output spikes were used to trigger intraspinal electrical microstimulations on two different microelectrodes of the neural probe, one on each side of the spinal cord.

#### Statistical Analysis

The statistical significance of alternating patterns of bursting activity recorded on two ventral roots VR1 and VR2 was assessed by a circular Rayleigh test performed on the angular values of individual bursts. The amplitude of the Rayleigh statistics was compared to three significance levels (P < 0.05, P < 0.01, and P < 0.001) represented as different circles on the polar representations in **Figures 1C**, **3C**, **4C**. The angular values on which the test was performed were computed as follows. The peaks of integrated signals were first detected on each ventral root for each burst. Then, for each peak of VR2 occurring at time t2, we considered the two neighboring peaks occurring before and after t2 on VR1 (t1\_prev ≤ t2 and t1\_after ≥ t2) to compute a local angular value using the following relation:

$$\phi = \frac{\pi}{2} + 2\pi \frac{t\_2 - t\_{1\rho rv}}{t\_{1\rho lcr} - t\_{1\rho rv}}$$

#### RESULTS

#### Identification of Two Intraspinal Microstimulation Sites

An initial set of 10 preparations were used to identify the best levels of the spinal cord to target between T11 and L5 in order to elicit consistent responses on the L2/L5 ventral roots. This was done using either a single glass pipette microelectrode or a penetrating shank. We found that ISMS delivered at the L1 level were the most reliable. We then considered 6 other preparations in which we inserted a 4 × 8 probe transversally at the L1 level. Between 15 and 26 contacts were scanned successfully at different current intensity levels between 40 and 800µA. We initially tested classical symmetrical biphasic pulses (cathodic-first with a cathodic phase of 500µs), and found that they were less efficient than balanced but non-symmetrical pulses where the anodic phase was twice weaker and longer. Cathodic current intensities above 150µA were generally required to obtain reliable responses on the ventral roots. As shown in **Figure 2**, we found that ISMS at L1 level elicited different burst responses on bilateral L2/L5 ventral roots depending on the location of the stimulation site. ISMS delivered dorsolaterally elicited a burst on the ipsilateral L2 ventral root (**Figures 2A,B**). Stimulations delivered dorsomedially typically elicited a response on both ipsilateral L2 and L5 ventral roots (as in **Figure 2C**), and occasionally also on the contralateral L5 ventral root. Consistently across preparations, we identified two intraspinal sites, one on each side of the spinal cord located about 200µm laterally from the midline and approximately at a median position dorsoventrally near the central canal (about 400µm from the dorsal surface). We found that a stimulation delivered on either site elicited a burst response simultaneously on the ipsilateral L2 and contralateral L5 motor outputs (**Figures 2D,E**). This result thus opened the possibility to generate locomotor-like activity using coordinated stimulation between these two stimulation sites.

#### Generation of Alternating Patterns Using the Artificial CPG

The eight neurons of the core network had the following identical parameters that were found to lead to a robust alternating activity on the output neurons of the CPG: a = 0.02, b = 0.2, c = −65, d = 8, and Ibias =8. The time constant τsyn of the exponential decay of synaptic current was set to 100 ms. For output synapses from neurons N2R and N2L, we

intraspinal locations were found consistently across preparations and thus further used for the hybrid experiments. For all panels, a sequence of four stimulations is

illustrated.

used Wsyn = −1.26 and P = 0.1%; for input synapses onto neurons N2R and N2L, we used Wsyn = −3 and P = 0.03%. For all other inhibitory synapses, we used Wsyn = −3.8 and P = 0.03%. The output layer neurons N5R and N5L received excitatory AMPA-like connections respectively from N4R and N4L, both defined by Wsyn = 30 and P = 90%. The initial conditions were v = −20 and u= −4 for N1R, N3L, N4L, N2R neurons, v = 0 and u = 0 for N1L, N3R, N4R, N2L, and v = −65 and u = −13 for N5R and N5L. With these parameters, each CPG bursting activity pattern generated one spike on N5L and one spike on N5R, used as trigger for delivering microstimulation to the spinal cord. The stimulation is only triggered by the CPG output and consisted of a short train of 10 stimuli (see Methods). By changing τreg (from 4 to 12 s), the artificial CPG alternating period could be modulated between 2 and 7 s.

### Generation of Locomotor-Like Activity in a Whole Ex-vivo Spinal Cord Using a Hybrid Connection

In a first step, we made a hybrid connection between the artificial CPG and a whole spinal cord. The output neuron of each side of the artificial CPG controlled intraspinal stimulations on one of the intraspinal stimulation site (**Figure 3A**). Each spike of an artificial output neuron triggered one stimulation on the corresponding intraspinal site. Using this strategy, clear locomotor-like activity could be obtained on bilateral L2 and L5 ventral roots (**Figures 3B,C**). Once the artificial CPG was turned on, this activity established at the first or second stimulation, remained robust with a 1:1 correspondence to the artificially imposed pace as long as the artificial CPG was ON, and then vanished as soon as the CPG was turned off.

sites, respectively) driven by the artificial CPG at a cycle frequency of 6.7 s (integrated traces shown in gray), showing clear left-right and ipsilateral alternating patterns (top: 2 min, bottom 30 s). (C) Polar representation of this induced rhythm as in Figure 1C.

### Generation of Locomotor-Like Activity in a Transected Spinal Cord Ex-vivo Using a Hybrid Connection

### DISCUSSION

In a second step, we tested whether this result could also be obtained in a lumbar spinal cord fully isolated from descending inputs to mimic a lesion situation. We thus considered another preparation, which was transected at the T7 level (**Figure 4A**). The same type of hybrid connection was used between the artificial CPG and the intraspinal stimulation sites. As shown in **Figures 4B,C**, this strategy could also successfully elicit locomotor-like activity on bilateral L2 and L5 ventral roots. The locomotor pattern could be maintained for more than 7 min as long as the artificial CPG was ON. Moreover, several periods of alternating cycle were tested in this preparation spanning the range allowed by the artificial CPG (2.6, 3.1, 4, 5.2, and 6.7 s between successive left and right stimulations). We found that the spinal cord could follow the imposed rhythm in an exact 1:1 correspondence at these different frequencies for the whole durations of the hybrid connection tested: 560 s at of 6.7 s (84 left+right stimulations), 430 s at 5.2 s (85 stimulations), 369 s at 4 s (97 stimulations), 392 s at 3.1 s (123 stimulations), and 363 s at 2.6 s (217 stimulations). The artificial CPG was limited to these period values and we thus could not test higher speeds to see when the spinal cord would stop following the artificial network. Nevertheless, the tested frequencies covered the pace of a pharmacologically-evoked rhythm with inter-burst period around 5 s on each ventral root (as shown in **Figure 1B**).

The goal of this study was to test whether ISMS controlled by an artificial neural network could in principle be used to restore locomotor-like activity in a transected spinal cord. This proof of principle was achieved here in the neonatal rat spinal cord isolated ex vivo, using the advantage that this preparation can display locomotor rhythms while offering a direct access to intraspinal networks. This paradigm allowed identifying appropriate intraspinal sites for stimulation, which were localized on either side of the central canal at L1 level. Efficient stimulations could be obtained for current levels of the order of 150–300µA. These current amplitudes were higher than those generally used in previous in vivo studies to elicit limb movements using ISMS in cats (Guevremont et al., 2006) and rats (Shahdoost et al., 2014). One possibility could be the difference in the frequency of the pulse train between these studies and our paradigm. Here we used only 1 ms between two successive pulses of the same train, while in vivo ISMS typically used lower frequency 40–50-Hz trains, likely to be more efficient. Moreover, the level of excitability of the CPGs might be different between our ex vivo situation at room temperature and the in vivo context where the complete network including sensory feedbacks remain present. Also, the activity elicited here on the ventral roots did not result from a direct activation of the motoneurons since a stimulation delivered on a precise site in L1 elicited activity simultaneously on the ipsilateral L2 and the contralateral L5. Thus, we created an indirect activation of the motoneurons, likely

by activating a group of interneurons participating in the lumbar CPG. It is possible that higher stimulation intensities are required to robustly activate the whole CPG rather than generating more discrete muscle contractions. As a result and based on previous studies (Ranck, 1975; Yeomans et al., 1986; Joucla et al., 2012), it is likely that the currents we used typically activated neurons within a distance of about 250–500µm from the electrode. Nevertheless, we could verify experimentally the specificity of our ISMS. Indeed, many electrode positions of the same probe were tested for stimulation and, as illustrated in **Figure 2**, the response to ISMS on the four ventral roots greatly depended on the site of stimulation. In particular, there was a clear dissociation of the response for the two stimulation sites that we identified on each side of the central canal, which were separated by typically only 400µm and yet triggered completely different output patterns (opposite L2-L5 diagonals). Using adjacent sites of the same shanks distant by only 100µm from the optimal stimulation sites did not produce consistent activation of the L2-L5 diagonal. Actually, the fact that we were able to reproduce locomotor-like activity patterns by coordinating ISMS on these two sites was precisely based on this dissociation.

The localization of the two intraspinal sites was in the region of the lumbar CPGs, which have previously been localized at the upper lumbar region within T13 and L2 segments (Cazalets et al., 1995; Kjaerulff and Kiehn, 1996). The ISMS positions identified here fall in this region and are close to several classes of projecting commissural interneurons (Kiehn and Butt, 2003). Moreover, these locations are also consistent with a high density of neurons involved in alternating locomotor activity, identified between L1 and L2 levels with calcium imaging (Antri et al., 2011). Altogether this suggests that ISMS may activate elements of the lumbar CPGs responsible for the coordination of opposite L2 and L5 motoneuronal pools, maybe through the direct activation of commissural interneurons. It should be noted that each stimulation typically triggered only one burst episode on output roots (as clearly seen in **Figures 2**–**4**). Thus, all bursts were followers of the stimulation pulses. In particular, no wind-up was observed since bursts disappeared as soon as the stimulation stopped, and the frequency of the locomotor pattern could be imposed by the frequency of the artificial CPG. For these reasons, it is possible that each stimulation activated a group of interneurons participating in the lumbar CPG, and that the coordination of two stimulation sites could allow reproducing locomotor-like activity by the coordinated activation of these groups.

The quest for fully autonomous neural prosthesis based on hybrid connections between the CNS and artificial neural networks is a tremendous challenge. Here, we only performed a first step toward this goal by making a unidirectional (open-loop) connection between an artificial CPG and the spinal cord circuitry. In this work, the artificial network was initially configured with proper parameters to exhibit adequate alternating rhythmic activity. In particular, modifying the frequency of the rhythm required a manual reconfiguration and re-synthesis of the network in the FPGA. Further versions of the CPG hardware implementation will offer the possibility to modulate its dynamics in real time. This will then open the possibility to build bidirectional hybrid connections, where supralesional activity can be used to control the artificial network dynamically in order to achieve a close loop artificial connection over the lesion. In particular, future developments could include paradigms where the artificial network would be dynamically controlled and/or modulated by inputs from supraspinal and/or intraspinal supralesional information, and possibly modulated in real time by sensory feedbacks produced after ISMS in preparations preserving whole hindlimbs attached to the spinal cord.

As a further perspective, such hybrid approach will also need to be extended in vivo in order to assess whether it can also help recover locomotion capabilities in adult animals subject to chronic spinal cord lesions. While this has started to be addressed using ISMS in spinal cats (Guevremont et al., 2006; Mazurek et al., 2012), little ISMS work has yet been obtained in rats (Shahdoost et al., 2014). Previous lesion results in adult rats showed that destruction of the gray matter at T13-L2 level induce severe locomotor deficits, while more caudal lesions have more limited influence (Magnuson et al., 2005), suggesting a localization of neonatal lumbar CPGs consistent with those of adult animals. Yet, whether the two stimulation sites identified in this study remain conserved in adulthood to elicit locomotor movements in spinal animals remain to be tested. Moreover, in the case of chronic animal models of paralysis, lesions may induce remodeling of intraspinal circuits on the long term (Dietz and Müller, 2004), which may change the way networks could be activated. However, previous studies indicate that despite such plastic changes, lumbar CPG circuitry remain present and can be reactivated below the lesion with training to recover functional locomotion (Barrière et al., 2008; van den Brand et al., 2012). An open question is whether such rehabilitation perspective can also be obtained using active neural prosthesis solely based on ISMS. Chronical experiments will thus be necessary to determine if the spinal circuitry below the lesion may also be exploited through hybrid connections with artificial neural networks to

#### REFERENCES


recover locomotor functions with autonomous spinal neural prosthesis.

### CONCLUSION

In conclusion, the present study is a first demonstration of a hybrid interconnexion between a living spinal cord and an artificial neural network driving ISMS to restore functional activity. These results are a first step toward intelligent neural prostheses based on hybrid live/artificial connections for the restoration of lost function in the injured CNS.

#### AUTHOR CONTRIBUTIONS

SJ, SR, and BY designed the study. MA, TLe, SS, YB, NL, and SR designed the artificial CPG and associated electronics (Multimed system). SJ, MA, TLa, PC, and BY developed the experimental setup. SJ and MA performed the experiments. SJ and BY analyzed the data. SJ, TL, SS, NL, SR, and BY wrote the manuscript.

#### ACKNOWLEDGMENTS

This work was supported by the French National Research Agency (ANR) under the Hyrene project No ANR-BLAN-0316 and by the European Union's Seventh Framework Programme (ICT-FET FP7/2007-2103, FET Young Explorers scheme) under the grant agreement n◦ 284772 BRAINBOW. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. The authors also wish to thank Nathalie Argenta and Dominique Panzeri (INCIA) for taking care of the animals, Eric Basier (INCIA) and Gilles N'Kaoua (IMS) for technical help, and Lionel Rousseau and Gaëlle Lissorgues from ESIEE-Paris for constructive discussions.


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2016 Joucla, Ambroise, Levi, Lafon, Chauvet, Saïghi, Bornat, Lewis, Renaud and Yvert. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Bio-Inspired Controller on an FPGA Applied to Closed-Loop Diaphragmatic Stimulation

Adeline Zbrzeski 1, 2 \*, Yannick Bornat 1, 2, Brian Hillen<sup>3</sup> , Ricardo Siu<sup>3</sup> , James Abbas <sup>4</sup> , Ranu Jung<sup>3</sup> and Sylvie Renaud1, 2

*<sup>1</sup> Bordeaux INP, IMS, UMR 5218, Talence, France, <sup>2</sup> Univ. Bordeaux, IMS, UMR 5218, Talence, France, <sup>3</sup> Department of Biomedical Engineering, Florida International University, Miami, FL, USA, <sup>4</sup> School of Biological and Health Systems Engineering, Arizona State University, Tempe, AZ, USA*

Cervical spinal cord injury can disrupt connections between the brain respiratory network and the respiratory muscles which can lead to partial or complete loss of ventilatory control and require ventilatory assistance. Unlike current open-loop technology, a closed-loop diaphragmatic pacing system could overcome the drawbacks of manual titration as well as respond to changing ventilation requirements. We present an original bio-inspired assistive technology for real-time ventilation assistance, implemented in a digital configurable Field Programmable Gate Array (FPGA). The bio-inspired controller, which is a spiking neural network (SNN) inspired by the medullary respiratory network, is as robust as a classic controller while having a flexible, low-power and low-cost hardware design. The system was simulated in MATLAB with FPGA-specific constraints and tested with a computational model of rat breathing; the model reproduced experimentally collected respiratory data in eupneic animals. The open-loop version of the bio-inspired controller was implemented on the FPGA. Electrical test bench characterizations confirmed the system functionality. Open and closed-loop paradigm simulations were simulated to test the FPGA system real-time behavior using the rat computational model. The closed-loop system monitors breathing and changes in respiratory demands to drive diaphragmatic stimulation. The simulated results inform future acute animal experiments and constitute the first step toward the development of a neuromorphic, adaptive, compact, low-power, implantable device. The bio-inspired hardware design optimizes the FPGA resource and time costs while harnessing the computational power of spike-based neuromorphic hardware. Its real-time feature makes it suitable for *in vivo* applications.

Keywords: bio-inspired controller, spiking neural network (SNN), closed-loop paradigm, ventilatory control system, metabolic demands, spinal-cord injury (SCI), field programmable gate array (FPGA), assisted ventilation

### INTRODUCTION

Approximately 282,000 Americans (∼0.1% of the US population) have traumatic spinal cord injury (SCI), with approximately 17,000 new injuries being reported each year (National Spinal Cord Injury Statistical Center—NSCISC, 2016). The cervical cord is the most common site of injury (54% of all cases). An individual with high-level cervical SCI at or above the fifth cervical level may experience partial or complete loss of ventilatory control (Warren et al., 2014) because the motor fibers of

#### Edited by:

*Eberhard E. Fetz, University of Washington, USA*

#### Reviewed by:

*Jan-Marino Ramirez, University of Washington, USA Andrew Joseph Fuglevand, University of Arizona, USA*

> \*Correspondence: *Adeline Zbrzeski adelinezbrz@gmail.com*

#### Specialty section:

*This article was submitted to Neuroprosthetics, a section of the journal Frontiers in Neuroscience*

Received: *03 February 2016* Accepted: *01 June 2016* Published: *16 June 2016*

#### Citation:

*Zbrzeski A, Bornat Y, Hillen B, Siu R, Abbas J, Jung R and Renaud S (2016) Bio-Inspired Controller on an FPGA Applied to Closed-Loop Diaphragmatic Stimulation. Front. Neurosci. 10:275. doi: 10.3389/fnins.2016.00275* the phrenic nerve, which innervates the diaphragm, originate from the 3rd to 5th cervical level. Respiratory diseases and failure are the leading cause of death after SCI (67.4% of these were cases of pneumonia) (NSCISC, 2016). Ventilatory insufficiency is often managed with positive pressure mechanical ventilation, but such treatment is associated with significant discomfort, diaphragmatic atrophy, atelectasis and barotrauma (DiMarco, 2005). Most mechanical ventilators act in an open-loop manner, but some systems for partial ventilatory support offer dynamical control of the mechanical ventilation through proportional assist ventilation or neurally adjusted ventilatory assist (Yonis et al., 2015). These systems aim to amplify the patient's native intrinsic respiratory effort. The practicality of these designs is limited by the need for non-invasive and reliable measures of the mechanical characteristics of the respiratory system and need for a reliable positioning of an instrumented transtracheal catheter to obtain reliable measure of the diaphragm electrical activity (Navalesi and Costa, 2003; Terzi et al., 2012).

An alternative approach to mechanical ventilation uses trains of electrical pulses to elicit contractions of the paralyzed (or paretic) ventilatory musculature (DiMarco, 2009; Madsen, 2012; Ho et al., 2014). In many individuals, this approach has been used to achieve suitable ventilation, but the performance of existing assistive ventilatory control system technology is limited. Open-loop ventilatory control systems require iterative manual tuning by a practitioner of parameters of a fixed stimulation pattern. Open-loop systems do not have the ability to adjust these parameters if manual tuning does not produce a stimulation pattern that adequately meets metabolic demands. Furthermore, muscle fatigue and changes in electrode response properties can alter the degree of ventilation produced by an open-loop stimulation, which can also result in an unmet respiratory need. In other systems that use electrical stimulation of paralyzed muscles, traditional feedback controllers (e.g., Abbas and Chizeck, 1991; Schearer et al., 2014) and adaptive approaches (e.g., Abbas and Chizeck, 1995; Riess and Abbas, 2001; Fairchild et al., 2010) have been able to achieve high quality control of limb movements. For ventilatory assistance, a closed-loop adaptive control system that uses electrical stimulation could overcome the drawbacks of open-loop control and offer improved quality of life to individuals with paralysis that impairs ventilation.

Neural networks in the brain modulate the drive to the respiratory muscles to meet metabolic demands under diverse conditions. The spatially distributed pontine–medullary respiratory network activates the spinal motor neurons that innervate the respiratory muscles (Richter, 1996; Smith et al., 2009). The activity of the respiratory network is continually adjusted by two major types of feedback: mechanical feedback (lung volume mechanoreceptors) and chemical feedback (levels of CO<sup>2</sup> and O<sup>2</sup> in the blood and brain tissue chemical receptors). This activity can be modulated by higher brain centers (voluntary control, posture, emotion, etc.). Therefore, the respiratory network activity is sensitive to various stimuli to drive breathing and non-breathing behaviors (vocalization, eating, straining, airway protection reflexes, e.g., coughing, sneezing, or swallowing, etc.). The respiratory network has a robust yet highly flexible network organization that can permit multiple state-dependent modes of operation (Smith et al., 2009). The neural network can be computationally modeled as a network of spiking neurons. Spiking neural network (SNN) paradigms have gained significant attention in the past decades (Maass, 1997) and have inspired many hardware control systems used for robot control algorithms (Arena et al., 2010; Ayers et al., 2010; Nichols et al., 2013), classifier devices (Hsieh and Tang, 2012), new algorithms (Lagorce et al., 2015), or assistive devices for therapy or rehabilitation (Jung et al., 2001; Vogelstein et al., 2008; Sun et al., 2011; Ambroise et al., 2013). Some closed-loop ventilatory control systems have been proposed (Sharshar et al., 2003; Beck et al., 2007) but none have used a bio-inspired SNN control system.

In this paper, we address the need for an adaptive ventilatory control system with a new concept: a bio-inspired SNN controller (**Figure 1**). If the native diaphragmatic drive is totally or partially interrupted by SCI, then ventilation may not be sufficient. A closed-loop system can provide stimulation that adapts in response to measures of physiological variables. Our innovative closed-loop architecture for ventilation assistance is inspired by the pontine–medullary respiratory network to benefit from its robustness and flexibility. The neural network controller is implemented on an FPGA (Field Programmable Gate Array), which is a general approach that has been used since the early 1990s (Cox and Blanz, 1992). It is a fully reconfigurable digital integrated circuit that has low demands on power and computational resources. To achieve high computational efficiency, our controller uses fixed-point computation and implements low computational cost neuron and synapse models. Our design is inspired by the biological circuits but it is not intended to emulate a precise model at the network or cellular level. Our multidisciplinary collaboration has produced a hardware-in-the-loop real-time (HL) simulation platform that uses a computational model of rat ventilation and a fixed-point SNN controller model. The HL simulation was used to validate the bio-inspired SNN controller prior to its implementation in hardware and use in animal experiments.

We first assess the feasibility of implementing a bio-inspired SNN controller in digital hardware. Then, we develop a design methodology for a closed-loop ventilatory control system using our HL computational model and its FPGA implementation. We describe the models (both mathematical and animal) as well as their implementation and use in a HL simulated experiment. Finally, we provide proof-of-concept HL computational results that illustrate ability of the system to adapt the stimulation parameters to meet metabolic demands.

### MATERIALS AND METHODS

#### Bio-Inspired Controller Architectures

The neural networks underlying respiratory rhythm and pattern generation and control of ventilation have been studied extensively. Based on these investigations a pontine–medullary respiratory network model has been proposed for control of ventilation (Smith et al., 2000, 2009) that includes a tonic drive network in the pons, the retrotrapezoid nucleus/parafacial respiratory group, and in raphe nuclei; a respiratory rhythm

initiating and generating core in the Bötzinger and Pre-Bötzinger regions of the medulla; and follower rostral and caudal ventral respiratory neural network groups. In this model, the core network generates the respiratory rhythm and shapes inspiratory and expiratory motor patterns of the follower ventral respiratory group neurons. The latter contain the pre-motor neurons projecting to the phrenic motoneurons and bulbospinal expiratory neurons for the control of respiratory muscles. The respiratory neural populations are identified by their spiking discharge patterns and by their possible main contribution in ventilatory control (Richter, 1996; Nuding et al., 2009; Smith et al., 2009; Mellen and Thoby-Brisson, 2012; Richter and Smith, 2014). Computational scientists have proposed network models to mimic both the neural population activity patterns and specific modulation of the respiratory network (Lindsey et al., 2012; Molkov et al., 2014; Toporikova et al., 2015).

The bio-inspired SNN controller is based on experimental neuroscience and computational studies of the pontine– medullary respiratory network at a high level (i.e., the architecture) as well as at a low level (i.e., the population activity). We have developed two generations of architecture: an open-loop system (solely black architecture in **Figure 2A**) and a closed-loop system (both black and gray architecture in **Figure 2A**). In the network, each SNN controller neuron is inspired by the behavior of one population of the respiratory network. The basic organization includes three sub-networks: the FPGA core network defines the stimulation dynamics for each breath, the FPGA tonic network controls the core network, and the FPGA follower network converts the core network spiking activity into a stimulation signal to the diaphragm. The SNN controller's core and follower neurons are used in both

open and closed-loop architectures. Neuron names in the SNN refer to their discharge pattern (dec for decreasing, inc for increasing) during a respiratory phase (E for expiration, I for inspiration): decreasing activity pattern expiration (Edec neuron), spiking activity during the inspiration-expiration transition (IE neuron), decreasing activity pattern inspiration (Idec neuron), spiking activity during the expiration-inspiration transition (EI neuron), and increasing pattern inspiration (Iinc neuron). Other neurons enable reproduction of specific features of the dynamics of the biological network (e.g., additional triggering neurons such as IdecTON and EITON implement excitatory neuron's dynamics). Finally, additional neurons modulate in real-time the bio-inspired SNN controller behavior using sensor information: VolSpk directly influences volume; C1Spk and C2Spk increase and decrease CO2, respectively. CO<sup>2</sup> was chosen as the primary chemical controller for metabolic demand as end-tidal CO<sup>2</sup> which reflects the partial pressure of arterial CO<sup>2</sup> is measureable in vivo and provides a very strong influence to respiratory drive.

The bio-inspired SNN controller architecture reflects the respiratory rhythm and pattern generation network architectures presented by Richter and Smith (Richter, 1996; Smith et al., 2009) while being adapted to high density and low power consumption constraints of implanted technology. The Richter and Smith architectures mostly describe the same populations with little variation, such as the core population late-I in Richter (1996) is not explicitly represented in Smith et al. (2009). One common description is the network formed by reciprocal inhibitory connections between two neurons—called the primary inhibitory oscillator. The coupled Idec and Edec neurons mimic the so-called early-I and post-inspiration populations (or PI or post-I).

It should be noted that the SNN architecture does not possess intrinsic pacemakers as in the biological network. The FPGA controller uses a self-excitation connection to mimic intrinsic rhythmogenic capability: the IdecTON neuron triggers the Idec; the IE neuron triggers the Edec neuron; and the EITON triggers the EI neuron. Another example is the connections between the core and follower populations. The follower population could either be inhibited by most of the core population (Richter, 1996) or excited by one core population (pre-I/I in Smith et al., 2009) but in both cases it is an excitatory follower population. In the FPGA controller, a self-excitatory connection on the follower Iinc neuron is implemented. This self-excitatory connection was already presented in Richter (1996). The EI neuron triggers Iinc, as suggested by the short activation of the pre-I population prior to the follower activity in Richter (1996). Similarly, the Edec neuron inhibits the follower neuron Iinc as suggested by the inhibition of PI neuron on the follower neuron in Richter (1996).

The closed-loop architecture adds three tonic neurons to the open-loop architecture: Volspk, C1spk, and C2spk (gray in **Figure 2A**). Tonic neurons encode signals measured by sensors into spike patterns. The tonic neuron activities can update the strength of specific connections in the bio-inspired SNN controller at each computation step. There is scarce information on the network architecture and the activity of biological tonic populations., Spike-timing-dependent plasticity (STDP) as a Hebbian synaptic learning rule has been identified in biological neurons (Markram et al., 1997; Bi and Poo, 1998). It has been demonstrated in various neural circuits over a wide spectrum of species, from insects to humans (Caporale and Dan, 2008), and various types of STDP have been identified (Abbott and Nelson, 2000). We chose to use simplified STDP learning rules (**Figure 2B**) to modulate connections according to breathing rate and depth as well as for injury level and metabolic demands: P(f), P(a), and P(inj). While the open-loop controller is triggered by an external excitatory stimulation (not shown in **Figure 2A**), the closed-loop controller is triggered by the tonic neuron Volspk activity.

The electrical drive pattern reflects the follower neuron Iinc on-going activity. Neuroscience and computational studies usually refer to the neuronal population activity as integrated population activity (population spike frequency histograms) or cycle-triggered histograms (average firing rates of the neurons during the respiratory cycle) reported as spikes per second per neuron (Nuding et al., 2009; Smith et al., 2009). The stimulation ratio is the number of spikes out of 20 measured over the last 20 computation steps (i.e., 10 ms). It represents the activity of each neuron from the bio-inspired SNN. The stimulation ratio is the proportion of maximum activation (stimulus) of the respiratory muscle, 0–1 (in this case, 0 spikes = 0, 10 spikes = 0.5, and 20 spikes = 1, etc.).

#### Spiking Neural Network Model

The low complexity leaky integrate-and-fire (LIF) neuron formalism was chosen for its minimal computational demand while maintaining key timing properties of biological neurons. The neuron is modeled as a leaky integrator of its total current from all synapses Itot(t), defined in Equation (1):

$$
\pi\_m \frac{d\nu\_m(t)}{dt} = -\nu\_m(t) + R\_m I\_{tot}(t) \tag{1}
$$

where τ<sup>m</sup> is the membrane time constant, vm(t) the displacement of neuron voltage from the resting potential, and R<sup>m</sup> the input resistance. The LIF neuron fires a spike when the variable vm(t) reaches a fixed threshold. An absolute refractory period forbids the neuron to fire a spike within a fixed period of time after each spike.

The kinetic model for SNN connectivity is an adaptation of the alpha model presented by Destexhe et al. (Destexhe et al., 1994). This kinetic model requires a single multiplier component. It eases the implementation of various kinetics and it "accounts for saturation and summation of multiple synaptic events, obviating the need for event queuing" (Destexhe et al., 1994). The kinetic model is based on a fraction of bound receptors r(t). Destexhe et al. approximate kinetics by two expressions, described in Equations (2) and (3):

1. During a Spike (t0 < t =< t1):

$$r(t - t0) = \left(r\_{\infty} + (r(t0) - r\_{\infty})\right) \exp(-\frac{t - t0}{\tau\_r})$$
 
$$\text{with } r\_{\infty} = \frac{\alpha}{\alpha + \beta} \\$ \tau\_r = \frac{1}{\alpha + \beta} \tag{2}$$

2. After a Spike (t > t1):

$$r(t - t1) = \,\_r r(t1) \exp(-\beta(t - t1))\tag{3}$$

where r<sup>∞</sup> is the maximum fraction of bound receptors, τ<sup>r</sup> and 1/β are time constants of the two phases of r(t) (i.e., during and after spike), and α and β are the forward and backward rate constants for transmitter binding (i.e., how fast or slow the kinetic model is). Unlike the Hodgkin and Huxley conductance model (Hodgkin and Huxley, 1952), α and β in this model are not time-dependent.

The total current from all synapses Itot(t) is the sum of synaptic currents IS(t). Each synaptic current is taken to be proportional to the fraction of bound receptors r(t), defined in Equation (4):

$$I\_{\mathbb{S}}\left(t\right) = A\_{\text{SE}}\,r(t) \tag{4}$$

where ASE is the absolute synaptic strength, which can be exhibited only by activating all bound receptors. Nichols et al. (2013) used a similar approach to compute synaptic current to simulate a biologically inspired SNN for robot motion control. They used an alternative kinetic model with three interdependent derivative equations (Tsodyks et al., 1998). Although their system was easily implemented in software for simulation, it was not implemented in hardware. The R<sup>m</sup> Itot(t) variable, equivalent to the Total Post-synaptic Potential (TPP), is defined in Equation (5):

$$R\_m I\_{tot}(t) = \sum\_i P\_i \ r\_i(t) \tag{5}$$

where P is the absolute synaptic strength potential as determined by the multiplication of ASE by Rm—the input resistance of the neuron. The input resistance R<sup>m</sup> is fixed in this model; ASE could be modified by plasticity rules. Combining both variables into one, P, avoids the use of a multiplier component. Finally, absolute synaptic strength potential is used as the connection strength in the bio-inspired SNN network.

To summarize, the neural network model is defined by three processes. The "Kinetic" process updates the fraction of bound receptors r(t) according to defined kinetic model. The "TPP" process computes each neuron R<sup>m</sup> Itot(t) variable. The "V<sup>m</sup> & Spike" process updates the Vm(t) variable, the refractory period status, and may fire a new spike according to the defined LIF neuron model.

#### FPGA Implementation (Open-Loop)

The bio-inspired SNN controller was described in Very High Speed Integrated Circuits Hardware Description Language (VHDL) to define how to use and connect the hardware components available on the FPGA. The SNN controller was implemented on a Xilinx Spartan-6 FPGA (XC6SLX150: 4 Mbit full speed block RAM, 180 embedded multipliers, 92 k 6-input look-up tables and 184 kb distributed memory), hosted on a custom board, presented elsewhere (Zbrzeski et al., 2015). We used UART communication for storage and display of real-time variable data from the SNN.

A sequencer coordinates two computing phases of the SNN that update and share two dynamical variables, r(t) and Vm(t) (**Figures 3A,B**). Both computing phases rely on the previously defined models, taking advantage of their simplicity and their computational efficiency. Then, the sequencer controls a third idle phase to refresh the variables at a defined frequency (in order to achieve biological real-time). In this application, realtime is defined by the controller's ability to elicit diaphragm contraction with on-going real-time updates of the stimulation parameters. The stimulation frequency range is between 75 Hz (typical) and 500 Hz in open-loop stimulation on a rat phrenic nerve or diaphragm (injured or not) (Mantilla et al., 2010; Kowalski et al., 2013). Stimulation frequency is lower for larger animals (e.g., dog, 10–50 Hz) (Walter et al., 2011). The bioinspired controller updates the SNN behavior and variables at 2 kHz, i.e., the computation time-step dt is 0.5 ms. It does not depend on the number of neurons and it ensures accurate control of stimulation pattern.

The first phase is the Kinetic computing. The "Kinetic" process is the conversion over discrete dt time-steps of the fraction of bound receptors r(t) from the model defined in Equations (6)–(8) using forward Euler integration.

FIGURE 3 | Architecture of the Spiking Neural Network (SNN) implemented on an FPGA. (A) Spatial and (B) temporal SNN representations, with *r(t)* the fraction of bound receptors stored in the Kinetic RAM, *P* the absolute synaptic strength potential, *Rm I tot(t)* the Total Post-synaptic Potential (TPP), and *Vm* the displacement of neuron voltage from the resting potential stored in the Spike RAM as well as the spiking activity of the associated neuron. Small or bold arrows represent, respectively, a computing process reading "R" or writing "W" in a Random-Access Memory (RAM). The SNN sequencer (Seq.) manages two computing phases, followed by an idle phase. The first computing phase updates each kinetic variable *r(t)* associated with one pre-neuron. The second computing phase updates the spike status for each neuron by computing *Rm I tot(t)* variable with the TPP process, followed by computing *Vm*, the refractory period, and the spike status with the "Vm & Spike." The SNN connectivity is stored in mapping RAM.

1. During a spike (t0 < t = < t1)

$$r[t+1] = A \ r[t] + B \tag{6}$$

2. After a spike (t > t1)

$$r[t+1] = C \, r[t] \tag{7}$$

where A, B, C are constants defined as.

$$A = dt \,\alpha \quad B = 1 - dt \, (\alpha + \beta) \quad \text{C} = 1 - (\beta/dt) \tag{8}$$

**Table 1** presents five sets of kinetic parameters associated with one neuron. Each set consists of 4 parameters: the maximum fraction of bound receptors (r∞), the rising exponential time constant (τ <sup>r</sup>), and the forward (α) and backward (β) rate constants for transmitter binding. Each kinetic parameter set determines the values of constants (A, B, and C) that are



*Each row presents a pre-defined parameter set that is available for selection when implementing the controller. The* α *and* β *are the forward and backward rate constants for transmitter binding. r*<sup>8</sup> *is the maximum fraction of bound receptors.* τ *<sup>r</sup> is the time constant of the increasing exponential characterizing the r(t) behavior during a spike. A, B, C are the unsigned 14-bit constants used for implementation on the FPGA.*

implemented using 14-bit unsigned integers to streamline realtime computation. A 3-stage pipeline is used to update all five kinetic parameters connected to one neuron output. One multiplier is used per kinetic parameter. It takes 519 clock cycles to update 512<sup>∗</sup> 5 fractions of bound receptors r(t), saved in block RAM as 0.18 unsigned vectors (i.e., rational values between 0 and 1 coded on 18 bits).

The second phase consists of computing spikes. It is the conversion over discrete dt time-steps of the LIF model using forward Euler integration, defined in Equation (9). After a spike, a neuron has a minimum refractory time of one time-step during which membrane potential is set at its resting potential (in this case, −70 mV).

$$\nu\_m[t+1] = \nu\_m[t] + \frac{dt}{\pi\_m}(\nu\_{rest} - \nu\_m[t] + R\_m I\_{tot}[t])$$
 
$$\text{with } R\_m I\_{tot}[t] = \sum P \, r[t] \tag{9}$$

The ratio dt/τ<sup>m</sup> is set at 8 to use a shift operation instead of a multiplication. The "TPP" process updates the R<sup>m</sup> Itot[t] signal, as illustrated in **Figures 3A,B**. The R<sup>m</sup> Itot[t] signal is connected to the "V<sup>m</sup> & Spike" process. The implemented architecture does not include the plasticity mechanism, but the block RAM accesses are already available for such a mechanism. The "TPP" and "V<sup>m</sup> & Spike" processes each use a 3-stage pipeline to update all neurons. Four multipliers are used in the "TPP" process. It takes 1036 clock cycles to update 512 neurons during this second phase.

The SNN architecture can construct up to an 8-to-all 512 neuron network. Each connection is modeled by 1 among the 5 possible kinetic connections of **Table 1**. The implemented network has a low connectivity density. The implemented SNN controller uses the open-loop architecture (**Figure 2A**). **Table 2** presents the open-loop SNN synaptic parameters.

#### Hardware-in-the-Loop Real-Time Computational Model

The bio-inspired SNN controller can provide complementary diaphragmatic electrical stimulation to the native phrenic stimulation. The closed-loop version of the SNN controller adapts the stimulation (timing, amplitude) to meet the metabolic demands by adapting the depth and frequency of breathing. As a precursor to animal experiments, the SNN controller was evaluated in a computational model. **Figure 4A** illustrates a hardware-in-the-loop real-time (HL) computational model using a rat computational model and a fixed-point FPGA SNN controller model. The HL model maintains real-time processing capability (0.5 ms).

The computational model of rat ventilation is a real-time dynamical model with the ability to simulate rat ventilation for various breathing states. The bio-computational model includes a native drive model, an activation model (a linear summation of native and stimulated activation, a simplified first approximation), a musculoskeletal model, and a metabolism model. The model was developed to reproduce the phrenic native drive of an injured/non-injured animal, the combined muscle activation drive, eupneic respiratory volume, and a residual CO<sup>2</sup> quantity in the model.

The musculoskeletal model was scaled to fit the eupneic respiratory output of a non-injured rat. This model was developed from a human computational model in Simulink/SimMechanics implementing the physiologically realistic muscle model from MSMS (Hillen and Jung, 2014). Proportional muscle activation (0–1, proportion of spikes out of 20) as provided by the SNN was transformed into motor unit activation using the intramuscular functional electrical stimulation model from Virtual Muscle (Song et al., 2008) as implemented in MSMS which uses a single motor unit of each type where firing frequency is specified for stimulus intensity using a recruitment order from Singh et al. (2000) for intramuscular stimulation. The lung volume displacement was assumed linearly related to the diaphragm displacement, as illustrated in Richter (1996). This approximation was determined to be acceptable, especially when considering the system in eupneic conditions and when focusing on the volume shape during inspiration phase. Ventilatory compliance was modeled as a damped spring with non-linear stiffness. The nonlinear stiffness values for the rat thorax/diaphragm were chosen from experimental data (Young et al., 1992). Damped spring and activation amplitude from the human musculoskeletal model were adjusted such that the rat musculoskeletal model produced lung volume trajectories similar to experimental collected rat data (10 experimental data recordings, as presented in **Figure 4B**). Native activation was represented as a spaced sawtooth function and defined the pattern of the native model.

Experimental data from one male Sprague Dawley rat (n = 1, 360 g) (Siu et al., 2015) was used to determine the desired lung volume waveforms that were then used by the control system in the simulation studies. Data was collected with the approval of the Institutional Animal Care and Use Committee of Florida International University. The rat was maintained under eupneic conditions (21% O2, 0% CO2, balance N2) through the use of a gas mixer (GSM-3, CWE Inc.). The rat was anesthetized via IP delivery of pentobarbital (45 mg/kg) with supplemental isoflurane (0.5–1%). A pulse oximeter monitored SpO2, while end-tidal CO<sup>2</sup> was monitored via a CO<sup>2</sup> analyzer (Capstar-100, CWE Inc.) throughout the study to monitor and assess animal health. A pneumotachometer (8420A, Hans-Rudolph) collected air flow, which was then processed through a hardware



*Each row corresponds to a pre-synaptic neuron i; each column corresponds to a post-synaptic neuron, j. For each pair of connected neurons, the first value indicates the kinetic connection Ki*,*<sup>j</sup> between neurons i and j; the second value indicates the absolute synaptic strength potential Pi*,*<sup>j</sup> . Note that the ON neuron is used to automatically start the SNN controller after its initializations when used in open-loop mode.*

integrator (PI-1000, CWE Inc.) to obtain breath volume. Bipolar stainless steel barb electrodes were inserted into the diaphragm to record electromyography. Flow, volume, end tidal CO2, and electromyography measurements were collected at 6 kHz.

The metabolism model was created to dynamically represent the quantity of CO<sup>2</sup> in the computational model. The metabolism model describes generation of CO<sup>2</sup> (by mass) in the body and the exhalation of CO<sup>2</sup> with each breath (as determined by breath volume/rate). Baseline CO<sup>2</sup> production (MBC0) was set for netzero increase in CO<sup>2</sup> in the body during tidal breathing/eupnea. Increases in metabolic demand are represented by increases in CO<sup>2</sup> production rate. The eupneic computational condition was defined for a respiratory rate (RR) of 56 breaths per minute and a tidal volume of 2 mL. The CO<sup>2</sup> amount is constant on a breathby-breath basis. If there is an increased metabolic demand while the rat computational model RR and/or volume are not altered, then the CO<sup>2</sup> amount will increase on a breath-by-breath basis.

The native model output is the native drive (e.g., phrenic drive) with a fixed amplitude, and fixed or variable respiratory rate. In a biological rat, RR and tidal volume would increase to keep the CO<sup>2</sup> amount constant. A linear regression defines the relationship between RRs and MBC parameters to provide constant CO<sup>2</sup> amount. It defines the dynamical RR mechanism. Increased metabolic demand conditions of +11% and +17% are called MBC11 and MBC17. These two increased metabolic demands lead to a RR of 62.31 and 65.75 breaths per minute in the non-injured rat model to maintain constant average CO2. Also, the native model was varied to roughly mimic the effect of an incomplete spinal cord injury on ventilation by attenuation of the native drive through a weighted injury factor. In experimental studies, at 1 day post-injury and in the eupneic condition, rat tidal volumes were reported to be between 70 and 90% of the non-injured tidal volumes (Fuller et al., 2006; Navarrete-Opazo et al., 2015). For use in these simulations, we defined 6 pairs of injury level and metabolic demand conditions: 90%- MBC11, 90%-MBC17, 80%-MBC11, 80%-MBC17, 75%-MBC11, 75%-MBC17.

The SNN controller provides a simulated electrical drive. The SNN network was simulated using the computing phases, computing algorithms and pipeline implementations, and hardware limitations (fixed-point dynamic and accuracy), detailed in section FPGA Implementation (Open-Loop). The HL computational model predicts the behavior of the FPGA implementation of the SNN controller.

Computational results for both open and closed-loop SNN controllers are presented. The open-loop SNN controller was simulated with parameters that represent a complete injured rat computational model (native drive: 0%) with normal metabolic demands (MBC0). The closed-loop SNN controller was simulated with parameters that represent an incomplete injured rat computational model (native drive: 90, 80, 75%) with normal and increased metabolic demands (MBC0, MBC11, MBC17). The coefficient of variation (CV) of RR was computed over the last 10 breathing cycles to assess the variation of the RR over time. Constant RR was defined as breathing that exhibited a coefficient of variation of RR less than 4%.

The same SNN architecture was used for all models. The strength synaptic potentials P(inj) were set to optimize the efficiency of the bio-inspired controller for each pair of injury level and metabolic demand. P(a) and P(f) were set only in open-loop configuration, and were adapted by the controller in the closed-loop configuration. Breathing conditions are characterized with respiratory rate using the 10 last breathing cycles, and the related coefficient of variation.

#### RESULTS

### Complete Injury Experiment with Open-Loop Controller

The bio-inspired controller described in Section Materials and Methods was used with the complete lesion model (native drive: 0%): diaphragm activation was specified as described in Section Bio-Inspired Controller Architectures. The electrical drive mimics the activity of the follower neuron Iinc (**Figure 5A**). Both frequency and amplitude of the electrical drive are fixed in the open-loop paradigm. The open-loop bio-inspired SNN controller uses 7 neurons and 17 connections with low computational power (e.g., neuron and kinetic simplified behavioral models). Device utilization after synthesis of the SNN controller for the open-loop system is listed in **Table 3**. The design operates at 124 MHz. Mixed pipeline, parallel structure and shift operations were used to design a network with optimized hardware resource costs.

The activity of this bio-inspired network is similar to experimental reports (Richter, 1996), as reflected in activity patterns of individual populations and in temporal sequencing and coordination across the network (**Figure 5A**). The coordinated pattern of observed activity is as follows: the Idec activity increases quickly before slightly decreasing, this is followed by a slight then a fast increase of Iinc activity; the Iinc activity decreases abruptly when the IE activity quickly increases; this triggers Edec activity (and IE activity goes off); then Edec activity increases first quickly then slowly, which turns on EI activity to briefly activate Idec activity; and the activation sequence is repeated. The Matlab SNN computational model was designed to emulate the high level dynamic behavior as well as the exact data processing dynamics of the FPGA component. The HL computational delay is the computation time-step itself; there is no buffering effect at the network level. The MATLAB implementation was used to develop the FPGA implementation, which runs in real-time with sub-microsecond precision. Dynamic behaviors and values are strictly the same between the computational model running on Matlab and its FPGA implementation from the network to intrinsic neuron variables as illustrated in **Figure 5B**. This observation was verified for all

Results are computed using the hardware-in-the-loop (HL) real-time computational model. SNN model uses the first architecture presented Figure 2A with a complete injured model (native drive 0%). Activation drive Activ. and Volume V are reported. (B) Illustration of the exact match between FPGA measure and HL computation. Example using the displacement of neuron voltage from the resting potential (Vm) for EI neuron.

SNN variables: membranes voltages, spike states and fractions of bound receptors of the kinetic model (not shown).

The relationship between the accuracy of the fractions of bound receptors and the system's ability to drive a wide range of stimulated respiratory rates is presented in **Figure 6**. We evaluated the impact of r(t) accuracy (number of bits) on the SNN output resolution. The HL computational tool facilitates the identification of the most sensitive absolute synaptic strength potentials, P(f), to the modulation of electrical drive frequency (i.e., the node between pre-neuron IE and post-neuron Edec neurons). The HL computational tool helps to find appropriate tradeoffs between technology resources and application function in a varied respiratory context. All SNN variable accuracies are fixed except for the r(t) accuracy (kinetic model). The maximum RR variation between bio-inspired controller with the highest simulated r(t) accuracy (32-bit) and lower accuracies (under, at and above 18-bit) are, respectively 4, 2, and 1 Br.min−<sup>1</sup> . Another



*Results obtained using XST synthesis software from Xilinx Inc. with standard optimization effort on timing, under the ISE 14.7 development environment.*

comparison is set by targeting a range of rat eupneic RRs (e.g., between 56 and 67 Br.min−<sup>1</sup> ). The associated range of absolute synaptic strength potentials P with high accuracy (32 bit) is from 300 to 700. The bio-inspired controller with r(t) accuracy above 18-bit approximately shares the same lower range, and can vary up to 100 with the lowest accuracy. The r(t) accuracy has a stronger impact on the higher range, e.g., from 700 (32-bit) to 850 (18-bit). Therefore, using low accuracy could potentially limit the ability of the system to deliver stimulation at a high frequency. Finally, we used 18-bit accuracy for the fractions of bound receptors r(t) in the bio-inspired controller: it has shown to be closely equivalent to a 32-bit structure and it minimizes the number of multipliers (limited in this technology to a 18-bit accuracy).

### Incomplete Injury Experiment with Closed-Loop Controller

The closed-loop architecture of bio-inspired SNN controller was designed, developed, and modeled for HL computation. It can dynamically synchronize its electrical drive with the native drive. The electrical drive can modify simulated animal breathing depth as well as respiratory rate. **Figure 7** shows the effect of activating the bio-inspired controller in an incomplete injury model (native drive: 90%). Breathing was not optimal prior to activation of the bio-inspired controller; depth of breathing was lower than in the non-injured model. The metabolism model sets a slightly higher breathing rate in the injured than in the non-injured model to keep equivalent minute ventilation: respiration depth was higher in the injured model with bio-inspired controller as compared to the non-injured model. The CV of RR reaches a peak of approximately 5% at cycle #11. The stabilization of the amplitude and frequency of respiration occurs in the 16th breath cycle, about 15 s after activation of the bio-controller while the CV of the respiratory rate parameter drops below the 4% mark. The activation pattern after t = 15 s was consistent. The stabilization of P(f) (sensitive to electrical stimulation frequency) has a faster timescale of 4 cycles (t = 6 s). The absolute synaptic strength potential, P(a), did not vary much upon activation of the bio-inspired controller (not shown).

The closed-loop system demonstrated robust performance when tested on 3 types of incomplete injury (native drive: 90, 80, 75%) with perturbations induced by activating the bio-inspired controller and by alterations in the rat model (sudden increases in metabolism of 11 and 17%). The bioinspired controller performed well in all cases by responding to the perturbation. The bio-inspired controller demonstrated adaptability by modifying the electrical drive to assist the incomplete injured model to reach equivalent breath depth and rate as in a non-injured model. The HL computational results with respect to a non-injured model are presented in **Figure 8** for all pairs of injury level and metabolic demand and for all perturbations. The absolute synaptic strength potentials P(inj) are more sensitive to the injury level than the metabolic demand increase except for one of them (P between Volspk and IE neurons, not shown). Data quantifying the effectiveness of the bio-inspired controller are presented in **Table 4**. First, the breath depth and rate were evaluated when the bio-controller was turned ON (**Figure 8A**). Initially, with the bio-controller OFF, the respiratory rates were all higher when the level of injury was large: respiratory rate increased from 11 to 31% from a non-injured model. Respiratory rate was reduced in all cases when the bio-controller was turned ON: a 4–10% error was measured when compared to the non-injured model but with a 5–23% improvement when compared to incomplete injured models with the bio-inspired controller OFF. The stabilization of the synchronized bio-inspired controller was reached in 14 breathing cycles with high injury models (80 and 75%) vs. 17 cycles with the lowest injury model (90%). The performance after activation of the bio-inspired controller was independent of network parameters for each injury level: e.g., injury at 90% has both a synchronization time and stabilization time of 17 breathing cycles.

Bio-inspired controller behavior as a function of metabolic demand is shown in **Figures 8B,C**. All but one condition was stable between 10 and 15 cycles after a metabolic perturbation. In the particular case of 80%-MBC17, the Iinc neuron behavior is slower than other SNNs. At cycle #100, bio-inspired controllers

are synchronized and lead to a difference in respiratory rate of 3– 8% when compared to non-injured models in the same metabolic demand condition. All results are stable at least for 300 s after controller activation and 180 s after metabolic perturbation.

## DISCUSSION

This work demonstrates the potential benefit of using bioinspired SNN controllers to restore impaired functions, e.g.,

impaired breathing ability with incomplete spinal cord injury. In addition, this work shows exploitation of hardware-in-theloop (HL) real-time computation to facilitate efficient design and development of a closed-loop system in a multidisciplinary research environment.

TABLE 4 | Delay of the bio-inspired controller to adapt to various perturbations expressed in breathing cycles.


*Perturbation is either when the controller is turned ON (OFF/ON) or when the metabolic demand increases (from eupneic to* +*11% or* +*17%) while the controller is ON. Injury level is defined by the native drive percentage (90, 80, 75%). One spiking neural network architecture was set with 6 slight variations of few absolute synaptic strength potentials to optimize the efficiency of the bio-inspired controller for each pair of injury level/metabolic demand increase. The number of breathing cycles where the coefficient of variation of the respiratory rate was less than 4% is reported. Also, normalized averaged respiratory rate at cycle #0 and #100 are reported between brackets.*

The FPGA development of a closed-loop system benefits from the bio-inspired architecture. This has been demonstrated by the fact that an all-to-all connectivity pattern is not mandatory to achieve suitable performance. This bio-inspired controller is functional in open and closed-loop configurations with a connection density of 30.3 and 30.7%, respectively. The summary presented in **Table 5** indicates that connection and neuron numbers vary widely across architectures. The biologically-based half-center or pattern generator architectures (Jung et al., 2001; Vogelstein et al., 2008; Molkov et al., 2014) have larger all-to-all density than those with feedforward architecture (Arena et al., 2010; Ayers et al., 2010), which are inspired by artificial neural network paradigms. We observe similar connection densities (29.9 ± 7.6%) to the biologicallybased architectures when using 8-to-all connectivity. With our architecture, up to 1660 neurons can be implemented with the same multiplier resources and computation timestep by increasing the Spike computing phase and decreasing the idle phase. If hardware implementations used densities comparable to the computational network, then increasing the maximum number of connection per neuron to MaxC would result in a very large network. The networks presented in **Table 5** have different ratios between the maximum numbers of connection C per neuron N. Therefore, a better understanding of the biological neural networks upon which we base our controllers may be as important as our ability to increase both neurons and MaxC resources to significantly improve closedloop system functionality. The bio-inspired SNN controller functions appropriately with limited resources, as demonstrated in both open and closed-loop paradigms with perturbations (**Figures 6**–**8**). **Table 3** reports the device utilization where only 5 multipliers and 19 18 kb-RAM-blocks are required. As mentioned earlier, the implemented system can extend its number of neurons up to hundreds without requiring additional resources. Moreover, the architecture presented in **Figure 3** is easily scalable.

This bio-inspired ventilatory controller simultaneously adapts and coordinates the breathing rate and pattern of the electrical drive over long timescales (breaths) with high computational power using short timescales (spikes). The intrinsic activity of the bio-inspired SNN controller is similar to respiratory networks (compare **Figure 5A** to results reported in Richter, 1996). The primary inhibitory oscillator is described as a coupled population of early-I and post-inspiration (or PI or post-I) neurons; these neurons are equivalent to the Idec and Edec neurons of the bio-inspired controller.

Surprisingly, a 10-neuron bio-inspired SNN also exhibits secondary behavior that is similar to that of the biological system: the double peak activity of the EI neuron. Yet, using a population instead of a single neuron could potentially improve the activity of others, such as Idec neuron. Indeed, the short duration of Idec activity affects the duration of the EI second peak activity (longer than experimental measures). The functionality of the spike-based bio-inspired approach has been illustrated with various conditions. Bio-inspired controller activities were computed over an equivalent relatively long time (from 3 to 5 min) while showing good stability (coefficient of variation of respiratory rate remains under 5%). In addition, SNNs are known to be robust to noise or artifacts (Navalesi and Costa, 2003; Terzi et al., 2012). The capacity for stable regulation and dynamic properties of the biological system (Warren et al., 2014) are reproduced by the bio-inspired SNN controller, as shown in **Figures 7**, **8**. These results demonstrate smooth synchronization with native breathing activity and robust responses to perturbations (turning ON the bio-inspired controller or sudden metabolic demand increase, for 3 examples of incomplete injury). In contrast with existing ventilatory assist systems that use electrical stimulation, the controller does not impose a fixed breathing rate. Also, the contribution of the electrical stimulation was appropriate and sufficient enough to improve respiratory rate despite disruptions in 3 injury models, reproducing the original desired breathing pattern.

One important aspect of this work is the hardware-in-theloop (HL) real-time computational model. The spectrum of rehabilitation applications is wide, especially considering all possible contexts, but few HL models have been presented: epilepsy and Parkinson's disease (Detorakis et al., 2015; Ehrens et al., 2015). Experimental and clinical studies will always ultimately be required to validate the safety and functionality of closed-loop rehabilitation systems, but computational studies can be utilized as intermediary steps. Regulatory agencies are increasingly recognizing the potential contributions of simulation studies throughout the device development and evaluation process. One of the highest priorities for 2016 of the Center for Devices and Radiological Health of the US Food and Drug Administration is to develop computational modeling technologies to support regulatory decision making. Computational model results could be used as evidence to reduce the size of some clinical trials.

One important asset of the HL model is that it enables efficient investigation of controller performance under a wide range of conditions (physiological parameters and controller parameters). For example, we identified one of the main sensitive nodes in the bio-inspired SNN to regulate and synchronize


TABLE 5 | Literature review of connection densities for various spiking neural networks used in neuroscience or applicable controller research.

*MaxC is the maximum number of connections C per neuron N in the network. All MaxC and 8-to-all densities were computed using the number of connections in the network vs., respectively, all-to-all neurons, MaxC-to-all and 8-to-all connections. Connections from external signals in closed-loop network were not included in the computation but are reported in brackets.*

the electrical drive with the native drive (**Figures 6**, **7A**) prior to any animal experiment. HL results highlight strengths and weakness in the network architecture. Another example is the identification of the importance of adapting Iinc absolute synaptic strength potentials for the control of the electrical drive pattern: the Iinc related plasticity can improve the electrical drive behavior with sufficient resolution in realtime. However, the proposed closed-loop architecture does not provide high enough accuracy in this respect (**Figure 7**). The HL results advocate turning the single Iinc neuron into a subnetwork, or population. The HL tool allows such development without compromising the other closed-loop functions. A third example is the independent implementation of active expiratory drive. The presented bio-controller does not include expiratory neurons identified in the literature (Richter, 1996; Molkov et al., 2014) but it still provides significant ventilatory assistance. Finally, the HL results call for further examination and development of learning rules. Learning rules in this work assist the breath depth and rate. Six similar sets of absolute synaptic strength potentials P(inj) were set to optimize the efficiency of the bio-inspired controller for each couple of injury level/metabolic demand increase (**Figure 8**). This HL computational study shows the need for a third learning rule related to the customization between the patient condition (injury level/ metabolic demand) and absolute synaptic strength potentials using a single SNN controller architecture. This third learning rule could be seen as equivalent to meta-plasticity in neuroscience. Learning rules could be easily implemented on the FPGA due to its scalable and modular structure at no additional time cost: updating all absolute synaptic strength potentials will be done during the kinetic phase, described in **Figure 3B**. Implementation of that third rule is the next step of this study combined with upcoming in vivo rat experiments.

In this study, we developed a rat computational model along with the accurate hardware implementation model to evaluate the feasibility of a bio-inspired controller for ventilation assistance in spinal cord injury applications. Each component of this model could be improved to more closely match experimental results. For example, the activation could include results reported from investigations of mechanisms of nerve electrical stimulation. Also, the metabolism model has a limited sensitivity: respiratory rate CV should not be higher than 0 at any time to represent breathing variability (**Figure 8**). Increasing the sensor diversity and number of sensors could improve the performance and versatility of the closed-loop system. Further insight from studies investigating the primary and secondary roles of various chemoreceptors (Ballantyne and Scheid, 2001; Huckstepp and Dale, 2011) could drive further enhancements to the bio-inspired system. For eventual implementation in humans, sensors will be required to gather reliable measures in a manner that minimizes inconvenience and limitations on activities such as speech. To measure the variables used in the simulation studies presented here, a small tube placed in or near the mouth or nose could be used to sample expired CO2; chest expansion sensors could be used to monitor volume. Further developments of sensor technologies would greatly facilitate clinical translation of adaptive ventilatory control systems.

## CONCLUSION

This innovative controller is inspired by neuroscienctific investigations and computational models of the respiratory neural network. Its SNN architecture has been designed to be functional and resource- efficient when implemented on an FPGA device while using neuron and kinetic models with low demands on computational resources. The closed-loop architecture takes advantages of the SNN structure to adapt the ventilatory control system with potentially numerous and varied types of feedback signals as illustrated by our custom hardware-in-the-loop real-time platform simulations. This work helps to design cross-disciplinary research experiments that pave the way to the design of the next generation of ventilation control systems.

### AUTHOR CONTRIBUTIONS

AZ developed the bio-inspired controller architecture and the SNN model. AZ and YB developed and tested the FPGA hardware. AZ and BH developed the HL models. RS prepared experimental materials and provided processed experimental data. AZ and SR analyzed simulated results. All authors contributed to the closed-loop concept for ventilatory assistance. SR, RJ, and JA supervised all aspects of the work. All authors discussed the results and commented on the manuscript.

### REFERENCES


#### ACKNOWLEDGMENTS

The authors would like to acknowledge the technical support of Ashwin MANGALORE and Gilles N'KAOUA. The authors would like to thank Dr. Jennifer HASLER for valuable discussions. This work was supported by the US-French Collaborative Research in Computational Neuroscience program: Agence Nationale pour la Recherche (ANR-13-NEUC-0001) and National Institutes of Health (R01-NS086088).


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

The reviewer JR and the handling Editor declared their shared affiliation, and the handling Editor states that the process nevertheless met the standards of a fair and objective review.

Copyright © 2016 Zbrzeski, Bornat, Hillen, Siu, Abbas, Jung and Renaud. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# A Review of Control Strategies in Closed-Loop Neuroprosthetic Systems

James Wright <sup>1</sup> \*, Vaughan G. Macefield1, 2, 3, André van Schaik <sup>1</sup> and Jonathan C. Tapson<sup>1</sup>

*<sup>1</sup> Biomedical Engineering and Neuroscience, The MARCS Institute, University of Western Sydney, Sydney, NSW, Australia, <sup>2</sup> School of Medicine, University of Western Sydney, Sydney, NSW, Australia, <sup>3</sup> Neuroscience Research Australia, Sydney, NSW, Australia*

It has been widely recognized that closed-loop neuroprosthetic systems achieve more favorable outcomes for users then equivalent open-loop devices. Improved performance of tasks, better usability, and greater embodiment have all been reported in systems utilizing some form of feedback. However, the interdisciplinary work on neuroprosthetic systems can lead to miscommunication due to similarities in well-established nomenclature in different fields. Here we present a review of control strategies in existing experimental, investigational and clinical neuroprosthetic systems in order to establish a baseline and promote a common understanding of different feedback modes and closed-loop controllers. The first section provides a brief discussion of feedback control and control theory. The second section reviews the control strategies of recent Brain Machine Interfaces, neuromodulatory implants, neuroprosthetic systems, and assistive neurorobotic devices. The final section examines the different approaches

#### Edited by:

*Michela Chiappalone, Istituto Italiano di Tecnologia, Italy*

#### Reviewed by:

*Paolo Del Giudice, Italian National Institute of Health, Italy Dennis J. McFarland, Wadsworth Center for Laboratories and Research, USA*

> \*Correspondence: *James Wright james.wright@uws.edu.au*

#### Specialty section:

*This article was submitted to Neuroprosthetics, a section of the journal Frontiers in Neuroscience*

Received: *07 March 2016* Accepted: *21 June 2016* Published: *12 July 2016*

#### Citation:

*Wright J, Macefield VG, van Schaik A and Tapson JC (2016) A Review of Control Strategies in Closed-Loop Neuroprosthetic Systems. Front. Neurosci. 10:312. doi: 10.3389/fnins.2016.00312* to feedback in current neuroprosthetic and neurorobotic systems.

#### Keywords: neuroprosthetics, control theory, closed-loop, brain-machine interface, feedback

## INTRODUCTION

A neuroprosthetic is a device or system that has an interface with the nervous system and supplements or substitutes functionality in the patient's body. For the purpose of this review we have included neuromodulatory systems and brain computer interfaces under the general description of neuroprosthetics, as well as rehabilitation systems such as exoskeletons. The key identifying characteristic of the neuroprosthetic is that it has an interface with the subject's nervous system, as distinct from an implantable devices such as an pacemaker or an insulin pump. Consequently there are a broad range of devices that we consider neuroprosthetics. To date there have been a number of reviews of neuroprosthetic systems. There is significant literature on classification algorithms and detection strategies (Schwartz, 2004; Lotte et al., 2007; Green and Kalaska, 2011; Borton et al., 2014; Morimoto and Kawato, 2015), including comparison and evaluation of the relative strengths of different approaches. However, there are fewer examinations the authors are aware of that investigate the different control approaches that have been implemented in neuroprosthetic settings. Performing such a review is made more difficult due to the small number of studies that have compared different control approaches within the same experiment, and the fact that many neuroprosthetic studies have by necessity been conducted with very small sample sizes, sometimes involving a single subject. Additionally many of the devices examined in this review are experimental or investigational, and are not yet in use in clinical or therapeutic settings (Sun and Morrell, 2014).

When considering these devices in the context of control and feedback it can be helpful to place them along a number of axes (**Figure 1**) to partition the large variety of systems and approaches. The first axis we have considered is the location of the interface with the nervous system, with the Central Nervous System (CNS) subdivided into the brain and the spinal cord, and then the Peripheral Nervous System (PNS), consisting of afferent and efferent pathways. Along this axis and additional distinction can be drawn between single channel systems that use a single electrode as the interface to the subject's nervous system, and multichannel systems which utilize many parallel channels for interface. Finally the channel can be unidirectional for simplex communication, or bidirectional for half-duplex, or full duplex communication.

Another axis to consider is the invasiveness of the interface. At one end we place noninvasive interfaces such as Electroencephlogram (EEG), Magnetic Resonance Imaging (MRI), and Electromyography (EMG). These interfaces are hampered by a variety of issues, including low spatial resolution and low signal to noise ratios. The advantage they enjoy is that they can be deployed with little risk, which has meant they are a popular platform for neuroprosthetic development. Next are minimally invasive interfaces such as microwire EMG and microneurographic recordings from the PNS. With a more invasive interface there are better quality signals, but still minimal intervention with the subject's body, reducing risk of complications. These interfaces are susceptible to movement, leading to changes in the quality of the recorded signal. The most invasive interfaces require surgical implantation, and come with risks associated with surgery as well as risk of damage to the part of the nervous system. Within this category a distinction can be drawn between the less invasive, such as cuff electrodes around PNS sites and Electrocorticogram (ECog)—these interfaces do not disrupt the blood brain barrier; and the more invasive Multi Electrode Arrays (MEA) in PNS nerves, motor or somatosensory cortex, Deep Brain Stimulators (DBS) to treat Parkinson's Disease with electrodes in the Globus Pallidus, Sub Thalamic Nucleus, or the Pedunculopontine Nucleus. These multichannel interfaces offer high resolution, but comparatively small spatial coverage (Krook-Magnuson et al., 2015). These electrodes come with risks that are still not fully understood. These primarily involve the physical trauma due to the insertion of the electrode, with effects occurring over different timescales. Shortly after insertion there is bleeding and swelling, as well as physical damage to neurons (Fernandez et al., 2014). Over longer timescales the presence of the recording device compromises the blood-brain barrier, allowing ingress of cellular and molecular components from elsewhere in the body (Schwartz, 2004). Additionally the long term stability of the recording site is often compromised, due to immune and mechanical actions on the microelectrodes (Krook-Magnuson et al., 2015). At the furthest extent of this axis are the interfaces that reconfigure the subject's nervous system. Targeted Muscle Reinnervation (TMR) surgically rewires an amputee subject's PNS, by deinnervating muscles that have no biomechanical role after the amputation and redirecting the preserved nerves from the amputated limb to the deinnervated muscles (Kuiken et al., 2009), allowing for high quality EMG recordings as control inputs to a prosthetic. Optogenetic techniques offer a non-electrical interface to neurons, by using light to activate special ion-channels. This technique enables individual neurons to be targeted, which is extremely difficult with cortical microelectrodes, as well as the

assistance in the same Modality, have an equivalent level of Invasiveness, or interface with the user's nervous system in the same Location.

possibility of selectively activating a class of neurons such as excitatory neurons instead of inhibitory (Krook-Magnuson et al., 2015). However, optogentic approaches require the introduction of genes that encode for light activated ion-channels (Deisseroth, 2011), and the issues associated with this may preclude this technique in humans.

The final axis is the modality of the prosthesis. Sensory neuroprosthetics offer input from artificial sensors, as in a cochlear implant or bionic eye, or modulate sensory input as is the case in neurostimulators for treating chronic pain. Systems to treat motor impairment are frequently referred to as Brain Machine Interfaces (BMI) or Brain Computer Interfaces (BCI) and infer motor intent from the subject in order to control a virtual or physical effector. Cognitive devices modulate the activity of the CNS and include devices such as DBS stimulators for Parkinson's Disease, depression, and hippocampal stimulators for memory.

### NEUROPROSTHETICS

The following examples are representative, but not exhaustive, and are placed in **Figure 1** to illustrate the wide variety of devices that can be described as having a neuroprosthetic interface. Many of these devices are experimental demonstrations, and not clinically approved interventions. Adjacent to the choice of controller, there are choices about the location and method for acquiring a suitable input signal (Grill et al., 2009; Andersen et al., 2010), and questions about the consequences of long term use of these and similar devices that have not yet been answered.

Peripheral nervous system interfaces are attractive as they are less invasive than the central nervous system alternatives, while still offering a rich source of information for neuroprosthetic control. Inmann and Haugland (2004) used a cuff electrode around the median nerve to record nerve activation due to touch and that input was used to modulate the Functional Electrical Stimulation of the subjects muscles. Horch et al. (2011) demonstrated that human subjects who were provided with intrafascicular electrical stimulation of the median and ulnar nerves derived from sensors on a myoelectric prosthetic limb were able to use the feedback to perform object recognition tasks by digital manipulation. Tan et al. (2014) used nonpenetrating cuff electrodes on the median, ulnar and radial nerves of human subjects to provide natural sensations of touch while operating a myoelectric prosthetic allowing for improved performance of manual tasks. Raspopovic et al. (2014) showed that a bidirectional interface with the median and ulnar nerves could be used to provide artificial sensory feedback related to the forces exerted on a sensorized prosthetic limb, and that the artificial sensations allowed the subject to improve their ability to sense characteristics of the objects being manipulated. Targeted Muscle Reinnervation generates a rich high density signal for surface Electromyography (EMG) that enables simultaneous operation of multiple degrees of freedom in a myoelectric prosthetic limb (Kuiken et al., 2009). The tissue serves as a bioamplifier for the nerve signal, allowing a large array of surface electrodes to be deployed on the subject. The array provides a rich signal suitable for pattern recognition, and combined with a high performance prosthetic limb gives the subject an improved experience. As a sensory modality prosthetic, (Dommel et al., 2009) are testing a vision prosthesis for electrical stimulation of the retina. Spinal cord stimulation may be able to generate gait patterns suitable for locomotion in paralyzed patients. Vogelstein et al. (2008) describes the design of a system that is capable of generating primitive locomotion in anesthetized felines. Borton et al. (2014) developed an electrochemical spinal neuroprosthesis to reactivate the circuits in a damaged spinal cord, allowing hindlimb movement sufficient to enable walking in paralyzed rats.

Yanagisawa et al. (2011) used ECog electrodes placed over the sensorimotor cortex of a stroke patient in order to control a supernumerary robotic hand that was able to mimic the hand posture of the subject. Berger et al. (2011) implanted microwire electrodes in the hippocampus of rats and recorded the activity while the animals were trained to complete a memory task. Subsequent stimulation of the electrodes according to the trained model improved the performance of the rats at the cognitive task. Neuromodulators for Deep Brain Stimulation have been used to treat the symptoms of Parkinson's disease and depression (Grahn et al., 2014). Multi Electrode Arrays implanted in motor cortex have been successfully used to acquire signals for the multi degree of freedom control of robotic limbs (Hochberg et al., 2012; Wodlinger et al., 2015). Guggenmos et al. (2013) describe a Brain Machine Brain Interface in rats that utilizes microwire recordings from a premotor area to detect spiking activity leading to stimulation of the somatosensory area. The Activity Dependant Stimulation via the neuroprosthetic prototype enabled rats with injury to the motor area to recover reach and grasp behavior.

This list of devices is not exhaustive, and serves only to illustrate the different axes of Modality, Invasiveness and Location when considering neuroprosthetic designs. For a more thorough description of the available neuroprosthetic devices and interface technologies, the reader is directed to the following excellent reviews (Navarro et al., 2005; Grill et al., 2009; Micera et al., 2010; Ortiz-Catalan et al., 2012). These reviews focus on the interface techniques with the nervous system, and provide a detailed discussion of the limits of current interfaces.

### INTRODUCTION TO CONTROL

The design of a neuroprosthetic varies significantly between different modalities. Because it is a multidisciplinary field often the language used to describe the system can vary between devices. The terms "closed-loop," "feedback," and "online" may take on different meanings. Identifying and acquiring a suitable input signal is a non-trivial task (Krook-Magnuson et al., 2015). This makes the fabrication of a substitute system in the case of impairment a complex endeavor. Developing a suitable simplified model for embodiment as an open-loop system is often a first step. But it is not always straightforward; take motor control as example—many models of muscles exist. Highly biophysical models (such as cross-bridge models) of muscles become large systems of non-linear differential equations when describing whole muscles or limbs (Ionescu and De Keyser, 2006). Similarly modeling the individual neurons that act as part of the Basal Ganglia, in a complex network of interconnections across the CNS (Broccard et al., 2014), rapidly becomes an intractable problem when developing an open-loop model for DBS to treat Parkinson's Disease. As a consequence the Basal Ganglia has often been modeled internally by a neurologist (Hosain et al., 2014), rather than explicitly within the DBS device, with stimulation parameters adjusted by the clinician observing the patient's symptoms instead of in response to a model. It is also possible that the control problem for a given neuroprosthetic has more than a single loop that needs to be considered, possibly due to the interaction of different physiological systems and different timescales (Houk, 1988). In this case control may best be achieved by a hierarchy of controllers, or a series of adaptive controllers that can be tuned at different stages of design (McFarland et al., 2006).

### Nervous System Control

All parts of the human body have evolved to operate by extremely complex closed-loop control. Different subsystems, such as the cardiovascular system or the immune system operate under closed-loop control, with sensors and effectors operating at micro (LeDuc et al., 2011) and macroscopic scales (Houk, 1988). Sensory organs can be directly connected to the nervous system, such as stretch receptors, or they can be indirectly coupled by messenger systems such as hormone signaling. Command of these systems can be voluntary, or have a voluntary component, but they may also be completely automatic. Substituting or supplementing the performance of an element of the body is the aim of a neuroprosthetic device. Achieving this involves the fabrication of an effector, such as a prosthetic limb, that can replicate at least a subset of the body's functionality. But there is no function without control, so it is also necessary to model, and potentially integrate with, the different control loops within the human body. There has been some good success with simpler systems, possibly due to the high level state abstraction of the control within the nervous system (Holinski et al., 2013), as in the control of the hand. By examining the joint angles for fingers during different hand postures (grasping different objects) using principal component analysis it has been shown that the first 3 principal components can account for 90% of the variability. However, the grasp posture data describes the hand only in the final state, it does not describe the trajectory the fingers took to achieve the position around the object. When joint angle data was recorded continuously from subjects performing natural hand movements, 8–9 principal components were needed to describe 90% of the variance (Danziger, 2014). Thus, classifying hand states, and transitioning between fixed postures in a prosthetic is a more straightforward task than attempting the dexterous control of individual fingers (Aggarwal et al., 2011). Although we can treat a robotic effector as part of the body and nervous system of the operator, all current techniques for recording neural activity involve the projection of the high dimensional space of hundreds to thousands of neurons down through the recording electrode array to the much lower dimensionality of the end point of the effector (Carmena, 2013).

### Feedforward Control

Feedforward or open-loop control generates a command for the plant that is expected to produce the correct output. However, there is no measurement of the output from the plant, and hence no measurement of error, so the controller has no mechanism to modulate a command (Houk, 1988). A block diagram of open-loop control is shown as **Figure 2**. Implicit within openloop control is the assumption of a perfectly described system that can be used to generate a control. Leaving aside the difficulties in creating a perfect model of any system, openloop approaches do not take noise or measurement error into account.

### Feedback Control

Feedback, or closed-loop control requires the inclusion of sensors in the system under control. The feedback controller generates a command for the plant, and the sensors measure the output of the plant in response to the command. If a measurement, such as the angle of a joint differs from the expected output, then the error signal can be used by the feedback controller to modify the generated commands. There are many mathematical approaches that can be used to modify feedback controller output (Crago et al., 1996).

### Adaptive Control

Adaptive control can be applied to both feedforward and feedback controllers. By using sensors to measure the input and output of the system adaptive control strategies seek to adjust the controller in response to perturbations in the environment or the controlled system (Crago et al., 1996). Adaptive controllers enable the development of a control strategy without requiring complete knowledge of the system being controlled, however, as a consequence adaptive controllers are rarely optimal.

### Internal Model Control

Internal Model Control (IMC) is an approach to feedback controller design that incorporates a model of the system that

is achieved by the inclusion of the Sensor component, shown here as the dashed line. The Sensor measures the Output enabling the Controller to assess the error and adjust the next Input to the Plant.

is being controlled (García et al., 1989). The model can be developed based only on the relationship between the inputs and outputs of the system, or alternatively a partial model or complete model of the system can be utilized (LeDuc et al., 2011). At each time step the internal model is evaluated forward to a horizon, offering a prediction of the system behavior in response to the controller's input, and the control inputs are evaluated against a cost function to find the optimum command to be executed at the next time step (Pan et al., 2015). A block diagram illustrates IMC as **Figure 3**.

#### Classification

A Classifier breaks a system into discrete states, and maps a relationship between an input and a system state (Schwartz, 2004). Classifiers can be supervised or unsupervised. Supervised systems are trained on an input-output relationship in a data set, and attempt to generalize the relationship to new data, while unsupervised systems attempt to partition or cluster the dataset.

### Actor-Critic

The Actor-Critic architecture separates the control policy from the evaluation of the action. The Actor component of the systems choses a policy, which affects the state of the system. The Critic component assesses the state of the system in accordance with a cost function, and provides the evaluation to the Actor (Mahmoudi and Sanchez, 2011). The goal based evaluation differs from the error signal of other control approaches, and does not require that the Actor has a model of the system's behavior (Mahmoudi et al., 2013).

## CONTROL ALGORITHMS

**Table 1** summarizes the Control Algorithms.

Control policies can be implemented by classification, model independent, or model based approaches (Kameneva et al., 2015).

### Control Policies with Independent Models

Consider the system state to be independent at each time step.

#### Bang-Bang Control

Also referred to as On-Off control, in this scheme when a threshold for a measured variable is crossed a program is activated. Although simple this control scheme has been used successfully to automate tasks that have previously required human intervention, such as the delivery of cortical electrical stimulation after ECog seizure detection (Peters et al., 2001), or the mapping of stimulus thresholds in high electrode count implanted neurostimulators (Wilder et al., 2009).

#### Finite State Machine

A State Machine is a model of a system. It can be considered a more complex implementation of Bang-Bang control. The measurement of a system value, combined with the modeled system's current state triggers an action and a state transition (Markovic et al., 2014). If the modeled system is periodic, such as gait during walking, then it can be possible to have transitions due to timing (Holinski et al., 2013), in which case the neuroprosthetic enables state transitions in response to deviants from the periodic behavior, such as starting or stopping the gait.

### Population Vector Algorithm

The biomechanics of the arm make motor control a difficult problem. Additionally the mechanisim of control within the motor cortex remains an open question. Is the cortical representation in an area such as the activity recorded from M1 encoding the lengthening or shortening of individual muscles (Schwartz, 2004), or is it representing the kinematics of movement (Ajemian et al., 2008)? In either case there is evidence for a forward and inverse model representation of motor control existing within the brain (Andersen et al., 2010; Green and Kalaska, 2011). The Population Vector Algorithm (PVA) is a popular method to decode neural activity recorded from cortical MEA in the motor cortex for the control of a robotic effector, or cursor in a 2D or 3D space. This control algorithm rests on the observation that different neurons have directional preferences they have higher spike rates for movements in some directions (Shpigelman et al., 2009). Individual neurons do not offer enough specificity to be useful, but a large enough population of neurons, recorded from simultaneously, can be used to determine the intended direction and movement velocity by linear regression. Neuroprosthetic control can then be performed by relying on a "targeting" strategy of decoding the end point trajectory apparent in each cell's activity.

In Helms Tillery et al. (2003) a non-human primate's BMI was extracting an X, Y, and Z signal for the end effector on the robotic arm and the other degrees of freedom of the arm were under the control of the robotic device. In Hatsopoulos et al. (2005) a human participant with 128 electrode array implanted in the precentral gyrus was able to achieve 2D cursor control on a laptop.

Initial implementations of PVA utilized cortical recordings made while the participant watched cursor movement, or the movement of a limb. However, although a PVA decoder created in this way may show good performance in offline testing, the closed-loop performance will not be better, and may be worse (Chase et al., 2009).


**217**

#### Control Policies with Dynamic Models

There are methods that rely on having a model of the dynamic system describing the parameter the neuroprosthetic is controlling. This might be a model of the spiking behavior of a region of the CNS that is to be modulated by a DBS, or a model of the kinematics of the trajectory of a cursor in a motor BMI, or the kinematics of the gait of an exoskeleton. Models can be linear or non-linear. Using the model, and an error signal, and modeling feedback the next state can be predicted using a variety of methods described below. These approaches are iterative, and work well as part of a closed-loop as they represent the process of the subject modifying the input control signal in response to the feedback signal.

A simple linear state model for velocity can be represented as:

$$\begin{aligned} \mathbf{x}\_{t} &= A\mathbf{x}\_{t-1} + \mathbf{w}\_{t-1} \\ \mathbf{y}\_{t} &= C\mathbf{x}\_{t-1} + q\_{t-1} \end{aligned}$$

where **x**<sup>t</sup> is the velocity as a 3 dimensional vector at time step **t**, **A** is a matrix of parameters describing the trajectory and **w**t−<sup>1</sup> is a noise term. The second equation describes the measurement model of the neural activity, **y**<sup>t</sup> . **C** is a matrix of parameters that relate velocity to the neural activity and **q**<sup>t</sup> is a noise term. This model can be expanded with the inclusion of an additional term to model the input of the control signal on the system state as follows:

$$\mathbf{x}\_{t} = A\mathbf{x}\_{t-1} + Bu\_{t-1} + \mathbf{w}\_{t-1}$$

where **u**<sup>t</sup> is the control signal at time step **t,** and **B** is a matrix of parameters describing the trajectory. The task of the following algorithms is to predicte the state in order to generate an error, the difference between the predicted and observed state, which can be used to adjust the control signal.

There are a number of assumptions within the model, including that the sensory feedback to the subject of the current state **x**<sup>t</sup> is error free and instantaneous (Shanechi and Carmena, 2013). The matrix **B** is tightly coupled to a particular task, making it difficult for the subject to use the control input **u**<sup>t</sup> to drive the neuroprosthetic if the task order changes during use (Matlack et al., 2014). Williams et al. (2013) points to the utility in including both a "hold" state and a "rest" state in the design of tasks for motor BMI decoders, and which is often not included as part of the model state. Finally, Hogri et al. (2015) illustrate a clear box modeling approach (LeDuc et al., 2011) where a simplified cerebellar microcircuit is implemented as a VLSI chip and interfaced to anesthetized rats.

#### Kalman Filter

The Kalman Filter (KF) is a recursive optimal estimator and is good at extracting signal from noisy measurements. It has been widely deployed in industrial automation and control systems engineering for over half a century. In its original form the state and measurement models needed to be linear. The extended Kalman Filter (EKF) models non-linear processes where a linearization has been performed, and implementations such as the Unscented Kalman Filter (Li et al., 2009) can utilize non-linear models. A number of variants have been proposed for neuroprosthetic closed-loop control, including SmoothBatch (Orsborn et al., 2012) and ReFIT (Gilja et al., 2012) which capture elements of the neuroprosthetic task in the model and updates the decoder parameters during the operation of the system. Updating the decoder in this manner is referred to as closed-loop decoder adaption (CLDA). In Dangi et al. (2014) Recursive Maximum Likelihood is used as part of CLDA to continuously adjust the KF. This is probably useful because the recording of the neural signal may be non-stationary due to factors (electrode drift, movement artifacts, external noise) as well as the fact that the subject may have changes in attention during the operation of the device, and the learning process may change the parameters for error and modeling (Chase et al., 2009). When examining various parameters that can be tuned in the decoder, (Cunningham et al., 2011) determined that bin width has a large impact on the performance of the KF, and should be optimized. Potentially due to the subjects ability to interact with the closed-loop system, shorter bin widths of 25–50 ms provide improved performance over longer bin widths.

#### Point Process Filters

The activity of individual neurons in the ensemble can be modeled as point processes with each spike being an event, which enables the filter to respond much more rapidly then methods that rely on binned spike counts or estimates of instantaneous firing rates (Li et al., 2009).

#### Reinforcement-Learning

In Actor-Critic architectures two coupled systems work together with complimentary models of the task. The two systems adapt using a Reinforcement Learning approach (DiGiovanna et al., 2009). The user of the system supplies a signal indicating success or failure to the Critic, which supplies a training signal to the Actor to allow adaption (Mahmoudi et al., 2008). By trial and error the Actor interacts with the environment, and the Critic's feedback rewards successful actions. The Actor-Critic approach may also be well-suited to neuroprosthetic control in a real world usage scenario where the task and associated trajectory varies from moment to moment, and achieving the goal may be the only reinforcement signal available (DiGiovanna et al., 2009). Mahmoudi et al. (2008) describes a neuroprosthetic for a Sprague Dawley rat with 32 electrodes implanted in primary motor cortex (M1)—symmetrically, 16 electrodes per hemisphere. This was used to control a robotic arm, which the rat used to press levers. Meanwhile the Critic component is implemented as a computer agent that adapts via the Reinforcement Learning paradigm based on the rewards the rat user receives, and the rat user is learning to modulate its neural activation modifying the directional tuning of the units in M1. An extension to this approach involves extracting the goal success signal directly from the subject. By recording from the Nucleus Accumbens in the ventral striatum of rats, an area believed to associate sensory perception with motor tasks, (Mahmoudi and Sanchez, 2011) were able to use the rat subject's internal representation of goal success as the evaluative feedback signal to the Critic component.

#### PID Feedback Control

The Proportional-Integral-Derivative (PID) controller is an extremely common and widely deployed controller in closedloop systems. The three terms—proportional, integral, and derivative are calculated from the plant's response to the input and are summed to generate the error signal (Åström and Wittenmark, 1990). Chaos control approaches, such as delayed feedback control, utilize the dynamics of the system to modify the control input. By taking advantage of the chaotic system sensitivity to perturbation, system state can be changed with minimal cost. In Vlachos et al. (2016) the use of delayed feedback control enables closed loop control of a seizure model (a spiking neural network) and the recovery of the non-seizure dynamics, while in Slutzky et al. (2003) seizure activity induced in rat hippocampal slice preparations was moderately controlled.

### Control Policies with Classifiers

Classifiers don't need a model of the system instead they attempt to determine a relationship between a set of measurements and a given state. Some classifiers can be sensitive to changes in the data they use to determine the classes (Lotte et al., 2007), such as Artificial Neural Networks whereas Linear Discriminant Analysis is more robust in the face of changes to data used to train the classifier.

#### Linear Discriminant Analysis

Linear discriminant analysis (LDA), and other related techniques, are statistical methods that find the features in the measurement of a signal that indicate the probability that it belongs to a given class (Mika et al., 1999). The assignment to a class can be used to trigger a neuroprosthetic intervention, such as the detection via EEG of a motor command and the subsequent activation of an ankle exoskeleton (Xu et al., 2014).

#### Artificial Neural Network

The Artificial Neural Network is a data driven approach to classification that in contrast to LDA and other statistical methods does not rely on the assumption of the underlying probability distribution of the system (Zhang, 2000). ANNs are organized in layers, with nodes or neurons connected typically in an input, hidden and output layer structure (**Figure 4**). There are numerous topologies, but among the most popular is the Multi Layer Perceptron (MLP), a three layer feedforward network. ANNs are trained with the presentation of input data that has been identified as belonging to an output class, and a learning rule is applied to adjust the weights on the connections between the nodes, of which back propagation is the most well-known.

ANNs have been used to predict end-point gait parameters from the EMG recorded from the neuromuscular activation of subjects with Spina Bifida (SB; Chang et al., 2009), and to achieve realtime dexterous control of a myoelectric prosthetic hand from cortical recordings of rhesus monkeys (Aggarwal et al., 2008). Echostate Neural Networks are a Recurrent Neural Network (Sussillo et al., 2012) that have been used in non-human primates for a motor BMI, and have been able to outperform the Kalman Filter.

#### Support Vector Machines

The Support Vector Machine (SVM) is a supervised machine learning approach that can perform classification or regression. The SVM identifies a hyperplane that separates classes within the data, by non-linearly projecting data points into a higher dimensional space (Tapson et al., 2013).

#### Linear Solutions to Higher Dimensional Interlayers

Linear Solutions of Higher Dimensional Interlayers (LSHDI) are a class of networks that have some similarity in architecture to ANNs, with an input, hidden, and output layer. They are distinct due to the much larger hidden layer, the random generation of the weights on between the input and the hidden layer, and the linear response of the output layer (Tapson et al., 2013). The Neural Engineering Framework (NEF) builds systems out of networks that have LSHDI characteristics (Eliasmith and Anderson, 2004). The NEF has been used to design Spiking Neural Networks that implements the Kalman Filter as part of a cortical motor BMI (Dethier et al., 2011). The Synaptic Kernel Inverse Method (SKIM) is an LSHDI network for spiking input (Tapson et al., 2013) that can perform both classification and regression.

#### FEEDBACK

There can be more than one feedback loop in the neuroprosthetic system (Broccard et al., 2014). Feedback can be the visual observation of the robotic effector as it is in many motor BMIs, allowing the operator to modulate their neural activity before it is decoded. Or in the case of a DBS neuroprosthetic the feedback signal may be acquired from recording electrodes implanted alongside the stimulating electrodes, in which case the feedback signal is returned directly to the device (Herron and Chizeck, 2014). Bidirectional interfaces, via the PNS or the CNS, enable the transformed signals of sensors on the robotic effector to be transmitted into the operators nervous system and interpreted as a sensory percept (Armiger et al., 2013). Feedback can also be from sensors on the robotic effector directly to the controller, bypassing the operator (Broccard et al., 2014; Markovic et al., 2014). And finally, reversing the direction of information, natural sensors can be used as a source of feedback for the controller (Holinski et al., 2013; Mendez et al., 2013; Wright et al., 2015) and the brain can be used as the source of the error signal (Mahmoudi and Sanchez, 2011).

#### DISCUSSION

We have reviewed the control policies employed by recent neuroprosthetic systems. For the purposes of this review we included motor BMIs, assistive devices, neuromodulatory systems, and other devices with an interface into a subject's nervous system. Many of the devices reported on in the literature are being developed in non-human primate or other animal models, only a subset have been tested in human subjects. There is a frequent conflation of detection and classification approaches with digital or Bang-Bang control within the literature.

Although closed-loop systems have been demonstrated experimentally there remain significant limits on our ability to describe the activity in the brain, and consequently develop control policies to respond to that activity. Simulation of cortical activity (Ehrens et al., 2015; Sandler et al., 2015; Vlachos et al., 2016), the use of experimental platforms (Keren and Marom, 2014), and the use of animal models has enabled the development of a wide range of neuroprosthetic systems. However, the appropriate method to transition these systems in human subjects is not clear. Among the difficulties is the body's response to chronic implantation of microelectrodes (Fernandez et al., 2014), the appropriate place to collect a signal (Krook-Magnuson et al., 2015), and the possibility that longterm attempts to control a cortical system may compromise some of the desirable behavior (Keren and Marom, 2014).

Comparisons of control policies across different modalities, interfaces and levels of invasiveness are difficult. Even within a given neuroprosthetic category it can be difficult to perform a comparison due to the wide variety of task designs, different subject training regimes and varying reported metrics; (Koyama et al., 2010; Sussillo et al., 2012) are rare exceptions. Examination of **Table 1** reveals that the motor BMI discipline appears to have explored the widest variety of control policies, and that the use of the Kalman Filter as part of a closed-loop system has broad support. Improvements to the traditional Kalman Filter to allow non-linear models of neural activity combined with its ability to

#### REFERENCES


be implemented in real-time continue to make it an attractive approach.

Closed-loop motor BMI systems have had significant success with closed-loop decoder adaption (CLDA; Shanechi and Carmena, 2013), supporting the use of closed-loop control. However, the CLDA approach has identified two distinct strategies in motor neuroprosthetics—decoding vs. learning. The decoding approach aims to read the natural motor plan whereas the learning approach monitors the changing neural activity as the brain learns to operate the prosthetic. It is not known at this stage if a similar duality of strategies will be applicable in other modalities. The difficulty in specifying a model for use in many of the control policies previously describes arises from our continued uncertainty about specific action of many of the components of the nervous system. It has been observed that users of Cochlear Implants have improved speech recognition performance after completing training with the device (Doucet et al., 2006), which may argue for a learning interpretation. An important caveat for the learning approach is that the neuroprosthetic system must be stable as regards the interface and transform of the input signal to the effector output, to allow the subject the opportunity to develop the "prosthetic motor memory" necessary for skillful operation (Carmena, 2013).

Neuroprosthetic development of closed-loop systems has been driven in part by the concern that the risks involved in highly invasive interfaces need to be mitigated by a strong case for the therapeutic benefit. Patient abandonment for upper limb prosthetics is high, with many wearers ceasing upper limb prosthetic use within 12 months of receipt of the device. Concerns cited by users are weight, appearance and difficulty of use (Biddiss and Chau, 2007). Extrapolating to more invasive systems, it may be difficult to argue the cost benefit if patient dissatisfaction is very high. Although these devices cannot be abandoned in the same manner as a detachable prosthetic limb, there is some suggestive research indicating an unwillingness to participate in experimental trials, which may suggest that the perceived benefit of neuroprosthetic systems by the target patient populations remains low (Illes et al., 2011). By improving control we can offer improved functionality and increased therapeutic benefit.

### AUTHOR CONTRIBUTIONS

JW performed the review and drafted the manuscript. JT, AS, and VM revised the manuscript.

International IEEE/EMBS Conference on Neural Engineering (NER) (Cancún), 388–391. doi: 10.1109/NER.2011.5910568


neural response is modulated by limb biomechanics. Neuron 58, 414–428. doi: 10.1016/j.neuron.2008.02.033


spiking neural network control algorithm. Adv. Neural Inf. Process. Syst. 2011, 2213–2221.


Houk, J. C. (1988). Control strategies in physiological systems. FASEB J. 2, 97–107.


limbs: review and discussion. Biomed. Eng. Online 11:33. doi: 10.1186/1475-92 5X-11-33


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2016 Wright, Macefield, van Schaik and Tapson. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

## APPENDIX: LITERATURE SEARCH METHODOLOGY

To generate a list of papers for the review we performed a search on Scopus using the following search criteria, restricting the results to articles or conference proceedings:

Topic = (neuroprosthetic OR neurorobotic OR exoskeleton OR neurostimulator OR (brain AND interface)) AND Topic = (control OR controller AND (scheme OR algorithm OR strategy)) AND Topic = (closed-loop OR closedloop). The results were limited to Articles and Conference Proceedings from 1990 to 2015.

With the above search we obtained 147 papers from Scopus. We excluded papers that were from unrelated fields, as well as papers that described the design or fabrication of an exoskeleton without reference to a control system, or that described a control strategy used by an exoskeleton or rehabilitative device without a nervous system interface. We excluded papers that discussed computing architectures for experimental design, or detailed the fabrication of electronics. Once duplicates and excluded papers were removed we were left with 85 articles.

The papers were divided into five categories based on their primary topics.

### Control strategy

Papers mainly describing the design of an algorithm. Papers comparing the performance of algorithms by a particular neuroprosthetic. Papers describing a change to the control loop such as the addition of feedback or the use of additional signal processing.

### Neuroprosthetic design

Papers describing a neuroprosthetic system, including interface site, electrode fabrication, sensors, architecture and algorithms.

### Device testing

Papers that describe the performance of an experimental neuroprosthetic. Papers that describe an experimental neuroprosthetic in a non-human test subject. Papers that compare the performance of a neuroprosthetic in a patient with an existing clinical therapy.

### Simulators

Papers that describe a simulation environment for development or testing of neuroprosthetics, or papers that describe a model of the nervous system for a neuroprosthetic device or control algorithm to interact with.

### Reviews

Papers that are reviews of neuroprostheses, algorithms or interfaces.

# Control of an Ambulatory Exoskeleton with a Brain–Machine Interface for Spinal Cord Injury Gait Rehabilitation

Eduardo López-Larraz 1, 2 \*, Fernando Trincado-Alonso<sup>3</sup> , Vijaykumar Rajasekaran<sup>4</sup> , Soraya Pérez-Nombela<sup>3</sup> , Antonio J. del-Ama<sup>3</sup> , Joan Aranda<sup>4</sup> , Javier Minguez 1, 2, 5 , Angel Gil-Agudo<sup>3</sup> and Luis Montesano1, 2

<sup>1</sup> Departamento de Informática e Ingeniería de Sistemas, University of Zaragoza, Zaragoza, Spain, <sup>2</sup> Instituto de Investigación en Ingeniería de Aragón (I3A), Zaragoza, Spain, <sup>3</sup> Biomechanics and Technical Aids Unit, National Hospital for Spinal Cord Injury, Toledo, Spain, <sup>4</sup> Institute for Bioengineering of Catalunya, Universitat Politécnica de Catalunya, Barcelona, Spain, <sup>5</sup> Bit & Brain Technologies, Zaragoza, Spain

#### Edited by:

Timothée Levi, University of Bordeaux 1, France

#### Reviewed by:

Jose Luis Contreras-Vidal, University of Houston, USA Ren Xu, University of Göttingen, Germany

> \*Correspondence: Eduardo López-Larraz edulop@unizar.es

#### Specialty section:

This article was submitted to Neuroprosthetics, a section of the journal Frontiers in Neuroscience

Received: 29 January 2016 Accepted: 19 July 2016 Published: 03 August 2016

#### Citation:

López-Larraz E, Trincado-Alonso F, Rajasekaran V, Pérez-Nombela S, del-Ama AJ, Aranda J, Minguez J, Gil-Agudo A and Montesano L (2016) Control of an Ambulatory Exoskeleton with a Brain–Machine Interface for Spinal Cord Injury Gait Rehabilitation. Front. Neurosci. 10:359. doi: 10.3389/fnins.2016.00359 The closed-loop control of rehabilitative technologies by neural commands has shown a great potential to improve motor recovery in patients suffering from paralysis. Brain–machine interfaces (BMI) can be used as a natural control method for such technologies. BMI provides a continuous association between the brain activity and peripheral stimulation, with the potential to induce plastic changes in the nervous system. Paraplegic patients, and especially the ones with incomplete injuries, constitute a potential target population to be rehabilitated with brain-controlled robotic systems, as they may improve their gait function after the reinforcement of their spared intact neural pathways. This paper proposes a closed-loop BMI system to control an ambulatory exoskeleton—without any weight or balance support—for gait rehabilitation of incomplete spinal cord injury (SCI) patients. The integrated system was validated with three healthy subjects, and its viability in a clinical scenario was tested with four SCI patients. Using a cue-guided paradigm, the electroencephalographic signals of the subjects were used to decode their gait intention and to trigger the movements of the exoskeleton. We designed a protocol with a special emphasis on safety, as patients with poor balance were required to stand and walk. We continuously monitored their fatigue and exertion level, and conducted usability and user-satisfaction tests after the experiments. The results show that, for the three healthy subjects, 84.44 ± 14.56% of the trials were correctly decoded. Three out of four patients performed at least one successful BMI session, with an average performance of 77.6 1 ± 14.72%. The shared control strategy implemented (i.e., the exoskeleton could only move during specific periods of time) was effective in preventing unexpected movements during periods in which patients were asked to relax. On average, 55.22 ± 16.69% and 40.45 ± 16.98% of the trials (for healthy subjects and patients, respectively) would have suffered from unexpected activations (i.e., false positives) without the proposed control strategy. All the patients showed low exertion and fatigue levels during the performance of the experiments. This paper constitutes a proof-of-concept study to validate the feasibility of a BMI to control an ambulatory exoskeleton by patients with incomplete paraplegia (i.e., patients with good prognosis for gait rehabilitation).

Keywords: spinal cord injury (SCI), brain machine interfaces (BMI), ambulatory exoskeletons, gait rehabilitation, movement intention decoding, electroencephalography (EEG), event related desynchronization (ERD), movement related cortical potentials (MRCP)

#### 1. INTRODUCTION

Recovery of lower-limb function in spinal cord injury (SCI) patients is crucial to enhance the independence and quality of life in this population (Ditunno et al., 2008). Two-third of SCI patients are reported as paraplegic (Wyndaele and Wyndaele, 2006), from which a considerable percentage is able to regain certain locomotion function, especially those with low and incomplete lesions (Nene et al., 1996; Scivoletto et al., 2014). Technological advances, such as robotic exoskeletons, have emerged as a valuable option to rehabilitate and restore gait in paraplegic patients beyond traditional means such as crutches, walkers, and orthoses. These robotic systems can range from clinical devices to bioinspired wearable ones. Firstly, weightsuspended robotic systems with a treadmill, such as the Lokomat (Hocoma Medical Engineering Inc, Zurich, Switzerland) and the Gait Trainer (GT II, Rehastim, Berlin, Germany), have demonstrated their rehabilitative efficacy, but they are generally expensive and cannot be used for motor substitution (Wirz et al., 2005). Secondly, robotic walking devices with balance control, such as the REX (REX Bionics Ltd), can be used by people with high SCI (up to C4/5 level), as they completely substitute their gait function, but it may be too bulky and inefficient for those patients who maintain certain balance control. Hence, these systems can be more appropriate for assistive purposes only. Thirdly, ambulatory exoskeletons, such as the ReWalk (Bionics Research Inc) and the H2 (Technaid S.L., Spain), are designed to assist leg movement, but they can only be used by patients with lower-limb weakness who still can maintain balance. These systems are of particular interest, as they can be controlled using assist-as-needed paradigms, which may be more effective than other approaches for rehabilitation and functional compensation of patients with paraplegia and also for stroke sufferers (Pons and Torricelli, 2014).

In this context, there is a growing interest toward the development of robotic devices controlled by brain–machine interfaces (BMI) to assist and rehabilitate gait function (Pfurtscheller et al., 2006; Fitzsimmons et al., 2009; Alam et al., 2014). The contingent link between neural commands and the peripheral feedback given by means of a rehabilitation device can promote neuroplasticity (Mrachacz-Kersting et al., 2012). Whereas, a BMI constitutes a natural interface that provides an easier and more intuitive control of assistive devices (Millán et al., 2010). Non-invasive technologies such, as the electroencephalogram (EEG), constitute a relatively cheap and portable option to build these BMI systems (Wolpaw et al., 2002). The use of brain-triggered rehabilitative technologies is of special relevance for incomplete SCI patients. These individuals can maintain some intact fibers below the injury level, and recent studies have shown how these spared pathways can be reinforced by the continuous association between the activation of the brain during the intention of movement and the stimulation of the paralyzed limbs (Jackson and Zimmermann, 2012).

The closed-loop control of walking exoskeletons using neural commands presents two main challenges. The first one corresponds to the development of robust and reliable BMIs to decode neural signals associated with gait movement intention. In contrast to upper-limb, which is generally the focus of BMI research for motor rehabilitation/restoration of paralyzed patients (Lebedev and Nicolelis, 2006; Millán et al., 2010), decoding of gait has not been so deeply studied. The recording of neural signals during walking might be affected by motion artifacts, which could bias the decoding and lead to misinterpretation of the neural dynamics associated with the movement (Castermans et al., 2014), although there is evidence showing that the influence of these artifacts can be reduced by using carefully designed set-ups (Nathan and Contreras-Vidal, 2016). Nonetheless, recent studies with healthy subjects have shown that EEG neural correlates can be used to decode the gait initiation before it occurs (Jiang et al., 2015; Sburlea et al., 2015) and to distinguish between different walking directions (Velu and de Sa, 2013). However, pathologies like SCI entail a brain reorganization, which may complicate the decoding of motor information (López-Larraz et al., 2015a). Hence, it is important to validate how BMI systems can be applied effectively in these patients. The EEG signals of a paralyzed patient during his/her attempt to move the legs could be decoded without any overt movement and used to trigger the movement of an exoskeleton or prosthesis that assists his/her walking. The current state of the art in non-invasive BMI technology does not allow for precise decoding of fine limb kinematics. Therefore, an accepted approach in the literature is to have a shared control paradigm in which the brain activity is used to trigger the movement of a robot/prosthesis that can autonomously perform a functional task (e.g., walk forward two steps; Millán et al., 2010; Rohm et al., 2013).

The second challenge arises from the complexity of the set-up required to control a device for gait assistance with neural signals. In the recent years, pilot studies have shown how BMIs have been used to control weight-suspended robotic and prosthetic systems (Do et al., 2013; King et al., 2015). Furthermore, robotic exoskeletons with balance control have also been controlled using brain signals (Kilicarslan et al., 2013; Kwak et al., 2015). All these studies are performed with devices that support balance, which minimizes fall risks, and three of them demonstrated successful control with SCI patients (Do et al., 2013; Kilicarslan et al., 2013; King et al., 2015). However, the control of ambulatory exoskeletons with a BMI presents additional issues compared to those systems with balance support. Even if it is used by patients with a relatively good condition (e.g., legs weakness and/or certain degree of balance control), they are required to maintain the balance by holding on to a walker or to parallel bars, and to focus on the intention of motion to command the BMI. This kind of set-up would permit the development of assist-asneeded rehabilitative interventions for such patients, which may lead to higher motor improvements (Cai et al., 2006). Hence, the validation of a BMI to control an ambulatory exoskeleton requires the design of a protocol with special considerations, such as safety, timings and control of patients' fatigue levels during the experiments.

This paper presents an integrated system for the closedloop control of an ambulatory exoskeleton with a BMI. The exoskeleton works under an assist-as-needed control paradigm, which can be adapted to the capabilities of each patient and assist him/her only to the extent he/she needs. The EEG neural correlates of movement are used to decode the intention of gait initiation, which is used as a volitional control signal for the exoskeleton movement. The feasibility of the proposed system is validated with two sets of experiments. The first experiment shows the viability of the whole set-up with three healthy subjects. The second experiment demonstrates the viability of the system in a realistic clinical environment, involving four incomplete SCI patients. An experimental protocol is proposed to operate the BMI in an ecological set-up, with an emphasis on patients' safety. Decoding performance, exertion levels, and satisfaction and usability scores were measured as indicators of the viability of the system for clinical applications.

### 2. MATERIALS AND METHODS

#### 2.1. Participants

Three able-bodied subjects and four SCI patients participated in the study. Demographic data of both the healthy subjects (H) and the patients (P) can be seen in **Table 1**. The SCI patients were hospitalized at the Hospital Nacional de Parapléjicos, in Toledo (Spain), where all the experimentation sessions took place. The inclusion criteria for the patients were: (1) SCI with any lesion level, ASIA C or D with gait prognosis; (2) patients in walking rehabilitation; (3) patient's balance allows standing between parallel bars; (4) no orthostatic complications during standing; (5) upper-limb strength to manage a walker or crutches, and to transfer from the wheelchair to a chair; (6) age between 18 and 60 years; and (7) height 1.50–1.95 m and weight up to 90 kg. The exclusion criteria were: (1) inability to stand in upright position for at least 15 min; (2) any surgery in the previous 3 months; (3) spasticity higher than 3 in the Modified Ashworth Scale (Bohannon and Smith, 1987) in any of the lowerlimb muscles; (4) previous/current lower-limb bone fracture; (5) ulcers or sores in areas of contact with the exoskeleton and/or electrodes; (6) previous/current history of cardiovascular disease of any kind or exercise contraindications; (7) upperlimb pain that limits weight bearing on crutches/walker/parallel bars; (8) significant upper/lower extremity discrepancies; (9) uncontrolled autonomic dysreflexia; (10) pregnancy; and (11) cognitive impairment of any kind. The selected patients met all inclusion and no exclusion criteria. All the subjects were duly informed about the study, and all of them gave written consent before the first session. The experimental procedure was approved by the Ethics Committee of the Hospital Complex of Toledo (Spain) (C.E.I.C. 31/02-2014).

### 2.2. Clinical Assessment

We evaluated the clinical condition of the SCI patients before their enrollment in the study. Their injury severity, lower extremity strength, and mobility were measured using a set of clinical tests, according to the standardized ASIA clinical exams (Marino et al., 2003). The lower extremity motor score (LEMS) was used to measure muscle strength, with 5 key muscles examined in each leg: hip flexors, knee extensors, ankle dorsiflexors, long toe extensors, and ankle plantar flexors. The grading system for the muscle strength goes from 0 to 5 (0 = absence of muscle contraction, 5 = normal active movement with full range of motion against full resistance). The cumulative score for the lower extremities ranges between 0 and 50. Modified Ashworth score was used for lower-limb spasticity measurement, ranging from 0 (no spasticity) to 4 (affected part rigid in flexion or extension). Only patient P4 presented a very slight spasticity (score 1) in the right ankle, below the level established in the exclusion criteria. Walking index for spinal cord injury (WISCI II) was used to quantify the degree of assistance required by the patient during normal walking and 10 Meter Walk Test (10 MWT) to assess walking speed (Ditunno and Ditunno, 2001; van Hedel et al., 2005). WISCII II grading system ranges from 0 (patient is unable to stand and/or participate in assisted walking) to 20 (ambulates with no devices, no braces and no physical assistance). According to the recommendations of 10 MWT, walking speed was calculated discarding the 2 initial and the 2 final meters, to only consider walking at a constant speed. Distance (6 m) was divided by the time measured to obtain gait speed (m/s). The values for each patient can be found in **Table 2**. Walking tests were performed using as little assistance as possible to ensure patient safety.

#### 2.3. Experimental protocol

The present study was divided into two stages. The first stage aimed at validating the technology under a well-controlled scenario. This was done by performing experiments with healthy subjects and evaluating if the BMI could be effectively used to close the loop and control the ambulatory exoskeleton. The second stage sought to demonstrate that the proposed system and protocol could be safely used in a clinical environment. Experiments with SCI patients were conducted, in which the key point was to measure parameters such as exertion and fatigue levels, as well as usability and satisfaction scales.

The experimental protocol consisted of familiarization sessions and BMI sessions. The experiments with the healthy subjects included the familiarization and the BMI sessions in


TABLE 1 | Demographic information of both groups: healthy subjects and SCI patients.

TABLE 2 | Clinical scores obtained by the patients before the experiments.


1 day. The experiments with the SCI patients required one familiarization session and two BMI sessions in 3 separate days. The set-up included: the EEG equipment (only for the BMI sessions) with the amplifiers in a backpack carried by the subject, the exoskeleton attached to the subject's legs, and a walking aid to help keeping balance (**Figure 1**). In addition, a trolley table was used to carry the computers that processed the EEG signals and controlled the exoskeleton. Crutches, a walker, and parallel bars were tested as walking aids. Crutches did not provide enough balance control for the patients and were discarded. The walker, which is commonly used in gait rehabilitation by these patients, worked well with the healthy subjects. However, during some preliminary tests with the patients, we realized that they had difficulties to move it while walking with the exoskeleton. Therefore, all the SCI patients performed the BMI sessions using parallel bars, whereas the healthy subjects used the walker. The exoskeleton joints remained blocked whenever it was not in movement in order to partially support patients' weight.

The familiarization sessions allowed the subjects to get used to the protocol timings and the exoskeleton movements. On these sessions, one experimenter triggered the movements of the exoskeleton manually, warning the subject before every movement. For the healthy subjects, these sessions consisted of 5–10 min walking with the exoskeleton. For the patients, the sessions took between 20 and 30 min, in which 2 clinicians monitored every movement and informed the patient about the protocol and how to interact with the exoskeleton. If required, these sessions were repeated until both the patient and the clinicians confirmed that the patient was accustomed to the system, and ready for the first BMI session.

The BMI sessions consisted of screening blocks and closedloop feedback blocks. Given the nature of the set-up, a cue-guided BMI was proposed, in which the EEG signals were classified asynchronously. Hence, the exoskeleton moved as soon as the intention of movement was decoded, but only during specific periods of time, avoiding sudden and unexpected movements that may result in patients' falls. The participants performed 3 or 4 screening blocks of 20 trials each, which were used to calibrate the BMI decoder. During this screening phase, the participants were standing, wearing the exoskeleton, and holding the corresponding walking aid (i.e., the walker for the healthy subjects, and the parallel bars for the patients). Neither the healthy subjects nor the patients could actually move the legs during the screening blocks (as the exoskeleton joints were blocked). Therefore, in both cases, we consider the action performed as a movement attempt and not as a movement execution. The screening blocks were composed of rest and movement attempt (MA) intervals. The rest intervals had a random duration between 4 and 7 s. An audio cue indicated the start of the MA interval, which lasted 3 s. The participants were instructed to attempt to move their right leg, as if they started walking, immediately after they heard the audio cue. The rest of the time, they were asked to stay relaxed and move as little as possible. During the MA interval, participants were explicitly asked to avoid compensatory movements with the rest of the body, especially with the hip, and to attempt to move their right leg only. The closed-loop feedback blocks were composed of trials with four intervals: (i) "Rest," (ii) "Preparation," (iii) "Movement Attempt," and (iv) "Movement." The experiments with healthy subjects included 3 blocks of 20 trials each (amounting to 60 trials), in order to acquire enough movements to have a good estimation of the performance of the BMI system. In the experiments with the SCI patients, there was a variable number of trials, and they were asked to reach a distance of 10 m (i.e., the length of the parallel bars), which corresponds to around 20–25 gait cycles. During the "Rest" state (5 s), the subjects were not required to perform any task, but just to relax after the previous trial. After that, a low tone was played, which marked the beginning of the "Preparation" interval (3 s), during which they were instructed to relax and be prepared for the upcoming cue. A high tone denoted the start of the "Movement Attempt" interval (maximum 3 s), in which they were asked to attempt to move their right leg in the same way

they had done in the screening blocks. If the BMI detected the intention to move at any time during these 3 s, the system started the "Movement" interval, in which the exoskeleton controller unblocked the joints and moved for one gait cycle: one step with right leg and one with left leg (6 s). Otherwise, after the 3 s, a new trial started in rest state. Supplementary Video 1 shows some trials of subject P1 triggering the exoskeleton with his motor intention.

For safety reasons, every trial required that the experimenter explicitly pressed an activation button during the "Rest" or "Preparation" intervals. If that button was pressed, the BMI decoder started sending its outputs to the exoskeleton controller. This would trigger the start of the gait cycle if the patient attempted to move during the "Movement Attempt" interval. If the button was not pressed, the exoskeleton did not move even if the participant attempted to move his/her leg. This mechanism was included in order to avoid starting a movement with the patient being in an unsafe position after the previous gait cycle, and to skip trials to regularly ask the patients about their fatigue levels. When required, the patients could sit for a few minutes to rest, and the trials continued when they confirmed that they were ready.

Due to the complexity of the set-up, the therapist–patient interaction was integral for the correct execution of the BMI experiments. Apart from being the control signal for the exoskeleton, the BMI was used by the therapist to guide the patient during the executions (Pichiorri et al., 2011). A therapist interface was designed to show the experimenter information of the BMI decoder output and the patient's EEG activity in real-time. The experimenter could, for instance, ask the patient to relax if the BMI was detecting movement commands during periods in which the patient should rest (e.g., due to excessive movements required to keep balance) or ask the patient to concentrate further when the BMI was not decoding any movement when they were required. In addition, an option to send manual triggers was included in the therapist interface in order to manually start exoskeleton movements, and so, reduce frustration when the BMI repeatedly failed to decode the movements.

### 2.4. EEG Acquisition

The EEG was recorded using a commercial g.Tec system (g.Tec GmbH, Graz, Austria), with 32 channels placed at AFz, FC3, FCz, FC4, C5, C3, C1, Cz, C2, C4, C6, CP3, CP1, CPz, CP2, CP4, FP1, FP2, F7, F3, Fz, F4, F8, T7, T8, P7, P3, Pz, P4, P8, O1, and O2 (according to the international 10/10 system). The ground and reference electrodes were placed on FPz and on the left earlobe, respectively. The EEG was digitized at a sampling frequency of 256 Hz and power-line notch-filtered to remove the 50 Hz line interference. The amplifiers connected to the EEG cap were carried in a backpack by the subject. The amplifiers were connected via long cables to a laptop placed on the trolley table.

#### 2.5. Exoskeleton

The exoskeleton used to assist gait was a 6 degrees of freedom wearable lower-limb orthosis with anthropomorphic configuration (Bortole et al., 2015). It included three joints for each leg: hip, knee, and ankle, each of which was powered by a DC motor coupled with a harmonic drive gear. The exoskeleton was equipped with potentiometers and strain gauges to measure the joint angles and the human–robot interaction torques. Its control was conceived to work under an assist-as-needed paradigm in order to make rehabilitation more challenging for the patients. A predefined trajectory, obtained from healthy subjects, was used as the desired gait pattern. The controller updated the stiffness values in real time according to the subject's performance in order to assist him/her just to the extent he/she needed (Rajasekaran et al., 2015). The exoskeleton was connected with long cables to its controller and to the power supply, which were on the trolley table.

### 2.6. EEG-Based Movement Intention Decoder

After recording the screening blocks and before the closedloop blocks, the BMI classifier was trained to distinguish between rest and movement attempt (MA) classes. The BMI decoder was based on the one proposed in López-Larraz et al. (2014). The decoding of movement attempt was dependent on the combination of two EEG movement correlates: the event-related desynchronization (ERD) of sensorimotor rhythms (Pfurtscheller and Lopes da Silva, 1999) and the movementrelated cortical potentials (MRCP) (Shibasaki and Hallett, 2006). Signals from the screening datasets were trimmed down to 7-s trials (−4 to 3 s from the MA audio cue).

#### 2.6.1. Artifact Removal

Before training, an automatized procedure based on z-scores was applied to eliminate the trials containing artifacts (López-Larraz et al., 2014). For each trial, the power in delta (1–4 Hz), theta (4– 8 Hz), alpha (8–12 Hz), and beta (12–40 Hz) frequency bands, as well as the trial variance and the maximum amplitude were computed. Trials that went over a threshold set at 2.5 standard deviations of the mean in any of these parameters were discarded. Statistical methods like this are especially useful for clinical setups as the one presented here, since they do not require human supervision and can be used quickly to eliminate the artifacts before training the BMI (Nolan et al., 2010; Maeder et al., 2012). In principle, this method should be able to remove the most common types of artifacts that can be found in a set-up like this. For instance, analyzing the power in delta band and the signal amplitude may serve to remove low-frequency motion artifacts, whereas analyzing the power beta and signal variance could help to get rid of trials contaminated with EMG artifacts.

For the closed-loop blocks, we considered two options: (1) having an online system to detect EEG artifacts and stopping the BMI every time that it detected one; or (2) considering that the possible artifacts would not highly influence the performance of the system. Given that we trained the BMI with clean trials, the existence of artifacts during test should not deceitfully increase the performance, and at most, they would decrease it. Since we considered that this is not worse than stopping the BMI every time that an online artifact detector detects an artifact, no artifacting was performed during the closed-loop trials.

#### 2.6.2. Feature Extraction and BMI Training

Features were computed from the clean trials using a 1-s long window with a sliding step of 250 ms. Features corresponding to the rest and MA classes were computed on the [−3, −1] s and [0, 2] s intervals, respectively (with 0 being the time when the high tone indicating the start of the MA interval was played). On these windows, the features were computed as follows:


For each time window, 1192 features were extracted. Sparse discriminant analysis (SDA) was used to select the 30 most discriminant non-redundant features and as a linear classifier (Clemmensen et al., 2011; López-Larraz et al., 2014).

#### 2.6.3. Closed-Loop BMI

During the closed-loop blocks, the EEG was processed continuously. A sliding window was computed every 62.5 ms and its features were extracted following the same procedure detailed in Section 2.6.2, and the classifier trained with the screening blocks generated a new BMI output. For each sliding window, the BMI classifier determined if the signal corresponded to rest or to MA classes. In order to ensure a consistent brain activation, the BMI generated a movement trigger when five consecutive windows of MA class were detected (Ramos-Murguialday et al., 2013). If the BMI was active (i.e., if the experimenter pressed the activation button during the "Rest" or "Preparation" intervals of a trial, see Section 2.3), the movement trigger was sent to the exoskeleton controller; otherwise, the trigger was not sent. The controller ignored those triggers that arrived during the "Rest" and "Preparation" intervals to avoid starting an unexpected movement, which could make the patients fall. Therefore, on each feedback trial, the exoskeleton moved if the experimenter activated the BMI and the BMI generated a trigger during the "Movement Attempt" interval. In addition, the exoskeleton could also be moved if the experimenter sent a manual trigger during the "Movement Attempt" interval.

### 2.7. Exertion and Satisfaction Assessments

In order to evaluate the feasibility of the system for clinical applications, the patients were assessed with exertion and satisfaction scales.

The exertion level was assessed three times on each BMI session: before starting (i.e., when the patient was still sitting on the wheelchair), after the screening blocks, and after the closedloop blocks. The Borg scale was used with values ranging from 6 ("very, very light") to 20 ("very, very hard") (Borg, 1970).

After the last BMI session, the patients were asked to evaluate how satisfied they were with the system (i.e., the complete setup, including the exoskeleton and the EEG system) by using a modified version of the QUEST (Quebec user evaluation of satisfaction with assistive technology) scale (Demers et al., 2002).

## 3. RESULTS

### 3.1. Movement Attempt EEG Correlates

The features used by the BMI to decode the attempts of movement were based on two well-studied EEG correlates: the event-related desynchronization (ERD) of sensorimotor rhythms and the movement-related cortical potentials (MRCP). **Figure 2** shows a summary of these correlates computed using the signals recorded in the screening blocks after removing artifactual trials (see Section 2.6.1). For the SCI patients, the screenings from both BMI sessions were combined. Following the methodology proposed in López-Larraz et al. (2014), we used optimal spatial filters (OSF) to visualize the ERD and the MRCP activities by combining the electrodes placed over the motor cortex. Activity recorded on electrodes FC3, FCz, FC4, C3, C1, Cz, C2, C4, CP3, CP1, CPz, CP2, and CP4 was combined with an optimization

algorithm, which computed the coefficients that maximized the signal-to-noise ratio of both ERD and MRCPs (Graimann and Pfurtscheller, 2006; Niazi et al., 2011).

Two out of three healthy subjects (H1 and H2) showed strong ERD and MRCP activations, where as H3 showed weak activations of both correlates. For the SCI patients, the ERD patterns of P4 were similar to the ones of H1 and H2; P1 and P2 showed smoother ERD and only in the beta band; and P3 showed no ERD at all. Regarding the MRCPs, P1 and P4 showed similar morphology to H1 and H2, whereas P2 and P3 presented more noisy activity.

#### 3.2. BMI Performance

On average, 14.58% of the trials were rejected before training the BMI decoder (15 ± 1.25% for the healthy subjects and 14.43 ± 2.62% for the patients). **Figure 3** shows one representative trial for one healthy subject (H1) and two patients (P2 and P3). For each subject, the following information is shown: data of 3 EEG channels (i.e., C3, Cz, and C4), the decoder output, the triggers generated (i.e., the BMI triggers and the manual triggers), the system states during the trial, the angle of both knees, and the interaction torques measured by the strain gauges of both knees. As can be seen in the left plot, a BMI trigger generated out of the "Movement Attempt" state does not start any movement of the exoskeleton. When it is generated in the appropriate state, the "Movement" period starts, first with the right leg and then followed by the left leg. In the right plot, the BMI did not detect any movement. Instead, the experimenter sent a manual trigger to start the "Movement" phase. The movement of the exoskeleton seems to cause relatively large motion artifacts in the EEG, especially in the patients (center and right plots).

Decoding results for the healthy subjects are presented in **Table 3**, and for the SCI patients in **Table 4**. Each of the healthy subjects performed 60 trials (3 blocks of 20 trials each). On average, 84.44% of the trials were correctly decoded, generating a walking movement with the exoskeleton. For the correctly decoded trials of the three subjects, the average time between the auditive cue and the beginning of the exoskeleton movement was 1.07 ± 0.63 s.

Given the complexity of the set-up and the unfamiliarity of the patients with the technology, they were asked to attend two BMI sessions. In the first session, which can be considered as a BMI-familiarization session, they were carefully informed about how the BMI system works. Then, they performed the screening blocks and a few closed-loop trials to familiarize with the whole system and protocol. On the second session, patients also performed screening blocks, and subsequently, they started with the closed-loop blocks until reaching a distance of 10 m.

Patient P1 was the only one who performed two successful BMI sessions (i.e., reaching the 10 m distance). In the first one, he achieved a high performance (84%), whereas in the second one, it dropped to 55.56%. For patients P2 and P4, the first session was prematurely interrupted, but both of them performed a successful second session, with more than 85% of decoded trials. For patient P3, performances were low in both sessions, especially in the second one, in which the experimenter had to repetitively use the manual trigger due to the lack of movement attempt commands decoded by the BMI. Notice that the decoding performances

three EEG channels located over the motor cortex: C3, Cz, and C4. The decoder output indicates the classifier label (Rest or MA) provided for each 1-s window in real time. The BMI triggers were generated only when five consecutive classifier outputs corresponded to MA class, after being in Rest class (they were ignored when generated out of the "Mov." Attempt period–see left plot). The manual triggers correspond to the movements generated by an explicit command sent by the experimenter (see right plot). The states of the system during a normal trial were: "Rest," "Preparation," "Movement Attempt," and "Movement." The right and left knee angles (in a normalized scale) indicate the flexion of the knee joints of the exoskeleton. The right and left knee interaction torques (in a normalized scale) measure the forces performed by the subjects in the strain gauges located on the knee joint of the robot. The vertical lines indicate the change of state of the system. The red, green and blue lines correspond to the beginning of the "Preparation," "Movement Attempt," and "Movement" phases, respectively.

presented in **Table 4** are computed as the ratio between the number of trials that the BMI correctly decoded and the total number of trials in which the experimenter did not activate the exoskeleton manually. The average decoding accuracy of the successful sessions (i.e., P1 sessions 1 and 2; P2 session 2; and P4 session 2) was 77.61 ± 14.72%, and their average time was 1.35 ± 0.71 s.

Notice that the decoding times reported in **Tables 3**, **4** correspond to the average time that triggering the movement took with respect to the presentation of the audio cue that indicated the beginning of the "Movement Attempt" interval. Given that different types of delays have an influence on this time (e.g., computational, cognitive, and/or physiological), we also evaluated the average decoding time with respect to the negative peak of the MRCPs. These peaks have been observed to be aligned with the beginning of the muscular activity (Niazi et al., 2011), and hence, may constitute a better indicator of when the subjects started the attempt of movement. For the healthy subjects, the average MRCP negativity appeared 1.02 ± 0.02 s after the auditory cue (**Figure 2**, first row). Hence, the average decoding time was 50 ms after the MRCP peak. For the 3 SCI patients who performed successful BMI sessions (i.e., P1, P2, and P4), the MRCP negativity appeared on average 1.37 ± 0.20 s after the cue. Therefore, the average decoding time for the patients was 20 ms before the occurrence of their MRCP peak.

#### TABLE 3 | Decoding results of healthy subjects.


Given is the number of trials performed, the number of trials in which the BMI decoded the intention of motion (resulting in a walking movement), the decoding accuracy (i.e., the percentage of correctly decoded trials), and the time between the auditive cue and the exoskeleton movement.

Although the proposed protocol impeded that the exoskeleton could start moving during "Rest" and "Preparation" time intervals, we analyzed the offline movement triggers that were generated by the BMI in those intervals. To that end, we calculated the number of trials in which, at least, one movement trigger was generated during non-movement periods. For the healthy subjects, movement triggers were generated in 52.22 ± 16.69% of the trials during the "Preparation" interval, and in 66.67 ± 13.02% of the trials during the "Rest" interval. For the successful sessions with the patients, movement triggers were generated in 40.45 ± 16.98% of the trials during the "Preparation" interval, and in 63.42 ± 14.15% of the trials

#### TABLE 4 | Decoding results of SCI patients.


For each patient and session, given is the number of trials performed, the number of trials in which the BMI decoded the intention of motion (resulting in a walking movement), the number of manual triggers sent by the experimenter, the decoding accuracy (i.e., the percentage of correctly decoded trials), and the time between the auditive cue and the exoskeleton movement.

\*The decoding accuracy was calculated as the number of decoded trials divided by the number of trials in which the experimenter did not send a manual trigger: %Acc = #Gait cycles #Trials <sup>−</sup> #Manual triggers × 100.

\*\*This session was prematurely interrupted due to technical problems with the exoskeleton.

\*\*\*This session was prematurely interrupted due to temporal restrictions of the participant.

during the "Rest" interval. We performed 3 statistical tests to compare the percentages of trials with correct triggers during the "Movement Attempt" period (true positives) and with erroneous triggers generated during "Rest" (false positives during "Rest") and "Preparation" (false positives during "Preparation") periods. For each measure, a single vector was generated by concatenating the percentages of the three healthy subjects and the four successful sessions of the patients, and paired Wilcoxon signed rank tests were used to measure significant differences. The number of true positives was significantly higher than the number of false positives generated during the "Preparation" interval (p < 0.05), although true positive was not significantly higher than the number of false positives during the "Rest" interval (p = 0.08). In addition, the number of false positives during the "Rest" interval was significantly higher than the ones generated during the "Preparation" interval (p < 0.05).

#### 3.3. Features

As an automatic feature selection algorithm was used during the experiments, we performed a post-hoc analysis to evaluate the selected features for each participant. **Figure 4** shows the features selected by the SDA algorithm for the healthy subjects and the patients. For the patients, the reported results correspond to the second BMI session. As can be seen, more frequency (ERD) than temporal (MRCP) features were consistently selected in both groups. For the ERD features, pairs in the whole channel-frequency space were selected for all the subjects. Central and centroparietal electrodes were more consistently selected than frontocentral ones. In addition, certain subjects showed a higher density of features allocated in specific regions of the channel-frequency map. For instance, for subject H1, more features were selected in the alpha band (8–12 Hz), whereas for subject H2, there were more in beta band (15– 25 Hz). MRCP features were more scarcely chosen by the algorithm, although in all of the participants, some of them were selected.

#### 3.4. Exertion and Satisfaction Assessments

**Table 5** shows the values of the Borg scale given by each patient on each session. At the beginning of the session, all the patients reported the minimum exertion level. As described above, each patient performed 3 or 4 screening blocks (each of which lasted around 3 min), resting between blocks as long as they required. After these screening blocks, all the patients reported an increase between 3 and 5 points on their exertion level. Subsequently, they started the closed-loop blocks, in which they walked a maximum of 10 m with the exoskeleton. Here, exertion levels slightly increased for 1 or 2 points in most cases. The only exceptions were P2 in session 1 (which just performed 2 gait cycles due to technical problems), who did not report any increase after screening; and P3 in session 2, who increased 5 points. None of the values of the Borg scale exceeded 17, which is considered as the limit value for maximal exertion.

The results of the satisfaction test are presented in **Table 6**. The highest score for the QUEST scale was obtained in the questions about safety and security, and easiness of use (4.25 on average), whereas the lowest was obtained by the question about comfortability (2.5 on average).

#### 3.5. Exoskeleton Adaptive Control

An important feature of the system introduced in this paper was that, since we used an ambulatory exoskeleton, we could introduce an assist-as-needed control paradigm, which may serve to make the rehabilitation interventions more challenging (Pons and Torricelli, 2014). Although it was not one of the main goals of this study, we measured the degree of assistance that the control strategy provided to healthy subjects and patients. The gait assistance for the healthy and SCI individuals is provided based on the adaptive control model presented in Rajasekaran et al. (2015). The adaptive control applies an efficient stiffness to

FIGURE 4 | Features selected by the SDA algorithm for each subject. Upper/lower panel corresponds to the features of the healthy subjects/patients. The left part of each panel shows the ERD features that were selected as channel-frequency pairs. The right part of each panel shows the MRCP features that were selected as channel-time pairs.

each joint, which is computed based on the interaction torques and position error of each joint. Hence, the assistance level is defined based on the variation in the stiffness parameter. For healthy subjects, the stiffness values for hip, knee, and ankle were 60 ± 4 Nm/deg, 60 ± 2 Nm/deg, and 60 ± 5 Nm/deg, respectively. For the SCI patients, the stiffness for hip, knee, and ankle were 80 ± 2 Nm/deg, 82 ± 5 Nm/deg, and 80 ± 5 Nm/deg, respectively.

### 4. DISCUSSION

The present study proposed a novel system BMI with an ambulatory walking exoskeleton. Its feasibility has been shown with experiments performed by three healthy subjects and four spinal cord injury (SCI) patients. The BMI decoded the brain activity related to the intention of movement and sent the commands to the robotic system. The robot moved for two steps

functional recovery (Jackson and Zimmermann, 2012). The

(one with the right and one with the left leg), using an assist-asneeded strategy, which assisted the patients only to the extent they needed. The degree of assistance was shown to be higher for the SCI patients, which could not perform the movement autonomously, compared to the healthy subjects, who did not have to make a high effort to follow the exoskeleton during its walking movements. This is, to the best of our knowledge, the first study in which a walking exoskeleton with no weight or balance support is controlled by incomplete SCI patients with a BMI. This may serve as a proof-of-concept for future studies in which a larger sample could allow the assessment of the rehabilitative effects of this type of BMI-controlled ambulatory exoskeletons.

In spite of the fact that EEG-based BMI technology is still far from being a practical control system for gait-assistive devices, its applicability for rehabilitation within clinical environments may be available in the next few years. In fact, its use has a great potential for rehabilitation of incomplete SCI patients. In these patients, there are still some pathways communicating the brain and the limbs, which can support some degree of


For each patient and session, the exertion levels were measured when the patient arrived (Pre session), after performing the screening blocks (After screening), and at the end of the session (Post session). The values of this scale range from 6 ("very, very light") to 20 ("very, very hard").

persistent causal relationship between the brain activation during the intention of movement and the stimulation of the limb (e.g., with an exoskeleton or electrical stimulation) has demonstrated its viability to induce Hebbian plasticity in animal studies (Jackson et al., 2006). Most BMI studies aiming to control rehabilitative devices are focused on the upper-limb, and they have demonstrated the possibility of controlling robotic systems (Gomez-Rodriguez et al., 2011; Ramos-Murguialday et al., 2012, 2013; Bhagat et al., 2016) or functional-electrical stimulation (FES) (Pfurtscheller et al., 2003; Rohm et al., 2013). For the lower-limb, there are less examples in the literature of closedloop non-invasive brain-controlled systems. The group led by Dr. Nenadic (University of California, USA) demonstrated the BMI-based control of weight-suspended robotic (Do et al., 2013) and FES (King et al., 2015) gait-assistance systems. Furthermore, two recent studies have used brain signals to control the REX (REX Bionics Ltd), a robotic system that provides weight and balance support for patients with a high degree of gait disability (Kilicarslan et al., 2013; Kwak et al., 2015). The system proposed in this paper utilized an ambulatory exoskeleton, which does not rely on any weight support beyond the walker or the parallel bars. Hence, our approach targets patients with incomplete and lower lesions, who can walk short distances with the help of crutches or walkers, and who are the ones with best prognosis for gait rehabilitation (Nene et al., 1996; Scivoletto et al., 2014).

During the screening blocks, the exoskeleton joints were blocked, and hence, even the healthy subjects could just perform the attempt of movement, and not an overt movement. The EEG correlates of these movement attempts, namely the ERD and the MRCPs, were used as features to train the BMI decoder for the closed-loop blocks. The combination of both activation patterns has been shown to be beneficial to improve the movement intention decoding (Ibáñez et al., 2014; López-Larraz et al., 2014), as it may prime the use of features from the signals with higher degree of activation (e.g., P2 which showed a significant ERD, but an MRCP with a low amplitude, see **Figure 2**). In a posthoc analysis, we observed that, in general, ERD features were


1, Not satisfied at all; 2, Not very satisfied; 3, More or less satisfied; 4, Quite satisfied; 5, Very satisfied.

more consistently selected by the automatic feature selection algorithm than MRCP features. There is certain controversy with regard to how these correlates can be affected by motion artifacts during walking (Castermans et al., 2014; Nathan and Contreras-Vidal, 2016). In fact, we observed that slow oscillations were present in some trials during the movements of the exoskeleton, especially in the patients (see **Figure 3**, center and right plots). Anticipating this, we preferred to train the decoder using signals recorded during the attempt of movement and not during actual movement. To minimize artifacts in the training data, we used an automatic artifact rejection method that eliminated contaminated trials (e.g., by slow movement oscillations or EMG artifacts). Moreover, we carefully instructed all the participants to only perform the attempt of movement of their right leg, avoiding compensatory movements with the rest of the body. However, the attempt of movement of a paralyzed limb is a complex task, especially for the paraplegic patients, who had to concentrate on keeping balance at the same time. Therefore, in the protocol we proposed, it is not possible to guarantee that they were only attempting to move the right leg, as they may also be activating arm or trunk muscles, which normally help them to move their legs during their normal walking rehabilitation. This means that the brain activations we measured in the patients may include not only the attempt of movement of the leg but also compensations of other body parts. Comparison of the brain activation patterns between the groups of healthy subjects and patients was out of the scope of this work. Firstly, because the number of participants was small to perform an accurate neuroimaging study, and secondly, because the set-up and task performed by each group were slightly different (i.e., walker and attempt of movement of a healthy limb for the healthy subjects; parallel bars and attempt of movement of a paretic limb for the SCI patients). Nonetheless, in terms of the brain activations, we observed that the subjects who presented weaker brain activations during the attempt of movement were the ones with poorer decoding performances: H3 and P3. Three of four patients had long-term injuries, between 11 and 24 months. Curiously, these three patients were the ones with best decoding performances, showing similar values to healthy subjects, in contrast to P3, who had a 5-months injury but could not control the BMI correctly.

SCI modifies the brain activity related to movement (Müller-Putz et al., 2007; Castro et al., 2013; López-Larraz et al., 2015a). Although there is a large body of literature showing that movement intention can be decoded from EEG signals on these patients (Pfurtscheller et al., 2003; López-Larraz et al., 2012; Rohm et al., 2013; King et al., 2015), among others, this neuroplastic process may affect the reliability of rehabilitative and assistive BMI systems to be used by SCI patients. The heterogeneity of incomplete lesions will probably result in significant differences in the neural reorganization processes followed by the patients' brains (Freund et al., 2013). Hence, an interesting research pursuit for the next years will be the characterization of the brain changes following SCI, which will set the basis to personalize the systems to improve their applicability.

Regarding the decoding algorithms to detect the movement intention, extensive research is being conducted toward the optimization of signal processing and classification techniques to increase BMI performance (Bashashati et al., 2015; López-Larraz et al., 2015b). The procedure used in this work has been previously used to decode movements of the upper-limb with incomplete tetraplegic patients (López-Larraz et al., 2014), and the accuracies that we achieved were similar to other recent studies detecting gait initiation (Jiang et al., 2015; Sburlea et al., 2015). Five consecutive windows with the classifier indicating a motor attempt output were required to start the movement. Although it implied a constant delay of 250 ms with respect to the first output, this mechanism was used to ensure a consistent brain activation and not just a spurious change that may lead to false positive activations There is evidence stating that short delays are beneficial to facilitate plasticity in the brain (Mrachacz-Kersting et al., 2012), and recent studies have worked on developing methodologies to anticipate movements or to decode them with a short latency (López-Larraz et al., 2014; Xu et al., 2014; Jiang et al., 2015; Sburlea et al., 2015). However, these studies require a precise measurement of the time instant when the movement starts in order to calibrate the BMI. Due to the typology of the patients recruited for this study, we could not have a reliable signal to identify the movement onset, even measuring the muscle activity with electromyography. Therefore, we considered that the delay induced by our methodology to detect the movement intention could be acceptable to control the exoskeleton, as a similar procedure has demonstrated its efficacy for neurorehabilitation of stroke patients (Ramos-Murguialday et al., 2013). In a post-hoc analysis, we compared the latency of the decoder with respect to the MRCP peak negativity, and observed that the difference was minimal (+50 ms for the healthy subjects and −20 ms for the patients). MRCP negativity is, in general, aligned to the onset of muscular activity during voluntary movements (Niazi et al., 2011). Although it should not be used as a robust measurement of decoding latency, this metric allowed us to estimate how this time would be with respect to the EMG activations. In any case, we consider that more investigation is required to evaluate how different tradeoffs (e.g., priming temporal precision, or guaranteeing consistent brain activation) can affect BMI performance and rehabilitative outcomes.

A cue-guided BMI protocol was proposed so that the participants always knew in which phase of the trial they were. Furthermore, the shared control strategy implemented implied that the exoskeleton movements were only enabled in specific time periods (i.e., the "Movement Attempt" intervals), ensuring that no unexpected movements could happen when the patients were not ready. The BMI analyzed the brain signals continuously, meaning that the movement triggers were generated asynchronously at any time during the trial. If these triggers were generated during the "Movement Attempt" interval, then the gait cycle gets started, and otherwise, they were ignored. The offline analysis revealed that movement triggers were generated in a high percentage of trials during the "Rest" (i.e., accommodation period, in which subjects could move and rest) and "Preparation" (i.e., relaxation period preceding the "Movement Attempt") intervals. The movement attempts were correctly decoded in 84.44 ± 14.56% and 77.61 ± 14.72% of the trials, for healthy subjects and patients, respectively. These percentages were significantly higher than the number of trials with movement triggers generated during "Preparation" interval (52.22 ± 16.69% and 40.45 ± 16.98%), but not greater than the trials with triggers generated during "Rest" interval (66.67 ± 13.02% and 63.42 ± 14.15%). Notice that in this latter period, the participants sometimes moved to accommodate their position after the previous trial. These numbers suggest that the use of shared-control strategies designed to avoid non-desired robot movements, like the one proposed in this paper, can facilitate the integration of BMI technology in clinical set-ups (Rohm et al., 2013). These take special relevance in gait rehabilitation therapies, in which keeping the balance may cause movements during rest periods that lead to more false positives.

The proposed approach aims at decoding the intention of motion to generate a functional movement (i.e., a gait cycle) with the exoskeleton. The repetitive association between the brain activation related to the motion intention and the peripheral feedback may reinforce the corticospinal circuits and promote Hebbian synaptic plasticity (Jackson and Zimmermann, 2012). An alternative and interesting approach for future would be to develop a system that continuously controls the exoskeleton movements instead of decoding the intention of movement and triggering a predefined trajectory. This should be the preferred strategy for assistive devices. Presumably, it may also improve rehabilitative effects by a more consistent association between paired firing of neurons, which may accelerate the neuroplastic changes (Jackson and Zimmermann, 2012). To date, promising results toward the continuous control of gait rehabilitations devices with EEG have been shown in preliminary studies with weight-suspended and self-supported systems (Do et al., 2013; Kilicarslan et al., 2013; King et al., 2015; Kwak et al., 2015). However, there are still several issues that need to be improved before the effective implantation of this technology in clinical practice, such as signal processing techniques or artifacts management (Castermans et al., 2014; Nathan and Contreras-Vidal, 2016). Some of these issues are of especial relevance when using ambulatory exoskeletons, which require extra considerations to improve safety, as the control strategy implemented to deal with possible false positives. Furthermore, the rehabilitative effects of each type of intervention still have to be quantified with clinical studies. For now, the development and improvement of novel interventions, like the one proposed here, aim at increasing the number of possible interventions to rehabilitate gait. Eventually, the clinicians will be in charge of evaluating the risks and benefits to recommend the most suitable interventions for each specific patient (e.g., BMI continuous control of prosthesis, or BMI-triggered predefined movements, as the one proposed in this study).

The main goals of this study were to test the BMI-exoskeleton system and to propose a methodology that may be followed in future studies combining BMI and ambulatory exoskeletons. Therefore, the design of the protocol was a key point to be able to validate the technology and the set-up with the patients. We observed that a familiarization session with the exoskeleton was necessary before the BMI session to allow the participants to get used to the dynamics of the system. While for the healthy subjects, a 5–10 min familiarization was enough, the patients required a specific session due to their poor balance and walking capabilities. Several issues have to be tailored for each patient depending on his/her capabilities, like the cadence or the distance between joints. Rehabilitative devices such as the one used in this study have to prioritize patient's safety. The recruited patients were capable of standing and ambulating without the aid of a harness, which required a complex set-up and additional safety measures to avoid falls. In this respect, the experimenter played an important role by controlling the system, which functioned as a "dead-man's" switch. This means that at the beginning of every trial, the experimenter had to make sure that the patient's legs and feet position were appropriate and ask him/her if he/she was ready for another step. This methodology tried to imitate the procedure followed by physiotherapists for patients in the early stages of gait rehabilitation with rigid leg orthoses. In addition, the experimenter had to monitor the EEG signals and the BMI output in order to verify that everything was correct and guide the patients when they lost concentration or generated artifacts by excessive movements. The use of BMI technology to provide the therapist with objective information about the patient's performance has been stated to be very important for the implantation of this type of systems in clinical environments (Mattia et al., 2013; Asín Prieto et al., 2014). We consider that the good results achieved in this study were, in part, due to the therapist–patient interaction that was augmented—thanks to the designed therapist interface.

The fatigue level of the patients was also continuously monitored, and they could ask for a pause whenever they wanted to relax for some minutes. This is, probably, the reason of the relatively low levels of exertion shown by the patients when asked at the end of the session. In terms of usability and satisfaction, the patients were not so positive as we had expected, but they provided very useful information that encourages the authors to continue working to improve the wearability and comfort of the system. Based on the high scores of the questions about safety and security, we believe that the security measures implemented in this study succeeded and they could be applied for future studies. In general, two of the patients (P1 and P3) were rather satisfied with the system, whereas the other two (P2 and P4) reported that several issues could be improved, especially those related with comfort and wearability of both the exoskeleton and the EEG system. Nonetheless, all of them appraised the potential of the combination of both technologies. Patients found the use of the conductive gel for the EEG recording as one of the main inconveniences, which is in line with other works (Rupp, 2014). We believe that the potentiality of dry electrodes will have to be explored for future prototypes to be used with patients in clinical practice. Another important limitation of the system is the difficulty to wear the exoskeleton, as at least two people had to assist the process. This is an important factor to improve for future designs of exoskeletons with clinical applicability. Ideally, the patients should be able to wear the exoskeleton by themselves. All of them reported their willingness to keep participating in rehabilitation interventions with this technology, which is a promising sign for the future of BMI-based gait rehabilitation.

It is important to mention that due to the nature of the study and the typology of patients recruited, the number of trials recorded with these patients was small. Nonetheless, the BMI performances are in line with similar works performed for the upper- and lower-limbs. The main contribution of this work is to validate the feasibility of a BMI system and protocol to control an ambulatory exoskeleton for gait rehabilitation without weight or balance support. However, this preliminary study does not allow yet to draw conclusions about the rehabilitative potential of this technology. Further experiments with a larger population of patients and with a larger number of sessions will be required to evaluate if the therapeutic potential of BMI for gait rehabilitation approaches the traditional therapies. New challenges appear when it comes to pursuing clinical trials integrating this kind of novel technologies with patients. Adapting the systems for different pathologies and personalization of the technology will be of paramount importance for the use of BMI systems in clinical practice (Rupp, 2014). In addition, standardizing metrics to evaluate system performances and clinical outcomes will facilitate the validation of BMIs for their implantation in rehabilitative centers (Venkatakrishnan et al., 2014).

### REFERENCES


Clemmensen, L., Hastie, T., Witten, D., and Ersbøll, B. (2011). Sparse discriminant analysis. Technometrics 53, 406–413. doi: 10.1198/TECH.2011.08118


#### AUTHOR CONTRIBUTIONS

Conceived and designed the experiments: EL, FT, VR, AD, JA, LM. Performed the experiments: EL, FT, VR, SP. Analyzed the data: EL, FT. Drafted the manuscript: EL, FT, VR. Revised the manuscript: EL, FT, VR, SP, AD, JA, JM, AG, LM.

#### ACKNOWLEDGMENTS

This research has been partially supported by Spanish Ministry of Science project HYPER-CSD2009-00067, and DGA-FSE, grupo T04.

### SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: http://journal.frontiersin.org/article/10.3389/fnins. 2016.00359


bedside," in Proceedings of the 5th International Brain-Computer Interface Conference (Graz), 268–271.


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2016 López-Larraz, Trincado-Alonso, Rajasekaran, Pérez-Nombela, del-Ama, Aranda, Minguez, Gil-Agudo and Montesano. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# A Personalized Multi-Channel FES Controller Based on Muscle Synergies to Support Gait Rehabilitation after Stroke

Simona Ferrante<sup>1</sup> \* † , Noelia Chia Bejarano1 †, Emilia Ambrosini 1, 2, Antonio Nardone3, <sup>4</sup> , Anna M. Turcato3, 4, Marco Monticone2, 5, Giancarlo Ferrigno<sup>1</sup> and Alessandra Pedrocchi <sup>1</sup>

*<sup>1</sup> Neuroengineering and Medical Robotics Laboratory, Department of Electronics, Information and Bioengineering, Politecnico di Milano, Milan, Italy, <sup>2</sup> Physical Medicine and Rehabilitation Unit, Scientific Institute of Lissone, Fondazione Salvatore Maugeri (IRCCS), Lissone, Monza Brianza, Italy, <sup>3</sup> Posture and Movement Laboratory, Division of Physical Medicine and Rehabilitation, Scientific Institute of Veruno, Fondazione Salvatore Maugeri (IRCCS), Veruno, Novara, Italy, <sup>4</sup> Department of Translational Medicine, University of Eastern Piedmont, Novara, Italy, <sup>5</sup> Department of Public Health, Clinical and Molecular Medicine, University of Cagliari, Cagliari, Italy*

#### Edited by:

*Paolo Massobrio, University of Genoa, Italy*

#### Reviewed by:

*Erika G. Spaich, Aalborg University, Denmark Marco Knaflitz, Polytechnic University of Turin, Italy*

> \*Correspondence: *Simona Ferrante simona.ferrante@polimi.it †Co-first authors.*

#### Specialty section:

*This article was submitted to Neuroprosthetics, a section of the journal Frontiers in Neuroscience*

Received: *07 March 2016* Accepted: *30 August 2016* Published: *16 September 2016*

#### Citation:

*Ferrante S, Chia Bejarano N, Ambrosini E, Nardone A, Turcato AM, Monticone M, Ferrigno G and Pedrocchi A (2016) A Personalized Multi-Channel FES Controller Based on Muscle Synergies to Support Gait Rehabilitation after Stroke. Front. Neurosci. 10:425. doi: 10.3389/fnins.2016.00425* It has been largely suggested in neuroscience literature that to generate a vast variety of movements, the Central Nervous System (CNS) recruits a reduced set of coordinated patterns of muscle activities, defined as muscle synergies. Recent neurophysiological studies have recommended the analysis of muscle synergies to finely assess the patient's impairment, to design personalized interventions based on the specific nature of the impairment, and to evaluate the treatment outcomes. In this scope, the aim of this study was to design a personalized multi-channel functional electrical stimulation (FES) controller for gait training, integrating three novel aspects: (1) the FES strategy was based on healthy muscle synergies in order to mimic the neural solutions adopted by the CNS to generate locomotion; (2) the FES strategy was personalized according to an initial locomotion assessment of the patient and was designed to specifically activate the impaired biomechanical functions; (3) the FES strategy was mapped accurately on the altered gait kinematics providing a maximal synchronization between patient's volitional gait and stimulation patterns. The novel intervention was tested on two chronic stroke patients. They underwent a 4-week intervention consisting of 30-min sessions of FES-supported treadmill walking three times per week. The two patients were characterized by a mild gait disability (walking speed > 0.8 m/s) at baseline. However, before treatment both patients presented only three independent muscle synergies during locomotion, resembling two different gait abnormalities. After treatment, the number of extracted synergies became four and they increased their resemblance with the physiological muscle synergies, which indicated a general improvement in muscle coordination. The originally merged synergies seemed to regain their distinct role in locomotion control. The treatment benefits were more evident for one patient, who achieved a clinically important change in dynamic balance (Mini-Best Test increased from 17 to 22) coupled with a very positive perceived treatment effect (GRC = 4). The treatment had started the neuro-motor relearning process also on the second subject, but twelve sessions were not enough to achieve clinically relevant improvements. This attempt to apply the novel theories of neuroscience research in stroke rehabilitation has provided promising results, and deserves to be further investigated in a larger clinical study.

Keywords: functional electrical stimulation, stroke rehabilitation, locomotion, treadmill, muscle synergies

### INTRODUCTION

The rehabilitation of neurological patients strongly benefits of task-oriented, immersive, repetitive exercises when the patient experiences an enriched, augmented sensorial feedback. Indeed, such interventions stimulate the activity-dependent plasticity of the Central Nervous System (CNS) thus facilitating motor relearning (Ting et al., 2015). Activity-dependent plasticity is further enhanced when Functional Electrical Stimulation (FES) is synchronized with task-oriented volitional exercises (Sheffler and Chae, 2007; Chae et al., 2008; Gandolla et al., 2014, 2016; Kafri and Laufer, 2015), such as cycling (Ferrante et al., 2008; Ambrosini et al., 2011, 2012) and walking (Kesar et al., 2009; Embrey et al., 2010; Sabut et al., 2010, 2011; Daly et al., 2011; Kim et al., 2012). Indeed, the increased afferent feedback provided by FES modulates motor cortex function and excitability to facilitate recovery (Ridding et al., 2000; Gandolla et al., 2014, 2016).

The first FES-based gait systems were designed for the treatment of foot drop, combining single-channel stimulation of the peroneal nerve with a pressure sensor to detect the initial contact of the foot with the ground (Melo et al., 2015). Since then, multi-channel FES strategies have been proposed and tested in stroke patients (Kesar et al., 2009; Ambrosini et al., 2010; Embrey et al., 2010; Sabut et al., 2010, 2011; Daly et al., 2011; Kim et al., 2012). However, in all FES-based gait systems, only the two main gait phases (i.e., the stance and swing phase) were detected and used to trigger the stimulation of the different muscles involved in the movement. The stimulation waveforms were mainly trapezoidal (Melo et al., 2015). These waveforms use a ramp up of stimulation amplitude at a constant pulse width to avoid sudden and jerky muscle responses both in the agonist and antagonist muscles, and a ramp down to avoid a sudden and unpleasant slap of the foot on the ground. Biomimetic stimulation controllers, which modulate the stimulation amplitude based on physiological EMG activations, were proposed for a single-channel drop-foot stimulator and were tested on a single patient, resulting to be more efficient than trapezoidal profiles (O'Keeffe et al., 2003). Biomimetic multichannel FES systems have shown promising therapeutic benefits when applied in stroke patients during cycling (Ferrante et al., 2008; Ambrosini et al., 2011, 2012). However, to the authors' knowledge, a biomimetic multi-channel FES system has not yet been proposed and tested during gait.

To design a biomimetic FES controller, it is essential to mimic the neural solutions adopted by the CNS to generate movements. It has been largely suggested in neuroscience literature that in order to generate a vast variety of movements, the CNS recruits a reduced set of coordinated patterns of muscle activities, defined as muscle synergies or motor modules (d'Avella et al., 2003; d'Avella and Bizzi, 2005). Further, a study on spinalized rats has provided experimental evidence that the CNS simplifies the complexity and high dimensionality of neural commands and mechanical outputs by means of a modular organization at the neuromuscular level (Mussa-Ivaldi and Bizzi, 2000).

The concept of muscle synergy has been formalized with a mathematical model based on factorization algorithms that decompose the EMG signals into the product of two components. The weighting component reveals the muscle composition of each synergy and the relative level of activation of each muscle, whereas the temporal component reflects the activation timing of each synergy throughout the execution of the movement. Each muscle synergy contributes to the mechanical output needed to generate task-specific biomechanical functions, also called biomechanical correlates (Lacquaniti et al., 2012). Many studies on physiological gait have agreed in the definition of four synergies as responsible of the main biomechanical correlates on healthy subjects (Clark et al., 2010; Barroso et al., 2014; Routson et al., 2014):

Synergy 1 (weight acceptance): activation of the hip and knee extensors during early stance that is associated with weight acceptance;

Synergy 2 (push off): activation of the ankle plantar-flexors in late stance that is associated with forward propulsion;

Synergy 3 (foot clearance): activation of the rectus femoris and the tibialis anterior during early stance and early swing, which provides foot dorsi-flexion immediately after heel strike and ground clearance of the foot, respectively;

Synergy 4 (leg deceleration): activation of the hamstrings during late swing and early stance to decelerate the leg and propel the body.

An additional synergy can be found when the trunk muscles are also recorded (Ivanenko et al., 2005). Simulation studies have confirmed the validity of the biomechanical correlates of the muscle synergies (Neptune et al., 2009; Allen and Neptune, 2012).

Muscle synergies have been shown to be "fixed" because they are consistent across different subjects despite variability and noise in the neuro-musculo-skeletal system, but also "flexible" so that they can adapt to slight changes in the environment or be affected by pathologies and then modulated with rehabilitation training (Santello and Lang, 2014). For instance, during locomotion post-stroke individuals exhibit a reduced number of synergies in their paretic side due to the merging of motor modules that imply a non-functional muscle co-contraction reflecting walking dysfunctions (Bowden et al., 2010; Clark et al., 2010; Ting et al., 2015). It is likely that this reduction is caused by a lack of independence of the corticospinal drive to the spinal cord, which ultimately causes a poor muscle control.

Muscle-synergy analysis is currently considered a useful methodology to assess sensorimotor individual deficits (Safavynia et al., 2011). Further, it could be a potential ground upon which novel therapies aimed at enhancing motor relearning could be designed (Ting et al., 2015). In this scope, a FES training based on healthy muscle synergies has been recently proposed for a balance control task. However, the experimental apparatus was rather complex, discouraging its translation to the clinical practice (Galeano et al., 2014).

Our study merges the potentialities of FES-based gait treatments with the strength of muscle-synergy training approach. Indeed, this study was aimed at designing a personalized, biomimetic, multi-channel stimulation controller to support gait rehabilitation after stroke, integrating the following novel aspects:


Preliminary results obtained from two chronic stroke patients with the proposed FES gait controller will be presented in order to show the potentiality of this novel intervention.

## MATERIALS AND METHODS

#### The Stimulation Controller Architecture

The FES-controller architecture includes the subject that can walk overground or on a treadmill, a PC running Linux RTAI, which hosts the whole control system, and a current-controlled stimulator (Rehastim <sup>R</sup> , Hasomed GmbH) delivering biphasic pulses to surface electrodes (Pals <sup>R</sup> , Axelgaard Manufacturing Co., Ltd.) placed on up to 8 muscles of the paretic leg. The subjects wear two inertial sensors (Mtx <sup>R</sup> , Xsens Technology), on both shanks, which provide a real-time kinematic measure used to accurately synchronize the stimulation to the gait cycle. The control system comprises a graphical user interface (GUI) implemented in QtTMsoftware and two real-time applications. The GUI allows the therapist to customize the treatment on the single patient, to start, pause, and stop the treatment, to save data, and to access the stored data. The first real-time application of the control system is the gait segmentation block (see **Figure 1**), which receives the signals from the inertial sensors and estimates the Initial Contact (IC), the End Contact (EC), and the Mid-Swing (MS) gait events for each leg. This algorithm was adapted from Chia Bejarano et al. (2015b), in order to be used robustly also in a magnetically disturbed environment, and was validated on healthy subjects using the force-sensitive resistors as a gold standard (Chia Bejarano et al., 2015a). The algorithm demonstrated an excellent accuracy in detecting the IC and EC events (F1-score of 0.98 for the IC and 0.96 for the EC), and allows the detection of the following 6 gait phases: paretic double support, non-paretic initial swing, non-paretic terminal swing, non-paretic double support, paretic initial swing, paretic terminal swing (**Figure 1**).

The second real-time application is the multi-channel synergybased stimulation controller that is personalized on each patient following the steps reported in **Figure 2** and detailed below.

#### Healthy Muscle Synergies

The starting point of the stimulation controller is the definition of a set of representative healthy muscle synergies. Thirteen healthy subjects (7 men and 6 women; age: 24.8 ± 1.3 years; height: 1.73 ± 0.11 m, weight: 60.8 ± 11.4 kg) volunteered to participate in this study. They were asked to walk overground and on a treadmill at their self-selected speed. The EMG was measured on the following muscles of the dominant leg: gluteus maximus (GM), rectus femoris (RF), vastus medialis (VM), hamstring medialis (HM), hamstring lateralis (HL), gastrocnemius medialis (MG), and tibialis anterior (TA). Kinematics data were acquired at 50 Hz by means of 2 inertial and magnetic sensors (MTx, Xsens) worn on both shanks (Chia Bejarano et al., 2015b).

The EMG signals were acquired at 1024 Hz, band-pass filtered (3rd-order Butterworth filter, cut-off frequencies of 40 and 400 Hz), rectified, and low-pass filtered (3rd-order Butterworth filter, cut-off frequency of 5 Hz) to obtain the EMG envelopes. Afterwards, the envelopes were segmented into single strides using the first contacts of the ipsilateral foot with the pavement (IC events). Then, each stride was normalized in time by interpolating the signals into 100 points, and in amplitude by dividing the EMG signals of each muscle by the median maximum value obtained across strides for each walking condition (treadmill and overground). After removing the initial acceleration and the final deceleration phases from each acquisition, 20 representative strides for each subject and condition were extracted as suggested by Oliveira et al. (2014). The non-negative matrix factorization (NMF) algorithm was applied separately to the 20 envelopes obtained for each subject and walking condition, in order to extract their muscle synergies (Lee and Seung, 1999). The quality of the factorization was measured by computing the variability accounted for (VAF) and the number of muscle synergies was chosen as the smallest number that allowed the reconstruction with a total VAF higher than 90%, or that did not improve the single-muscle VAF more than 5% when adding a new synergy (Clark et al., 2010). The individual muscle synergies of each walking condition were extracted using the most representative number of synergies obtained according to the chosen VAF criterion. Then, the weights of each individual muscle synergy were normalized to have a unitary norm, applying the corresponding transformation to their respective activations profiles, to maintain constant their product. For each walking condition, the average set of muscle synergies across subjects was calculated. To compare the healthy muscle synergies obtained during overground and treadmill walking, the mean and standard deviation of the following metrics were computed: (1) the similarity, i.e., the normalized scalar product between the weights (W) extracted in the two walking conditions; (2) the circular cross correlation between the activation profiles (H) extracted in the two conditions; (3) the

arrows respectively. GUI, graphical user interface; freq, frequency; PW, pulse width; A, amplitude.

lag in percentage of gait cycle in which the maximal circular cross correlation was obtained. If the two walking conditions were comparable in terms of muscle synergies, the healthy synergies extracted from the overground walking condition could be used both as a reference to evaluate the overground waking coordination of patients before and after treatment, and to build the biomimetic stimulation strategy to be applied during treadmill training.

#### Pre-treatment Assessment of the Patient's EMG

Before starting the treatment, the patient was asked to walk overground and the surface EMG activation signals were measured on eight muscles of the paretic leg following the same protocol described in the previous paragraph for healthy subjects. Analogously, the signal processing procedure described above was used to obtain the EMG envelopes.

#### NNR of the Patient's EMG Envelopes with the Healthy Muscle Synergies

The mean set of weights (WHEALTHY) and activation profiles (HHEALTHY) obtained during overground walking in healthy subjects were used to perform the two Non-Negative Matrix Reconstructions (NNR) of the EMG envelopes obtained on the paretic side during the patient's pre-treatment assessment. The NNR was applied by fixing the synergy vectors as WHEALTHY and letting only the synergy activation coefficients H update at every algorithm iteration, according to the following multiplicative update rule:

$$H \leftarrow H \frac{\left(\begin{array}{c} \left(\boldsymbol{W}\_{HEALTHY}^{T} \cdot \boldsymbol{M} \right) \\ \hline \left(\boldsymbol{W}\_{HEALTHY}^{T} \cdot \boldsymbol{W}\_{HEALTHY} \cdot \boldsymbol{H} \right) \end{array} \right)}{\left(\begin{array}{c} \boldsymbol{W}\_{HEALTHY}^{T} \cdot \boldsymbol{H} \end{array} \right)}$$

where M is the matrix of the EMG envelopes measured on the 8 muscles during 20 gait cycles. Each vector of WHEALTHY was normalized to unit norm before applying NNR.

Afterwards, the NNR was applied to the EMG envelopes of the specific patient by fixing the synergy activation coefficients HHEALTHY and deriving the patient's weights, using the following update rule:

$$W \leftarrow \ W \frac{\left(M \cdot H\_{\text{HEADITY}}^T\right)}{\left(W \cdot H\_{\text{HEADITY}} \cdot H\_{\text{HEADITY}}^T\right)}$$

#### Identification of the Patient's Impaired Muscle Synergies

Each of the four reconstructed patient's muscle synergies were compared to the mean healthy synergies by computing the following metrics:


• the activation duration was defined as: Act = 1 − ActHp−ActHhealthy <sup>100</sup> , where ActHp and ActHhealthy are the durations, in percentage of the gait cycle, of the activation phases. These were computed on the patient's reconstructed activation profile and on the mean activation profile of the healthy group, respectively. The activation duration was defined based on the onset and offset values, which were determined from the activation profile using a threshold fixed at the minimum of each profile plus 20% of the cycle peak-to-peak amplitude.

For all metrics a value close to 1 indicates a behavior similar to the healthy subjects. The metrics were first computed on the group of healthy subjects in order to obtain the normality ranges and the specific thresholds to be used to discriminate the impaired muscle synergies. For each metric, a cut-off point of the mean −2·SD was chosen to define a threshold common to all muscle synergies. A patient's muscle synergy was defined as impaired when at least one of the metrics resulted under threshold.

#### Definition of the Personalized Biomimetic Stimulation Strategy

The individual muscle activations were reconstructed from the representative physiological muscle synergies by multiplying the mean muscle weights and the mean activation profiles of the synergies that resulted to be impaired in the patient as follows: EMG <sup>N</sup>×<sup>100</sup> = Whealthy N×<sup>J</sup> ∗ HhealthyJ <sup>×</sup> <sup>100</sup> where N is number of considered muscles (N = 8) and J the number of impaired synergies (J ≤ 4).

To avoid excessive fatigue due to FES, the stimulation of each muscle was set to zero when the profile was lower than a threshold defined as the value of the baseline plus the 20% of the peak-to-peak amplitude. Finally, when muscles were characterized by very similar activation profiles, if possible, they were grouped to be activated by a single stimulation channel using stimulation electrodes covering both muscles. The stimulation strategy modulated the stimulation pulse width between 0 and a predefined maximum value of 400µs. The stimulation frequency was common to all muscles and was set to 20 Hz, whereas the stimulation amplitude was identified individually for each muscle, during an initial calibration procedure, in order to produce a visible contraction without discomfort. To identify the stimulation amplitude a pulse width of 400µs was used.

The control system adapted the predefined biomimetic stimulation strategy to changes in walking speed within session. Indeed, when a subject entered a new gait phase, the average of its duration over the last five strides was computed. This estimate was used to stretch or expand the corresponding part of the stimulation profile in order to fully adapt to the subject's gait timing.

### A Preliminary Evaluation of the FES Treatment Effect on Two Chronic Stroke Patients

Two patients with chronic hemiparesis due to ischemic stroke (**Table 1**) were asked to undergo a 4-week intervention consisting of 30-min sessions of FES-supported treadmill walking three

TABLE 1 | Patient details.


times per week. Each session consisted of 5 min of warming up without FES, 20 min of gait supported by the multi-channel personalized FES controller, and 5 min of cooling down without FES. The patient was asked to select his preferred walking velocity during the warming up phase. Before and after the end of the intervention, two clinical scales were assessed: the motor subscale of the Functional Independence Measure (FIM) which evaluates the patient's motor disability during daily life activities and ranges from 13 to 91 (independent), (Kidd et al., 1995) and the Mini Best test (MBT) which evaluated the dynamic balance and ranges from 0 to 28 (normal balance; Franchignoni et al., 2010). To evaluate specific improvements in terms of walking ability, the same test used to identify the patient's impaired muscle synergies was repeated at the end of the intervention. Both kinematics and EMG data were collected. The mean cadence was computed from the kinematics data. EMG envelopes were computed and the NMF algorithm was applied to extract the muscle synergies as previously described for healthy subjects. At the end of the intervention, the patients were also asked to rate the global perceived effect of the treatment using the global rating change (GRC), which is an 11-point scale (−5 = made things worse; 0 = not changed; 5 = completely recovered; Kamper et al., 2009).

The protocol was approved by the Central Ethics Committee of the Fondazione Salvatore Maugeri (IRCCS) and both patients provided their written informed consent before participation.

### RESULTS

#### Functioning of Multi-Channel Synergy-Based Stimulation Controler Healthy Muscle Synergies

The WHEALTHY and HHEALTHY during overground walking are reported in **Figure 3**. All healthy subjects were characterized by four muscle synergies corresponding to the four gait subfunctions identified in literature: Weight Acceptance (WA), Push Off (PO), Foot Clearance (FC) and Leg Deceleration (LD). The same modular organization, both in terms of spatial composition and temporal recruitment, was obtained during treadmill walking. Indeed, comparing the extracted muscle synergies in the two walking conditions and averaging across subjects, a mean (Standard Deviation, SD) similarity of 0.89 (0.11), a circular cross correlation of 0.94 (0.06), and a time lag of 2 (1) in percentage of the gait cycle were found. This confirms that the two walking conditions share the same muscle coordination and thus it is possible to define both a treadmill and an overground treatment based on the same set of physiological muscle synergies.

The synergies extracted from the overground walking condition were used to perform the two NNR of the EMG envelopes obtained on the paretic side during the patient's pretreatment assessment.

#### NNR of the Patient's EMG Envelopes with the Healthy Synergies

**Figure 4** shows the NNR results obtained for both patients. The obtained VAF values were 0.85 and 0.77 for S1, and 0.90 and 0.84 for S2 when the NNR was applied fixing WHEALTHY and HHEALTHY, respectively.

#### Identification of the Patient's Impaired Muscle Synergies

**Table 2** reports for each metric and each muscle synergy, the thresholds computed on the healthy subjects group (last column) and the values obtained by the two patients during the pretreatment assessment.

The metrics confirmed what was visually observed by the reconstructed synergies: S1 had an impaired spatial composition in the WA synergy (similarity was under threshold for WA), a wider activation timing of PO and LD synergies, and a delayed recruitment of the FC synergy. Concerning S2, a low cross correlation was found for FC and LD synergies. Thus, all four synergies were defined as impaired for S1 and only FC and LD were considered impaired synergies for S2.

#### Definition of the Personalized Stimulation Strategy

The final FES strategy obtained and used for both patients is shown in **Figure 5**. The medial and lateral hamstrings and the medial gastrocnemius and soleus showed similar activation profiles and therefore the FES strategy coupled into one stimulation channel both hamstrings, and the calf muscles into another. Thus, a total of six muscle groups were stimulated.

S1 had a FES strategy based on all four healthy synergies, whereas for S2 only the FC and LD synergies was used to obtain the muscle stimulation profiles. When a reduced number of synergies was used to create the stimulation strategy, a subset of the six muscle groups was stimulated. In particular, the calf muscles, which were not recruited by the FC and LD healthy synergies, were not stimulated for S2.

**Figure 5** also shows the different kinematic patterns of the two patients. Indeed, both patients were characterized by a prolonged double support, but S1 extended the paretic double support phase (gait phase 1) whereas S2 extended the non-paretic double support phase (gait phase 4). In both cases, the FES strategy was able to adapt to these changes in gait pattern, mapping the stimulation profiles accordingly.

For both patients the first (upper panels) and last (lower panels) sessions of the intervention are shown to highlight the differences in gait speed together with slight differences in the kinematic pattern. In the first session of FES-supported gait, S1 presented an impaired kinematic pattern characterized by a double support phase of the paretic leg (phase 1) lasting the 48% of the gait cycle and a very short paretic single support (21% of the gait cycle). In the last treatment session, this kinematic pattern changed: the duration of the paretic double support was the 25% of the gait cycle and the duration of the paretic single support was 29%. These improvements in the kinematic pattern

corresponded to a walking speed that in the last session doubled its value with respect to the first session. Concerning S2, the kinematics pattern in the first session was characterized by a reduced paretic swing phase that was augmented by 52% in the last session, achieving a final duration equal to the 38% of the gait cycle.

### The FES Treatment Effect on the Two Chronic Stroke Patients

Both patients completed the treatment without difficulties and reported a positive global perceived effect of the treatment (GRC was +4 (improved a lot) and +2 (improved) for S1 and S2, respectively). The treadmill speed used in the first and last day of treatment increased from 0.43 to 0.83 m/s, and from 0.38 to 0.68 m/s for S1 and S2, respectively.

The extracted muscle synergies before and after the treatment during the overground walking tests are shown in **Figures 6**, **7** for S1 and S2, respectively. The treatment induced an increase of the number of extracted synergies in both patients from 3 to 4 indicating a general improvement in muscle coordination. The VAF was 0.87 before and 0.88 after treatment, and 0.90 before and 0.93 after treatment for S1 and S2, respectively. The visual comparison between the extracted synergies obtained before intervention for S1 and the healthy synergies (**Figure 6**) suggests that the first extracted synergy (S1-1) resembles the FC synergy except for the GM activation, the second extracted synergy (S1-2) mostly recruits the MG and SO muscles as it is in the PO synergy with an anticipatory activation profile, and the third synergy (S1-3) merges the WA and LD synergies. After treatment, four muscle synergies were found, generally resembling the spatial composition of the healthy muscle synergies in **Figure 3**. An early recruitment of the plantar-flexors is still present in the PO synergy.

Concerning S2 (**Figure 7**), before treatment the first synergy (S2-1) can be associated to the WA synergy, the second (S2-2) seems to be the merging of the FC and LD synergies, and the third (S2-3) seems to be the PO synergy with a slight contribution of the hamstring muscles. After treatment, four synergies were obtained displaying a behavior more similar to healthy subjects; however, an early activation of the plantar-flexors is still visible in PO synergy.

The clinical evaluation of the two patients is reported in **Table 3**. Both patients had a mild motor disability at baseline. In both cases the improvement gained in muscle coordination was not yet enough to be transferred into a significant difference in general motor disability.

the single-stride profile (thinner lines) and the mean profile (thicker line) are reported.

TABLE 2 | Metrics computed on the reconstructed muscle synergies of S1 and S2 for each muscle synergy.


*The last column indicates the thresholds obtained in the group of healthy subjects. The values below the predefined threshold are highlighted in bold.*

## DISCUSSION

The high heterogeneity of stroke patients and the high variability in their response to treatments demand for novel personalized assessment methodologies able to unveil the specific impaired sub-functions to be recovered and for novel training procedures adapted to single subject's disability. Most of the outcome measures used to assess the patients' condition are focused on the overall motor function (such as the walking speed) and do not have the power to discriminate specific impairments that underlie the general functional deficit (Ting et al., 2015). Recent neurophysiological studies recommended the analysis of muscle synergies to finely assess the impairment of the subject, to design rehabilitation treatments personalized on the specific nature of the individual impairment, and to assess the eventual treatment outcome (Clark et al., 2010; Ting et al., 2015). In this scope, we developed a multi-channel FES controller to support gait training based on physiological muscle synergies and we personalized it to the individual impairment obtained by means of a baseline assessment of gait muscle synergies. The proposed treatment is goal-oriented, task-specific, and challenging, since the subjects are asked to walk for 30 min at a comfortable speed. FES patterns have been accurately aligned in time with the gait phases of the subject so to assure the maximal synchronization between the FES input and the volitional activity. Thus, it should harness activity-dependent neuro-plasticity (Lamontagne and Fung, 2004; Gandolla et al., 2014).

FIGURE 5 | The personalized stimulation strategy obtained for S1 (Left panels) and S2 (Right panels) in the first (Upper panels) and last (Lower panels) sessions of the intervention. Same labels as in Figure 3.

In literature, FES controllers have been based on very simple segmentation algorithms able to discriminate between the stance and swing phases and the FES strategy has been linearly mapped through the gait stride adopting sub-optimal time rules in order to automatically deactivate stimulation (Daly et al., 2011). The novelty of our proposed control system lies in the capability to accurately map the subject's gait timing based on the realtime detection of six gait phases. This mapping is able to stretch or extend the stimulation profiles according to the actual duration of all six phases. For instance, if a patient's gait pattern is characterized by an extended double support phase, the stimulation profile of the muscles supporting this phase are extended accordingly in order to follow the correct muscle timing and coordination involved in this phase (see **Figure 5**). This choice avoids the unwanted activation of antagonist muscles in the extended kinematic phase that could increase the instability

of the gait instead of fostering the relearning process. This novel modality to map the individual kinematics is an important personalization factor, since the duration of each gait phase is highly dependent on the patient's level of impairment (Olney and Richards, 1996).

The starting point to design the proposed biomimetic multichannel FES controller was the identification of the physiological set of synergies during walking. Results obtained in healthy subjects confirmed that a common motor-control strategy based on four muscles synergies, was shared across walking conditions (overground and treadmill). The identified muscle synergies corresponded well to the four biomechanical functions proposed in the literature (Clark et al., 2010) and confirmed the equivalence in motor coordination between the two walking conditions (Kautz et al., 2011). The physiological muscle synergies used to design the stimulation strategy were extracted from a group of healthy subjects, which were younger than the average stroke patient; nevertheless, it has been shown that synergies are invariant with aging (Monaco et al., 2010).

The two patients included in the pilot study were quite independent in daily-life activities before treatment and were characterized by a mild gait disability (gait speed higher than 0.8 m/s; Tilson et al., 2010). However, a more specific analysis based on muscle synergies during overground gait highlighted an altered motor coordination characterized by only 3 independently recruited muscle synergies for both patients. The underlying gait abnormality was different for the two patients. S1 merged the control of the proximal extensors with the hamstrings (i.e., the WA and LD synergies) and showed an early activation of the plantar-flexors, which is a typical behavior of stroke patients (Clark et al., 2010). S2 merged the FC and LD synergies and also showed an early recruitment of the plantar-flexors presumably connected to an increased excitability of the monosynaptic stretch reflex (Crenna and Frigo, 1987). These two different impairments in terms of muscle coordination produced a proper personalized treatment that resulted in a FES strategy based on all 4 or just 2 muscle synergies for S1 and S2, respectively. Comparing the first and last session of treatment (**Figure 5**), a different kinematic pattern is noticeable. The first patient was able to reduce by half the duration of his paretic double support phase. This represents a big improvement although it was still higher than the physiological duration, which is about 10% of the gait cycle (Perry and Burnfield, 1992). Analogously, the paretic single support increased by 38% becoming closer to the physiological value (Perry and Burnfield, 1992). The second subject improved the duration of his paretic swing phase. For both subjects the improvements across treatment sessions in gait timing were coupled with a faster speed in the training execution.

The assessment after the end of the intervention showed that both patients improved their muscle coordination; indeed, four muscle synergies were extracted for both of them. The originally merged synergies seemed to regain their distinct role in locomotion control, even if the PO synergy maintained an anticipated recruitment and a prolonged timing in both patients. The benefits of the treatment were more evident for S1 and this was confirmed by the patient's perceived effect; indeed, S1 stated that, thanks to the treatment, he was able to achieve a very good walking improvement (GRC = 4). The improvement in motor coordination was also coupled with a clinically important improvement in the dynamic balance. Indeed, the pre-post change of the Mini Best Test overcame 4 points (**Table 3**) that is the minimal clinical important difference for patients with balance disorders (Godi et al., 2013). Concerning S2, the treatment was able to induce a beginning of neuromotor relearning, but probably the 12 treatment sessions were not enough to translate the motor-coordination progress into relevant clinical improvements, in line with the lower benefit perceived by the patient (GRC = 2).

The duration of the donning and setting up of the FES controller was performed by the physiotherapists and, excluding the first day of training, it lasted an average of 5–10 min, which was compatible with a clinical use.

This study has two main limitations. First, the treatment was tested only with two chronic stroke patients and the results obtained, even if encouraging, should be considered as

TABLE 3 | Comparison of the outcome measures before and after training.


preliminary. Second, the speed used during training was too slow. For instance, the second subject never reached his self-selected speed overground during training. Recent studies demonstrated that combining a 2-channel FES with fast treadmill walking yielded larger improvements in gait mechanics than when FES was combined with self-selected speed treadmill walking (Kesar et al., 2011; Awad et al., 2016). Thus, future studies using this personalized biomimetic FES controller should consider the use of fast speed during training.

#### CONCLUSION

In this study we have developed and tested a personalized multichannel FES controller to support gait rehabilitation after stroke. The treatment was personalized to the specific gait abnormality of each patient. Indeed, once the impaired biomechanical functions were revealed by an assessment based on muscular synergies analysis, the exercise was shaped in order to train only the muscle coordination associated with those biomechanical functions. The muscle-synergy analysis was also exploited to assess the effects of treatment and confirmed to be very effective in identifying improvements in motor coordination. The results presented in this pilot case study were encouraging; however, they should be confirmed by a wider statistical study (e.g., a randomized

#### REFERENCES


controlled study). Additionally, the application of this multichannel FES controller could be extended to people with postacute stroke, whose lack of a well-learnt compensatory strategy by the CNS could help improving the benefits obtained with the proposed treatment in case of chronic patients.

#### AUTHOR CONTRIBUTIONS

SF designed the work, drafted the manuscript, refined the data analysis, interpreted the results, revised and approved the manuscript. NC designed the work, collected the data, carried out the data analysis, interpreted the results and revised and approved the manuscript. EA designed the work, interpreted the results, revised and approved the manuscript. AN designed the work, recruited the patients, supervised the intervention, revised and approved the manuscript. AT performed the clinical scale assessment, carried out the intervention, revised and approved the manuscript. MM revised and approved the manuscript. GF designed the work, revised and approved the manuscript. AP designed the work, revised and approved the manuscript.

#### FUNDING

The work was partially supported by the PRIN project titled "Fall risk estimation and prevention in the elderly using a quantitative multifactorial approach" (grant no.: 2010R277FT) and partly by the project titled "Fall prevention and locomotion recovery in post-stroke patients: a multimodal training towards a more autonomous daily life" funded by the Italian Ministry of Health (grant no.: GR-2010-2312228).

### ACKNOWLEDGMENTS

The authors would like to acknowledge Axelgaard Manufacturing Ltd. for donating the stimulation electrodes.


treadmill and over-ground training. Neurorehabil. Neural Repair 25, 588–596. doi: 10.1177/1545968311400092


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2016 Ferrante, Chia Bejarano, Ambrosini, Nardone, Turcato, Monticone, Ferrigno and Pedrocchi. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Low Latency Estimation of Motor Intentions to Assist Reaching Movements along Multiple Sessions in Chronic Stroke Patients: A Feasibility Study

Jaime Ibáñez 1, 2, Esther Monge-Pereira<sup>3</sup> , Francisco Molina-Rueda<sup>3</sup> , J. I. Serrano<sup>4</sup> , Maria D. del Castillo<sup>4</sup> , Alicia Cuesta-Gómez <sup>3</sup> , María Carratalá-Tejada<sup>3</sup> , Roberto Cano-de-la-Cuerda<sup>3</sup> , Isabel M. Alguacil-Diego<sup>3</sup> , Juan C. Miangolarra-Page<sup>3</sup> and Jose L. Pons <sup>1</sup> \*

<sup>1</sup> Neural Rehabilitation Group, Spanish National Research Council, Cajal Institute, Madrid, Spain, <sup>2</sup> Sobell Department of Motor Neuroscience and Movement Disorders, Institute of Neurology, University College London, London, UK, <sup>3</sup> Motion Analysis, Ergonomics, Biomechanics and Motor Control Laboratory, Department of Physical Therapy, Occupational Therapy, Rehabilitation and Physical Medicine, Faculty of Health Sciences, Rey Juan Carlos University, Madrid, Spain, <sup>4</sup> Neural and Cognitive Engineering Group, Centro de Automática y Robótica, Universidad Politécnica de Madrid (UPM), Spanish National Research Council, Madrid, Spain

#### Edited by:

Timothée Levi, University of Bordeaux 1, France

#### Reviewed by:

Ren Xu, Guger Technologies, Austria Ning Jiang, University of Waterloo, Canada

> \*Correspondence: Jose L. Pons jose.pons@csic.es

#### Specialty section:

This article was submitted to Neuroprosthetics, a section of the journal Frontiers in Neuroscience

Received: 09 December 2016 Accepted: 28 February 2017 Published: 17 March 2017

#### Citation:

Ibáñez J, Monge-Pereira E, Molina-Rueda F, Serrano JI, del Castillo MD, Cuesta-Gómez A, Carratalá-Tejada M, Cano-de-la-Cuerda R, Alguacil-Diego IM, Miangolarra-Page JC and Pons JL (2017) Low Latency Estimation of Motor Intentions to Assist Reaching Movements along Multiple Sessions in Chronic Stroke Patients: A Feasibility Study. Front. Neurosci. 11:126. doi: 10.3389/fnins.2017.00126 Background: The association between motor-related cortical activity and peripheral stimulation with temporal precision has been proposed as a possible intervention to facilitate cortico-muscular pathways and thereby improve motor rehabilitation after stroke. Previous studies with patients have provided evidence of the possibility to implement brain-machine interface platforms able to decode motor intentions and use this information to trigger afferent stimulation and movement assistance. This study tests the use a low-latency movement intention detector to drive functional electrical stimulation assisting upper-limb reaching movements of patients with stroke.

Methods: An eight-sessions intervention on the paretic arm was tested on four chronic stroke patients along 1 month. Patients' intentions to initiate reaching movements were decoded from electroencephalographic signals and used to trigger functional electrical stimulation that in turn assisted patients to do the task. The analysis of the patients' ability to interact with the intervention platform, the assessment of changes in patients' clinical scales and of the system usability and the kinematic analysis of the reaching movements before and after the intervention period were carried to study the potential impact of the intervention.

Results: On average 66.3 ± 15.7% of trials (resting intervals followed by selfinitiated movements) were correctly classified with the decoder of motor intentions. The average detection latency (with respect to the movement onsets estimated with gyroscopes) was 112 ± 278 ms. The Fügl-Meyer index upper extremity increased 11.5 ± 5.5 points with the intervention. The stroke impact scale also increased. In line with changes in clinical scales, kinematics of reaching movements showed a trend toward lower compensatory mechanisms. Patients' assessment of the therapy reflected their acceptance of the proposed intervention protocol. Conclusions: According to results obtained here with a small sample of patients, Brain-Machine Interfaces providing low-latency support to upper-limb reaching movements in patients with stroke are a reliable and usable solution for motor rehabilitation interventions with potential functional benefits.

Keywords: electroencephalography, motor-related cortical potentials, event-related desynchronization, functional electrical stimulation, stroke, neurorehabilitation

#### INTRODUCTION

Upper-limb function recovery after a stroke is in many cases insufficient despite intensive physical therapy. In order to actually get meaningful functional changes in these patients, it has been suggested that traditional physical therapies need to be paralleled with brain modulation interventions aimed to guide plastic changes in the brain (Belda-Lois et al., 2011).

Experimental neuromodulation paradigms using electrophysiological acquisition and stimulation techniques to produce long-term plastic changes at supraspinal and spinal levels have been proposed to treat motor dysfunction in stroke (Lefaucheur, 2006; Daly and Wolpaw, 2008). Among these, paradigms using Brain-Machine Interfaces (BMI) linking cortical motor-related activity with afferent information from limbs have been used to efficiently induce cortical plastic changes in healthy subjects (Xu et al., 2014b; Kraus et al., 2015) and in patients (Ramos-Murguialday et al., 2013; Várkuti et al., 2013).

The electroencephalographic (EEG) activity over the premotor and motor cortical areas presents characteristic variations in the periods before self-initiated movements. Two main motor related EEG patterns are known to reflect mental states related to motor planning and execution processes: the Bereitschaftspotential (BP; Shibasaki and Hallett, 2006) and the Event-Related Desynchronization (ERD; Pfurtscheller and da Silva, 1999). ERD and BP have been used in BMI experiments aimed to improve motor neurorehabilitation (Bhagat et al., 2016; Grimm et al., 2016; Lopez-Larraz et al., 2016). Previous studies have used these cortical patterns to detect the onset of voluntary movements in healthy subjects with temporal precisions of 200–500 ms with respect to the onset of muscle activations in the limbs (Lew et al., 2014; Xu et al., 2014a). The possibility of identifying this information of motor intentions allows establishing a tight temporal association of movement-specific cortical activations with proprioceptive afferent feedback from the moved limbs for rehabilitation purposes. Based on this idea, previous studies conditioned the cortico-muscular descending tract to the lower-limbs in control subjects and stroke patients by temporally associating motor intentions to perform analytical ankle movements with electrical or mechanical stimuli (Xu et al., 2014b). These studies showed that significant plastic changes were visible after a single session intervention if small latencies between the cortical activations and the peripheral stimuli were maintained. Moreover, this intervention concept has proven to be potentially beneficial for stroke rehabilitation (Mrachacz-Kersting et al., 2016, 2017).

Mostly, studies of low-latency detectors of motor intentions with the upper limb have only been carried out in offline conditions. Moreover, no interventions so far have tested BMI platforms decoding pre-movement BP and ERD patterns online in patients with brain damage due to a stroke. While BP detections online in healthy subjects doing ankle dorsiflexions have demonstrated to be reliable for BMI approaches, BPs in upper-limb movements (Hadsund et al., 2016; Martínez-Expósito et al., 2017), and specially in stroke patients (Daly et al., 2006) present particularities that make them less reliable for BMI applications, which may limit their usability in BMIs. Recently, it was demonstrated that an appropriate combination of BP- and ERD-based classifiers could lead to reliable and low-latency estimation of stroke patients' upper-limb motor intentions (Ibáñez et al., 2014a,b). In addition, it was shown that the use of Functional Electrical Stimulation (FES) can assist patients to perform functional complex (multi-joint) movements (Resquin et al., 2016). Here it is hypothesized that, in patients with a stroke and chronic arm motor dysfunction, the possibility of timely matching motor intentions with FES assisting specific motor functions opens a window for targeted neuromodulation interventions aimed at improving function-specific motor neural circuits. The simultaneous neuromodulation of ERD and BP phenomena might induce changes in cortical activity related to both motor planning and execution, unlike the existent approaches, thus boosting neurorehabilitation. To achieve this, it needs to be assessed the feasibility and impact of EEG-based (ERD + BP) low-latency decoder of motor intentions triggering FES in upper-limbs in patients along a certain period of time.

In this study, the ability of a BMI system to assist upperlimb functional movements of stroke patients based on their pre-movement cortical changes decoded online and on a singletrial basis was tested for the first time. Moreover, the study reports results of a multisession intervention (eight sessions during 1 month) using FES to assist upper-limb reaching movements of four chronic stroke patients when EEG-based lowlatency estimations of motor intentions are detected. The BMI system performance, patients' functional changes as well as their subjective reports regarding the received intervention are used to discuss on potential benefits of the proposed intervention.

#### MATERIALS AND METHODS

#### Patients

Four chronic stroke patients (age 54 ± 12 years, mean ± SD; all males) with a lesion in the territory of the middle cerebral artery and a predominance of brachial hemiparesis were recruited for this study (see details in **Table 1**). Patients met the following inclusion criteria: (i) ability to manipulate most objects; (ii) spasticity less than or equal to two in the


TABLE 1 | Patients' clinical data.

Modified Ashworth Scale; (iii) ability to understand instructions and actively participate in tasks. Patients with cognitive decline, sensory aphasia, visual impairment, behavioral disorders, articular rigidity, irreversible contractures and dysmetria, and those who had been treated with botulinum toxin or baclofen <6 months before the start of the study were excluded from the study.

The experimental protocol for this study was approved by the Ethical Committee of the "Universidad Rey Juan Carlos" (Alcorcón, Spain) and warranted to be in accordance with the Declaration of Helsinki. All patients signed a written informed consent.

#### Study Protocol

All experiments were carried out in a sound- and light-attenuated ward of a clinical university. During BMI interventions, patients were seated on comfortable seats and with their arms resting on a desk and movements were performed with the affected upper limbs of the patients.

Patients participated in 10 sessions carried out in different days during 1 month (see **Figure 1**). The whole intervention with each patient consisted of eight BMI-FES sessions (two sessions per week). Two additional sessions, right before and after the intervention phase, were scheduled to assess patients' functional evolution and their subjective evaluation of the received intervention (the latter only being carried out at the end of the whole process).

#### Assessment Sessions

#### Clinical Scales

Clinical experts performed functional tests in the first and last sessions to analyse possible patients' improvements.

Upper-extremity sensorimotor function was assessed using Fügl-Meyer Assessment for Upper Extremities (FMA-UE). The four domains evaluated included: upper-extremity motor function (maximum score = 66), sensory function (maximum score = 12), passive joint motion (maximum score = 24), and joint pain (maximum score = 24). Items were scored on a 3 point ordinal scale from 0 (cannot perform) to 2 (performs fully). Summative scores were generated for each domain, scores ranged between 0 and 126 (Duncan et al., 1992; Wagner et al., 2008).

The Stroke Impact Scale 16 (SIS-16) was used to assess patients' health status following stroke. Duncan et al. (2003) developed the SIS-16 to assess physical function in patients with stroke using items from the composite physical domain of the Stroke Impact Scale (SIS) version 3.0. The SIS-16 can differentiate lower levels of disability. The SIS-16 consists of 16 items: seven activities of daily living items, eight mobility items, and a single hand function item. Each item is rated in a 5-point Likert scale in terms of the difficulty the patient has experienced in completing each item. Summative scores are generated for each domain. Scores range from 16 to 80 (Duncan et al., 2003).

#### Kinematics Analysis

To analyse kinematics, patients performed five repetitions of the reaching task while being measured with a motion capture platform based on the optoelectronic system Vicon Motion (Oxford Metrics, Oxford, UK). Patients sat on a comfortable chair close to a desk. The patient-to-desk distance was 8-10 cm and the angle of the chair was 90◦–100◦ . In the starting position the patient's trunk rested firmly against the back of the chair. Patients were asked to put their hands on the desk (palms down) with shoulder at around 20◦ of abduction and elbow at around 95◦ of flexion. A hard plastic glass (diameter = 5.5 cm, height = 15 cm) was used as target. The glass was placed on the desk in line with the patient's sternum and at a distance equal to 75% of the maximum reachable distance with the paretic arm.

Patients were instructed to reach the glass from the starting position using their paretic hand. All patients practiced the reaching task before motion capture trials. Once this phase was completed, a static calibration recording was performed. Using this recording, it was checked that each marker was visible from the scanning cameras and analyzed movements were registered. In these, after the verbal instruction "Get ready...go," patients had to lift the arm and reach and grasp the glass at a comfortable speed (similar to the one used in the BMI-FES interventions). Three seconds after reaching the target patients had to move back to the initial position. The time needed to perform the movement was defined as the time interval between the hand movement onset until the hand reached the glass. We analyzed the shoulder, elbow and thorax positions when the hand reached the glass.

#### Satisfaction Assessment

We evaluated patients' perceived comfort and acceptability of the BMI-FES platform. Five items are rated on a Likert-type scale from 1 to 5 (strongly disagree—strongly agree): (1) "you are satisfied with the intervention"; (2) "this intervention has been useful in order to carry out activities of daily living"; (3) "you would recommend this intervention to other subject in the same situation"; (4) "The instrumentation is uncomfortable." The arithmetic mean across all items provides the total satisfaction score.

### Intervention Sessions BMI-FES Platform

During the intervention sessions movements of the paretic arm were measured with solid-state gyroscopes, which allowed easy and robust recordings of transitions between resting and movement phases (Ibáñez et al., 2014a). Two gyroscopes (Technaid S.L., Madrid, Spain) were placed on the distal third of the forearm, and the middle of the arm. Data were sampled at 100 Hz and stored in a PC running a real-time OS (QNX Software Systems, Ottawa, Canada).

EEG signals were recorded from 31 positions (AFz, F3, F1, Fz, F2, F4, FC3, FC1, FCz, FC2, FC4, C5, C3, C1, Cz, C2, C4, C6, CP3, CP1, CPz, CP2, CP4, P3, P1, Pz, P2, P4, PO3, PO4, and Oz, all according to the International 10–20 system) with active Ag/AgCl electrodes (Acticap, Brain Products GmbH, Germany). The reference was set to the voltage of the earlobe contralateral to the arm moved. AFz was used as ground. The signal was amplified (gUSBamp, g.Tec GmbH, Austria) and sampled at 256 Hz. A standard PC was used to acquire and process the EEG data using a custom-made Simulink model (The Mathworks Inc., Natick MA, USA). This PC sent digital signals to the real-time PC using a USB DAQ (USB-6008, National Instruments, Austin TX, USA).

FES was delivered at the anterior deltoids, triceps and wrist extensors with a multichannel monopolar neurostimulator with charge compensated pulses (UNA Systems, Belgrade, Serbia). Traditional surface electrodes (Pals Platinum—rectangle 5 × 5 cm) were used. The common electrode was located on the oleocranon. Pulse width and frequency were set to 350 µs and 30 Hz, respectively. FES current was adapted in each session with each patient to achieve comfortable stimulation levels that elicited muscle contractions. Current values ranged between 20 and 50 mA (depending on the motor threshold of each muscle and to the patients' acceptance of the received stimulation). Due to the FES configuration and to the weight of the patients' arms, FES alone was not able to lift the arm unless it was successfully triggered by the BMI when patients attempted to perform the reaching task (in which case it provided assistance to the attempted movement). The stimulator was controlled by the PC storing gyroscopic data, which in turn received activation commands from the computer recording the EEG activity via a digital signal. Each time FES was activated it was done in a sequential manner (first deltoids and 250 ms later triceps and wrist extensors) so that the arm could first be lifted from the table and then extended toward the target.

#### EEG-Based Detection of the Motor Intentions with Low Latencies

The classifier used to detect movement intentions from EEG was based on the one presented in Ibáñez et al. (2014a). A logistic regression was used to detect the onset of the voluntary movements based on the characterization of the ERD and BP cortical patterns observed in patients (**Figure 2**).

#### **Detection of the onsets of movements from gyroscopes**

Locations of onsets of voluntary movements were estimated based on the gyroscopic signals. For this, gyroscopic recordings were low-pass filtered (Butterworth, order 3, <10 Hz). For each patient, the sensor that first showed changes during the execution of reaching movements (between the one placed on the forearm and the one on the arm) was used. The peak amplitude performing the movements was estimated in each session. A threshold amplitude for the detection of the onsets of the movements was set to 5% of this peak amplitude. Finally, visual correction of the detected onsets was carried out to ensure that involuntary and residual movements were not taken into account for training and BMI validation purposes.

#### **Feature extraction**

EEG signals recorded in the pre-intervention calibration trials in each session were used to extract the best features for the posterior online decoder.

For the ERD detection, band-pass filtering (Butterworth, 3th order, 6 Hz < f1, 35 > f2) and small Laplacian filtering were

applied to the EEG signal. Power values in segments of 1.5 s and in the frequency range 7–30 Hz (with steps of 1 Hz) were obtained from the frontal, fronto-central, central, centro-parietal, and parietal channels. The Welch's method was used to get power estimations (Hamming windows of 1 s; 50% overlap). The estimations in the training trials from –3 to –0.5 s (with respect to the movement onsets) were labeled as examples of the resting state. Estimations at the movement onsets were labeled as examples of the movement state. The Bhattacharyya distance was used to select the 10 best channel-frequency pairs to build the classifier, i.e., the 10 with the largest distance between the resting state and the movement onset estimations.

For BP, a Butterworth low-pass filter (1 Hz > fc, 1st order) was applied to extract the low-frequency components of EEG signals. A modified version of the large laplacian filter using as reference the average information from eight peripheral channels in the EEG electrode layout was used in order to minimize the weight of individual reference channels (Ibáñez et al., 2014a). Three virtual channels were generated by subtracting the average recordings of channels F3, Fz, F4, C3, C4, P3, Pz, and P4 to channels C1, Cz, and C2. These three de-referenced central channels were considered since the late part of the BP typically presents a lateralization in upper-limb movements (Shibasaki et al., 1980) and the spatial distribution of motor cortical activations in stroke patients may be altered due to their brain lesions (Serrien et al., 2004). The average BP of the resulting channels was obtained using the training data. The channel showing the highest peak at the movement onset relative to the average amplitude in the interval [−3,−2] s (with respect to the movement onset) was selected for BP-based detection of movement onsets.

#### **Classifier construction**

A naïve Bayes classifier of independent features was used to detect the ERD pattern preceding the onset of the reaching movements by using 10 channel-frequency pairs previously selected.

A matched filter of length 1.5 s was designed using the selected channel for BP detection. The filter was obtained by removing the baseline level (first 500 ms) of all 1.5 s trials in the training dataset and then averaging the BPs.

To train the logistic regression classifier that combined ERD- and BP-based estimations of motor intentions, training examples of the resting condition were taken from outputs of both classifiers (ERD and BP) between −3 s and −0.5 s, and estimations between ± 125 ms with respect to the onsets of movements were used to model the movement state.

#### **Online decoding**

In the online decoding phase, the logistic regression classifier generated estimations of movement intentions every 100 ms. The decoder yielded a binary output depending on whether the probabilistic output from the logistic regression classifier was over or under a certain threshold, which in turn activated the FES. The threshold was obtained from the training dataset and, if needed, it was further adjusted based on the reports of the patient in a few number of pre-intervention calibration trials. The threshold was initialized following the criterion of maximizing the percentage of good trials (GT), i.e., trials with a true positive (TP), and with no false positives (FP). TP were movements detected by the BMI with a detection latency within the range of ±750 ms with respect to the reference onsets estimated with the gyroscopes. EEG-based movement intention detections during resting phases were considered FPs. The precision of the detector was characterized by computing the number of FP per minute (FP/min). The percentage of GT was obtained by counting the amount of trials with no FP and a TP. Finally, latencies of the TP with respect to the onsets of movements were computed to analyse the temporal accuracy of the system. The definition of all these metrics is further elaborated in Ibáñez et al. (2014a).

To achieve a stable BMI system, outputs of the BMI were processed by a block ensuring that consecutive FES stimuli were separated by at least 5 s of time.

#### BMI-FES Intervention and FES Configuration

Once the BMI system had been calibrated, the intervention phase of the session begun. Patients performed 60–80 movements assisted with FES triggered by the BMI. The specific amount of trials performed in each session depended on the patients' arousal and their willingness to continue. Patients were allowed to rest and talk in the middle of the sessions if they needed to. Throughout the intervention trials, patients were asked to concentrate and have the FES activated with their movements.

Each time a trigger from the BMI was received by the PC controlling the FES system, the stimulation pattern described in Section BMI-FES Platform was triggered.

#### Validation of the BMI Performance

In order to validate the performance of the EEG-based decoder of motor intentions during the interventions, the GT and the detection latencies during the intervention trials were computed. To facilitate the evaluation of the BMI system, patients were instructed not to perform movements when FES stimulation arrived before they have planned to start the movement. The times at which FES stimuli were triggered were compared to the onsets of movements according to the data from gyroscopes.

#### Statistical Procedures

All statistical analyses were performed using SPSS 17.0 (IBM Corp., New York USA) and Matlab2011 (The Mathworks Inc., Natick MA, USA). Due to the small sample size, Shapiro-Wilk test was applied to check normality of BMI performance and clinical scores. Given that all analyzed samples violated the statistical normality, Wilcoxon signed-rank test was used to compare the clinical scales scores before and after the intervention. The Friedman test for repeated measures was used to compare the BMI performance scores between sessions. Only the sessions with data from all patients were included in the analysis. A linear least square fitting was applied to estimate the tendency of the BMI performance measures along sessions, obtaining the squared error R 2 and the gradient of the line m.

All results are reported as the mean ± SD, and considered significant if P < 0.05.

### RESULTS

### Patients' ERD and BP

ERD and BP patterns were used by the BMI platform to control FES assistance. **Figure 3** shows the average (across sessions) ERD and BP patterns of each patient taking part in the experiments. Fieldtrip's ft\_multiplotER and ft\_multiplotTFR functions were used to obtain the patterns (Oostenveld et al., 2011). As shown in the figure, there is a large variability between patients in terms of the magnitude of the ERD and BP patterns and in terms of their spatial, frequency, and temporal distributions. In all cases, ERD and BP start before the onset of the movements. In addition, it is observed that in all cases, the BP minimum peak is delayed several hundreds of milliseconds with respect to the movement onset. Neither the laterality nor the degree of change of the ERD/BP patterns showed a correlation with the patients' upper limb function.

#### BMI Performance

Average percentages (across sessions) of GT for P1, P2, P3, and P4 were 67.4 ± 15.5, 52.8 ± 6.7, 81.1 ± 12.1, and 66.1 ± 14.8%, respectively (left panel in **Figure 4**). In the best session for each patient (green bars in the left panel of **Figure 4**), GT results were 80.9% (P1), 64.4% (P2), 91.7% (P3), and 81.2% (P4). The average TP rate and number of FP per session (considering all stored sessions and patients) were 71.1 ± 19.5% and 8.1 ± 4.9 FP/session, respectively. No statistically significant differences in any measure were found between sessions for all patients [GT: χ 2 (5) = 5.875, p = 0.320; TP: χ 2 (5) = 4.000, p = 0.549; FP: χ 2 (5) = 5.109, p = 0.403].

Results improved along sessions in three cases (see **Table 2**). TP and GT showed an increasing tendency along sessions for all patient, especially P1, P4, and less markedly in the first sessions of P2. On the contrary, FP showed increasing tendency for P1 and P2 and decreasing for P3 and P4 (see **Table 2**).

The average detection latencies (considering all sessions) were 202 ± 266 ms (P1), 130 ± 316 ms (P2), 3 ± 190 ms (P3), and 103 ± 254 ms (P4). No pair of sessions differed statistically in the average detection latency [χ 2 (5) = 7.587, p = 0.164].

A tendency toward smaller detection latencies could be observed in all patients (P3, P4, and less in P1, P2) when analysing the evolution along the different sessions (**Table 2**; right panel, **Figure 4**).

Unsuccessful results of the BMI-FES intervention were only observed in the sixth session with patient P2 (unreliable

estimations of motor intentions were generated in that case). As a result of this unreliable BMI function, this intervention session was interrupted since the patient reported an uncomfortable interaction with the FES.

**Figure 4** also includes information on how patients were able to control the BMI-FES interface by performing imaginary instead of actual movements (left panel, blue bars). This condition was tested at the end of the experimentation to ascertain that motor-related activity was robust enough to trigger FES regardless of whether it was accompanied by overt movements. In all cases, GT for motor imagery condition were similar but lower than GT for non-imagined movements.

#### Changes in Functional Scales

**Table 3** summarizes the observed changes in the two evaluated functional scales (FMA-UE and SIS) after the intervention period. The FMA-UE score increased in 11.5 points after the intervention, with increases in all patients being observed. All patients showed improvements in the passive range of motion and sensation in FMA-UE scores. In addition, three patients (P1, P2, and P4) showed increases in motor function scores. The SIS score presented an average 10.5 points increase after the intervention. No significant differences in any of the two scales were observed (p = 0.114 and Z = −1.826 for SIS changes; p = 0.068 and Z = −1.461 for FMA-UE).

#### Analysis of the Kinematics

**Table 4** reports joint positions (degrees) when the affected hand reached the glass during the kinematics assessments. On average, after the interventions shoulder flexion was slightly increased (0.7◦ ) and shoulder abduction was reduced (5.45◦ ). Additionally, elbow and thorax flexion were reduced (2.09 and 4.97◦ , respectively). There were no significant differences after the intervention in any of the joint angular rotations measured (p = 0.465 and Z = −0.730 for shoulder and elbow flexion changes; p = 0.273 and Z = −1.095 for shoulder abduction changes, p = 0.068 and Z = −1.826 for thorax flexion changes).

#### Usability Assessment

The perceived comfort and acceptability of the intervention platform proposed here varied across patients. Three patients (P1, P2, P3) "agreed or strongly agreed" with the intervention, while P4 "neither agreed nor disagreed" with it. One patient (P2) reported that the intervention was useful in order to carry out activities of daily living ("strong agreement"), but the other three patients reported "strong disagreement" (P3, P4) or "neither agreement nor disagreement" (P1) in this regard. Regarding the degree of recommendation of the received intervention; all participants declared to "agree or strongly agree." Finally, two participants (P1 and P4) reported to be in "agreement or strong agreement" with the instrumentation process carried out during the intervention; however, the other two participants indicated to be in "disagreement or strong disagreement."

#### DISCUSSION

This study has tested for the first time the ability of a BMI system to assist upper-limb functional movements of stroke patients based on their pre-movement cortical changes decoded online

TABLE 2 | Linear least square fitting parameters (m, slope; R 2 , squared

error) of the BMI performance measures along sessions.


and on a single-trial basis. Such use of anticipatory EEG activity allows the timely assistance of patients during the motor tasks performed. The study aimed to evaluate the usability of the proposed technology and its potential effects when applied in a prolonged in time intervention. Overall, patients could reliably control the interface by spontaneously performing movements and low average detection latencies (<200 ms) were obtained. Moreover, measured FMA-UE changes were higher than the minimal clinically important difference.

Most BMI interventions involving patients with a stroke have used synchronous paradigms, that is, patients performed movements paced by an external signal, and ERD patterns were used to characterize the movement phases (Ang et al., 2014; Kraus et al., 2015; Bhagat et al., 2016; Irimia et al., 2016). This approach allows a more reliable function of BMIs (FP can be avoided), but it can underestimate the relevance of the temporal coupling between patients' intentions of movements and the perceived afferent feedback. In a series of studies by Mrachacz-Kersting et al., the temporal association between motor intentions and peripheral stimuli proved to be relevant in order to efficiently guide cortical changes related to ankle movements (Mrachacz-Kersting et al., 2012, 2016). While some of these studies involved BMI approaches with healthy subjects, interventions with stroke patients typically used visual cues and fixed (patient-specific, according to BP) stimulation onsets. Such offline BMI-like approach is an excellent solution for practical and robust EEG-based neuromodulation interventions, but it assumes that movements are always performed in identical conditions with respect to the external cues, and hinders the demands on patients for planning movements actively. Being able to have a BMI system asynchronously providing reliable and timely estimations of motor intentions allows higher adaptability to inter-trial changes in movement-related cortical activities and gives rise to using ecological rehabilitation scenarios where patients cannot automatize the task performance according to external guides. In this regard, the present study is an original attempt to demonstrate the suitability of purely asynchronous BMIs for motor neurorehabilitation after a stroke.

Importantly, although a small sample is considered here, results are comparable (and better in some cases) to those


TABLE 4 | Analysis of reaching movement kinematics before and after the intervention (values represent joints' rotations in degrees).


obtained in previous similar studies with healthy subjects (Ibáñez et al., 2014a; Xu et al., 2014b; Lin et al., 2016), and are the first demonstration of an online low-latency BMI system tested with patients. The average detection latencies obtained in this study are slightly higher than those obtained in a previous study using the same EEG-based classifier in patients with stroke (112 ± 278 vs. 35.9 ± 352.3 ms). This is probably due to the fact that the previous study was carried out offline while this present study used the BMI online to trigger FES. Since FES was programmed to support patients' movements, it likely had a priming effect on forthcoming movements, i.e., a stimulus arriving when the patient was about to move but with anticipation would in turn anticipate the patients' generation of the intended movement. In any case, detection latencies were in general low enough in order to expect facilitatory effects in the motor cortex (Xu et al., 2014b).

Importantly, the potential impact of motor-related artifacts in the obtained BMI performances, although possible, is estimated to be small. Muscle artifacts, on the one hand, lead to increases of the cortical activity in frequencies within the beta band, which are in the opposed direction to ERD changes learned from the pre-movement EEG signals in the training stage. On the other hand, post-movement low frequency components in the EEG are typically spread along the scalp (which allows spatial filtering techniques to cancel them) and also they typically start with positive changes of the EEG amplitudes (Shibasaki et al., 1980). Furthermore, results obtained with the motor imagery condition tested in the last intervention session are comparable (although smaller) to those obtained with actual movements, which implies that patients with very limited motor capabilities could have the possibility to interact with the proposed BMI-FES platform.

Patients' ERD and BP were marked and started around 0.5– 1 s before movements could be observed from gyroscopes. Both patterns showed alterations compared to ERD and BP patterns in healthy subjects in terms of spatial and temporal distributions and in line with previous findings (Serrien et al., 2004; Daly et al., 2006; Fang et al., 2007). ERD patterns tended to show a higher involvement of cortical areas around the vertex (P3 and P4) or of contralesional areas (P2). Regarding the temporal characteristics, BP in patients showed a delayed peak hundreds milliseconds after the actual onset of the movement, in line with previously published studies (Daly et al., 2006). This delayed peak makes it more challenging to generate estimations of intentions to initiate voluntary movements with temporal accuracy. In this study, only the BP part that preceded the movements and finished at t = 125 ms was used to model the movement intention class. However, the implications of this decision in the hypothesis that afferent stimulation has to be triggered at the BP peak are not clear, and therefore further research should be carried out to describe the role of post-movement BP parts in patients with stroke. In this study, the validity of the stimuli timings was given not only by their comparison with the actual movements (recorded with gyroscopes) but also by patients' reports indicating that, in most cases, they perceived FES in time with their attempts to perform the reaching task.

Changes in patients as a result of the intervention were observed in terms of changes in the BMI performance across sessions. GT in patients P1, P2, and P4 improved in the first 3–4 sessions, and remained high in P3. In addition, average detection latencies decreased with sessions in P3, P4, and less consistently in P2. These results indicate that patients were able to modulate preparatory cortical activity that released the movements, and therefore reinforces the idea that asynchronous BMI approaches as the one here are suited to reinforce and maximize motor planning in stroke.

Apart from changes in BMI performance, no other neurophysiological changes are described here. Using ERD and BP phenomena simultaneously gives raise to the hypothesis that the intervention might induce changes in both motor planning and execution concurrently, contrasting the existing approaches that singled only one of the former phenomena out. However, despite changes in EEG due to the intervention period were explored, no consistent results were found in terms of ERD, BP, or power in motor cortical rhythms during rest (results not included here). Such changes, if they exist, should be derived from a larger sample of patients, given the high intra-patient variability in EEG information across days (Shenoy et al., 2006). Changes in cortical excitability derived from non-invasive brain stimulation are not reported here either. Such changes have been observed in other neuromodulation interventions with muscles having large cortical representation areas, but not with proximal muscles as the ones here stimulated. From the authors' perspective, the variability obtained in responses to brain stimuli targeting proximal upper-limb muscles requires further investigation in order to use these metrics to validate BMI systems as the one proposed here.

Regarding functional changes, the analysis of reaching kinematics led to small but consistent results. Reaching movements need an adequate range of motion toward thorax extension, shoulder flexion, and elbow extension. The altered kinematic of reaching movements in stroke are typically characterized by compensatory trunk and shoulder movements (Roby-Brami et al., 2003a,b). Taken together, the positions of joints observed after the intervention pointed to an improved reaching movement execution, with higher shoulder flexion (increased 0.7◦ ), lower elbow and thorax flexion (2.09 and 4.97◦ , respectively). In addition, patients showed a reduction of 5.45◦ in the compensatory shoulder abduction after the intervention period. In general, a more symmetrical reaching pattern could be observed in most cases. Additionally, FMA-UE scores obtained immediately after the intervention were higher than the minimal clinically important difference (MCID) established for the FMA-UE in chronic stroke patients, which ranged from 4.25 to 7.25 points (Page et al., 2012). All participants showed improvements in the total score in FMA-UE. Improvements were found in passive joint motion scores, which may be related to the repeated training of specific movements with assistance. Additionally, motor function and sensation in FMA-UE scores improved, likely reflecting a favorable effect of repeated motor activity using FES to assist upper-limb reaching (Wang, 2007). These results are in line with previous reports suggesting that combined modulation of voluntary movement, proprioceptive sensory feedback, and electrical stimulation can play a relevant role in improving impaired sensory-motor integration by FES therapy (Hara, 2010). The SIS-16 after the intervention was also increased in the patients (10.5 points). This increment was between the MCID range established for the SIS-16 (9.4–14.1 points; Fulk et al.,

#### REFERENCES

Ang, K. K., Guan, C., Phua, K. S., Wang, C., Zhou, L., Tang, K. Y., et al. (2014). Brain-computer interface-based robotic end effector system for wrist and hand rehabilitation: results of a three-armed randomized controlled trial for chronic stroke. Front. Neuroeng. 7:30. doi: 10.3389/fneng.2014.00030

2010). These results point to a positive effect of the intervention in the participants' health status.

With respect to the satisfaction assessments, according to the patients two aspects should be improved in order to carry out this intervention in further studies. First, it is necessary to carry out more sessions and more arm movements to achieve a better learning transfer. Second, the instrumentation has to be simplified because this aspect may generate fatigue and discomfort in the participants.

This study was carried out with a small sample of patients. This implies that, despite consistent changes could be observed across patients, no statistically significant changes were observed in the clinical or functional metrics, likely due to the small statistical power. To demonstrate the benefits of the intervention proposed here as compared to (or in conjunction with) more traditional therapies, future studies in line with the present manuscript should involve larger groups of patients, with samplematched control groups and blind assessments to avoid observer's bias. Moreover, the analysis of the proposed intervention in subacute stroke would also be relevant to test if the functional impact is larger in patients more susceptible to neuromodulation interventions.

#### CONCLUSIONS

This manuscript represents an approach to BMI-FES interventions for the upper limb in stroke patients, exploiting the predictive properties of EEG signals related to motor processes. Results show a potentially beneficial effect of the BMI-FES intervention in terms of clinical scales and kinematic analysis. In addition, the study demonstrates the suitability of the proposed EEG-based decoding algorithms for their use with patients.

#### AUTHOR CONTRIBUTIONS

JI, FM, EM, and JS participated in the design of the experiment, in the data collection process, in the data analysis, and in writing the manuscript. MD and JP designed the experiments, carried out the experimentation, and wrote the manuscript. RC, AC, MC, IA, and JM collected the patients, carried out the experiments, and edited the manuscript.

#### ACKNOWLEDGMENTS

This research has been supported by Spanish Ministry of Science project HYPER-CSD2009-00067. We acknowledge support of the publication fee by the CSIC Open Access Publication Support Initiative through its Unit of Information Resources for Research (URICI).


machine interface (BMI) to an upper-limb exoskeleton for stroke survivors. Front. Neurosci. 10:122. doi: 10.3389/fnins.2016.00122


tasks in people with hemiparesis after stroke. Phys. Ther. 88, 652–663. doi: 10.2522/ptj.20070255


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2017 Ibáñez, Monge-Pereira, Molina-Rueda, Serrano, del Castillo, Cuesta-Gómez, Carratalá-Tejada, Cano-de-la-Cuerda, Alguacil-Diego, Miangolarra-Page and Pons. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Corrigendum: Low Latency Estimation of Motor Intentions to Assist Reaching Movements along Multiple Sessions in Chronic Stroke Patients: A Feasibility Study

Jaime Ibáñez 1, 2, Esther Monge-Pereira<sup>3</sup> , Francisco Molina-Rueda<sup>3</sup> , J. Ignacio Serrano<sup>4</sup> , Maria D. del Castillo<sup>4</sup> , Alicia Cuesta-Gómez <sup>3</sup> , María Carratalá-Tejada<sup>3</sup> , Roberto Cano-de-la-Cuerda<sup>3</sup> , Isabel M. Alguacil-Diego<sup>3</sup> , Juan C. Miangolarra-Page<sup>3</sup> and Jose L. Pons <sup>1</sup> \*

<sup>1</sup> Neural Rehabilitation Group, Spanish National Research Council, Cajal Institute, Madrid, Spain, <sup>2</sup> Sobell Department of Motor Neuroscience and Movement Disorders, Institute of Neurology, University College London, London, United Kingdom, <sup>3</sup> Motion Analysis, Ergonomics, Biomechanics and Motor Control Laboratory, Department of Physical Therapy, Occupational Therapy, Rehabilitation and Physical Medicine, Faculty of Health Sciences, Rey Juan Carlos University, Madrid, Spain, <sup>4</sup> Neural and Cognitive Engineering Group, Centro de Automática y Robótica, Universidad Politécnica de Madrid, Spanish National Research Council, Madrid, Spain

Keywords: electroencephalography, motor-related cortical potentials, event-related desynchronization, functional electrical stimulation, stroke, neurorehabilitation

Edited and reviewed by: Timothée Levi,

Université de Bordeaux, France

\*Correspondence: Jose L. Pons jose.pons@csic.es

#### Specialty section:

This article was submitted to Neuroprosthetics, a section of the journal Frontiers in Neuroscience

> Received: 28 June 2017 Accepted: 05 July 2017 Published: 18 July 2017

#### Citation:

Ibáñez J, Monge-Pereira E, Molina-Rueda F, Serrano JI, Castillo MDd, Cuesta-Gómez A, Carratalá-Tejada M, Cano-de-la-Cuerda R, Alguacil-Diego IM, Miangolarra-Page JC and Pons JL (2017) Corrigendum: Low Latency Estimation of Motor Intentions to Assist Reaching Movements along Multiple Sessions in Chronic Stroke Patients: A Feasibility Study. Front. Neurosci. 11:422. doi: 10.3389/fnins.2017.00422

#### **Low Latency Estimation of Motor Intentions to Assist Reaching Movements along Multiple Sessions in Chronic Stroke Patients: A Feasibility Study**

by Ibáñez, J., Monge-Pereira, E., Molina-Rueda, F., Serrano, J. I., del Castillo, M. D., Cuesta-Gómez, A., et al. (2017). Front. Neurosci. 11:126. doi: 10.3389/fnins.2017.00126

In the recently published article, there were incorrect and missing contents in the Acknowledgments section, which should have read as follows.

### ACKNOWLEDGMENTS

**A corrigendum on**

This research has been supported in part by Grant #H2020-MSCA-IF-2015-700512 from the European Commission (JI), by Spanish Ministry of Science and Innovation, project Associate (799158449-58449-45-514) (JP), by "NeuroMOD (DPI2015-68664-C4-1-R)" (Md and JS) and by project HYPER-CSD2009-00067. We acknowledge support of the publication fee by the CSIC Open Access Publication Support Initiative through its Unit of Information Resources for Research (URICI).

The authors apologize for this error and state that this does not change the scientific conclusions of the article in any way.

**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2017 Ibáñez, Monge-Pereira, Molina-Rueda, Serrano, Castillo, Cuesta-Gómez, Carratalá-Tejada, Cano-de-la-Cuerda, Alguacil-Diego, Miangolarra-Page and Pons. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Closed-Loop Neuroprosthesis for Reach-to-Grasp Assistance: Combining Adaptive Multi-channel Neuromuscular Stimulation with a Multi-joint Arm Exoskeleton

#### Florian Grimm and Alireza Gharabaghi\*

Division of Functional and Restorative Neurosurgery, Department of Neurosurgery, and Centre for Integrative Neuroscience, Eberhard Karls University Tuebingen, Tuebingen, Germany

#### Edited by:

Timothée Levi, University of Bordeaux, France

#### Reviewed by:

Gernot Müller-Putz, Graz University of Technology, Austria J. Luis Lujan, Mayo Clinic, USA

\*Correspondence: Alireza Gharabaghi alireza.gharabaghi@uni-tuebingen.de

#### Specialty section:

This article was submitted to Neuroprosthetics, a section of the journal Frontiers in Neuroscience

Received: 10 February 2016 Accepted: 07 June 2016 Published: 23 June 2016

#### Citation:

Grimm F and Gharabaghi A (2016) Closed-Loop Neuroprosthesis for Reach-to-Grasp Assistance: Combining Adaptive Multi-channel Neuromuscular Stimulation with a Multi-joint Arm Exoskeleton. Front. Neurosci. 10:284. doi: 10.3389/fnins.2016.00284

Stroke patients with severe motor deficits cannot execute task-oriented rehabilitation exercises with their affected upper extremity. Advanced rehabilitation technology may support them in performing such reach-to-grasp movements. The challenge is, however, to provide assistance as needed, while maintaining the participants' commitment during the exercises. In this feasibility study, we introduced a closed-loop neuroprosthesis for reach-to-grasp assistance which combines adaptive multi-channel neuromuscular stimulation with a multi-joint arm exoskeleton. Eighteen severely affected chronic stroke patients were assisted by a gravity-compensating, seven-degree-of-freedom exoskeleton which was attached to the paretic arm for performing reach-to-grasp exercises resembling activities of daily living in a virtual environment. During the exercises, adaptive electrical stimulation was applied to seven different muscles of the upper extremity in a performance-dependent way to enhance the task-oriented movement trajectory. The stimulation intensity was individualized for each targeted muscle and remained subthreshold, i.e., induced no overt support. Closed-loop neuromuscular stimulation could be well integrated into the exoskeleton-based training, and increased the task-related range of motion (p = 0.0004) and movement velocity (p = 0.015), while preserving accuracy. The highest relative stimulation intensity was required to facilitate the grasping function. The facilitated range of motion correlated with the upper extremity Fugl-Meyer Assessment score of the patients (p = 0.028). Combining adaptive multi-channel neuromuscular stimulation with antigravity assistance amplifies the residual motor capabilities of severely affected stroke patients during rehabilitation exercises and may thus provide a customized training environment for patient-tailored support while preserving the participants' engagement.

Keywords: functional electrical stimulation, robot-assisted rehabilitation, feedback, virtual reality, individualized therapy, hemiparesis, upper-limb assistance, hybrid assistance

## INTRODUCTION

The majority of stroke survivors remain dependent on others for activities of daily living due to a dysfunctional upper extremity (Jørgensen et al., 1999; Dobkin, 2005; Feigin et al., 2008). However, when clinically meaningful improvements are achieved, they correlate positively with the dose of therapy (Lohse et al., 2014; Pollock et al., 2014). Many studies, therefore, aimed to further increase the number of task-oriented exercises by applying assistive robotic technologies for stroke rehabilitation; often resulting in improved arm/hand function and muscle strength, albeit respective trials have, as yet, provided only low-quality evidence (Mehrholz et al., 2015). However, critical voices attribute technology-assisted improvements such as these to unspecific influences like increased enthusiasm for novel interventions on the part of both patients and therapists (Kwakkel and Meskers, 2014). In the same vein, when compared to dose-matched conventional physiotherapy, robot-assisted training showed no clinically relevant, additional benefits in controlled trials (Lo et al., 2010; Klamroth-Marganska et al., 2014).

This dilemma might be illustrated, for example, by the most advanced commercially available training system for the upper limb; an active robotic exoskeleton with seven actuated axes (i.e., degrees of freedom) that provides antigravity support for the paretic arm and enables patients with severe impairment to perform task-oriented movements (Klamroth-Marganska et al., 2014; Kwakkel and Meskers, 2014; Brauchle et al., 2015). This device provided slightly more functional gain for the participating stroke survivors, but was less effective in restoring arm strength than conventional therapy (Klamroth-Marganska et al., 2014), probably because it was too supportive when providing assistance as needed during the exercises (Chase, 2014; Brauchle et al., 2015).

In this context, neurophysiological parameters might constitute a means of preserving patient engagement and of avoiding under-challenge. Other studies applied surface electromyography to infer the person's intention to perform a particular movement and used it as a control signal for robotic assistance (Maciejasz et al., 2014). For severely impaired stroke patients, however, who might benefit most from robotic therapy (Klamroth-Marganska et al., 2014; Brauchle et al., 2015), this physiological parameter might be inadequate due to paralysis and/or abnormally co-activated muscles (Wright et al., 2014). Novel robotic devices, therefore, move only when the brain is most responsive to the feedback by the multi-joint exoskeleton (Brauchle et al., 2015). More specifically, patients control the rehabilitation robots with their brain signals—i.e., via motor imagery-related oscillations of the ipsilesional cortex—within the framework of a brain-robot interface (BRI) for stroke rehabilitation (Brauchle et al., 2015). Although, this technique makes it possible to successfully link three-dimensional robotic training to the participants' own effort, some findings also suggest that sustained brain self-regulation for brain-controlled robotic training is challenging and that it may even be characterized by a significant association with the experience of frustration for the participants (Fels et al., 2015). This potential drawback of connecting rehabilitation exercises to physiological parameters might possibly be overcome, if the resources available for coping with the mental load, that occurs in conjunction with BRI technology, are taken into consideration and when the task difficulty is adjusted accordingly (Naros and Gharabaghi, 2015; Bauer and Gharabaghi, 2015a,b; Naros et al., 2016a). At the same time, however, a direct comparison of the perceived workload of BRI tasks and classical rehabilitation exercises on the basis of voluntary muscle contraction suggested that the experience of frustration and over-challenge was taskindependent, thus supporting the notion that the perceived workload was influenced by the characteristics of the individual subject (Fels et al., 2015).

Accordingly, assistance as needed has to be individually adjusted during stroke rehabilitation and, if not used precisely, is constantly confronted with the dangers of both under- and over-challenge, no matter what assistive technology is applied. However, current assisted approaches usually take an all-ornothing approach, e.g., by providing active robotic guidance to complete a movement as soon as the patient failed to reach the defined goal (Klamroth-Marganska et al., 2014); or by triggering functional electrical stimulation (FES) for overt muscle contraction as soon as a predefined physiological state (recorded with EMG or EEG) is achieved (Howlett et al., 2015).

More targeted assistance might, therefore, be necessary during these rehabilitation exercises to maintain engagement without compromising the patients' motivation; i.e., providing support as little as possible and as much as necessary. Along these lines, we explored an alternative approach to classical assistive technology in this feasibility study. Instead of applying standard robot-guided rehabilitation or triggered FES, we minimized the robotic assistance to pure antigravity support while providing performance-dependent, neuromuscular electrical stimulation with subthreshold modulation of individual upper limb muscles. Notably, the robotic assistance was passive, and the electrical stimulation was non-functional, i.e., elicited no overt movement. We hypothesized, however, that this combined, closed-loop approach leads to a wider range of motion than any one of these assistive tools by itself.

## METHODS

Eighteen stroke patients (female/male: 6/12; right/left hemispheric stroke: 13/5, ischemic/hemorrhagic: 13/5; mean age: 56 ± 9.8 [34 69] years) in the chronic phase after stroke (78 ± 55.3 [8 244] months) presented with a severe and persistent hemiparesis. The modified upper extremity Fugl-Meyer-Assessment score (i.e., mean motor UE-FMA score without coordination, speed, and reflexes; Naros and Gharabaghi, 2015) of our group of patients was 15.6 ± 4.9 [9 25]. This study, which was approved by the ethical review committee of the local medical faculty, involved two sessions of reach-to-grasp training with a multi-joint exoskeleton attached to the paretic arm. Each session lasted approximately 30 min. and consisted of 150 trials. The exoskeleton, virtual reality, and task design have been described in detail elsewhere (Grimm et al., 2016) and are cited here.

### Exoskeleton and Virtual Reality

We used a commercially available (Armeo Spring, Hocoma, Volketswil, Switzerland) rehabilitation exoskeleton for shoulder, elbow and wrist joints with seven axes (i.e., degrees of freedom), providing antigravity support for the paretic arm and registration of movement kinematics and grip force. The un-weighing was realized via two springs incorporated into the device. This device could be used to make individual adjustments e.g., of gravity compensation, thereby supporting patients with severe impairment in performing task-oriented practice within a motivating virtual environment. We extended these features in-house by using the real-time sensor data of the exoskeleton to display a three-dimensional multi-joint visualization of the user's arm in virtual reality. This entailed capturing the angles of all arm joints and the grip force from a shared memory block using a file mapping communication protocol. The virtual arm engine was programmed in a Microsoft XNATM framework. The arm model utilized by the engine was constructed as a meshed bone-skin combination with 54 bones (3Ds Max 2010TM, Autodesk). The measured joint angles (accuracy 0.1◦ ) and grip forces of the device were used to modify the bone-vectors of the meshed model according to the movements of the user, thereby providing online closed-loop feedback. The joint angles of the exoskeleton were directly represented in virtual reality, whereas the grip forces were augmented to feedback natural hand function. Prior to each session, participants were instructed to perform a natural reach-to-grasp movement during the task by using distal (elbow) rather than proximal (shoulder) movements. The three-dimensional visualization of the arm was then applied during each task as an implicit online feedback of the movement since explicit information can disrupt motor learning in stroke patients (Boyd and Winstein, 2004; Cirstea and Levin, 2007). Various virtual training paradigms were designed to allow for different rehabilitation exercises resembling activities of daily living.

### Task Design

In this study, participants performed self-paced, threedimensional (in x-, y-, and z-direction) reach-to-grasp movements in virtual space. Patients could interact within the virtual space via the virtual arm representation described above. The position of the virtual arm changed in real-time according to the patient's arm position tracked by the orthosis. The grasping and releasing of the virtual ball was performed by applying force to the grip sensor and opening the hand, respectively. The relationship between the force applied to the grip sensor and the virtual movement was adjusted individually to each user.

After system setup, the exercise was presented on a screen in front of the patient. This exercise consisted of a transfer movement, i.e., a ball had to be grasped in virtual space and transferred to a basket. The position of the ball and the basket in space and in relation to each other was randomly distributed in x- (left to right/ right to left), y- (up to down/down to up), or z-direction (front to back/ back to front). After presenting the objects in virtual space, the patients had to move the virtual hand toward the presented ball. The movements were self-paced and no distinct timing was given. After grasping the ball, threedimensional transfer movements toward a basket were necessary, i.e., the ball had to be grasped, carried to a distant basket and then released again. The timing for this transfer movement was self-induced. The virtual hand could interact with the ball as soon as it entered a defined range around the latter. The ball changed its color according to the hand position (white: out of range, green: possible to grasp, yellow: possible to transfer, red: possible to release). After releasing the ball in the basket, the next exercise started by presenting the next ball randomly distributed in virtual space.

### Closed-Loop Neuromuscular Stimulation

We integrated a neuromuscular electrical stimulation device in the exoskeleton-based training environment (Rehastim, 8 channel stimulator, Hasomed GmbH, Magdeburg, Germany), and applied biphasic square impulses (frequency: 30 Hz, pulse width: 500µs). The stimulation intensity of this integrated neuroprosthesis was updated in a closed-loop, real-time iteration at 60 Hz via a controller area network (CAN)/universal serial bus (USB) port using a custom-made algorithm. This made it possible to stimulate seven different muscles / muscle groups relevant for reaching and grasping, while the output current was adapted continuously for each of them: M. extensor digitorum communis, M. flexor digitorum superficialis, M. biceps brachii, M. triceps brachii, M. pectoralis major, M. infraspinatus/M. teres minor (i.e., muscle group), M. deltoideus pars anterior. In pairs of antagonist muscles/muscle groups, only one of them was stimulated at the same time; i.e., either M. extensor digitorum communis or M. flexor digitorum superficialis, either M. biceps brachii or M. triceps brachii, either M. pectoralis major or M. infraspinatus/M. teres minor. This resulted together with the M. deltoideus pars anterior—in up to four simultaneously stimulated, co-activated muscles/muscle groups (**Figure 1**).

According to a biomechanical movement model (**Figure 2**) on the basis of the vector positions of the virtual arm, the neuromuscular stimulation pattern and intensity was calculated (**Figure 3**).

More specifically, the target vector and the estimated movement vector of each individual muscle group were calculated on the basis of the real-time arm position measured by the exoskeleton, while using the cosine similarity functions between the two vectors for weighting. This function gives a maximum output of 1 when the target vector and the estimated movement vector of the corresponding muscle group are pointing in the same direction. When the vectors point in opposite directions, the function would result in a negative output and is then set to zero, i.e., resulting in no stimulation. For positive outputs, i.e., when target vector and estimated movement vector point in the same direction, the stimulation amplitudes are calculated by multiplying the weighting of the muscle vectors with a Boltzmann-fitting of the time course of a ramping stimulation toward maximum stimulation strength

FIGURE 1 | Set-up of multi-channel neuromuscular stimulation integrated into a gravity-compensating, seven-degree-of-freedom exoskeleton.

(Equation 1). This stimulation began with a 2 s delay to avoid instability of the induced movement.

$$\begin{array}{l} c\_{i}(t) = \left(1 - \frac{1}{\varepsilon^{l}}\right) c\_{i \max} \frac{\overrightarrow{\overline{\boldsymbol{x}\_{l}}} \overrightarrow{\boldsymbol{v}\_{l}^{\*}}}{\|\overrightarrow{\overline{\boldsymbol{x}\_{l}}}\| \|\overrightarrow{\overline{\boldsymbol{v}\_{l}}}\|} \\ c\_{i} \in \mathcal{R}\_{0+}: \text{ simulation current, muscle group i} \\ \boldsymbol{x}\_{lt}: \text{ estimated movement vector, muscle group i} \\ \overset{\scriptstyle \mathsf{X}\_{lt}}{\underset{\scriptstyle l}{\longrightarrow}}: \text{ estimated target vector} \end{array} \tag{1}$$

Each patient performed two consecutive exoskeleton-supported training sessions—one with and one without concurrent stimulation—in randomized order. Both the exoskeleton and the maximum stimulation intensity (Stimmax) were individually calibrated: The exoskeleton was adjusted to provide optimized gravity compensation for every joint and to allow for goaldirected movements in three-dimensional space. The gravity compensation was provided by a spring mechanism of the orthotic device, which was calibrated individually to balance the weight of the patient's paretic arm. Thereby, the exoskeleton was adjusted to the corresponding functional anatomy of the participant before each session. Particularly, the shoulder position, forearm, and upper arm length were considered for the adjustments.

For calibration of the stimulation intensity, the different muscles/muscle groups where identified anatomically before applying the self-sticking FES electrodes (Han-Sen GmbH, Hamburg, Germany; 50 mm diameter and 50 × 80 mm). The electrode positions were optimized by subsequent test stimulation. Since all participants suffered from severe upper limb impairment, prolonged supra-motor threshold stimulation was perceived as painful and ineffective and was therefore not applied. The Stimmax for each muscle group was empirically determined as the output current perceived as comfortable and approaching the motor threshold, but remaining still subthreshold. The motor threshold was identified by a visible joint movement. Each muscle group was stimulated separately before the training session to determine the individual maximum stimulation strength.

#### Outcome Measures

The kinematic assessment included movement smoothness, temporal efficiency, and range of motion (volume). Movement smoothness was captured by calculating changes of movement direction along an optimal direct path toward the targets, by estimating the distance function between the hand-position and the final endpoint, and by calculating the second derivative of the function to determine the number of turning points for each task (Cirstea et al., 2006). In order to avoid compensatory shoulder movements the stimulation patterns for shoulder and biceps activation were equally distributed for an inbound trajectory. Temporal efficiency was captured as the time required to complete each task, and as the mean and peak velocity of the hand between the targets while calculating their distance for x-, y-, and z-directions in virtual units (vu). The overall range of motion (volume, vu<sup>3</sup> ) was measured as the complete space covered during the exercises. The range of sensor data from the grip-sensor was estimated as the mean change in grip pressure. The range of shoulder, upper arm, and forearm movement was measured in degrees.

The maximum (Stimmax) and mean stimulation amplitude (in mA) was calculated per channel, i.e., muscle group. In addition, we captured the stimulation period for different stimulation intensities, i.e., <25% Stimmax, 25–50% Stimmax, >50–75% Stimmax, >75% Stimmax.

#### Statistics

Statistical analysis was performed on a Matlab 2010b Engine. For paired data points a t-test for paired samples was performed. The significance level was set at p = 0.0125 for all tests after conservative Bonferroni correction. Correlation coefficients r and respective p-values were calculated between the overall range of motion and the UA-FMA score.

#### RESULTS

The three-dimensional reach-to-grasp exercises of this study could be completed only with the help of assistive technology. None of the patients was able to complete grasping exercises in unsupported conditions, i.e., they all scored 0 out of a possible 4 points in the related FMA sub-scores ("grasp cylinder," "grasp tennis ball"). However, neuromuscular stimulation alone was not sufficient in our severely impaired patient group, i.e., none of the targeted muscles was stimulated in a functionally relevant way to allow for overt muscle contraction. The patients were, therefore, unable to perform a reach-to-grasp movement per se, even when neuromuscular stimulation was applied. Instead, the multi-joint antigravity assistance was essential to facilitate the goal-oriented grasping exercises in the 3D-virtual environment and required commitment from the patients.

Fifteen of the patients were able to complete all 150 trials in each session. The amount of training had to be reduced for three patients, two of whom completed 75 trials and the third 50 trials in each session. This resulted in a group mean of 135 trials (±32.2808, [50 150]). The reach-to-grasp direction was randomly distributed (x: 44.4 ± 12.1 vu, y: 45.1 ± 11.2 vu, z: 44.1 ± 12.6 vu).

Closed-loop neuromuscular stimulation could be integrated well into the exoskeleton-based training; this neuroprosthesis increased the task-related range of motion (ROM) in 16 out of 18 participants as well as the mean ROM of all patients (p = 0.0004). More distant targets in the virtual training space were achieved in all three x-, y-, z-directions and the participants were able to perform longer transfer movements, i.e., inter-target distances (**Table 1**, **Figure 4**).

Moreover, the movement range of the shoulder, upper arm, and forearm increased significantly in the neuroprosthesis


FIGURE 4 | Comparison of paired plots for mean values of the range of motion (volume) and distances in x-, y-, and z- direction with and without stimulation.

condition, while the grip pressure showed a trend (**Table 2**, **Figure 5**). The range of motion correlated with the upper extremity Fugl-Meyer Assessment score of the patients for both the antigravity orthosis (r = 0.62, p = 0.01) and the neuroprosthetic condition (r = 0.58, p = 0.028).

The neuroprosthesis increased the movement velocity (3.8 vs. 3.5 vu/s, p = 0.015) with a trend toward a faster task completion (6.9 vs. 7.2 s, p = 0.02) while preserving the smoothness of the trajectory (9.3 vs. 9.31, p = 0.46; **Figure 6**).

The stimulation was applied after the self-induced movements of the patients by adapting the intensity in accordance with the output of the dynamic biomechanical arm model and the estimated target vector for each targeted muscle group (**Table 3**, **Figure 7**). Completion of the overall task took on average 6.9 s, while most of the time (mean 4.8 ± 0.6 s, i.e., 82% of the time) no or only minimal stimulation, i.e., <25% of Stimmax, was applied. The same was true for the virtual transfer task of the ball into the basket, which was supported by minimal stimulation, i.e., <25% of Stimmax, in 78% of the trajectory.The highest relative stimulation intensity was necessary to facilitate the grasping function, i.e., the grip strength in transfer movements, by applying stimulation intensities of >75% Stimmaxto the finger flexion muscle for about 22% of the time. The performancedependent adaptation of stimulation resulted in a decrease in both stimulation intensity (on average by 35.4%) and period (on average by 36.77%) in the course of the session, which is suggestive of motor learning.

### DISCUSSION

The present study demonstrated the feasibility of integrating multichannel closed-loop neuromuscular stimulation in an exoskeleton-based training; this neuroprosthesis increased the goal-oriented range of motion and movement velocity while preserving accuracy in chronic stroke patients with a severe impairment of the upper-extremity. The multi-joint exoskeleton for the paretic arm enabled the patients to perform task-oriented practice within a virtual environment (Housman et al., 2009), which they were unable to perform without assistance. Notably, unlike other studies with similarly affected stroke patients, in which robots completed a movement that the patients had begun (Klamroth-Marganska et al., 2014; Brauchle et al., 2015), this hybrid technology delivered antigravity-support only, i.e., provided no active assistance. Thus, the patient engagement was maximized by default in the present study, leaving no room for slacking; the continuous visual feedback of the arm kinematics enabled the patients to adjust their action online during each task; an approach closely resembling natural motor learning.

Such a closed-loop framework adheres to an operant conditioning rationale (Bauer et al., in press), providing contingent feedback to facilitate the targeted activity considered to be beneficial to recovery and which might ultimately lead to functional gain (Bauer and Gharabaghi, 2015a). One drawback of such restorative approaches, however, is that the considerable challenge of these exercises (Fels et al., 2015; Bauer and Gharabaghi, 2015b) might condition the patients to explore alternative, i.e., therapeutically undesirable strategies such as compensatory shoulder movements (Cirstea and Levin, 2000) or co-activation of non-targeted muscles (Gharabaghi et al., 2014b). Moreover, particularly in patients with severe impairments, non-successful trials may frustrate the participants, thereby compromising their motor learning.

In this context, adaptive neuromuscular stimulation, as applied in the present study, may support the exercises by extending the range of motion in accordance with the actual ability of each patient. More specifically, the range of motion correlated with the upper extremity Fugl-Meyer Assessment score of the patients for both the non-NMES and the NMES condition, indicating a targeted assistance of the genuine movement capability of each patient. Importantly, to avoid under-challenge, stimulation was applied adjunct to voluntary contraction and not as an alternative. Moreover, such an additive stimulation approach has proved effective in assisting reaching and grasping exercises in severely impaired, chronic stroke patients for repetitive task practice (Thrasher et al., 2008; Oujamaa et al., 2009; Mann et al., 2011). Unlike these previous approaches, however, our stimulation paradigm was (i) multichanneled, i.e., targeting seven different muscles, (ii) modelbased to follow the three-dimensional movement trajectory, (iii) performance-dependent to enhance task-oriented training, and (iv) subthreshold to avoid slacking:

(i) Previous approaches combining functional electrical stimulation (FES) with mechanical support for the upper limb usually stimulated one or two muscles. Only recently, FES of thee joints, i.e., shoulder, elbow, and wrist, was implemented and shown to be effective in reducing upper limb impairment following stroke (Meadmore et al., 2014). Notably, more functional motor activities of the upper limb could be performed following this intervention: a finding that could not be achieved in an earlier study conducted by the same research group using exactly the same therapy dose (18 sessions, 60 min each) with FES to two proximal muscles only (Meadmore et al., 2012). Future studies will


FIGURE 5 | Comparison of paired plots for mean values of kinematic data (for shoulder movement, upper arm movement, forearm movement, and grip force) with and without stimulation.

#### TABLE 3 | Stimulation parameters.


FIGURE 6 | Comparison of paired plots for mean values of distance to target, velocity, time per task, and smoothness of trajectories with and without stimulation.

reveal whether a more fine-graded targeting of even more muscles—as shown to be feasible in the present study for stimulation-assisted task-oriented 3D exercises—leads to further functional gains when applied repetitively within a multi-session intervention.

(ii) Conventional FES of the upper limb, even when physiologically triggered, follows an all-or-nothing concept. Only few research groups have explored modelbased stimulation paradigms to precisely control FES for goal-oriented movements of the upper limb (Hughes et al., 2009; Meadmore et al., 2012, 2014). The most advanced approach used iterative learning control, which applied data from previous attempts in an effort to update the FES control signal on the current attempt (Meadmore et al., 2014). The reduction of error between real and reference trajectories within a biomechanical model thereby corresponded to improved performance over successive attempts. In addition, the supplied FES was reduced as performance improved to optimize motor learning (Meadmore et al., 2014). Our approach complements this strategy; instead of adjusting the stimulation from trial to trial, we tuned it within each trial. Rather than aiming to reduce the error between actual and reference trajectory with suprathreshold FES, we applied a ramping stimulation which, nevertheless, remained subthreshold throughout the task-oriented attempt (see below). Instead of one (Hughes et al., 2009), two (Meadmore et al., 2012), or three muscles (Meadmore et al., 2014), we integrated a total of seven muscles into our biomechanical model. By adding more muscles a larger number of movement directions could be addressed, thereby, covering a three-dimensional volume with movements in x-, y-, and z-direction. Despite these differences, the supplied multi-channel stimulation was reduced in our feasibility study as well. This could already be observed in the course of one session, suggesting that, even when applied subthreshold, an online adaptation of stimulation has immediate effects on motor learning.

(iii) The performance-dependent stimulation applied in the present study was more subtle than in other approaches. Since no functional muscle contraction was achieved by the actual stimulation, performance was instead captured by gradual modulations of self-initiated, orthosis-assisted movements. It is therefore somewhat surprising that this novel approach resulted in task-adapted stimulation

intensities for each of seven targeted muscles during the task-oriented exercises, e.g., with the highest relative stimulation intensity for facilitation of grip-strength occurring during transfer movements. This information about the different levels of assistance required by the muscles for specific goal-oriented tasks might be used in future studies to customize the exercises and training protocols in such a way as to target specific weaknesses, e.g., particular muscles/muscles groups in the course of a long-term training program.

(iv) Neuromuscular stimulation alone, however, was not enough to facilitate reach-to grasp movements in our severely impaired patient group. Instead, multi-joint antigravity assistance was required to facilitate the task-oriented training in the 3D-virtual environment. Although, none of the targeted muscles was stimulated in a functionally relevant way to allow for overt muscle contraction, the cumulative effect of multi-channel subthreshold stimulation resulted in an increased range of motion and movement velocity while preserving smoothness during the goal-oriented exercises. This finding suggests a general facilitation of sensorimotor networks, which might provide a novel restorative approach in chronic stroke patients with a severe impairment of the upper-extremity. However, it has to be borne in mind that the applied ramping stimulation, based on a Boltzmann-fitting algorithm during each task, led to minimal stimulation intensity, i.e., <25% of Stimmax, during most of the training period. Future studies need to explore whether different adaptive stimulation paradigms may achieve larger kinematic gains, e.g., by applying more neuromuscular stimulation, or by utilizing transcranial direct current stimulation to facilitate exoskeleton-based motor leaning (Naros et al., 2016b).

However, the current approach resulted in kinematic gains while still encouraging effort from the participants. To further generate a challenge for motor learning, the progression of training is necessary (Guadagnoli and Lee, 2004) and might be realized by reducing the FES support level (Meadmore et al., 2014) or by automated adaptation of training difficulty during robotassisted stroke rehabilitation (Metzger et al., 2014). Both of these requirements are integrated into the presented neuroprosthetic set-up and need to be examined in more detail with regard to their respective clinical relevance in the targeted patient population by performing intervention studies with repetitive sessions.

The presented neuroprosthesis sparks hope for a general capacity for even larger gains, e.g., when additional interventions

#### REFERENCES


such as brain state-dependent cortical stimulation (Kraus et al., 2016a) are applied to maximally exploit the salvaged restorative potential. In particular, the task-related and musclespecific facilitation provided by this hybrid device during reach-to-grasp exercises of severely impaired stroke patients, may deliver the framework for concurrent cortical stimulation. Activity-dependent transcranial magnetic stimulation, for example, may constitute such an additional input during robot-assisted training (Gharabaghi, 2015; Massie et al., 2015). Associative brain state-dependent stimulation (Royter and Gharabaghi, 2016) during brain-robot interface exercises has the potential to unmask latent corticospinal connectivity after stroke (Gharabaghi et al., 2014a). The application of such state-dependent stimulation synchronized to maximum gains of assisted range of motion may consolidate the involved corticospinal circuits in accordance with Hebbian-like plasticity rules. More specifically, neuroprosthetic exercises based on brain-robot feedback may result in connectivity changes of cortico-cortical motor networks (Vukeliæ et al., 2014; Vukelic´ and Gharabaghi, 2015a,b) and lead to a re-distribution of cortico-spinal connections (Kraus et al., 2016b). This advanced assistive rehabilitation technology may thereby constitute a back-door to the motor system to further improve the scope for recovery (Bauer et al., 2015).

In summary, combining robotic assistance with adaptive closed-loop neuromuscular stimulation provides customized rehabilitation environments for severely impaired stroke patients, and may increase kinematic parameters while preserving the voluntary effort of patients, during rehabilitation training. Whether these technological refinements also lead to relevant functional gains requires investigation in controlled intervention studies in comparison to dose-matched, conventional physiotherapy.

### AUTHOR CONTRIBUTIONS

FG participated in the study design, software development, data acquisition and analysis. AG participated in the study design and data analysis. Authors jointly drafted and approved the final manuscript.

#### ACKNOWLEDGMENTS

AG was supported by grants from the German Research Council [DFG EC 307], and from the Federal Ministry of Education and Research [BFNT 01GQ0761, BMBF 16SV3783, BMBF 0316064B, BMBF16SV5824].


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2016 Grimm and Gharabaghi. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Closed-Loop Task Difficulty Adaptation during Virtual Reality Reach-to-Grasp Training Assisted with an Exoskeleton for Stroke Rehabilitation

#### Florian Grimm, Georgios Naros and Alireza Gharabaghi\*

*Division of Functional and Restorative Neurosurgery, and Centre for Integrative Neuroscience, Eberhard Karls University of Tuebingen, Tuebingen, Germany*

#### Edited by:

*Paolo Bonifazi, Tel Aviv University, Israel*

#### Reviewed by:

*Agnes Roby-Brami, French Institute of Health and Medical Research (INSERM), France Disha Gupta, Weill Cornell Medical College, USA*

\*Correspondence:

*Alireza Gharabaghi alireza.gharabaghi@uni-tuebingen.de*

#### Specialty section:

*This article was submitted to Neuroprosthetics, a section of the journal Frontiers in Neuroscience*

Received: *25 December 2015* Accepted: *26 October 2016* Published: *15 November 2016*

#### Citation:

*Grimm F, Naros G and Gharabaghi A (2016) Closed-Loop Task Difficulty Adaptation during Virtual Reality Reach-to-Grasp Training Assisted with an Exoskeleton for Stroke Rehabilitation. Front. Neurosci. 10:518. doi: 10.3389/fnins.2016.00518* Stroke patients with severe motor deficits of the upper extremity may practice rehabilitation exercises with the assistance of a multi-joint exoskeleton. Although this technology enables intensive task-oriented training, it may also lead to slacking when the assistance is too supportive. Preserving the engagement of the patients while providing "assistance-as-needed" during the exercises, therefore remains an ongoing challenge. We applied a commercially available seven degree-of-freedom arm exoskeleton to provide passive gravity compensation during task-oriented training in a virtual environment. During this 4-week pilot study, five severely affected chronic stroke patients performed reach-to-grasp exercises resembling activities of daily living. The subjects received virtual reality feedback from their three-dimensional movements. The level of difficulty for the exercise was adjusted by a performance-dependent real-time adaptation algorithm. The goal of this algorithm was the automated improvement of the range of motion. In the course of 20 training and feedback sessions, this unsupervised adaptive training concept led to a progressive increase of the virtual training space (*p* < 0.001) in accordance with the subjects' abilities. This learning curve was paralleled by a concurrent improvement of real world kinematic parameters, i.e., range of motion (*p* = 0.008), accuracy of movement (*p* = 0.01), and movement velocity (*p* < 0.001). Notably, these kinematic gains were paralleled by motor improvements such as increased elbow movement (*p* = 0.001), grip force (*p* < 0.001), and upper extremity Fugl-Meyer-Assessment score from 14.3 ± 5 to 16.9 ± 6.1 (*p* = 0.026). Combining gravity-compensating assistance with adaptive closed-loop feedback in virtual reality provides customized rehabilitation environments for severely affected stroke patients. This approach may facilitate motor learning by progressively challenging the subject in accordance with the individual capacity for functional restoration. It might be necessary to apply concurrent restorative interventions to translate these improvements into relevant functional gains of severely motor impaired patients in activities of daily living.

Keywords: robot-assisted rehabilitation, robotic rehabilitation, individualized therapy, hemiparesis, motor recovery, upper-limb assistance, reinforcement learning

## INTRODUCTION

Despite their participation in standard rehabilitation programs (Jørgensen et al., 1999; Dobkin, 2005), restoration of arm and hand function for activities of daily living is not achieved in the majority of stroke patients. In the first weeks and months after stroke, a positive relationship between the dose of therapy and clinically meaningful improvements has been demonstrated (Lohse et al., 2014; Pollock et al., 2014). In stroke patients with long-standing (>6 months) upper limb paresis, however, treatment effects were small, with no evidence of a dose-response effect of task-specific training on the functional capacity (Lang et al., 2016). This has implications for the use of assistive technologies such as robot-assisted training during stroke rehabilitation. These devices are usually applied to further increase and standardize the amount of therapy. They have the potential to improve arm/hand function and muscle strength, albeit currently available clinical trials provide on the whole only low-quality evidence (Mehrholz et al., 2015). It has, notably, been suggested that technology-assisted improvements during stroke rehabilitation might at least partially be due to unspecific influences such as increased enthusiasm for novel interventions on the part of both patients and therapists (Kwakkel and Meskers, 2014). In particular, a comparison between robotassisted training and dose-matched conventional physiotherapy in controlled trials revealed no additional, clinically relevant benefits (Lo et al., 2010; Klamroth-Marganska et al., 2014). This might be related to saturation effects. Alternatively, the active robotic assistance might be too supportive when providing "assistance-as-needed" during the exercises (Chase, 2014). More targeted assistance might therefore be necessary during these rehabilitation exercises to maintain engagement without compromising the patients' motivation; i.e., by providing only as much support as necessary and as little as possible (Grimm and Gharabaghi, 2016). In this context, passive gravity compensation with a multi-joint arm exoskeleton may be a viable alternative to active robotic assistance (Housman et al., 2009; Grimm et al., 2016a). In severely affected patients, performancedependent, neuromuscular electrical stimulation of individual upper limb muscles integrated in the exoskeleton may increase the range of motion even further (Grimm and Gharabaghi, 2016; Grimm et al., 2016b). These approaches focus on the improvement of motor control, which is defined as the ability to make accurate and precise goal-directed movements without reducing movement speed (Reis et al., 2009; Shmuelof et al., 2012), or using compensatory movements (Kitago et al., 2013, 2015). Functional gains in hemiparetic patients, however, are often achieved by movements that aim to compensate the diminished range of motion of the affected limb (Cirstea and Levin, 2000; Grimm et al., 2016a). Although these compensatory strategies might be efficient in short-term task accomplishment, they may lead to long-term complications such as pain and joint-contracture (Cirstea and Levin, 2007; Grimm et al., 2016a). In this context, providing detailed information about how the movement is carried out, i.e., the quality of the movement, is more likely to recover natural movement patterns and avoid compensatory movements, than to provide information about movement outcome only (Cirstea et al., 2006; Cirstea and Levin, 2007; Grimm et al., 2016a). This feedback, however, needs to be provided implicitly, since explicit information has been shown to disrupt motor learning in stroke patients (Boyd and Winstein, 2004, 2006; Cirstea and Levin, 2007). Information on movement quality has therefore been incorporated as implicit closed-loop feedback in the virtual environment of an exoskeleton-based rehabilitation device (Grimm et al., 2016a). Specifically, the continuous visual feedback of the whole arm kinematics allowed the patients to adjust their movement quality online during each task; an approach closely resembling natural motor learning (Grimm et al., 2016a).

Along these lines, virtual reality and interactive video gaming have emerged as treatment approaches in stroke rehabilitation (Laver et al., 2015). They have been used as an adjunct to conventional care (to increase overall therapy time) or compared with the same dose of conventional therapy. These studies have demonstrated benefits in improving upper limb function and activities of daily living, albeit currently available clinical trials tend to provide only low-quality evidence (Laver et al., 2015). Most of these studies were conducted with mildly to moderately affected patients. In the remaining patient group with moderate to severe upper limp impairment, the intervention effects were more heterogeneous and affected by the impairment level, with either no or only modest additional gains in comparison to dosematched conventional treatments (Housman et al., 2009; Byl et al., 2013; Subramanian et al., 2013).

With respect to the restoration of arm and hand function in severely affected stroke patients in particular, there is still a lack of evidence for additional benefits from technology-assisted interventions for activities of daily living. The only means of providing such evidence is by sufficiently powered, randomized and adequately controlled trials (RCT).

However, such high-quality RCT studies require considerable resources. Pilot data acquired earlier in the course of feasibility studies may provide the rationale and justification for later largescale RCT. Such studies therefore need to demonstrate significant improvements, with functional relevance for the participating patients. Then again, costly RCT can be avoided when innovative interventions prove to be feasible but not effective with regard to the treatment goal, i.e., that they do not result in functionally relevant upper extremity improvements in severely affected stroke patients.

One recent pilot study, for example, applied brain signals to control an active robotic exoskeleton within the framework of a brain-robot interface (BRI) for stroke rehabilitation. This device provided patient control over the training device via motor imagery-related oscillations of the ipsilesional cortex (Brauchle et al., 2015). The study illustrated that a BRI may successfully link three-dimensional robotic training to the participant's effort. Furthermore, the BRI allowed the severely impaired stroke patients to perform task-oriented activities with a physiologically controlled multi-joint exoskeleton. However, this approach did not result in significant upper limb improvements with functional relevance for the participating patients. This training approach was potentially too challenging and may even have frustrated the patients (Fels et al., 2015). The

patients' cognitive resources for coping with the mental load of performing such a neurofeedback task must therefore be taken into consideration (Bauer and Gharabaghi, 2015a; Naros and Gharabaghi, 2015). Mathematical modeling on the basis of Bayesian simulation indicates that this might be achieved when the task difficulty is adapted in the course of the training (Bauer and Gharabaghi, 2015b). Such an adaptation strategy has the potential to facilitate reinforcement learning (Naros et al., 2016b) by progressively challenging the patient (Naros and Gharabaghi, 2015). Recent studies explored automated adaptation of training difficulty in stroke rehabilitation of less severely affected patients (Metzger et al., 2014; Wittmann et al., 2015). More specifically, both robot-assisted rehabilitation of proprioceptive hand function (Metzger et al., 2014) and inertial sensor-based virtual reality feedback of the arm (Wittmann et al., 2015) benefit from assessment-driven adjustments of exercise difficulty. Furthermore, a direct comparison between adaptive BRI training and non-adaptive training (Naros et al., 2016b) or sham adaptation (Bauer et al., 2016a) in healthy patients revealed the impact of reinforcement-based adaptation for the improvement of performance. Moreover, the exercise difficulty has been shown to influence the learning incentive during the training; more specifically, the optimal difficulty level could be determined empirically while disentangling the relative contribution of neurofeedback specificity and sensitivity (Bauer et al., 2016b).

In the present 4-week pilot study, we combined these approaches and customized them for the requirements of patients with severe upper extremity impairment by applying a multi-joint exoskeleton for task-oriented arm and hand training in an adaptive virtual environment. Notably, due to the severity of their impairment, these patients were not able to practice the reach-to-grasp movements without the exoskeleton. The set-up was, however, limited to pure antigravity support, i.e., it provided passive rather than active assistance. Furthermore, it tested the feasibility of closed-loop online adaptation of exercise difficulty and aimed at automated progression of task challenge.

#### METHODS

We recruited five stroke patients (2 female, mean age: 52 ± 9 [from 41 to 63] years) in the chronic phase after stroke (65 ± 59 [from 8 to 156] months) who provided written, informed consent and presented with a severe and persistent hemiparesis (for details, see **Table 1**). The modified upper extremity Fugl-Meyer-Assessment score (i.e., mean motor UE-FMA score without coordination, speed and reflexes) of our group of patients was 14.3 ± 5.3 [from 9 to 22.4]. This study was approved by the ethical review committee of the local medical faculty. It involved a 20-session training program in the course of 4 weeks. Each session consisted of brain self-regulation and proprioceptive feedback with a hand robot (Naros and Gharabaghi, 2015) prior to a physiotherapy training with a multijoint exoskeleton attached to the impaired arm (Grimm et al., 2016a). Each physiotherapy session consisted of 150 trials of taskoriented reach-to-grasp exercises resembling activities of daily living which were randomly distributed in the directions x, y and z (a total of 50 trials in each direction). The general experimental set-up has already been described in detail elsewhere (Grimm and Gharabaghi, 2016; Grimm et al., 2016a,b) and is cited here when applied in the same way.

## Exoskeleton and Virtual Reality

We used a commercially available (Armeo Spring, Hocoma, Volketswil, Switzerland) rehabilitation exoskeleton for shoulder, elbow and wrist joints, with seven axes (i.e., degrees of freedom) providing antigravity support for the paretic arm and registration of movement kinematics and grip force (**Figure 1**, upper row). This device allowed individual adjustments e.g., of gravity compensation, thereby supporting subjects with severe impairment in performing task-oriented practice within a motivating virtual environment.

We extended these features in-house by using the real-time sensor data of the exoskeleton to display a three-dimensional multi-joint visualization of the user's arm in virtual reality (**Figure 1**, lower row). This provided feedback as to the movement quality, i.e., the absence or presence of compensatory movements. Such a feedback is more liable to recover movement patterns used by the subject before suffering a stroke. It can also avoid compensatory movements rather than merely providing information about movement execution (Cirstea and Levin, 2007). For this purpose, we used a file mapping communication protocol to capture the angles of all arm joints and the grip force from a shared memory block. The virtual arm engine was programmed in a Microsoft XNATM framework. The arm model utilized by the engine was constructed as a meshed boneskin combination with 54 bones (3Ds Max 2010TM, Autodesk). The joint angles and grip forces of the device measured with the exoskeleton were used to modify the bone-vectors of the meshed model in accordance with the movements of the user,



*ACI, internal carotid artery; ACM, middle cerebral artery.*

thus providing online closed-loop feedback. The joint angles of the exoskeleton were directly represented in virtual reality, whereas the grip forces were augmented to feedback natural hand function. More specifically, the maximum grip pressure measured by the force sensor resulted in a full closure of the virtual hand to a fist independent of the subjects' actual ability to perform this particular movement. Prior to each session, subjects were instructed to perform a natural reach-to-grasp movement during the task by using distal (elbow) rather than proximal (shoulder) movements. The three-dimensional visualization of the arm was then applied during each task as an implicit online feedback of the movement, since explicit information can disrupt motor learning in stroke patients (Boyd and Winstein, 2004; Cirstea and Levin, 2007). More specifically, delivery of explicit instructions has been shown to disrupt implicit motor learning after stroke regardless of task (either continuous or discrete movement tasks) or lesion location (involving either the sensorimotor cortical areas or basal ganglia); this disruption did not occur in healthy control subjects (Boyd and Winstein, 2006). In the current set-up, various virtual training paradigms were

provided visual feedback in virtual reality (lower row).

designed to allow for different rehabilitation exercises resembling activities of daily living.

#### Task Design

In this study, subjects performed a reach-to-grasp movement toward a ball which changed its position in virtual space after each trial, thus necessitating three-dimensional transfer movements. The ball had to be grasped, carried to a distant basket and then released without the necessity for a final wrist movement. As soon as it entered a defined range around the ball, the virtual hand could react with the former. The ball changed its color according to the hand position (white: out of range, green: possible to grasp, yellow: possible to transfer, red: possible to release). The grasping and releasing of the virtual ball was performed by applying force to the grip sensor and opening the hand, respectively. The respective thresholds of the grip sensor were adjusted to the individual strength of the user.

### Closed-Loop Adaptation of Task Difficulty

Modification of task difficulty was achieved by adjusting the virtual training space, i.e., the distance between the ball and the basket, in the course of one session, and from session to session. More specifically, during the device calibration, an individual base point was estimated for every subject at the beginning of the training and remained stable throughout the sessions. This base point was projected in the middle of the sagittal body axis in front of the subject, serving as a reference for symmetrical transfer movements in x (right-left), y (up-down) and z (frontback) direction. The basket and ball were randomly distributed in the virtual space, allowing for 6 movement directions (right, left, up, down, forward, backward). The distances reached during each task were recorded throughout the training and gradually enlarged by the training algorithm. The starting distance between ball and basket was set at 5 virtual units in x, y or z direction (vu), corresponding to 7 cm. Upon successful completion of the task, which was not limited in time, the next task was immediately presented. Whenever the task was successfully accomplished twice, an auto-adaptive algorithm progressively enlarged this distance. In this case, the distance between the objects was enlarged by 7 cm in the corresponding direction. The reached distances were stored at the end of each session and provided the starting distances for the next training day. If the task could not be accomplished, i.e., if one object (ball or basket) could not be reached, the distance was reduced again. To allow for enough time to complete the movement, a timeout of 2 s was given. Following this period, the object moved slowly toward the virtual hand at a velocity of 0.5 vu/s until it could be reached. The new, reduced distance was stored for the next task. Similarly, the grip force required for initiating the augmented closing and opening of the fingers of the virtual model was also progressively increased whenever the respective threshold was achieved three times in a row, and decreased when the necessary force could not be applied. These performance-dependent adjustments enabled the subjects to complete the tasks at their respective capability levels. The subjects were instructed to perform the tasks as quickly and as accurately as possible. To maintain their motivation, they received additional feedback via a point score system: the larger the accomplished distance and the faster the performance, the higher the score per trial. In addition, the total score and the five highest trial scores were displayed to the subjects at the end of each session (**Figure 2**).

### Outcome Measures

The training space of the exoskeleton (real space) and the virtual space correspond linearly with an arbitrary point O (0/0/0) localized in the center of the shoulder joint. All quantitative data are transformed to SI-Units. Since no direct conversion is available, raw sensor data are displayed for the grip force. The kinematic assessment included accuracy, temporal efficiency and range of motion (volume). Movement accuracy was captured by calculating changes of movement direction along an optimal path toward the targets, by estimating the distance function between the hand-position and the final endpoint, and by calculating the second derivative of the function to acquire the number of turning points for each task (Cirstea et al., 2006). Temporal efficiency was captured as the time required to complete each task, and as the mean and peak velocity of the hand between the targets while calculating their distance for x-, y-, and zdirections in virtual units (vu). The range of motion (volume) was measured according to the orthosis and displayed in degrees. The range of sensor-data from the grip-sensor was estimated as the mean change in grip pressure. Furthermore, the raw movement data of all joints (shoulder, upper-arm, elbow and wrist) was acquired in degrees. Movements were allowed in 3D space, i.e., moving simultaneously in x-, y-, and z-direction, as illustrated in **Figure 2**. However, the outcome measure "mean distance" refers to an arithmetic mean, since the targets were aligned in one axis (x, y, or z) for each task. The average distance covered in the corresponding direction thus reflected the increase of the inter-target distance. Providing the distance in 3D space would have provided (particularly in the first sessions) false positive values due to large inaccuracies during movement execution. As a cumulative parameter of the performance evolution in 3D space we computed the total training volume, which grew along with the subjects' abilities. This volume was estimated on the basis of the performed movement in 3D space (not on the basis of the inter-target distance).

#### Statistics

Statistical analysis was performed on a Matlab (2010b) Engine. The kinematic data (volume, distance, grip pressure and joint movement) was tested for linear distribution using the Lillieforstest (2-sided goodness-of-fit test). The non-parametric Kruskal-Wallis was used for group comparisons of the UE-FMA score between pre- and post-training. To estimate the evolution of

distance in the course of the trials, i.e., at the beginning of the session (A) and in the middle of the session (B). Lower row: evolution of achieved distances in x-, y- and z-direction in the course of one training session (same as above). The trials shown above are marked with (A) and (B).

parameters during training, a robust multilinear regression model was fitted. Since the Lilliefors-test revealed normality of the data, a robust multilinear regression analysis was applied to minimize the impact of outliers (Holland and Welsch, 1977). The fitting function was based on an iteratively reweighted least squares algorithm. The weight of each iteration was calculated by applying a bi-squared function to the residuals of the previous iteration. The slope b of coefficient estimates and the ratio of the standard error of coefficient estimates (t) are presented for every fitting function. The significance level was set to p = 0.05 for all tests.

#### RESULTS

All subjects were able to perform the reach-to-grasp exercises in virtual space due to gravity compensation and alteration of the grip force of the exoskeleton.

In the course of 20 sessions, the auto-adaptive algorithm led to a progressive increase of the training space in accordance with the subjects' abilities (**Figure 3**: individual subjects, **Figure 4**: group data normalized to baseline, **Figure 5**: group data normalized to maximum). The results are presented in **Table 2**. The gain was particularly high in the first 2–3 sessions, and reached a plateau in the last 3–4 sessions. The mean distance, and the distances for the y-direction and the z-direction in virtual space all showed a significant increase throughout all sessions. The trend in the x-direction (p = 0.057) for all sessions reached significance when considering the evolution before the saturation effect, i.e., sessions 1–18.

This learning curve was paralleled by an improvement of kinematic parameters (**Figure 5**, **Table 2**): The mean training volume increased over the time course of training (pre: 18054 cm<sup>3</sup> ± 26053 cm<sup>3</sup> ; post: 35572 cm<sup>3</sup> ± 15069 cm<sup>3</sup> ), reaching a robust average increase of at least 100% of the starting volume from the 6th session on. This improvement was paralleled by a temporary (i.e., sessions 7–18) increase of volume variability, indicating the potential for relevantly larger gains in some of the subjects.

This gain in range of motion was not at the expense of other kinematic parameters. By contrast, both the inaccuracy (number of turning points) and movement speed- related parameters such as peak velocity and time per task also improved. The peak velocity revealed a robust average increase of at least 50% of the starting speed from the 10th session on. The variability also increased steadily, suggesting that subjects have different specific slopes of increased speed.

Notably, these kinematic gains were also paralleled by significant motor improvements for grip force and elbow movement. The degree of elbow movement increased throughout all sessions by an average of 50% from the 11th to the 16th session, before reaching a saturation level later on. The average

FIGURE 3 | Evolution of mean arithmetic distance of all directions together and distances for x-, y- and z-directions in the course of the training for each patient. Each point represents the mean across 50 trials in each direction for each subject. The color indicates the different patients. One session was performed per day. The solid lines indicate the linear regressions in the course of training.

grip force also increased relevantly, but showed the largest variability of all the parameters (**Figure 6**: individual subjects, **Figure 7**: group data normalized to baseline, **Figure 8**: group data normalized to maximum). Shoulder movement and upper-arm movement showed an improvement but missed significance; the wrist movement did not change in the course of the training. The UE-FMA score changed significantly (p = 0.026) from 14.3 ± 5.4 [from 9 to 22] before to 16.9 ± 6.1 [from 10 to 26] after the intervention.

#### DISCUSSION

This pilot study demonstrates the feasibility of progressively increasing the range of motion of chronic stroke patients with a severe impairment of the upper extremity in the course of 20 training sessions. A multi-joint exoskeleton for the paretic arm allowed the subjects to perform task-oriented practice within a virtual environment (Housman et al., 2009). Notably, unlike other studies with similarly affected stroke patients, where active robots completed a movement when started once (Klamroth-Marganska et al., 2014; Brauchle et al., 2015), this assistive technology delivered antigravity-support only and provided no guidance. Patient engagement was maximized by default in the present study, leaving no room for slacking; the continuous visual feedback of the arm kinematics enabled the patients to adjust their action online during each task; an approach that closely resembles natural motor learning.

Such a closed-loop framework follows an operant conditioning rationale. It provides contingent feedback to facilitate the targeted activity which is considered to be

FIGURE 5 | The figure shows the across subject evolution of the distance traveled in the x, y, and z directions. The mean distance is estimated across subjects, where the distance is the average distance covered in the corresponding direction across trials per session per subject. The data is normalized to the maximum performance achieved in each parameter. The subplots shows the evolution of the (A) arithmetic mean distance, (B) the distance in the x- direction, (C) the distance in the y-direction and (D) the distance in the z-direction.


#### TABLE 2 | Parameter progression over training.

beneficial for recovery, and might ultimately lead to functional gains (Gharabaghi et al., 2014c,d; Bauer and Gharabaghi, 2015a). These restorative approaches may, however, pose a considerable challenge for the patients (Bauer and Gharabaghi, 2015b; Fels et al., 2015) who might explore alternative, i.e., therapeutically undesirable, strategies (Gharabaghi et al., 2014b). Moreover, particularly in patients with severe impairments, non-successful trials may cause frustration, thereby limiting motor learning. In this context, closed-loop adaptation of exercise difficulty, as practiced in the present

lines indicate the linear regressions in the course of training.

study, may help to avoid frustration by tailoring the range of motion in accordance with the actual ability of each patient.

Previous adaptation approaches provided different types of assistance (Colombo et al., 2012), applied a lead-lag performance model for robotic assistance (Chemuturi et al., 2013), or adjusted the robot/patient's interaction forces (Vergaro et al., 2010). The adaptation approach implemented in this study was differed conceptually from the previous algorithms in that it modulated the virtual task difficulty, not the degree of assistance. This passive gravity compensation remained stable throughout the exercises. Nonetheless, the patients were challenged continuously in our study since the difficulty level increased progressively as soon as task accomplishment was repeated successfully. This performance-dependent online adjustment of task challenge facilitated reinforcement learning and resulted in a progressive increase of the virtual training space with a concurrent improvement of real world range of motion and other kinematic parameters such as accuracy and movement velocity. Notably, these gains followed unsupervised training algorithms and were paralleled by motor improvements such as increased elbow movement, grip force and upper extremity Fugl-Meyer-Assessment score. Whether or not these motor improvements were caused by the specific performance-dependent training algorithm applied here cannot be concluded from the present data, since a control group, i.e., dose-matched training without online adaptation of task difficulty, was not included in this study. Furthermore, this set-up did not assess whether its effects would be limited to chronic and severely affected stroke survivors. The dataset was also small and the heterogeneity of subjects, injuries or time from stroke might influence the gains observed.

However, these limitations do not compromise the major finding of this study, namely the feasibility of progressively increasing the assisted range of motion of severely impaired stroke patients by applying closed-loop virtual reality feedback for unsupervised motor learning. As in all previous studies in chronic stroke patients with severe motor impairments

of the upper extremity (e.g., Lang et al., 2016), the clinical improvement within 20 training sessions was, in any case, too modest to lead to relevant functional gains of the patients in their activities of daily living. A sufficiently powered, randomized and adequately controlled but costly trial is, therefore, currently not justified on the basis of this specific approach and the dose of practice applied here. However, the implemented setup may prove suitable as the basis and training framework for other concurrently applied restorative interventions (see below).

Different, mutually non-exclusive reasons might be responsible for the current limited functional gain: Since the dose of stroke rehabilitation therapy has been shown to correlate positively with clinically meaningful improvements (Lohse et al., 2014; Pollock et al., 2014), the approximately 3000 movement attempts, i.e., exercise trials, performed in the course of 20 sessions during this 4-week study might have been insufficient to induce functionally more relevant improvements. On the other hand, even higher doses of motor therapy (i.e., 6400 or 9600 repetitions in the course of 8 weeks and 32 sessions, 4 days/week) in chronic stroke patients with long-standing (>6 months) upper limb paresis, did not result in a larger functional improvements than in patients who received a therapy dose (3200 repetitions) similar to the one applied here (Lang et al., 2016).

However, the trajectories of kinematic and clinical parameters in the course of the training of the present study suggest that a plateau level of improvement, i.e., a ceiling effect, has not been achieved yet and that further practice sessions, i.e., a longer intervention period, would result in larger gains. Moreover, the huge performance variability of the patients in some sessions, e.g., between 100 and 1000% increased ranges of motion, suggests a general capacity for even larger improvements for at least some of the patients. These windows of opportunity might, however, necessitate additional interventions to maximally exploit and consolidate the salvaged restorative potential.

Brain stimulation may facilitate such additive effects for assisted reach-to-grasp exercises: Bilateral transcranial direct current stimulation, for example, has led to improved motor performance of healthy patients beyond the natural learning

curve when applied prior to training with the very same multi-joint arm exoskeleton as applied in the present work (Naros et al., 2016a). Brain state-dependent transcranial magnetic stimulation has, moreover, been demonstrated to induce robust increases of corticospinal excitability (Kraus et al., 2016b; Royter and Gharabaghi, 2016) and may thereby amplify use-dependent plasticity when applied in conjunction with assistive rehabilitation devices (Gharabaghi, 2015; Massie et al., 2015). Concurrent state-dependent transcranial magnetic stimulation may thereby unmask latent corticospinal connectivity after stroke (Gharabaghi et al., 2014a) which can be detected and monitored with refined motor mapping techniques (Kraus and Gharabaghi, 2015, 2016; Mathew et al., 2016). Applying phase-dependent stimulation (Raco et al., 2016) synchronized to maximum gains of assisted range of motion, may furthermore consolidate the involved corticospinal circuits in accordance with Hebbian-like plasticity rules.

The scope for recovery may also be improved when using advanced assistive rehabilitation technology based on brain-robot interfaces, since these devices were found to constitute a back-door to the motor system (Gomez-Rodriguez et al., 2011; Bauer et al., 2015). Exercises based on brain-robot feedback of motor-imagery related sensorimotor beta-band desynchronization may result in connectivity changes of cortico-cortical motor networks (Vukelic et al., 2014; Vukeli ´ c and Gharabaghi, 2015a,b ´ ), lead to a re-distribution of cortico-spinal connections (Kraus et al., 2016a) and to behavioral gains (Naros et al., 2016b). Combining these tools with an adaptive virtual environment similar to that applied in this study may thus maximize the impact of both approaches on sensorimotor function.

In summary, combining gravity-compensation with autoadaptive closed-loop feedback in virtual reality provides customized rehabilitation environments for severely affected patients and may facilitate unsupervised motor learning by balancing the patient's challenge in accordance with the individual capacity for functional restoration; a proposal that requires investigation in a larger cohort of stroke patients in comparison to sham adaptive and non-adaptive feedback as well as to dose-matched, conventional physiotherapy.

### AUTHOR CONTRIBUTIONS

FG participated in the study design and software development, supervised the measurement sessions and carried out most of the data analysis. GN supervised the measurement sessions. AG participated in the study design and data analysis, and wrote the manuscript. Authors jointly approved the final manuscript.

#### REFERENCES


#### ACKNOWLEDGMENTS

AG was supported by grants from the German Research Council [DFG EC 307], and from the Federal Ministry for Education and Research [BFNT 01GQ0761, BMBF 16SV3783, BMBF 0316064B, BMBF16 SV5824].

resting-state networks: personal trait or task-specific challenge? J. Neural Eng. 12:046029. doi: 10.1088/1741-2560/12/4/046029


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2016 Grimm, Naros and Gharabaghi. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Hybrid Neuroprosthesis for the Upper Limb: Combining Brain-Controlled Neuromuscular Stimulation with a Multi-Joint Arm Exoskeleton

Florian Grimm<sup>1</sup> , Armin Walter <sup>2</sup> , Martin Spüler <sup>2</sup> , Georgios Naros <sup>1</sup> , Wolfgang Rosenstiel <sup>2</sup> and Alireza Gharabaghi <sup>1</sup> \*

<sup>1</sup> Division of Functional and Restorative Neurosurgery, Centre for Integrative Neuroscience, Eberhard Karls University Tuebingen, Tuebingen, Germany, <sup>2</sup> Department of Computer Engineering, Wilhelm Schickard Institute for Computer Science, Eberhard Karls University Tuebingen, Tuebingen, Germany

#### Edited by:

Paolo Massobrio, University of Genoa, Italy

#### Reviewed by:

Alissa Fourkas, National Institutes of Health, USA Marc Slutzky, Northwestern University, USA

Mukta Vaidya contributed to the review of Marc Slutzky

\*Correspondence:

Alireza Gharabaghi alireza.gharabaghi@uni-tuebingen.de

#### Specialty section:

This article was submitted to Neuroprosthetics, a section of the journal Frontiers in Neuroscience

Received: 10 February 2016 Accepted: 25 July 2016 Published: 09 August 2016

#### Citation:

Grimm F, Walter A, Spüler M, Naros G, Rosenstiel W and Gharabaghi A (2016) Hybrid Neuroprosthesis for the Upper Limb: Combining Brain-Controlled Neuromuscular Stimulation with a Multi-Joint Arm Exoskeleton. Front. Neurosci. 10:367. doi: 10.3389/fnins.2016.00367 Brain-machine interface-controlled (BMI) neurofeedback training aims to modulate cortical physiology and is applied during neurorehabilitation to increase the responsiveness of the brain to subsequent physiotherapy. In a parallel line of research, robotic exoskeletons are used in goal-oriented rehabilitation exercises for patients with severe motor impairment to extend their range of motion (ROM) and the intensity of training. Furthermore, neuromuscular electrical stimulation (NMES) is applied in neurologically impaired patients to restore muscle strength by closing the sensorimotor loop. In this proof-of-principle study, we explored an integrated approach for providing assistance as needed to amplify the task-related ROM and the movement-related brain modulation during rehabilitation exercises of severely impaired patients. For this purpose, we combined these three approaches (BMI, NMES, and exoskeleton) in an integrated neuroprosthesis and studied the feasibility of this device in seven severely affected chronic stroke patients who performed wrist flexion and extension exercises while receiving feedback via a virtual environment. They were assisted by a gravity-compensating, seven degree-of-freedom exoskeleton which was attached to the paretic arm. NMES was applied to the wrist extensor and flexor muscles during the exercises and was controlled by a hybrid BMI based on both sensorimotor cortical desynchronization (ERD) and electromyography (EMG) activity. The stimulation intensity was individualized for each targeted muscle and remained subthreshold, i.e., induced no overt support. The hybrid BMI controlled the stimulation significantly better than the offline analyzed ERD (p = 0.028) or EMG (p = 0.021) modality alone. Neuromuscular stimulation could be well integrated into the exoskeleton-based training and amplified both the task-related ROM (p = 0.009) and the movement-related brain modulation (p = 0.019). Combining a hybrid BMI with neuromuscular stimulation and antigravity assistance augments upper limb function and brain activity during rehabilitation exercises and may thus provide a novel restorative framework for severely affected stroke patients.

Keywords: functional electrical stimulation, robot-assisted rehabilitation, brain-robot interface, brain-machine interface, brain-computer interface, functional restoration, motor recovery, upper-limb assistance

#### Grimm et al. Brain-Controlled Hybrid Neuroprosthesis for the Upper Limb

## INTRODUCTION

Standard of care leaves the majority of stroke survivors with a dysfunctional upper extremity and, consequently, with a long-term dependency on others for activities of daily living (Jørgensen et al., 1999; Dobkin, 2005; Feigin et al., 2008). Attempts to improve recovery in this patient group are numerous and embrace advanced rehabilitation technology for motor relearning such as brain-interface based neurofeedback training (Ang et al., 2015; Morone et al., 2015; Pichiorri et al., 2015), robot-assisted rehabilitation devices (Lo et al., 2010; Klamroth-Marganska et al., 2014) and activity-dependent neuromuscular stimulation techniques (Thrasher et al., 2008; Oujamaa et al., 2009; Mann et al., 2011). Recent approaches combine these different methods in a bid to maximize the overall benefits (Meadmore et al., 2014; Brauchle et al., 2015; Hortal et al., 2015; Grimm and Gharabaghi, 2016). However, there is still a critical need in the rehabilitation community to provide options for stroke patients with chronic impairments. In this context, movement-related desynchronization (ERD) in the contralateral sensorimotor cortex has been shown to be compromised in stroke patients compared to healthy controls; notably, the more severe the patient's motor impairment, the less beta-band ERD (Rossiter et al., 2014). Accordingly, increasing this oscillatory modulation range again would provide a therapeutic target for a restorative training approach.

In the present proof-of-principle study, we explored an integrated approach for providing assistance as needed to amplify the task-related range of motion (ROM) and the movement-related brain modulation during rehabilitation exercises of severely impaired patients; we have, therefore, combined different rehabilitation tools: brain-controlled neurofeedback training, an upper limb multi-joint exoskeleton, and activity-dependent neuromuscular electrical stimulation (NMES). These different components served the following goals: The brain-controlled neurofeedback training based on motor imagery has recently been shown to increase task-related oscillatory modulation, specifically in the beta-frequency band, in correlation with corticospinal excitability (Kraus et al., 2016a) and motor learning (Naros et al., 2016a). Moreover, previous findings indicated that NMES amplifies both cortical ERD (Müller et al., 2003) and excitability when combined with motor imagery (Reynolds et al., 2015) or volitional effort (Stein et al., 2013). More specifically, during NMES movement a prominent ERD was found similar to that observed after active or passive movements suggesting that the sensorimotor processing during NMES involves some of the processes which are also involved in voluntary hand movements (Müller et al., 2003). Finally, multi-joint gravity compensation of the upper extremity has recently been shown to increase the movement range of severely affected stroke patients (Grimm et al., 2016), particularly when combined with NMES (Grimm and Gharabaghi, 2016).

However, the presented multifaceted device differs from previous approaches in several ways: the brain-controlled neurofeedback was not provided by an active robotic exoskeleton (Brauchle et al., 2015) but by NMES combined with a passive un-weighting exoskeleton (Meadmore et al., 2012; Hortal et al., 2015); in addition, NMES was not applied to proximal (Meadmore et al., 2012; Hortal et al., 2015) but to distal muscles (Meadmore et al., 2014), and was not controlled by kinematic information (Meadmore et al., 2012, 2014), but by physiological signals (Brauchle et al., 2015; Hortal et al., 2015) while applying a hybrid brain-machine interface (BMI) based on both sensorimotor cortical desynchronization (ERD) and electromyography (EMG) activity. Moreover, NMES induced no overt support (Meadmore et al., 2014; Hortal et al., 2015) but remained subthreshold (Grimm and Gharabaghi, 2016).

These modifications aimed to address limitations of current rehabilitation technologies, which usually take an all-or-nothing approach, e.g., by providing active robotic guidance to complete a movement as soon as the patient fails to reach the defined goal (Klamroth-Marganska et al., 2014); or by triggering NMES for overt muscle contraction, also referred to as functional electrical stimulation (FES), as soon as a predefined physiological state (recorded with either EMG or EEG) is achieved (Howlett et al., 2015). This all-or-nothing approach offers an important experience for patients who have not been able to move their hand or arm for years. From a motor learning perspective, however, it might be more successful to provide such rewarding feedback, e.g., robot-assisted movement of the paretic hand, only when a certain level of effort is made by the participant and gradually increased in the course of the training (Naros and Gharabaghi, 2015; Naros et al., 2016a). More targeted assistance might, therefore, be necessary during the rehabilitation exercises to maintain engagement without compromising the patients' motivation; i.e., by providing support—as little as possible and as much as necessary.

We, therefore, hypothesized that the adjustments implemented in our integrated approach provide assistance as needed to amplify the task-related ROM and the movementrelated brain modulation during rehabilitation exercises of severely affected stroke patients without compromising their engagement.

### METHODS

Patients were selected for this study when they were in the chronic phase after stroke (>6 months) presenting with a severe and persistent hemiparesis [modified upper extremity Fugl-Meyer-Assessment score (mUE-FMA) < 25]. Seven stroke patients (mean age: 59 ± 9.3 [41 89] years; 66.43 ± 16.6 [34 80] months post stroke; 14.3 ± 4.7 [9 23] mUE-FMA; male: female, 6:1; ischemic (middle cerebral artery): hemorrhagic stroke, 3:4; right: left hemisphere, 6:1). The mUE-FMA (without coordination, speed, and reflexes) was used to ensure that our results were comparable to earlier studies (Brauchle et al., 2015; Naros and Gharabaghi, 2015). This study, which was approved in accordance with the guidelines of the ethics committee of the local medical faculty, involved two sessions of wrist training with a multi-joint exoskeleton attached to the paretic arm. Each session consisted of approximately 30–40 movement trials with alternating wrist extension and flexion. Each movement period (extension or flexion) lasted for 5 s and was preceded

by a 5 s rest period. This study is part of a larger research program on assisted reach-to-grasp movements in severely affected stroke patients. Within this framework, recent studies have revealed the importance of anti-gravity support with a multi-joint exoskeleton. We therefore applied this exoskeletonbased setup in this study as well to facilitate the transfer of the present findings into the overall research program. The exoskeleton and virtual reality have been described in detail elsewhere (Grimm and Gharabaghi, 2016; Grimm et al., 2016) and are cited here where applicable.

#### Exoskeleton and Virtual Reality

We used a commercially available (Armeo Spring, Hocoma, Volketswil, Switzerland) rehabilitation exoskeleton for shoulder, elbow and wrist joints with seven axes (i.e., degrees of freedom) to provide antigravity support for the paretic arm and to register movement kinematics and grip force. Unweighing was realized via two springs that were incorporated into the device. This device could be used to make individual adjustments of, for example, the gravity compensation, thereby supporting patients with severe impairment in performing task-oriented practice within a motivating virtual environment. We extended these features in-house by using the real-time sensor data of the exoskeleton to display a three-dimensional multi-joint visualization of the user's arm in virtual reality. This entailed the use of a file mapping communication protocol to capture the angles of all arm joints and the grip force from a shared memory block. The virtual arm engine was programmed in a Microsoft XNATM framework. The arm model utilized by the engine was constructed as a meshed bone-skin combination with 54 bones (3Ds Max 2010TM, Autodesk). Using the measured joint angles and grip forces of the device, the bone-vectors of the meshed model were modified according to the movements of the user to provide online closed-loop feedback. The joint angles of the exoskeleton were directly represented in virtual reality, whereas the grip forces were augmented (i.e., amplified in virtual reality on the screen) to feedback natural hand function. This allowed visualizing finger movements on the screen, even though this information was not used for the study. However, the threedimensional visualization of the fingers and wrist was applied during each task as an implicit online feedback of the movement. Prior to each session, participants were instructed to perform a natural wrist movement during the tasks aiming at maximum extension and flexion, respectively. The ROM of wrist movement was calculated as the sum of maximum extension and flexion and computed as the mean of each session.

#### Neuromuscular Electrical Stimulation

We integrated a NMES (De Marchis et al., 2016) device (Rehastim, 8- channel stimulator, Hasomed GmbH, Magdeburg, Germany) into the exoskeleton-based training environment with two bipolar, self-adhesive electrodes (diameter: 40 mm), and applied biphasic square impulses (frequency: 30 Hz, pulse width: 500µs). The stimulation of this integrated neuroprosthesis (**Figure 1**) was updated in a closed-loop, real-time iteration at 60 Hz via a Controller Area Network/Universal Serial Bus (CAN/USB) port using a custom-made algorithm. Whenever the BMI classifier output was positive (see below), NMES was applied for 3 s to the M. extensor carpi ulnaris during wrist extension or to the M. flexor carpi radialis during flexion movement.

Each patient performed two exoskeleton-supported training sessions—one with and one without BMI-controlled NMES. Both the exoskeleton and the maximum stimulation intensity (Stimmax) were individually calibrated. The exoskeleton was adjusted to provide optimized gravity compensation for every joint and to allow for unrestricted wrist movements in threedimensional space. The Stimmax for each muscle group was empirically determined as the output current approaching the motor threshold but that was still perceived as comfortable. Since all participants suffered from severe upper limb impairment, prolonged supra-motor threshold stimulation was perceived as painful and was therefore not applied. The stimulation intensity

FIGURE 1 | Integrated neuroprosthesis with feedback via a virtual environment. Assistance is provided by a gravity-compensating, seven degree-of-freedom exoskeleton attached to the paretic arm. Neuromuscular electrical stimulation is applied to the wrist extensor and flexor muscles during the exercises and is controlled by a hybrid brain-machine interface based on both sensorimotor cortical desynchronization and electromyography activity.

was thus set in accordance with each patient's comfort level and just below motor-threshold, i.e., no visible joint movement, and resulted in a mean of 10.5 mA (±4.4 mA) and 9.5 mA (±4.4 mA) for the wrist flexor and extensor, respectively.

### Data Acquisition

Electroencephalographic (EEG) signals were recorded with BrainAmp DC amplifiers and an antialiasing filter (BrainProducts, Munich, Germany) from 32 Ag/AgCl scalp electrodes (sampling rate: 1000 Hz) in accordance with the international 10–20 system (FP1, FP2, F3, Fz, F4, FC5, FC3, FC1, FCz, FC2, FC4, FC6, C5, C3, C1, Cz, C2, C4, C6, CP5, CP3, CP1, CPz, CP2, CP4, CP6, P3, POz, P4, POz, O1, O2; reference: FCz, ground: AFz). Electrode impedances were maintained below 10 k. Since it often exceeds the frequency range of the physiological signals, ambient noise may compromise the recordings. To avoid an aliasing error due to undersampling of this noise, we, therefore, made every effort to remove all potential sources of electrical noise from the experimental environment, i.e., the high-frequency noise was deliberately avoided during the experiment and verified offline. Thanks to this approach, we observed no high-frequency noise in our recordings (Gharabaghi et al., 2014a; Vukelic et al., 2014; Bauer et ´ al., 2015; Naros and Gharabaghi, 2015; Vukelic and Gharabaghi, 2015a,b ´ ).

Since EMG contaminations via compensatory movements are known to compromise EEG-based BMI training (Gharabaghi et al., 2014b), experienced examiners, who were trained to recognize these artifacts, instructed the patient to minimize them. As in previous studies with healthy subjects (Vukelic´ et al., 2014) and severely affected stroke patients (Naros and Gharabaghi, 2015), the patient was also instructed to avoid blinking, chewing, and any head and body movements other than the wrist movements. Together with visual inspection and feedback by the examiner, this approach proved to be a feasible method of preventing alternative BMI control. In addition, the EEG data was reanalyzed offline by visual inspection to remove all artifacted trials due to movement artifacts or current drifting; this resulted in a mean of 4.5 ± 3.8 excluded trials.

#### Data Analysis

Band pass (2–150 Hz) and notch filtering (50 Hz) were applied to the EEG raw signal. After epoching the filtered data into trials, visual artifact rejection was performed. This yielded an average of 26 ± 4 and 31 ± 3 (mean ± SD) trials in the non-NMES and NMES sessions, respectively. The power spectrum was normalized to the mean spectral distribution of the 5 s premovement rest period of the session. Mean movement-related spectral perturbation (ERSP) of the feedback electrodes were calculated for each session using the EEGLAB-Toolbox (Delorme and Makeig, 2004).

Surface electromyography (EMG) of the M. extensor carpi ulnaris and M. flexor carpi radialis were recorded with a bandpass filter of 0.1–1000 Hz and a sampling rate of 1000 Hz. The first task was used to set an individual EMG-threshold (area under the curve, AUC), to calibrate the EMG-classifier. Discrimination between movement and rest was performed by analyzing the activity of the measured EMG-channels. To this end, the EMG data of these channels was bipolarized and a Butterworth highpass filter with an order of n = 2 and a cutoff at 1 Hz was applied. The waveform length WL (ti) = Pt<sup>i</sup> t=ti−w+1 |x (t + 1) − x (t)| was calculated for each bipolarized EMG channel within a sliding window of w = 200 ms length. The sliding window was moved over the data in steps of 40 ms and corresponded to the waveform length of both channels. The waveform length feature of EMG has already been used to successfully decode different movements from EMG activity (Tenore et al., 2009). To correct for a delayed response of the subject to the cues, we calculated the crosscorrelation of a vector W = WL(t\_i) containing the waveform length feature with a vector P = P(t\_i) which encodes the trial phase, where P(t\_i) = 1 if t\_i is part of the movement phase (otherwise 0). We used the latency of the maximum of the crosscorrelation sequence as an offset to improve the assignment of the waveform length to the movement or rest class (MWL or RWL, respectively). We identified the threshold T for the discrimination between the two distributions MWL and RWL with a Receiver Operating Characteristic (ROC) analysis. The criterion for threshold selection was set such that the falsepositive rate was lower than 5% to ensure high specificity (≥0.95) of the classifier.

### Brain-Machine Interface (BMI)

The BMI environment was designed to stimulate the patient's wrist during the movement (recorded by EMG) as soon as movement-related event-related desynchronization (ERD) in the β-band was detected in the ipsilesional hemisphere (Walter et al., 2012; Gharabaghi et al., 2014a). NMES stimulation was not triggered unless both the EMG and EEG classifier gave a positive output (**Figure 2**). We hypothesized that this hybrid approach improves the stability of classification (Leeb et al., 2011) and expected that the effects on ROM and ERD are bigger when using BMI+NMES than the exoskeleton alone.

During the NMES session, the same EMG filtering and feature extraction strategy as described above was employed. After bipolarization and filtering, the samples of each data packet

when the task-specific effort of the participant is detected.

from these channels were joined together to form a 200 mslong queue. The waveform length was computed, summed up for both channels and compared to the threshold T for movement detection. If it exceeded T, the EMG classifier gave a positive output.

The EEG algorithm was based on the spectral power values between 16 and 22 Hz for three selected channels (FC4, C4, and CP4). We applied the same frequency-range and setup as in our previous BMI studies (Gharabaghi et al., 2014a; Vukelic et al., ´ 2014; Bauer et al., 2015; Naros and Gharabaghi, 2015; Vukelic and ´ Gharabaghi, 2015a,b). The spectral power was calculated using an autoregressive model order of 16 (McFarland and Wolpaw, 2008). This was fitted to the last 500 ms of the signal and updated every 40 ms. Classifier output was positive when 5 consecutive 40 ms epochs (i.e., 200 ms) were classified as ERD-positive. An epoch was not regarded as ERD-positive until the output of the classifier exceeded a threshold θ (Walter et al., 2012; Gharabaghi et al., 2014a; Naros and Gharabaghi, 2015; Naros et al., 2016a). The online signal processing was performed with the standard algorithm of the BCI2000 software (Mellinger et al., 2007). With a bin width of 2 Hz and targeted bin centers of 17, 19, and 21 Hz, the resulting frequency band was 16–22 Hz and corresponded to a wave length of between 45 and 62 ms. Choosing a data window of 500 ms enabled us to capture several cycles of these frequencies for reliable power analysis. This approach has already proved to be reliable in studies with the very same BMI setup (Walter et al., 2012; Gharabaghi et al., 2014a; Vukelic et al., 2014; Bauer et al., ´ 2015; Naros and Gharabaghi, 2015; Vukelic and Gharabaghi, ´ 2015a,b).

The sensitivity and specificity of the classifier of a linear discriminant analysis were indicated by the true-positive rate (TPR) and the true-negative rate (TNR), respectively; the false-positive rate (FPR) equaled 1-TNR. TPR and TNR were calculated by

$$TPR = \frac{\text{pN}\_{\text{move}}}{\text{N}\_{\text{move}}} \tag{1}$$

$$TNR = \frac{\text{nN}\_{\text{rest}}}{\text{N}\_{\text{rest}}} \tag{2}$$

with N as the total number of sample blocks in either the rest or move period, and pN and nN as the positively and negatively classified sample blocks, respectively.

The classification accuracy (CA) of a BMI system was defined by

$$\text{CA} = \frac{\text{TPR} + \text{TNR}}{2} \tag{3}$$

and estimated for the different classifier modalities, i.e., EEG, EMG, and hybrid EEG/EMG. In addition, the correct response rate (CRR) was calculated as the ratio between the number of actions (i.e., BMI controlled NMES assistance) and the number of trials.

#### Statistics

Statistical analysis was performed on a Matlab 2010b Engine. Data was tested for normal distribution using the Lilliefors-test (2-sided goodness-of-fit test). For normally distributed data, a dependent t-test for paired samples was performed; otherwise a Wilcoxon's signed ranks test was used. The significance level was set at p = 0.05 for all tests.

#### RESULTS

Subthreshold NMES could be well integrated into the exoskeleton-based training; the effects on ROM and ERD were bigger when using BMI+NMES than the exoskeleton alone. More specifically, this combined approach increased the task-related ROM of the wrist from 18 ± 6 ◦ to 26 ± 8 ◦ (p = 0.009, **Figure 3**).

The patients showed ERD both in the non-supported and the NMES-supported tasks. The ERD maximum for the decoded channels and frequencies was −2.47 and −2.83 dB in the non-supported and NMES-supported tasks, respectively. The intervention modulated the movement-related brain activity by amplifying the desynchronization (**Figure 4**) in the feedback frequency band (16–22 Hz) as well as by inducing significant (p = 0.019) additional broadband ERD throughout the task period in the low beta (14–16 Hz), delta (2–5 Hz), and gamma band (45–47 Hz) (**Figure 5**).

The hybrid BMI, i.e., combining the classification output of the EEG and the EMG classifier, was used during the task for online control. By achieving a mean classification accuracy of 66 ± 9.6% compared to 55 ± 6.4% (offline analysis with the EEG-classifier only) and 55 ± 4.6% (offline analysis with the EMG-classifier only, **Figure 6**), the hybrid BMI controlled the stimulation significantly better than either the EEG (p = 0.028) or the EMG (p = 0.021) modality. This gain was achieved by increasing the specificity of the classification, i.e., by significantly

reducing the false positive rates to 22 ± 7.1% with the hybrid approach as compared to 37 ± 6.3% with the EMG (p = 0.037) and 53 ± 5.1% with the EEG modality (p = 0.007). On average, the device was triggered in 24 out of 31 trials, i.e., achieving a CRR of 77%.

This improved accuracy with the hybrid approach was also reached for the offline analysis of the non-NMES sessions (when no BMI or classification took place) by achieving 63 ± 6.9% compared to 56 ± 5.9% (EEG-classifier) and 55 ± 4.6% (EMGclassifier, **Figure 6**); the hybrid BMI classified significantly better than either the EEG (p = 0.031) or the EMG (p = 0.038) modality, notably, without a potential bias by the actual application of this classifier and the BMI-NMES during the task.

#### DISCUSSION

This proof-of-principle study has demonstrated the feasibility of an integrated neuroprosthesis combining a hybrid BMI—based on both cortical and muscle activity—with an exoskeleton and NMES for neurofeedback training via a virtual environment; this neuroprosthesis increased the ROM of wrist movement in chronic stroke patients with a severe impairment of the upperextremity. Unlike other studies with similarly affected stroke patients, in which robots completed a movement initiated by the patients (Klamroth-Marganska et al., 2014; Brauchle et al., 2015), the technology applied here provided antigravity-support only (Housman et al., 2009), i.e., rendered no active assistance, thereby exploiting patient engagement and avoiding underchallenge during neurorehabilitation. However, future studies need to disentangle the contributions and mechanisms of BMI, NMES, and exoskeleton practice separately. Moreover, future intervention studies need to apply multiple sessions to explore whether cumulative increases of ROM and ERD can be achieved with this approach.

In this context, brain-controlled neurofeedback training aims to modulate cortical physiology and is applied to increase the responsiveness of the brain to subsequent physiotherapy (Pichiorri et al., 2015). When used in conjunction with commercially available robotic rehabilitation technology, these devices are also referred to as brain-robot interfaces (BRI; Bauer et al., 2015; Fels et al., 2015; Kraus et al., 2016a; Naros et al., 2016a). Such brain-robot interfaces can be applied for both restorative and assistive purposes. Even though both methods employ similar technology, restorative interfaces differ in concept substantially from brain-controlled assistive devices, which aim to compensate for lost function (Hochberg et al., 2012; Collinger et al., 2013). While the latter approach intends to maximize speed and classification accuracy for high-dimensional control (Spüler et al., 2014, 2016), the former aims to facilitate selfregulation of brain activity, which is considered beneficial for recovery and might ultimately lead to persistent functional gains (Naros and Gharabaghi, 2015). Such a restorative goal necessitates methodological specifications, e.g., in the areas of constrained feature space, regularized feature weights, cognitive load, feedback modality, and threshold adaptation to facilitate reinforcement learning of brain self-regulation and corticospinal connectivity (Bauer et al., 2016a,b; Bauer and Gharabaghi, under review). Proprioceptive feedback, for example, has been shown to enhance brain self-regulation of beta-band oscillations in comparison to visual feedback only (Vukelic and Gharabaghi, ´ 2015a); these self-regulated beta-oscillations, in turn, correlated with the increase in corticospinal excitability following BRI training (Kraus et al., 2016a).

These specifications are, however, often not taken into consideration when brain signals are applied during rehabilitation practice, e.g., to control robotic devices or NMES. Instead, classification algorithms are applied to maximize accuracy in an unconstrained feature space, e.g., with support vector machines computing optimal features of an extended oscillatory frequency band, thereby resembling the approach usually chosen for assistive brain-interfaces (Hortal et al., 2015). Following the requirements of restorative neurofeedback training, e.g., providing feedback to beta-band ERD may, however, result in relatively low classification accuracy—as also observed in the present study—and frustrate the participants (Bauer and Gharabaghi, 2015a; Fels et al., 2015). This is particularly true of the severely affected patient group since movement-related beta-ERD in the ipsilesional primary cortex is compromised in stroke patients in comparison to healthy controls, i.e., the more severe the patient's motor impairment, the less beta-ERD (Rossiter et al., 2014).

In this context, we recently argued (Naros and Gharabaghi, 2015) that the fact that beta oscillations are less optimal for classification purposes—e.g., for differentiating movementrelated brain states in many stroke patients—does not compromise but rather qualifies this physiological marker as a therapeutic target. We referred to an analogy to the concept of constraint-induced movement therapy in stroke patients, where the affected rather than the healthy body side is trained to facilitate restoration instead of compensation of motor function (Naros and Gharabaghi, 2015); and proposed that restorative

neurofeedback training should follow the therapeutic goal of restoring the sensorimotor loop via improved beta-band modulation rather than aiming to train the brain state that enables the patient to control the exercising device best. The latter is a strategy that is implicitly followed when selecting individual frequency bands with best classification properties, i.e., that best separate the rest and the task condition (Hortal et al., 2015; Pichiorri et al., 2015).

Under these circumstances, complementary strategies such as continuous threshold adaptation (Bauer and Gharabaghi, 2015a; Naros and Gharabaghi, 2015; Bauer et al., 2016a) or hybrid classifiers that consider both brain signals and electromyography (EMG) activity (Leeb et al., 2011) are necessary to improve patient control over the training devices. The latter approach proved to be effective in the present feasibility study by increasing the classification accuracy from 55 to 66% with the hybrid BMI, compared to the EEG- or EMG-classifier, and resulting in 77% task-related neuroprosthetic support. Notably, this improvement was achieved by increasing the specificity of the feedback, i.e., by decreasing the false positive rate, which is particularly relevant for reinforcement learning with brain-interface based neurofeedback (Bauer and Gharabaghi, 2015a), since the considerable challenge

of these exercises (Bauer and Gharabaghi, 2015b; Fels et al., 2015) might condition the patients to explore alternative, i.e., therapeutically undesirable, strategies (Gharabaghi et al., 2014b). Moreover, this hybrid approach enabled patients to achieve BMI controlled NMES assistance in more than 70% of the tasks, a level which is regarded as necessary for achieving a sense of self-efficacy during motor learning with assistive technology (Metzger et al., 2014). Notably, EMG signals alone were insufficient for classification in this study and might in general be inadequate as a control signal in the targeted patient group due to paralysis and/or abnormally co-activated muscles (Wright et al., 2014), a condition especially relevant in the severely impaired stroke patients who might benefit most from assistive rehabilitation technology.

Furthermore, the presented closed-loop framework facilitated the beta-band ERD, thereby adhering to the operant conditioning rationale, i.e., reinforcing the targeted activity considered to be beneficial for recovery and which might ultimately lead to functional gain (Bauer and Gharabaghi, 2015b; Naros and Gharabaghi, 2015; Naros et al., 2016b). However, whether this effect was achieved directly via the subthreshold NMES or mediated by the increased ROM in the NMES condition remains to be clarified. The spectral changes beyond the feedback frequency band suggest the former since the stronger wrist movement in the neuroprosthetic condition as compared to the orthotic condition is unlikely to result in broadband modulation of cortical activity in itself. Future studies, however, need to test this hypothesis by comparing different movement extensions with the same intervention, i.e., either neuroprosthetic or orthotic support. Importantly, recent findings indicated that NMES amplifies both ERD and cortical excitability when combined with motor imagery (Reynolds et al., 2015) or volitional effort (Stein et al., 2013). The facilitated ERD might, therefore, provide the substrate for future gains following repetitive application since the task-related ERD during brain-robot training have been shown to correlate with the cortico-spinal excitability after the intervention (Kraus et al., 2016a).

In recent approaches for stroke rehabilitation, patients controlled the rehabilitation robots with their brain signals, i.e., via motor imagery-related oscillations of the ipsilesional cortex, thereby successfully linking three-dimensional robotic training for reach-to-grasp movements to the participant's effort (Brauchle et al., 2015). The findings suggest, however, that sustained brain self-regulation for brain-controlled robotic training might be challenging (Brauchle et al., 2015) and may even be characterized by a significant association with the experience of frustration for the participants (Fels et al., 2015). To avoid this over-challenge, the brain-control assistance should probably be applied with more precision. In the same vein, complementary approaches applied NMES concurrently with antigravity support with a multi-joint exoskeleton (Meadmore et al., 2012; Hortal et al., 2015), thereby directly addressing the strength of specific muscle groups. However, these approaches stimulated proximal muscles of the upper limb, while the activation of wrist and hand muscle might be particularly important for functionally relevant improvements (Meadmore et al., 2014). The brain-controlled NMES in the present study has therefore been focused on wrist movement while continuous antigravity support via a passive multi-joint exoskeleton was provided to the rest of the upper limb.

In the context of neurorehabilitation, NMES is usually applied at supra-motor threshold intensity (referred to as FES) to train either arm or leg function; advanced approaches applied this stimulation to the upper extremity in conjunction with braininterface technology for spinal cord injury patients (Pfurtscheller et al., 2003; Kreilinger et al., 2013; Rohm et al., 2013; Vuckovi ˇ c´ et al., 2015) and stroke survivors (Ethier et al., 2015; Hortal et al., 2015).

In this context, the present study was the first to apply BMIcontrolled subthreshold NMES to support the wrist exercises by extending the ROM in accordance with the actual ability of each patient. Importantly, to avoid under-challenge, stimulation was applied adjunct to voluntary contraction and not as an alternative. An additive stimulation approach such as this was shown to be effective for repetitive task practice of upper limb exercises in severely impaired, chronic stroke patients (Thrasher et al., 2008; Oujamaa et al., 2009; Mann et al., 2011). However, our neuromodulation paradigm remained subthreshold during the task, whereas the aforementioned NMES studies of the upper limb, even if physiologically triggered, followed an allor-nothing concept with supra-threshold stimulation. Our statedependent stimulation, which was controlled by the hybrid BMI, was, therefore, more subtle than in these earlier approaches. Due to the fact that functional muscle contraction was not realized by the stimulation itself, the increased performance was attained by modulations of self-initiated, orthosis-assisted movements. This outcome indicates an overall facilitation of sensorimotor networks by the subthreshold NMES and could constitute a novel restorative strategy in chronic stroke patients suffering from severe impairment of the upper extremity. Further research should investigate whether greater kinematic gains can be attained with other stimulation paradigms, such as the application of increased neuromuscular stimulation or concurrent transcranial current stimulation to facilitate exoskeleton-based motor leaning (Naros et al., 2016b). Our approach, however, led to kinematic gains while still encouraging our patients to participate. Progression of training is required to provide a further challenge for motor learning (Guadagnoli and Lee, 2004). This could be achieved either by means of a decrease in the NMES support level (Meadmore et al., 2014) or by automatic adaption of the level of training during robotassisted stroke rehabilitation (Metzger et al., 2014). Both methods could in future be integrated into this neuroprosthetic set-up without difficulty and, by performing repetitive sessions within intervention studies, their respective clinical relevance in the targeted patient population should be examined more closely.

The neuroprosthesis introduced here holds the promise of bringing even more gains, e.g., via the simultaneous application of further interventions such as brain statedependent cortical stimulation (Kraus et al., 2016b; Royter and Gharabaghi, 2016) to make full use of the salvaged restorative potential. Particularly, during exercises with severely impaired stroke patients, the task-related and muscle-specific facilitation that this device generates could provide the framework for concurrent cortical stimulation. For example, activity-dependent transcranial magnetic stimulation during robot-assisted training could provide such an additional input (Gharabaghi, 2015; Massie et al., 2015) Post-stroke latent corticospinal connectivity may be unmasked during brainrobot interface exercises by associative brain state-dependent stimulation (Gharabaghi et al., 2014a). As per Hebbian-like plasticity rules, such state-dependent stimulation synchronized to maximum gains of assisted ROM could consolidate the corticospinal circuits involved. More specifically, brain-robot feedback-based neuroprosthetic exercises may cause connectivity changes in cortico-cortical motor networks (Vukelic et al., 2014; ´ Vukelic and Gharabaghi, 2015a ´ ) and result in a redistribution of cortico-spinal connections (Kraus et al., 2016a). Therefore, advanced assistive rehabilitation technology such as the one presented here could offer a backdoor to the motor system and

### REFERENCES


provide better prospects of recovery (Bauer et al., 2015). When patients do not gain volitional control of this technology with beta-modulation via a standard EEG-based approach despite the strategies mentioned above (Naros and Gharabaghi, 2015)—e.g., due to an extended cortical lesion and distorted physiology epidural recordings of field potentials may nonetheless facilitate the detection and neurofeedback training of this physiological target (Gharabaghi et al., 2014b). Such an approach closer to the neural signal source may also induce clinical gains after a shorter therapy time than is usually applied with the standard EEG technique (Gharabaghi et al., 2014c) and may even serve as a bi-directional interface for concurrent brain stimulation (Gharabaghi et al., 2014d).

In conclusion, during rehabilitation exercises, the combination of a BMI with neuromuscular stimulation and antigravity assistance has cumulative effects on both ROM and cortical modulation and, as such, may constitute a novel restorative framework for severely affected stroke patients while retaining their voluntary effort. Whether, such technological refinements also result in relevant functional gains will need to be investigated by comparing them in controlled intervention studies with dose-matched, conventional physiotherapy.

### AUTHOR CONTRIBUTIONS

FG participated in the study design and software development, supervised the measurement sessions and carried the data analysis. AW, MS, and WR participated in the software development. GN supervised the measurement sessions. AG participated in the study design and data analysis, and wrote the manuscript. Authors jointly drafted and approved the final manuscript.

### ACKNOWLEDGMENTS

AG was supported by grants from the German Research Council [DFG EC 307], and from the Federal Ministry of Education and Research [BFNT 01GQ0761, BMBF 16SV3783, BMBF 0316064B, BMBF16SV5824]. WR was supported by the Baden-Württemberg Stiftung (GRUENS) and the German Research Council (SP 1533/2-1).


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2016 Grimm, Walter, Spüler, Naros, Rosenstiel and Gharabaghi. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Design and Optimization of an EEG-Based Brain Machine Interface (BMI) to an Upper-Limb Exoskeleton for Stroke Survivors

Nikunj A. Bhagat <sup>1</sup> \*, Anusha Venkatakrishnan1 †, Berdakh Abibullaev <sup>1</sup> , Edward J. Artz <sup>2</sup> , Nuray Yozbatiran<sup>3</sup> , Amy A. Blank <sup>2</sup> , James French<sup>2</sup> , Christof Karmonik <sup>4</sup> , Robert G. Grossman<sup>4</sup> , Marcia K. O'Malley 2, 3, Gerard E. Francisco<sup>3</sup> and Jose L. Contreras-Vidal 1, 4

#### Edited by:

*Michela Chiappalone, Istituto Italiano di Tecnologia, Italy*

#### Reviewed by:

*Surjo R. Soekadar, Department of Psychiatry and Psychotherapy, Germany Alireza Gharabaghi, Functional and Restorative Neurosurgery, Germany*

> \*Correspondence: *Nikunj A. Bhagat nabhagat@uh.edu*

#### †Present Address:

*Anusha Venkatakrishnan, Palo Alto Research Center Inc. (A Xerox Company), Palo Alto, CA, USA*

#### Specialty section:

*This article was submitted to Neuroprosthetics, a section of the journal Frontiers in Neuroscience*

Received: *26 November 2015* Accepted: *13 March 2016* Published: *31 March 2016*

#### Citation:

*Bhagat NA, Venkatakrishnan A, Abibullaev B, Artz EJ, Yozbatiran N, Blank AA, French J, Karmonik C, Grossman RG, O'Malley MK, Francisco GE and Contreras-Vidal JL (2016) Design and Optimization of an EEG-Based Brain Machine Interface (BMI) to an Upper-Limb Exoskeleton for Stroke Survivors. Front. Neurosci. 10:122. doi: 10.3389/fnins.2016.00122* *<sup>1</sup> Non-Invasive Brain Machine Interface Systems Laboratory, Department of Electrical Engineering, University of Houston, Houston, TX, USA, <sup>2</sup> Mechatronics and Haptics Interfaces Laboratory, Department of Mechanical Engineering, Rice University, Houston, TX, USA, <sup>3</sup> NeuroRecovery Research Center at TIRR Memorial Hermann and University of Texas Health Sciences Center, Houston, TX, USA, <sup>4</sup> Houston Methodist Research Institute, Houston, TX, USA*

This study demonstrates the feasibility of detecting motor intent from brain activity of chronic stroke patients using an asynchronous electroencephalography (EEG)-based brain machine interface (BMI). Intent was inferred from movement related cortical potentials (MRCPs) measured over an optimized set of EEG electrodes. Successful intent detection triggered the motion of an upper-limb exoskeleton (MAHI Exo-II), to guide movement and to encourage active user participation by providing instantaneous sensory feedback. Several BMI design features were optimized to increase system performance in the presence of single-trial variability of MRCPs in the injured brain: (1) an adaptive time window was used for extracting features during BMI calibration; (2) training data from two consecutive days were pooled for BMI calibration to increase robustness to handle the day-to-day variations typical of EEG, and (3) BMI predictions were gated by residual electromyography (EMG) activity from the impaired arm, to reduce the number of false positives. This patient-specific BMI calibration approach can accommodate a broad spectrum of stroke patients with diverse motor capabilities. Following BMI optimization on day 3, testing of the closed-loop BMI-MAHI exoskeleton, on 4th and 5th days of the study, showed consistent BMI performance with overall mean true positive rate (TPR) = 62.7 ± 21.4% on day 4 and 67.1 ± 14.6% on day 5. The overall false positive rate (FPR) across subjects was 27.74 ± 37.46% on day 4 and 27.5 ± 35.64% on day 5; however for two subjects who had residual motor function and could benefit from the EMG-gated BMI, the mean FPR was quite low (< 10%). On average, motor intent was detected −367 ± 328 ms before movement onset during closed-loop operation. These findings provide evidence that closed-loop EEG-based BMI for stroke patients can be designed and optimized to perform well across multiple days without system recalibration.

Keywords: brain machine interface (BMI), movement related cortical potentials (MRCPs), motor intent detection, robotic exoskeleton, stroke rehabilitation

## 1. INTRODUCTION

Functional restoration of arm and hand movements is a major goal of post-stroke rehabilitation therapy (Langhorne et al., 2009; Basteris et al., 2014). There exists evidence to suggest that robotassisted therapy improves upper-limb functional assessment scores (Kwakkel et al., 2008; Klamroth-Marganska et al., 2014) and strength (Milot et al., 2013), by inducing activity-dependent cortical plasticity (Hogan et al., 2006; O'Malley et al., 2006; O'Dell et al., 2009). Yet, these improvements fail to reach relevant additional benefits over dose-matched conventional therapy (Kwakkel et al., 2008; Lo et al., 2010; Mehrholz et al., 2012; Klamroth-Marganska et al., 2014) or transfer into functional ability for performing daily living activities (Basteris et al., 2014). It has been suggested, that the slight benefits of robotassisted therapy might be due to unspecific influences such as increased enthusiasm for novel interventions on the part of both patients and therapists (Kwakkel and Meskers, 2014). Notably, robotic training was less effective at restoring arm strength than conventional therapy in the study by Klamroth-Marganska et al. (2014), possibly because the device was too supportive when providing "assistance-as-needed" during the training (Chase, 2014; Brauchle et al., 2015).

Current robot-assisted therapies provide high intensity and repetitive training, but are inadequate in ensuring patient engagement, motivation, and reward, which are important factors for inducing cortical plasticity (Hogan et al., 2006; Basteris et al., 2014; Goodman et al., 2014). Therefore, recent research in robotic therapy has focused on detecting and responding to patient's motor intent, to ensure active participation of the patient during the therapy (Krebs et al., 2003; Blank et al., 2013, 2014; Hu et al., 2013). Typically, motor intent is detected via force (Kahn et al., 2006; Loureiro and Harwin, 2007; Gupta et al., 2008), or electromyography (EMG) activity (Krebs et al., 2003; Hu et al., 2009; Tong et al., 2010; Ho et al., 2011; Lenzi et al., 2012; Vaca Benitez et al., 2013) from the impaired limb's movement and the robot's motion is triggered once the intent is detected. However, these methods are only appropriate for patients who are able to produce some voluntary movement or high enough levels of muscle activity. For more severely impaired patients and to ensure patient engagement, motor intent can also be detected using noninvasive scalp electroencephalography (EEG; Wang et al., 2009; Gomez-Rodriguez et al., 2011; Frisoli et al., 2012; Venkatakrishnan et al., 2014), which is the focus of our work.

Advances in non-invasive scalp EEG have made it possible to analyze neural activity and provide feedback to the patient in real-time via a brain machine interface (BMI) through virtual and physical environments (Farina et al., 2013; Nakagome et al., 2015). Such neurofeedback can facilitate cortical plasticity and motor learning to enhance motor recovery and the resulting BMI paradigm is termed as restorative BMI (Soekadar et al., 2014). In this context, EEG-based restorative BMIs are easy to set up, pose no risks as compared to invasive techniques, and can be readily deployed in a clinical setting for providing rehabilitation therapy in both acute and chronic states. Hence, in recent years, several studies have proposed a neurorehabilitation regimen that augments existing robot-assisted therapy with closed-loop EEG-based BMI (Daly et al., 2008; Gomez-Rodriguez et al., 2011; Ramos-murguialday et al., 2014; Xu et al., 2014b; Ang et al., 2015) or magnetoencephalography (MEG)-based BMI (Buch et al., 2008). The BMI-Robot system usually deploys a robot or exoskeleton to command or guide the patient's movement whenever it detects the patient's voluntary motor intent. However, due to high trial-to-trial variability and poor signal-to-noise ratio (SNR) of EEG signals, detection of intent from single-trials is a daunting task (Bai et al., 2011) and poses a serious challenge to the clinical viability of EEG-based neurorehabilitation therapies. Therefore, the goal of the current study was to develop an asynchronous BMI that can detect voluntary motor intent from chronic stroke patients using EEG and command an upper-limb powered exoskeleton to provide assistance and sensory feedback. The exoskeleton used was the MAHI Exo-II (French et al., 2014), an upper-extremity exoskeleton that guided movements once intent was detected. The main focus of this feasibility study was to design and optimize an EEG-based BMI for intent detection in stroke patients, and hence we did not expect any functional changes during this short-term study.

Generally for EEG-based intent detection, either power modulations in different frequency bands (e.g., µ-rhythms, 8–12 Hz) or time domain amplitude fluctuations (e.g., slow movement related cortical potentials (MRCP) in delta band, 0.1–4 Hz) can be used. Sensorimotor (SMR) or µ-rhythms are characterized by decrease in power (desynchronization) over the contralateral sensorimotor cortex during planning and execution of imagined as well as real limb movements (Buch et al., 2008; Daly et al., 2008; Bai et al., 2011; Gomez-Rodriguez et al., 2011; Muralidharan et al., 2011b; Ramos-murguialday et al., 2014; Ang et al., 2015). In contrast, MRCPs or slow cortical potentials (SCPs) are negatively increasing potentials that occur -1.5 seconds(s) to -2 s before movement onset and reach negative peak at the onset of either self-initiated or predictably-cued movements (Cui and MacKinnon, 2009). The initial negative slope of MRCP preceding self-paced movement is often called Bereitschaftspotential (BP) or Readiness Potential (RP), whereas a similar slow negative potential observed before an imperative stimuli to externally cued movement is termed as Contingent Negative Variation (CNV) (Shibasaki and Hallett, 2006). MRCPs have been used previously to detect intention for self-paced reaching movements (Lew et al., 2012), imagined or attempted ankle dorsiflexion (Xu et al., 2014a,b), sitting and standing transitions (Bulea et al., 2014) and even for discriminating movement direction (Lew et al., 2014). BMIs that detect intent by simultaneously combining information from different types of input signals: MRCPs, µ-rhythms, and β-rhythms (Fatourechi et al., 2008; Ibáñez et al., 2014), as well as brain-neural computer interface systems which use eye movements measured via electrooculography (EOG) for interrupting unintended motion and enhance safety of an EEGbased hand exoskeleton (Witkowski et al., 2014; Soekadar et al., 2015), have also been developed.

Detecting intent from MRCPs is desirable for two reasons: (i) the magnitude and slope of MRCPs modulate with movement characteristics such as force, speed, task complexity, etc., thus providing a versatile motor control signal for capturing patient motor intent (Cui and MacKinnon, 2009; Jochumsen et al., 2013); (ii) the changes in the peak amplitude and latency of MRCPs, could potentially serve as neural indicators of cortical reorganization following motor learning and hence can further help in evaluating the efficacy of BMI-based neurorehabilitation (Yilmaz et al., 2015). Previous studies based on MRCPs have mainly dealt with healthy subjects (Bulea et al., 2014; Xu et al., 2014a,b) and/or in the case of stroke patients, have been conducted offline (Lew et al., 2012; Ibáñez et al., 2014). The brain activity of stroke patients varies to a large extent from that of a healthy intact brain, resulting in significantly differing EEG features for identical tasks (Leamy et al., 2014). Moreover, results obtained with healthy subjects rarely translate to stroke patients, and hence, it is essential to validate the closed-loop performance of MRCP-based BMI in patients with stroke. Therefore, to address this gap in the literature as well as to benefit from the aforementioned MRCP properties, we selected MRCPs for intent detection in this study.

In Section 2, our experimental procedure and methods for BMI calibration as well as for BMI control in real-time are presented. Section 3 presents the results from offline calibration and closed-loop performance using EEG-based BMI. The implications of this study are discussed in Section 4 and the conclusions are presented in Section 5. This study is registered on ClinicalTrials.gov (Identifier: NCT01948739).

### 2. MATERIALS AND METHODS

Four subjects (3 male) with chronic stroke participated in this study, which involved five experimental sessions (or days) per participant. The first 3 days were reserved for BMI calibration, followed by 2 days for testing closed-loop BMI control. Below we provide details for each of the components within this study. Preliminary findings in one stroke and three healthy subjects were reported in Bhagat et al. (2014).

#### 2.1. Subjects

This study was carried out in accordance with the recommendations of the Institutional Review Boards of University of Houston, Rice University, University of Texas Health Science Center, and Methodist Hospital with written informed consent from all subjects. All subjects gave written informed consent in accordance with the Declaration of Helsinki. The inclusion criteria were: (1) age: 18–75 years; (2) chronic stroke (≥ 6 months post-stroke); (3) upper limb hemiparesis associated with stroke, with Manual Muscle Testing (MMT) score ranging from 2 to 4 in the elbow and wrist flexors; (4) no joint contracture or severe spasticity in the affected upper limb; (5) sufficient sitting balance to participate with robotic activities; (6) no hemineglect that would preclude participation in the study protocol; (7) no history of chemodenervation or nerve block for spasticity or pain relief to the affected limb in the past 4 months and no planned alteration in upper-extremity therapy or medication for muscle tone during the course of the study; and (8) no condition (e.g., severe arthritis, central pain) that would interfere with the administration of motor function tests. The exclusion criteria identified were: (1) orthopedic conditions of either upper extremity that would affect performance on the study; (2) untreated depression that may affect motivation to participate in the study; and (3) pregnancy.

#### 2.1.1. Post-Experiment Assessments

For this study, baseline clinical scores were not measured, since we did not expect them to change during the short intervention of this study. Instead, clinical and functional assessments were performed post-experiment to determine the subject's physical and cognitive impairment levels as a result of stroke. Muscle spasticity and motor impairment were evaluated using the Modified Ashworth Scale (MAS, range 0–4, 4 being maximum spasticity) and Fugl-Meyer Arm Assessment (FMA, range 0– 66, 66 being normal function). MAS scores for only elbow portion of the test, i.e., flexor and extensor muscles of the affected hand are reported here, since these muscle groups were predominantly used for operating the exoskeleton. To test for cognitive impairments, the Folstein's Mini-Mental State Exam (MMSE, range 0–30, ≥ 27 implies normal cognition) was conducted. In addition, the NIH Stroke Scale (NIHSS, range 0– 42, 42 meaning severe stroke impairments) was evaluated. Lastly, grip strength was measured using a hand-held dynamometer. **Table 1** provides demographic details and subjects' performance on standard clinical and functional assessment tests that were conducted after completion of study. All subjects recruited were right-handed prior to onset of stroke, although S4 had used his left-hand for writing. S4 had Moyamoya disease and had suffered two strokes, ischemic followed by hemorrhagic, which occurred within a span of 1 month.

T1-weighted Magnetic Resonance Images (MRI) were obtained at the Houston Methodist Research Institute MRI core using a 3T Ingenia (Philips) full body MRI scanner for the purpose of conducting source imaging. A MRI scan protocol with the following acquisition parameters was used: number of acquisitions = 1; acquisition matrix = 252 × 227; TR = 8 ms; field of view = 250 × 200; duration = 5 min, 30 s; slice thickness = 2 mm; flip angle = 8 ◦ ; reconstructed in-plane resolution = 0.78 mm. Scan parameters were adjusted if necessary to account for the anatomy of the subject (such as changing the field of view or number of slices depending on the need for anatomical coverage). MRI images were acquired for all subjects, except for subject S3 who declined the MRI scan because of claustrophobia.

## 2.2. Experimental Setup

#### 2.2.1. Electroencephalography (EEG)

Scalp EEG was recorded using a 64-channel, active-electrode system (actiCAP system, Brain Products GmbH, Gilching, Germany). The EEG amplifier was configured for sampling frequency = 500 Hz, resolution = 16-bit, dynamic range = ±3.2768 mV, and bandwidth = 0−1000 Hz. The EEG electrodes were positioned according to the International 10–20 system (Klem et al., 1999). The ground and reference electrodes were attached to the subject's ears, one on the unimpaired side (ground) and other on the impaired side (reference).



*MAS, Modified Ashworth Scale (range 0–4, 4 indicating maximum spasticity). Only elbow portion of the scores for flexor and extensor muscles of the affected hand are reported here. FMA, Fugl-Meyer Arm score (range 0–66, higher scores representing better arm function).*

*MMSE, Mini Mental State Exam (range 0–30, higher scores representing normal cognition).*

*NIHSS, NIH Stroke Scale (range 0–42, higher score imply severe motor impairments).*

*GS, Grip strength for affected hand reported as percentage of the unaffected hand score.*

*n/a, Data not available, since subject declined the MRI scan because of claustrophobia.*

Four peripheral active electrodes FT9-10, TP9-10 were instead used to record EMG activity from the impaired hand. For this, the active electrodes were replaced with shielded passive electrodes using a splitter box (EIB-64A, Brain Products). Then a pair of shielded electrodes 5 cm apart (bipolar configuration) was placed on each of the biceps and triceps muscles. The EMG ground electrode was attached to the skin at the olecranon process of the unimpaired elbow joint and combined with the EEG ground at the splitter box. The amplifier range for these 4 channels was scaled to ± 327.68 mV using recording software from Brain Products. Thus, EEG and EMG signals were synchronized.

#### 2.2.2. Exoskeleton

The MAHI Exo-II has four actuated degrees of freedom (DOF), but the current study only focused on controlling a single DOF elbow joint and hence, the wrist and forearm actuators were held in a fixed position using set-point proportional-derivative control. The exoskeleton allowed adjusting the range of elbow movement for each subject within 0–60◦ . The exoskeleton's elbow movement was mapped to a solid green ball on the screen using a graphical user interface (GUI), for providing visual feedback. A detailed description of the exoskeleton is reported elsewhere (French et al., 2014). It was operated in two training modes for BMI calibration: user-triggered and user-driven. In the user-triggered mode, the user initiated the movement by pushing against a slight resistive force, and then the robot guided the user in performing the movement. In the userdriven mode, the user initiated and performed the movement without any guidance from the exoskeleton. Further, in the user-driven mode, the exoskeleton was back-drivable with low friction and inertia and only passively recorded the motion kinematics. As compared with the user-triggered mode, the userdriven mode required greater physical effort from the subject during the task. Consequently, subjects with excessive muscle weakness were unable to complete the task in the user-driven mode and hence for such subjects, we used the user-triggered mode only.

The exoskeleton's controller recorded elbow position and velocity by sampling high-resolution encoders at 1000 Hz. The exoskeleton also synchronized data capture with the EEG/EMG system by generating 5 V TTL trigger pulses. Within each trial, triggers were generated when the targets were shown (target-onset), when the subject initiated movement (movement-onset) and when a target was hit (target-reached). Movement-onset was determined during data acquisition whenever the joint velocity exceeded a predetermined threshold value. This threshold was determined on day 1 for each subject by having them move the exoskeleton for five practice trials, at a comfortable speed in user-driven mode. The threshold was then taken as 5% of the average peak velocity obtained from practice trials. For subjects that could not use the user-driven mode, the velocity threshold was heuristically adjusted until the subjects were able to comfortably initiate the exoskeleton's movement in the user-triggered mode. **Figure 1A** depicts the EEG-based BMI to the MAHI Exo-II exoskeleton.

#### 2.2.3. BMI Calibration Task

For calibrating the BMI, subjects attempted self-initiated elbow flexion or extension to move the exoskeleton from the center position toward either an upper or lower target, respectively (Bhagat et al., 2014). The subjects were instructed to first consciously think about their preparation for the impending movement and when ready, move the exoskeleton toward the target as fast as they could. The movements were self-paced with inter-trial fixation for 4–6 s. Trials were presented in blocks of 20 and up to 8 blocks of calibration trials were recorded per day. The calibration routine was repeated on the subsequent day to account for the day-to-day EEG variability when training the BMI. Identical task design was followed for both user-driven and user-triggered modes. The data collection process was tailored depending on the subject's motor ability as shown in **Figure 2**. For subjects S2 and S4 that were able to use both calibration modes, we recorded 8 blocks/day (i.e., 4 consecutive blocks for each mode). On day 2, for these subjects, the order for userdriven and user-triggered modes was swapped from that of day 1. Subjects S1 and S3 could not use the user-driven mode due to excessive motor impairment and hence for them, we decided to use only the user-triggered mode for calibrating the BMI. For all

FIGURE 1 | (A) Schematic of the asynchronous EEG-based BMI. Motor intents detected from EEG activity of a chronic upper-limb impaired stroke patient, are gated by EMG from impaired hand to trigger robot guided movement using MAHI Exo-II (currently elbow flexion/extension only). (B) Timeline for each trial during closed-loop BMI control task. Exoskeleton's motion (forearm extension) and corresponding visual feedback displayed on the graphical user interface (GUI) at significant events on the timeline are shown. Exoskeleton or home (H) position and target (T) position are also indicated. (C) Raster plot displays time-series of selected EEG (MRCP) channels and their spatial average used to detect motor intent, EMG from biceps and triceps muscles, and exoskeleton's kinematics (elbow position, velocity), during closed-loop BMI control. Alternating attempted trials (shaded) from target-onset to target-reached and fixation intervals are shown. Markers indicate BMI predictions (unfilled triangles) and successful intent detection with EMG-gated BMI (filled triangles). Trial #2 shows examples of spurious BMI-only intents i.e., false positives that were successfully rejected by EMG-gate and a missed subject attempt i.e., false negative (marked by arrow) which the BMI failed to detect. Note also the incorrect BMI-predicted motor intent during fixation interval preceding Trial #2, which was rejected by EMG-gate.

subjects, on day 3, we trained a BMI classifier for each calibration mode using data from previous days and additionally fine-tuned the classifier's parameters, which were thereafter kept fixed for closed-loop BMI control.

#### 2.2.4. Closed-Loop BMI Control Task

Once calibrated, the BMI's performance was tested in realtime during days 4 and 5. During closed-loop BMI control, the subject's goal was to use the BMI and initiate flexion or extension movement of the exoskeleton in order to reach the target. In contrast to a calibration trial, where the subjects were free to choose one of the two targets, during BMI control the target was under computer control and alternated between the two spatial positions on the screen (top or down). S1 and S3 had trained with user-triggered mode only and hence for them, we tested the same BMI classifier on both days. For S2 and S4, however, on day 4 we tested the BMI classifier that was calibrated using user-driven mode and on day 5 we tested the classifier trained using usertriggered mode. Regardless of the BMI classifier used, the BMI only triggered the movement of the exoskeleton in both modes. Hence the subjects, which were unknown to the classifier used, attempted the task in the same way.

**Figures 1A,B** show the closed-loop BMI implementation as well as the timeline for a typical trial during online testing. As shown in **Figure 1B**, the robot's current position was shown to the subjects by a solid green ball (home), whereas the fixation and target positions were shown using black and green crosses, respectively. Each trial was preceded by 4–6 s of fixation and lasted for up to 15 s during which the subjects could attempt to start exoskeleton movement using the BMI. During this span, the robot remained stationary and actively resisted any force exerted by the subject. Once the BMI detected intent, the system validated the BMI's decision by comparing it with the EMG activity from biceps and triceps of the impaired limb (Mattia et al., 2013). If EMG activity was detected in either of these muscles within 1 s following the BMI's decision, the algorithm triggered the exoskeleton to execute a pre-recorded motion sequence in order to reach the target. However, if EMG activity was absent following the BMI's decision, then the algorithm rejected the BMI's decision and did not trigger the exoskeleton's movement. The EMG-gated BMI strategy was deployed for reducing the false positives of the BMI classifier. In case the subject was unable to complete the task within the 15 s allotted, a "Timed-out" message was displayed on the screen briefly, followed by the fixation for the next trial. A raster plot of the physiological and kinematics signals along with markers for time points when the BMI had detected intent are shown in **Figure 1C**.

To help evaluate misclassification or false positives, a few randomly selected trials within a block were presented as "catch" trials or rare events. During a catch trial, the subjects were instructed to not think/attempt to move the robot (i.e., a planned No-go) for the entire 15 s interval. To distinguish a catch trial from a regular trial, the target was shown as a large red ball. If the EMG-gated BMI did detect intent during the catch trial it triggered the robot to move and its decision was recorded as a false positive. Each block contained from 1-5 catch trials and their order was randomized. The unbalanced ratio of catch trials (rare events) to regular trials was selected in order to allow subjects to practice BMI control of the exoskeleton and be able to learn to use the BMI for performing the movement. A balanced distribution of trials, was however maintained during offline cross-validation to get an initial estimate of classifier's performance, as described in Section 2.3.3.

On average, an entire block (i.e., 20 trials) was completed in 6.55 ± 0.64 minutes. The number of blocks completed during closed-loop BMI control varied across subjects due to subject fatigue and availability.

#### 2.3. BMI Decoder Calibration

#### 2.3.1. Signal Processing

Offline data analysis was performed using MATLAB's Signal Processing and Statistics toolboxes (MATLAB, 2012), EEGLAB (Delorme and Makeig, 2004), and R Programming Language's Signal and R.matlab packages (Signal Developers, 2013; Bengtsson, 2014; R Core Team, 2014). To detect MRCPs using features extracted from EEG signals, a classifier was trained as described below and as shown in **Figure 3**.

EEG data recorded on days 1 and 2 were appended and filtered in the low frequency delta band (0.1–1 Hz) (Lew et al., 2012). The filters were applied in succession, i.e., initially EEG signals were high-pass filtered (causal, 4th order Butterworth, −3 dB cutoff freq. = 0.1 Hz), then re-referenced using Large Laplacian spatial filter (McFarland et al., 1997) and finally low-pass filtered (causal, 4 th order Butterworth, −3 db cutoff freq. = 1 Hz). Although Butterworth (IIR) filters introduce non-linear phase distortion and sometimes can be unstable, they are recommended over FIR filters when computational efficiency, sharp cutoffs and high throughput causal systems are required (Widmann et al., 2015). Also, by using a causal filter offline, we ensured that the phase distorted EEG signals used to train the classifier, would be similar to those which the classifier will encounter during real-time.

The filtered signals were downsampled to 20 Hz and segmented into epochs extending from [−2.5 s 1 s] with respect to target-onset and movement-onset triggers. Epochs aligned to movement-onset corresponded to the subject's preparation for

2.4).

movement, during which MRCPs are known to be generated (Shibasaki and Hallett, 2006; Cui and MacKinnon, 2009; Lew et al., 2012). Hence, these epochs (or trials) were labeled as belonging to "Go" class. Similarly, epochs aligned to target-onset were labeled as "No-go" class, since the subjects were at rest and fixating during this interval. All Go epochs were visually inspected for corruption by eye blinks or movement artifacts and the corrupted epochs were removed. For each Go epoch removed, the corresponding No-go epoch was also removed, to maintain equal class distribution. Thus, 154 ± 10 epochs per class were retained across subjects.

Next, the Go epochs were baseline corrected, by subtracting from each epoch its mean amplitude over [−2.5 s −2.25 s] interval prior to movement-onset and then time averaged, to obtain a grand-averaged waveform for each EEG channel. Baseline correction was used only during the computation of grand-averages to aid in the visualization of MRCPs and was not applied during classifier design, since it did not affect the classifier's accuracy. Previous studies show that MRCPs are observed in grand-averaged Go epochs of EEG channels over the primary motor cortex, pre-motor and supplementary motor areas (Shibasaki and Hallett, 2006; Lew et al., 2012). In addition, for stroke patients the MRCPs are distributed bilaterally over both cortices as compared to dominant contralateral distributions observed in healthy subjects (Yilmaz et al., 2015). Therefore, we visually inspected the grand-averages for channels over sensorimotor cortex and selected only those channels for which MRCPs were identified. From these channels, we manually selected a subset of channels that achieved best classification accuracy. While automated channel selection may be preferred over the manual approach taken here, a previous study found that the classifier performed equally well for both approaches (Lew et al., 2012).

Further, for each trial, the EEG epochs (V<sup>k</sup> (t)) from above selected MRCP channels (= M) were merged using a spatial average that is given by, X(t) = 1 M P<sup>M</sup> <sup>k</sup>=<sup>1</sup> V<sup>k</sup> (t). Spatial Averaging or mean filtering is a standard image processing technique for smoothing and reducing noise in images by reducing the intensity variations in neighboring pixels (Fisher et al., 2003). We applied spatial averaging for smoothing the single-trial variations of MRCP channels and thus computed a global MRCP representation for motor intent. Trials for which the spatial averaged MRCP peaked earlier than −1.5 s before movementonset were most likely corrupted by artifacts and such trials were removed from the training set.

#### 2.3.2. Optimal Window for Segmenting Go and No-go Epochs

In order to extract EEG features, we segmented the Go and No-go epochs using two equal length windows. To specify a window we determined two parameters: the location of its leading edge (i.e., onset time) and its length (i.e., looking back into the past starting from onset time). Previous studies have used fixed location windows with pre-decided length. For example, in Lew et al. (2012) a fixed window from [−0.75 s −0.25 s] was used for segmenting the Go epochs, across all subjects. As shown in Section 3.2, this technique may result in poor classifier performance due to trial-to-trial variability of MRCPs. To overcome this drawback, we propose an adaptive window technique where the window location and its length for each subject is optimized to best capture the negative slope of MRCPs and compensate for its trial-to-trial variability. For this, in each trial, the location of the Go window was kept variable and made to coincide with the time when the spatially averaged MRCP reached its negative peak. Since we expected the MRCPs to be absent during fixation interval, the location of the No-go window was arbitrarily fixed at −0.5 s before target-onset.

Subsequently, the length of the Go and No-go windows were iteratively increased from 0.5 s to 1 s in steps of 50 ms. In each iteration, the classifier's performance was evaluated by computing the area under its receiver operating characteristics (ROC). Finally, the shortest window length that achieved the maximum area under the ROC curve was selected as the optimal window length (wlO). The window length optimization loop was also applied to the conventional fixed window technique and its performance was compared with the adaptive window technique.

#### 2.3.3. Feature Extraction and Classifier Design

After segmenting the Go and No-go epochs, four time domain features were computed from the segmented epochs, namely slope, negative peak amplitude, area, and Mahalanobis distance. Thus, for N trials, we have N Go and No-go epochs each, resulting in a 2N × 4 feature matrix. The Mahalanobis distance (d) for each windowed epoch is calculated as its distance from the cluster of all windowed epochs belonging to the Go class. Thus,

$$d = [(\mathbf{x} - \boldsymbol{\mu})^\top \boldsymbol{\Sigma}^{-1} (\mathbf{x} - \boldsymbol{\mu})]^{-\frac{1}{2}}$$

where **x** is a vector of signal amplitude for each Go or Nogo epoch, µ and 6 are the mean and covariance matrix for the cluster of all Go samples (Duda et al., 2012). It is reasoned that during classification, a target or unlabeled epoch containing MRCP will be similar in shape to the known or labeled Go epochs and hence will have a smaller Mahalanobis distance (ideally 0). To minimize computation time during closed-loop BMI control, µ and 6 were saved during calibration and re-used later in real-time.

A binary Support Vector Machine (SVM) classifier was trained to discriminate between the Go and No-go epochs. The SVM classifier was implemented using LIBSVM library (Chang and Lin, 2011). The library's C-Support Vector Classification (C-SVC) formulation with Radial Basis Function (RBF) kernel defined as K(**xi**, **xj**) = e −γ k**xi**−**xj**k 2 , γ > 0 was used. The regularization and kernel parameters (C, γ ) were optimized using the grid search technique for different combinations of C ǫ {10,100,1000} and γ ǫ {0.2, 0.5, 0.8, 1}. LIBSVM extends traditional SVM implementation and provides a probability estimate, i.e., P(y = Go | **x**), given a sample vector **x** (Chang and Lin, 2011). To classify a testsample as Go, it is required that P(y = Go | **x**) ≥ τ , where τ is the detection threshold (ideally τ = 0.5).

Stratified 10-fold cross-validation was used to evaluate the classifier's offline performance. During cross-validation, to test the classifier on an unseen trial, we used a sliding window that was shifted every 50 ms from [−2.5 s 1 s] with respect to either movement-onset or target-onset. The sliding window's length was set equal to the Go window length during that iteration of the optimization loop. This cross-validation scheme more closely resembled real-time BMI control by preserving the chronological order of the data and provides a more conservative estimate of accuracy than a conventional cross-validation scheme (Lew et al., 2012; Niazi et al., 2013).

As the sliding window shifted through a trial, if three consecutive windows were predicted as Go, then that trial was classified as Go. In this case, the average probability over the three consecutive Go decisions was assigned to that trial. Alternately, if the decision was No-go then the average probability over all No-go decisions within the trial, was assigned to that trial. By grouping the assigned probability estimate on all test trials from the 10 folds, the classifier's ROC curve was computed (Fawcett, 2006). The ROC curve was computed for each window length iteration and the shortest window length that resulted in the maximum area under ROC curve was chosen as optimal window length (wlO). After deciding wlO, the classifier with the highest accuracy amongst the 10 cross-validation folds for that wlO, was selected for closed-loop BMI implementation.

To test whether our classifier performed better than chance, we shuffled the class labels for 1000 times and for each permutation we calculated the mean classification accuracy after repeating the 10-fold cross validation. If the classifier performed better on the original training set than 95% of randomized samples, i.e., if empirical p−value < 0.05, then the difference in the mean classification accuracy was considered significant (Ojala and Garriga, 2009).

### 2.4. Closed-loop BMI Implementation

For closed-loop BMI control, a custom MATLAB graphical user interface was developed that streamed EEG and EMG signals in real-time using Brain Products's streaming library. After filtering and downsampling to 20 Hz, the spatial average of selected MRCP channels was computed. A sliding window of length equal to wl<sup>O</sup> generated the classifier's prediction every 50 ms. If the prediction's probability estimate exceeded the decision threshold (τc) for N<sup>c</sup> number of consecutive windows, only then the BMI made a Go decision. The parameters τ<sup>c</sup> and N<sup>c</sup> were empirically tuned on day 3 for each subject and for each calibration mode and subsequently were kept fixed on days 4 and 5.

Furthermore, to implement the EMG-gated BMI strategy, EMG signals (biceps and triceps) from the impaired hand were band-pass filtered (30–200 Hz, 8th order, Butterworth) and their root mean square (RMS) amplitude over a 300 ms interval was computed. The RMS amplitude was compared to pre-set thresholds for the biceps and triceps in order to detect EMG activity. As soon as the BMI predicted motor intent, a one second timer was started. If EMG activity was detected before the timer overflowed, then the BMI's decision was accepted and the exoskeleton performed the movement. Otherwise, the BMI's decision was rejected. EMG activity from either biceps or triceps can be used for gating both movements, i.e., initiate exoskeleton's movement, irrespective of whether the desired motion was flexion or extension.

#### 2.5. Performance Evaluation

True Positive Rate (TPR) and False Positive Rate (FPR) were used to evaluate the BMI's performance on days 4 and 5. TPR was defined as the fraction of attempted trials for which the motor intent was correctly detected, within each block. FPR was defined as the fraction of catch trials for which the motor intent was incorrectly detected, within each block. Two-sided Wilcoxon Rank Sum test was used to determine if the BMI's performance significantly differed between days 4 and 5. As subjects S2 and S4 attempted the task in the same way during closed-loop BMI control, regardless of whether the BMI was calibrated using user-driven or user-triggered mode, we compared the BMI's performance in their case as well.

In addition, based on the time required by the BMI to detect intent within a 15 s trial, we estimated the number of motor intents the BMI could detect per minute (min). This metric, referred simply as Intents per min = 60 × (Time(s) to detect intent)−<sup>1</sup> , measures the responsiveness of the BMI to the subject's motor intention. Also, we calculated the coefficient of variation (CoV) for Intents per min, to measure how dispersed their distribution was within a block. CoV was defined as the ratio of standard deviation to mean values of intents per min for a block. Furthermore, we computed the latency between motor intent detection by the BMI and the physical onset of subject's movement during closed-loop control. Physical movement onset was determined from the kinematic data, i.e., when the joint velocity exceeded a pre-set threshold.

We also asked the subjects to provide feedback on the accuracy of the BMI during closed-loop using a 5-point Likert scale. After each trial, the participants were asked: "How accurate was the BMI's decision in this trial?". In response, the subjects provided a rated score from 1-5 where: 1-completely inaccurate, 2-moderately inaccurate, 3-not sure, 4-moderately accurate, 5 completely accurate.

Finally, to help elucidate the neural networks involved in the generation of intent in stroke patients we localized the neural signals generated in the time interval leading to the detection of motor intent during closed-loop BMI control. Cortical sources of MRCP were estimated for each subject on a trial by trial basis for days 4 and 5. The average source activation for each block and the grand-average across blocks for each day was then computed. For details on source analysis and its outcomes, refer to Supplementary Materials.

## 3. RESULTS

### 3.1. MRCPs in Stroke Subjects

**Figure 4** depicts the MRCPs for subject S4. This subject was able to use both BMI calibration modes and the left and right columns correspond to the user-driven and user-triggered modes, respectively. **Figures 4C,D** show grand-averaged traces with 95% confidence bounds for all channels shown in **Figures 4A,B**. Note that the negative peak of MRCP lags by ∼0.5 s with respect to movement-onset due to the non-linear phase distortion of IIR filters used for preprocessing EEG. As seen in **Figures 4C,D**, channels FCz, FC1, Cz, C1-C3, CPz, CP1-CP3 illustrate strong MRCPs whereas the remaining channels FC2- FC4, C4 and CP4 do not show any discernible MRCPs. **Table 2** lists MRCP channels identified for all subjects. MRCP channels that were later used by the BMI classifier for detection of motor intent are marked by red circles in **Figures 4A,B** and shown in bold-face in **Table 2**. In addition, **Figures 4E,F** show raster plots of color-coded single-trial EEG epochs (only for Go epochs), for the selected MRCP channels and their spatial average. The

classifier. Shaded blue and black circles represent reference and ground electrodes respectively, which were attached to the subject's ears. (C,D) show baseline corrected grand-averages ± 95% confidence intervals using thick and thin black lines, respectively. In the figures, the peak of MRCP is lagging (∼0.5 s) the time of movement-onset (MO) due to the non-linear phase distortion of IIR filters. (E,F) display raster plots of single-trial EEG amplitudes, without baseline correction, for channels used to train the classifier (columns 1–4) and their spatial average (column 5). The trials were sorted in increasing order of latency, which is defined as the time interval starting from 0.5 s up to the negative peak of spatial average. In column 5, trials for which the peak negativity of spatial average occurred earlier than −1.5 s (vertical black line) with respect to movement-onset, were rejected when training the classifier since these trials are most likely corrupted by artifacts.

raster plots were created using EEGLAB's erpimage() function (Delorme and Makeig, 2004). As compared to 1-D grandaverages, ERP-image plots provide a 2-D representation (epoch times × epoch amplitudes) of single-trial MRCPs and help in visualizing their inter-trial variability. The ERP-images were first sorted and then vertically smoothed using a moving average filter of length = 2 trials. The sorting order was determined from the time instant at which the spatially averaged MRCP reached a negative peak, within the interval [-2 s 0.5 s]. Epochs for which the negative peak occurred closer to 0.5 s after movement-onset, where ranked higher than other epochs, whereas epochs with negative peak occurring earlier than −1.5 s were rejected during classifier training. **Table 2** shows the initial number of trials (per class) as well as the number of trials that satisfied this criterion. Amongst trials that satisfied our criteria, we found approximately equal distribution of trials between days 1 and 2. This is also indicated by the number within parenthesis in **Table 2**.

## 3.2. Comparison of Fixed and Adaptive Window Techniques

**Figure 5** compares the fixed and adaptive window techniques using a sample dataset (subject S4, user-triggered mode). For comparison outcomes in other subjects, refer to Supplementary Materials. In **Figure 5A**, a few single-trial spatial-averaged MRCP epochs from calibration data recorded for subject S4 (usertriggered mode) are shown. In the left column, a fixed window is shown that was shifted by 0.5 s after movement-onset to compensate for the filtering delays. As seen in this figure, the fixed window approach often fails to capture the negative MRCP slope in all trials and instead segments a mixture of rising and falling signal trends. Alternatively, as seen in **Figure 5A** (right column), the adaptive window approach consistently captures the negative slope of MRCP for each trial. **Figure 5B** shows the 4-D feature space using 2-D scatter plots (top and bottom), for both the fixed and adaptive windows.


TABLE 2 | Optimized parameters for offline calibration and closed-loop testing of BMI control.

*UD, user-driven mode; UT, user-triggered mode. Optimal window length (wlO) is reported here for only the adaptive window approach.* τ*<sup>c</sup> and N<sup>c</sup> are the decision thresholds for classifier's probability estimate and number of consecutive windows, respectively.*

\**Channel with bold-faced labels were later used for training the classifier.*

*† Initial no. of trials* = *Total calibration trials recorded* − *trials rejected by visual inspection.*

*‡ Number of trials eventually used for training the classifier, after rejecting trials which did not meet our criteria (see Section 2.3.1 for details). Additionally, the parenthesis indicates number of trials belonging only to day 1, which were short-listed for classifier training.*

**Figure 5C** shows the ROC curves for the classifier performance when using fixed and adaptive window techniques, whereas **Figure 5D** shows the optimal window length (wlO) selected for each technique. In addition, **Figure 5E** compares the 10-fold cross-validation accuracies that were obtained during calibration (offline), for each subject and calibration mode. Here accuracy refers to the percentage of correct predictions from the total predictions. Using one-sided Wilcoxon Rank Sum test, the classification accuracy for adaptive window was found to be significantly better than for fixed window for subjects S3 (p < 0.05) and S4 (p < 0.01). The median and maximum classification accuracy across all subjects was higher for adaptive window over fixed window. Higher classification accuracy is important because the classifier with the highest cross-validation fold accuracy amongst the 10 folds, was selected for closed-loop BMI implementation. Besides accuracy, the adaptive window approach also achieved larger area under the ROC curve in a majority of the cases, except for subject S2 in user-driven mode, as can be seen from **Figures 5C,D** and Supplementary Figures S-4–S-7. Interestingly, all classifiers were significantly better than random chance (49.6 ± 2.2%), irrespective of whether fixed or adaptive windows were used. Since the adaptive window performed better than the fixed window, we selected the classifier trained using adaptive window for closed-loop BMI control. **Table 2** lists the adaptive window lengths that were optimized for each subject, as well as the closed-loop BMI parameters which were fine-tuned on day 3.

#### 3.3. Closed-loop BMI Performance

**Figure 6** shows the median and interquartile range for block wise TPR and FPR that were obtained on days 4 and 5. Additionally, Table S-3 (Supplementary Materials) presents the mean ± SD values for the different metrics that were considered in this study to evaluate the closed-loop BMI performance. When considering each subject's performance individually, all subjects except S1, showed significant difference in TPR between both days. For S2, the difference was negative i.e., BMI performed very well on day 4 (user-driven mode) with maximum TPR = 100% and 0 false positives and on day 5 (user-triggered mode), however the TPR significantly decreased (p < 0.05) as well as the FPR marginally increased. On the other hand, for S4, the results were quite the opposite. In S4's case, the TPR on day 5 was significantly better (p < 0.01) than day 4 and there were only a few false positives on day 5.

To further understand how the BMI's performance evolved within each session and across both sessions, we estimated the number of intents per minute. In **Figure 7**, the block wise intents per min for days 4 and 5 are shown. Within each block, the intents per min were calculated for only those trials for which the BMI correctly detected intent. Underneath each boxplot for intents per min, we plot the block wise coefficient of variation (CoV). The overall distribution of intents per min and its CoV for each day is shown by accumulating the values obtained for that day.

In general, the number of intents detected per min by the BMI largely fluctuated across blocks. However, for S2 (day 4), using regression analysis we found that the median values for intents per min significantly increased across blocks. Since, within a session the BMI was kept fixed, this suggests that within a single session, with repeated practice S2 had learned to effectively control the BMI. Moreover, the increase in intents per min also corroborates well with our previous result when we found that for S2 on day 4, the BMI performed almost perfectly. A similar

trend was also found for S1 on day 4, which tended towards significance (p = 0.055). Overall on both days, the median number of intents detected per min hovered around 7-12. The CoV estimates remained fairly uniform for S1 (day 4) and S2 (days 4 and 5). However, for S4 we found that on each day, the CoV significantly decreased as the subject practiced with more blocks. This suggests that with more block repetitions, the variance in BMI's performance decreased and it was able to consistently detect the subject's intent. The subjects' rating of the BMI performance averaged 3.15 ± 1.68 over the two days of closed-loop testing, indicating the subjects felt the BMI system responded on average to their intent (Table S-3, Supplementary Materials).

conditions. UD, user-driven mode; UT, user-triggered mode.

#### 4. DISCUSSION

In this study we designed and optimized an asynchronous EEG-based BMI to perform goal-oriented movements using feasibility of the BMI system was validated in four chronic stroke patients over two days. The proposed BMI can be calibrated using either the user-driven or user-triggered modes of the exoskeleton, to accommodate patients with varying levels of motor impairment. Further, the BMI's false positive rate was substantially reduced by incorporating an EMG-gate as a ground truth for the subject's motor intentions. The BMI paradigm was designed to be asynchronous such that the subjects were free to attempt the trial any time after the target appeared (in fact, an instruction stimulus informed the subjects that they could start the volitional trial any time they wished) and before the trial timed out, while the BMI was continuously analyzing the ongoing brain activity (Leeb et al., 2007). This approach differs from a synchronous BMI, wherein the EEG is analyzed in predefined time intervals and the participants are instructed to imagine their movement following a auditory cue presented by the system (Brauchle et al., 2015).

an upper extremity powered exoskeleton (MAHI Exo-II). The

While both asynchronous and synchronous are feasible for the current application, the former approach provides more flexibility by allowing the user to control the timing of the exoskeleton's movement or otherwise remain idle (Leeb et al., 2007).

The overall performance across all subjects, combined over both days was TPR = 64.86 ± 18.35% and FPR = 27.62 ± 36.37%. Also, the mean TPR on day 5 (67.08 ± 14.55%) was consistent with the mean TPR for day 4 (62.71 ± 21.43%). Although the mean FPR including all subjects was 27.74 ± 37.46 on day 4 and 27.5 ± 35.64 on day 5, when considered individually, S2 and S4 had very low FPR (< 10%). As seen from **Table 1**, subjects S1 and S3 are older and more severely impaired (mean age = 63 years, mean FMA score = 16.5), as compared to S2 and S4 (mean age = 34 years, mean FMA score = 25.5). Moreover, due to excessive motor impairment, S1 and S3 were unable to use the user-driven mode and their EMG signals were weak and unreliable. These factors could have contributed to excessive FPR in these subjects. However, for S1, while there was no change in the median TPR and FPR on both days, the variability in TPR reduced considerably on day 5. This was also true for S3, where in fact the TPR on day 5 significantly (p < 0.05) improved over day 4. This suggests that these subjects were adapting well to the BMI paradigm, despite their severe motor impairment and possibly age-related cognitive decline.

In **Table 3**, we compare our results with previous offline and online BMI studies that have tested EEG-based intent detection, specifically with stroke patients. While a majority of the online BMI studies have focused on sensorimotor rhythms (SMR) for detecting intent, we observed comparable performance using MRCPs. Interestingly, using MRCPs alone we were able to achieve offline true positive rates (82 ± 16%) matching that of an hybrid (SMR + MRCP) BMI (82 ± 10%) which was proposed in Ibáñez et al. (2014).

In addition to online intent detection accuracy, the latency for intent detection is also a significant factor in determining the clinical viability of BMI-based neurorehabilitation therapy. Ideally, the intent for movement should be detected well in advance to allow a casual and seamless transfer from motor intention to movement execution via the exoskeleton (Grosse-Wentrup et al., 2011; Niazi et al., 2013). Moreover, the concomitant activation of the motor cortex during movement planning and the afferent sensory feedback provided by the exoskeleton is necessary for inducing neural plasticity as per Hebbian theory (Grosse-Wentrup et al., 2011; Muralidharan et al., 2011a).

Therefore, it is encouraging that the proposed BMI was able to detect intent before actual movement onset in nearly all subjects (Table S-3, Supplementary Materials). The overall detection latency across both days was -367 ± 328 ms prior to the subjects' physical movement onset. These results are comparable to the latencies reported in previous studies with stroke and healthy subjects: -620 ± 250 ms (Bai et al., 2011), -460 ± 85 ms (Lew et al., 2012), -152 ± 238 ms (Niazi et al., 2013), -317 ± 73 ms (Jochumsen et al., 2013), etc. and support the feasibility of detecting motor intent in patients with stroke using MRCPs.

The EMG-gated BMI approach, presented in this study, acts like a logical AND between the BMI and EMG predictions and hence its performance represents a lower bound on the TPR and FPR of the EMG-only condition, i.e., the TPR/FPR for EMG-only condition will be at least as much as EMG-gated BMI or higher. To confirm this, for subjects S2 and S4 who had residual motor function and could benefit from EMG-gating, an offline analysis was performed to compute the TPR and FPR when considering an EMG-only controller. The results were for S2, on day 4, TPR = 91 ± 10%; FPR = 4 ± 11% and on day 5, TPR = 88 ± 9%; FPR = 33 ± 44%. For S4, on day 4, TPR = 80 ± 9%; FPR = 17 ± 19% and on day 5, TPR = 89 ± 6%; FPR = 6 ± 17%. The higher FPR obtained in the EMG-only condition for S2 (days 4 and 5) and S4 (day 4) occurs because it uses simple thresholding as compared to the conservative classification approach applied by EMG-gated BMI (see Table S-3, Supplementary Materials). The trade-off however, is that the TPR of the EMG-gated BMI is also reduced. Thus, as compared to using an EMG-only controller, the EMG-gated BMI approach improves the specificity of intention detection at the cost of reduced sensitivity.

It is interesting to note that some evidence of operant conditioning of neural activity may have occurred in S2 wherein a linear increase in number of motor intents was detected across sessions (**Figure 7**). The combination of visual and proprioceptive feedback associated with robot-assisted arm movement could have promoted increased volitional control of

for intents per min and individual values of CoV across all blocks for that day, along with their slopes. UD, user-driven mode; UT, user-triggered mode.

movement-related cortical activity in the patient. While seen only in one patient in our initial feasibility study, this finding is particularly interesting as it provides additional support to the hypothesis that BMI-assisted robotic rehabilitation therapies can trigger neural plasticity, similar to the findings of (Naros and Gharabaghi, 2015). In our future clinical trial, we plan to further study and leverage the effect of such operant conditioning to enhance effectiveness of each training session.

Previous studies have found that MRCPs occur bilaterally over the scalp during motor preparation, and gradually become more lateralized before and during the movement execution (Platz et al., 2000; Shibasaki and Hallett, 2006). Thus, in our application, bilateral activation for detection of motor intent was expected. Furthermore, due to the maladaptive higher involvement of the unaffected hemisphere during motor preparation of paretic hand, ipsilateral over-activation (i.e., higher negative amplitudes) and a contralateral lower activation can be observed (Yilmaz et al., 2015). This was also observed in the current study in **Figures 4A,B** for subject S4, who was impaired in the left-hand. Therefore, we used brain activity over both hemispheres for implementing the BMI. This approach differs from conventional restorative BMIs that rely completely on ipsilesional brain activity (Soekadar et al., 2014). However, the afferent sensory feedback provided by exoskeleton movement is provided to the affected arm, thus encouraging patients to actively participate in the therapy and thereby achieve better functional recovery (Daly and Wolpaw, 2008; Venkatakrishnan et al., 2014).

### 4.1. Trial-to-Trial Variability of MRCPs

In the literature, different signal processing and machine learning techniques have been proposed to improve SNR of EEG signals and reduce variability of MRCPs for single-trial intent detection. In Garipelli et al. (2013), the authors propose an optimal spectral filter with pass-band [0.1–1 Hz] and a combination of common average reference and smoothening spatial filters for preprocessing EEG signals, followed by Linear Discriminant


#### TABLE 3 | Review of EEG-based motor intent detection studies in stroke patients.

*SMR, Sensorimotor or* µ*-rhythms; MRCP, Movement related cortical potentials; AUC, performance reported as area under the ROC curve.*

*<sup>a</sup>Study included both healthy and stroke participants. Here we mention only the number of stroke participants. If available, we report only the BMI accuracy that was obtained with stroke patients.*

*<sup>b</sup>Study reported both online and offline accuracies. Here we only consider online accuracy.*

*<sup>c</sup> Overall TPR and FPR computed offline during BMI Calibration. Note that EMG gating was not used when computing offline accuracy.*

Analysis (LDA). Alternately, a high dimensional time-embedded feature matrix, which at each time point incorporates MRCP samples from up to 50 ms in the past, followed by dimensionality reduction and classification using Gaussian Mixture Models, has been proposed in Bulea et al. (2014). Yet another approach, combines high dimensional spatio-temporal ERP features and subsequently classifies into target vs. non-target using either regularized-LDA or multiple Logistic Regressors (Blankertz et al., 2011; Marathe et al., 2014). Interestingly, in all the above studies for extracting features for training the classifier, a fixed window was used. In this study, we addressed this issue by proposing an adaptive window technique for extracting MRCP features during classifier training.

Single-trial EEG variability has been traditionally attributed to changes in background neural activity and other non-neural artifacts (Blankertz et al., 2011; Garipelli et al., 2013). However, it is possible that the temporal and amplitude variability in EEG reflects changes in task performance, neural adaptation/learning and endogenous changes in global brain state due to fluctuations in sustained attention, fatigue, etc. (Goldman et al., 2009; Marathe et al., 2014). Studies examining the relation between MRCPs and movement speeds have found that for faster movements, the onset of MRCP (or BP) was delayed and it peaked sooner than for slower movements (Shibasaki and Hallett, 2006; Gu et al., 2009). These findings suggest that single-trial variability of MRCPs could also be influenced by the subject's volition to select the movement speed and direction for a trial. In addition, changes in fatigue and attention can introduce variability in EEG, especially if the same task is repeated over several trials. Although a detailed analysis of this conjecture is outside the scope of this paper, it led to the design of the adaptive window technique, for minimizing the effect of MRCP variability on the classifier's performance.

#### 4.2. Study Limitations

One potential limitation of the current study is the effect of artifacts on the classifier's performance. In this study however, we use low frequency, narrow delta band (0.1–1 Hz) EEG activity before movement onset, which according to previous studies (Lew et al., 2012; Bulea et al., 2014), is unlikely to be contaminated by motion or muscular artifacts. Also, it has been found that ocular artifacts mainly affect the frontal EEG channels (Lew et al., 2012), which we did not use for detecting intent. Moreover, we used only the central EEG electrodes over the sensorimotor cortex, which are less likely to be corrupted by any muscular or ocular artifacts, if any. In addition, during offline calibration, we visually rejected noisy epochs from the training set to minimize their effects. Therefore, it is likely the effect of artifacts, if any, on the classifier's performance was negligible. It is also important to note that the manual selection of channels for analyzing MRCP used in this study will not scale for a larger number of patients. Therefore, in future work we plan to further investigate methods to automate channel selection personalized to each patient.

As seen in **Figure 5E**, the higher variations in the crossvalidation accuracy of adaptive approach could have resulted from overfitting, since it uses smaller number of training samples (117 ± 16) per subject, which were retained using our criteria (see Section 2.3.1); whereas for the fixed window approach we used all training samples (154 ± 10) per subject. To overcome this limitation, larger number of training samples will be recorded and techniques for preventing overfitting of cross-validation data (Ng, 1997), will be considered in future. The proposed BMI performed well for less affected patients that could, in addition, benefit from EMG-gating. However, for more severely affected patients, other solutions than the one presented here may be required and should be explored further. Closed-loop control of BMI systems also has the potential to actively engage learning and adaptation and therefore change cortical activity (Orsborn and Carmena, 2013). The present feasibility study did not investigate this possibility. This question would be better addressed in a longitudinal study with a larger cohort of stroke patients.

#### 5. CONCLUSIONS

This study demonstrates the feasibility of using movement related cortical potentials (MRCPs) recorded via EEG, to design a closed-loop BMI system for detecting motor intent of chronic stroke patients over multiple days and without recalibrating the BMI. Using the adaptive window approach proposed here together with calibration data from multiple days, we demonstrated closed-loop BMI performance, in spite of intertrial variability and poor SNR of MRCPs. Our methods were validated in four stroke patients with varying severity of motor impairments, who were able to use the EEG-based BMI in real-time to control an upper-limb exoskeleton (MAHI-Exo II). We are currently testing our BMI approach in a clinical trial

REFERENCES


involving a larger population of chronic stroke patients to assess the potential benefits of using a personalized closed-loop BMI system for robot-based upper-limb rehabilitation.

#### AUTHOR CONTRIBUTIONS

JC, MO, and GF conceived the project. NB, AV, EA, MO, GF, and JC designed the experiments. MO supplied the exoskeleton (MAHI Exo-II) and provided inputs to EA and JF for adapting the exoskeleton. AV, NY, and GF recruited the subjects. NB, AV, EA, and AB ran the experiments and collected the data. EA, AB were responsible for operating the exoskeleton. NY performed clinical assessment. CK, RG were responsible for MRI scans and interpretation. NB, BA, and EA analyzed the data with supervision by JC. NB performed the BMI design and optimization with input from JC. NB drafted and wrote the manuscript. BA performed the source analysis (refer Supplementary Materials). AV, AB, RG, MO, GF, and JC reviewed the drafts and made substantial comments. All authors read and approved the final manuscript.

#### ACKNOWLEDGMENTS

This work was supported by NIH Grant R01NS081854 under the National Robotics Initiative.

#### SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: http://journal.frontiersin.org/article/10.3389/fnins. 2016.00122

a novel upper limb robotic rehabilitation system for stroke," in Engineering in Medicine and Biology Society (EMBC), 2014 36th Annual International Conference of the IEEE (Chicago, IL), 4127–4130.


Duda, R. O., Hart, P. E., and Stork, D. G. (2012). Pattern Classification. Wiley.


robotic assistance of motor recovery. J. Rehabil. Res. Dev. 43, 605–618. doi: 10.1682/JRRD.2005.06.0103


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2016 Bhagat, Venkatakrishnan, Abibullaev, Artz, Yozbatiran, Blank, French, Karmonik, Grossman, O'Malley, Francisco and Contreras-Vidal. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# What Turns Assistive into Restorative Brain-Machine Interfaces?

#### Alireza Gharabaghi\*

*Division of Functional and Restorative Neurosurgery, and Centre for Integrative Neuroscience, Eberhard Karls University Tuebingen, Tuebingen, Germany*

Brain-machine interfaces (BMI) may support motor impaired patients during activities of daily living by controlling external devices such as prostheses (assistive BMI). Moreover, BMIs are applied in conjunction with robotic orthoses for rehabilitation of lost motor function via neurofeedback training (restorative BMI). Using assistive BMI in a rehabilitation context does not automatically turn them into restorative devices. This perspective article suggests key features of restorative BMI and provides the supporting evidence: In summary, BMI may be referred to as restorative tools when demonstrating subsequently (i) operant learning and progressive evolution of specific brain states/dynamics, (ii) correlated modulations of functional networks related to the therapeutic goal, (iii) subsequent improvement in a specific task, and (iv) an explicit correlation between the modulated brain dynamics and the achieved behavioral gains. Such findings would provide the rationale for translating BMI-based interventions into clinical settings for reinforcement learning and motor rehabilitation following stroke.

#### Edited by:

*Michela Chiappalone, Fondazione Istituto Italiano di Technologia, Italy*

#### Reviewed by:

*Vittorio Sanguineti, University of Genoa, Italy David Guggenmos, University of Kansas Medical Center, USA*

\*Correspondence:

*Alireza Gharabaghi alireza.gharabaghi@uni-tuebingen.de*

#### Specialty section:

*This article was submitted to Neuroprosthetics, a section of the journal Frontiers in Neuroscience*

Received: *21 December 2015* Accepted: *21 September 2016* Published: *13 October 2016*

#### Citation:

*Gharabaghi A (2016) What Turns Assistive into Restorative Brain-Machine Interfaces? Front. Neurosci. 10:456. doi: 10.3389/fnins.2016.00456* Keywords: assistive technology, neurorehabilitation, stroke, rehabilitation robotics, brain-computer interface, brain-robot interface

In stroke patients with severe and persistent motor deficits, restitution of useful function is very limited. Despite intensive rehabilitation programs, these patients are often left with a dysfunctional upper extremity and, consequently, with a long-term dependency on others for activities of daily living (Jørgensen et al., 1999; Dobkin, 2004; Feigin et al., 2008). There are many technologydriven efforts to improve recovery in this patient group on the basis of motor re-learning. Brain-machine interfaces (BMI), for example, have been applied lately to bridge the impaired connection in the sensorimotor loop. Unlike classical BMIs that assist motor impaired patients, for example by controlling external devices such as prostheses, their restorative counterparts provide brain-state dependent proprioceptive feedback by way of orthotic devices attached to the hand or arm of the patient to facilitate rehabilitation training toward functional restoration. Such supported movements facilitate the detection of motor intention even in the absence of actual movements (Gomez-Rodriguez et al., 2011; Brauchle et al., 2015). When used in conjunction with commercially available robotic rehabilitation technology (Bauer et al., 2015; Brauchle et al., 2015; Vukelic and Gharabaghi, 2015a ´ ), these devices are also known as brain-robot interfaces (BRI). Patient control over these robotic training devices is usually mediated by motor imageryrelated sensorimotor oscillations of the ipsilesional cortical electroencephalogram (EEG) or electrocorticogram (Gharabaghi et al., 2014b). However, the translation from assistive toward restorative BMI cannot be realized simply by applying the identical method in altered environments for different goals. We, therefore, propose a conceptual framework for restorative BMI distinct from their assistive predecessors, summarize the most recent supporting evidence (Kraus et al., 2016a; Naros et al., 2016), and project future developments and perspectives in this field. We exemplify this concept for the area of movement recovery after stroke. The underlying assumption is that learning to modulate sensorimotor beta-oscillations might facilitate post-stroke functional restoration. Current evidence for this assumption is circumstantial: (i) Movement-related betaoscillations are compromised in stroke patients and correlate with the impairment level (Rossiter et al., 2014). (ii) Volitional modulation of sensorimotor beta-oscillations can be learned via BMI and correlates with corticospinal excitability increases (Kraus et al., 2016a) and motor learning (Naros et al., 2016) in healthy subjects. However, there is currently only one pilot study available in literature that has addressed this concept of "learning beta-band self-regulation" for post-stroke rehabilitation (Naros and Gharabaghi, 2015). This perspective article intends to outline in detail the rationale for this approach and to initiate a discussion on necessary features and prerequisites of restorative BMI for stroke rehabilitation.

### FROM ASSISTANCE TO RESTORATION

Despite the application of novel techniques such as BMI training combined with physiotherapy (for an overview, see Naros and Gharabaghi, 2015), there is still a lack of relevant functional improvement of the hand and finger function in the patient group with persistent deficits of the upper limb. This has attracted interest in the specificity and mechanisms of BMI therapy, since the underlying neurophysiology of this intervention (Kraus et al., 2016a), together with appropriate strategies to optimize learning and motor gains (Naros et al., 2016) have not yet been fully explored.

The BMI approach in rehabilitation comes into play once standard physical practice is no longer possible in the targeted patient group. Consequently, the lost motor function following stroke limits the re-learning of movements (Doyon and Benali, 2005; Halsband and Lange, 2006; Naros et al., 2016). In such cases, motor imagery (MI) might constitute an alternative for physical practice (Halsband and Lange, 2006; Boe et al., 2014) since it activates the sensorimotor system without any overt movement (Gao et al., 2011; Szameitat et al., 2012; Vukelic´ and Gharabaghi, 2015a; Naros et al., 2016). This volitional modulation of oscillatory activity during MI can be supported by providing BMI feedback about the user's current brain state to facilitate operant learning of oscillatory patterns considered beneficial to recovery (Vukelic and Gharabaghi, 2015a; Naros ´ et al., 2016). The concept of restorative BMI training, therefore, is based on the premise that associative learning facilitates selfregulation of such MI-related brain activity by connecting the neural correlates of movement intention and the contingent feedback. Notably, for behavioral gains, this approach requires not only operant learning of brain self-regulation but also the progression of the trained brain dynamics (see also paragraph on Adaptive BMI feedback). More specifically, movementrelated desynchronization (ERD) in the contralateral, ipsilesional sensorimotor cortex is compromised in stroke patients compared to healthy controls, i.e., the more severe the patient's motor impairment, the less ERD (Rossiter et al., 2014). Accordingly, a restorative training approach would need to increase this oscillatory modulation range again.

However, most BMI studies do not demonstrate such a progressive modulation range, i.e., the improved skill of brain self-regulation in the course of the training, even when behavioral gains are reported after the intervention (for an overview, see Naros et al., 2016). This suggests that the reported BMI use had general priming effects on subsequent physiotherapy rather than genuine effects, i.e., intervention specific motor gains (for an overview, see Naros et al., 2016). The demonstration of direct brain-function and/or brain-behavior relationships would, however, be a requirement for the concept of restorative BMI, which is based on the assumption that operant conditioning of the targeted brain state and dynamics facilitates taskspecific motor gains (Naros and Gharabaghi, 2015). Unlike previous ambivalent findings during BMI motor rehabilitation, neurofeedback-induced operant conditioning of the targeted brain states was indeed successfully achieved in the cognitive domain and led to task-specific functional gains (Zoefel et al., 2011). This ambiguity between the findings in the cognitive and the motor domain might be related to the methodological limitations of earlier BMI approaches in the motor domain. This applies in particular to the cortical frequency-band trained by ERD in previous studies, i.e., alpha- instead of beta-oscillations, the feedback strategy and/or the application or lack of additional brain stimulation (Naros and Gharabaghi, 2015; Naros et al., 2016).

### ADAPTIVE BMI FEEDBACK

Classical BMI approaches maximize the classification accuracy of the device to optimally detect task-related MI (Thomas et al., 2013; Thompson et al., 2013; Spüler et al., 2014; Bauer and Gharabaghi, 2015a). This approach has been applied in most previous studies with stroke patients without resulting in a gain of the skill for BMI control as one might have expected in a continuous learning experience (for an overview, see Naros and Gharabaghi, 2015). On the basis of learning principles, a certain degree of challenge for the participant, therefore, seems to be required to reinforce continuous effort and progression of brain self-regulation (Bauer and Gharabaghi, 2015a). In this context, mathematical simulations (Bauer and Gharabaghi, 2015b) and empirical data (Naros and Gharabaghi, 2015; Naros et al., 2016) suggest that dynamic threshold adaptation of the oscillatory desynchronization level that has to be achieved to control the BMI in the course of the training, i.e., adjusting the difficulty level of the neurofeedback task on the basis of the performance, is a more appropriate method for achieving BMI reinforcement learning than a fixed threshold at maximum classification accuracy, i.e., an unchanged oscillatory desynchronization level that has to be achieved to control the BMI (Theodoridis and Koutroumbas, 2009).

Along these lines, a recent study with healthy subjects over a 3-day training period was the first to demonstrate that dynamic threshold adaptation is instrumental in facilitating learning of movement-related brain self-regulation. By contrast, subjects

who trained with a classical BMI concept, i.e., without threshold adaptation, failed to progressively modulate the targeted brain activity (Naros et al., 2016). This matched the concept that BMI paradigms which focus on the maximization of classification accuracy optimize the metabolic cost (Jackson and Fetz, 2011; Naros et al., 2016). Restorative BMIs should, therefore, be designed in such a manner as to provide incentives not only for achieving but also for enhancing the targeted brain activity, e.g., progressing the level of ERD (Carmena, 2013; Naros and Gharabaghi, 2015; Naros et al., 2016). Although, BMI tasks are potentially linked to the experience of frustration (Fels et al., 2015), a less demanding task structure of reaching the brain state only once so as to be rewarded with feedback did not result in improved brain self-regulation (Naros et al., 2016). By contrast, the more demanding task of providing or withholding feedback contingent to the targeted brain self-regulation was essential for achieving sustained ERD (Naros et al., 2016). More specifically, when comparing different BMI training conditions in a parallelgroup design [(i) adaptive classifier thresholding and contingent feedback, (ii) adaptive classifier thresholding and non-contingent feedback (iii) non-adaptive classifier thresholding and contingent feedback, (iv) non-adaptive classifier thresholding and noncontingent feedback], contingent neurofeedback and adaptive classifier thresholding were critical for learning brain selfregulation which, in turn, led to behavioral gains after the intervention. Contingent feedback to successful brain selfregulation meant that as soon as the predefined ERD level was achieved the participants were rewarded by the robotic opening of the hand. However, if the targeted brain state could not be sustained, the robotic movement ceased again but could be resumed within the same trial if the predefined brain state was attained again (Naros et al., 2016). Furthermore, adaptive classifier thresholding throughout the intervention was realized by adjustments of the task difficulty before each training session in the course of a multi-session program. These adjustments were made in accordance with the BMI performance in the preceding session based on an algorithm that has been shown to support reinforcement learning of self-regulated beta-oscillations (Naros and Gharabaghi, 2015).

In this context, future studies may evaluate the impact of different task thresholds, i.e., targeted ERD levels, on the learning incentive, thereby empirically determining the optimal difficulty level for brain self-regulation and disentangling the relative contribution of neurofeedback specificity and sensitivity (Bauer et al., 2016a). Moreover, future approaches may investigate alternative approaches to balancing the mental effort involved, for example by adjusting the task demands on the basis of self-ratings by the participants (Bauer et al., 2016b).

#### BRAIN-FUNCTION INTERACTION

While the progression of sensorimotor self-regulation is a necessary requirement for restorative BMI, such local modulation would not be sufficient by itself. Such an intervention would necessitate more global network effects as well to bring about behavioral gains. But how would BMI feedback training translate self-regulated modulation of local oscillations into changes of distant functional networks? In other words, how is such a brain-function interaction physiologically mediated?

Imaging studies based on multi-channel electroencephalography revealed that sensorimotor brain self-regulation and BMI feedback entrained an extended cortical motor network that includes frontal and parietal brain areas (Vukelic´ et al., 2014; Vukelic and Gharabaghi, 2015a ´ ) with distributed, but spatially selective frequency-specific effects on cortico-cortical connectivity that last beyond the intervention period (Vukelic and Gharabaghi, 2015b ´ ). This motor network modulation is critically linked to the proprioceptive feedback provided by the BMI (Vukelic and Gharabaghi, 2015a ´ ). Notably, those subjects who were particularly capable of performing sensorimotor brain self-regulation could be predicted by a distributed alpha-band resting state network measured before the intervention (Bauer et al., 2015). Similarly, the resting state functional connectivity of the motor cortex seems to be related to motor learning (Mottaz et al., 2015) and to the prediction of functional improvement after stroke (Nicolo et al., 2015). Moreover, functional coupling of coherent theta-band oscillations during the BMI task correlated with the skill of sensorimotor modulation, thus indicating a motor learningrelated network (Vukelic and Gharabaghi, 2015a ´ ). These findings match well with the neurophysiological concepts that link these various frequency domains to working memory and sensorimotor integration (Fell et al., 2011; Cruikshank et al., 2012), sensory processing and multi-modal integration (Palva and Palva, 2007; Weisz et al., 2014), and the retrieval of stored motor schemata and bottom-up integration of sensory and motor information (Caplan et al., 2003; Cruikshank et al., 2012; Vukelic and Gharabaghi, 2015a ´ ).

With regard to the intended behavioral improvements, the modulation of corticospinal connectivity by BMI feedback may represent the even more important functional network effect. Neurofeedback interventions have already been shown to increase the effective corticospinal connectivity, i.e., the sensorimotor excitability evaluated by transcranial magnetic stimulation (TMS) and motor evoked potentials (MEP) (Pichiorri et al., 2011; Shindo et al., 2011; Mokienko et al., 2013). However, until very recently, these measurements did not provide a specific link between the modulated brain activity and the changed connectivity to the periphery. Methodological improvements with refined TMS maps (Kraus and Gharabaghi, 2015, 2016) closed this gap by demonstrating robust changes of corticospinal connectivity for the BMI-trained muscle, but not for the control muscle. The largest MEP gains were found in those cortical areas that were most strongly modulated by the intervention (Kraus et al., 2016a). Furthermore, this target selectivity and topographic specificity were paralleled by a functional correlation between the modulated brain activity and the increased connectivity to the periphery, i.e., the largest MEP gains were observed in the subjects with the biggest modulation range (Kraus et al., 2016a).

Future studies are required to evaluate whether these functional network changes of corticocortical and corticospinal connectivity during the intervention and in the following resting state persist during behavioral tasks after the intervention, and to what extent they influence the respective performance. Furthermore, different feedback modalities, such as functional electrical stimulation and/or closed-loop TMS (Gharabaghi et al., 2014a; Raco et al., 2016; Royter and Gharabaghi, 2016), may be explored in conjunction with BMI technology to compare their differential impact on network modulations with that one of proprioceptive feedback provided by the robotic orthoses in earlier studies. It will be particularly important to explore these brain-function interactions for the whole upper extremity so as to translate them to activities of daily living. This might also entail studying the impact of brain-machine interfaces connected to multi-joint exoskeletons (Grimm and Gharabaghi, 2016; Grimm et al., 2016a,b) for three-dimensional reach-tograsp movements (Brauchle et al., 2015) on the corticospinal excitability of different muscle groups, including synergetic and antagonistic interactions.

#### BRAIN-BEHAVIOR INTERACTION

Unlike in the cognitive domain (Zoefel et al., 2011), a brainbehavior interaction in the motor domain, i.e., a direct link between the brain state/dynamics modulated by BMI feedback and subsequent improvements in an actual motor task, was not demonstrated until very recently. In addition to the factors mentioned in earlier paragraphs, the targeted brain state might be one of the major reasons for this lack; this has already been outlined in detail elsewhere and resonates here (Naros et al., 2016): Despite the eligibility of beta-ERD as a control signal in brain interfaces (Bai et al., 2008) the majority of BMI studies up to now preferred to use alpha-ERD (Naros and Gharabaghi, 2015). This was due to the fact that, in stroke patients, alpha-ERD was more effective than beta-ERD in classifying brain states related to movement (Gomez-Rodriguez et al., 2011). Although, these two frequency bands are modulated by motor execution and MI in much the same way (Van Wijk et al., 2012; Kilavik et al., 2013; Brinkman et al., 2014), it is becoming clearer that they perform different tasks. The function of alpha-ERD is to gate the inhibition of regions which are irrelevant for the task (Pineda, 2005; Mazaheri and Jensen, 2010; Sabate et al., 2011), whereas beta-ERD is responsible for mediating sensorimotor cortex disinhibition (Siegel et al., 2012; Kilavik et al., 2013) and muscular proprioceptive feedback (Salmelin et al., 1995; Mima et al., 2000; Riddle and Baker, 2006; Kristeva et al., 2007; Aumann and Prut, 2014). On the basis of these differences in function, we postulated that, of the two frequency bands, beta-oscillations constitute the better therapeutic option for BMI therapy in patients suffering from motor impairment following stroke (Brauchle et al., 2015; Naros and Gharabaghi, 2015). This approach thus allowed the first demonstration of a frequency-specific correlation between the modulation of cortical physiology with MI-based BMI training and later motor performance (Naros et al., 2016). Such a correlation was, however, not observed between alpha-activity and motor performance. Promoting the ability to voluntarily control beta-oscillations on the basis of proprioceptive feedback might, therefore, facilitate the communication between the motor cortex and muscles in the same frequency band (Brown, 2007; Darvishi et al., under review), thereby resulting in improved motor control in behavioral tasks (Naros et al., 2016).

The next step will be to draw a direct comparison between the operant conditioning of different frequency bands, for example between alpha- and beta-band ERD, to ascertain which particular oscillatory pattern is responsible for this improvement. Further interventions may also be required to gain maximal exploitation and consolidation of the patients' remaining ability for motor learning and brain self-regulation. One such additional input during robot-assisted training may be activity-dependent brain stimulation (Gharabaghi, 2015; Massie et al., 2015). During BMI training, for example, concurrent state-dependent transcranial magnetic stimulation is capable of unmasking latent corticospinal connectivity following stroke (Gharabaghi et al., 2014a). On the basis of Hebbian-like plasticity, state-dependent stimulation synchronized to maximum ERD may serve to stabilize the corticospinal circuits involved (Kraus et al., 2016b).

Future studies will explore whether the behavioral gains resulting from beta-ERD modification also lend themselves to other motor tasks. This would be instrumental in transforming such an approach into a clinical application. Some patients may, however, be unable to gain volitional control of this technique using beta-modulation in a standard EEG-based setting on account of an extended cortical lesion and/or distorted physiology. In such instances, the detection and neurofeedback training may be accomplished by epidural recordings of field potentials (Gharabaghi et al., 2014b). This alternative approach, which is nearer to the neural signal source, may not only require a shorter period of therapy to induce clinical gains than is customary using the standard EEG technique (Gharabaghi et al., 2014c), but may also act as a bi-directional interface for concurrent brain stimulation (Gharabaghi et al., 2014d).

In summary, BMIs may be referred to as restorative tools when demonstrating subsequently (i) operant learning and progressive evolution of specific brain states/dynamics, (ii) correlated modulations of functional networks related to the therapeutic goal, (iii) subsequent improvement in a specific task, and (iv) an explicit correlation between the modulated brain dynamics and the achieved behavioral gains. Such findings would provide the rationale for translating BMI-based interventions into clinical settings for reinforcement learning and motor rehabilitation following stroke.

## AUTHOR CONTRIBUTIONS

The author confirms being the sole contributor of this work and approved it for publication.

### ACKNOWLEDGMENTS

AG was supported by grants from the German Research Council [DFG EC 307], and from the Federal Ministry of Education and Research [BFNT 01GQ0761, BMBF 16SV3783, BMBF 0316064B, BMBF 16SV5824]. There is no conflict of interests.

### REFERENCES


Granger causality. Neuroimage 54, 1280–1288. doi: 10.1016/j.neuroimage.2010. 08.071


**Conflict of Interest Statement:** The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2016 Gharabaghi. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.