# BIOMECHATRONICS: HARMONIZING MECHATRONIC SYSTEMS WITH HUMAN BEINGS

EDITED BY : Dingguo Zhang, Venketesh Dubey, Wenwei Yu and Kin Huat Low PUBLISHED IN : Frontiers in Neuroscience

#### Frontiers Copyright Statement

© Copyright 2007-2019 Frontiers Media SA. All rights reserved. All content included on this site, such as text, graphics, logos, button icons, images, video/audio clips, downloads, data compilations and software, is the property of or is licensed to Frontiers Media SA ("Frontiers") or its licensees and/or subcontractors. The copyright in the text of individual articles is the property of their respective authors, subject to a license granted to Frontiers.

The compilation of articles constituting this e-book, wherever published, as well as the compilation of all other content on this site, is the exclusive property of Frontiers. For the conditions for downloading and copying of e-books from Frontiers' website, please see the Terms for Website Use. If purchasing Frontiers e-books from other websites or sources, the conditions of the website concerned apply.

Images and graphics not forming part of user-contributed materials may not be downloaded or copied without permission.

Individual articles may be downloaded and reproduced in accordance with the principles of the CC-BY licence subject to any copyright or other notices. They may not be re-sold as an e-book.

As author or other contributor you grant a CC-BY licence to others to reproduce your articles, including any graphics and third-party materials supplied by you, in accordance with the Conditions for Website Use and subject to any copyright notices which you include in connection with your articles and materials.

All copyright, and all rights therein, are protected by national and international copyright laws.

The above represents a summary only. For the full conditions see the Conditions for Authors and the Conditions for Website Use. ISSN 1664-8714 ISBN 978-2-88945-737-3 DOI 10.3389/978-2-88945-737-3

#### About Frontiers

Frontiers is more than just an open-access publisher of scholarly articles: it is a pioneering approach to the world of academia, radically improving the way scholarly research is managed. The grand vision of Frontiers is a world where all people have an equal opportunity to seek, share and generate knowledge. Frontiers provides immediate and permanent online open access to all its publications, but this alone is not enough to realize our grand goals.

#### Frontiers Journal Series

The Frontiers Journal Series is a multi-tier and interdisciplinary set of open-access, online journals, promising a paradigm shift from the current review, selection and dissemination processes in academic publishing. All Frontiers journals are driven by researchers for researchers; therefore, they constitute a service to the scholarly community. At the same time, the Frontiers Journal Series operates on a revolutionary invention, the tiered publishing system, initially addressing specific communities of scholars, and gradually climbing up to broader public understanding, thus serving the interests of the lay society, too.

#### Dedication to Quality

Each Frontiers article is a landmark of the highest quality, thanks to genuinely collaborative interactions between authors and review editors, who include some of the world's best academicians. Research must be certified by peers before entering a stream of knowledge that may eventually reach the public - and shape society; therefore, Frontiers only applies the most rigorous and unbiased reviews.

Frontiers revolutionizes research publishing by freely delivering the most outstanding research, evaluated with no bias from both the academic and social point of view. By applying the most advanced information technologies, Frontiers is catapulting scholarly publishing into a new generation.

#### What are Frontiers Research Topics?

Frontiers Research Topics are very popular trademarks of the Frontiers Journals Series: they are collections of at least ten articles, all centered on a particular subject. With their unique mix of varied contributions from Original Research to Review Articles, Frontiers Research Topics unify the most influential researchers, the latest key findings and historical advances in a hot research area! Find out more on how to host your own Frontiers Research Topic or contribute to one as an author by contacting the Frontiers Editorial Office: researchtopics@frontiersin.org

# BIOMECHATRONICS: HARMONIZING MECHATRONIC SYSTEMS WITH HUMAN BEINGS

Topic Editors:

Dingguo Zhang, Shanghai Jiao Tong University, China Venketesh Dubey, Bournemouth University, United Kingdom Wenwei Yu, Chiba University, Japan Kin Huat Low, Nanyang Technological University, Singapore

Image: Dean Drobot/Shutterstock.com

This eBook provides a comprehensive treatise on modern biomechatronic systems centred around human applications. A particular emphsis is given to exoskeleton designs for assistance and training with advanced interfaces in human-machine interaction. Some of these designs are validated with experimental results which the reader will find very informative as building-blocks for designing such systems. This eBook will be ideally suited to those researching in biomechatronic area with bio-feedback applications or those who are involved in high-end research on manmachine interfaces. This may also serve as a textbook for biomechatronic design at post-graduate level.

Citation: Zhang, D., Dubey, V., Yu, W., Low, K. H., eds. (2019). Biomechatronics: Harmonizing Mechatronic Systems with Human Beings. Lausanne: Frontiers Media. doi: 10.3389/978-2-88945-737-3

# Table of Contents

*05 Editorial: Biomechatronics: Harmonizing Mechatronic Systems With Human Beings*

Dingguo Zhang, Venketesh N. Dubey, Wenwei Yu and Kin Huat Low

#### SECTION A

#### NOVEL EXOSKELETONS FOR ASSISTANCE AND TRAINING

*08 Design of a Soft Robotic Elbow Sleeve With Passive and Intent-Controlled Actuation*

Tze Hui Koh, Nicholas Cheng, Hong Kai Yap and Chen-Hua Yeow

*20 Design and Preliminary Feasibility Study of a Soft Robotic Glove for Hand Function Assistance in Stroke Survivors*

Hong Kai Yap, Jeong Hoon Lim, Fatima Nasrallah and Chen-Hua Yeow

*34 Design and Validation of Exoskeleton Actuated by Soft Modules Toward Neurorehabilitation—Vision-Based Control for Precise Reaching Motion of Upper Limb*

Victoria W. Oguntosin, Yoshiki Mori, Hyejong Kim, Slawomir J. Nasuto, Sadao Kawamura and Yoshikatsu Hayashi


Juan Fang, Qing Xie, Guo-Yuan Yang and Le Xie

## SECTION B

#### ADVANCED HUMAN-MACHINE INTERFACES IN BIOMECHATRONICS


Qin Zhang, Runfeng Liu, Wenbin Chen and Caihua Xiong


Yubo Wang and Kalyana C. Veluvolu

*137 Decoding Lower Limb Muscle Activity and Kinematics From Cortical Neural Spike Trains During Monkey Performing Stand and Squat Movements*

Xuan Ma, Chaolin Ma, Jian Huang, Peng Zhang, Jiang Xu and Jiping He


Carlos Rodriguez-Guerrero, Kristel Knaepen, Juan C. Fraile-Marinero, Javier Perez-Turiel, Valentin Gonzalez-de-Garibay and Dirk Lefeber

*181 Physiological Signal-Based Method for Measurement of Pain Intensity* Yaqi Chu, Xingang Zhao, Jianda Han and Yang Su

# SECTION C

#### EXPERIMENTAL OUTCOMES AND VALIDATION

*194 Phase-II Clinical Validation of a Powered Exoskeleton for the Treatment of Elbow Spasticity*

Simona Crea, Marco Cempini, Stefano Mazzoleni, Maria Chiara Carrozza, Federico Posteraro and Nicola Vitiello


Farahiyah Jasni, Nur Azah Hamzaid, Nor Elleeiana Mohd Syah, Tze Y. Chung and Noor Azuan Abu Osman

# Editorial: Biomechatronics: Harmonizing Mechatronic Systems With Human Beings

#### Dingguo Zhang<sup>1</sup> \*, Venketesh N. Dubey <sup>2</sup> \*, Wenwei Yu<sup>3</sup> and Kin Huat Low<sup>4</sup>

*<sup>1</sup> School of Mechanical Engineering, Robotics Institute, Shanghai Jiao Tong University, Shanghai, China, <sup>2</sup> Faculty of Science and Technology, Bournemouth University, Poole, United Kingdom, <sup>3</sup> Center for Frontier Medical Engineering, Chiba University, Chiba, Japan, <sup>4</sup> School of Mechanical and Aerospace Engineering, Nanyang Technological University, Singapore, Singapore*

Keywords: biomechatronics, exoskeleton, prosthesis, rehabilitation robot, human-machine interface, brainmachine interface, electromyography, electroencephalography

**Editorial on the Research Topic**

#### **Biomechatronics: Harmonizing Mechatronic Systems With Human Beings**

There has been a growing body of research in the recent years on human-robot interactions, human-machine interfaces and intelligent devices that are centered around human application, however, these works by and large lacked in focus on how to harmonize the interactions between mechatronic systems and users in the loop. This is one of the key areas for evaluating the success of any mechatronic system implementation on human. The collection of papers in this volume is touching upon the frontiers of this research area as to how the efficacy of such biomechatronic systems could be evaluated and improved. There are a total of 19 papers looking into various aspects of human-machine interfaces (HMIs) using electromyography (EMG) and electroencephalography (EEG), tactile feedback, external devices such as exoskeletons and prosthetic devices for assistance and rehabilitation, novel techniques like machine learning and intelligent computation, and experimental evaluation or validation. The following paragraphs aim to give a glimpse of the contents presented in this eBook. Specifically, these are categorized under three distinct headings: (A) Novel exoskeletons for assistance and training, (B) Advanced human-machine interfaces in biomechatronics, and (C) Experimental outcomes and validation.

#### Edited and reviewed by:

*Mikhail Lebedev, Duke University, United States*

#### \*Correspondence:

*Dingguo Zhang dgzhang@sjtu.edu.cn Venketesh N. Dubey vdubey@bournemouth.ac.uk*

#### Specialty section:

*This article was submitted to Neural Technology, a section of the journal Frontiers in Neuroscience*

Received: *31 August 2018* Accepted: *03 October 2018* Published: *25 October 2018*

#### Citation:

*Zhang D, Dubey VN, Yu W and Low KH (2018) Editorial: Biomechatronics: Harmonizing Mechatronic Systems With Human Beings. Front. Neurosci. 12:768. doi: 10.3389/fnins.2018.00768*

# A. NOVEL EXOSKELETONS FOR ASSISTANCE AND TRAINING

There has been a surge of interests in implementing "soft robots" into every aspect of robotic assistance and training since it can provide compliance with inherent safety features to users. The paper by Koh et al. discussed a soft robotic elbow sleeve with full design features and functionality using elastomeric and fabric-based pneumatic actuators; what was more interesting was the intent based actuation control. The user intent was detected by surface electromyography (sEMG) on healthy subjects and the results showed that all participants were capable of achieving EMG control of the elbow sleeve. An extension of the above idea by the same group was explored by Yap et al. using soft robotic gloves with fabric-reinforced soft actuators, which targeted hand function assistance. They integrated elastic fabric with soft actuators to enhance the driven force for finger extension. A pilot test on stroke survivors showed improved grasping performance with the assistance of the soft glove. In a slightly different approach Oguntosin et al. have demonstrated functional properties of the exoskeleton actuated by soft modules. The exoskeleton has 3D printed parts with passive joints and utilizing the gravity compensation scheme to make it ultra-light. The exoskeleton was proposed for precise reaching control in neurorehabilitation for individuals with specific motor impairments. These papers highlighted the advantage of using soft modules for rehabilitation.

**5**

In an unusual way Huang et al. have studied a cable based robot for assessment of motor control during 3D movements tracking with position varying gravity compensation. They have shown that there was a significant difference in control strategies with and without gravity compensation which might assist subjects in performing optimal movements during rehabilitation. Fang et al. have realized the fact that people swing their arms synchronously with leg movement in the normal gait. So they developed a rotational orthosis for walking with arm swing for gait training to provide coordinated interlimb performance. This could potentially improve gait rehabilitation outcomes of patients undergoing training; the device was tested for functional assessment with normal subjects and was reported to provide a stronger feeling of walking with arm swing than without.

# B. ADVANCED HUMAN-MACHINE INTERFACES IN BIOMECHATRONICS

Advanced human-machine interfaces (HMIs) are the key to harmonizing mechatronic systems with human beings in the loop; many developed systems simply fail because they lack of good interfaces for interpreting user motion intention/states and providing tactile feedback in the systems. Liu et al. have developed a model for continuous and simultaneous decoding of multi-joint dynamic arm movements based on multichannel surface EMG signals. This was used to myoelectrically control exoskeletons for upper-limb rehabilitation which showed the potential for effortless control of impaired arms. On a similar approach Zhang et al. have used sEMG signals for simultaneous and continuous estimation of shoulder and elbow kinematics using principal/independent component analysis and artificial neural network for learning electromechanical association. This was done to achieve natural and intuitive human-machine interaction particularly for applications in exoskeletons, prostheses and other arm rehabilitation techniques. In an extension to the above approach Wu et al. developed grip force and three-dimensional push-pull force estimation based on sEMG and generalized regression neural network. This work was proposed to meet the requirements of force control of the intelligent prosthetic hand which employed EMG and grip force sensors to measure the output force of the human hand. Since all the above reported work employed EMG sensors, there must be a better way of acquiring stable signals. Jiang et al. have proposed a Polypyrrole-coated nonwoven fabric sheet for designing EMG sensors. The design can be customized for shape and size and can be sewn onto elastic band for close contact with skin for practical control of prosthetic hand. The sensors provided comparable results with conventional Ag/AgCl electrodes typically used in EMG sensors.

In a slight depart from the EMG based HMIs using muscle signals, EEG based HMIs employ brain signals, which are well-known as brain-machine interfaces (BMIs). Some previous non-invasive BMIs adopted Fourier based methods for time-frequency decomposition for feature extraction of EEG. However, this technique was challenging for multichannel EEG signals. To improve performance of multi-channel EEG classification, Wang and Veluvolu developed evolutionary algorithms for optimizing the features to reduce the overall dimensionality and preserve the class-related information. Their results show that the combination of Fourier based method with covariance matrix adaptation evolution strategy has the best overall performance. Compared with non-invasive BMIs based on scalp EEG, invasive BMIs use high-quality brain signals, which are crucial for advanced control of dexterous robotic arms or prosthetic hands. Ma et al. have conducted a trial with monkeys to decode lower limb muscle activity and kinematics from cortical neural spike recordings during stand/squat movements. They provided detailed analyses of recorded data and new insights into invasive BMIs that could be used for controlling complex biomechatronic systems.

To explore the mechanism of tactile sensation under transcutaneous electrical nerve stimulation, a 3D computational model was developed for estimating tactile nerve fiber excitability (Zhu et al.). The simulation showed comparable results with psychophysical experimental data with healthy subjects. This study has potential for providing tactile feedback for prosthesis and realizing bi-directional neural-machine interfaces in future. In order to improve challenge/skill ratio in a multi-modal interface for human-robot interaction, Rodriguez-Guerrero et al. have investigated simultaneous adaptation of game difficulty and haptic assistance through psychophysiological signals (heart rate, skin conductance level, and skin conductance response frequency) and performance feedback. They came up with a new matric (FlowIndex) to numerically quantify and visualize the challenge/skill relation which improved the balance between augmented performance and user satisfaction. Different from the traditional diagnosis based on the patient's self-description, Chu et al. have developed a method of pattern recognition to classify the pain intensity based on multiple physiological signals (blood volume pulse, electrocardiogram, and skin conductance level). This has implications for the patients with major cognitive and communicative impairments who are not able to describe their level of pain. The experimental results on healthy subjects showed that the method can provide objective and quantitative evaluation of pain intensity.

# C. EXPERIMENTAL OUTCOMES AND VALIDATION

The level of harmonization between mechatronic systems and human beings could not be assessed without validating these devices under experiments with human subjects. There are several studies that focused on experimental trials to prove the efficacy of their devices. For example, Crea et al. have reported Phase II of clinical validation of a powered exoskeleton for the treatment of elbow spasticity. The rehabilitation outcomes from 17 post-stroke patients with robotic elbow exoskeleton showed that intensive early treatment prevented the occurrence of spasticity at a later stage. Buzzi et al. proposed a non-disruptive method to study the arm endpoint stiffness in a teleoperated robotic system. Based on a musculoskeletal model, they could use arm kinematics and muscular activation (sEMG) to estimate the Zhang et al. Editorial: Biomechatronics

endpoint stiffness. Here, they just focused on experimental study to investigate how different master devices and tasks influence the regulation of arm endpoint stiffness and its relation with task performance, hand speed and acceleration. This experimental study may further help to understand which characteristics and parameters play a fundamental role in compliant humanrobot interactions. In order to generate sufficient and accurate motion data for positive rehabilitation using exoskeletons, Liu et al. conducted experimental study based on their custommade Neo-Arm exoskeleton. At each joint of Neo-Arm, an angular displacement sensor was equipped through the axis of the rotation, by which the angular information could be captured. Their methods have been compared with the conventional method i.e., a camera-based (Vicon) system. The experiments were conducted on 8 healthy subjects performing a series of movements, including five actions and five hovering postures, and the results showed that the satisfactory data could be achieved with suitable precision for upper limb motion tasks without the need for platform based systems. Since motor learning is a critical component of the rehabilitation process, Jarrett and McDaid conducted a simulated pilot study with healthy subjects to evaluate motor learning in case of physical impairment. They developed a technique where simulated impairment is applied to constrain the motion, such that motor learning could be studied independently of physical impairment. Their tool could be used to get insight into underlying causes of motor learning deficits. This could be used to provide subject– specific therapy. Jasni et al. have analyzed the interaction among the voluntary and mechanical joints and segments in users wearing lower-limb prosthesis/orthosis using cyclograms. The cyclogram model was developed using data from 20 healthy able-bodied subjects and 25 prosthesis and orthosis users (10 transtibial amputees, 5 transfemoral amputees, and 10 orthosis users).This cyclogram model enables quantitative judgment of the effect of changing a particular parameter in the prosthetic leg gait. This experimental study is particularly useful for understanding coordinated movement of several joints and limb segments wearing external devices.

So looking at the collection of papers in this research topic we are happy to endorse that this eBook provides a comprehensive appraisal of the latest developments in the field of biomechatronics including exoskeletons, prosthesis, and advanced human-machine interfaces and followed by experimental validation of some of the developed techniques. We hope that these papers will provide readers new insights into the aspects of harmonizing mechatronic systems with human beings.

## AUTHOR CONTRIBUTIONS

DZ was invited to prepare this research topic. He invited VD, WY, and KL to be co-editors in it. They all contributed to soliciting papers, inviting guest-authors, revising their manuscripts, and handling revisions with invited reviewers. Specifically, DZ edited 12 papers, VD edited 2 papers, WY edited 4 papers, and KL edited 1 paper. VD drafted the editorial.

# FUNDING

This work was supported by the National Natural Science Foundation of China (No. 51475292, No.61761166006), and was also supported by the EU International Research Staff Exchange Scheme.

**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Zhang, Dubey, Yu and Low. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Design of a Soft Robotic Elbow Sleeve with Passive and Intent-Controlled Actuation

Tze Hui Koh<sup>1</sup> , Nicholas Cheng<sup>1</sup> , Hong Kai Yap1, 2 and Chen-Hua Yeow<sup>1</sup> \*

*<sup>1</sup> Evolution Innovation Laboratory, Department of Biomedical Engineering, National University of Singapore, Singapore, Singapore, <sup>2</sup> NUS Graduate School for Integrative Sciences and Engineering, National University of Singapore, Singapore, Singapore*

The provision of continuous passive, and intent-based assisted movements for neuromuscular training can be incorporated into a robotic elbow sleeve. The objective of this study is to propose the design and test the functionality of a soft robotic elbow sleeve in assisting flexion and extension of the elbow, both passively and using intent-based motion reinforcement. First, the elbow sleeve was developed, using elastomeric and fabric-based pneumatic actuators, which are soft and lightweight, in order to address issues of non-portability and poor alignment with joints that conventional robotic rehabilitation devices are faced with. Second, the control system was developed to allow for: (i) continuous passive actuation, in which the actuators will be activated in cycles, alternating between flexion and extension; and (ii) an intent-based actuation, in which user intent is detected by surface electromyography (sEMG) sensors attached to the biceps and triceps, and passed through a logic sequence to allow for flexion or extension of the elbow. Using this setup, the elbow sleeve was tested on six healthy subjects to assess the functionality of the device, in terms of the range of motion afforded by the device while in the continuous passive actuation. The results showed that the elbow sleeve is capable of achieving approximately 50% of the full range of motion of the elbow joint among all subjects. Next, further experiments were conducted to test the efficacy of the intent-based actuation on these healthy subjects. The results showed that all subjects were capable of achieving electromyography (EMG) control of the elbow sleeve. These preliminary results show that the elbow sleeve is capable of carrying out continuous passive and intent-based assisted movements. Further investigation of the clinical implementation of the elbow sleeve for the neuromuscular training of neurologically-impaired persons, such as stroke survivors, is needed.

Keywords: robotic, rehabilitation, elbow, stroke, assistive, electromyography-driven, wearable, soft-robotic

# 1. INTRODUCTION

According to the World Health Organization, every year approximately 15 million people around the world suffer from stroke (Rodrigo et al., 2013), and 70–80% experience upper-extremity impairment (Hu et al., 2007). The upper extremity, which includes the shoulder, elbow, wrist and hand, is important for the spatial movement and manipulation of objects, as well as for gesturing. It is therefore essential that stroke survivors regain a certain level of functionality so that they are able to independently carry out the activities of daily living.

#### Edited by:

*Kin Huat Low, Nanyang Technological University, Singapore*

#### Reviewed by:

*Danilo Emilio De Rossi, University of Pisa, Italy Kyujin Cho, Seoul National University, South Korea*

> \*Correspondence: *Chen-Hua Yeow rayeow@nus.edu.sg*

#### Specialty section:

*This article was submitted to Neural Technology, a section of the journal Frontiers in Neuroscience*

Received: *19 December 2016* Accepted: *10 October 2017* Published: *25 October 2017*

#### Citation:

*Koh TH, Cheng N, Yap HK and Yeow C-H (2017) Design of a Soft Robotic Elbow Sleeve with Passive and Intent-Controlled Actuation. Front. Neurosci. 11:597. doi: 10.3389/fnins.2017.00597*

**8**

Traditionally, stroke rehabilitation is done in the form of therapeutic intervention with the help of physiotherapists and occupational therapists, which, while effective, is timeconsuming and labor intensive (Hu et al., 2007). To reduce the burden on caregivers and the need for oversight by trained therapists, robotic devices have been developed, which aim at supporting the patients in carrying out rehabilitation at their convenience and with minimal supervision. These robotic devices are typically devised to carry out: (1) continuous passive movements, and (2) active-assisted movements. Continuous passive movements are useful for reducing muscle tone to improve the mobility of the joint (Hu et al., 2009), while activeassisted movements help in recovering motor control through the positive shaping of the cortical reorganization following brain injuries such as stroke (Hesse et al., 2003). Current robotic rehabilitation devices include end-effector robots, such as the MIT Manus and MIME, and exoskeletons, such as the Myomo, ARMin, OrthoJacket and KIST Wearable Robotic Arm (Zhang et al., 2015). While these devices have demonstrated good efficacy in rehabilitation, they are costly, non-portable and rigid, which limits them to the clinical setting, and their rigidity also makes it difficult for them to be aligned with the patients' joints during rehabilitation (Oguntosin et al., 2015).

Soft robotics is an emerging area of robotics research resulting from the melding of materials chemistry and robotics, and aims to tackle the challenges faced by traditional robotics (Polygerinos et al., 2015b). As its name suggests, soft robotics consists of robots that are manufactured with flexible or elastomeric structural elements. In particular, soft pneumatic actuators are an important component used in soft wearable robots because they are lightweight, flexible and able to distribute pressure evenly along the joint (Oguntosin et al., 2015; Polygerinos et al., 2015a,b). Additionally, their response to actuation may be customized to a large degree to accomplish motions typically difficult with hard robotic devices, possibly overcoming alignment problems through the direct attachment of actuators to the limbs (In et al., 2015). Some current developments in soft robotics for rehabilitative purposes include a soft robotic glove for robotassisted hand therapy (Yap et al., 2015, 2016b; Yeo et al., 2016), as well as a soft robotic sock for robot-assisted ankle exercise (Low et al., 2016).

The provision of timely assistance upon detecting user-intent (Lenzi et al., 2012) is a common issue in the development of robotic devices for active-assisted rehabilitation. Current stateof-the-art devices such as the Myomo and OrthoJacket have builtin electromyography (EMG) capabilities that help to estimate the joint torque needed to perform movement (Benitez et al., 2013). Studies have shown that intention-based rehabilitation enhances the therapeutic effect, accelerating the speed with which functionality is recovered (Zhang and Zhou, 2012).

The above-mentioned developments open up possibilities in developing myoelectric-based lightweight wearable robotics, which have the potential to both restore normal function in patients affected by stroke, as well as to enhance the outcome of their neuro-muscular rehabilitation. In this paper, we present the design of a soft robotic elbow sleeve for enabling flexion and extension of the elbow both passively and intent-based motion reinforcement, which may be useful as a neuro-muscular training device. In the following sections, we discuss the methods for fabricating the actuators for the elbow sleeve, the design of the control system for the elbow sleeve, methods for mechanical characterization of the actuators, as well as a description of the range of motion study that was conducted to evaluate the functionality of the device. Finally, we present the results of the mechanical characterization and the range of motion study, and discuss their implications.

# 2. MATERIALS AND METHODS

# 2.1. Design Overview

The soft robotic elbow sleeve spans across the elbow joint, from below the biceps of the arm to the middle of the forearm (**Figure 1**), and comprises the following components: (1) two pneumatic actuators to individually effect elbow joint flexion and extension; (2) a programmable pump-valve control system to switch between modes of operation; (3) surface EMG sensors which are attached to the biceps and triceps; (4) an external pump for providing air supply to the actuators; and (5) hook and loop straps which secure the actuators to the arm.

The working principle of the soft robotic elbow sleeve is based on the controlled air pressurization of two separate pneumatic actuators working antagonistically with respect to one another. Mechanically, pressurization of the flexion actuator causes the elbow to flex, while pressurization of the extension actuator causes the elbow to extend. The two resulting configurations (flexed and extended) of the arm are shown in **Figure 2**. Electronic control for the mechanical action of the actuators allows for two distinct mode of operations of the soft robotic sleeve—passive rehabilitation and intent-based rehabilitation. This is accomplished through the valves, pressure sensor, microcontroller, which are packed within a custom-made control box, and two surface EMG sensors. The details of the actuators and the control system are given in the following sections.

#### 2.2. Actuator Design and Fabrication 2.2.1. Flexion Actuator

The flexion actuator in the soft robotic sleeve is an elastomeric construct that bends upon an input air pressure. To obtain a desired angular configuration upon an input air pressure, a fiber-fabric-reinforced approach adapted from Polygerinos et al. (2015b) was adopted in the development of the flexion actuator. The molds for the flexion actuator body (which described a length of 24 cm and radius of 1 cm) were first designed using a Computer-Aided Design software (Dassault Systèmes SOLIDWORKS Corp., USA) and 3D-printed with Acrylonitrile Butadiene Styrene (ABS). These molds were then used to fabricate the actuator body using a soft elastomeric material, Dragon Skin 20 (**Figure 3A**). The actuator body was then sealed with a flat piece of elastomeric layer (Dragon Skin 20), and a woven nylon strip (which acts as a strain-limiting layer) was glued to this flat surface to enable bending of the actuator (**Figure 3B**). Following that, monofilament thread (Nylon) was hand-wound around the actuator, with the thread pitch (5 mm) determined by the grooves imprinted onto the actuator (**Figure 3C**). This

serves to radially reinforce the actuator. Two of these flexion actuators were made and assembled for the elbow sleeve. They were wrapped with a Nylon textile where bending is not required, so as to allow for an angular configuration in bending (**Figure 4**).

Dragon Skin 20 was chosen as the elastomer for fabricating the actuator due to its ability to withstand higher pressure and provide higher force output at Shore A hardness of 20 as compared to other tested materials such as Dragon Skin 10. The length of the actuator (24 cm) was determined based on the distance of the center of gravity of the average human forearm from the elbow joint (approximately 41.7% from the elbow joint, which is 18.7 cm, inclusive of hand) (Drillis et al., 1964) as well as the force that each actuator is able to produce, in order to ensure that sufficient torque can be generated to assist flexion. The Nylon monofilament thread that was wound around the actuator was introduced to limit radial expansion and thus improve the actuator's durability.

#### 2.2.2. Extension Actuator

As the extension actuator is placed interior to the flexion actuator along the elbow sleeve, we cannot simply replicate the method used for the flexion actuator (elastomeric bending actuator) in developing the extension actuator as it would hinder the operation of the flexion actuator due to its sheer volume. An alternate design, following an inflatable beam model, was instead utilized, involving the use of the torque generated from the inflation of the actuator to extend the elbow.

Nylon ripstop sheets coated with thermoplastic polyurethane were heat sealed to form an air tight chamber with a single inlet, resulting in a fabric actuator with a length of 20 cm. While deflated, the actuator does not take up much space, and provides little hindrance to the action of the flexion actuator (**Figure 5A**). Through inflation, the actuator is then able to act in a manner similar to that of a rigid beam and resist deflection (Sanan et al., 2009; Yap et al., 2016c) (**Figure 5B**).

## 2.3. Actuator Characterization

To evaluate the force-pressure relationship of the flexion and extension actuators, two experimental platforms were developed, as illustrated in **Figure 6**.

For the flexion actuator (**Figure 6A**), following a similar protocol introduced by Polygerinos et al. (Polygerinos et al., 2015b; Yap et al., 2016a), the actuator was constrained in a rigid fixture to minimize nonlinear effects, with the distal tip placed in contact with a force sensor. The pressure in the actuator was radial reinforcement.

FIGURE 4 | Illustrating the effect of wrapping Nylon around the flexion actuator to acquire a desired angular configuration. (A) An unconstrained elastomeric actuator in flexed formation, with the yellow curve indicating a rounded curvature upon air pressure input. (B) A fabric constrained elastomeric actuator upon pressurization, with the yellow lines indicating the angular curvature achieved.

gradually increased as the force sensor recorded the bending force exerted by the tip. This enables the measurement of the maximum force generated by the tip of the actuator at each pressure.

For the extension actuator (**Figure 6B**), experimental deflection tests were conducted by fixing the actuator at one end with varying weights hung from the other end of the actuator. For each weight, the pressure was decreased to the point at which the actuator began to deflect. This setup is suitable for determining the relationship between pressure and load capacity. A similar mode of testing was utilized by Sanan et.al to find the relationship between load and deflection (Sanan et al., 2009).

# 2.4. Control System

The programmable pump-valve control system of the robotic elbow sleeve allows for switching between the two rehabilitation modes—passive and active modes—and uses a commercial pneumatic pump and valve for the supplying of air pressure for actuation, controlled by an Arduino Uno micro-controller. Two pneumatic pressure sensors are used to individually monitor the pressures within the flexion and extension actuators, and two surface EMG sensors are used for intent-detection by measuring EMG signals from the biceps and triceps. Feedback control of the system for both passive and EMG rehabilitation modes uses a closed-loop proportional-integral-derivative (PID) regulator, following the basic equation:

$$u(t) = K\_p e(t) + K\_i \int\_0^t e(t)dt + K\_d \frac{de(t)}{dt} \tag{1}$$

where the u(t) is the control variable, in this case, the pressure value; e(t) is the error value; Kp, K<sup>i</sup> , and K<sup>d</sup> are the proportional, integral, and derivative gains, respectively; and t is time. The controller uses pressure as an input and in turn regulates the duty cycle of the pulse-width-modulation (PWM) as a means to control the pressure according to a pre-set point. This also acts as a safety feature to ensure that the pressure within the actuators do not reach failure point. To streamline the process of switching between modes, calibrating the EMG sensors, and the processing of generated data, a Graphical User Interface (GUI) was specially developed in C++ using the Processing Integrated Development Environment.

In the passive rehabilitation mode, the device is actuated in cycles, alternating between flexion and extension. This continuous repetitive movement was enabled by utilizing the internal timer of the micro-controller to track time. When the defined time interval was reached, the system would switch from flexion to extension and vice versa, and the timer would be reset.

Conversely, in active intent-based rehabilitation, the detection of user intent is achieved by the surface EMG sensors attached to the biceps and triceps, in which the EMG signals are passed

beam and resists bending deformation.

through a logic sequence to allow for flexion or extension of the elbow. The EMG control scheme uses rectified, amplified and then integrated surface EMG signals captured from the biceps and triceps. Notably, the nature of the surface EMG signals captured is highly dependent on their spatial location from the muscle fibers, which is difficult to regulate with high precision. The strength of the EMG signals also typically changes with different subjects as well as different muscles (Halaki and Ginn, 2012). To rectify this, for comparison purposes, both signals from the biceps and triceps are normalized by first requesting maximum voluntary contraction (MVC) effort from the user in an initiation sequence. The MVC value obtained is then used as a reference point against which all muscle signals are compared to during the task. This allows assessment of the level of activity and improves the reproducibility of the data. The signals are subsequently passed through a series of conditions as shown in **Figure 7** that cause the activation and deactivation of the actuators, based on a time-over-threshold algorithm. For example, the condition for flexing is met when the processed signal from the bicep crosses a percentage threshold for an interval of 20 ms, while ensuring that the signal from the biceps is stronger than the signal from the triceps. A similar set of conditions exists for extension to occur. This method of control minimizes the occurrence of involuntary muscle contractions that may cause misinterpreted commands.

## 2.5. Range of Motion Study

Six healthy human subjects (20–25 years old), three males and three females, were enrolled in this study. Written and informed consent were obtained from all the subjects, with approval from the university's Institutional Review Board (NUS-IRB) prior to the experiment. All experimental trials were performed in a gait analysis laboratory.

Each trial comprises of three segments: (1) voluntary active elbow flexion and extension without the elbow sleeve; (2) use of the elbow sleeve in passive rehabilitation mode; and (3) use of the elbow sleeve in active rehabilitation mode.

In the first experimental segment, subjects were seated and requested to perform three vertical elbow flexions and extensions in succession, for a total of three trials, followed by three horizontal elbow flexions and extensions in succession for another three trials (**Figure 8**). All trials were started from an extended elbow configuration of the subjects (**Figure 8C**). This is representative of activities of daily living such as lifting an object (vertical motion) as well as wiping a table (horizontal motion).

In the second experimental segment, the subjects were then tasked to perform the trials in the first segment, with assistance provided by the soft robotic elbow sleeve in the absence of muscular signal input. After donning the soft robotic elbow sleeve, the passive mode of the elbow sleeve system was engaged, causing the subjects' arms to move in the same sequence of activities as in the first segment.

In the third experimental segment, the control mode of the elbow sleeve was switched to the active rehabilitation mode. Two surface EMG sensors (Myoware, Advancer Technologies, LLC) were mounted on the bicep and triceps muscle groups on the arm without the elbow sleeve. The subjects were then tasked with completing only horizontal elbow flexion using activation signals from the two muscle groups on the free arm to control the state of the elbow sleeve.

The Vicon motion capture system was used to track the arm positions of the subjects. Retroreflective markers were attached to each subjects' arm at positions shown in **Figure 9**, and their 3D coordinates were sampled at a frequency of 100 Hz. At each sampling point, the resultant elbow flexion angle, formed between the shoulder-to-elbow segment and the elbow-to-wrist segment (as marked out by the retroreflective markers), was computed and recorded for analysis. As such, a fully extended elbow (no flexion) corresponded to a flexion angle of 0◦ . The range-of-motion (ROM) thus refers to the maximum angle of elbow flexion attainable by each subject.

For the recording of EMG signals from the muscles, the Delsys Trigno Wireless EMG system was used, with a sampling frequency of 1,000 Hz. To minimize the effects of skin resistance,

the area was swabbed with alcohol prior to the placement of the EMG sensors on the biceps and triceps. The inherent variability of the EMG signals necessitates post-processing of the recorded signals. In lieu of this the EMG signals were full-wave rectified, filtered with a second-order band pass Butterworth filter and normalized against the maximum muscle signal output during the first experimental segment.

# 3. RESULTS

# 3.1. Actuator Characterization

The experimental results of the bending force experiments conducted on the flexion actuators are presented in **Figure 10A**, showing a reproducible non-linear relationship between input pressure and tip force. The maximum force measured at a pressure of 350 kPa was 67.4N, which is well above the sufficient 20N required for the actuation of the elbow (data acquired from body segment data). A force higher than 20N is necessary to compensate for the forces which are required to bend the actuator itself and ensure that sufficient bending force is supplied to the arm segments.

For the extension actuator, the pressure-force relationship showed a linear trend (**Figure 10B**). Linear regression was performed on the data to fit a calibration line for the elbow sleeve (R <sup>2</sup> = 0.9951), which allows for an estimation of the forces produced at varying pressure magnitudes. At 80kPa, the average deflective force of 16.5N the actuator produced was capable of enabling full elbow extension.

# 3.2. Control Characterization

**Figure 11** shows the control of pressure within the actuators for the passive rehabilitation mode of the elbow sleeve. The PID controller system was able to maintain pressure at 350 kPa for the flexion actuator and 80 kPa for the extension actuator with small variations of approximately 10 kPa. This is adequate for generating the necessary flexion and extension for the elbow joint in the range of motion study.

# 3.3. Range of Motion Study

#### 3.3.1. Efficacy of Soft Robotic Elbow Sleeve

To quantify the efficacy of the elbow sleeve in assisting elbow flexion and extension, the elbow joint angles measured in the course of the first and second experimental segments were compared. The first experimental segment involved voluntary active elbow flexion and extension without the elbow sleeve while the second segment involved the use of the elbow sleeve in passive rehabilitation mode, flexing and extending a still elbow.

In each experimental segment, elbow flexion and extension were performed in two different directions, vertically and horizontally. As a representation, **Figure 12A** shows a single subject's elbow joint angle from three cycles of active elbow flexion and extension (experimental segment 1), while **Figure 12B** shows the same subject's elbow joint angle under passive contraction (experimental segment 2). Looking at the results from the different genders, for the female subjects, it was found that the average active maximum ROM was 127.8◦ while the average passive maximum ROM was 69.3◦ . This translates to the soft robotic elbow sleeve's capability in achieving 54.2% of the active ROM (with standard deviation of 27.2◦ ). In contrast, for male subjects, it was found that the average active maximum ROM was 132.1◦ while the average passive maximum ROM was 59.7◦ . This meant that the robotic sleeve was able to achieve 45.2% of the active ROM (with standard deviation of 23.4◦ ).

For the results in the horizontal direction, as a representation, **Figure 12C** shows a single subject's elbow joint angle from three cycles of active elbow flexion and extension (experimental segment 1), while **Figure 12D** shows the same subject's elbow joint angle under passive contraction (experimental segment 2). For the female subjects, it was found that the average active maximum ROM was 118.9◦ while the average passive maximum ROM was 63.3◦ . This translates to the soft robotic elbow sleeve's capability in achieving 53.2% of the active ROM (with standard deviation of 15.6◦ ). In contrast, for male subjects, it was found that the average active maximum ROM was 103.3◦ while the average passive maximum ROM was 56.1◦ . This meant that the robotic sleeve was able to achieve 54.3% of the active ROM (with standard deviation of 9.7◦ ).

The results for the different genders across the two experimental segments are summarized in **Table 1**. In all trials, extension of the elbow back to the starting point was achieved.

#### 3.3.2. Electromyography-Driven Active Control

We reproduce here the results of the third segment of the ROM study, in which the control mode of the elbow sleeve was switched to the active rehabilitation mode, and the subjects were tasked with completing horizontal elbow flexion and extension cycles using only activation signals from the two muscle groups on the opposite arm (non-sleeved arm) to control the state of the elbow sleeve.

All the subjects were successful in controlling the state of the elbow sleeve with muscle effort from the target muscle groups detected by the surface EMG sensors. As a representation, **Figure 13** shows the processed EMG data from the biceps and triceps of a single subject normalized to the threshold of 50%, together with the change in elbow joint angle upon EMG activation by the user. It was observed that there is a tendency for co-contraction during extension (as observed by high EMG signals from both muscles). This was possibly due to the action of joint stabilization by both muscle groups, which did not affect the functionality of the elbow sleeve, for the algorithm continues to function so long as the threshold is met by the intended muscle.

# 4. DISCUSSION

In this paper, we presented the design and testing of a soft robotic elbow sleeve that could be used for passive and intentbased neuro-muscular training. The novelty of this design comes from the use of lightweight fiber-reinforced and fabric-based soft actuators for elbow moving, and this device represents the first of its kind in integrating both flexion and extension capabilities. The elbow sleeve's capability in enabling full extension is particularly useful for stroke patients. In studies assessing the motor impairment and activity limitations of stroke patients, it was found that spasticity, which is a velocity-dependent hyperexcitability of muscles to stretch, is a common complication encountered, at an estimate of 60% of post-stroke patients (Watkins et al., 2002; Sommerfeld et al., 2004). This is manifested in the form of a slightly flexed elbow with the forearm across the abdomen. Another common problem that arises in stroke is that of muscle contracture, which also leads to similar complications as spasticity. Typical physiotherapy treatments for such ailments include slow progressive elbow extensions, which

FIGURE 10 | Graphs showing the relationship between force and pressure for the actuators. (A) Graph of tip force vs. pressure of flexion actuator. (B) Graph of resistive force vs. pressure of extension actuator.

has seen reasonable success for ROM reestablishment (Bonutti et al., 1994). As such, this means of therapy can be potentially fulfilled by our elbow sleeve due to the inclusion of extension actuators.

An important design constraint—that of ensuring that extension actuators do not hinder the flexion actuators determined that each of the actuators has to be manufactured differently. In this regard, the flexion actuator is a fiber-reinforced silicone actuator coupled with nylon fabric to influence its bending behavior, whereas the extension actuator is a inflatable fabric actuator which provides resistive forces through a beam bending model. Both actuators show differences in their pressure-force relationships, with the flexion actuator exhibiting a nonlinear sigmoid-shaped curve, and the extension actuator showing a linear relationship. The non-linearity of the flexion actuator is largely due to the anisotropy that is incorporated within the flexion through fiber and fabric reinforcements, while the extension actuator displayed a linear pressure-force relationship in the absence of modifying reinforcements to the material.

vertical sub-experiment. (B) Passive vertical sub-experiment. (C) Active horizontal sub-experiment. (D) Passive horizontal sub-experiment.

TABLE 1 | Summary of experimental results for segments 1 and 2.


In examining the control system of the elbow sleeve, while it is successful in generating and maintaining the necessary flexion and extension for the elbow joint in the ROM study, there remains an undesirable frequency switching which led to a slight oscillatory movement of the elbow sleeve during actuation. To overcome this, a dead-zone can possibly be introduced at the valve controller, creating boundaries around the pressure readings that allow the valves to remain at their current states until the change in pressure is sufficiently large, which is a suggested method by Guo et. al on modifications to improve the PID controller (Guo et al., 2002).

Preliminary studies were carried out on healthy subjects to quantify the basic functionality of the elbow sleeve. From our results across all the experiments, it was observed that while the extension actuators were able to ensure full extension of the elbow, the flexion actuators had difficulties in enabling full flexion of the elbow, achieving an average of approximately 50% of the full ROM across all subjects. As such, there is room for improvement with respect to the ROM achievable by the device. To improve the ROM, it is recommended that finite element modeling of the fiber-fabric-reinforced actuators be conducted to determine the maximum possible force output of these actuators. Once the full potential of the actuators is quantified in these computational models, feedback control loops may then be implemented to control the extent of actuation by monitoring the angle that the elbow is flexed to. Additionally, alternative

materials for actuator fabrication, or an increase in the number of flexion actuators on the device may also be considered.

In comparing the results gathered from each gender, it was observed that the average percentage ROM achieved from using the elbow sleeve was lower for the male subjects, which was expected due to male subjects having a generally higher segment mass as compared to female subjects. On a larger front, this analysis also suggests that there is a need to tailor the mechanical properties of the actuators to fit each individual's requirements.

In comparing the results gathered from elbow motion in the vertical and horizontal directions, it was observed that the variance in vertical motion is lower as compared to that in the horizontal direction of motion. This is possible due to the decreased effects of gravity acting in the horizontal direction as compared to the vertical direction, thereby making it easier to flex the elbow horizontally as compared to vertically. This set of results is useful in further optimizing the control system toward sensing and providing different bending forces suited for the mechanical task at hand.

The EMG mode of control was successfully implemented during the preliminary user studies. This indicates the functionality of the elbow sleeve as a means of providing active-assisted movements. Going forwards, by placing the surface EMG electrodes on the non-supported arm, bimanual practice may be enabled. The significance of this in stroke recovery is that the consensual operation of the non-affected upper limb may cause the stimulation of ipsilateral corticospinal projections to the paretic muscles, which is of relevance to hemiplegic recovery (Hesse et al., 2003). The logic scheme used for the EMG control is primarily based on an on-off algorithm, and a possible improvement to the system would be to integrate angular control into the device by modifying the algorithm. If bimanual control is maintained, two flex sensors calibrated to obtain angle of bending may be placed onto the back of the elbow. Through comparison of the angle bent by both elbows, a third PID regulator may be introduced to identify the error and adjust the supported elbow, using the non-supported elbow as the set-point angle. Another possible solution would be to record signals from more muscle groups. The array of signals and their amplitudes from these muscle groups would then provide a response pattern as type of fingerprint that may be used to identify different points of elbow actuation. These signals may be classified using either a fuzzy approach or an artificial neural network approach (Chan et al., 2000). In this manner, the surface EMG sensors may be placed on the supported arm to detect post-stroke using residual muscle signals.

Finally, the testing of the system was done with six healthy subjects. While this is sufficient for the purpose of demonstrating the functionality of the system in enabling robot-assisted flexion and extension of the elbow with and without intent detection,

## REFERENCES


clinically-relevant conclusions may not be reached as the system has to be tested on individuals with neurological impairments. A limitation of testing healthy subjects is such that, healthy subjects, in contrast with individuals with neurological impairments, have a tendency to activate their agonist muscles during assistive movement. In this study, we found one subject to exhibit biceps activation during passive mode rehabilitation. Moving forwards, the next step to this study would be to conduct clinical trials on stroke patients, who are the intended end-users of the elbow sleeve. These patients would present differing physical conditions (such as their EMG signal amplitudes) which would better elucidate the efficacy of the elbow sleeve, and the improvements that needed to be made.

# AUTHOR CONTRIBUTIONS

TK and NC collected, processed and analyzed the data, drafted the manuscript and contributed equally to the work. TK, NC, HY, and CY designed the study. HY contributed to the interpretation of findings and drafted the manuscript. CY oversaw its coordination and helped to draft the manuscript. All authors read, edited and approved the final manuscript.

# FUNDING

This work was supported in part by the Ministry of Education AcRF Tier 2 grant under Grant R-397-000-203-112.

rehabilitation for chronic stroke. Neurorehabil. Neural Repair 23, 837–846. doi: 10.1177/1545968309338191


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2017 Koh, Cheng, Yap and Yeow. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Design and Preliminary Feasibility Study of a Soft Robotic Glove for Hand Function Assistance in Stroke Survivors

#### Hong Kai Yap1, 2, Jeong Hoon Lim<sup>3</sup> , Fatima Nasrallah<sup>4</sup> and Chen-Hua Yeow<sup>1</sup> \*

<sup>1</sup> Department of Biomedical Engineering, National University of Singapore, Singapore, Singapore, <sup>2</sup> NUS Graduate School for Integrative Sciences and Engineering, National University of Singapore, Singapore, Singapore, <sup>3</sup> Department of Medicine, National University of Singapore, Singapore, Singapore, <sup>4</sup> Queensland Brain Institute, University of Queensland, St. Lucia, QLD, Australia

#### Edited by:

Venketesh N. Dubey, Bournemouth University, United Kingdom

#### Reviewed by:

Brent Winslow, Design Interactive (United States), United States Victor Manuel Pulgar, Wake Forest School of Medicine, United States Arturo Forner-Cordero, University of São Paulo, Brazil Agnes Roby-Brami, Institut National de la Santé et de la Recherche Médicale, France

> \*Correspondence: Chen-Hua Yeow rayeow@nus.edu.sg

#### Specialty section:

This article was submitted to Neural Technology, a section of the journal Frontiers in Neuroscience

Received: 31 March 2017 Accepted: 20 September 2017 Published: 09 October 2017

#### Citation:

Yap HK, Lim JH, Nasrallah F and Yeow C-H (2017) Design and Preliminary Feasibility Study of a Soft Robotic Glove for Hand Function Assistance in Stroke Survivors. Front. Neurosci. 11:547. doi: 10.3389/fnins.2017.00547 Various robotic exoskeletons have been proposed for hand function assistance during activities of daily living (ADL) of stroke survivors. However, traditional exoskeletons involve the use of complex rigid systems that impede the natural movement of joints, and thus reduce the wearability and cause discomfort to the user. The objective of this paper is to design and evaluate a soft robotic glove that is able to provide hand function assistance using fabric-reinforced soft pneumatic actuators. These actuators are made of silicone rubber which has an elastic modulus similar to human tissues. Thus, they are intrinsically soft and compliant. Upon air pressurization, they are able to support finger range of motion (ROM) and generate the desired actuation of the finger joints. In this work, the soft actuators were characterized in terms of their blocked tip force, normal and frictional grip force outputs. Combining the soft actuators and flexible textile materials, a soft robotic glove was developed for grasping assistance during ADL for stroke survivors. The glove was evaluated on five healthy participants for its assisted ROM and grip strength. Pilot test was performed in two stroke survivors to evaluate the efficacy of the glove in assisting functional grasping activities. Our results demonstrated that the actuators designed in this study could generate desired force output at a low air pressure. The glove had a high kinematic transparency and did not affect the active ROM of the finger joints when it was being worn by the participants. With the assistance of the glove, the participants were able to perform grasping actions with sufficient assisted ROM and grip strength, without any voluntary effort. Additionally, pilot test on stroke survivors demonstrated that the patient's grasping performance improved with the presence and assistance of the glove. Patient feedback questionnaires also showed high level of patient satisfaction and comfort. In conclusion, this paper has demonstrated the possibility of using soft wearable exoskeletons that are more wearable, lightweight, and suitable to be used on a daily basis for hand function assistance of stroke survivors during activities of daily living.

Keywords: soft robotic, hand exoskeleton, rehabilitation, activities of daily living, soft actuators

# INTRODUCTION

The ability to perform basic activities of daily living (ADL) impacts a person's quality of life and independence (Katz, 1983; Andersen et al., 2004). However, an individual's independence to perform ADLs is jeopardized due to hand motor impairments, which can be observed in patients with neurological disorders such as stroke. In order to improve hand motor functions in terms of strength and range of motion (ROM) (Kutner et al., 2010), stroke survivors undergo rehabilitation programs comprising repetitive practice of simulated ADL tasks (Michaelsen et al., 2006). Normally, patients undergo rehabilitation exercises in a specialized rehabilitation center under the guidance of physiotherapists or occupational therapists. However, due to increasing patient population, it is foreseen that there will be a shortage of physiotherapists to assist in the rehabilitative process. Thus, there will be comparatively less therapy time, which will eventually lead to a slower recovery process for the patients. Over the past decade, technological developments in robotics have facilitated the rehabilitative process and have shown potential to assist patients in their daily life (Maciejasz et al., 2014). One example of such a device is the hand exoskeleton, which is secured around the hand to guide and assist the movement of the encompassed joints. However, due to the complexity of the hand, designing a hand exoskeleton remains a challenging task.

Traditional hand exoskeletons involve the use of rigid linkagebased mechanisms. In this kind of mechanism, rigid components, such as linear actuators, rotary motors, racks, and pinions as well as rigid linkages are normally involved (Worsnopp et al., 2007; Rotella et al., 2009; Martinez et al., 2010). To assist hand movements that have high degrees of freedom (DOFs), traditional exoskeletons can be incorporated with a substantial number of actuators to achieve the requirement. However, this means that their application is limited due to the increasing bulkiness for higher DOFs. Therefore, these devices are normally restricted in clinical settings and not suitable for performing home therapy. Additionally, their rigidity, weight and constraint on the non-actuated DOFs of the joints pose complications. As a result, the level of comfort and safety of patients is reduced. In view of this, there is an apparent need for the development of exoskeletons that may be used in both clinical and home settings. A lightweight and wearable exoskeleton may allow patients to bring back home to continue daily therapy or to serve as an assistive device for the ADLs.

The development of wearable robotic exoskeletons serves to provide an alternative approach toward addressing this need. Instead of using rigid linkage as an interface between the hand and the actuators, wearable exoskeletons typically utilize flexible materials such as fabric (Sasaki et al., 2004; Yap et al., 2016a) and polymer (Kang et al., 2016), driven by compliant actuators such as cables (Sangwook et al., 2014; Xiloyannis et al., 2016) and soft inflatable actuators (Polygerinos et al., 2015d; Yap et al., 2016c). Therefore, they are more compliant and lightweight compared to the rigid linkage-based mechanism. Cable-driven based exoskeletons involve the use of cables that are connected to actuators in the form of electrical motors situated away from the hand (Nilsson et al., 2012; Ying and Agrawal, 2012; Sangwook et al., 2014; Varalta et al., 2014). By providing actuations on both dorsal and palmar sides of the hand, bi-directional cable-driven movements are possible (Kang et al., 2016). These cables mimic the capability of the tendons of the human hand and they are able to transmit the required pulling force to induce finger flexion and extension. However, the friction of the cable, derailment of the tendon, and inaccurate routing of the cable due to different hand dimensions can affect the efficiency of force transmission in the system.

On the other hand, examples of the soft inflatable actuators are McKibben type muscles (Feifei et al., 2006; Tadano et al., 2010), sheet-like rubber muscles (Sasaki et al., 2004; Kadowaki et al., 2011), and soft elastomeric actuators (Polygerinos et al., 2015b,c; Yap et al., 2015); amongst which, soft elastomeric actuators have drawn increasing research interest due to their high compliance (Martinez et al., 2013). This approach typically embeds pneumatic chamber networks in elastomeric constructs to achieve different desired motions with pressurized air or water (Martinez et al., 2012). Soft elastomeric actuators are highly customizable. They are able to achieve multiple DOFs and complex motions with a single input, such as fluid pressurization. The design of a wearable hand exoskeleton that utilizes soft elastomeric actuators is usually simple and does not require precise routing for actuation, compared to the cabledriven mechanism. Thus, the design reduces the possibility of misalignment and the setup time. These properties allow the development of hand exoskeletons that are more compliant and wearable, with the ability to provide safe human-robot interaction. Additionally, several studies have demonstrated that compactness and ease of use of an assistive device critically affect its user acceptance (Scherer et al., 2005, 2007). Thus, these exoskeletons provide a greater chance of user acceptance.

**Table 1** summarizes the-state-of-art of soft robotic assistive glove driven by inflatable actuators. Several pioneer studies on inflatable assistive glove have been conducted by Sasaki et al. (2004); Kadowaki et al. (2011) and Polygerinos et al. (2015a,b,c). Sasaki et al. have developed a pneumatically actuated power assist glove that utilizes sheet-like curved rubber muscle for hand grasping applications. Polygerinos et al. have designed a hydraulically actuated grip glove that utilizes fiber-reinforced elastomeric actuators that can be mechanically programmed to generate complex motion paths similar to the kinematics of the human finger and thumb. Fiber reinforcement has been proved to be an effective method to constrain the undesired radial expansion of the actuators that does not contribute to effective motion during pressurization. However, this method limits the bending capability of the actuators (Figure S1); as a result, higher pressure is needed to achieve desired bending.

This paper presents the design and preliminary feasibility study of a soft robotic glove that utilizes fabric-reinforced soft pneumatic actuators. The intended use of the device is to support the functional tasks during ADLs, such as grasping, for stroke survivors. The objectives of this study were to characterize the soft actuators in terms of their force output and to evaluate the performance of the glove with healthy participants and stroke survivors. The glove was evaluated on five healthy participants

#### TABLE 1 | Hand assistive exoskeletons driven by inflatable actuators.


in order to determine the ROM of individual finger joints and grip strength achieved with the assistance of the glove. Pilot testing with two stroke survivors was conducted to evaluate the feasibility of the glove in providing grasping assistance for ADL tasks. We hypothesized that with the assistance of the glove, the grasping performance of stroke patients improved.

Specific contributions of this work are listed as follows:


## DESIGN REQUIREMENTS AND RATIONALE

The design requirements of the glove presented in this paper are similar to those presented by Polygerinos et al. (2015a,b,c) in terms of design considerations, force requirements, and control requirements. For design considerations, weight is the most important design criterion when designing a hand exoskeleton. Previous studies have identified the threshold for acceptable weight of device on the hand, which is in the range of 400–500 g (Aubin et al., 2013; Gasser and Goldfarb, 2015). Cable-driven, hydraulic, and pneumatic driven mechanisms are found to be suitable options to meet the criteria. To develop a fully portable system for practical use in home setting, reduction in the weight of the glove as well as the control system is required. The total weight of the control system should not exceed 3 kg (Polygerinos et al., 2015a,b,c). In this work, the criteria for the weight of the glove and control system are defined as: (a) the weight of the glove should be <200 g, and (b) the weight of the control system should be <1.5 kg.

Considering the weight requirement, hydraulic systems are not ideal for this application, as the requirement of a water reservoir for hydraulic control systems and actuation of the actuators with pressurized water will add extra weight to the hand. The second consideration is that the hand exoskeleton should allow fast setup time. Therefore, it is preferable for the hand exoskeleton to fit the hand anatomy rapidly without precise joint alignment. Compared to cable-driven mechanisms, soft pneumatic actuators are found to be more suitable as they allow rapid customization to different finger length. Additionally, they do not require precise joint alignment and cable routing for actuation as the attachment of the soft pneumatic actuators on the glove is usually simple. Therefore, in this work, pneumatic mechanisms were selected. Using pneumatic mechanism, Connelly et al. and Thielbar et al. have developed a pneumatically actuated glove, PneuGlove that is able to provide active extension assistance to each finger while allowing the wearer to flex the finger voluntarily (Connelly et al., 2010; Thielbar et al., 2014). The device consists of five air bladders on the palmar side of the glove. Inflation of the air bladders due to air pressurization created an extension force that extends the fingers. However, due to the placement of the air bladders on the palmar side, grasping activities such as palmar and pincer grasps were more difficult. Additionally, this device is limited to stroke survivors who are able to flex their fingers voluntarily.

In this work, the soft robotic glove is designed to provide functional grasping assistance for stroke survivors with muscle weakness and impairments in grasping by promoting finger flexion. While the stroke survivors still preserve the ability to modulate grip force within their limited force range, the grip release (i.e., hand opening) is normally prolonged (Lindberg et al., 2012). Therefore, the glove should assist with grip release by allowing passive finger extension via reinforced elastic components, similar to Saeboflex (Farrell et al., 2007) and HandSOME (Brokaw et al., 2011). The elastic components of these devices pull the fingers to the open hand state due to increased tension during finger flexion. Additionally, the glove should generate the grasping force required to manipulate and counteract the weight of the objects of daily living, which are typically below 1.07 kg (Smaby et al., 2004). Additionally, the actuators in the glove should be controlled individually in order to achieve different grasping configurations required in simulated ADL tasks, such as palmar grasp, pincer grasp, and tripod pinch. For the speed of actuation, the glove should reach full grasping motion in <4 s during simulated ADL tasks and rehabilitation training.

For the actuators, we have recently developed a new type of soft fabric-reinforced pneumatic actuator with a corrugated top fabric layer (Yap et al., 2016a) that could minimize the excessive budging and provide better bending capability compared to fiber-reinforced soft actuators developed in previous studies (Polygerinos et al., 2015c,d). This corrugated top fabric layer allows a small initial radial expansion to initiate bending and then constrains further undesired radial expansion (**Figure 1**). The detailed comparison of the fiber-reinforced actuators and fabric-reinforced actuators can be found in the Supplementary Material.

Upon air pressurization, the top surface of the actuators expands due to the inflation of the embedded pneumatic chamber (**Figure 1B**). With inextensible layers attached at the bottom of the actuators, bending motion can be obtained due to asymmetrical strain along the length of the actuators. To support a complete hand closure, previous studies, including our group, have developed multi-segment soft actuators that are mechanically programmed to achieve bending and extending motions by controlling the placement of the inextensible bottom layers at different localities along the length of the actuators (Polygerinos et al., 2015b,c; Yap et al., 2017). The bending motion supports finger joint flexion while the extending motion offsets the increased distance due to skin stretching during finger flexion. Bending and extending segments can be pre-designed to conform to the finger anatomy of different patients. This feature demonstrates the high customizability of the soft actuators. However, as the hand dimension varies for different patients, pre-designing actuators with different lengths and controlling the placement of the bending and extending segments for each patient can be time consuming.

Therefore, in this work, we have designed the actuators without pre-programmed bending and extending motions. Instead of controlling the placement of the inextensible bottom layers to achieve bending and extending motions, a single elastic fabric with higher elastic modulus (0.5 N/mm) than the silicone rubber is placed at the bottom of the actuators. With the elastic fabric, the actuators are able to achieve both bending and extending motions (**Figure 1B**). As the actuators are compliant, they are able to conform to the shape of the object during actuation. A bending motion is preferred at the

FIGURE 1 | (A) A fabric-reinforced soft actuators with a corrugated fabric layer and an elastic fabric later [Actuator thickness, T = 12 mm, and length, L = 160 mm (Thumb), 170 mm (Little Finger), 180 mm (Index & Ring Fingers), 185 mm (Middle Finger)]. (B) Upon air pressurization, the corrugated fabric layer unfolds and expands due to the inflation of the embedded pneumatic chamber. Radial budging is constrained when the corrugated fabric layer unfolds fully. The elastic fabric elongates during air pressurization and stores elastic energy. The actuator achieves bending and extending motions at the same time. (C) A bending motion is preferred at the finger joints (II, IV, VI). An extending motion is preferred over the bending motion at the finger segments (I, III, V) and the opisthenar (VII).

location with lower impedance, i.e., the finger joints. At the finger segments that possess higher impedance, the bending motion of the actuators is limited and thus the extending motion is preferred (**Figure 1C**).

# MATERIALS AND METHODS

#### Actuator Fabrication

A two-part 3D-printed mold is used to fabricate the actuators. A lower-part mold (chamber mold) is used to create a pneumatic chamber inside the actuators, which will inflate upon air pressurization. An upper-part mold (top layer mold) is used to impose a corrugated outer layer for the placement of fabric at the top of the actuators (**Figure 1A**). The detailed description of the fabrication process as well as the mold dimension can be found in the Supplementary Material.

#### Actuator Characterization

The actuators were characterized in terms of their blocked tip force and grip force upon pressurization. The blocked tip force

exerted by the actuator was measured over increasing pressures using a customized force measurement system (**Figure 2A**). The system consisted of a compression load cell (FC22, Measurement Specialties Inc., USA) and a mounting platform. The proximal end of the actuator was mounted on the platform and connected to the air source. The distal end of the actuator was in contact with the load cell. A constraining platform was positioned on top of the actuator. During pressurization, the actuator flexed and the tip of the actuator came into contact with the constraining platform, which constrained the height and the curvature of the actuator. This force measurement setup was similar to the setup presented in previous studies (Polygerinos et al., 2015c,d). Constraining the top surface of the actuator minimized the non-linear effects caused by the bending of the actuator when pressurized (Polygerinos et al., 2015d). Thus, this setup could measure the maximum blocked tip force generated by the actuator regardless of the bending angle. The pressure was increased from 0 to 120 kPa. The experiment was repeated three times. Additionally, a theoretical model was adapted from the model presented by Polygerinos et al. (2015d) and Wang et al. (2017) to predict the tip force output of the actuators at this configuration (Figure S4).

For hand exoskeleton application, the force that is of particular interest is the normal grip force (**Figure 2B**) generated by the gripping of the actuators on an object and the frictional grip force (**Figure 2C**) that counteracts the weight of that particular object to prevent it from falling. Both the normal and frictional grip forces were measured using a universal testing machine (Model 3345, Instron, MA, USA). Four actuators, which corresponded to four fingers, were pressurized to 120 kPa to enclose and grasp a cylinder, which was of 50 and 75 mm in diameter, in two orientations (i.e., horizontal orientation to measure the normal grip force and vertical orientation to measure the frictional grip force). The cylinder was pulled upward by the Instron at a fixed velocity (8 mm/s) until the cylinder was released from the actuators' grip. The setup was similar to the setup reported in previous literature (Galloway et al., 2016).

Both the normal and frictional grip forces were measured when they were resisting the upward motion of the cylinder. The normal grip force is the sum of the forces applied by four actuators normal to the cylinder surface. The measured force was that resisting the upward motion of the cylinder, which tried to pull the actuators straight from the default bending state (**Figure 2B**). The frictional grip force is the force exerted by the actuators as the cylinder moves and slides across them (**Figure 2C**). The experiment was repeated three times and the results were averaged.

# Integration and Evaluation of Soft Wearable Robotic Glove

The overall structure of the soft robotic glove is a glove with five finger-actuator pockets attached on the dorsal side (**Figure 3A**). The actuators are integrated to the glove via the actuator pockets (**Figure 3B**). Open palm design is adopted for easy donning and doffing of the glove. The glove is secured to the wearer's hand via the finger pockets and a wrist strap. The finger pocket consists of

anti-slip material to enhance the grip strength. To wear the glove, the wearer just needs to insert the fingers into the finger pockets and secure the glove to the wrist via the wrist strap. The glove base serves as a compliant interface between the actuators and the human hand, providing minimal mechanical impedance to the finger motion when it is being worn and ensuring kinematic transparency.

can be adjusted and attached to the base of the glove via Velcro.

The actuators can be easily inserted into the actuator pockets, which are made from stretchable lycra fabrics (**Figure 3C**). The lycra fabrics serve as second constraining layers for the actuators, which further prevent over expansion of the actuators. Each actuator is isolated with respect to the others. The assistance of each finger can be achieved independently, which allows execution of different simulated ADL tasks. The length of the finger-actuator pockets can be adjusted and attached to the base of the glove via Velcro, in order to accommodate different finger lengths (**Figure 3D**). Both the Velcro loop and the glove cover constrain the bending movement of the actuators at the proximal part of the hand (**Figure 3D**).

The total weight of the glove is ∼180 g, which is much lower than the design requirement. The thickness of the glove (including the actuators) and the width of each finger-actuator pocket is <2 cm. Additionally, as the actuators work under air pressure, inflation of the actuators does not add a significant amount of extra weight to the hand, as compared to hydraulically actuated actuators.

# Pump-Valve Control System

To realize isolated control of each actuator, a pump-valve control system was assembled and integrated into a portable waist belt pack (**Figure 4A**). The control system consisted of a a microcontroller (Arduino Mega, Arduino), a miniature diaphragm pneumatic pump (D737-23-01, Parker, USA), five miniature solenoid valves (X-Valve, Parker, USA), and five air pressure sensors (MPX5500DP, Freescale, USA) (**Figure 4B**). The microcontroller regulated the measured air pressure (P) to track the desired pressure (Pref) and used pulse width modulation (PWM) to control the activation and deactivation of the valves and pump based on the readings of the pressure sensors. The control system could be powered by a 12 V rechargeable lithium polymer battery (DC12300, China). The total weight of the control system was ∼1.26 kg, which was lighter than the control system (3.3 kg) presented by Polygerinos et al. (2015c).

A Proportional Integral Derivative (PID) control algorithm was used to ensure that the measured air pressure (P) of the actuators was close to desired air pressure (Pref). The PID control scheme was implemented on the microcontroller with a sampling frequency of 100 Hz. The valves have a nominal response time of 20 ms and a PWM frequency of 50 Hz was used. The control parameters were selected as K<sup>p</sup> = 10, K<sup>i</sup> = 0.7, and K<sup>d</sup> = 0.1. The desired pressure Pref was set at 120 kPa, which was the pressure that corresponded to full finger flexion based on the results from ROM test. The actuator for the index finger was pressurized from 0 to 120 kPa. The control loop was tested in a step response experiment (**Figure 4C**). The closed-loop system had a rise time of 1.36 s and a settling time (5%) of 1.63 s. The closed-loop system bandwidth was found to be 0.342 Hz (Figure S5).

## Glove-Assisted Range of Motion

Five healthy participants (3 males, 2 females, age: 26.2 ± 2.4 yr) were recruited in order to evaluate the performance of the glove in terms of its assisted-finger ROM and grip strength. All participants gave written informed consent in accordance with the Declaration of Helsinki. The study protocol was approved

by the Institutional Review Board of the National University of Singapore.

Two sessions were conducted for each participant. In the first session, which consisted of three active trials, the participants were instructed to close and open their right hand with the presence of the glove. The length of the finger-actuator pockets was adjusted to fit the finger length of the specific subject. In the second session, which consisted of three glove-assisted trials, the participants were instructed to relax their muscles and not to exert any voluntary finger action. The hand closing and opening were assisted by the actuation of the soft actuators. EMG signals of the finger flexors and extensors were recorded during the trials to identify whether the subject was exerting voluntary control of the hand during glove-assisted trials.

The assisted-finger ROM and the EMG data were obtained through the test method reported in our previous study (Yap et al., 2016c). Additional details regarding the test method are reported in the "Test Method for Glove-assisted ROM" section of the Supplementary Material.

## Glove-Assisted Grip Strength

A portable setup with a digital dynamometer (HF-50, China) was developed to measure the glove-assisted grip strength of the participants (**Figure 5A**). Upon pressurization to 120 kPa, the glove assisted participants with grasping a cylinder (50 and 75 mm in diameter) (**Figure 5B**). The participants were instructed to not exert voluntary force and hence, the grasping of the cylinder would be solely due to the force generated by the inflated actuators. The cylinder was tied to a Kevlar cable attached to the dynamometer. As the participants grasp and lift the cylinder upwards, the cylinder would be slowly released from the actuators' grip, when the tension in the cable became larger than the frictional force applied by the actuators (**Figure 5C**). This setup allowed the measurement of frictional grip force applied by the actuators, with the glove worn by the participants. The setup and the mechanism of the slip of the cylinder from the hand are similar to those reported in previous literature (Westling and Johansson, 1984; Johansson and Westling, 1988; In et al., 2017).

# Pilot Test with Stroke Survivors

A pilot test that evaluated the feasibility of the glove in providing hand grasping assistance for ADL tasks was performed on two stroke survivors. All participants gave written informed consent in accordance with the Declaration of Helsinki. The study protocol was approved by the NHG Domain Specific Review Board. The subject demographics are shown in **Table 2**.

The patients were instructed to use their affected hand to grasp an empty water bottle (Weight: 20 g, Diameter: 60 mm) and a tin can (Weight: 454 g, Diameter: 75 mm) typically used in Jebsen


TABLE 2 | Stroke patients' demographic data.

Hand Function Tests (Beebe and Lang, 2009). The same task was repeated having the patients wear the glove. The task was considered successful if the patient was able to grasp, lift, and put down the object. The participants were allowed to use their nonparetic hand to support their paretic forearm if they could not lift their paretic arm. The time taken to complete the tasks (grasp, lift, and put down the bottle or can) was recorded. The maximum time allowed to complete the task was 90 s. The experiment was repeated three times for each object and the results were averaged across six trials for each patient in each condition (i.e., with and without glove assistance).

In this work, we employed a straightforward button control strategy to control the glove. A phone application with three virtual buttons, (namely grasping, pinching, and tripod pinching) was built to interface with the control system through wireless Bluetooth communication (Figure S7). The patients were instructed to click the virtual buttons using their non-paretic hand to activate the glove for grasping activities. To deactivate the glove, the patients just simply clicked the specific activated button again.

After the test, the patients' feedback on the device was obtained. The patients were instructed to fill in the Usefulness-Satisfaction-and-Ease-of-use questionnaire (USE) (Lund, 2001) and a questionnaire focusing on comfort level, desire to use, and desire to purchase the device. Both questionnaires use a seven-point Likert rating scale that focuses on the experience and feedback of the system.

# RESULTS

# Blocked Tip Force Output

The average values of the experimental force and the force obtained from a theoretical model were compared and shown in **Figure 6**. The force increased with increased pressure (**Figure 6A**). The soft actuators could generate a maximum tip force of 9.12 N at 120 kPa. Similarly, the force values obtained from the theoretical model increased with increased pressure, with a tip force value of 9.47 N at 120 kPa.

# Normal and Frictional Grip Force Output

**Figures 6B,C** plotted the force (normal/frictional grip force) displacement of the cylinder (distance traveled by the cylinder from initial position) characteristics of the force measurement scenario. The average values of the normal and frictional grip force applied by the actuators are shown in **Figures 6B,C**. The results showed that four actuators could generate a normal grip force of up to 36.2 ± 4.5 N (Diameter: 75 mm) and 28.1 ± 3.6 N (Diameter: 50 mm) (**Figure 6B**). The actuators could generate a frictional grip force of up to 18.7 ± 0.9 N (Diameter: 75 mm) and 12.4 ± 1.7 N (Diameter: 50 mm) (**Figure 6C**). The normal and

frictional grip forces decreased with decreased cylinder diameter. As the objects of daily living do not weigh more than 1.5 kg (Matheus and Dollar, 2010), the frictional force was found to be sufficient to lift most of the objects.

### Glove-Assisted Range of Motion

During active trials, the maximum achievable joint angles were 82.3 ± 6.8◦ , 84.1 ± 7.9◦ , and 44.8 ± 4.4◦ at MCP, PIP, and DIP joints. The sums of finger joint angles during active trials were measured to be ∼211◦ . During glove-assisted trials, the maximum achievable joint angles were 73.9 ± 10.4◦ , 79.9 ± 4.2◦ , and 46.0 ± 3.3◦ at MCP, PIP, and DIP joints. The sums of finger joint angles were measured to be ∼200◦ (**Figure 7A**). Statistical analysis was conducted using the non-parametric Wilcoxon's Signed-Rank tests. The tests showed no significant difference between active and glove-assisted trials (p = 0.38 for MCP joint, P = 0.32 for PIP joint, and P = 1.25 for DIP joint).

#### Muscle Activation

The muscle activation signals showed clear differences between active and glove-assisted trials (**Figure 7B**). During the glove-assisted trials, the amplitude of the normalized EMG signal confirmed that the participants exerted minimal muscle effort. Therefore, we concluded that the participants exerted

FIGURE 7 | (A) Averaged range of motion of individual finger joints of five participants during active and glove-assisted trials. MCP, Metacarpophalangeal joint; PIP, Proximal interphalangeal joint; DIP, Distal interphalangeal joint. (B) Averaged muscle activation profiles of finger flexors and extensors of one representative subject (Subject 2) during active and glove-assisted trials. Exercise Cycle, Hand closing followed by opening.

minimal voluntary movement during the glove-assisted trials. The hand closing and opening motions were completely assisted by the glove.

# Glove-Assisted Grip Strength

When the glove was worn by the participants, the glove could generate a frictional grip force up to 8.4 ± 1.8N (Cylinder diameter: 75 mm) and 5.8 ± 1.7 N (Cylinder diameter: 50 mm) to counteract the weight of an object. Statistical analysis was conducted using the Wilcoxon's Signed-Rank tests. The frictional grip force decreased with decreased cylinder diameter. Significant was concluded (P = 0.03).

#### Simulated ADL Tasks

**Figure 8** demonstrates the ability of the glove to hold and lift a 454 g-weighted tin can in vertical (**Figure 8A**) and horizontal (**Figure 8B**) orientation. Additionally, we demonstrated that the glove was also able to assist with palmar grasp and pincer grasp that are required for simulated ADL tasks, such as grasping and pinching (**Figures 8C,D**).

## Pilot Test with Stroke Survivors

Both patients were able to activate the glove using the phone application with virtual buttons (**Figure 9**). Without the assistance of the glove, patient S1 was able to finish the tasks in an average time of 9.0 ± 1.4 s (**Figure 9B**). When patient S1 was grasping and lifting the water bottle, the patient could not grasp the bottle effectively and the bottle toppled. With the assistance of the glove, patient S1 could finish the tasks in an average time of 8.0 ± 0.7 s (**Figure 9C**).

The hand of the patient S2 was slightly clenched. As a result, the patient could not finish the tasks within 90 s as the patient could not grasp both objects (**Figure 9B**). The patient could finish the tasks in an average time of 12.3 ± 2.7 s with the assistance of the glove (**Figure 9C**).

Statistical analysis was conducted using the non-parametric tests. Due to poststroke heterogeneity and small sample size, Wilcoxon's Signed-Rank tests on individual participant data were used to determine each patient's improved time-taken to complete the task with the assistance of the glove. Significance was concluded on patient S2 (p = 0.02). Although significance was not concluded on patient S1 (p = 0.06), we could still observe that patient S1 could grasp the bottle more effectively with the assistance of the glove. Therefore, as hypothesized, the grasping performance of stroke patients improved with the assistance of the glove.

For patients' feedback, the mean and standard deviation (SD) of the questionnaire parameters are presented in **Table 3**. Generally, the patients reported scale of at least five out of seven-point Likert rating.

# DISCUSSION

Compared to the fiber reinforced actuators developed for soft robotic glove application in previous studies, which operate at the pressure range of 275–375 kPa (Polygerinos et al., 2015c), the fabric-reinforced soft actuators designed in this study required

FIGURE 9 | Evaluation of the glove with stroke patient S1 and S2. (A) Hand conditions of patients. (B) Comparison of grasping performances with and without assistance of glove. (C) Time taken to complete the grasping tasks (n = 6 for each patient; P = 0.06 for S1, P = 0.02 for S2, paired t test, \*p < 0.05.). The actuators of the glove were pressurized at 120 kPa.

a lower range of operating pressures to achieve similar force output. With this advantage, we can select pumps and valves that operate at a lower pressure range and consume less power, which will lead to a more portable system.

Compared to the frictional grip force applied by the actuators when the glove was not worn (section Normal and Frictional Grip Force Output), the frictional grip force reduced when the glove was worn by the participants (section Muscle Activation). This might be due to the joint stiffness of the participants that imposed an opposing torque that the actuators needed to overcome during finger flexion. Nevertheless, the force generated was considered sufficient to grasp and achieve most of the ADLs (Matheus and Dollar, 2010; Polygerinos et al., 2015c). The frictional grip force is dependent on the object size, which can be observed from the grip force experiment. The experiment showed that the force output decreased with smaller cylinder diameter. This

achieved with the assistance of the glove.

TABLE 3 | Questionnaire parameters and relative ratings of the USE questionnaire and a questionnaire focusing on comfort level, desire to use, and desire to purchase the device.


SD, standard deviation.

was because the force exerted by the actuators changed with the distance (i.e., the force decreased with increasing actuator bending angle). Additionally, the frictional grip force is also affected by the friction coefficient of the surface of the object and the anti-slip material of the finger pockets on the glove. The friction coefficient between the anti-slip material on the finger pocket and the surface of the cylinder could be estimated from the relationship between normal grip force and frictional grip force. In the measurement scenario described in this work, the coefficient was estimated at between 0.4 and 0.5.

The average functional ROM of the hand, which are sufficient to perform more than 90% of daily functional activities, at MCP, PIP, and DIP are 61, 60, and 39◦ , respectively (Hume et al., 1990). Additionally, as the actuators designed in this study are able to extend to offset the increased distance due to skin stretching during finger flexion, the reported assisted ROM at MCP is higher than the assisted ROM reported in our previous study (Yap et al., 2016c). The EMG experiment confirmed that the participants exerted no voluntary effort during the glove-assisted trials and the hand grasping action was completely assisted by the glove. Therefore, our results demonstrated that the glove could achieve sufficient ROM and provide assistance with grasping motions typically found in ADL, in the absence of voluntary muscle control.

The choice of pneumatic over hydraulic allows the control system to be more portable and lightweight than the control system presented by Polygerinos et al. (2015b,c), as it does not require a water reservoir. However, the limitation of the pneumatic system is that it suffers from slow dynamics due to slow valve discharging speed and slow response times when depressurizing, as presented in previous related works on soft robotic gloves (Polygerinos et al., 2015c; Yap et al., 2017). In this work, the closed-loop system bandwidth of the control system was found to be 0.342 Hz. The value (0.342 Hz) is similar to the bandwidth of the control system presented by Polygerinos et al. (0.25 Hz). Although the bandwidth was less than the common bandwidth of rigid linkage-based and cable-driven hand exoskeletons (Agarwal et al., 2015; Xiloyannis et al., 2016), it can be considered sufficient for patients to conduct functional tasks for ADLs. The valves used in current system could be replaced with valves with faster flow rate in the future to increase the bandwidth of the actuators.

The limitation of the previous soft actuator design is that the actuators apply forces to flex the finger upon air pressurization. Upon depressurization, the actuators assist finger extension passively with the elastic properties of the actuators and the elastic textile materials of the actuator pockets (Polygerinos et al., 2015b,c; Yap et al., 2016a, 2017). The extension force might not be enough for patients with hypertonicity and increased finger flexor tone. In this work, the bottom elastic fabric layer is able to further enhance the extension force required to bring the fingers to the open hand state. The elastic modulus of the elastic fabric (0.45 N/mm) was chosen to match the elastic modulus of the elastic cords described in previous devices that allow the patients to flex their fingers voluntarily and provide assistance in the hand opening (Brokaw et al., 2011). The elastic cords are attached to the distal phalanx of each finger at the dorsal side. The tensions in the elastic cords increase with increasing finger flexion and pull the fingers to the open hand state. Assuming an offset distance of 10 mm due to skin stretching during finger flexion, the elastic fabric is able to provide an estimated 4.5 N of extension force to pull the fingers to the open hand state when the actuators are depressurized.

Pilot testing with two stroke survivors demonstrated the feasibility of the glove in providing functional grasp assistance for ADL. The glove is suitable for stroke survivors with a flaccid hand and without increased flexor tone (Modified Ashworth Scale > 2). Without the assistance of the glove, the patients could not grasp cylindrical objects effectively, due to weak grasp strength and contracture. With the presence of the glove, the patients could then grasp the objects more effectively as the fingers were slightly more extended due to the elastic components of the glove. When the actuators were pressurized, the glove provided an additional grasping force to the patients' hand and assisted with the grasping activities. Therefore, the preliminary results showed that with the presence and assistance of the glove, the patients' grasping performance improved. The results of the USE questionnaire and the patients' feedback revealed that the patients were generally satisfied with the glove. Additionally, patients reported high level of comfort and ease of use. The parameter with the lowest rating is "Desire to purchase". The patients pointed out that they would purchase if the selling price was within their affordable range, which was between 500 and 1,000 USD. Otherwise, they would prefer rental. The most positive aspects that the patients pointed out included (1) The glove serves the purposes of mobilizing the finger joints and assisting their grasping activities and (2) The glove motivates the patient to do rehabilitation exercises. On the other hand, the most negative aspect is that the fabric of the glove should be able to be sanitized and washed easily for hygiene purpose.

#### CONCLUSION

This paper presented a soft robotic glove designed to assist stroke survivors with grasping tasks during their ADLs. Fabricreinforced actuators with a corrugated fabric layer and a reinforced elastic layer were developed and experimental results have shown that the actuators could support finger motions with desired force output at lower operating air pressure, compared to the required operating pressure of previously developed actuators. A control system was built to allow isolated control of each actuator and integrated into a portable waist belt back. Both the glove and the control system are more lightweight than previously developed glove systems. The glove was evaluated with five healthy participants in terms of the assisted ROM and grip strength. Our results demonstrated that the glove was able to achieve sufficient ROM and provided assistance with motions typically found in ADL, in the absence of voluntary muscle control. A pilot test on two stroke survivors with reduced hand function also demonstrated that the glove allowed the patients to perform functional grasping activities more effectively.

# FUTURE WORK

In this work, the experiments on grip force, glove-assisted ROM, and pilot tests with stroke survivors were conducted at a fixed actuator pressure (120 kPa). While the results from section Blocked Tip Force Output have shown that the actuator force increased with increased pressure, it is possible to conduct a more extensive study in the future by obtaining objective measurements, such as force, ROM, and grip strength, as a function of actuator pressure and presenting the time course of the pressure during each task. The force, ROM and grip strength can be controlled by adjusting the air pressure according to patient's condition. This feature will be further explored in subsequent work.

Future iterations of the glove will include sensor components, such as joint angle sensors and force sensors, which can provide joint angle and grasp force feedback. Joint angle feedback allows closed-loop position control and will enable the pumpvalve control system to identify whether the desired handfinger posture has been achieved. The grasp force feedback will permit us to examine the force output of the glove and to facilitate impedance control strategies (Marchal-Crespo and Reinkensmeyer, 2009).

While stroke survivors are the target patient population of the glove presented in this work, the glove can also potentially assist patients with incomplete spinal cord injuries or amyotrophic lateral sclerosis. Apart from serving as an assistive device for ADLs, the glove described in this work can potentially serve as a rehabilitative device, providing task-specific rehabilitation training. Several research groups working on hand exoskeletons have shown that incorporation of these devices into hand therapy can benefit the stroke survivors. Significant improvements in functional outcome have been observed with the combination of assist-as-needed control strategy and user intent detection strategy (Hu et al., 2013; Thielbar et al., 2014, 2017). For example, Theilbar et al. have developed a cable-driven robotic glove with the combination of voice and EMG to detect user intention. Their study has shown that incorporation of the glove into the rehabilitation therapy has the potential for clinical use to improve hand function.

For stroke patients with increased flexor tone and spasticity (Modified Ashworth Scale > 2), the passive extension mechanism might not have enough force to extend the fingers. For these patients, soft robotic glove that is capable of providing active flexion and extension will be preferable. Finally, a larger clinical study will be conducted in order to evaluate and improve the actuator and glove design, and to study the long-term efficacy of the glove-assisted intervention in rehabilitation.

# ETHICS STATEMENT

This study was carried out in accordance with the recommendations of Institutional Review Board of the National University of Singapore (B-14-141) and the NHG Domain Specific Review Board (2015/00975) with written informed consent from all subjects. All subjects gave written informed consent in accordance with the Declaration of Helsinki. The protocol was approved by the Institutional Review Board of the National University of Singapore (B-14-141) and the NHG Domain Specific Review Board (2015/00975).

# AUTHOR CONTRIBUTIONS

HY, JL, FN, and CY designed the study. HY collected, processed and analyzed the data and drafted the manuscript. JL, FN, and CY contributed to the interpretation of findings. CY oversaw its coordination and helped to draft the manuscript. All authors read, edited and approved the final manuscript.

# FUNDING

This work was funded by the A<sup>∗</sup> STAR Medtech Innovation Grant (R397-000-260-305) and the National University Health System (NUHS)-Clinical Imaging Research Center (CIRC) Seed Funding Grant (R-172-000-323-511).

# ACKNOWLEDGMENTS

The authors wish to thank all the participants and their family members who volunteered to take part to the experiment. The authors would like to express their appreciation NUS Graduate School for Integrative Sciences and Engineering for providing scholarship support to HY.

# SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fnins. 2017.00547/full#supplementary-material

# REFERENCES


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2017 Yap, Lim, Nasrallah and Yeow. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Design and Validation of Exoskeleton Actuated by Soft Modules toward Neurorehabilitation—Vision-Based Control for Precise Reaching Motion of Upper Limb

Victoria W. Oguntosin<sup>1</sup> , Yoshiki Mori <sup>2</sup> , Hyejong Kim<sup>2</sup> , Slawomir J. Nasuto<sup>1</sup> , Sadao Kawamura<sup>2</sup> and Yoshikatsu Hayashi <sup>1</sup> \*

<sup>1</sup> Brain Embodiment Lab, Biomedical Engineering, School of Biological Sciences, University of Reading, Reading, United Kingdom, <sup>2</sup> Department of Robotics, Ritsumeikan University, Shiga, Japan

#### Edited by:

Dingguo Zhang, Shanghai Jiao Tong University, China

#### Reviewed by:

Yi Sun, National University of Singapore, Singapore Min Li, Xi'an Jiaotong University, China

> \*Correspondence: Yoshikatsu Hayashi y.hayashi@reading.ac.uk

#### Specialty section:

This article was submitted to Neural Technology, a section of the journal Frontiers in Neuroscience

Received: 16 February 2017 Accepted: 07 June 2017 Published: 07 July 2017

#### Citation:

Oguntosin VW, Mori Y, Kim H, Nasuto SJ, Kawamura S and Hayashi Y (2017) Design and Validation of Exoskeleton Actuated by Soft Modules toward Neurorehabilitation—Vision-Based Control for Precise Reaching Motion of Upper Limb. Front. Neurosci. 11:352. doi: 10.3389/fnins.2017.00352 We demonstrated the design, production, and functional properties of the Exoskeleton Actuated by the Soft Modules (EAsoftM). Integrating the 3D printed exoskeleton with passive joints to compensate gravity and with active joints to rotate the shoulder and elbow joints resulted in ultra-light system that could assist planar reaching motion by using the vision-based control law. The EAsoftM can support the reaching motion with compliance realized by the soft materials and pneumatic actuation. In addition, the visionbased control law has been proposed for the precise control over the target reaching motion within the millimeter scale. Aiming at rehabilitation exercise for individuals, typically soft actuators have been developed for relatively small motions, such as grasping motion, and one of the challenges has been to extend their use for a wider range reaching motion. The proposed EAsoftM presented one possible solution for this challenge by transmitting the torque effectively along the anatomically aligned with a human body exoskeleton. The proposed integrated systems will be an ideal solution for neurorehabilitation where affordable, wearable, and portable systems are required to be customized for individuals with specific motor impairments.

Keywords: 3D printed exoskeleton, soft actuators, compliant assistance, active and passive mechanisms, gravity compensation, reaching motion

# 1. INTRODUCTION

Although an increased effort has been placed on the recovery process of patients following a stroke, with recent advances in technology for monitoring the brain functions, lack of human resources in therapeutic training of patients implies that patients generally may not reach their full recovery potential when discharged from hospital following initial rehabilitation (Loureiro and Harwin, 2007). Neurological illness often manifests in clinical condition resulting in weakness of the muscles controlling the elbow and rehabilitation devices are used in order to help in recovery of muscular power (Maciejasz et al., 2014). In stroke patients, damage to the motor-, and somatosensory cortices, makes it difficult to control and move the arm, forearm, and fingers. These patients often use assistive devices and therapy to encourage them to use their damaged limb so that the re-growth of a neural circuitry can take place that would eventually aid recovery through neuro-plasticity (Jack et al., 2001). Therefore, there is a growing interest in using robotic devices to deliver effective assistance. The aim of the robotic assistance is to provide a set of intensive and repetitive therapies to enhance the motor recovery of patients, decreasing the amount of work of a therapist. It has been found that it is important for a patient to undergo continuous exercise for successful rehabilitation (Heo et al., 2012) which robots are able to provide.

Typically, rehabilitation methods are either highly human intensive or involve attachment of rigid robotic system controlling the upper limb. Robot-based rehabilitation has been shown to have positive effects by reducing impairment (Loureiro et al., 2014). Robots provide an acceptable performance measurement test and control of the amount of exercise delivered to the subject (Kutner et al., 2014). Hard robotic structures are often based on joints connected by rigid links; this makes them heavy, with expensive and complicated control. In addition, these robotic systems are less compliant than the joints they actuate making it difficult to be directly attached to the human body unless through the use of specialized end effectors (Krebs et al., 2000), thereby mitigating their use for personal use by patients. To overcome these challenges, safe human robotic interactions, and inexpensive design can be achieved through a soft wearable robotic device (Tondu and Lopez, 2000; Tsagarakis and Caldwell, 2000).

A novel approach to rehabilitation is making use of inexpensive and soft actuators. Soft actuators are usually actuated with compressed air or SMAs (Shape Memory Alloy) (Chiu et al., 2013). SMA actuation is accomplished by embedding the trained alloy into the silicone rubber and using electric current to produce heat to deform the robot to the trained shape. Soft robots can be realized with the use of McKibben artificial muscles to produce a compliant motion similar to the skeletal muscle (Tsagarakis and Caldwell, 2000) and are actuated using a pneumatic source (Takagi et al., 2009).

Another novel approach uses embedded pneumatic networks of channels in elastomers (Ilievski et al., 2011) to achieve bending and crawling motions (Ilievski et al., 2011). Soft rehabilitation robots have been developed to generate assistive force for grasping (Shepherd et al., 2011) and gait rehabilitation (Shepherd et al., 2011). They make use of air channels embedded in elastomers to perform bending motions similar to the motion of the fingers. Other robots composed of silicone rubber for angular displacement (Sun et al., 2013) and as a 2D (Marchese et al., 2014) or 3D (Martinez et al., 2013) manipulators have been developed. An upper limb orthosis called Orthojacket actuated by pneumatics and designed for elbow rehabilitation has been developed (Schulz et al., 2011).

There have been a lot of research in developing and implementation of robotic devices for upper limb rehabilitation especially index and finger rehabilitation in the form of exoskeletons (Chiri et al., 2012; Maciejasz et al., 2014). These hand exoskeletons make use of cables (Dovat et al., 2008) and rubber (Polygerinos et al., 2015) for actuation purposes. Virtual reality has been used together with these hand exoskeletons to further aid in recovery (Ueki et al., 2012). Target tracking have also been incorporated in order to increase motivation and cooperation from the user as well as individualize exercise difficulty depending on strength the user (Jack et al., 2001).

In traditional, robotic rehabilitation systems interacting with the patient using a single distal attachment point on the forearm by means of an orthosis [for example, MIT-MANUS (Krebs et al., 2004)], exercises are defined in the XYZ Cartesian space relative to the robots single end-effector attachment point, and the assistance magnitude is modulated using impedance/admittance control schemes in the robot task-space. To support comfortably upper limb flexion and extension, WREX, Wilmington Robotic EXoskeleton, was designed at A. I. DuPont Hospital for Children to provide assistance for the wide range of natural motion in daily settings (Sanchez et al., 2006; Tariq Rahman et al., 2006). The JAECO WREX is a light weight exoskeleton with two links and four degrees of motion that approximates normal human anatomy. It incorporates elastic band elevation assists for both the shoulder and elbow to eliminate influence of gravity on the extremity (Sanchez et al., 2006; Tariq Rahman et al., 2006). The unique design of the shoulder and elbow joints allows for a significant improvement in the available range of motion when compared to other assistive devices.

A new type of wearable robots called soft exoskeletons or exosuits is becoming more popular, aiming at therapeutic intervention of upper limb by mirroring the motion of the other limb (Panasonic, 2005). Removing the external metal frame, soft exoskeletons become compliant, much lighter, and could be significantly less expensive than rigid frame wearable robots. However, soft exoskeleton's main drawback is the same as their main strength: they have no external rigid frame to support the body parts and transfer force effectively from actuator to some area of the body. We think that the effective transfer of the force should take into account the anatomical structure of limbs and joints, requiring the solid structure of the assistive device in order not to add unnecessary strain on the body parts that might damage patient joints, muscles, or tendons.

As a unique approach to a lightweight and flexible robotic structure, inflatable structures have been developed for the robotic arm as an attractive alternative method of imbuing robots with the desired lightness. Kim et al. developed the robotic arm that comprised of the inflatable links, air bag actuators, and ABS joint structures (Kim et al., 2015). Also, the novel control schemes have been applied on the basis of visual feedback control, which do not require the precise inverse kinematics for the position control (Nishida and Kawamura, 2012).

In our study, aiming at the reaching motion of the upper limb, we developed and validated the Exoskeleton Actuated by Soft Modules (EASoftM) to fulfill the four conditions; (1) A few degrees of freedom, (2) Precise motion control, (3) Wearable assistive robots, (4) Compliant assistance. We proposed the exoskeleton aligned with the anatomical structure, actuated by the soft modules located at the joint position, and the soft actuators realize the compliant motion based on pneumatic actuation and visco-elastic properties of soft materials such as plastic and rubber.

From the patient perspective, the ultra-light EASoftM offers upper limb assistance, i.e., it can assist the reaching motion of the upper limb, supporting the movement of the elbow and shoulder, and can record trajectories of hand reaching a target based on the visual feedback control. It also has high compliance and high safety integrity from the perspective of the patient, because it is entirely composed of rubber and soft modules, and attached to the body via Velcro straps. We first describe the design specification of the EASoftM with the scene setting (Sections 2.1 and 2.2), and secondly describe the detailed kinematics and the vision-based control schemes (Sections 3.1– 3.3). After explaining the production of the plastic modules (Section 3.4), their torque characteristics and reliability were presented (Section 4). In the Result section (Section 5.1), a series of experiments is described for the motion of one and two degrees of freedom, first tested by using the weight. Finally, we presented the participant experiments, demonstrating the closedloop reaching motion on precise position control (Section 5.2). The General discussion is given in Section 6.

As a contribution to neurorehabilitation, the EASoftM can be integrated with the Brain Computer Interface, for example, for detecting the motor command generation by online electroencephalogram (EEG) analysis, in order for the EASoftM to guide the hand to a target position in a rehabilitation therapy. The advantage of using EASoftM is that it does not include any motors which generate the elecro-magnetic waves which inevitably influence the electric signals from brain activity.

# 2. DESIGN AND SPECIFICATIONS

#### 2.1. Scene Setting for Rehabilitation

To minimize detrimental effects of technology on the rehabilitation process, some of the technical requirements present in the design of most rehabilitation robots take into consideration the ergonomics of the system and its ability to cope with a variety of patient demographic and anthropomorphic parameters. An imperative requirement in the design of the human machine interface is that it should mimic behavior of the human therapist. That is, it should be compliant when assisting movement, provide full support within the patients passive range of motion and nourish the patient's confidence and motivation levels through goal-oriented motion with informative biofeedback.

Reaching motion is one of the most critical behaviors in our daily life at home; at the table, we reach and grasp a cup to drink, and in the kitchen we reach for the spice jars or pots. Thus, it is important to assist the motion for reaching purposes. required system characteristics can be decomposed into the following;


the shoulder and elbow to assist the reaching motion.

5. Allowance for small deviation from the planned trajectories: Passive joints should should accommodate small spontaneous motion with compliance, and soft modules should provide compliance naturally by contained air and soft material.

We aim at rehabilitation for people with neuromuscular weakness such as muscle disease, cerebral palsy, spinal cord injury, and stroke, that effect the spontaneous motion of upper limbs. Also, given a proper balance between passive and active joints, the proposed system should provide an appropriate active assistance in guiding the hand to a target position. The integrated system should act as a functional aid in activities of daily living ideally at home, serving as a cost effective exercise/therapy device for people recovering from stroke.

# 2.2. Design Specifications

All of the considerations in Section 2.1 lead to five main specifications to be fulfilled by the Exoskeleton Actuated by Soft Modules which can assist the reaching motion;


# 3. DESIGN, MATERIAL, AND METHODS

Fulfilling the design specifications in the Section 2.2 in order to assist comfortable flexion and extension of the upper limb, we mainly concentrated on three characteristics; (1) A few degrees of freedom; anatomically structured exoskeleton with the parallel links for gravity compensation, (2) Compliant assistance: soft modules to actuate the elbow and shoulder joints, and (3) Precise motion control: visual control loop to guide the hand to the target at high precision.

# 3.1. Kinematic Structure of Exoskeleton Actuated by Soft Modules (EASoftM)

The overall structure of the proposed EASoftM is given in **Figure 1** with a schematic representation of a participant,

FIGURE 1 | Kinematic structure of Exoskeleton Actuated by Soft Modules (EASoftM) (A) Overall structure of the proposed EASoftM with a participant. (B) Close-up picture of upper body. P R represents the base coordinate of the robot, P C1 the camera coordinate used in the vision-based control law, and P C2 the camera coordinate to record the trajectories of the end-effector. The exoskeleton has two parallel links and four degrees of motion that approximates normal human anatomy of upper limb. (C) Overall structure of the proposed EASoftM (view from top). The attachment to the upper limb was set up at the end of exoskeleton to support the wrist of a participant. (D) Denavit-Hartenberg parameters of exoskeleton structure. The soft modules are attached to the joints (q1 and q4) to rotate the joints, and the rubber bands are attached to the parallel structure of link L2 and L5 to fix the joint angles of q2 and q5 within a certain range, resulting in compensation of the gravity. (E) Kinematic structure of exoskeleton in 2D work plane. As the passive joints (q2 and q5) lift up the upper limb,the active joints rotates shoulder and elbow (q1 and q4) to make a reaching motion possible within this 2D plane.


E L5 0 0 0

TABLE 1 | DH parameters for the kinematic structure.

where P R represents the base coordinate of the robot, P C1 the camera coordinate used in the vision-based control law, and P C2 the camera coordinate to record the trajectories of the end-effector. The exoskeleton has two parallel links and four degrees of motion that approximate normal human anatomy of the upper limb. The attachment to the upper limb was set up at Robot link L<sup>5</sup> to support the wrist of a participant.

TABLE 2 | Length of links of exoskeleton.


The kinematic structure exoskeleton is given in **Figure 1D** with DH (Denavit-Hartenberg) parameters in **Table 1** (Denavit and Hartenberg, 1955). The length of the links is provided in **Table 2**. Four joints were represented by q where q<sup>1</sup> and q<sup>4</sup> were active joints and q<sup>2</sup> = −q<sup>3</sup> (parallel link) and q<sup>5</sup> were passive joints.

Before actuation of the soft modules, these two passive joints, q<sup>2</sup> and q<sup>5</sup> were adjusted by the elastic property of the rubber band in order to shift the equilibrium point of upper limb by compensating the gravity. Two soft modules were attached to q<sup>1</sup> and q<sup>4</sup> of the exoskeleton to actuate pneumatically the rotation of joints.

By combining the passive and active joints, we can predetermine the 2D work plane, in which two active joints guide the wrist to the target position. The soft actuators made of polyethylene were attached to individual joints of exoskeleton, providing the visco-elastic property in guidance of motion. All the parts were printed by the 3D printer using ABS plastic. The ultra-light polyethylene modules were used as ideal material for designable and wearable soft robotic modules.

#### 3.2. Control Schemes of Exoskeleton Actuated by Soft Modules (EASoftM)

This study used a control scheme which exploited direct visualmotor loops in the feedback control to actuate robotic arms (Nishida and Kawamura, 2012). Based on the transformations from visual space to actuator space, the proposed control scheme will allow a robot to execute visually directed reaching motion with its end-effector without reference to its precise kinematics and necessity to calculate the inverse kinematics. The block diagram of the control law is given in **Figure 2**.

Specifically, we apply the Proportional and Integral (PI) feedback control laws to the relative distance between the target and the end effector of the exoskeleton in the 2D work plane defined in **Figure 1E**. First, we calculate the Jacobian of the exoskeleton in this 2D plane (**Figure 1E**), which is used in the control law. In this 2D work plane, the position of the endeffector is given by

$$\begin{aligned} \mathbf{X} &= \begin{bmatrix} \mathbf{x} \\ \mathbf{y} \end{bmatrix} \\ &= \begin{bmatrix} (L\_1 - L\_3)\sin q\_1 + L\_2\cos q\_1 + L\_4\sin(q\_1 + q\_4) \\ + L\_5\cos(q\_1 + q\_4) \\ (L\_1 - L\_3)\cos q\_1 - L\_2\sin q\_1 + L\_4\cos(q\_1 + q\_4) \\ + L\_5\sin(q\_1 + q\_4) \end{bmatrix} \end{aligned} \tag{1}$$

Thus, the Jacobian, **J**(**q**) is given in the 2D plane by **X˙** = **J**(**q**)**q˙**, where **q˙** = - **q˙**1 **q˙**4 T . **J**(**q**) is given by

$$\mathbf{J}\_{(\mathbf{q})} = \begin{bmatrix} a\_{11} \ a\_{12} \\ a\_{21} \ a\_{22} \end{bmatrix},\tag{2}$$

$$\begin{aligned} a\_{11} &= (L\_1 - L\_3)\cos q\_1 - L\_2 \sin q\_1 + L\_4 \cos(q\_1 + q\_4) \\ &- L\_5 \sin(q\_1 + q\_4) \\ a\_{12} &= L\_4 \cos(q\_1 + q\_4) - L\_5 \sin(q\_1 + q\_4) \\ a\_{21} &= -(L\_1 - L\_3)\sin q\_1 - L\_2 \cos q\_1 - L\_4 \sin(q\_1 + q\_4) \\ &+ L\_5 \cos(q\_1 + q\_4) \\ a\_{12} &= -L\_4 \sin(q\_1 + q\_4) + L\_5 \cos(q\_1 + q\_4) \end{aligned}$$

Generally speaking, a **PI** control law for the target torque is given by

$$\pi\_{\mathbf{d}}(\mathbf{t}) = \mathbf{K}\_{\mathbf{P}} \mathbf{J}\_{\mathbf{(q)}}^{\mathrm{T}} \Delta \mathbf{X} + \int\_{\mathbf{0}}^{\mathbf{t}} \mathbf{K}\_{\mathbf{I}} \mathbf{J}\_{\mathbf{(q)}}^{\mathrm{T}} \Delta \mathbf{X} \mathbf{d} \boldsymbol{\delta}, \tag{3}$$

with τ**d**(**t**): Target torque, **J T** (**q**) : Transposed Jacobian, **KP**: Partial gain parameter, 1**X**: Relative distance between the target and the end-effector in the robot coordinate, 6R, **K<sup>I</sup>** : Integral gain parameter.

Here, we need to perform the coordinate transformation from the camera coordinate, 6C1, to the robotic coordinate, 6<sup>R</sup> (**Figure 1**). Thus, 1**X** is calculated in the following way;

$$
\Delta \mathbf{X} = \mathbf{C\_1} \mathbf{W\_1} \Delta \mathbf{U\_1},\tag{4}
$$

where **C<sup>1</sup>** is the coordinate direction transformation matrix, **W<sup>1</sup>** is the coordinate rotation transformation matrix, and the relative distance between the target and the end-effector, 1**U<sup>1</sup>** is represented by

$$
\Delta \mathbf{U\_1} = \begin{bmatrix} u\_{d1} - u\_1 \\ \nu\_{d1} - \nu\_1 \end{bmatrix}, \tag{5}
$$

where ud<sup>1</sup> : Pixel value of the target on x-axis in 6R, vd<sup>1</sup> : Pixel value of the target on y-axis in 6R, u1: Pixel value of the eneffector on x-axis in 6R, v1: Pixel value of the en-effector on y-axis in 6R.

In this study the **W<sup>1</sup>** is given by

$$\mathbf{W\_{I}} = \begin{bmatrix} \cos(\theta\_{\text{\text{\text{\text{\text{\text{\text{\text{\text{\text{\text{\text{\text{\text{\text{\text{\text{\text{\text{\text{\text{\text{\text{\text{\text{\text{\text{\text{\text{\text{\text{\text{\text{\text{\text{\text{\text{\text{\text{\text{\text{\text{\text{\text{\text{\text{\text{\text{\text{\text{\text{\text{\text{\text{\text{\text{\text{\text{\text{\text{\text{\text{\text{\text{\text{\text{\text{\text{\text{\text{\text{\text{\text{\text{\text{\text{\text{\text{\text{\beta}}}}}}}}}}}}}}}}}}}}}}}}}}}}} $$
} 

with θx<sup>1</sup> = 60 [deg]: Angle between x-axis of 6<sup>R</sup> and P C1 , θy<sup>1</sup> = 30 [deg]: Angle between x-axis of 6<sup>R</sup> and P C1 .

However, **W<sup>1</sup>** is always accompanied by the error from calibration, thus, we approximate **W<sup>1</sup>** to be unit matrix.

**C<sup>1</sup>** is given by

$$\mathbf{C\_{I}} = \begin{bmatrix} 0 & -1 \\ -1 & 0 \end{bmatrix}. \tag{7}$$

Using the coordinate transformation matrix and the video image obtained from the camera, and from Equation (3), we apply the following control scheme to calculate the target torque;

$$\mathbf{r\_{d}(t)} = \mathbf{K\_{P}} \mathbf{J\_{(qd)}}^{T} \mathbf{C\_{I}} \Delta \mathbf{U\_{I}} + \int\_{0}^{t} \mathbf{K\_{I}} \mathbf{J\_{(qd)}}^{T} \mathbf{C\_{I}} \Delta \mathbf{U\_{I}} \mathbf{d} \delta \tag{8}$$

where **J**(**qd**) **<sup>T</sup>** is a transposed Jacobian (Equation 2) which is approximated by the one close to the target position and kept constant throughout the reaching motion. **K<sup>P</sup>** is a gain factor for proportional control, and **K<sup>I</sup>** for integral control.

As a next step, let us calculate the control law based on the robotic coordinate, 6R. Using the coordinate transformation from the camera to the work space coordinate system, Equations (4) and (8) becomes

$$\pi\_{\mathbf{d}}(\mathbf{t}) = \mathbf{K}\_{\mathbf{P}} \mathbf{\tilde{J}}^{\mathbf{T}} \Delta \mathbf{X} + \int\_{0}^{t} \mathbf{K}\_{\mathbf{I}} \mathbf{\tilde{J}}^{\mathbf{T}} \Delta \mathbf{X} d\boldsymbol{\delta} \tag{9}$$

where ˜**J T** has been modified as follows,

$$\mathbf{j}^{\mathrm{T}} = \mathbf{J}\_{(\mathbf{q}\mathbf{d})}{}^{\mathrm{T}} \mathbf{C}\_{1} \mathbf{W}\_{1}{}^{-1} \mathbf{C}\_{1}{}^{-1}. \tag{10}$$

Note here that ˜**J T** is the approximated Jacobian, which contains errors. However, the convergence of 1**X** is guaranteed as long as there is not so much difference between ˜**J T** and **J**(**qd**) **T** (Nishida and Kawamura, 2012). Thus, the proposed control scheme here does not require any encoders to measure the angle of the joints, contributing to the ultra-light weight of the exoskeleton.

In order to output the target pressure to control the soft modules, it is necessary to convert the target torque to the target pressure, which require the detail modeling and the non-linear mapping. This relation should be generally given by **P<sup>d</sup>** = **B**(**q**)τ**d**. However, in this work, we simply related the desired torque with the desired pressure by **P<sup>d</sup>** = **B0**τ**d**. Note here, that this simple relation was made possible due to the positive and negative pressure actuation. We set B<sup>0</sup> = 1, and only use **K<sup>P</sup>** and **K<sup>I</sup>** as the gain parameters to be tuned. The gain parameters will be determined empirically for each experimental condition. The schematic summary of the control scheme is given in **Figure 2**. As experimental results, we will show the trajectories of the end-effector in the scale of [mm], using 1**U**.

# 3.3. Camera Coordinate to Record the Trajectories of the End-Effector

In this section, we explain how we set up the camera coordinate to record the trajectories of the end-effector in **Figure 1**. While performing the experiments, another camera located above the 2D work plane records the trajectories of the end-effector. Based on the robot coordinate, 6R, the relative distance between the target and the end-effector, 1**X** is given by

$$
\Delta \mathbf{X} = \mathbf{C\_2} \mathbf{W\_2} \Delta \mathbf{U\_2}, \tag{11}
$$

where **C2**: Coordinate direction transformation matrix from 6C<sup>2</sup> to 6R, **W2**: Coordinate rotation transformation matrix from 6C<sup>2</sup> to 6R.

1**U<sup>2</sup>** is the relative distance between the target position and the end-effector based on 6C<sup>2</sup> in **Figure 1** given by

$$
\Delta \mathbf{U\_2} = \begin{bmatrix} u\_{d2} - u\_2 \\ v\_{d2} - v\_2 \end{bmatrix}, \tag{12}
$$

with ud<sup>2</sup> : Pixel value of the target position on x-axis, vd<sup>2</sup> : Pixel value of the target position on y-axis, u2: Pixel value of the endeffector position on x-axis, v2: Pixel value of the end-effector position on y-axis.

In this study, **W<sup>2</sup>** is calculated by setting the camera on θx<sup>2</sup> = 0 [deg], θy<sup>2</sup> = 0 [deg].

$$\mathbf{W\_2} = \begin{bmatrix} \cos(\theta\_{\text{j2}}) & 0\\ \sin(\theta\_{\text{x2}})\sin(\theta\_{\text{j2}}) & \cos(\theta\_{\text{x2}}) \end{bmatrix} = \begin{bmatrix} 1 & 0\\ 0 & 1 \end{bmatrix},\tag{13}$$

with θx2: Angle between x-axis of 6<sup>R</sup> and 6C2, θy2: Angle between y-axis of 6<sup>R</sup> and 6C2.

And **C<sup>2</sup>** is given by

$$\mathbf{C\_2} = \begin{bmatrix} -1 & 0\\ 0 & 1 \end{bmatrix}.\tag{14}$$

FIGURE 6 | Torque as a function of (A) Pressure and (B) Joint angle when applied positive pressure.

# 3.4. Production of Soft Actuators and Exoskeleton

The polyethylene soft modules developed for the active joints were designed to be attached to the joints (q<sup>1</sup> and q<sup>4</sup> in **Figure 1D**), and have multi-cells with one end glued together by thermal adhesion (**Figure 3**). The ABS frames were used to maintain the structure of each cell, when deflating the modules by applying negative pressure.

The inflatable actuator was pressurized by pneumatic actuators through silicon tubes, and as the individual cells start to push against each other, the whole module produces the torque in the fan-shaped manner as a function of pressure. Also, the modules could produce negative torque by being deflated. The pressure could be as low as −30 [kPa] and range up to 30 [kPa]. This range of pressure was sufficient to actuate the exoskeleton carrying the weight of upper limb of participants.

The forces generated by the soft modules are distributed along the entire length of L1, L3, and L4, providing the torque to rotate the shoulder and elbow joint respectively as in **Figure 1E**. Note here, that reaching motion can be only realized in the two dimensional plane defined by the kinematic structure in **Figure 1E**. The ABS parts of exoskeleton were printed by a 3D printer. The parts for the parallel link structures were assembled, using the ABS made bolts.

#### 3.4.1. Development of Control Circuits

The control diagram is given in **Figure 4**. We used a USB camera (logicool HD WEBCAM C525) located above the shoulder of exoskeleton for the visual feedback control to calculate the relative distance between the target and the end-effector. The marker was attached to the forearm to represent the hand position. The visual processing was programmed in C++ (Microsoft Visual Studio 2013). The control law in Equation (8) outputs the desired pressure to actuate the inflatable plastic modules. The desired pressure was transmitted to the regulator which controlled the valves of the air flow via DA board.

In order to measure the pressure at the vicinity of soft modules, the pressure sensors (SMC ISE10-M5-C) were placed,

line) and y(t) (Gray line) as a function of time, using the camera on the shoulder. (B) Visual deviation (1U1) x(t) (Black line) and y(t) (Gray line) as a function of time, using the camera from above. (C) The pressure as a function of time. The black line represents the target pressure, and the gray line represents the measured pressure.

and AD board (Contec AI-1616LI-PE) was used to transmit the signals to the PC. The positive pressure regulator (SMC ITV2050- 212BS) and the negative pressure regulator (SMC ITV20290- 212BS5) was connected to each soft actuator through the flow regulator valve (SMC VEF3121-1) which switched the pressure between positive and negative. The positive pressure regulator was connected to the tank with the compressor (IWATA TFP04B-10 C) and the negative pressure tank to the vacuum pump (IWATA SCROLL VACUUM PUMP).

# 4. TORQUE CHARACTERISTICS OF SOFT ACTUATORS

The torque characteristics of soft actuators, torque as a function of the joint angle, was investigated using the experimental setup shown in **Figure 5**. We performed the torque measurement of the soft modules in the following procedure;


Thus, in **Figures 6**, **7** we measured the torque produced by the soft module at the equilibrium state by fixing the angle of the joint in **Figure 5**. We characterized the functional properties of the soft modules. The torque was measured as a function of (1) pressure in the range from −30 to 30 [kPa] at the incremental step of 3.0 [kPa]; and (2) an angle when the soft module was actuating the rotational joint with two rigid arms (**Figure 5**).

**Figure 6** shows the torque as a function of (a) Pressure and (b) Joint angle. The general trend can be observed; (1) the torque increased as a function of pressure and (2) the torque decreased as a function of angle. For the case of applying the negative pressure to the soft module, **Figure 7** shows the torque as a function of (a) Pressure and (b) Joint angle. The general trend can be noted; (1) the torque increased as a function of pressure and (2) the torque increased as a function of an angle.

We expect that the largest hysteresis when switching the pressure from positive to negative, as it takes a certain time for the air flow in pipes and plastic modules to relax to the steady state. However, we aim at the upper limb assistance in slow motion, thus, air flow would be equilibrated throughout the actuation.

# 5. FUNCTIONAL PROPERTIES OF THE EXOSKELETON ACTUATED BY SOFT MODULES (EAsoftM)

The developed Exoskeleton Actuated by Soft Modules is shown in **Figure 8**. The rubber bands were attached to the parallel structures of the exoskeleton in order to compensate the gravity, and the soft modules were attached in-between two plastic plates on shoulder and elbow joints. The total weight of the EASoftM was 967 [g], composed of assembled 3D printed arm (724 [g]), soft module (31 [g]), and attachment (181 [g]). The end-effector was attached to the end-point of the links, and the camera was located on the shoulder position to apply the visual feedback control in **Figure 1**.

In the following sections, we validated the functional properties of the EASoftM, first by putting the weight of 1.0 [kg] on the attachment of the exoskeleton arm, and secondly, by

performing the participant experiments as shown in **Figure 8D**. In both experiments, the relative distance between the endeffector and target, and trajectories in the 2D camera frames were measured as a function of time as well as the pressure applied to the individual soft modules.

# 5.1. Functional Properties of the Exoskeleton Actuated by Soft Modules (EASoftM) under the Weight Load

In this section, we demonstrated the functional properties of EASoftM with the weight load of 1.0 [kg] attached to the exoskeleton arm. To validate the motion of the EASoftM for positive and negative pressure actuation, we characterized the motion of the exoskeleton for flexion and extension, respectively, from one degree of freedom (elbow) to two degrees of freedom (elbow and shoulder). Finally, we performed more complex reaching motion which combined the extension and flexion by switching target positions.

# 5.1.1. 1D Elbow Motion Using the Elbow Joint of the Exoskeleton Actuated by Soft Modules (EASoftM)

#### **5.1.1.1. Flexion motion toward the target (positive pressure actuation)**

For the 1D rotation of the elbow of the EASoftM, the gain parameters were set to kp<sup>1</sup> = kp<sup>2</sup> = 2.0 and ki<sup>1</sup> = ki<sup>2</sup> = 0.2 in Equation (8). **Figures 9A,B** shows the visual deviation (relative distance between the target and the end-effector, 1**U<sup>1</sup>** in Equation 5) of x(t) and y(t) as a function of time, using the camera on the shoulder. Note here, that this visual deviation was input to the control law in Equation (8) whereas **Figure 9B** shows the visual deviation (relative distance between the target and the end-effector, 1**U<sup>2</sup>** in Equation 12), using the camera from above to record the trajectories of the end-effector. It takes about 12 s to reach the target set at x = y = 0.

**Figure 9C** shows the pressure of the soft module as a function of time. Since the relative distance between the target and the tracer is highest at the initial position of the tracer, the pressure profile shows the slow convergence to the target pressure, reflecting the slow pneumatic actuation of the soft modules. Due to the loss of the pressure through the pipelines, the deviation between the target pressure and the measure pressure remained.

The final visual deviation was △x = 0.44 [mm] and △y = −0.20 [mm] in **Figure 10A**, demonstrating the substantial accuracy for reaching motion. This accuracy also proves that the visual-based control as represented in Equation (8) can work efficiently with the approximated kinematics. The trajectory of the tracer attached to the attachment of the exoskeleton shows the circular trajectory as a result of the motion of the elbow rotation in **Figure 10B**.

#### **5.1.1.2. Extension motion to the target (negative pressure actuation)**

To actuate the soft modules by the negative pressure, the gain parameters were set to kp<sup>1</sup> = kp<sup>2</sup> = 2.5 and ki<sup>1</sup> = ki<sup>2</sup> = 0.2 in Equation (8). We could successfully show that extension of the elbow of the EMSoftM was possible by applying the negative pressure as shown in **Figure 11**. The convergence was shown to the target position (**Figures 11A,B**) and to the target pressure (**Figure 11C**). The ABS frame inserted to each plastic cell worked to prevent the collapse of the structure of the cells when deflating air from them. **Figure 12** shows the trajectories of the endeffector, demonstrating the extension of the exoskeleton. The final visual deviation was △x = 0.30 [mm] and △y = 1.65 [mm] in **Figure 12A**.

#### 5.1.2. 2D Motion Using the Elbow and the Shoulder Joints of the Exoskeleton Actuated by Soft Modules (EASoftM)

As a next step, to validate the 2D motion using the elbow and the shoulder rotation, we performed the experiments using the weight on the attachment under following conditions; (1) Flexion motion to the target, (2) Extension motion to the target, (3) Triangular motion with three targets.

#### **5.1.2.1. Flexion motion to the target (positive pressure actuation)**

The gain parameters were set to kp<sup>1</sup> = 2.0, kp<sup>2</sup> = 2.0 for the elbow joint, and kp<sup>1</sup> = 0.2, kp<sup>2</sup> = 0.2 for the shoulder in Equation (8). For the case of 2D motion using the elbow and shoulder joints

the Exoskeleton Actuated by Soft Modules. (A) Visual deviation (1U1) x(t) (Black line) and y(t) (Gray line) as a function of time, using the camera on the shoulder. (B) Visual deviation (1U2) x(t) (Black line) and y(t) (Gray line) as a function of time, using the camera from above. (C) The pressure as a function of time. The black line represents the target pressure, and the gray line represents the measured pressure.

of the EASoftM, it was shown that the reaching motion was also possible as shown in **Figures 13A,B**.

The drop in pressure around 14 s in **Figures 13C,D** was due to the change of torque generated by the posture transformation and the gravitational force, as the exoskeleton arm flexed. The non-linear nature of this drop should come from the kinematics of the exoskeleton arm, which is highly non-linear itself. The flow regulator valve controls the constant air flows from the positive and negative regulator, and the pressure within a range of −5.0 and 5.0 [kPa] is outside the precision of the air flow/pressure regulators. However, due to the friction between the ABS frames, this low pressure range will not result in any substantial torque generation. Since the relative distance between the target and the tracer is highest at the initial position of the tracer, the pressure profile shows the slow convergence reflecting the slow pneumatic actuation of the soft modules. The trajectories of the end-effector are shown in **Figure 14**.

The relative distance to the target was achieved to be △x = 0.65 [mm] and △y = −1.75 [mm]. This precision is surprising as the control laws in Equation (8) only used the visual information of the relative distance between the target and the end-effector as input to the control law. Note here, that the kinematics of the exoskeleton is only provided as approximation and is kept constant throughout the reaching motion. In the classical control theory, once a target has been defined in the robots work space using its sensors, the necessary final pose to intercept the object must be calculated using some form of inverse kinematics: the explicit transformation of work space coordinates into joint angles. However, the inverse kinematics is very sensitive to error, thus, the visual-based control should be appropriate for control of soft modules.

#### **5.1.2.2. Extension motion to the target (negative pressure actuation)**

The gain parameters were set to kp<sup>1</sup> = 2.0, kp<sup>2</sup> = 2.0 for the elbow joint, and kp<sup>1</sup> = 0.2, kp<sup>2</sup> = 0.2 for the shoulder joint in Equation (8). It was shown that the extension motion is also possible when applying the negative pressure to achieve the flexion motion as shown in **Figures 15A,B**. Due to the integral control accumulating the visual deviation and the friction between the ABS frames, there were, around 20 and 35 s, two steep changes in values of the measured pressure in **Figures 15C,D**. Note here that the pressure range between −5.0 and 5.0 [kPa] is out of the controllable range of the flow/pressure regulators in **Figure 4**. The measured negative pressure in the flow regulation valve is attributed to the constant deflation of the air from the negative pressure regulator. **Figure 16** shows the trajectories of the end-effector. The relative distance to the target was achieved to be △x = −0.93 [mm] and △y = −1.57 [mm].

#### **5.1.2.3. Reaching motion with three target positions**

We placed three targets to allow the end-effector to demonstrate the closed loop motion, i.e., the end-effector should come back to its initial place. The target was switched to another target when the relative distance between the target and the end-effector reaches <5.0 [mm]. The gain parameters were set to kp<sup>1</sup> = 2.0, kp<sup>2</sup> = 2.0 for the elbow joint, and kp<sup>1</sup> = 0.2, kp<sup>2</sup> = 0.2 for the shoulder joint in Equation (8).

**Figures 17A,B** show the position of the target, x(t) and y(t), using the camera on the shoulder. We can clearly observe that the end-point reaches the first target, the second target, and goes back to the initial position where the third target was located. **Figure 18** shows the target and measured

target is denoted by the circle.

pressure as a function of time for the shoulder (a) and the elbow (b) joint. When switching the target position, there were corresponding jumps in the target pressure, but note here that the measured pressure followed a smooth curve from the positive to negative pressure around 7 s in **Figures 18A,B**. It is a promising feature of the soft module used to actuate the exoskeleton to demonstrate the smooth transition for switching the sign of the pressure, leading to the smooth transformation of motion from flexion to extension.

For the case of reaching motion by switching three targets sequentially, it was shown that the reaching motion could demonstrate the closed loop motion as shown in **Figure 19**, even though there was some overshoot in the relative distance and

FIGURE 14 | Trajectories of the end-effector (positive pressure actuation) in x-y plane, using (A) the camera on the shoulder and (B) the camera from above. The target is denoted by the circle.

time, using the camera from above. (C) The pressure of the shoulder module as a function of time. The black line represents the target pressure, and the gray line represents the measured pressure. (D) The pressure of the elbow module as a function of time. The black line represents the target pressure, and the gray line represents the measured pressure.

the measured pressure as a function of time when switching the targets.

# 5.2. Functional Properties of the Exoskeleton Actuated by Soft Modules (EASoftM) Validated by Participant Experiments

To evaluate the functionality of EASoftM in providing assistance with a wide range of reaching motion and precision to reach a target, we performed the experiments with one participant. The ethical approval was permitted by the ethical committee of Ritsumeikan University (BKC-medical-2016-051). **Figure 8D** shows the Exoskeleton system attached to the wrist of the participant. The equilibrium point of the upper limb was shifted to match the desired work space appropriate for reaching motion. The visual process using the camera on the shoulder could successfully monitor the position of the tracer to apply the visual control law (Equation 8).

#### 5.2.1. Flexion Motion to the Target (Positive Pressure Actuation)

The gain parameters were set to kp<sup>1</sup> = 2.0, kp<sup>2</sup> = 2.0 for the elbow joint, and kp<sup>1</sup> = 0.2, kp<sup>2</sup> = 0.2 for the shoulder in Equation (8). For the case of 2D motion using the elbow and shoulder joints of the EASoftM, it was shown that the reaching motion is also possible, as shown in **Figures 20A,B**.

The sudden rise in pressure around 8 s in **Figures 20C,D** was due to the configuration changes of the soft modules under the friction. However, this type of fluctuation within the low pressure range does not produce enough torque to affect the rotation of the joints Since the relative distance between the target and the tracer is highest at the initial position of the tracer, the pressure profile shows the slow convergence reflecting the slow pneumatic actuation of the soft modules. The trajectories of the end-effector are shown in **Figure 21**. As the exoskeleton itself was designed based on the anatomical nature of human body, the two link motion in parallel to the human upper limb was smooth. The relative distance to the target was achieved to be △x = −0.89 [mm] and △ y = 2.66 [mm].

#### 5.2.2. Extension Motion to the Target (Negative Pressure Actuation)

The gain parameters were set to kp<sup>1</sup> = 2.0, kp<sup>2</sup> = 2.0 for the elbow joint, and kp<sup>1</sup> = 0.2, kp<sup>2</sup> = 0.2 for the shoulder joint in Equation (8). It was shown that the extension motion is also possible when applying the negative pressure to achieve the flexion motion as shown in **Figures 22A,B**. **Figures 22C,D** show the pressure of the shoulder and elbow module respectively as a function of time. After 3 s from the initial targets, their pressure converged to the target pressure. **Figure 23** shows the trajectories of the end-effector. The relative distance to the target was achieved to be △x = 10.12 [mm] and △ y = 9.92 [mm].

FIGURE 18 | Reaching motion using the elbow and shoulder joint of the Exoskeleton Actuated by Soft Modules, using the camera on the shoulder. The black line represents the target pressure, and the gray line represents the measured pressure. (A) The pressure of the shoulder module as a function of time. (B) The pressure of the elbow module as a function of time.

the second target by square, and the third target by triangle.

#### 5.2.3. Reaching Motion with Three Target Positions

By switching three targets sequentially, we demonstrated the closed loop motion, i.e., the end-effector should come back to its initial place. The target was switched to another target when the relative distance between the target and the end-effector reached <5.0 [mm]. The gain parameters were set to kp<sup>1</sup> = 2.20, kp<sup>2</sup> = 2.20 for the elbow joint, and kp<sup>1</sup> = 0.50, kp<sup>2</sup> = 0.50 for the shoulder joint in Equation (8).

**Figures 24A,B** show the position of the target, x(t) and y(t), using the camera on the shoulder. We can clearly observe that the end-point reaches the first target, the second target, and goes back to the initial position where the third target was located. **Figure 25** shows the target and measured pressure as a function of time for the shoulder (a) and the elbow (b) joint. When switching the target position, there were corresponding jumps in the target pressure. Overall, it is a promising feature of the soft module used to actuate the exoskeleton to demonstrate the smooth transition in switching the sign of the pressure, leading to the smooth transition from flexion to extension.

For the case of reaching motion, by switching three targets sequentially it was shown that the EASoftM could demonstrate the closed loop motion as shown in **Figure 26**, even though there were some overshoots in the relative distance and the measured pressure as a function of time when switching the targets. Note here, that the straight trajectories can be realized by tuning the transformation matrix from visual to work space in Equation (8) (Henry Eberle and Hayashi, 2017).

## 6. GENERAL DISCUSSION

Therapeutic intervention using the assistive robots has been proven to be effective when including the following design specification (Loureiro et al., 2014);

1. A few degrees of freedom; Motion in daily life requires the coordinated joint motion. Thus, the assistive robots are expected to support a set of joints in upper or lower limbs.

FIGURE 20 | Flexion motion (positive pressure actuation) of the elbow and the shoulder of the Exoskeleton with the participant. (A) Visual deviation (1U1) x(t) (Black line) and y(t) (Gray line) as a function of time, using the camera on the shoulder. (B) Visual deviation (1U1) x(t) (Black line) and y(t) (Gray line) as a function of time, using the camera from above. (C) The pressure of the shoulder module as a function of time. The black line represents the target pressure, and the gray line represents the measured pressure. (D) The pressure of the elbow module as a function of time. The black line represents the target pressure, and the gray line represents the measured pressure.


To realize the precise motion with a few degrees of freedom, the conventional methods would suggest the industrial type robot such as metal exoskeleton which is a hard, rigid structure and heavy (Krebs et al., 2000), not fulfilling the condition 3. A new

(Black line) and y(t) (Gray line) as a function of time, using the camera on the shoulder. (B) Visual deviation (1U2) x(t) (Black line) and y(t) (Gray line) as a function of time, using the camera from above. (C) The pressure of the shoulder module as a function of time. The black line represents the target pressure, and the gray line represents the measured pressure. (D) The pressure of the elbow module as a function of time. The black line represents the target pressure, and the gray line represents the measured pressure.

type of a wearable robot called soft exoskeletons or exosuits is an interesting alternative. Removing the external metal frame, soft exoskeletons become compliant, much lighter, and could be significantly less expensive than rigid frame wearable robots (Panasonic, 2005). However, soft exoskeletons main drawback is the same as their main strength: they have no external rigid frame to support the body parts and transfer force effectively from actuator to some area of body.

We think that the effective transfer of the force should take into account the anatomical structure of limbs and joints, requiring the solid structure of the assistive device. Elderly and disabled people already have weakened bones and muscles, so if the force is transmitted in the wrong manner, adding unnecessary strain to their body parts, it might lead to joint degeneration, damage their muscles or tendons. Thus, in our study, aiming at the reaching motion of upper limb, we developed

FIGURE 24 | Reaching motion using the elbow and shoulder joint of the Exoskeleton, using the camera on the shoulder. (A) x(t) as a function of time. (B) y(t) as a function of time.

FIGURE 25 | Reaching motion using the elbow and shoulder joint of the Exoskeleton, using the camera on the shoulder. The black line represents the target pressure, and the gray line represents the measured pressure. (A) The pressure of the shoulder module as a function of time. (B) The pressure of the elbow module as a function of time.

and validated the exoskeleton actuated by soft modules to fulfill these four conditions. We proposed the exoskeleton aligned with the anatomical structure actuated by the soft modules located at the joint positions, and these soft actuators realized the compliant motion based on pneumatic actuation and viscoelastic properties of soft materials such as plastic and rubber. As far as we are aware of, this proposed structure demonstrated the first possibility of integrating the rigid exoskeleton and the soft actuators to assist the reaching motion, as well as the vision-based control law showing its effective functionality in guiding the hand of the participant to the target.

This vision-based control has a certain advantage when the exoskeleton is used in the daily life, i.e. the visual analysis can be easily extended to detect the target such as cups and pens in the work space, and it does not require any encoders and a set of cables transmitting the signals, which add technical complexity to the integrated system. Thus, closing the control loop only by visual information is a flexible solution that can easily integrate low level motion control with higher level intelligent control. More specifically, since we aim at slow reaching motion, the necessary torque mainly will be used to compensate the gravity. Although, it is in principle possible to use the active actuator for this purpose, however, it would be hard to produce the wearable device aligned with the anatomical structure, as it would increase the number of actuators required. Thus, in our study we used the passive actuator for gravity compensation to decrease the weight for wearable purposes. As a next step, we developed the pneumatic actuators made of plastic sheets, which has good tensile strength and is light, and could produce substantial torque for our purposes under 1 atmospheric pressure. Conventionally, antagonist actuator is necessary to actuate the single joint, resulting in more weight and higher number of tubes. In our study, we could successfully demonstrate that the positive and negative pressure can be applied subsequently to produce the torque to increase and decrease the angle of joints. The low pressure actuation contributed to the smooth transition of the pressure as shown in **Figures 6**, **7**.

Most of the assistive robots were made of rigid links and motors which used metallic materials. To realize the compliant motion, the mechanical structure required additional component such as spring, which also added more weight onto the whole structure (Loureiro et al., 2011). Also, considering the assistive soft robotics, though the pneumatic actuators are light weighted, they were used with the metal links and pulleys, thus, the total weight tend to increase, making them inappropriate for the wearable solution (for example, see Schulz et al., 2011). Our proposed exoskeleton aimed at the ultra-light solution, using the 3D printed links and the soft modules (plastic sheets) for actuation, and reducing the number of actuations by employing

#### REFERENCES

Chiri, A., Vitiello, N., Giovacchini, F., Roccella, S., Vecchi, F., and Carrozza, M. C. (2012). Mechatronic design and characterization of the index finger module of a hand exoskeleton for post-stroke rehabilitation. the negative pressure actuation and gravity compensation. In addition, despite of the challenges imposed by such choice of hardware components materials, we did not lose the precision of position control since the vision-based control was employed.

The further work is planned to make a whole system portable and wearable. This can be achieved by the pneumatic actuators, the regulators and the power supplies being mounted in a waist belt pack so that the participants can freely move around. Also, aiming at the wearable and portable system, we can use the small motors to pump in the air, and the simple electric circuits to control the pressure of the soft modules (Oguntosin et al., 2015). By placing those electric components in the waist belt, the electromagnetic waves should not interfere with the EEG measurement. In the near future, we will integrate the robotic system with an eye tracking system to transmit the intention of the participant identifying the desired targets, as a natural extension of the vision-based control schemes toward the daily exercise. As a mid-term plan, we will integrate the proposed robotic system with Brain Computer Interface to estimate the user's motion intention in order to trigger the assistive motion (Hayashi et al., 2012; Kondo et al., 2015), as the system discussed in this article provides an optimal solution for the measurement of the EEG signals, since its construction made of plastic (metal-free structure and actuation) in principle does not produce any electromagnetic waves.

#### ETHICS STATEMENT

This study was carried out in accordance with the recommendations of Ritsumeikan medical ethical guidelines and Ritsumeikan Ethical Committee with written informed consent from all subjects (BKC-medical-2016-051). All subjects gave written informed consent in accordance with the Declaration of Helsinki. The protocol was approved by the Ritsumeikan Ethical Committee.

#### AUTHOR CONTRIBUTIONS

VO and YM substantially contributed to the development of the systems, data acquisition, and analysis. HK contributed to the performance of experiments, and the interpretation of the results. SN, SK, and YH contributed to the design of the work and overall discussion. All of us contributed to the writing of the paper.

## ACKNOWLEDGMENTS

This work is partially supported by the School of Biological Sciences, University of Reading and by the Department of Robotics, Ritsumeikan University.

Chiu, M. C., Yeh, L. J., and Hung, T.-C. (2013). The development of wrist rehabilitation assistive devices using a shape memory alloy. Adv. Mater. Res. 740, 408–415. doi: 10.4028/www.scientific.net/AMR.740.408

IEEE ASME Trans. Mechatr. 17, 884–894. doi: 10.1109/TMECH.2011.21 44614


based on flexible elastomers. Adv. Mater. 25, 205–212. doi: 10.1002/adma.201 203002


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2017 Oguntosin, Mori, Kim, Nasuto, Kawamura and Hayashi. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Assessment of Motor Control during Three-Dimensional Movements Tracking with Position-Varying Gravity Compensation

Yao Huang1, 2 †, Qianqian Yang1, 2 †, Ying Chen1, 2 and Rong Song1, 2 \*

*<sup>1</sup> Key Laboratory of Sensing Technology and Biomedical Instrument of Guangdong Province, School of Engineering, Sun Yat-sen University, Guangzhou, China, <sup>2</sup> Guangdong Provincial Engineering and Technology Center of Advanced and Portable Medical Devices, School of Engineering, Sun Yat-sen University, Guangzhou, China*

#### Edited by:

*Dingguo Zhang, Shanghai Jiao Tong University, China*

#### Reviewed by:

*Steven Weidong Su, University of Technology, Sydney, Australia Qinghua Huang, South China University of Technology, China Wei Zhou, Xiamen University, China*

> \*Correspondence: *Rong Song songrong@mail.sysu.edu.cn*

*† These authors have contributed equally to this work.*

#### Specialty section:

*This article was submitted to Neural Technology, a section of the journal Frontiers in Neuroscience*

Received: *31 December 2016* Accepted: *20 April 2017* Published: *16 May 2017*

#### Citation:

*Huang Y, Yang Q, Chen Y and Song R (2017) Assessment of Motor Control during Three-Dimensional Movements Tracking with Position-Varying Gravity Compensation. Front. Neurosci. 11:253. doi: 10.3389/fnins.2017.00253* Active movements are important in the rehabilitation training for patients with neurological motor disorders, while weight of upper limb impedes movements due to muscles weakness. The objective of this study is to develop a position-varying gravity compensation strategy for a cable-based rehabilitation robot. The control strategy can estimate real-time gravity torque according to position feedback. Then, the performance of this control strategy was compared with the other two kinds of gravity compensation strategies (i.e., without compensation and with fixed compensation) during movements tracking. Seven healthy subjects were invited to conduct tracking tasks along four different directions (i.e., upward, forward, leftward, and rightward). The performance of movements with different compensation strategies was compared in terms of root mean square error (RMSE) between target and actual moving trajectories, normalized jerk score (NJS), mean velocity ratio (MVR) of main motion direction, and the activation of six muscles. The results showed that there were significant effects in control strategies in all four directions with the RMSE and NJS values in the following order: without compensation > fixed compensation > position-varying compensation and MVR values in the following order: without compensation < fixed compensation < position-varying compensation (*p* < 0.05). Comparing with movements without compensation in all four directions, the activation of muscles during movements with position-varying compensation showed significant reductions, except the activations of triceps and in forward and leftward movements, the activations of upper trapezius and middle parts of deltoid in upward movements and the activations of posterior parts of deltoid in all four directions (*p* < 0.05). Therefore, with position-varying gravity compensation, the upper limb cable-based rehabilitation robotic system might assist subjects to perform movements with higher quality and improve the participation of robot-aided rehabilitation training. Further studies are needed to explore the effectiveness and clinic application across pathologies.

Keywords: gravity compensation, upper limb rehabilitation, cable-based rehabilitation robotics, arm tracking, muscle activation

# INTRODUCTION

Since the upper limb dysfunctions after stroke seriously affect daily lives, it is essential for patients to restore the affected motor functions through rehabilitation training. Post-stroke reaching movements is affected by gravity weight of upper limb due to the weakness of those anti-gravity muscles (Beer et al., 2007). Position control was adopted in traditional robots (Lahouar et al., 2009; Yuan et al., 2014) and rehabilitation robots (Lum et al., 2002; Staubli et al., 2009), to follow predetermined trajectories. Although, position control in rehabilitation robots could counteract the effect of gravity weight of upper limb, it focused on control accuracy and ignores patients' voluntary participation in task execution, and its effectiveness in rehabilitation therapy needed further improvement. For patients who suffer from muscle weakness, gravity support showed promising results in minimizing the gravity-induced interference for tasks execution (van Elk et al., 2005; Cheng et al., 2015; Runnalls et al., 2015).

Gravity compensation can be provided by support devices (Herder et al., 2006; Kloosterman et al., 2010) or robotics (Kahn et al., 2006; Ball et al., 2007; Ladenheim et al., 2013), and can be generally grouped into three categories. Firstly, fixed or manually adjusted compensation were commonly used to counteract the gravity weight of upper limb (Nef et al., 2007; Stopforth, 2013; Lenzo et al., 2015). Similarly, a fixed external vertical force was applied to an upper limb robot by a motorized vertical cabling system for gravity compensation (Ball et al., 2007). Secondly, passive compensations by flexible force from elastic materials were chosen (De Luca et al., 2005; Stienen et al., 2009). As an assistive arm exoskeleton, T-WREX provided gravity compensation from the elastic bands whose number could be adjusted at different arm support levels (Housman et al., 2009). Arm orthosises with springs was designed to passively compensate arm weight, and could help patients with little moving ability to reach and grasp (Herder et al., 2006; Kramer et al., 2007). Freebal, a device with gravity compensation at different levels from cables connected to springs, could facilitate upper limb movements tracking in the horizontal and vertical planes (Prange et al., 2009; Kloosterman et al., 2010; Coscia et al., 2014). Thirdly, position-varying compensations can be provided according to human physical characteristics, since gravity torque of upper limb is highly coupled with the dynamics of the limbs and dependent on the postures and positions of moving limbs. Hsu et al. proposed an active control strategy to estimate the subject's movement intention and the control strategy included a gravity compensation term modified by the upper limb dynamics (Hsu et al., 2012). Cheng et al. developed a two degrees of freedom (2-DOF) compliant beam which could compensate the torques on each joint based on the dynamic of the upper limb (Cheng et al., 2015). A torque-angle model containing a gravity compensation term was proposed by Lin et al. to evaluate motion quality of adhesive capsulitis patients (Lin et al., 2014).

Since the gravity torque is highly dependent on positions of upper and fore arm during movements, fixed or varying elastic compensation strategies did not consider the position-coupling effect. Although, Hsu et al. and Li et al. included compensation strategies according to human physical characteristics, both their strategies were combined with other control terms (Hsu et al., 2012; Li et al., 2015) which might affect voluntary participation. Gravity compensation should provide suitable assistance to subjects without affecting voluntary participation, but few previous studies about compensation strategy considered both varying position of upper limb and voluntary participation. Meanwhile, how varying position affects movement performance and muscle activations is rarely reported.

The objective of this study is to develop a position-varying gravity compensation strategy for a cable-based rehabilitation robot. A gravity torque estimation model according to position feedback is proposed for estimating real-time upper limb gravity torque. Then, the performance of this strategy is compared with the other two kinds of gravity compensation strategies (i.e., without compensation and with fixed compensation) during four different directions of man-machine cooperation movements tracking. Root mean square error (RMSE) between target and actual moving trajectories, normalized jerk score (NJS), mean velocity ratio (MVR) of main motion direction, and the mean activation of six muscles are used for assessing different compensation strategies with a cable-based rehabilitation robot.

# MATERIALS AND METHODS

#### Participants

Seven healthy men (mean age: 23.7 ± 1.1 yrs., mean weight: 63.5± 9.3 kg, mean height: 172.5 ±4.5 cm) were recruited in this study. All the subjects were able to lift their right arm against gravity, and had no musculoskeletal or neurological problems. All subjects provided their written informed consent prior to participating in this study. All experimental procedures were approved by the human ethic committee of the First Affiliated Hospital of Sun Yat-sen University.

## Experimental Apparatus

A cable-based rehabilitation robotic system was recruited for providing assistance to counteract the influence of gravity during multi-joints upper limb movements (**Figure 1**). The robotic system consisted of a mechanical part, a motion capture system with four cameras (OptiTrack, NaturalPoint, USA), a sixchannel surface electromyographic (EMG) signals amplifier, a 16 bit analog-to-digital data acquisition card (PXI-6229, National Instruments, USA), and a personal computer. The mechanical part composed of a cubic base frame made of aluminum links, a splint, three cables, and three motors (DM1B-045G, Yokogawa, Japan) with three servo drivers (UB1DG3, Yokogawa, Japan; Yang et al., 2016). The splint was controlled by three motors through three bundled cables, resulting in 3 degrees of freedom (DOF). The group of motors could apply force on the split and assist the user performing movements in a 3-dimensional (3D) space. Three markers were attached at the dorsal centers of three joints (i.e., wrist, elbow, and shoulder), respectively, to record actual positions by the motion capture system. The sampling rate of the motion capturing was set at 100 Hz, and raw position data were filtered with a second-order Butterworth filter with a cutoff frequency of 6 Hz. Bi-polar surface EMG of six superficial muscles of the upper extremity [i.e., biceps (BIC), triceps (TRI),

anterior (DA), middle (DM), posterior (DP) parts of deltoid, and upper trapezius (TRA)] were recorded by attaching electrodes on the subject's skin. The raw EMG data from the six muscles were converted by the data acquisition card. The EMG signals were recorded at 1,000 Hz, amplified with a gain of 5,000, and band-filtered by a 4th-order Butterworth filter with a band of 10–400 Hz.

## Experimental Procedure

Before experiment, subjects should be seated in a chair and their trunks were strapped by soft fabrics to minimize the trunk movements. The initial posture of upper limb positioned at: the elbow flexion was at 90◦ , the forearm pronation was at 90◦ , the upper arm hung down vertically and was close to the trunk, the wrist was extended and the fingers were closed. A computer screen was placed in front of the subject for providing real-time visual feedback of the target and actual wrist positions. A yellow slide block represented the user's wrist showed the actual wrist position, and a red sliding block represented the target position. As shown in **Figure 2A**, each subject went through four kinds of multi-joint tracking tasks with the right arm [i.e., move upward (A)/forward (B)/leftward (C)/rightward (D) to the place 0.2 m away from the initial point along a straight line]. During one movement series, subjects are asked to finish three movement sessions including three kinds of gravity compensation strategies. And each session required the execution of six consecutive laps.

#### Gravity Compensation Strategies

The position-varying gravity compensation strategy was based on real-time estimation of upper limb gravity torque. A gravity torque estimation model was built to estimate the gravity torque from the joint positions and moments of the shoulder, elbow, and wrist:

$$T = \mathbf{G}\_{\mathbf{u}} L\_{\text{sub}} + \mathbf{G}\_{f} L\_{f\text{sh}} + \mathbf{G}\_{\text{h}} L\_{\text{hsh}} \tag{1}$$

where **T** was the gravity torque of upper limbs about shoulder,**Gh**,**G<sup>f</sup>** ,**G<sup>u</sup>** were the gravity of hand, forearm and upper arm, respectively, and Lhsh, Lfsh, L ush were the moment arms from limb centroids of hand, forearm and upper arm to centroid of shoulder joint, respectively.

The equivalent force to the wrist can be calculated from the gravity torque as Equation (2):

$$F\_{\mathfrak{g}} = T / L\_{\text{shw}} \tag{2}$$

where **F<sup>g</sup>** here was the equivalent force of the arm, Lshw was moment arm from limb centroids of shoulder to centroid of wrist joint.

The dynamic characteristics of the robotic system can be analyzed based on the static force model when the splint moved slowly and stably (**Figure 2B**). The resultant compensation force (FOUT) then can be expressed as Equation (3)

$$F\_{OUT} = -(F\_{\mathcal{J}} + \mathbf{G}\_{split})\tag{3}$$

where **Gsplit** was the gravity of the split.

Orientations of the three cables should be considered when driving the motors and controlling the split. The cable-based structure required all cables kept in tension, and the tensile forces needed during arm movements can be presented as Equation (4):

$$F\_T = J^{-1} F\_{OUT} \tag{4}$$

where **F<sup>T</sup>** = [**FT**1, **FT**2, **FT**3] **<sup>T</sup>** was the tensile force matrix, and J = [**u**1, **u**2, **u**3] was the unit vector matrix of the mechanism structure.

There were the other two kinds of gravity compensation strategies (i.e., without compensation and with fixed compensation). The fixed compensation strategy provided a fixed gravity torque calculated when the wrist was in the initial place. When the strategy without compensation was applied, subjects were asked to finish movements without the assistance of the cable-based robot. The weight of the split is 2 kg, thus when finishing movements without assistance, subjects were required to carry an extra 2 kg weight to guarantee the same experiment condition and to simulate the patients with arm weakness.

trajectories during movements tracking with different gravity compensation strategies. \*Significant difference was found between two kinds of gravity compensation strategies (*p* < 0.05).

#### Huang et al. Assessment of Position-Varying Gravity Compensation

#### Data Analysis

RMSE between the target trajectory and the actual trajectory of wrist was used to evaluate movement accuracy.

$$\text{RMSE} = \sqrt{\sum\_{i=1}^{N} \left( (X\_{a\_i} - X\_{t\_i})^2 + (Y\_{a\_i} - Y\_{t\_i})^2 + (Z\_{a\_i} - Z\_{t\_i})^2 \right) / N} \tag{5}$$

Where i refers to the sampling point, Xa, Ya<sup>i</sup> , Za<sup>i</sup> refer to the actual values of 3D coordinates and Xt<sup>i</sup> , Yt<sup>i</sup> , Zt<sup>i</sup> refer to the target values of 3D coordinates.

In order to evaluate the control abilities of arm, NJS of the actual trajectory during movement, was adopted to represent movement smoothness in 3D space (Hogan and Sternad, 2009).

$$\text{NJS} = \sqrt{\frac{1}{2} \times \frac{T^5}{D^2} \times \int \left| \stackrel{\cdots}{s(t)}^2 dt \right.} \tag{6}$$

Where t refers to the actual time, s(t) refers to the position at the time of t, and T, D refer to the duration time, distance during the movement, respectively.

The MVR in each main motion direction is proposed for quantifying the relative velocity deviation in desired direction during movements tracking. It was assessed for movement efficiency, and was calculated as:

$$\text{MVR} = \sum\_{i}^{N} \left( V\_{m\_i} / \sqrt{V\_{x\_i}^2 + V\_{y\_i}^2 + V\_{z\_i}^2} \right) / N \tag{7}$$

Where i refers to the sampling point, Vm<sup>i</sup> refers to the velocity in the main motion direction at the point of i, and Vx<sup>i</sup> , Vy<sup>i</sup> , Vz<sup>i</sup> refer to the velocity in X, Y, Z direction at the point of i, respectively. N refers to the total sampling points.

Envelopes of EMG signals were obtained after a full-wave rectification and a low-pass filter at 20 Hz. The muscle activation was calculated for each muscle and mean muscle activation of one muscle was the mean value during the whole sampling time in one tracking movement.

Two-way ANOVA (Analysis of Variance) as a statistical model was applied to test the main effect of compensation strategy (without, with fixed, and with position-varying gravity compensation), tracking direction (upward, forward, leftward, and rightward) and the interaction effect of these two factors on the RMSE, NJS, MVR, and mean muscle activation values. In the following, post-hoc Tukey tests as multiple comparisons were performed to test the difference on RMSE, NJS, and MVR. Paired t-tests were then utilized to compare RMSE, NJS, MVR, and mean muscle activation among the three compensation strategies in each direction. The significance level for all statistical tests was set at 0.05. All statistical analysis was conducted with SPSS version 22.0 (SPSS Inc., Chicago, IL).

#### RESULTS

**Figures 3A–C** displayed RMSE (**Figure 3A**), NJS (**Figure 3B**), and MVR (**Figure 3C**) during movements tracking with different gravity compensation strategies. The results of ANOVA tests were shown in **Table 1**. As shown in **Table 1**, both compensation strategy and movement direction showed notable influences on RMSE (P < 0.05), NJS (P < 0.05), and MVR (P < 0.05). No significant interaction between compensation strategy and movement direction was found. The results showed that there were significant effects of compensation strategies in RMSE and NJS values represented by the following order: without compensation > fixed compensation > positionvarying compensation, and MVR values represented by the following order: without compensation < fixed compensation < position-varying compensation. According to the posthoc analysis, RMSE during upward and forward movements were remarkably higher than those during the leftward and rightward movements while MVR during upward and forward movements were significantly smaller than those during the leftward and rightward movements. Additionally, there was a significant decrease in NJS during upward movements compared with those during rightward movements. Based on paired t-test, RMSE of position-varying compensation strategy reduced significantly in leftward and rightward movements when comparing with those of without and fixed compensation strategy. In upward and forward movements, significant difference between the RMSE values of the position-varying and fixed compensation strategies were found. As for NJS, the significant difference between the position-varying and without compensation strategies during upward and forward movements and the significant difference between the position-varying and



\**Indicated significant difference (P* < *0.05).*

without compensation strategies during upward movement were found. All the MVR values with fixed and with position-varying gravity compensation were larger than those without gravity compensation. Comparisons of MVR between each two of three gravity compensation strategies showed significant difference in three directions: upward, leftward, and rightward. The actual trajectories in four movements from a same subject were plotted in **Figure 3D**.

**Figure 4** represented EMG envelope of the six muscles for one subject during the movements tracking with three different compensation strategies. The results indicated that when the subject moved upward, leftward, and rightward with gravity compensation, BIC, DA, DM, and TRA were less activated while he moved rightward, BIC and DA were less activated. Two-way ANOVA showed that effect of compensation strategy on mean muscle activation of BIC, TRI, DA, DM, and TRA (P < 0.05) together with effect of direction on mean muscle activation of DA, DM, and TRA were significant (P < 0.05). No significant interaction between compensation strategy and movement direction was found. In **Figure 5**, mean muscle activation of per muscle during four direction movements with different gravity compensation strategies were displayed. Compared with movements without compensation, the mean activation of some muscles during movements with position-varying compensation

showed significant reductions [i.e., BIC and DA in all four directions, the mean activation of DM and TRA in three directions (forward, leftward, and rightward), and the mean activation of TRI in both upward and rightward]. The results revealed that the activation of BIC, TRI, DA, and DM with position-varying gravity compensation showed a significant decrease compared with fixed compensation during the upward movements.

# DISCUSSION

The purpose of this study was to compare three different gravity compensation strategies by a cable-based robotic system. Kinematics parameters in terms of RMSE, NJS, and MVR together with mean muscle activations were applied to evaluate their movement performance. RMSE can reflect the general tracking accuracy, and is determined by sensory perception, motor planning and execution (Seppänen et al., 2013). The lower RMSE values with position-varying gravity compensation reflected improved tracking accuracy, which was in agreement with previous studies (Grimm et al., 2016). NJS reflected movement smoothness and was generally utilized to evaluate the arm control abilities (Adamovich et al., 2009). The significant decrease in NJS with compensation than without compensation were also reported by Coscia et al. who found significant difference in normalized jerk between without and with gravity compensation during arm reaching movements (Coscia et al., 2014). MVR is utilized to quantify the relative velocity deviation in desired direction during movements tracking. A similar parameter, normalized angular velocity ratio, was proposed by Kim et al. for assessing locomotion precision (Kim et al., 2014). The higher MVR values with position-varying gravity compensation indicated that subjects could more voluntarily participate in the desired movements. The effects of direction on movement kinematics were significant which was in accordance with previous studies in sagittal (Papaxanthis et al., 1998), frontal (d'Avella et al., 2008), or horizontal plane (Beer et al., 2004). Both RMSE and MVR could describe movement accuracy from different points of view, and higher RMSE during upward and forward movements were corresponding to lower MVR. There explanation of the result might be that, the variation of gravity torques during upward and forward movement might be larger.

There were significant reductions in main contributed muscle activations with position-varying and with fixed gravity compensation when compared with without compensation. Previous studies also found that BIC, DA, and TRA were the mainly contributed muscles for gravity support of upper limb, and the activations of these muscles also decreased significantly during movements in horizontal plane (Sabatini, 2002; Prange et al., 2009) and frontal plane (Kloosterman et al., 2010; Coscia et al., 2014). In addition, McCrea et al. proposed that DM was recruited if the primary anti-gravity muscles were not capable during upper limb movements (McCrea et al., 2005). The findings of this study were in accordance with the above-mentioned studies. Although, upper limb muscles activations were dependent on direction (Hughes et al., 2009), activations of DP and TRI were not affected by direction in this study. It might be explained by that they were not the main contributed muscles indicated by in low activations in the four directions. As reported, subjects could maintain satisfy dynamic criteria during movements such as optimization of motor command (Nakano et al., 1999) or energy expenditure (Soechting et al., 1995) by appropriately activating arm muscles. The reduction of the mean muscle activation due to gravity compensation indicated that subjects could put more focus on target-directed movements, and result in a larger range of motion or more repetitions of movements. Previous studies also reported subjects had larger range of motion on the horizontal panel with gravity compensation (Housman et al., 2009; Wang and Dounskaia, 2012). Furthermore, muscle activation is related to muscle forces(Lloyd and Besier, 2003), therefore, lower activation of muscles might reflect the robot is able to share the gravity loading, and muscle forces mainly focus on desired movement execution.

This study found that position-varying gravity compensation provided by the cable-based rehabilitation robot could improve man-machine cooperation movements in 3D working space in terms of RMSE, NJS, and MVR, which was the first explorative investigation on the position-varying gravity compensation to our knowledge. Moreover, since the position-varying gravity compensation strategy could reduce activations of anti-gravity muscles, it could assist patients with muscle weakness to perform rehabilitation training in clinic. The advantages of the positionvarying compensation strategy are located on two aspects: firstly, the assistance is provided to counteract the effect of gravity; secondly, the assistance does not let the training in the passive way, and voluntary residual motor efforts can be focused on target-directed task. The reorganization in brains of subjects after stroke can be facilitated by active rehabilitation training (Cauraugh et al., 2000). In the future, more parameters should

#### REFERENCES


be adopted to clarify performance of the position-varying gravity compensation, and patients should be employed to explore the clinical effectiveness.

# CONCLUSION

The present study firstly explored the effect of position-varying gravity compensation strategy provided by a cable-based robot on kinematics and muscle activations in comparison with the other two gravity compensation strategies. The improvement in kinematics and less activated anti-gravity muscles with the position-varying gravity compensation indicated its potential in robot-aided rehabilitation therapy. More studies across pathologies with gender-matched subjects are needed to validate whether training with position-varying gravity compensation is clinically feasible and effective.

# AUTHOR CONTRIBUTIONS

YH and RS conceived and designed the study. YH and QY performed the experiments. YH and QY wrote the paper. RS reviewed and edited the manuscript. YH and QY contributed to the work equally and should be regarded as co-first authors. YC made a contribution to experiments. All authors had read and approved the manuscript.

## FUNDING

The project was supported by the National Natural Science foundation of China (Grant No. 61273359 and 91520201), the Guangdong Science and Technology Planning Project (Grant No. 2014B090901056 and 2015B020214003) and the Guangzhou Research Collaborative Innovation Projects (Grant No. 201604020108).

muscle synergies during reaching movements. J. Neuroeng. Rehabil. 11:22. doi: 10.1186/1743-0003-11-22


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2017 Huang, Yang, Chen and Song. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Development and Feasibility Assessment of a Rotational Orthosis for Walking with Arm Swing

Juan Fang1, 2 , Qing Xie<sup>3</sup> , Guo-Yuan Yang<sup>2</sup> and Le Xie2, 4 \*

*<sup>1</sup> Jiangsu Key Laboratory of Advanced Food Manufacturing, Equipment and Technology, Jiangnan University, Wuxi City, China, <sup>2</sup> The Joint Lab of the Institute of Rehabilitation Centre and Chejing Robotics Technology (Shanghai) Co., Ltd., Med-X Research Institute, Shanghai Jiao Tong University, Shanghai, China, <sup>3</sup> Department of Rehabilitation Medicine of Ruijin Hospital, School of Medicine, Shanghai Jiao Tong University, Shanghai, China, <sup>4</sup> School of Material Science and Engineering, Shanghai Jiao Tong University, Shanghai, China*

Interlimb neural coupling might underlie human bipedal locomotion, which is reflected in the fact that people swing their arms synchronously with leg movement in normal gait. Therefore, arm swing should be included in gait training to provide coordinated interlimb performance. The present study aimed to develop a Rotational Orthosis for Walking with Arm Swing (ROWAS), and evaluate its feasibility from the perspectives of implementation, acceptability and responsiveness. We developed the mechanical structures of the ROWAS system in SolidWorks, and implemented the concept in a prototype. Normal gait data were used as the reference performance of the shoulder, hip, knee and ankle joints of the prototype. The ROWAS prototype was tested for function assessment and further evaluated using five able-bodied subjects for user feedback. The ROWAS prototype produced coordinated performance in the upper and lower limbs, with joint profiles similar to those occurring in normal gait. The subjects reported a stronger feeling of walking with arm swing than without. The ROWAS system was deemed feasible according to the formal assessment criteria.

#### Edited by:

*Venketesh N. Dubey, Bournemouth University, UK*

#### Reviewed by:

*Brent Winslow, Design Interactive, USA Hans-Eckhardt Schaefer, University of Stuttgart, Germany*

> \*Correspondence: *Le Xie lexie@sjtu.edu.cn*

#### Specialty section:

*This article was submitted to Neural Technology, a section of the journal Frontiers in Neuroscience*

Received: *01 October 2016* Accepted: *16 January 2017* Published: *01 February 2017*

#### Citation:

*Fang J, Xie Q, Yang G-Y and Xie L (2017) Development and Feasibility Assessment of a Rotational Orthosis for Walking with Arm Swing. Front. Neurosci. 11:32. doi: 10.3389/fnins.2017.00032*

Keywords: interlimb neural coupling, arm swing, normal gait, coordinated movement, rehabilitation robotics

# INTRODUCTION

People swing their arms synchronously with leg movement during walking due to interlimb neural linkage, in addition to mechanical factors. Although the arms have no direct function for propulsion (Barbeau et al., 1987), people normally swing their arms so as to improve gait stability (Behrman and Harkema, 2000; Bovonsunthonchai et al., 2012) and energy efficiency (Dietz, 2002; Collins et al., 2009). Apart from such behavioral relevance, many phenomena imply that arm swing during walking is a neural-coordinated motor output. Rhythmic muscle activity was observed in the constrained arms during walking overground (Eke-Okoro et al., 1997), which implies the existence of neural coupling between the upper and lower limbs. Furthermore, adding mass to one ankle induced adaptive changes in both arms, in addition to changes in EMG from the leg muscles (Donker et al., 2002). This resulted in a coordinated movement pattern similar to that seen in unloaded normal gait. Studies of walking on a split-belt treadmill with different speed ratios between the legs resulted in coordinated locomotion in the legs and arms (Dietz et al., 2001). Short accelerations or decelerations randomly applied to the right leg during treadmill walking produced EMG response in the bilateral arm muscles, in addition to that in the right leg (Dietz et al., 2001). Interlimb neural interaction thus appears to be an underlying neural mechanism of human bipedal locomotion.

The theory of interlimb neural coupling brings new requirements for gait rehabilitation robotics. Interlimb modulation is active during walking, but not in standing or sitting (Dietz et al., 2001; Zehr et al., 2012). The neural interaction between the upper and lower limbs is maintained in patients with injury to central nervous system (Visintin and Barbeau, 1994; Stephenson et al., 2010). The task-specific practice strategy suggests gait restoration robotic systems should provide locomotion–like movements to improve gait control and functional ability (Harkema, 2001). Judging from the implication of interlimb neural coupling and the fact of arm swing during walking, it was suggested that gait training after neurological injury should incorporate simultaneous upper limb and lower limb rhythmic exercise to take advantage of neural coupling (Ferris et al., 2006).

In spite of the existence of many types of rehabilitation robots, there is no system which activates both the upper and lower limbs in the same way as during walking. Over the last few decades many types of rehabilitation systems have emerged, including systems for gait restoration (Díaz et al., 2011) or for upper limb rehabilitation (Lum et al., 2005). Several lowerlimb exoskeletons are commercially available to assist walking restoration, such as the Lokomat (Hidler et al., 2008) and the G-EO (Hesse et al., 2010) systems. They induce upright walking movement at variable speeds in the lower limbs. The arms often hold horizontal fixed bars to support the body. There are also several systems, such as the Armeo (Nef et al., 2006) and GENTLE/s (Loureiro et al., 2003), for those who have functional impairments in the upper limbs. The users often practice various arm movements in a sitting position. To the best of the authors' knowledge, there is no gait orthosis which incorporates arm swing.

Based on these limitations, a new rehabilitation system was to be developed in the present work. As the early initiation of gait rehabilitation is generally deemed important (Fang et al., 2011), the requirements of the proposed system included:


The aim of this work was to develop and evaluate the feasibility of a Rotational Orthosis for Walking with Arm Swing (ROWAS). The formal criteria for feasibility assessment were (Bowen et al., 2009): (i) implementation—was the system technically implementable? (ii) acceptability—was the system acceptable to the users? and (iii) responsiveness—was there a measurable movement that was close to the target joint trajectories?

## METHODS

The starting point for this work was a gait analysis experiment to determine the target joint trajectories for the ROWAS system. After the ROWAS was developed and implemented as a prototype, a feasibility assessment was conducted using the formal feasibility criteria of implementation, acceptability and responsiveness.

# Gait Analysis Experiment

In order to provide reference data for the design of a system actuating both the upper and lower limbs, a gait experiment was performed using a Vicon motion analysis system (Oxford Metrics Ltd., Oxford, UK) in Ruijin Hospital, Shanghai Institute of Orthopaedics and Traumatology, China. Ethical approval was obtained from the Ethics Committee at the Med-X Research Institute, Shanghai Jiao Tong University, Shanghai, China. Twenty-four able-bodied subjects were recruited and they provided written informed consent prior to participation.

The experimental set-up consisted of a ten-camera Vicon MX13 motion capture system (Vicon Peak, Oxford, UK). Thirtynine reflective markers of 14 mm diameter were affixed to specific anatomical landmarks (Plug-In Gait Marker Set, Vicon Peak, Oxford, UK) to record the segmental trajectories with a sampling frequency of 100 Hz (Kadaba et al., 1990). The details of this gait experiment can be found in our previous paper (Fang et al., 2016).

The angle trajectories of the shoulder, hip, knee and ankle joints within a gait cycle during overground walking were analyzed. The duration of one gait cycle was normalized to 100%, with heel strikes at 0 and 100%. The joint data were visually observed to remove outliers, and smoothed with the loess or rloess Matlab functions. The joint data from the overground walking experiment provided reference information of the joint profiles for the ROWAS system.

#### Development of the ROWAS System

Based on the design requirements, the ROWAS system was implemented as a rotational bed linking frames for the upper and lower limbs. To achieve the required functional features, modules for a retractable wheel-set, a ground-simulation plate and a foot platform were further developed. A 3D CAD design software tool (SolidWorks, Version 2016, Dassault Systèmes SolidWorks Corporation, Massachusetts, the USA) was used to develop the structure of the ROWAS system.

#### Rotational Bed Frame

In order to provide rehabilitation to users who are in an early post-injury phase, a rotational bed frame was developed, which allows practice in various positions. Depending on their physical situation, users can initiate walking training in a lying position (**Figure 1A**). Then as the rehabilitation progresses, the users can gradually be tilted up until in an upright position (**Figure 1B**).

We adopted a linear actuator MD60 from the Moteck Electric Corporation to rotate the bed up. As **Figure 2** shows, the linear actuator allows the bed plate to be: (i) in a lying position when it is retracted to the shortest length d; (ii) in an upright position when it is extended to its longest length p. The following equations

describe the device geometry:

$$d^2 = \left(a - c\right)^2 + b^2\tag{1}$$

$$p^2 = (b+c)^2 + a^2 \tag{2}$$

$$\theta = \arctan{\frac{b}{a-c}}\tag{3}$$

We assumed the original height of the bed was 400 mm, i.e., b = 400. To secure an acceptable efficiency of the actuator, the angle of the actuator and the horizontal level θ should be larger than 45◦ . For the chosen linear actuator noted above, stroke = 300 mm; d = 530 mm; p = 830 mm; thus meeting these requirements. Using Equations (1) and (2), a = 560.2 mm and c = 212.5 mm. In this case, the angle between the linear actuator and the horizontal level θ was about 50◦ , which satisfies the above requirement.

#### Retractable Wheel Support

For ease of translation, a wheel-set was employed at the bottom of the bed frame. However, during gait training, the system should be supported by a stable base. Thus, a retractable wheel support was designed with the requirements: (i) to support the system when the system was to be moved; (ii) to allow the system to be supported by the bed leg when the system is in use for rehabilitation training.

The wheel-set was attached by four parallel bars, which are lifted by a linear actuator. When the system is used for rehabilitation training, the wheel-set is retracted under the bed (**Figure 3A**) and the system is supported by the bed legs. When pushed by the linear actuator, the wheel-set was extended to the lowest position (**Figure 3B**), and the system is lifted up and supported by the wheels. The geometry for this linear actuator is similar to Equations (1)–(3). We also chose an MD21 from Moteck Electric Corporation, but with a stroke of 150 mm.

#### The Ground-Simulation Plate and Foot Platform

We designed a plate to support the leg during simulation of the stance phase. According to our systematic study on toe trajectories (Fang et al., 2016), it was found that the toe trajectory relative to the hip joint is curved. Therefore, the groundsimulation plate (**Figure 4**) was designed to have two ends taking a curved shape.

supporting state. (1) The bed leg, (2) the wheel-set, (3) the parallel bars, (4) the wheels, and (5) the linear actuator.

In order to enable the subject to walk on the groundsimulation plate, we further designed a shoe platform (**Figure 5**) to support the foot and simulate the ground reaction force. With six wheels at the bottom, the shoe can roll on the ground-simulation plate. Two air bags were fixed on the shoe sole to simulate the events of heel-strike and toe-off. The springs between the foot base and the wheels were to provide space between the shoe platform and the ground-simulation plate.

#### Joint Drives

For ease of control, we mounted the drives for the shoulder, knee, and ankle joints bilaterally at the sides (**Figure 6**). In order to get the upper limbs, especially the forearms, swing naturally as in normal walking, the hip joints require free space on each lateral side. Thus, we positioned the hip drives between the legs, as shown in **Figure 6B**. The drive for the right hip was mounted pointing to the joint center, while the drive for the left hip was mounted behind the left joint and the power was transmitted to the joint by a synchronous belt.

The rigs for the upper and lower limbs were designed adjustable in length to align the mechanical joints with the human joints. Size adjustment was implemented using a sliding nut that could be locked at any position on the frame by a clamp. The rigs for the lower limbs were covered with plates, which could support the leg during the transfer of the patient to the system.

#### The Whole ROWAS System

With a wheel set support at the bottom, the system finally included two leg rigs and two upper limbs rigs which were fixed on a rotational bed (**Figure 7**). Secured by the body-weight support system, the users lie upon the bed. After their arms and legs are fixed to the upper and lower limb frames, the bed is tilted to a position where the users feel comfortable. Then the user starts walking training with arm swing. If the users can be tilted to an upright position, the shoe platforms roll on the ground simulation plate. When the system needs to be moved, the wheel-set can be lowered.

The ROWAS prototype was manufactured as illustrated (**Figure 8**). Using strut profiles (Bosch Rexroth group), the bed frame was manufactured. The other structures were manufactured using aluminum alloy. The support frame and the joints of the ROWAS system had to be strong enough to transfer the necessary forces and torques to support a subject. To ensure this, the exoskeleton should be capable of developing the joint torques of a person of 135 kg

walking at various speeds in various postures. For supine and upright walking, the maximal loads were derived from data presented in the literature (Fang et al., 2014a). The motors we selected were Yaskawa AC motors with harmonic gearboxes. The drive components for the two hip joints were a combination of a 400W AC motor (SGM7J-04AFC6S+SGD7S-2R8A00A) and a gearbox with ratio of 160 (LCSG-25-160), while the drive assemblies for the shoulder, knee and ankle joints were a combination of a 100W AC motor (SGM7J-01AFC6S+SGD7S-R90A00A) and a gearbox with ratio of 120 (LCSG-17-120).

An eight-axis Googol motion controller (GTS-800-PV-VB-PCI, GoogolTech Limited) was adopted to control the joint motors. The joint performance data from the overground experiment was the target movement for motors. To obtain satisfactory movement, the time course of each joint angle was fit using Fourier series functions. The fitted reference movements were sent to the corresponding motors via C++ software. Then the PID control algorithm imbedded in the Googol motion controller was employed to move the ROWAS prototype according to the target joint profiles.

# Experimental Evaluation of the ROWAS Prototype

Experimental evaluation had two subtests: a test on the prototype to assess only the system function, and tests on able-bodied subjects to collect user feedback on the prototype. As a preliminary trial on this novel training modality, we recruited five able-bodied subjects to evaluate the performance (**Table 1**). Ethical approval was obtained from the Ethics Committee at the Med-X Research Institute, Shanghai Jiao Tong University, Shanghai, China. Subjects provided written informed consent prior to participation.

#### Experimental Procedure

During the experiment, the system produced walking at a cadence of 30 steps/min. In the preliminary test on the prototype only, the bed frame was rotated up at a speed of 1.5◦ /s to 75◦ relative to the ground. Then the system moved the frames for the upper and lower limbs. During the tests on the able-bodied subjects, we firstly collected their anthropometric data (**Table 1**). We then asked them to lie on the bed and secured them with the body-weight support system. After fixing their arms and lower

drive, (4) the knee drive, (5) the ankle drive.

FIGURE 8 | The ROWAS prototype. (A) The front view, (B) the back view.


TABLE 1 | Subject information.

limbs with Velcro straps, we rotated the bed to a position which was 75◦ relative to the ground. Finally we actuated the joint motors. The subjects tried to totally relax and follow the motion that the prototype produced. The test on each subject included two sessions: (i) the prototype actuated both the upper and lower limbs; (ii) it only moved the lower limbs, while the upper limb frames were left unactuated. Please see the Supplementary Video to see the performance of a representative subject during the test. The prototype produced walking for 45 s in each session. The joint performance was recorded via encoders. After the test, all the subjects were asked the questions below so as to collect their feedback:


#### Data Analysis

To compare the joint movements produced by the ROWAS prototype with those recorded during overground walking, three parameters were analyzed: difference of ROM (Rerror), phase shift (Terror) and root-mean-square error (RMSerror) for the angle performance (Fang et al., 2011, 2014b). The ROM of the hip joint from the overground walking experiment and the ROWAS test were ROH and RRH, respectively, while the onsets of maximal hip extension in the overground walking experiment and the ROWAS test within one gait cycle T were TOH and TRH. The angle amplitudes of the hip joint from the ROWAS and the overground walking were AOH and ARH. The differences of the ROM, phase and RMS regarding the hip joint (RerrorH, TerrorH and RMSerrorH) between overground walking and the ROWAS test were obtained from Eqs (4-6). A similar procedure was adopted to compare the ROM difference, phase shift and RMS difference of the shoulder (RerrorS, TerrorS, RMSerrorS), knee (RerrorK, TerrorK, RMSerrorK) and ankle (RerrorA, TerrorA, RMSerrorA) angle profiles from overground walking and the experiment. In order to obtain the phase shift of the shoulder and knee joints, the onsets of the maximal shoulder extension and knee flexion were analyzed. Thus,

$$R\_{errorH} = \frac{R\_{RH} - R\_{OH}}{R\_{OH}} \times 100\% \tag{4}$$

$$T\_{errorH} = \frac{T\_{RH} - T\_{OH}}{T} \times 100\% \tag{5}$$

$$RMS\_{errorH} = \frac{\sqrt{\frac{1}{N} \sum\_{i=1}^{N} (A\_{RH} - A\_{OH})^2}}{R\_{OH}} \times 100\% \tag{6}$$

#### Criteria for Feasibility Assessment

The criteria employed for assessment of the feasibility of the ROWAS system were (Bowen et al., 2009): (i) implementation—was the system technically implementable? (ii) acceptability—was the system acceptable to the users? and (iii) responsiveness—was there a measurable movement that was close to the target joint trajectories? The ROWAS was considered to have satisfied responsiveness criteria if Rerror, Terror, and RMSerror were less than 10%.

#### RESULTS

When the system was in a horizontal position, the dimension of the whole system was 2145 mm in length and 850 mm in width. The bed surface was 755 mm above the ground. It could be used by subjects with a height between 1.50 and 1.85 m. When the system was tilted up to be 90◦ relative to the ground (vertical), it was 2345 mm in height.

The target performance of the ROWAS system was taken from the gait experiment as described above. The joint angles of a representative subject with a height of 1.72 m were selected for illustration (the dotted lines in **Figure 9**). The fit data from the calculated Fourier functions served as the final reference values for the prototype (the dashed lines in **Figure 9**).

In the preliminary test on the prototype only, we applied a PID controller to actuate the joints. We observed that the parameters kp = 2 (proportional gain), ki = 0 (integral parameter) and kd = 0 (derivative parameter) provided acceptable results (the solid lines in **Figure 9**). During the test on able-bodied subjects, we applied the same PID parameters. The angle performance from the all subjects was close to that of preliminary tests on the prototype without users (dash-dot lines with shading in **Figure 9**). Compared to overground walking, the joint angle

FIGURE 9 | Joint performance. Lines with diamonds: measured experimental values; dashed lines: reference values; solid lines: the encoder readings from the prototype without users; dash-dot lines with shading: the mean ± SD performance from the prototype with users. (A) Hip angles, (B) knee angles, (C) ankle angles, (D) shoulder angles.

performance had a phase shift of about 3.5%, mean ROM errors of less than 8%, and RMS errors of less than 9% (**Table 2**).

Feedback obtained from the subjects showed that S2, S4, and S5 had no sense of insecurity. S1 and S3 reported a sense of insecurity when tilted up, but not during walking sessions. All users reported a stronger sense of walking when the arms were actuated compared to when they were not. S2 and S4 said that the medial sides of both knee joints were somewhat uncomfortable. S1, S2, and S4 reported that the arm movement was not comfortable. S1 thought the step length should be smaller.

## DISCUSSION

The aim of this work was to develop and evaluate the feasibility of the prototype ROWAS system. Feasibility assessment, as suggested by Bowen et al. (2009) and Saengsuwan et al. (2015), considered the formal criteria of implementation, acceptability and responsiveness.

The results demonstrated that the gait system with arm swing was technically implementable. Developed via computer modeling, ROWAS was implemented as a prototype. The main features were: (i) the system included a rotational bed which TABLE 2 | The difference of the ROM error (%), the phase shift (%) and RMS error (%) between the ROWAS and overground data of all five subjects.


allowed the potential user to try various positions from supine to upright walking; (ii) the main support area was flat, such as the trunk support plate and the support plate for the lower limbs, which facilitates easy transfer of the users from their wheelchair to the system; (iii) the shoe platform produced ground reaction forces on the foot sole; and (vi) synchronous movement between the upper and lower limbs. Inclusion of arm swing could be simply implemented by adding the upper limb frames with drives. The challenge was that the arm swing function restricted the mounting options for the hip joint actuators. For ease of control, it was desirable for the drives to directly actuate the joints. However, arm swing required free space on both sides, especially around the hip joint area. Therefore, we put the hip drives between the legs. Furthermore, the ROWAS system was easy to move due to a retractable wheel support. The ROWAS prototype met the design requirements and is an innovative system.

The ROWAS system was tested regarding the acceptability criterion by the able-bodied subjects. Currently available gait orthoses normally have the arms fixed using a horizontal level bar for balance (Lum et al., 2005; Díaz et al., 2011). Recent research on interlimb neural coupling suggests that the inclusion of arm swing is helpful for gait restoration (Ferris et al., 2006). Therefore, the novel ROWAS system was developed with the inclusion of arm swing, and we were interested in testing whether this new training modality was accepted or not. Lying on the bed, and further secured by the body-weight support system, the subjects in the preliminary tests did not report any sense of insecurity during the walking session. After experiencing passive walking with and without arm swing in ROWAS, all the subjects reported a stronger sense of walking when arm movement was activated.

Regarding the responsiveness criterion, the test measurements proved that the prototype produced accurate responses close to the target profiles, i.e., a walking-like coordinated movement among the shoulder, hip, knee and ankle joints of both sides. Our study was motivated due to the absence of a gait system with arm swing. Although there are tricycles which produce movements in both the upper and lower limbs (Hunt et al., 2012), users are in a sitting position, with the limbs moving in a circular path. Thus, the movement profiles are quite different from those occurring during normal gait. The measured joint trajectories from our system demonstrated that a synchronized movement was produced among the upper and lower limbs, with a maximal phase shift of about 3.5%. Compared to overground normal walking, the mean ROM error of less than 8% and a maximal RMS error of 9% demonstrated that a natural walking style was generated in ROWAS. It was thus demonstrated that the ROWAS prototype was feasible regarding the responsiveness feasibility criterion.

Our study and the ROWAS prototype have some limitations. As a feasibility study, we only recruited five subjects. More subjects are desirable to obtain a more detailed evaluation of the prototype. Nevertheless, these subjects provided useful feedback on how to improve the ROWAS prototype for general application. S2 and S4 reported pressure around the knee joint. This was because the knee joint of the frame was not properly aligned with their knee joint. To address this limitation, we plan to add one more adjusting mechanism for the knee joint alignment. Three out of five subjects reported that the arm movement felt uncomfortable. This was because the shoulder joint was actuated in the sagittal plane only. We therefore plan to use a spherical joint to obtain more degrees of freedom. The ROWAS prototype had a fixed hip joint height, which was 1250 mm above the ground when the bed was in an upright position. People with different heights were aligned with the system via the hip joint. Therefore, shorter subjects (S1 and S3) had a sense of being lifted too high and had a sense of insecurity when tilted up. This insecurity resolved after the subjects got used to the height and it did not disturb their walking session. To solve this problem, the bed base could be improved to have an adjustable height. Another limitation was the fixed ROMs of the upper and lower limbs. Because of this, shorter users such as S1 thought the step length was too large. A variable walking style should be implemented.

Future work should prepare a modified ROWAS system for clinical tests on patients from target populations with neurological impairments. The height of the bed base should be reduced so that the patients can be easily transferred to the system from their wheel chair. To provide clinical testing, we need to design a better body-weight support system which secures them, but without discomfort in the bottom area. Finally, the control algorithms need to be extended to allow compliant movement, and also to allow potential subjects, especially hemiplegic patients, to use their healthy side to control the impaired side. EMG responses should be investigated when patients use the system, as this will allow further investigation of interlimb neural coupling.

## CONCLUSIONS

In conclusion, we designed, manufactured and tested a ROWAS. During testing with five able-bodied subjects, the system could produce coordinated joint performance in the upper and lower limbs which was similar to normal overground gait. This system was deemed feasible, as far as the formal implementation, acceptability and responsiveness criteria were concerned.

# AUTHOR CONTRIBUTIONS

JF mainly developed this project, designed the system, manufactured and tested the system, analyzed the data and wrote the paper; QX helped develop the concept of this project and modified the manuscript; GY developed this project, supervised the experiments and analyzed the data and checked the paper; LX developed this project, helped to develop the design concepts, analyzed the data and checked the paper.

## ACKNOWLEDGMENTS

This work was supported by grants from the National Natural Science Foundation of China (JF: Grant No. 81401856, LX: Grant Nos. 61672341 and 61190124), Science and Technology Commission of Shanghai Municipality (LX: Grant No. 14441900800) and from the youth research funding scheme of Jiangnan University (JF: Grant No. JUSRP115A12). The authors acknowledge the staff of the Chejing Robotics Technology (Shanghai) Co., Ltd. for their support in manufacturing the prototype; and we particularly thank the mechanical research group for suggestions on the development of the ROWAS system.

## SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: http://journal.frontiersin.org/article/10.3389/fnins. 2017.00032/full#supplementary-material

# REFERENCES


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2017 Fang, Xie, Yang and Xie. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# EMG-Based Continuous and Simultaneous Estimation of Arm Kinematics in Able-Bodied Individuals and Stroke Survivors

Jie Liu<sup>1</sup> , Sang Hoon Kang<sup>2</sup> , Dali Xu<sup>1</sup> , Yupeng Ren<sup>1</sup> , Song Joo Lee<sup>3</sup> and Li-Qun Zhang4, 5 \*

<sup>1</sup> Sensory Motor Performance Program, Rehabilitation Institute of Chicago, Chicago, IL, United States, <sup>2</sup> School of Mechanical, Aerospace, and Nuclear Engineering, Ulsan National Institute of Science and Technology, Ulsan, South Korea, <sup>3</sup> Center for Bionics, Biomedical Research Institute, Korea Institute of Science and Technology, University of Science and Technology, Seoul, South Korea, <sup>4</sup> Department of Physical Therapy and Rehabilitation Science and Department of Orthopaedics, University of Maryland, Baltimore, MD, United States, <sup>5</sup> Department of Bioengineering, University of Maryland, College Park, MD, United States

Among the potential biological signals for human-machine interactions (brain, nerve, and muscle signals), electromyography (EMG) widely used in clinical setting can be obtained non-invasively as motor commands to control movements. The aim of this study was to develop a model for continuous and simultaneous decoding of multi-joint dynamic arm movements based on multi-channel surface EMG signals crossing the joints, leading to application of myoelectrically controlled exoskeleton robots for upper-limb rehabilitation. Twenty subjects were recruited for this study including 10 stroke subjects and 10 able-bodied subjects. The subjects performed free arm reaching movements in the horizontal plane with an exoskeleton robot. The shoulder, elbow and wrist movements and surface EMG signals from six muscles crossing the three joints were recorded. A non-linear autoregressive exogenous (NARX) model was developed to continuously decode the shoulder, elbow and wrist movements based solely on the EMG signals. The shoulder, elbow and wrist movements were decoded accurately based only on the EMG inputs in all the subjects, with the variance accounted for (VAF) > 98% for all three joints. The proposed approach is capable of simultaneously and continuously decoding multi-joint movements of the human arm by taking into account the non-linear mappings between the muscle EMGs and joint movements, which may provide less effortful control of robotic exoskeletons for rehabilitation training of individuals with neurological disorders and arm impairment.

Keywords: electromyogram (EMG), non-linear autoregressive exogenous model, continuous decoding, exoskeleton robot, computational neuroscience

# INTRODUCTION

Rehabilitation robots have been developing rapidly and used for therapeutic training of patients with neurological disorders, including stroke, cerebral palsy, and spinal cord injury (Dipietro et al., 2005; Krebs et al., 2008; Song et al., 2008; Marchal-Crespo and Reinkensmeyer, 2009; Pons, 2010; Frisoli et al., 2012; Heo et al., 2012; Reinkensmeyer and Boninger, 2012; Zariffa et al., 2012; Ren et al., 2013). Advances have been made to build more practical and functional upper-limb powered robotic exoskeleton devices (Nef et al., 2007; Perry et al., 2007; Gupta et al., 2008; Kim et al., 2012; Mao and Agrawal, 2012; Ren et al., 2013; Shao et al., 2014). The advances in powered

#### Edited by:

Ioan Opris, Leonard M. Miller School of Medicine, United States

#### Reviewed by:

Domenico Caputo, Sapienza Università di Roma, Italy Giovanni Mirabella, Sapienza Università di Roma, Italy

> \*Correspondence: Li-Qun Zhang zhangl@umd.edu

#### Specialty section:

This article was submitted to Neural Technology, a section of the journal Frontiers in Neuroscience

Received: 14 November 2016 Accepted: 14 August 2017 Published: 25 August 2017

#### Citation:

Liu J, Kang SH, Xu D, Ren Y, Lee SJ and Zhang L-Q (2017) EMG-Based Continuous and Simultaneous Estimation of Arm Kinematics in Able-Bodied Individuals and Stroke Survivors. Front. Neurosci. 11:480. doi: 10.3389/fnins.2017.00480

**73**

exoskeletons imply great promise to allow neurologically impaired patients to perform versatile activities, therefore helping restore strength and flexibility of their limbs. In contrast, relatively less attention has been given to providing less effortful control of exoskeleton robots. Previous studies have shown that motor commands are generated by the combination of a small number of muscle synergies, which allows the coordinated recruitment of groups of muscles with specific amplitude balances (Jiang et al., 2010; Overduin et al., 2015; d'Avella, 2016). Muscle synergies can be used to predict the movement of multiple degrees of freedom (DOFs; Ison and Artemiadis, 2014; Jiang et al., 2014a). Controlling a multiple DOFs robotic device requires sophisticated techniques for identification of various movements from recorded electromyography (EMG) signals (Fleischer and Hommel, 2008; Lo et al., 2010; Peerdeman et al., 2011; Scheme and Englehart, 2011; Fougner et al., 2012). A neural control interface is crucial to providing accurate, natural and less effortful control of powered exoskeletons (Kiguchi and Hayashi, 2012; Lenzi et al., 2012).

Among the potential biological signals for human-machine interaction (brain, nerve, and muscle signals), EMG, the ensemble electrical activity of a muscle may be the only experimentally non-invasive record of the motor commands to the muscles that enables routine clinical applications. EMG is generated by the neural activation from the brain and spinal cord and therefore contains substantial movementrelated information. It is worth noting that EMG signals do not necessarily reflect the overall computations carried on by the motor system. In fact, they are unlikely to catch neural signals related to a key executive function for shaping motor behavior, i.e., the ability of cancelling pending movements (Mirabella, 2014; Mirabella and Lebedev, 2017). However, in practice EMG signals could enable efficient control of robotic exoskeletons by extracting those motor commands that reach the muscles. Inhibitory control is fundamental for achieving a proper behavioral flexibility due to the fact events cannot be fully predicted practically. In many instances, preplanned actions must be aborted to avoid catastrophic consequences. Often suppression of a planned action occurs within the central nervous system, and thus the related neural activity does not reach the muscles. It is not by chance that brain–machine interfaces enacting inhibitory control have proposed to reproduce goaldirected behaviors in a more naturalistic way recently (Mirabella, 2012; Mirabella and Lebedev, 2017).

Use of EMG in decoding motor commands is one of the most robust and accurate interfaces for controlling robotic devices (Farina and Aszmann, 2014). As a non-invasive measurement containing rich motor control information, surface EMG is an important input for the control of powered robotic devices (Parker et al., 2006; Pons, 2010). As a result, surface EMG is increasingly recognized as one of the important control signals for assistive or rehabilitative devices in robot-aided therapy (Song et al., 2008; Hincapie and Kirsch, 2009; Marchal-Crespo and Reinkensmeyer, 2009; Jiang et al., 2010; Smith and Brown, 2011). Myoelectric control is a promising approach for controlling the multiple DOFs of multifunctional dexterous exoskeletons (Fleischer and Hommel, 2008). However, a major challenge in myoelectric control is to provide simultaneous and proportional control signals for robotic devices with multiple DOFs (Jiang et al., 2010; Fougner et al., 2012). To facilitate a less effortful myoelectric control paradigm, myoelectric controllers should provide proportional control of multiple DOFs simultaneously. This has been addressed in a few recent studies (Ameri et al., 2014a; Farmer et al., 2014; Fougner et al., 2014; Hahne et al., 2014; Ngeo et al., 2014). To provide simultaneous, independent and proportional control of multiple DOFs, various linear and non-linear estimators have been used, including artificial neural networks (Koike and Kawato, 1995; Cheron et al., 1996; Au and Kirsch, 2000; Shrirao et al., 2009; Pulliam et al., 2011; Jiang et al., 2012; Zhang et al., 2012; Ameri et al., 2014b; Farmer et al., 2014; Ngeo et al., 2014), regression techniques (Chen et al., 2013; Ameri et al., 2014a; Hahne et al., 2014), and state-space models (Artemiadis and Kyriakopoulos, 2010, 2011; Pan et al., 2014). Recent research has shown that continuous decoding plays increasingly important role in myoelectric control (Ameri et al., 2014a; Farmer et al., 2014; Fougner et al., 2014; Hahne et al., 2014; Ngeo et al., 2014).

Previous studies have shown that artificial neural networks, being a widely used supervised non-linear approach, outperformed linear regression (a supervised linear approach) and non-negative matrix factorization (a linear unsupervised method) in the EMG decoding paradigm (Hahne et al., 2014; Jiang et al., 2014b). It was possible to predict wrist joint angle instead of forces from EMGs with artificial neural networks when the subject was performing free dynamic movements (Jiang et al., 2012). In particular, a non-linear autoregressive exogenous (NARX) model was utilized to continuously map the kinematics of a transtibial prosthesis and EMG activity to estimate the prosthetic ankle angle in transtibial amputees in a recent study (Farmer et al., 2014). However, few investigators have been able to draw on any systematic research into decoding dynamic multi-joint arm movements. Most studies on EMG decoding have only focused on the estimation of kinematics of the leg and fingers. The purpose of this study was to develop a novel multi-input multi-output decoding method based on the NARX neural network model and predict dynamic multi-joint arm movements simultaneously based only on multichannel EMG inputs, which can potentially be used to achieve user-friendly, less effortful myoelectric control of robot-aided multi-joint movements.

#### METHODS

#### Subjects

Twenty subjects were recruited for this study including 10 stroke subjects (**Table 1**) and 10 with no neurological disorder (5 males and 5 females, age: 46.1 ± 14.6 year). All subjects gave written informed consent, which was approved by the Institutional Review Board of Northwestern University. A screening examination and clinical assessment to determine the eligibility for each stroke subject were performed by a physical therapist. Inclusion criteria for participation of the study include: (a) age between 21 and 65 years old; (b) experience of stroke,

TABLE 1 | Physical Characteristics of Subjects.


Duration: years since the onset of stroke. Paretic: the side of hemiparesis. F-M (UE): the Fugl-Meyer assessment scale of the paretic upper extremity (total score: 66). MSS: the motor status scale measures shoulder, elbow (maximum score = 40), wrist, hand, and finger movements (maximum score = 42).

initial onset > 6 months; (c) medically stable with clearance to participate.

#### Experimental Procedure

An upper limb exoskeleton robot, called IntelliArm and capable of controlling the shoulder, elbow and wrist individually or simultaneously in the horizontal plane (Ren et al., 2013), was used in the study to conduct supported free arm reaching and movement, as commonly used in stroke rehabilitation (Lo et al., 2010). The subject was seated with the upper arm, forearm, and hand strapped to the IntelliArm through the corresponding braces (**Figure 1**). The subject's shoulder horizontal adductionabduction, elbow flexion-extension and wrist flexion-extension axes were aligned with the corresponding mechanical axes of the IntelliArm robot (Ren et al., 2013). The IntelliArm robot was made back-drivable under the internal model based impedance control (IMBIC; Kang et al., 2009), thus the subject could move the arm voluntarily with little resistance from the robot. Six wireless pre-amplified, single differential surface EMG electrodes (Trigno, Delsys Inc; Boston, Massachusetts) were used to record EMG signals from the anterior deltoid, posterior deltoid, biceps brachii, long head of the triceps brachii, flexor carpi radialis, and extensor carpi radialis muscles. The subject was instructed to move their arm voluntarily in the horizontal plane with large concurrent movements across the shoulder, elbow, and wrist joints at the subject's self-selected comfortable speeds (<70 degrees/second), as the speed and movement duration are more likely to vary in real-world settings. The motors used to control the motion of the robot in other planes were locked in this study. The movement range of each of the joints was about 90◦ . Relative long single trial instead of a few short time trials were used for data recording when movements are freely repeated several times in order to capture both the commonality and the variability of the EMG and motion properties. Thus, the subject performed a single movement trial over a span of about 3 min in this study. Previous studies have used a similar approach for modeling functional relationships between EMG and limb kinematics (Cheron et al., 1996, 2003). The reliability of an experimental study could be affected by several factors such as motivation or boredom. For example, during series of trials, the second one is likely better than the first because subjects intend to improve their performance or because they may benefit from the learning experience of the first one. By contrast, a decreased performance between the first trial and the following trial could possibly be explained by lack of motivation or fatigue (Hopkins, 2000).

#### Data Collection and Signal Processing

The surface EMG signals were sampled at 1,000 Hz by using the Trigno EMG system, rectified, and then low-pass filtered with a six-order Butterworth filter (cutoff frequency = 4 Hz) to obtain the linear envelope (LE). Multi-joint position signals from the IntelliArm robot were sampled at 1,000 Hz and synchronized to the EMG data through a trigger signal sent to the aforementioned IntelliArm and EMG data acquisition systems simultaneously. The trigger was implemented as a push button switch. The resultant EMG LE and position signals were then down sampled to 20 Hz for subsequent data analysis. **Figure 2** illustrates the representative EMG and kinematic data recorded during a typical experimental trial.

#### NARX Model

The multi-joint dynamic relationship between the six arm muscles and the shoulder, elbow and wrist movements was characterized by an AutoRegressive with eXogeneous inputs (NARX) model for discrete-time non-linear systems (Leontaritis and Billings, 1985; Billings, 2013) and is represented as

$$\begin{split} \boldsymbol{y}(t) &= \boldsymbol{f}\left[\boldsymbol{y}\left(t-1\right), \dots, \boldsymbol{y}\left(t-n\_{\mathcal{V}}\right), \boldsymbol{x}\left(t-1\right), \dots, \boldsymbol{x}\left(t-n\_{\mathcal{X}}\right)\right] \\ &+ \; \boldsymbol{e}(t)\left(t\right)\left(t-1\right)t - n\boldsymbol{x} \end{split} \tag{1}$$

where x(t) and y(t) are the input and output of the model at discrete time step t, respectively. While the input order n\_x ≥ 1, the output order n\_y ≥ 1, and n\_x ≤ n\_y (Leontaritis and Billings, 1985; Billings, 2013). f(·) is a non-linear function, which can be approximated by a standard multilayer perceptron (MLP) neural network. The resulting connectionist architecture is called a NARX recurrent neural network, which has been used in modeling non-linear dynamic systems (Chen et al., 1990; Lin et al., 1996, 1997; Menezes and Barreto, 2008; Farmer et al., 2014).

Assuming that e(t) has zero mean and finite variance, the minimized mean-square error (MSE) predictor for the NARX model is approximated by

$$
\hat{\boldsymbol{y}}(t) = \boldsymbol{y}(t) - \boldsymbol{e}\left(t\right) = \boldsymbol{f}\left[\boldsymbol{y}\left(t-1\right), \dots, \boldsymbol{y}\left(t-n\_{\mathcal{V}}\right), \boldsymbol{x}\left(t-1\right), \dots, \boldsymbol{x}\left(t-n\_{\mathcal{V}}\right), \dots, \boldsymbol{x}\left(t-n\_{\mathcal{V}}\right)\right] \tag{2}
$$

$$
\dots, \boldsymbol{x}\left(t-n\_{\mathcal{X}}\right)] \tag{2}
$$

The NARX model has a delay structure, creating embedded memory within the neural network. The MSE one-step predictor is a non-linear function of a finite number of past outputs and past inputs.

In contrast to other recurrent networks, NARX neural networks have feedback from the outputs rather than from the hidden states (Siegelmann et al., 1997). As a result, NARX neural networks are able to learn more effectively, converge faster and exhibit better generalizations (Lin et al., 1996, 1997). In addition, the NARX neural networks are computationally as powerful as Turing machines (Siegelmann and Sontag, 1991, 1995), which means that theoretically any problem can be modeled and solved using a NARX neural network. Thus, we used a NARX neural network to predict dynamic arm movements by modeling the non-linear dynamic relationship between the multiple muscle activities and arm kinematics.

#### NARX Neural Network Structure

The NARX network model used in this study was developed by using the Neural Network Toolbox in Matlab (Mathworks, Natick, MA), which consisted of an input layer containing the sixchannel EMG LE signals recorded from the arm, a hidden layer containing three neurons, and a linear output layer containing three outputs corresponding to the shoulder, elbow, and wrist angular positions (**Figure 3**).

To continuously decode the multi-joint positions simultaneously from the EMG LE signals, the following NARX network model was used:

$$\begin{aligned} U\_n(t) &= f\_1 \left( \sum\_{i=1}^{N\_\mathbf{x}} \sum\_{k=0}^{n\_\mathbf{x}} A\_{ni} \begin{pmatrix} k \end{pmatrix} X\_i \begin{pmatrix} t - m - k \end{pmatrix} \\ &- \sum\_{j=1}^{N\_\mathbf{y}} \sum\_{k=0}^{n\_\mathbf{y}} B\_{\mathbf{n}j} \begin{pmatrix} k \end{pmatrix} Y\_j \begin{pmatrix} t - m - k \end{pmatrix} + b\_{\mathbf{1}n} \right), \\ n &= 1, 2, 3 \end{aligned} \tag{3}$$

$$Y\_j(t) = f\_2\left(\sum\_{n=1}^3 C\_{\text{jn}}(t)\,U\_n(t) + b\_{2\text{j}}\right), \quad j = 1, 2, 3 \tag{4}$$

where U<sup>n</sup> (t) represents output of one of the non-linear nodes of the hidden layer, Y<sup>j</sup> (t) is the shoulder, elbow or wrist angular position predicted by the NARX network model at time t; N<sup>x</sup> (=6) is the number of EMG LE signals; n<sup>x</sup> (=2) is the number of past input time lags used in the NARX model; N<sup>y</sup> (= 3) is the number of joint angles, n<sup>y</sup> (=2) is the number of past output time lags used in the NARX model; m (=2) is the number of time lags, which specifies the embedding delay (τ = m1t) and correspond to the prediction interval; X<sup>i</sup> t − m − k is the ith EMG LE at time lag m + k; Y<sup>j</sup> t − m − k is a joint angle predicted by the NARX model at time lag m + k; Ani, Bnj and Cjn are the weight vectors. b1n and b2j are the bias weights. f<sup>1</sup> and f<sup>2</sup> are the sigmoid function and linear function, respectively.

The NARX neural network model has 51 parameters in total, which are estimated in training the neural network offline by using a supervised learning procedure to minimize the errors

between the NARX network model outputs and experimentally measured angles.

#### Performance Measure

For each subject, a five-fold cross-validation procedure was used to evaluate the performance. The EMG and arm joint kinematic data were divided into five segments with equal length. Each time, four out of five segments of data were used as the training set and the remaining segment was used as the validation set (training: 80%, validation: 20%). The procedure was repeated with each of five segments used as the validation set once. To evaluate the performance of the trained decoder, the variance accounted for (VAF) was calculated between the measured and decoded joint angles based on the fresh data (Kearney and Hunter, 1990; Fagg et al., 2009).

One-way ANOVA with repeated measures was utilized to compare the decoding performance across the multiple joints. The group factor had three levels, corresponding to the shoulder, elbow, and wrist joints. The null hypothesis was that there was no difference between the decoding performance of the shoulder, elbow and wrist joints. Multiple comparisons with Bonferroni correction were conducted to compare pairwise differences between the joints.

# EXPERIMENTAL RESULTS

## Decoding Performance and the Effect of Prediction Interval

The effect of the length of prediction interval on the decoding performance was evaluated as suggested by previous research (Farmer et al., 2014). The relationship between decoding accuracy and the prediction interval was plotted for the shoulder, elbow, and wrist joint angles predicted based on only six muscle EMG signals, which shows that the decoding performance varies with the length of prediction interval (**Figure 4**). An increase of window length from 50 to 100

ms led to consistent improvement in decoding performance across the joints, as indicated by the higher VAF. A further increase of the prediction interval to 300 ms was accompanied with a slight decrease in the decoding performance. For each prediction interval, no significant differences were observed between different joints [One-way repeated measures ANOVA, F(2, 15) = 0.16, P = 0.85, partial eta squared η<sup>p</sup> <sup>2</sup> = 0.02]. Averaged across all the able-bodied subjects, the prediction interval of 100 ms yielded the best performance with the highest average VAF, which was used in the subsequent analysis for decoding.

#### Position Decoding Results

The offline decoding performance of NARX neural network was assessed in terms of its performance to decode the shoulder, elbow and wrist joint positions during active movements using only the six muscle EMG LEs as inputs. High performance in position decoding was achieved for all subjects by using the NARX model, which has a hidden layer containing three neurons, and a linear output layer containing the shoulder, elbow and wrist angle outputs with the prediction interval of 100 ms. Joint angle predictions made using the NARX neural network model based on only the multiple EMG inputs matched very closely with the measured shoulder, elbow, and wrist positions (**Figure 5**).

Very close matches between the decoded and measured joint angles were achieved for all 10 able-bodied subjects, as shown in **Figure 6**. The ANOVA showed a significant difference among the decoding performances for the shoulder, elbow and wrist joints [One-way ANOVA, F(2, 18) = 5.61, P = 0.0092, partial eta squared ηp <sup>2</sup> = 0.29]. The decoding accuracy for the wrist was lower than that achieved for the shoulder and elbow (Bonferroni test, P < 0.05).

Similarly, we have further tested the feasibility of the NARX decoder in 10 stroke subjects showing that the decoded and measured joint angles were close for all 10 stroke subjects who had various levels of functional impairment (Motor status scale 14.67–66.02, Fugl-Meyer score 16–50), as shown in **Figure 7**. The ANOVA showed a significant difference among the decoding performances for the shoulder, elbow and wrist joints [Oneway ANOVA, F(2, 18) = 7.31, P = 0.0029, partial eta squared ηp <sup>2</sup> = 0.35]. The decoding accuracy for the wrist was lower than that achieved for the shoulder and elbow (Bonferroni test, P < 0.05).

# DISCUSSION AND CONCLUSION

The study investigated decoding multi-joint dynamic arm movements based solely on surface EMG signals from muscles crossing the joints by extracting those motor commands that reach the muscles. This study contributes to rapidly growing research on myoelectric control of limb movements by demonstrating continuous decoding of arm kinematics using surface EMG signals based on the NARX model. In the current study, taking into account the non-linear dynamical relationship between the muscle EMGs and arm movements, we successfully estimated shoulder, elbow and wrist joint movements using only the six EMG signals by adopting time delays and feedback stream into an artificial neural network. These results are consistent with recent studies (Jiang et al., 2012; Farmer et al., 2014; Hahne et al., 2014; Ngeo et al., 2014) in which proportional and simultaneous estimation of limb kinematics was achieved toward myoelectric control, and suggest the use of EMG as a robust and accurate interface for robot control (Farina and Aszmann, 2014). These findings further support the approach to decoding dynamic limb movements using surface EMG signals by taking advantage of artificial neural networks.

Motivated by the fact that NARX neural networks have a recurrent dynamic nature and are computationally equivalent to Turing machines (Siegelmann and Sontag, 1991, 1995), we assessed the NARX networks for continuous decoding of multi-joint dynamic arm movements using surface EMG signals. The performance of the decoder was evaluated by offline analysis using EMG signals from the six muscles and kinematic data measured by the robot during dynamic multijoint arm movements in the horizontal plane. The decoding accuracies in terms of VAF, were above 98% for the shoulder, elbow and the wrist joints possibly due to the fact that the

noise of EMG recordings was suppressed by low-pass filtering the EMG signals with a six-order Butterworth filter (cutoff frequency = 4 Hz). The decoding performance of the wrist was slightly inferior to that obtained for the shoulder and elbow, possibly because the kinematic data collected from the wrist joint were noisy due to the relatively large noise caused by

fresh validation data (50 s long) were 0.9999, 0.999, and 0.9977, respectively.

the Harmonic Drive gear compared to the torque generated by subjects during the voluntary wrist movement under impedance control. This finding is in agreement with that reported by Farmer et al. who showed a NARX model could continuously estimate the prosthetic ankle angle in transtibial amputees using the EMGs recorded from the residual lower limb (Farmer et al., 2014). Furthermore, the method is able to continuously decode movements of multiple joint simultaneously instead of a single joint movement.

Previous studies with able-bodied individuals showed promising results, which need to be interpreted with caution, as the findings might not be directly transferable to neurologically impaired patients, because voluntary surface EMG signals might be contaminated by spontaneous motor activity in studies involving neurological impairments. For instance, when recording EMG signals from paretic muscles of stroke or spinal cord injury patients, abnormal hyper-excitable motor unit discharges may induce spontaneous spurious spikes, compromising the voluntary EMG signals (McKay et al., 2011). Previous studies with incomplete spinal cord injury have shown that the resulting involuntary background spikes can contaminate surface EMG recordings thus result in inappropriate interpretation of the signals (Liu et al., 2014a,b). Therefore, we have further tested the feasibility of the NARX decoder in 10 stroke subjects to evaluate the current method on neurologically impaired patients, we observed that the high decoding accuracy (VAF > 98% for all three joints) was consistent across different stroke subjects who had various levels of functional impairment (Motor status scale 14.67–66.02, Fugl-Meyer score 16–50). It was observed that the stroke subjects have more variable decoding results compared to the healthy subjects with respect to the wrist joint angle, In contrast, the healthy subjects have more variable decoding results for the shoulder and elbow joints, This result may be explained by the fact that the healthy subjects have bigger motion ranges, high speeds and velocity changes. These stroke subjects might have different clinical features, because they are very likely of having undergone very different degree of brain plasticity. We observed that nothing would change by removing patients with more than 10 years since the onset of stroke. However, with a small sample size, caution must be applied, as the findings might not be extrapolated to all patients with different clinical features.

Note that unlike the time delay neural network (TDNN) approach for predicting joint-angle trajectories based on EMG signals during arm movements (Au and Kirsch, 2000; Pulliam et al., 2011; Smith and Brown, 2011), the NARX network makes full use of the model output feedback and has powerful modeling, learning and generalization capacities (Chen et al., 1990; Lin et al., 1996, 1997; Menezes and Barreto, 2008; Farmer et al., 2014). The presence of direct feedback from the output gives strong decoding power to the NARX model. More specifically, the NARX model makes use of the AR model's implicit characterization of the limb kinematics vis-à-vis the feedback. In contrast, the absence of such a feedback in the TDNN model makes it less powerful in decoding. Thus, the NARX network outperforms conventional neural network based decoders, such as the TDNN network (Menezes and Barreto, 2008) qualitatively. Despite this, we acknowledge that future systematic comparison studies on different decoding methods are needed.

EMG signals have been used to control joint angular velocity indirectly by commanding the stiffness equilibrium angle (Ha et al., 2011). However, from a practical point of view, models characterizing multiple muscles controlling human joints are rather complex even for a single joint due to the complex nature of neuromusculoskeletal systems, and usually they require exhaustive, time-consuming offline analysis of neuromusculoskeletal system identification problem (Koo and Mak, 2005). Thus, using such models for real-time decoding is difficult and their real-time application remains to be demonstrated, especially for applications to neurologically impaired cases.

Despite these promising results, questions remain. A limitation is that the high accuracy may be based on the fact that the subjects was training in performing a simple and repetitive arm movement and the arm movements were restricted to a horizontal plane with arm-weight-supported, as opposed to 3-D arm movement. Horizontal plane arm movements have been widely studied and commonly used in rehabilitation of patients post stroke (Hollerbach and Flash, 1982; Lo et al., 2010). It could be argued that the positive results were due to the repeated

#### REFERENCES

Ameri, A., Kamavuako, E., Scheme, E., Englehart, K., and Parker, P. (2014a). Support vector regression for improved realtime, simultaneous myoelectric control. IEEE Trans. Neural Syst. Rehabil. Eng. 22, 1198–1209. doi: 10.1109/TNSRE.2014.23 23576

simplified movements in 2-D space. Still, further research should be done to investigate different arm movements in 3-D space without arm-weight-support. Furthermore, although the IntelliArm robot was controlled to be backdrivable, it might still induce some external loading to the subject's arm during the multi-joint arm movements, which might affect the EMG signals and the myoelectric control behavior of human arm movement. Everyday tasks involve behaviors uncertainty originating from a large variety of sources, which in some circumstances might require to suppress preplanned actions (Mirabella, 2014). However, in this study the presence of behaviors uncertainty was very limited if not completely absent. Therefore, it was not possible to examine the EMG signal variability in the presence of unpredictable changes. This is an important issue for future research.

In summary, the proposed approach is capable of continuously decoding multiple joint movements of the human arm simultaneously by taking into account the non-linear mappings between the multi-joint arm kinematics and multiple muscle EMGs, which may help improve myoelectric control of robotic exoskeletons and robot-guided arm rehabilitation in neurological disorders.

## ETHICS STATEMENT

Twenty subjects were recruited for this study including ten stroke subjects (**Table 1**) and ten with no neurological disorder (5 males and 5 females, age: 46.1 ± 14.6 year). This study was carried out in accordance with the recommendations of guidelines from the Collaborative Institutional Training Initiative (TICI) program, and the Institutional Review Board of Northwestern University. All subjects gave written informed consent, which was approved by the Institutional Review Board of Northwestern University.

## AUTHOR CONTRIBUTIONS

Conceived and designed the experiments: JL and LZ. Performed the experiments and data acquisitions: JL, SK, SL, DX, and YR. Analyzed and the data interpretation: JL, DX, YR, and LZ. Contributed reagents/materials/analysis tools: JL, SK, DX, YR, and SL. Wrote the paper: JL and LZ. Final approval of the version to be published: JL and LZ.

#### FUNDING

This work was supported in part by the National Institute on Disability, Independent Living and Rehabilitation Research, National Science Foundation, and DGIST.


**Conflict of Interest Statement:** LZ and YR hold equity positions in Rehabtek LLC, which is involved in developing the rehabilitation robot used in this study.

The other authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2017 Liu, Kang, Xu, Ren, Lee and Zhang. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Simultaneous and Continuous Estimation of Shoulder and Elbow Kinematics from Surface EMG Signals

#### Qin Zhang\*, Runfeng Liu, Wenbin Chen\* and Caihua Xiong\*

*The State Key Laboratory of Digital Manufacturing Equipment and Technology, Institute of Rehabilitation and Medical Robotics, Huazhong University of Science and Technology, Wuhan, China*

#### Edited by:

*Dingguo Zhang, Shanghai Jiao Tong University, China*

#### Reviewed by:

*Lizhi Pan, North Carolina State University, United States Zhong Yin, University of Shanghai for Science and Technology, China Zhaojie Ju, University of Portsmouth, United Kingdom*

#### \*Correspondence:

*Qin Zhang qin.zhang@hust.edu.cn Wenbin Chen wbchen@hust.edu.cn Caihua Xiong chxiong@hust.edu.cn*

#### Specialty section:

*This article was submitted to Neural Technology, a section of the journal Frontiers in Neuroscience*

Received: *19 January 2017* Accepted: *01 May 2017* Published: *30 May 2017*

#### Citation:

*Zhang Q, Liu R, Chen W and Xiong C (2017) Simultaneous and Continuous Estimation of Shoulder and Elbow Kinematics from Surface EMG Signals. Front. Neurosci. 11:280. doi: 10.3389/fnins.2017.00280* In this paper, we present a simultaneous and continuous kinematics estimation method for multiple DoFs across shoulder and elbow joint. Although simultaneous and continuous kinematics estimation from surface electromyography (EMG) is a feasible way to achieve natural and intuitive human-machine interaction, few works investigated multi-DoF estimation across the significant joints of upper limb, shoulder and elbow joints. This paper evaluates the feasibility to estimate 4-DoF kinematics at shoulder and elbow during coordinated arm movements. Considering the potential applications of this method in exoskeleton, prosthetics and other arm rehabilitation techniques, the estimation performance is presented with different muscle activity decomposition and learning strategies. Principle component analysis (PCA) and independent component analysis (ICA) are respectively employed for EMG mode decomposition with artificial neural network (ANN) for learning the electromechanical association. Four joint angles across shoulder and elbow are simultaneously and continuously estimated from EMG in four coordinated arm movements. By using ICA (PCA) and single ANN, the average estimation accuracy 91.12% (90.23%) is obtained in 70-s intra-cross validation and 87.00% (86.30%) is obtained in 2-min inter-cross validation. This result suggests it is feasible and effective to use ICA (PCA) with single ANN for multi-joint kinematics estimation in variant application conditions.

Keywords: principle component analysis (PCA), independent component analysis (ICA), artificial neural network (ANN), simultaneous and continuous motion estimation, myoelectric control

# 1. INTRODUCTION

Surface Electromyography (EMG) signal represents ample electric information of muscle contractions innervated by neural signals. It has been used as a viable and effective neural interface in interpreting human's movement intent into control command especially for various robotic systems (Farina et al., 2014; Maciejasz et al., 2014). Traditionally, the predefined motion patterns are classified from EMG features through the trained motion classifier (Kuiken et al., 2009; Chen et al., 2013; Ju and Liu, 2014). As long as a predefined motion pattern is selected, the robot will be driven along a predefined trajectory although the user would like change their movement intent. For multi-DoF coordinated motions, motion patterns of each DoF are respectively classified and sequentially employed to drive the robot to achieve the intended coordinated movement. Such motion decoding

**84**

style based on pattern-classification technology is different from the natural neuromuscular control strategy where the human's intent (represented by EMG) is consecutively mapped to the human's motion (Farina et al., 2014). Therefore, simultaneous and continuous motion estimation has been proposed as an alternative manner to achieve natural and intuitive motor control.

Comparing with the motion pattern classification, the simultaneous and continuous motion estimation provides concurrent motion information (such as joint angle, muscle force) rather than motion trigger command together with the EMG recording. With simultaneous and continuous motion intent decoding method, the estimates of the motion information can be simultaneously obtained which is more efficient especially for coordinated motion estimation with multiple DoFs involved. This procedure continuously proceeds with the persisted recording and interpreting of the EMG signals. Although the continuous motion estimation of one or two DoFs at one joint (such as elbow Zhang et al., 2013; Han et al., 2015, wrist and/or hand Jiang et al., 2012; Muceli and Farina, 2012 or finger Ngeo et al., 2014; Pan et al., 2014) has drawn a lot of attention, there is few research on continuous estimation of multiple DoFs across shoulder and elbow joint (Au and Kirsch, 2000; Artemiadis and Kyriakopoulos, 2010, 2011). Thus, the prosthetics (Miller et al., 2008), exoskeletons (Kiguchi et al., 2008; Kiguchi and Hayashi, 2012) and other assistive devices for upper limb rehabilitation (Vogel et al., 2013; Jang et al., 2016) cannot benefit from the continuous motion estimation technology. This paper investigates the feasibility of the simultaneous and continuous estimation of 4-DoF kinematics across shoulder and elbow from EMG signals during coordinated arm movements.

Due to the nonlinearity of the mapping between the EMG and the human motions as well as the non-stationarity of the EMG signals over time, the performance of myoelectrical control heavily depends on the accuracy and robustness of the human motion estimation. To achieve this purpose, various research efforts were placed on exploring the correlation between EMG and kinematics or dynamics during different physiological or application conditions as well as developing feasible and effective EMG feature extraction and estimation methods to obtain desired kinematic and/or dynamic information. However, due to the structural and functional heterogeneity of muscles and the inherent stochastic nature of EMG signals, methodologies of motion estimation from EMG have been researched with various kinematic or dynamic constraints (Staudenmann et al., 2010; Zhang et al., 2011, 2013; Clancy et al., 2012; Li et al., 2014). Most estimation methods are developed based on various machine learning algorithms, such as linear-regression (LR), non-negative matrix factorization (NMF) and artificial neural network (ANN). In the context of multi-DoF estimation, a time-delayed artificial neural network was applied to predict the shoulder and elbow joint angles with average root-mean-square (RMS) errors of 20◦ on able-bodied subjects during 30-s arm movements (Au and Kirsch, 2000). By using principle component analysis (PCA) for both the EMG and motion data decomposition, the shoulder and elbow motions were estimated in the low-dimensional space by a state-space decoding model (Artemiadis and Kyriakopoulos, 2010, 2011). Although the estimation accuracy was not explicitly reported, it was sufficient for the tele-operation of a robotic arm in the 3-D Cartesian space (Artemiadis and Kyriakopoulos, 2010). However, the non-stationarity characteristics of the EMG signals were still troublesome (Artemiadis, 2012), so that the decoding model had to be switched in the course of motion completion according to the muscle contraction states of each muscle involved in order to compensate the motion estimation deviation over time in 3-min arm movements (Artemiadis and Kyriakopoulos, 2011).

Apart from the nonlinearity and non-stationarity, subjectspecific and crosstalk problems have always companied with the application of EMG whose characteristics apt to be influenced by both physiological and nonphysiological factors during dynamic movements (Beck et al., 2006; Artemiadis, 2012). Thus, it was difficult to find consistent patterns among subjects regarding the motor control strategies (Beck et al., 2006). Even for single subject, since the concentric and eccentric muscle contractions adopted different motor control strategies, the joint angle had different effects on related EMG signals (Qi et al., 2011). Therefore, when shoulder and elbow coordinate to generate large-range arm motions, the EMG-based simultaneous and continuous estimation of the involved joint kinematics is complicated. This paper attempts to simultaneously and continuously estimate three shoulder angles and one elbow angle without switching the motion estimation model, which is validated with up to 2-min arm motions on six subjects.

This work proposes an alternative method to simultaneously and continuously estimate the joint angles from EMG signals during multi-joint coordinated arm movements. With respect to the previous works, EMG-driven estimation of 4 DoFs across 2 joints (shoulder and elbow) is investigated in this work during coordinated arm movements which are involved in daily activities. The estimation performance was evaluated by multiple motion repetitions up to 2 min in order to evaluate the robustness of the proposed methods to time-variant characteristics of EMG. During all the estimation processes, only one estimation model is learned and used for motion estimation for single subject without the model switching during the course of motion completion.

# 2. METHODS

The coordinated upper limb movements, such as reaching, pointing, exercising and manipulating, play important roles in human daily life. Although the three main joints of upper limb, shoulder, elbow and wrist, represent distinct kinematic and dynamic properties, their coordination determines the completion of human intended movement. Correspondingly, each muscle activating these joints has different spatial and temporal pattern. Wherein, the shoulder and elbow joints confer the wide range of motion for upper limb, so the coordination of the 3 DoFs at shoulder (flexion/extension, abduction/adduction and pronation/supination) and 1 DoF at elbow (flexion/extension) were investigated in this work with the wrist joint being excluded for simplification as did in Artemiadis and Kyriakopoulos (2010) and Artemiadis and Kyriakopoulos (2011). In our opinion, it is more meaningful to estimate shoulder and elbow DoFs simultaneously and continuously as there are more variabilities during a wide range of their motion comparing with wrist. The pattern recognition-based humanmachine interface (HMI) cannot represent such instantaneous variations except static and discrete motion pattern. This study was approved by the local ethical committee of Huazhong University of Science and Technology, and all subjects signed informed consent forms.

#### 2.1. Experiment Setup

Six able-bodied subjects (all male, 23 ± 1 year-old, 62 ± 4.5 kg) participated in this study. All of them are right-handed dominant and have no reported neuromuscular disorders of their upper limb. They were ever instructed to practise the intended movements until they feel comfortable to the experimental setup.

#### 2.1.1. EMG Recordings

Surface EMG signals were acquired by a commercial EMG acquisition system (Me6000, Mega Electronics Ltd, Kuopio, Finland). The configuration of the EMG recording is shown in **Figure 1**. Eight predominant muscles activating the four shoulder and elbow DoFs were selected to be the muscle of test, that is, biceps, triceps, deltoid (anterior), pectoralis major (clavicular head), deltoid (middle), deltoid (posterior), trapezius, and teres major muscles. Eight channel of bipolar differential amplifier were carefully placed on these muscles according to both the anatomy and hand touch experience. The active EMG electrodes of each channel were positioned at the muscle belly along the muscle fiber direction with the reference electrode orthogonal to the midline of the active electrodes according to the recommendation of Me6000. The skin underneath the electrodes was cleaned with alcohol patch to reduce the resistance between

FIGURE 1 | Illustration of the experiment setup. The configuration of the EMG electrodes and reflective marker clusters is shown on the left plot with the bony landmarks being removed. Correspondingly, the motion recording marker clusters and EMG traces are respectively shown on the top right and bottom right. (The subject in the present image provided written informed consent for the publication of this image.)

the skin and the electrodes. The EMG signals were amplified (gain 305) and sampled at 1 KHz.

#### 2.1.2. Motion Recordings

A 6-camera motion capture system (Vicon F20-MX3, Vicon Motion Systems Ltd, Oxford, UK) was used to record the motions of the upper limb. The Euler angles were calculated from the reflexible marker positions as proposed previously (Chen et al., 2010). During the dynamic motion recording, only four marker clusters were needed (as shown in **Figure 1**). The recording marker clusters were attached respectively at the flat part of the acromion, the lateral upper arm just below the insertion of the deltoid, the dorsal surface of the distal forearm. The reference marker cluster was located at the flat part of sternum close to the jugular notch. Before the dynamic motion recording, seven bony landmarks were added to specific positions together with the marker clusters according to the recommendation on definitions of joint coordinate systems (Wu et al., 2005). The position relationship between the bony landmarks and the marker clusters was established in a static recording trial. The position of the bony landmarks could then be reconstructed by the marker clusters and their relationship after the bony landmarks were removed. The advantage of such process, reconstructing the bony landmarks from the rigid marker clusters rather than directly recording the bony landmark positions, is to eliminate the recording errors due to the relative bone movement across specific joints. The motion recording was sampled at 50 Hz and synchronized with the EMG recording through the motion capture system.

## 2.2. Experimental Protocol

During the experiment, the subjects were quietly seated in the chair with their torso keeping upright and their right hand keeping relaxing. Considering both the functional tasks of upper limb (van Andel et al., 2008) and the present experimental setup (for example, some movements having serious occlusion problems of the reflexive markers during the course of movement have to be excluded), four motions shown in **Figure 2** were designed to obtain sufficient range of motion of the right upper limb. All the motions were initiated when the right arm was

FIGURE 2 | Illustration of the movements and their trajectories involved in this study. All these movements were resulted from the coordination of multiple muscles and multiple joints. Each movement was initiated from the lowest position and returned to the initial position after the completion of the movement.

freely hanging and close to the torso. The traces in **Figure 2** indicated the movement trajectory from the initial position to the destination position in a single trip of each motion. Motion M1 simulated delivering water to drink. In motion M2, the subjects were asked to touch their right shoulder with their right hand in the sagittal plane. In motion M3 (M4), the subjects were asked to lift their right arm in the sagittal (coronal) plane as they can. After the completion of each movement, the right arm returned to the initial position. As most daily-life movements of human are quasi-rhythmic after sufficient learning and training except when unexpected events happen (Churchland et al., 2012; Kandel et al., 2012), these four arm movements are freely repeated several times in order to capture both the commonality and the variability of the EMG and motion properties.

In order to collect sufficient experimental data and catch most variabilities during the experiments, we designed two separate motion sessions with each session consisting of fifteen repetitions of the motion (see **Figure 3**) for each motion of each subject. The muscles were allowed to relax shortly (around 3 s) before initiating the next motion repetition. Around 5-min rest was allowed before the start of the second test session. All the motions were performed naturally without any kinematic or dynamic constraints of the right arm. The exact duration of the single motion completion and the rest between adjacent motion repetitions were controlled by the subjects themselves as they felt comfortably. Such experimental setup without strict kinematic constraints was also suitable to verify the feasibility of the proposed joint angle estimation method in practical myoelectric control scenario.

#### 2.3. Data Processing

With the above EMG and motion recording strategies, both of the EMG and motion data were saved in a computer and treated off-line in Matlab (The MathWorks, Version 7.10.0.499, 64-bit, 2010). The purpose of data processing is to extract suitable signal features, including EMG and motion features, for estimation

model learning. In addition, the test data used for cross validation were also processed as did for the training data.

As we know, the muscle synergies and motion synergies contribute to reducing the complexity of neural control in the redundant human motor system. Mathematics techniques, such as PCA, can be used to parse the complex data into a small number of components, for example, only two synergies were needed to account for more than 96% of the activation patterns of eleven human arm muscles during free arm motions (Artemiadis and Kyriakopoulos, 2011). Independent component analysis (ICA) was also effective to extract statistically independent muscle activity source signal from their combinations from multi-channel EMG (Nakamura et al., 2004; Garcia et al., 2005) and high-density EMG signals. ICA has been considered superior to PCA in improving force estimation accuracy from EMG (Staudenmann et al., 2007). Inspired by these previous works, two mode decomposition methods, PCA and ICA, were respectively conducted following band-pass filtering (6th Butterworth with 10–400 Hz cut-off frequency) of the EMG signals.

The multi-channel of EMG recordings are usually noisy and have super-Gaussian or Gaussian distributions in different activation levels (Nazarpour et al., 2013). PCA is selected to process the EMG signals as it is especially efficient in dimension reduction when the signals are highly correlated. With the PCA processing, the recorded EMG signals can be on one hand transformed to the low-dimensional principal components (PCs) with noise free. On the other hand, the dependence among different recording channels can be broken resulting in uncorrelated signals. As for the ICA algorithm, it works under the hypothesis that the original signals are linearly mixed and have non-Gaussian distribution, so that it is reasonable to perform PCA as a preprocessing for ICA to discard the irrelevant structures among the EMG signals and satisfy the hypothesis of ICA algorithm. From now on, whenever we mentioned ICA, it means ICA was conducted after preprocessing of PCA while PCA means only PCA was conducted. The detailed algorithm of PCA and ICA can be found in Johnson and Wichern (2007) and Hyvarinen et al. (2001).

Noth that, the feature vector obtained by the training data with PCA was saved for the testing data to be projected in the same low dimensional space as did on the training data. With ICA, the estimation of unmixing matrix can be obtained through different ways, such as the approximations of negentropy, the minimization of mutual information and the maximum likelihood estimation. We achieved the estimation of unmixing matrix through negative entropy maximization due to its advantages such as fast convergency and robustness (Hyvarinen et al., 2001). Similarly, the unmixing matrix obtained with the training data was kept for the testing data to be unmixed in the same way as did on the training data.

After the PCs (ICs) of the recorded EMG signals were induced from PCA (ICA), the EMG feature set, including mean absolute value (MAV), root mean square, waveform length, zero crossing, variance, median frequency and mean frequency were further calculated in a 40 ms analysis window with 50% overlapping. Although such feature set was applied or partly applied in previous works (Artemiadis and Kyriakopoulos, 2011; Jiang et al., 2012), we did not find consistent improvement in estimation quality when using the combined feature set comparing with using only MAV feature. Therefore, we only present the analysis and results by using only MAV as the EMG feature in the following description.

For the motion data, 2-order butterworth filter with cutoff frequency of 6 Hz was applied firstly to the recorded reflective marker position data. The three joint angles at shoulder and one joint angle at elbow were then calculated from the marker position data according to the method proposed previously (Chen et al., 2010). The 4 DoFs are defined as shoulder abduction/adduction (SAA), shoulder flexion/extension (SFE), shoulder pronation/supination (SPS) and elbow flexion/extension (EFE). Although the joint motion dimension can be reduced from original 4-fold into 2-fold representing 98% of the total variance as did in Artemiadis and Kyriakopoulos (2010), since we found this processing did not contribute in reducing the complexity or improving the estimation accuracy significantly, the original 4 joint angles were directly used as the estimation target rather than the low-dimensional 2 components. In **Figure 4**, the 4 DoFs are respectively rectified and normalized by their maximum value to represent in the same range of [0, 1]. As we expect, for the same DoF, it differs over time during different movement repetitions. Among all the DoFs, their correlation differs during different movement repetitions as well, which reveals clear evidence of redundancy in achieving the same movement with different activating strategies of multiple DoFs. In addition, the behavior of single DoF and the correlation among all the DoFs are both different in different subjects for the same movement. Because of these variabilities, simultaneous and continuous estimation of

FIGURE 4 | Examples of 4 motion repetitions of M1 in two subjects. It is clear to find evidence of redundancy in achieving the same movement with different activating strategies of multiple DoFs in the same subject. The behavior of single DoF and the correlation among all the DoFs are both different in different subjects for the same movement.

the joint angles is important to interpret instantaneous neural variations for natural and accurate myoelectric control.

Until now, we obtained both the EMG features and joint angle features to be the input and output respectively of the motion estimation model.

#### 2.4. Joint Angle Estimation with ANN

Multi-layer perceptron (MLP), a feedforward ANN model, was used to learn the mapping from the low-dimensional EMG feature to the shoulder and elbow joint angles. The multiple MLPs (mMLPs) has been popularly used for training where one MLP outputs one joint angle in order to obtain accurate estimation for each joint angle (Jiang et al., 2012; Muceli and Farina, 2012). In addition to the separate MLP training for each joint angle estimation, we attempted to use single MLP (sMLP) for concurrent estimation of all the joint angles. The sMLP was then assessed to see if it could be used instead of mMLPs with comparable estimation accuracy and compact training procedure. In each MLP training neural network, there were three hidden layers with five neurons in each layer. The neurons in the hidden layers and output layer had a tangent sigmoid transfer function and a Linear transfer function respectively. The MLP was trained through the Leverberg-Marquardt back-propagation algorithm. All these network training parameters were fixed during the training and testing processes.

The training and testing of the sMLP strategy are illustrated in **Figure 5**. In the training phase, the EMG features and joint angles were fed into the MLP, the correlation between them was learned and kept for the following testing phase. In the testing phase, the trained MLP outputs the joint angle estimates in the presence of new EMG feature data. The performance index R 2 is calculated from the measured and estimated joint angles through Equation (1). Note that, in the feature extraction procedure, either PCA or ICA was applied before the calculation of MAV of EMG.

feature extraction, the raw EMG signals were sequentially processed with band-pass filtering, PCA or ICA decomposition and MAV calculation.

#### 2.5. Performance Index

The estimation performance can be evaluated by different indices, such as RMS error, correlation coefficient (CC) and coefficient of determination R 2 . In order to compare the performance of the proposed method with other works, global R <sup>2</sup> was calculated across all the joint angles as below:

$$R^2 = 1 - \frac{\sum\_{i=1}^{D} \sum\_{t=0}^{N} (\widehat{\wp\_i(t)} - \wp\_i(t))^2}{\sum\_{i=1}^{D} \sum\_{t=0}^{N} (\wp\_i(t) - \overline{\wp\_i(t)})^2} \tag{1}$$

where <sup>y</sup>i(t), <sup>y</sup>i(t) and <sup>y</sup>di(t) are respectively the real joint angle calculated from motion capture system, its average over time and its estimates from only EMG signals at time instant t for the ith DoF. N is the number of samples of each joint angle. D is the number of DoF involved in the study.

All the estimations were performed on the experimental data recorded from six able-bodied subjects. In a word, the human motion estimation process consists of two phases: training and testing. In the traditional estimation method, k-fold cross validation (k = 4 or 5) was popularly applied, where one out of the k repetitions of each movement was used as testing set and the rest repetitions as training set (Jiang et al., 2012; Muceli and Farina, 2012). In comparison, we applied leave-p-out strategy for the cross-validation where p motion repetitions were used for testing with the rest for training. As the variabilities over time likely deviate the estimation result, this strategy is helpful to evaluate the possibility of the trained model in estimating joint angle in continued movement repetitions. If the training and testing data coming from the same test session, we call such traditional cross-validation process as intra-cross validation. In addition, we proposed to conduct inter-cross validation to reveal more convincing results where the training and testing data were coming from two test sessions for each motion in each subject (see **Figure 3**). Comparing with the intra-cross validation, the inter-cross validation is closer to the practical online condition in myoelectric control, where the testing was performed online following the offline training.

## 3. RESULTS

The purpose of the current work is to propose a simultaneous and continuous joint angle estimation method for multi-joint coordinated upper limb myoelectric control. In the course of joint angle estimation, several factors, such as signal feature extraction method, time-variant properties of EMG and MLP training/testing strategy, may have unexpected influence on the estimation accuracy. Any failure on the estimation course may result in the failure or high error in the joint angle estimation. Therefore, the estimation performance was evaluated in several aspects.

1) **EMG Mode Decomposition:** The estimation performance with ICA and PCA decomposition was compared, suggesting their contribution and advantages for the estimation quality. Original 8-channel raw signals were reduced to 3∼5-manifold for different subjects representing 95% of the total variances by PCA processing. When the ICA algorithm was conducted, the uncorrelated principal components obtained by PCA was treated as the mixed signals.


#### 3.1. Multi-DoF Motion Estimation during Intra-Cross Validation

An example of the joint angle estimation during intra-cross validation is presented in **Figure 6**. The blue lines indicate the joint angles calculated from motion capture system which are considered as the real joint angles. The red lines indicate the joint angle estimates from the EMG and MLP (trained with the training datasets). The estimation was obtained by sMLP and the EMG was processed by ICA. The global estimation accuracy is 94.6% represented by R 2 . In such arm movement, multiple muscles and multiple joints were involved to achieve the coordinated muscle contractions and joint movements. Although the motion velocities and the trajectories are both different among different motion repetitions as shown in **Figure 4**, all the joint angles can be still estimated in the rest, initial concentric and final eccentric motion phase, indicating the feasibility and accuracy of the joint angle estimation method from EMG even in the presence of some kinematic variabilities.

Although the estimates of joint angles can be directly used as the control commands for the robot control, the spacial trajectory of the end-effector is also what we concern. The trajectory must be natural and smooth, otherwise, the joint coordination may be problematic in practice. Therefore, we evaluated the 4 joint angle estimation performance not only by comparing the estimated

joint angles with the measured angles, but also by comparing the trajectories of the end-effector driven by the estimated angles and measured angles. The forward kinematics analysis was conducted using the measured and estimated joint angles respectively to compare the measured and estimated trajectories of the endeffector. As the wrist joint was excluded in this work, the center of the wrist joint was considered as the end-effector with the wrist joint being fixed. **Figure 7** shows the motion trajectory of the end-effector constructed by the measured joint angles (blue) and the estimated joint angles (red) in **Figure 6**. In order to show the movement trajectory intuitively, the original coordinate is set at (x0, y0, z0) as shown in **Figure 1**. In this motion (M1), both the shoulder and elbow joint move in a large range of motion. The determination of R <sup>2</sup> of the trajectory matching is 94.58%. The estimation error mainly exists at the beginning of the movement. Comparatively, the midline and the destination of the trajectory calculated from the estimated joint angles matches the recorded quite well.

#### 3.1.1. Performance Comparison with Different EMG Features and Training Strategies

PCA and ICA were both considered necessary and useful in EMG mode decomposition and represented their suitability in EMGbased joint angle or force estimation (Staudenmann et al., 2007; Artemiadis and Kyriakopoulos, 2010). We intended to evaluate their contribution to the motion estimation performance. Thus, the motion estimation accuracies were calculated and compared between PCA and ICA processing of the EMG signals in this study. For the neural network training, previous works applied mMLPs to yield one angle from each MLP. To simply the training strategy, we compared the estimation performance with sMLP and mMLPs. Finally, sMLP and mMLPs grouping with PCA and ICA algorithm respectively were used for neural network training. All the estimation accuracies of each motion across all the subjects were averaged and summarized in **Figure 8**. We find that the sMLP is sufficient to yield comparable even better estimation accuracy with respect to the mMLPs either with PCA or ICA for EMG mode decomposition. In addition, the training complexity and computation time is superior when training by sMLP. For the comparison between ICA and PCA, both of them can give high estimation accuracy for each motion. If we average the estimation accuracies across all the motions, the four combinations, that is, ICA+mMLPs, ICA+sMLP, PCA+mMLPs and PCA+sMLP, have respectively yielded 89.23, 91.12, 88.7, and 90.23% estimation accuracy comparing with the measured joint angles. By conducting two-way ANOVA test between the two factors, EMG decomposition method (ICA/PCA) and training strategy (sMLP/mMLPs), the p-values for the ICA/PCA, 0.5919, the sMLP/mMLPs, 0.1409, and the interaction between the two factors, 0.8225, indicating all these combinations get similar motion estimation performance and no significant interactions between the two factors. Considering the potential versatility of the ICA and the simple configuration of the sMLP, in the following study, if we did not explain especially, the EMG was processed by ICA and the sMLP was chosen as the training strategy.

#### 3.1.2. Performance Comparison between Short-Term and Middle-Term Estimation

Nonstationary properties of EMG is a troublesome effect for myoelectric control. For example, the estimation accuracy decreased from around 30 s without model switching strategy in a free arm movement (Artemiadis and Kyriakopoulos, 2011). The inherent cause may come from the complex electrochemical behaviors of the skeletal muscles. The robustness of the estimation method to the time-related factors was evaluated by comparing the estimation accuracies in the short-term and middle-term application. If any stage of the estimation process, including feature extraction and training, can not overcome the effects of the time-variant factors, the estimation quality may deteriorate over time. In **Figure 9**, the estimation performance in short-term and middle-term was summarized in the condition of intra-cross validation. The global estimation accuracy is the averaged R <sup>2</sup> when all the test data were used for estimation. When the first half of the testing data were estimated, the short-term accuracy was calculated, whereas the last half of the testing data corresponding to middle-term estimation. The performance across the estimation duration was not statistically significant (p=0.9676), indicating the estimation performance is similar during the 70-s arm movements.

#### 3.2. Multi-DoF Motion Estimation during Inter-Cross Validation

In intra-cross validation, part of data from the same motion session was used for training with the other part for testing. In practice, when the training phase finished, a new motion session starts and new EMG signals may result in corresponding estimates. Therefore, we also tested the motion estimation using EMG signals from two different sessions for training and testing respectively, which was more practical from the muscle states and estimation conduction procedure. An example of

the joint angle estimation during the inter-cross validation is shown in **Figure 10**. The estimation accuracy is 98.95% for the motion M4 of this subject in the 2-min estimation. In this motion, the subjects were asked to lift their right arm in the coronal plane. Due to no kinematic or dynamic constraints being exerted, the subject could perform this motion as they felt comfortably. For example, they may flex their elbow to compensate the uncomfortability during the motion. In this motion, the 4 joint angles were ranged quite differently, while the joint angle estimates were quite close to the measured angles. The corresponding motion trajectory estimation accuracy is 91.38% (shown in **Figure 11**). Although the practical initiation positions and trajectories (blue) in different motion repetitions

are scattering in each plane, the constructed estimate of the motion trajectories (red) are still smooth and accurate in reaching destination.

The estimation comparison between ICA and PCA in this inter-cross validation is shown in **Figure 12**. The average estimation accuracy with ICA and PCA of four motions across six subjects is 87 and 86.3% respectively. The estimation performance seems similar in total with ICA or PCA for EMG mode decomposition in this inter-cross validation (p = 0.8531), except for the motion M2 where ICA presents apparent

FIGURE 10 | An example of the 4-DoF estimation result during the inter-cross validation. The motion M4 was selected to be shown here. The sMLP was trained after ICA processing.

advantages comparing with PCA. With respect to the intra-cross validation where 91.12% (90.23%) estimation accuracies were presented in **Figure 8**, the estimation in the inter-cross validation

is a little worse. However, considering the practical condition of myoelectric control and the estimation accuracies of the previous works as described in the introduction, this estimation performance is quite satisfactory. Moreover, the proposed method also works well during 2-min estimation process in the inter-cross validation. The average estimation accuracy was summarized in **Figure 13** during middle-term estimation with ICA. There are also no significant differences among the global, short-term and middle-term estimation performance (p = 0.8870).

#### 4. DISCUSSION

This paper investigated the feasibility of simultaneous and continuous motion estimation from EMG during arm movements involving coordinated activation of four DoFs across shoulder and elbow joints. In comparison to single joint (such as wrist or elbow joint), the coordination of shoulder and elbow joints offer more contribution for the upper limb to reach objects in a large range of motion. Thus, the simultaneous and continuous estimation of shoulder and elbow motion is important and challenging due to the complex activation of multiple muscles and multiple DoFs (Prilutsky et al., 2011). Although only four arm motions were involved in this work, all of them were functional movements popularly in daily life and achieved by activating all the four DoFs under the condition of no kinematic or dynamic constraints. It is especially meaningful in rehabilitation robotics systems where functional movements are practically important for the users (Takahashi et al., 2008). Moreover, such rehabilitation robot is usually able to provide functional but few movements due to the tradeoff between the complexity of mechanical configuration and the complexity of control system (Maciejasz et al., 2014).

Myoelectric control based on simultaneous and continuous motion estimation has been considered superior to that based on pattern recognition as it provides continuous motion (joint angle) estimates as long as the EMG signals are available. In this work, the proposed method either with PCA or ICA for EMG mode decomposition yields accurate joint angle estimation of the 4DoFs at shoulder and elbow joints. The estimation performance was not only evaluated by intra-cross validation and inter-cross

square.

validation, but also evaluated in short-term and middle-term condition. In intra-cross validation, ICA and PCA with sMLP training respectively gives 91.12 and 90.23% estimation accuracy on average over four arm motions across six subjects. In intercross validation, the corresponding estimation accuracy is 87.14 and 86.30% respectively. This estimation accuracy is high in the context of simultaneous and continuous estimation of multiple DoFs. In addition, the estimation model was not needed to be switched even during 2-min movement, which is relatively stable and efficient for online myoelectric control.

In this preliminary study, only able-bodied subjects were involved. In this situation, the method can be directly used for tele-operation of assistive robots as in Artemiadis and Kyriakopoulos (2011) and any mechatronics device as long as we would like to control it by our movement intent. This method can be also applied for myoelectrical control of exoskeletons of upper limb during the rehabilitation stage of muscle force enhancement. Furthermore, this method is quite promising to be applied for arm prosthetics control benefiting from more motor information provided by targeted muscle reinnervation (TMR) which has been proved able to use EMG decoding to regain intuitive and simultaneous functional control (Miller et al., 2008; Kuiken et al., 2009; Hargrove et al., 2013). In the proposed method, PCA and ICA both provided satisfactory estimation accuracies in the proposed simultaneous motion estimation, while the latter is a little better due to its capacity in further separating the source muscle activities following muscle synergies extraction in the surface EMG recording scenario. Moreover, ICA is expected to be more advantageous comparing with PCA in the presence of more muscle activity cross-talks through highdensity EMG recordings or multiple EMG recordings from the targeted muscles after TMR surgery.

The accurate and stable estimation of multiple DoFs from muscle activities can not only contribute to practical myoelectric control in various robotic systems, but also reveal the inherent correlation between the muscle activities and the coordinated joints through suitable feature extraction and correlation learning methods. Such correlation may be different among different motion tasks, yet determinable for the same motion task even multiple joints and multiple muscles were involved to obtain the coordinated motions.

# 5. CONCLUSIONS AND PERSPECTIVES

EMG signals have been popularly used to decode human motion intent for myoelectric control. Traditional motion estimation method uses pattern recognition to provide binary control commands which can only move the robot in a predefined pattern. Simultaneous and continuous motion estimation from EMG has been proposed to provide more natural and accurate motion control command with respect to pattern-recognition based estimation. However, few works have achieved significant progress in simultaneous and continuous estimation of multijoint coordinated motions. In this work, we proposed a simultaneous and continuous estimation method which can accurately estimate 4 joint angles across two joints for coordinated upper limb motions only from multi-channel EMG signals. EMG mode decomposition through PCA and ICA both revealed their effectivities and feasibilities to obtain accurate joint angle estimation with ICA a litter better. A sMLP network was proposed to estimate the joint angles from the EMG signals after short training. The experimental data from six able-bodied subjects were used to train the neural network and validate the joint angle estimation performance. By comparing the estimated joint angles with the calculated joint angles from the motion capture system, we can find the joint angle estimation accuracy is quite high which indicates that the inherent correlation between the muscle activities and joint angles during complex coordinated motion tasks is feasible to be explored through the proposed mode decomposition and training methods. As the application of EMG decoding in the assistive robot teleoperation (Artemiadis, 2012) and the robotic leg control after a TMR surgery (Kuiken et al., 2009; Hargrove et al., 2013), the accurate arm motion estimation is promising to be applied to control the assistive robot, prosthetic arm or any mechatronics system that we expect to control by our movement intent. In the early future, the proposed method will be further researched on myoelectric control of an exoskeleton our team developed (Liu et al., 2013) and increasing the motion of range of upper limb.

## AUTHOR CONTRIBUTIONS

QZ: Conceived this study. QZ and RL: Performed the experiments and collect the data. WC: Calculated the joint angle from position data. QZ and CX: Proposed the EMG signal processing method. QZ and RL: Analyzed the data and implemented the motion estimation. QZ: Wrote the manuscript. All the co-authors helped revising the manuscript and agreed with the present version.

#### ACKNOWLEDGMENTS

This work was partly supported by the National Natural Science Foundation of China (Grant No. 51305148, 91648203, 51335004), the International Science and Technology Cooperation Program of China (Grant No. 2016YFE0113600), and the Natural Science Foundation of Hubei Province (Grant No.2015CFA004).

### REFERENCES


electrical stimulation. IEEE Trans. Biomed. Eng. 60, 2299–2307. doi: 10.1109/TBME.2013.2253777


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2017 Zhang, Liu, Chen and Xiong. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Grip Force and 3D Push-Pull Force Estimation Based on sEMG and GRNN

#### Changcheng Wu1, 2, Hong Zeng<sup>1</sup> , Aiguo Song<sup>1</sup> \* and Baoguo Xu<sup>1</sup>

*<sup>1</sup> School of Instrument Science and Engineering, Southeast University, Nanjing, China, <sup>2</sup> College of Automation Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing, China*

The estimation of the grip force and the 3D push-pull force (push and pull force in the three dimension space) from the electromyogram (EMG) signal is of great importance in the dexterous control of the EMG prosthetic hand. In this paper, an action force estimation method which is based on the eight channels of the surface EMG (sEMG) and the Generalized Regression Neural Network (GRNN) is proposed to meet the requirements of the force control of the intelligent EMG prosthetic hand. Firstly, the experimental platform, the acquisition of the sEMG, the feature extraction of the sEMG and the construction of GRNN are described. Then, the multi-channels of the sEMG when the hand is moving are captured by the EMG sensors attached on eight different positions of the arm skin surface. Meanwhile, a grip force sensor and a three dimension force sensor are adopted to measure the output force of the human's hand. The characteristic matrix of the sEMG and the force signals are used to construct the GRNN. The mean absolute value and the root mean square of the estimation errors, the correlation coefficients between the actual force and the estimated force are employed to assess the accuracy of the estimation. Analysis of variance (ANOVA) is also employed to test the difference of the force estimation. The experiments are implemented to verify the effectiveness of the proposed estimation method and the results show that the output force of the human's hand can be correctly estimated by using sEMG and GRNN method.

#### Keywords: grip force, 3D push-pull force, force estimation, EMG, GRNN

# INTRODUCTION

Prosthetic hand is a kind of human-machine interface. The upper limb amputees can recover some hand function with the help of the prosthetic hand. In recent years, many kinds of prosthetic hands have been investigated to meet the requirements of the amputees (Davidson, 2002; Zaini and Ahmad, 2011; Maat et al., 2017). Among these hands, the prosthetic hand based on the EMG has received lots of attention due to its simple operation and that it is in accordance with the operation habits of the natural hand. **Figure 1** shows a typical block diagram of the EMG prosthetic hand. Firstly, the EMG signals are captured from the amputee's remaining arm by the EMG sensors on the skin surface. Then, the motion recognition unit outputs the user's motion intention based on analysis of the captured EMG signals. In the motion recognition unit, the features extracted from EMG signals are used to identify the user's motion intention. The controller sends the control commands to the prosthetic hand according to the user's motion intention which has been recognized in the motion recognition unit. In such manner, the prosthetic hand can be

#### Edited by:

*Dingguo Zhang, Shanghai Jiao Tong University, China*

#### Reviewed by:

*Tjeerd W. Boonstra, University of New South Wales, Australia Lizhi Pan, North Carolina State University, United States*

#### \*Correspondence:

*Aiguo Song a.g.song@seu.edu.cn*

#### Specialty section:

*This article was submitted to Neural Technology, a section of the journal Frontiers in Neuroscience*

Received: *31 December 2016* Accepted: *01 June 2017* Published: *30 June 2017*

#### Citation:

*Wu C, Zeng H, Song A and Xu B (2017) Grip Force and 3D Push-Pull Force Estimation Based on sEMG and GRNN. Front. Neurosci. 11:343. doi: 10.3389/fnins.2017.00343*

controlled by the amputee's EMG signals. The motion recognition unit plays an important role in this control mode. Whether the prosthetic hand movement is in line with the user's intention is directly decided by the motion recognition unit. As a matter of fact, many scholars have made a lot of investigations in this area (Zhang et al., 2013; Xie et al., 2015; Hofmann et al., 2016; Segil et al., 2016).

The hand movement is described by type, speed, force, and amplitude of the action, and etc. And all these are important in the control of the prosthetic hand, especially type and force. Therefore, the main tasks of the motion recognition are type classification and force estimation of the action.

In the process of type classification of the action, plenty of research work has been done in the past decades (Li et al., 2010; Hioki and Kawasaki, 2012; Ngeo et al., 2013; He et al., 2015; Pan et al., 2015). Nishikawa et al. used two channels of the EMG signals and the real-time learning method to discriminate a maximum 10 forearm motions including 4 wrist motions and 6 hand motions (Nishikawa et al., 1999). Ten different features extracted from four channels of the EMG signals are used to classify the eight hand motions in Al Omari et al. (2014). And the authors reported that an accuracy of 94% was achieved by using LDA (Linear Discriminant Analysis) based on the combination of wavelength, Willson amplitude, and root mean square. Wu proposed an EMG self-learning recognition method to reduce the effects of the individual difference on the EMG motion recognition (Wu et al., 2015). Ju and Liu used multiple sensors to analyze the human hand motions and reported that there exist significant relationships between the muscle signals and the finger trajectories as well as between the muscle signals and the contact force (Ju and Liu, 2014). To improve the performance of the EMG prosthetic hand, some scholars attempted to obtain the continuous motor variables such as the limb positions and force of the action. Ngeo used a multi-output convolved Gaussian Process to estimate the finger joint kinematics from the EMG signals (Ngeo et al., 2014).

The other part of the hand movement intention, force of hand movement (grip force, push force, pull force, etc.), is also important in the control of the prosthetic hand, especially in dexterous manipulation. There are mainly two kinds of force of the action output from the human hand. One is the grip force and the other is the push force and pull force in three dimensional space (we call it 3D force). In the process of grip force estimation, the most common method is detecting the strength of the muscle contraction from the EMG signals and then mapping the measured strength to the expecting grip force. During the bilateral grasping task, Kamavuako et al. investigated the use of the EMG features in order to estimate the grip force on the ipsilateral and contralateral hand (Kamavuako et al., 2012). And the author also reported that one channel of EMG signals measured from the flexor digitorum profundus can be used to represent the grip force within the entire range of force (Kamavuako et al., 2013). Castellini et al. used five channels of the EMG signals and the SVM (Support Vector Machine) to achieve the grip force estimation and their estimation error is <7% (Castellini et al., 2009). In 3D force estimation, Nielsen et al. used the artificial neural network to estimate the isometric force in multiple degrees of freedom from the wrist (Nielsen et al., 2011). Hashemi et al. researched the EMG based on force estimation in dynamic contractions by using the parallel cascade identification modeling (Hashemi et al., 2012, 2015).

Although, force estimation of the action is also important in the control of the prosthetic hand, the amount of the research work in this area is apparently less than the work in type classification of the action. Actually, not only the grip force but also the 3D force plays an important role in the control of the dexterous prosthetic hand. For the prosthetic hand with movable wrist, the 3D push/pull force is important in the process of moving objects. The estimate of 3D push/pull force can be used to control the rotation of the wrist.

In this paper, we present a force estimation method for the purpose of estimating the force of hand movement comprehensively. Firstly, we set up the experimental platform which is used to capture the EMG signals and force of the action. The eight channels of the EMG signals are captured from the eight different positions on the arm skin surface. The grip force and the 3D force are captured by a grip force sensor and a three dimensional force sensor, respectively. Secondly, the time domain feature extraction method is employed to extract the features from the captured EMG signals. Then, the generalized regression neural network (GRNN) is employed to estimate force of the action which includes the grip force and the 3D force, by using the extracted EMG features and the captured force of action. At last, experiments are implemented to verify the effectiveness of the proposed force estimation method of the action.

The rest of the paper is organized as follows. Section "Experimental Platform" introduces the experimental platform in detail. The detailed process of force estimation of the action is described in Section "Force Estimation of Hand Movement." The experiments are implemented in Section "Experiment and Result," and the Conclusions are in Section "Conclusion."

#### EXPERIMENTAL PLATFORM

An experiment platform as shown in **Figure 2** is set up in this section. The experimental platform consists of several EMG sensors, a three dimensional force sensor, a data collector, a computer, and the application software.

The EMG sensors and the 3D force sensor are respectively used to capture the surface EMG signals generated on the arm

skin surface and the force signals output by the hand while the hand is moving and grasping. The EMG signals and the force signals are digitalized in the data collector. The data collector has a USB interface via which the data can be transmitted between the data collector and computer.

FIGURE 3 | The EMG sensor and disposable electrode.

#### EMG Sensor

The EMG signal is a kind of weak signal. To meet the requirements of EMG signal measurement, we design a surface EMG sensor shown in **Figure 3**. The sensor's pass-band is 10– 500 Hz and the voltage gain is about 1,000. The disposable moisture Ag/AgCl electrodes are used in the sensor. There are two snap-fasteners in the EMG sensor. The distance between these two snap-fasteners is 2 cm. And the snap-fastener is made of conductive metal. There is also a snap-fastener in the disposable electrode. The snap-fastener in the disposable electrode should be buckled into the snap-fastener in the EMG sensor.

There is a conditioning circuit in the EMG sensor. The structure diagram of the conditioning circuit is shown in **Figure 4**. The voltage gains of the two amplifiers, A1 and A2, are set as 15 and 40, respectively. The high-pass filter (cut-off frequency: 10 Hz) is used to remove the direct current component from the signal. The low-pass filter (cut-off frequency: 500 Hz) is used to eliminate the high-frequency noises. The voltage gains of these two filters are all about 1.3. The notch filter is used to

reduce the 50 Hz power-line interference. Since the input range of the data collector is from 0 to 3 V, we design the level-rising circuit to make the voltage of the signals >0 V. **Figure 5** shows a typical signal measured by the EMG sensor.

# Grip Force Sensor

**Figure 6** shows the grip force sensor and its manipulation sketch. The grip force sensor mainly contains a grasping mechanism (GM), an elastic beam, a signal conditioning circuit, and a sensor shell. There is a connecting rod between the elastic beam and the grasping mechanism. The elastic beam is fixed in the sensor shell. Grasping the GM with force will lead to the deformation of the elastic beam. There are two strain gauges, sg<sup>1</sup> and sg2, attached on the surface of the elastic beam. These two strain gauges are used to measure the deformation of the elastic beam. According to the knowledge of the mechanics of the material, we know that there exists a relationship between the deformation of the beam and the force applied to the beam. Therefore, we can measure the grip force by detecting the deformation of the strain gauges. The widely used measurement circuit, Wheatstone bridge, is employed to measure the deformation of the strain gauges. And the output signals of the Wheatstone bridge are amplified by a differential amplifier.

The calibration is necessary for the reason that the relationship between the force applied to the elastic beam and the deformation of the beam is non-linear and there are some other factors which may influence the measurement accuracy. After the calibration experiment, the accuracy of the designed grip force sensor is 0.1N.

# 3D Force Sensor

A three dimension (3D) force sensor (measurement range: −30N ∼30 N, accuracy: 2%FS), which is designed by the Robot Sensor and Control Lab in Southeast University, is introduced to measure the force signal (Ma and Song, 2012; Ma et al., 2013). **Figure 7** shows the signals measured by the 3D force sensor while the force is applied to the sensor optionally. The value >0 indicates that the direction of the force is in forward direction, and the value <0 indicates that the direction of force is in reverse direction.

As shown in **Figure 8**, the 3D force sensor is fixed on a vertical placed flat. And a cylindrical handle is installed at the end of the sensor. Holding the handle, human hand can apply force to the sensor in different directions.

# Data Collector

Analogy signals captured by the sensors can be digitalized by the data collector. The designed data collector can realize analogy

signal sampling in 16 channels simultaneously. The AD (analogy to digital) converters in the data collector are 10 bit. The sampling rate of the data collector can be set from 200 to 2000 Hz per channel. The voltage range of the input signals is from 0 to 3 V. The data collector has a USB interface through which data can be exchanged between the data collector and computer.

# Application Software

The application software running on the computer is used to display the EMG signal waves and force signal waves in real time. And the data of these signals can be recorded when needed. In addition, the application software has the function of changing the data sampling rate by means of sending commands to the data collector.

# FORCE ESTIMATION OF HAND MOVEMENT

The relation between force and EMG signal is varied with many factors, such as force value, speed, path, etc. (Orizio et al., 2010; Kamavuako and Rosenvang, 2012). In order to study the relationship between the EMG signals and the force signals output by the hand and to estimate the force based on the EMG signal, we designed the signal processing diagram as shown in **Figure 9**.

Since the EMG signal is a kind of complex non-linear signal and the hand movement is completed by the cooperation of many muscles, it is difficult to describe the relationship between the EMG signals and the force output by the hand by using the linear model.

Under the assumption that there is a non-linear function, φ, which can describe the relationship between the EMG signals and the force output by the hand, the force output by the hand can be estimated by measuring the EMG signals on the arm skin surface.

$$F = \phi(X) \tag{1}$$

where, φ is the non-linear function, X are the EMG signals measured from the arm skin surface, F is the force output by the hand.

In this paper, the GRNN is employed to describe the nonlinear relationship between the EMG signals and the force for its good non-linear mapping capability and high degree of parallel processing information capacity.

#### EMG Signal Acquisition and Feature Extraction

In this paper, eight different positions on the arm skin surface are selected to acquire the EMG signals. The distribution of the EMG sensors and the serial number of the EMG sensors are shown in **Figure 10** and **Table 1**.

EMG signal captured by the data collector is a time series signal which can describe the characteristics of the hand movement after necessary preprocessing and feature extraction.

The feature extraction method of EMG signals is usually the time domain method, frequency domain method, and time-frequency domain method. The time domain method has the advantages of less computation comparing with the other two methods. The usual time domain feature extraction methods are mean absolute value (MAV), variance (VAR), zeros crossings (ZC), and Willison Amplitude (WA). **Table 2** shows the mathematical equation of each introduced feature. All features are extracted by using Matlab.

**Figure 11** shows a typical EMG signal and its time domain feature.

TABLE 1 | The distribution location and the serial number of the EMG sensors.


## The Generalized Regression Neural Network

The GRNN was proposed by Donald F. Specht in 1991. It is a kind of RBF (Radial Basis Function) neural network. It has the advantage of non-linear mapping, great robustness, and a high degree of parallel processing information capacity. The performance and the learning speed of the GRNN are better than the general neural network (Altiparmak et al., 2009).

The GRNN has a network structure of 4 layers as shown in **Figure 12**. These four layers are input layer, pattern layer, sum layer, and output layer, respectively (Altiparmak et al., 2009).

X = [x1, x2,...,xm] T is the input vector of the GRNN, and Y = [y1, y2,...,y<sup>l</sup> ] T is the output of the GRNN. The number of the neuron in pattern layer is as the same as the number of the training samples. Each neuron in pattern layer corresponds to a



*Where, x<sup>j</sup> is the jth sample data, N is the length of the sliding window.*

training sample data. The transfer function of the ith neuron in pattern layer is:

$$P\_i = \exp[\frac{(X - X\_i)^T (X - X\_i)}{2\sigma^2}], i = 1, 2, \dots, n \tag{2}$$

where, X is the input of the GRNN, X<sup>i</sup> is the learning sample corresponding to ith neuron, σ is the smoothing parameter.

The neuron in sum layer can be divided into two classes. One class corresponds to Equation (3), and the number is only one. The other class corresponds to Equation (4).

$$S\_D = \sum\_{i=1}^{n} P\_i \tag{3}$$

$$S\_{\mathbf{N}\dot{j}} = \sum\_{i=1}^{n} \mathbf{y}\_{i\dot{j}} P\_i, \ j = 1, 2, \ldots, L \tag{4}$$

where, yij is jth element in Y<sup>i</sup> , Y<sup>i</sup> is the ith output sample. L is the dimension of the output vector.

The output of the GRNN is defined as:

$$\mathcal{Y}\_{i} = \frac{\mathcal{S}\_{\text{Ni}}}{\mathcal{S}\_{\text{D}}} \; i = \; 1, 2, \dots, L \tag{5}$$

To improve the performance of the GRNN in our study, the samples should be distributed as uniformly as possible within a certain range.

#### Force Estimation of Hand Movement

In this paper, the GRNN is employed to study the relationship between the EMG signals and the force output by the hand. As shown in **Figure 9**, the force estimation can be divided into two steps. One is offline training of the GRNN, and the other is the online force estimation. In the prosthetic hand based on EMG, the GRNN is offline trained by collecting the EMG signals and the force signals. The trained GRNN is used to estimate force of the action in real time. It is that mapping the extracted EMG features into the force of hand movement by using the trained GRNN.

The software in the computer records the eight channels of the EMG signals (X), the grip force signals (F<sup>g</sup> ), and the 3D force signals (F3D) in real time.

$$X = \begin{bmatrix} X\_1 \\ X\_2 \\ \vdots \\ X\_n \end{bmatrix} = \begin{bmatrix} \varkappa\_{11} & \varkappa\_{12} & \cdots & \varkappa\_{1m} \\ \varkappa\_{21} & \varkappa\_{22} & \cdots & \varkappa\_{2m} \\ \vdots & \vdots & \vdots & \vdots \\ \varkappa\_{n1} & \varkappa\_{n2} & \cdots & \varkappa\_{nm} \end{bmatrix} \tag{6}$$

$$\begin{array}{c} \begin{array}{c} X\_n \ \end{array} \ \end{array} \begin{array}{c} \ \begin{array}{c} \ x\_{n1} \ \ x\_{n2} \ \vdots \ \ddots \ \end{array} \ \begin{array}{c} \ x\_{nm} \ \end{array} \end{array} \end{bmatrix} $$

$$\begin{array}{c} F\_{3D} = \begin{bmatrix} F\_x \\ F\_y \\ F\_z \end{bmatrix} = \begin{bmatrix} f\_{x1} \ f\_{x2} & \cdots & f\_{xm} \\ f\_{y1} \ f\_{y2} & \cdots & f\_{ym} \\ f\_{z1} \ f\_{z2} & \cdots & f\_{zm} \end{bmatrix} \end{array} \tag{7}$$

$$F\_{\mathfrak{F}} = \begin{bmatrix} f\_{\mathfrak{F}^1} \ f\_{\mathfrak{F}^2} \ \cdots \ f\_{\mathfrak{G}^m} \end{bmatrix} \tag{8}$$

where, X<sup>1</sup> ∼ X<sup>8</sup> are the eight channels of the EMG signals captured by the eight EMG sensors, respectively, xij is the jth sample data captured by the ith EMG sensor. Fx, Fy, and F<sup>z</sup> are the force signal in x, y, and z direction measured by the 3D force sensor, respectively; fxi, fyi, fzi are the ith sample data in x, y, and z direction, respectively; fgi is the ith sample data captured by the grip force sensor and m is the number of the sampling.

Time domain feature extraction methods described in **Table 2** are used in the processing of the EMG signals. And we can get the

Frontiers in Neuroscience | www.frontiersin.org

feature matrixes: XFMAV, XFVAR, XFZC, and XFWA.

$$XF\_{MAV} = \begin{bmatrix} X\_{1MAV} \\ X\_{2MAV} \\ \vdots \\ X\_{8MAV} \end{bmatrix} XF\_{VAR} = \begin{bmatrix} X\_{1VAR} \\ X\_{2VAR} \\ \vdots \\ X\_{8VAR} \end{bmatrix} \tag{9}$$

$$XF\_{ZC} = \begin{bmatrix} X\_{1ZC} \\ X\_{2ZC} \\ \vdots \\ X\_{8ZC} \end{bmatrix} XF\_{WA} = \begin{bmatrix} X\_{1WA} \\ X\_{2WA} \\ \vdots \\ X\_{8WA} \end{bmatrix} \tag{10}$$

Combining the feature matrixes and the force signals, we can get a new matrix Sam.

$$\mathbf{S}\_{am} = \begin{bmatrix} XF \\ F \end{bmatrix} \tag{11}$$

where, XF is one of the XFMAV, XFVAR, XFZC, and XFWA. F is one of the F3<sup>D</sup> and F<sup>g</sup> .

In the offline training step of the GRNN, the k-fold cross validation is introduced to evaluate the performance of the GRNN.


In the k-fold cross validation method, k is set to two in the paper. The mean absolute value of the estimation error (MAVE), the root mean square (RMS) of the estimation error and the correlation coefficient (ρ) between F and F˜ are introduced to evaluate the accuracy of the estimation results. Take the force in direction x for an example:

$$MAE\_{\mathbf{x}} = \frac{\sum\_{i=1}^{N} abs\left(\tilde{f}\_{\mathbf{x}i} - f\_{\mathbf{x}i}\right)}{N} \tag{12}$$

$$RMS\_x = \sqrt{\frac{\sum\_{i=1}^{N} (\tilde{f}\_{xi} - f\_{xi})^2}{N - 1}} \tag{13}$$

$$\rho\_{\tilde{f}\_{\mathbf{x}}f\_{\mathbf{x}}} = \frac{\text{cov}\{\tilde{f}\_{\mathbf{x}}, f\_{\mathbf{x}}\}}{\sigma\_{\tilde{f}\_{\mathbf{x}}}\sigma\_{f\_{\mathbf{x}}}} = \frac{E[\tilde{f}\_{\mathbf{x}} - \mu\_{\tilde{f}\_{\mathbf{x}}} \mathbf{(}f\_{\mathbf{x}} - \mu\_{f\_{\mathbf{x}}})]}{\sigma\_{\tilde{f}\_{\mathbf{x}}}\sigma\_{f\_{\mathbf{x}}}} \tag{14}$$

where, ˜ fxi is the estimate of force in direction x; fxi is the force in direction x measured by the 3D force sensor; N is the length of the data.

A greater MAVE indicates that the estimation result is poorer. A greater RMS indicates that the estimation result has a larger fluctuation. And a greater ρ indicates that the estimated result has a higher similarity with the force measured by the force sensor.

In grip force estimation experiments, One-way ANOVA is conducted to compare the performance of different features In One-way ANOVA, the factor is EMG feature (MVA, VAR, ZC, WA). In 3D force estimation experiments, Twoway ANOVA is conducted. The factors of the Two-way ANOVA are force direction (x, y, z) and EMG feature (MVA, VAR, ZC, WA). Tukey comparison is performed when significance is detected. The significance level for all tests is set at 0.05.

#### EXPERIMENT AND RESULT

To verify the effectiveness of the proposed method in this paper, the experiment was implemented.

Six right hand-dominant healthy subjects without any neuromuscular disorders (4 males, 2 females, and aged between 21 and 28) were chosen as the participants the experiment. And the participants were informed consent prior to study participation.

Before the experiment, the locations of the EMG sensors on the arm skin surface were cleaned by medical alcohol. Then, the electrodes were buckled into the EMG sensors and the EMG sensors were attached on the arm skin surface. The locations of the EMG sensors are shown in **Table 1** and **Figure 10**.

During the experiments, the EMG signals and force signals of the action were displayed as waveforms in the computer screen and recorded by the software. The data sampling rate of the data collector was set as 1 kHz per-channel.

In the experiments, the data captured from different individuals are used to train the GRNN, respectively.

## The Grip Force Estimation Experiment

Firstly, the grip force estimation experiment is implemented. During the grip force estimation experiment, the subjects are asked to grasp the grip force sensor several times with different grip force. During the grasping, the subjects should not move their arms. The duration of the grasping is about 1–2 s. The maximum grip force is <30N.

In the experiment, four channels of the EMG signals were selected to estimate the grip force. **Figure 13** shows the EMG signals and the grip force while the subject 1 was grasping the grip sensor three times continuously with different grip force.

Four kinds of EMG time domain features which were extracted from the same raw EMG signals were used to train the GRNN, respectively. And the trained GRNNs were used to estimate the grip force, respectively.

The experiment results of the subject 1 are shown in **Figure 14**. It shows that the results of the WAV and VAR are better than the results of ZC and WA. When the grip force increases from 0 to 8N, the results of all features show a good estimation effect. When the grip force is larger than 10N, the estimation accuracy of ZC and WA decreases, especially when the force is larger than 15N. In the results of ZC, the fluctuation may appear in the estimation results when the grip force is larger than 15N. And when the grip force is larger than 15N, the estimation accuracy is decline sharply in the result of ZC. When the grip force is larger than 20N, the estimation accuracy is also decline in the result of WA. In general, the estimate of force by using these four features and the actual force have the similar trend.

The results of all six subjects are shown in **Table 3**. The average values of the six subjects' results are calculated and shown in **Table 4** and **Figure 15**. The variation range of the all subjects' results is also shown in the form of vertical red line, as shown in **Figure 15**. It shows that the result of MVA is better than the other features'. The result of ZC has a larger fluctuation than that of the other three features.


TABLE 3 | The results of the grip force estimation experiments.

TABLE 4 | The average results of the grip force estimation experiments across all the subjects.


The results of One-way ANOVA and Tukey HSD are shown in **Tables 5**–**8**. Results in **Table 5** indicate that there are significant differences (p < 0.05) among four features. The results of the Tukey HSD show that there are significant differences between each two features except VAR and WA. Four features can be divided into three subsets.

The above results indicate that the result of the WAV is better than the other three features' results by all three evaluation index. The result of the VAR and the result of the WA are similar. Among all the results, the ZC get the

#### TABLE 5 | The results of the One-way ANOVA.


TABLE 6 | The results of the Tukey HSD for *MAVE.*


worst one. The results also show that all of these four features can realize the estimation of the grip force in an acceptable performance.

## The 3D Force Estimation Experiment

TABLE 7 | The results of the Tukey HSD for *RMS.*

The 3D force is the representative of the force output by the human hand in three-dimensional space, which can be measured by 3D force sensor. During the 3D force estimation experiment, the subjects were asked to grasp the handle which was set at the end of the 3D force sensor, and to apply force to the handle in x, y, and z direction. The duration of the subjects applying force to each direction is about 1 s. The maximum force is <20N.

In this experiment, eight channels of the EMG signals were selected to estimate the grip force. **Figure 16** shows the EMG

TABLE 8 | The results of the Tukey HSD for ρ*.*



TABLE 9 | The results of the 3D force estimation experiments.

TABLE 10 | The average results of the 3D force estimation experiments across all the subjects.


TABLE 11 | The results of the Two-way ANOVA for *MAVE.*

Source Type III Sum of squares df Mean square F Sig. Corrected model 4.099 5 0.820 9.471 0.000 Intercept 24.863 1 24.863 287.250 0.000 Force direction 0.093 2 0.047 0.538 0.586 EMG\_Feature 4.006 3 1.335 15.426 0.000 Error 5.713 66 0.087 Total 34.675 72 Corrected total 9.811 71

TABLE 12 | The results of the Two-way ANOVA for *RMS.*


signals and the 3D force while subject 1 applied force to the 3D force sensor.

The same as the grip force estimation experiment, four kinds of EMG time domain features which were extracted from the same raw EMG signals were used to train the GRNN, respectively. And the trained GRNNs were used to estimate the 3D force, respectively.

The experimental results of the subject 1 are shown in **Figure 17**. In **Figure 17**, the red line represents the actual force and the blue line represents the estimated force. The result of WAV has the best performance among the results of all these four features. And the result of ZC is the worst. When applying force to z-direction, which means pushing the 3D force sensor, the estimation results are poor by using the features of VAR and ZC. On the whole, the 3D force estimation results are acceptable.

The results from all six subjects are shown in **Tables 9**, **10**. The average values and the variation range of the six subjects' results are shown in **Figure 18** indicated by bar graph and vertical red line, respectively. It shows that the result of MVA is better than the other features'. The result of ZC has a larger fluctuation than that of the other three features. In the zdirection, the result of VAR also has a lager fluctuation. And in the z-direction, the result of VAR is worst by all three evaluation index.

The results of Two-way ANOVA and Tukey HSD are shown in **Tables 11**–**13**. The results of the Tukey HSD are shown in **Tables 13**–**16**. The results of Two-way ANOVA indicate that there are significant differences (p < 0.05) among four features. For all three evaluation index, the effects of force direction are not significant (p > 0.456). The results of Tukey HSD show that the effects of MAV, VAR, and

TABLE 13 | The results of the Two-way ANOVA for ρ.


TABLE 14 | The results of the Tukey HSD for *MAVE.*


WA are similar. Four features can be divided into two subsets.

The above results indicate that all of these four features can achieve the estimation of the 3D force successfully. The result of MAV is best. But compare with VAR and WA, there is no significant difference. The result of ZC is worst.

#### CONCLUSION

In order to meet the requirements of the dexterous control of the prosthetic hand, the paper proposes a force estimation method of hand movement based on the sEMG and GRNN. The estimated force of hand movement includes the grip force and the 3D force. An experimental platform is set up to measure the multi-channels of the sEMG signals, the grip force and the 3D push-pull force. Based on this platform, the sEMG on the arm skin surface and force of the action output by the hand can be measured synchronously. The widely used time domain feature extraction methods are employed to pre-process the sEMG signals. Then the extracted EMG features are mapped to force of the hand movement by using GRNN.

The experiments are implemented to verify the effectiveness of the proposed force estimation method of hand movement. And the results show that the proposed method can realize the force estimation of hand movement with an acceptable performance under the condition that grip force is <30N and the 3D push-pull force is <20N. In grip force estimation, the result of MVA is best. In 3D force estimation, the result of MAV is also best, but compare with VAR and WA, there is no significant difference. The performance of ZC is worst both in grip estimation and 3D force estimation.


TABLE 16 | The results of the Tukey HSD for ρ*.*


For the future work, we will research the estimation of the parameters of the kinematics and dynamics based on the sEMG for the purpose of further improving the performance of the dexterous manipulation of the EMG prostheses.

#### ETHICS STATEMENT

The study is exempt from ethics approval. We put up a petition in Chinese for sEMG clinical measurement to the Nanjing Tongren Hospital (http://tongren.ecaihr.com/). The relevant organization of the Nanjing Tongren Hospital consider our clinical measurement without ethical problem. All the subjects were gave written, informed consent. The permission was granted by the subject to reproduce their likeness in **Figures 2**, **10**.

#### REFERENCES


#### AUTHOR CONTRIBUTIONS

CW: Project design; Experimental study; Writing paper. HZ: Data analysis. AS: Project design. BX: Data analysis.

#### ACKNOWLEDGMENTS

This paper is supported by the National Science Foundation of China under Grant No. 91648206, 61325018, 61403080, 61673105, 61673114. Jiangsu Technology Development Project under Grant No. BE2014132. Jiangsu Natural Science Foundation under Grant No. BK20140641. Young Teacher Startup Foundation of Nanjing University of Aeronautics and Astronautics under Grant No. 56YAH17027.


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2017 Wu, Zeng, Song and Xu. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# sEMG Sensor Using Polypyrrole-Coated Nonwoven Fabric Sheet for Practical Control of Prosthetic Hand

#### Yinlai Jiang<sup>1</sup> \*, Masami Togane<sup>2</sup> , Baoliang Lu<sup>3</sup> and Hiroshi Yokoi 1, 2

*<sup>1</sup> Brain Science Inspired Life Support Research Center, University of Electro-Communications, Tokyo, Japan, <sup>2</sup> Department of Mechanical Engineering and Intelligent Systems, University of Electro-Communications, Tokyo, Japan, <sup>3</sup> Department of Computer Science and Engineering, Shanghai Jiao Tong University, Shanghai, China*

One of the greatest challenges of using a myoelectric prosthetic hand in daily life is to conveniently measure stable myoelectric signals. This study proposes a novel surface electromyography (sEMG) sensor using polypyrrole-coated nonwoven fabric sheet as electrodes (PPy electrodes) to allow people with disabilities to control prosthetic limbs. The PPy electrodes are sewn on an elastic band to guarantee close contact with the skin and thus reduce the contact electrical impedance between the electrodes and the skin. The sensor is highly customizable to fit the size and the shape of the stump so that people with disabilities can attach the sensor by themselves. The performance of the proposed sensor was investigated experimentally by comparing measurements of Ag/AgCl electrodes with electrolytic gel and the sEMG from the same muscle fibers. The high correlation coefficient (0.87) between the two types of sensors suggests the effectiveness of the proposed sensor. Another experiment of sEMG pattern recognition to control myoelectric prosthetic hands showed that the PPy electrodes are as effective as Ag/AgCl electrodes for measuring sEMG signals for practical myoelectric control. We also investigated the relation between the myoelectric signals' signal-to-noise ratio and the source impedances by simultaneously measuring the source impedances and the myoelectric signals with a switching circuit. The results showed that differences in both the norm and the phase of the source impedance greatly affect the common mode noise in the signal.

Keywords: surface electromyography, myoelectric control, prosthetic hand, conductive polymer, electrode

# INTRODUCTION

Robotic prosthetic hands have benefitted from significant improvements due to the quick advance of robotics and biomedical engineering in the last decade. The prosthetic hand is expected to restore the lost hand's function through hardware that can operate like a human hand and an interface that can transfer the user's motion intention to the hardware. In regard to the hardware, many dexterous prosthetic hands with multiple degrees of freedom (DoFs), such as the Vincent hand by Vincent Systems, the iLimb hand by Touch Bionics, the Bebionic hand by RSL Steeper, and the Michelangelo hand by Otto Bock, have been developed and are commercially available (Belter et al., 2013). In regard to the human interface, electromyography (EMG) signals associated with

#### Edited by:

*Dingguo Zhang, Shanghai Jiao Tong University, China*

#### Reviewed by:

*Yinfeng Fang, University of Portsmouth, UK Jie Shang, Ningbo Institute of Industrial Technology (CAS), China*

> \*Correspondence: *Yinlai Jiang jiang@hi.mce.uec.ac.jp*

#### Specialty section:

*This article was submitted to Neural Technology, a section of the journal Frontiers in Neuroscience*

Received: *19 October 2016* Accepted: *17 January 2017* Published: *06 February 2017*

#### Citation:

*Jiang Y, Togane M, Lu B and Yokoi H (2017) sEMG Sensor Using Polypyrrole-Coated Nonwoven Fabric Sheet for Practical Control of Prosthetic Hand. Front. Neurosci. 11:33. doi: 10.3389/fnins.2017.00033* residual muscle contraction, from which the user's motion volition is recognized according to the remnant muscle activity, have been studied (Asghari Oskoei and Hu, 2007). Studies on the classification of EMG signals have reported remarkable results with advanced algorithms based on machine learning methods such as nonnegative matrix factorization, linear regression, linear discriminant analysis, support vector machines, and artificial neural networks (ANN), which have been proposed to identify as many patterns as possible from limited EMG sources and to deal with the unsteadiness, nonlinearity, individual difference, and low reproducibility of EMG signals (Al-Timemy et al., 2013; Young et al., 2013; Amsüss et al., 2014; Hahne et al., 2014; Jiang et al., 2014b; Earley et al., 2016).

However, despite the significant improvement of the mechanics and the EMG analysis methods, most of the currently commercially available myoelectric prosthetic hands used by amputees are still controlled by very simple algorithms that process the surface electromyogram (sEMG) to control 1◦C of freedom (DoF) at a time. Threshold-level control, which generates a command to control one motion with one EMG source according the signal amplitude, is still the most widely used method (Dosen et al., 2010; Young et al., 2012). The main problem for applying advanced algorithms to control dexterous prosthetic hands in real life is the difficulty of measuring stable myoelectric signals. There are many factors, such as electrode shift (Young et al., 2012; Pan et al., 2015), noises in the signals (Chowdhury et al., 2013), and fatigue that causes the change of EMG (Beck et al., 2014), for the fault of applying advanced algorithms. Some studies turn to multi-modal sensing to overcome the difficulties of myoelectric control (Fang et al., 2015). The difficulty to measure useful signals with a high signalto-noise (S/N) ratio using the current measurement methods is one of the main reasons for keeping amputees away from the advanced prosthetic hands. Most of the algorithms reported are tested in labs with only high-quality signals which are unavailable for used by people in their daily lives. Furthermore, the shape, size, and site of the residual limbs are different from one amputee to another, especially for young amputees whose residual limbs are still changing with their growth (Vasluian et al., 2013). Commercially available sEMG sensors allow little customization to adapt to these various and variable conditions. In real-life applications, the differences caused by analysis algorithms are less than those caused by measurement methods, since the quality of the EMG signal determines the usability and the stability of the myoelectric control.

A sEMG sensor is an electrochemical transducer that detects biopotentials by using electrodes placed on the skin. Since the measured potentials have small amplitudes, <30 mV when measured on the muscle fiber and <1 mV when measured on the skin surface, the potentials are susceptible to electromagnetic interference. Consequently, circuits that amplify the signals and filter external noise are usually required. A traditional way to measure EMGs for myoelectric control is differential amplification, in which the electrodes and the amplifier are the pre-amplifier and are the most important factors affecting the S/N ratio. Ag/AgCl electrodes, so-called "wet electrodes" utilizing an electrolytic gel to form a conductive path between skin and electrode, are currently the most widely used in hospitals and laboratories. Despite the fact that the low electrode-skin impedance of Ag/AgCl electrodes contributes to a high S/N ratio, dry electrodes made from metals, e.g., MyoBock 13E200- 50 electrodes (Otto Bock HealthCare GmbH) made of pure titanium, are always adopted in real-life myoelectric control, since wet Ag/AgCl electrodes are disposable and not suited for long-term measurement. Electrodes reported recently include implantable electrodes (Merrill et al., 2011), invasive fine wire electrodes (Smith et al., 2016), low invasive percutaneous implant (Hahne et al., 2016), noninvasive metal electrodes with special curved microneedle array-based structures (Kim et al., 2015), and high-density multichannel electrodes (Popovic-Maneski et al., 2013). The implantable and invasive electrodes are suitable for measuring high-quality signals for dexterous control and diagnosis in hospitals, but the risk of infection makes them difficult for long-term everyday use. Most of the noninvasive electrodes made of metals have a fixed shape and size and are not customizable.

The objective of this study is therefore to develop usable and customizable sEMG sensors for practical control of a prosthetic hand in daily life. In preliminary studies, we developed both multi-DoF prosthetic hands with an interactive-wire driven mechanism (Seki et al., 2013; Morishita et al., 2014) and two DoF prosthetic hands simplified for quick practical use (Jiang et al., 2014a). An fMRI study with amputees showed the adaptation process in the brain for control of the prosthetic hand (Kato et al., 2009). However, for daily use of the prosthetic hand by a person with a disability, the number of recognizable sEMG patterns is limited by the usability and stability of the sEMG electrodes. This study introduces a sEMG electrode made of polypyrrole-coated nonwoven fabric sheet (PPy electrode) that can be easily customized to any person and can be worn onehanded for stable measurement of sEMG signals. Preliminary results on the PPy electrodes, including the manufacturing method, and basic performance for the control of myoelectric prosthetic hands, were reported in conference proceedings (Jiang et al., 2015). This study further tested the PPy electrodes by amputees and compared them with traditional disposable wet Ag/AgCl electrodes. Furthermore, we investigated the relation between the S/N ratio and the source impedances when using the PPy electrodes by simultaneous measurement of the source impedances and sEMG signals. The results showed that source impedances can serve as a useful reference when manufacturing the PPy electrodes. The possibility of improving the S/N ratio by balancing the source impedances was also indicated.

# MATERIALS AND METHODS

#### Development of the sEMG Sensor Electrode Material

A sEMG sensor using polypyrrole-coated nonwoven fabric sheet (Japan Vilene Company, Ltd) as electrodes (hereinafter referred to as PPy electrodes) is proposed for people with disabilities to control prosthetic limbs in daily life. The material is shown in **Figure 1**. The base material is polyethylene terephthalate and nylon, coated with polypyrrole, a kind of conducting

polymer. The thickness of the sheet is 0.5 mm and the surface resistivity is <1 × 10<sup>5</sup> /sq. Compared with poly(3,4 ethylenedioxythiophene)-poly(styrenesulfonate) (PEDOT-PSS), which has been reported for electrodes measuring bioelectrical signals (Yamato et al., 1995; Richardson-Burns et al., 2007; Lang et al., 2009; Tsukada et al., 2012; Isik et al., 2015; Nagamine et al., 2016; Pani et al., 2016; Papaiordanidou et al., 2016), the conductivity of polypyrrole is lower, but it is more stable in an ambient environment that may be humid and wet due to skin sweating with a lower watersolubility and a lower hygroscopicity (Lee et al., 2001; Jang et al., 2004; Xu et al., 2006; Lang et al., 2009; Elschner et al., 2010).

#### Structural Design of the Sensor Band

PPy electrodes are easily customized to fit the size and the shape of the residual parts of an amputee. To attach PPy electrodes to the stump and to keep them on the skin, the PPy electrodes are sewn on an elastic band, as shown in **Figure 2**. Length l and width w of the electrodes, and distance d between pairs of electrodes can be adjusted to match a specific user. The elasticity of the band leads to close contact to the skin, and thus reduces the skin-electrode impedance and enables stable sEMG measurement with a high signal-to-noise ratio. Stainless swaging parts are used to transduce the electrical potentials from the PPy electrodes to the circuit of amplifier and filter.

#### Trial Use

Myoelectric control of a prosthetic hand with PPy electrodes was tested by a 14-year-old girl with a congenital right upper limb deficiency and a 36-year-old man with a right upper limb amputation that happened 5 years earlier. All the experimental protocols of this study were approved by the ethics board of the University of Electro-Communications [No. 10006(4)]. All subjects were fully informed about the procedures, risk, and benefits of the study, and written informed consent was obtained from all subjects before the study.

**Figure 3** shows the 3-channel sEMG sensor band with six recording electrodes for differential amplification and a common ground electrode. The potentials detected by the PPy electrodes were amplified and filtered by the circuit shown in **Figure 3**. The circuit was connected to the electrodes as closely as possible to ensure the electrical contact avoided noise. The circuit consisted of a preamplifier using an instrument amplifier AD620 (Analog Devices, USA) with a gain of 40 dB, a band pass filter with a 1– 1000 Hz bandwidth, a 50 Hz notch filter, and a second amplifier. The output voltage range of the circuit was (−9 v, 9 v).

The subjects wearing the sensor band tried to control a prosthetic hand developed in a preliminary study (Jiang et al., 2014a). The prosthetic hand consisted of a lightweight robotic hand with two motors to realize power grasp, precision grasp, and lateral grasp, a socket to mount the hand to the residual forearm, and a controller (SH72544R, Renesas Electronics Corporation, Japan) that performed EMG signal processing, pattern recognition, and motor control. The sEMG sensor band was worn on the stump covered by the socket. **Figure 4** shows that the stumps of the subjects are quite different. They were able to one-handed attach the sensor bands customized for them by themselves. With the sensor bands, they tried to control the myoelectric prosthetic hand to perform some daily task, e.g., picking up a cup or building blocks, as shown in the figure.

# COMPARISON WITH WET AG/AGCL ELECTRODES

The trial by the subjects illustrated the practicability of the sensor band. To further verify the performance and characteristics of the PPy electrodes, these electrodes were compared with traditional wet Ag/AgCl electrodes in two experiments: one to investigate the correlation of the measured sEMG signals and the other to

FIGURE 4 | Controlling prosthetic hand with the proposed PPy electrodes.

compare the accuracy of sEMG signal pattern recognition for myoelectric control.

## Analysis of Signal Correlation

To investigate the quality of the signals measured by the proposed sensor, we carried out simultaneous measurement of sEMG

with the PPy electrodes and the widely used wet electrodes, the Ag/AgCl electrodes with electrolytic gel (GE Healthcare, Japan) from the same muscle fibers. The measurement settings are shown in **Figure 5**. Four able-bodied subjects participated in the first experiment. Each subject was measured three times. The sampling rate was 1600 Hz and the quantization bit length was 16. The correlation coefficient of the root mean square (RMS) of the signals measured by the two kinds of electrodes and the FFT results were calculated and compared.

# Pattern Recognition for Myoelectric Control

To test the practicality of the PPy electrode, a pattern recognition experiment was conducted. In this second experiment, the sEMG signals were used for classification of multiple hand and wrist movements. **Figure 6** shows the five basic motions selected for myoelectric control.

The 3-channel sEMG sensor band with the same structure as shown in **Figure 3** was worn below the elbow, and the wet Ag/AgCl electrodes for comparison were adjusted to the same positions (**Figure 6**). The basic architecture of the pattern recognition algorithm was developed in the preceding study (Jiang et al., 2014a). The data were sampled at 1600 kHz. After A/D conversion, the SMG data were segmented into 128 ms epochs every 10 ms and FFT was performed on each epoch. The power spectrum was smoothed by the moving average. The powers of frequencies 31, 55, 78, 102, 148, 195, 258, and 320 Hz were extracted as features for input into the artificial neural network (ANN). The ANN is a 3-layer feed-forward neural network with 24 nodes in the input layer, 32 nodes in the hidden layer, and 5 nodes in the output layer.

Six able-bodied subjects participated in the experiment. The ANN was trained beforehand for each subject with a back propagation algorithm. The subjects then performed each of the five tasks for 2 s, during which the sEMG was classified 200 times. Each task was performed three times by each subject. The ratio of correct recognition to total recognition was used as the metric to compare the effectiveness of the electrodes.

#### Results

Examples of the RMS of the signals are shown in **Figure 7**. Although the amplitude of the signals measured by the wet Ag/AgCl electrodes was slightly larger, the wave shape of the signals is quite similar, with respect to the PPy electrodes. The correlation coefficients of the 12 tests are listed in **Table 1**. Individual differences were significant, with Subjects 1 and 2

being lower and Subjects 3 and 4 being higher, but the average correlation coefficients of the 4 subjects were all above 0.82. The overall average correlation coefficient of the 4 subjects was 0.87 (SD = 0.065, n = 12). Examples of the FFT results are shown in **Figure 8**. The power spectra of the signals measured with the two kinds of electrodes were very similar in the low-frequency area under 150 Hz. In the high-frequency area over 150 Hz, the power of the signals measured with the PPy electrodes was higher than that with the wet Ag/AgCl electrodes.

The results of the pattern recognition are listed in **Table 2**. Of these tasks, Relax was recognized most accurately at nearly 100%. The other four tasks were recognized with similar accuracies around 85%. The recognition accuracies of each subject were different but consistent, with both sets of accuracies with the different electrodes being relatively lower or higher for each subject. Although the recognition accuracy was quite different according to the subject and the task, no consistent difference was observed in the accuracy with respect to the electrode types. The mean recognition accuracies for each subject showed little


TABLE 1 | Correlation coefficients of the 12 tests.

difference between the types. The accuracy difference was due much more to the subject difference than to the electrode types.

#### Discussion

PPy nonwoven sheet was adopted as the material for constructing sEMG electrodes that address the problem of customization to improve the usability and practicality of myoelectric control. The size and shape of the sheet are easily adjusted to amputees whose stumps have different residual muscles. The PPy electrodes were compared with the widely used wet Ag/AgCl electrodes to investigate the signal quality and to confirm their feasibility for myoelectric control of multi-DoF prosthetic hands. The simultaneous measurements with the two types of electrodes suggest that signal quality was quite similar and the difference for myoelectric control was negligible. The displacement of electrodes and electrode material were two considerable reasons for the difference in the correlation coefficient. The displacements in the longitudinal direction with respect to the muscle fibers may affect the EMG characteristics (Young et al., 2012). It was hard to place the electrodes onto the skin strictly along the muscle fiber. Interface capacitance existed between skin and the dry PPy electrodes without utilize an electrolyte, as in the gel of the wet Ag/AgCl electrodes. The different electrical property of the electrode materials might contribute to the difference in the sEMG signals.

From the results of the general FFT and the conventional ANN with a simple structure implementable on a microprocessor, the average recognition accuracy of the PPy electrodes for 5 classes of movement was nearly 90%, which was comparable to the average accuracy of the wet Ag/AgCl electrodes. These experimental results show that the proposed PPy electrodes, which are highly usable and customizable, are as effective as the wet electrodes for myoelectric control. Besides the conditions of the subject, the performance of EMG based motion classification mainly depends on measurement, feature extraction, and classification. Many methods have been reported for the analysis and classification of sEMG signals (Chowdhury et al., 2013). The methods, with their advantages and disadvantages, lead to different classification results which might further elucidate the characteristics of the measurement method. We adopted the general FFT and the conventional ANN in this study to show the performance of the PPy electrodes for the practical control of prosthetic hand. Future work will include other methods to further investigate the characteristics of the proposed PPy electrodes.

## RELATION BETWEEN SEMG QUALITY AND SOURCE IMPEDANCE

The electrodes act as a transducer by converting ionic current within the tissue to electrical current to the amplifier. The source impedance, including the tissue impedance, the skinelectrode impedance, and the electrode impedance, is one of the most important factors affecting the S/N ratio. The mismatch of source impedance decreases the common mode rejection ratio (CMRR), which degrades the S/N ratio of the signal (Degen and Jackel, 2008). The proposed PPy electrodes can be easily cut into different shape and size with different conductivity and skin-electrode impedance. The balance of source impedances may be a criterion in preparation of the PPy electrodes. An experiment was therefore conducted to investigate the relation between sEMG quality and source impedance.

#### Analysis of Signal Correlation

The experimental system for simultaneous measurement of the sEMG and the source impedance is shown in **Figure 9**. Sources 1 and 2 are the origins of EMG signals. Z<sup>1</sup> and Z<sup>2</sup> indicate the combination of tissue impedance, skin-electrode impedance, and electrode impedance. R<sup>1</sup> and R<sup>2</sup> are variable resistances (0–200 k) to enable manual adjustment of the source impedances, since the source impedances to the differential amplifier include variable resistances. By manually adjusting R<sup>1</sup> and R<sup>2</sup> to increase the imbalance of the source impedances, the CMRR declines, thus causing a decrease in the S/N ratio. The photo at the top right of **Figure 9** shows the variable resistances installed before the amplification circuit. An LCR meter (ZM 2371, NF Corporation, Japan) was used to measure the magnitude and phase angle of the source impedances at 300 Hz and 1 Vrms. Switches 1 and 2, controlled by Arduino Uno (Arduino, Italy), switch the measurement of source impedance 1 and source impedance 2 with the LCR meter. It is important to note that the source impedance exhibits

Subject 1 Subject 2 Subject 3 Subject 4 0.82 0.83 0.87 0.76 0.87 0.83 0.83 0.85 0.88 0.97 0.96 0.96

#### TABLE 2 | Results of pattern recognition.


\**wet, wet Ag/AgCl electrodes; PPy, PPy electrodes.*

a number of amplitude- and frequency-dependent nonlinear effects (Dorgan and Reilly, 1999). The source impedances at 300 Hz and 1 Vrms were measured as representative values, since measurements at multiple frequencies would cost time and decrease the simultaneity between impedances and the sEMG. During the impedance measurement, the 1 Vrms input signal was also amplified, which caused amplifier saturation. The saturation started with the start and ended with the end of the input signal. The output of the differential amplifier during the impedance measurement was therefore excluded from the sEMG analysis.

The sEMG of an able-bodied subject doing flexion was measured by placing the PPy electrodes above the digitorum profundus muscle. To investigate the relation between the source impedance balance and the S/N ratio, we adjusted the noise level by tuning R<sup>1</sup> and R<sup>2</sup> during the rest state, and then measured the sEMG and the source impedances at the noise level. The three noise levels are illustrated in **Figure 10**. Levels 1, 2, and 3 show little noise, noise in the range of (−6 v, 6 v), and noise in the range of (−9 v, 9 v), respectively. At each level, the sEMG and source impedances were measured ten times. The differences of the magnitude and phase angle were used as an index of source impedance imbalance, and the FFT of the sEMG signals were calculated to analyze the noise.

#### Results

**Table 3** shows the source impedance imbalance at the three noise levels. Both the magnitude differences and the phase angle differences are significant. At noise level 1, the imbalance of magnitude is around 1500 and the phase angles are nearly the same. To increase the noise to levels 2 and 3, the imbalance was increased in magnitude to around 8000 and 20,000 , and the phase angle was increased to around 10 deg and 30 deg, respectively. Accordingly, the SD of the imbalances in the magnitude and phase angle also increased with the increase of noise level.

The power spectra of the measured sEMG signals at different noise levels are shown in **Figure 11**. The noise mainly came from the power line radiation (50 Hz) and its harmonics, and the measurement frequency (300 Hz) of the LCR meter. At noise level 1, the noise was mainly caused by the LCR meter at 300 Hz. With the decline of the CMRR due to the imbalance of source impedances, more noise was mixed in with the sEMG signals. Especially at the frequencies of 100, 200, and 300 Hz, the power of the noise was much higher than that of the sEMG signal.

#### Discussion

An important factor of the dry electrodes is their ability to maintain electrical contact with skin, since the electrical contact affects the electrode-skin impedance and the source impedance balance. The dry electrodes do not utilize an electrolyte, as in the gel of the wet Ag/AgCl electrodes. High interface capacitance therefore exists between the skin and the electrode, so that the source impedances are larger when using dry electrodes than those when using wet electrodes. A comparison study (Searle and

Kirkup, 2000) investigated the contact impedances, the resistance to electrical interference, and the movement artifacts of the dry and wet electrodes. However, the relation between contact impedances and S/N ratio was not quantitatively measured.

In this study, the simultaneous measurement of the source impedance balance and the sEMG by using the switching circuit suggest that the imbalances in the magnitude and phase angle of the source impedances were the reason for the increase of noise level. By adjusting the variable resistances to unbalance the source impedances, the CMRR of the differential amplifier declines, and thus the common mode noises, caused by harmonics of the power line radiation and the measurement signal, were also amplified. Conversely, if the source impedances are balanced, the S/N ratio can be improved. Source impedances, rather than the shape and the size of the electrodes, are important for sEMG measurement. Therefore, two electrodes connected to the same amplifier do not have to have the same shape and size as long as the source impedances are close. The proposed flexible PPy electrodes provide high freedom of shape and size adjustment to fit a specific amputee.

# GENERAL DISCUSSION

We developed a novel PPy electrode for the convenient measurement of sEMG signals. The conductive nonwoven fabric sheet is easily cut and sewn to fit an individual. Currently, most amputees need to be checked as to whether they are suitable to use a myoelectric prosthetic hand at hospitals or other medical institutions. Before amputees get myoelectric prosthetic hands, which are custom-made according to the prescription by the prosthesis therapist, they are required to participate in training up to 2 months in order to generate a stable sEMG signal. This process decreases the accessibility of myoelectric prosthetic hands, especially for those who are poor and living in undeveloped areas. The goals of our current study and our preliminary studies (Jiang et al., 2014a, 2015) are to

TABLE 3 | Magnitude and phase angel differences at the three noise levels (Impedances were measured at 300 Hz and 1 Vrms).


develop a low-cost and highly usable myoelectric prosthetic hand system in order to increase its accessibility, both economically and technically. In order to provide the technology and the opportunities necessary for people with disabilities to maximize their quality of life by using prosthetic hands, the whole system including the sEMG measurement should be conveniently adaptable to any person, since each person represents different conditions. Usability is one of the main factors that affects prosthesis acceptance or rejection (Biddiss and Chau, 2007; Vasluian et al., 2013). The trial use experiment illustrated that the PPy electrodes provide a convenient and customizable measurement method for amputees. The comparison with wet Ag/AgCl electrodes suggest that the PPy electrodes are usable and capable of recording sEMG signals for myoelectric control.

To address this challenge of sEMG measurement, many kinds of electrode construction using pure or mixed materials such as stainless steel, Si3N4, Ag/AgCl, and Au have been proposed (Searle and Kirkup, 2000). Recently, conductive polymers, especially PEDOT-PSS, have been widely studied as promising bioelectrode materials (Richardson-Burns et al., 2007; Lang et al., 2009; Tsukada et al., 2012; Isik et al., 2015; Nagamine et al., 2016; Pani et al., 2016; Papaiordanidou et al., 2016). PEDOT-PSS has highly electroconductive and biocompatible characteristics that meet the requirements for biomedical electrode materials (Takamatsu et al., 2015; Pani et al., 2016). However, because it is more water-soluble and hygroscopic, PEDOT-PSS exfoliates from the substrates and absorbs water in wet conditions, which will reduce its mechanical strength and electrical characteristics (Elschner et al., 2010). Although the conductivity is lower, polypyrrole is more stable than PEDOT-PSS in the sEMG measurement environment, in which the electrode may become wet with sweat. The low conductivity can be compensated to some extent by the use of active electrodes (where buffering and amplification are performed at the electrode site) and high-quality amplifiers. The experimental comparison with wet Ag/AgCl electrodes suggests that although the conductivity of the PPy electrodes is much less than that of the wet Ag/AgCl electrodes, the signal quality was quite similar and negligible for myoelectric control.

One major cause of the noise in the sEMG is the common mode interference that is mixed in the signal when the CMRR declines due to the skin-electrode impedance change (Yokus and Jur, 2016). The switched simultaneous measurements of the sEMG and source impedances suggest the importance of source impedance balance for signal quality. Theoretically, any conductive material has the potential to become an electrode if the amplifier is ideal, with infinitely great input impedance. Since such amplifiers do not exist, most of the studies focus on the electrical property in order to develop electrodes with skin-electrode impedance that is as low as possible (Hahne et al., 2016). However, for a practical use such as myoelectric control, a tradeoff exists between reducing the contact impedance and improving the usability and customizability. The electromechanical stability of the contact between sEMG electrodes and skin directly affects the quality of the signal (Roy et al., 2007), but the stability is hard to guarantee when using metallic electrodes to control a prosthetic hand in a real-life environment, since impedance changes can be induced by motion (Cömert and Hyttinen, 2014). With the improvement in electronic devices, more flexible conductive materials with relatively low conductivity can be alternatives to developing practical surface bioelectric electrodes. The simultaneous measurements of the source impedance balance and the sEMG also show the possibility of improving the S/N ratio by balancing the source impedances. If the source impedances can be monitored and adjusted with variable resistances to decrease the impedance differences, the CMRR of the differential amplifier could be improved to exclude the common mode interferences.

In preparation of the PPy electrodes, source impedance and interelectrode distance (Young et al., 2012) are two important factors to determine their shape and size. Desirable characteristics of electrode-skin impedance are a low value of the impedance and a balance of values between the electrodes. Small electrodes with less contact area may lead to high electrodeskin impedance and the difference of electrode-skin impedance may cause source impedance unbalance. High density electrodes are often adopted in scientific studies to obtain fine EMG characteristics or to improve the robustness by redundancy. However, in practical control of prosthetic hands, most of which have <5 DoFs currently (Belter et al., 2013), three effective channels of sEMG signal are enough. Another non-negligible factor is the conditions of the amputees (i.e., the number and distribution of residual muscles). The flexible PPy electrodes can be easily cut into the shape and size that fit a specific amputee. Metal electrodes also can be customized into different shapes and sizes. But metallic processing is not as easy as cutting a fabric sheet. And metal electrodes are not flexible to fit closely to the body.

In sum, the PPy electrodes investigated here showed their usability and effectiveness in sEMG recording for myoelectric control of prosthetic hands. Nevertheless, because the trial use by amputees and the experiments were carried out over a short period of time (<1 day for the trial use and 1 week for the experiments), the durability needs to be further validated. The stability of conductive polymers including PPy and PEDOT:PSS is different with respect to dopants, synthesis methods, environment, and usage in applications (Yamato et al., 1995; Cui and Martin, 2003; Jang et al., 2004; Khodagholy et al., 2011; Andreoli et al., 2013). To the best of our knowledge, there is no studies comparing PPy and PEDOT:PSS in the form of fabric sheet. Since PPy is stable under wet and humid conditions, the durability is expected to be better than that of electrodes made of PEDOT:PSS. Furthermore, the properties of the nonwoven fabric sheet, including the surface texture and the geometric configuration, need to be further investigated. The performance

#### REFERENCES

Al-Timemy, A. H., Bugmann, G., Escudero, J., and Outram, N. (2013). Classification of finger movements for the dexterous hand prosthesis control with surface electromyography. IEEE J. Biomed. Health Inform. 17, 608–618. doi: 10.1109/JBHI.2013.22 49590

of the PPy electrodes may be improved by optimizing their properties.

## CONCLUSION AND OUTLOOK

We have proposed a sEMG sensor using PPy electrodes for practical myoelectric control of prosthetic hands in daily life. The flexible electrodes are highly customizable to fit the size and shape of different stumps. The PPy electrodes enable highly usable and effective sEMG measurements in real-life settings. By sewing the electrodes on elastic webbing, it is easy to construct a band that can be put on one-handed to record multiple channel sEMG signals. Experiments comparing PPy electrodes with wet Ag/AgCl electrodes suggest that the PPy electrodes are as effective as the widely used wet Ag/AgCl electrodes for myoelectric control of prosthetic hands. The simultaneous measurements of source impedances and sEMG suggest that the source impedance imbalance is a main reason for the low S/N ratio, and so the S/N ratio may be improved by balancing the source impedances. We will continue to investigate the durability of the sEMG sensor using PPy electrodes and to improve it by optimizing the properties of the fabric sheet. Long-term home trial use together with prosthetic hands by amputees is under planning. PPy electrodes with different surface textures and geometric configurations will also be tested in the near future.

#### AUTHOR CONTRIBUTIONS

YJ, MT, BL, and HY conceived the study concept. YJ, MT, and HY developed the electrode with the polypyrrole-coated nonwoven fabric sheet. YJ and MT designed and executed the experiments. YJ, MT, BL, and HY interpreted the experimental data. YJ drafted the manuscript. MT, BL, and HY revised the manuscript.

## ACKNOWLEDGMENTS

This research was partially supported by JSPS KAKENHI Grant Numbers JP16K01539, JP16K12951, and JP16K12503. It was also supported by the "Brain Machine Interface Development" and the "Adaptable and Seamless Technology Transfer Program through Target-driven R&D" by the Japan Agency for Medical Research and Development (AMED). The authors gratefully acknowledge the support by Japan Viline for providing us with polypyrrole-coated nonwoven fabric sheet. Also, we appreciate the assistance of S. Sakoda in setting up the experimental system and S. Morishita in data analysis.

Amsüss, S., Goebel, P. M., Jiang, N., Graimann, B., Paredes, L., and Farina, D. (2014). Self-correcting pattern recognition system of surface EMG signals for upper limb prosthesis control. IEEE Trans. Biomed. Eng. 61, 1167–1176. doi: 10.1109/TBME.2013.2296274

Andreoli, E., Liao, K.-S., Haldar, A., Alley, N. J., and Curran, S. A. (2013). PPy:PSS as alternative to PEDOT:PSS in organic photovoltaics. Synthetic Metals 185–186, 71–78. doi: 10.1016/j.synthmet.2013.10.005


IEEE Trans. Biomed. Eng. 60, 1250–1258. doi: 10.1109/TBME.2012.22 32293

**Conflict of Interest Statement:** The handling Editor declared a shared affiliation, though no other collaboration, with one of the author BL and states that the process nevertheless met the standards of a fair and objective review.

The other authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2017 Jiang, Togane, Lu and Yokoi. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Evolutionary Algorithm Based Feature Optimization for Multi-Channel EEG Classification

Yubo Wang1, 2 and Kalyana C. Veluvolu<sup>2</sup> \*

*<sup>1</sup> School of Life Science and Technology, Xidian University, Xi'an, China, <sup>2</sup> School of Electronics Engineering, College of IT Engineering, Kyungpook National University, Daegu, South Korea*

The most BCI systems that rely on EEG signals employ Fourier based methods for time-frequency decomposition for feature extraction. The band-limited multiple Fourier linear combiner is well-suited for such band-limited signals due to its real-time applicability. Despite the improved performance of these techniques in two channel settings, its application in multiple-channel EEG is not straightforward and challenging. As more channels are available, a spatial filter will be required to eliminate the noise and preserve the required useful information. Moreover, multiple-channel EEG also adds the high dimensionality to the frequency feature space. Feature selection will be required to stabilize the performance of the classifier. In this paper, we develop a new method based on Evolutionary Algorithm (EA) to solve these two problems simultaneously. The real-valued EA encodes both the spatial filter estimates and the feature selection into its solution and optimizes it with respect to the classification error. Three Fourier based designs are tested in this paper. Our results show that the combination of Fourier based method with covariance matrix adaptation evolution strategy (CMA-ES) has the best overall performance.

#### *Edited by:*

*Dingguo Zhang, Shanghai Jiao Tong University, China Keum-Shik Hong, University of Illinois at Urbana-Champaign, USA Jianjun Meng, University of Minnesota, USA*

> *\*Correspondence: Kalyana C. Veluvolu veluvolu@ee.knu.ac.kr*

#### *Specialty section:*

*This article was submitted to Neural Technology, a section of the journal Frontiers in Neuroscience*

*Received: 01 September 2016 Accepted: 16 January 2017 Published: 01 February 2017*

#### *Citation:*

*Wang Y and Veluvolu KC (2017) Evolutionary Algorithm Based Feature Optimization for Multi-Channel EEG Classification. Front. Neurosci. 11:28. doi: 10.3389/fnins.2017.00028* Keywords: BCI, evolutionary algorithm, Fourier linear combiner, feature optimization

# 1. INTRODUCTION

Brain-computer interface (BCI) is defined as an alternative communication pathway which translates the measured brain activity into control commands (Pfurtscheller et al., 2008). Among the existing brain activity measurement techniques, EEG has been extensively employed for BCI applications due to its non-invasiveness, ease of implementation, and cost-efficiency (Lotte et al., 2007). Recently, the dry-electrodes and wireless EEG systems have also been studied in various experimental settings which has further expanded the scope of EEG based BCI systems (Tangermann et al., 2012; Gao et al., 2014).

The motor imagery has been one of the successful methods for EEG based BCI systems (McFarland et al., 2010; Hill et al., 2014). The application of motor imagery task to BCI systems has enabled subjects to sufficiently control a moving cursor in 2D space or a quadcopter in 3D space (Wolpaw et al., 2004; LaFleur et al., 2013). As shown in Muller-Gerking et al. (1999) and Guger et al. (2003), a contra-lateral amplitude decrease in α band can be found in most subjects during motor imagery tasks. Therefore, the motor imagery based BCI systems require transforming the recorded EEG signal into frequency domain in quasi real-time. Then the extracted frequency domain features are fed to machine learning algorithms for classification (Lotte et al., 2007; Robinson et al., 2013).

As frequency analysis of EEG is a major concern, many approaches such as the band pass filter bank, autoregressive (AR) model, Fourier transform, and Wavelet transform have been employed (Turnip and Hong, 2012; Wang et al., 2012b; Robinson et al., 2013; Hsu, 2014; Khan et al., 2014). However, all these methods have certain limitations. The frequency resolution of a band-pass filter is constrained by the order of the filter. The AR model coefficients only provide limited frequency information as the envelope of the spectrum reconstructed from AR coefficients is limited by the order of AR model (Shumway and Stoffer, 2011). The discrete wavelet transform is considered as a bandpass filter bank with orthogonal basis that has better frequency and temporal resolution (Wang et al., 2013b). However, the obtained wavelet coefficients are redundant in general. Further, post-processing is required to stabilize the performance of the classifier (Qin and He, 2005).

The EEG signal considered for BCI is generally bandlimited. To decompose such signal into frequency domain and to maintain the balance between temporal and frequency resolution, the band-limited multiple Fourier linear combiner (BMFLC) has been developed (Veluvolu et al., 2012). BMFLC models the signal by adopting a truncated Fourier series. Adaptive filter algorithm is then employed to obtain amplitude estimation of individual frequency components in a pre-fixed frequency band. For application to motor imagery, BMFLC is applied to provide time-frequency mapping of the µ rhythm. The estimated amplitude of each frequency is then fed to a classifier (Wang et al., 2012a, 2013b). It was found in Wang et al. (2012a) that the performance of the BMFLC based BCI systems is improved if a subject-specific narrow band can be identified within the range of µ rhythm. The identification criterion and existence of such band have been established in Veluvolu et al. (2012). The reduced dimensionality in the feature vector resulted in better performance of the classifier.

Earlier approaches based on BMFLC only employed EEG signals that were recorded from sensorimotor cortex (i.e., C3 and C4 from international 10/20 system, Jurcak et al., 2007; Wang et al., 2012a, 2013b). Since only two EEG channels were considered, it was straightforward to perform the subject-specific band identification on the averaged time-frequency mapping (Wang et al., 2013b). Although the two channel system is simple in implementation, there is very limited scope for performance improvement. Due to the volume conduction of skull and scalp, the signal generated by the cortex of interest may be distributed to a vicinity of the area on the scalp and picked up by surrounding electrodes with different amplitude (Blankertz et al., 2008). To further enhance the classification performance, more data from EEG channels is often required for more complex problems.

The increase in the number of EEG channels poses two challenges. First, as the number of channel increases, direct identification of the subject-dependent reactive band becomes infeasible and an automated identification process is required. Secondly, as the feature vector includes all frequency estimates from all the channels, the dimension increases drastically. High dimensionality in feature vector will effect the performance of the classifier (Muni et al., 2006).

The above mentioned two problems are related as they both require optimization of the features to reduce the overall dimensionality and preserve the class-related information. For BCI application, the common spatial filter (CSP) is usually employed to solve the above two problems by reducing the whole EEG montage to a few spatial filtered channels in which the task-related information is significant (Lotte, 2014; Meng et al., 2015). The CSP works to find a projection that maximizes signal variance in one class while minimizing the variance in another class (Lotte and Guan, 2011). It has been shown in Wang et al. (2013a) that the application of the Tikhonov regularized CSP (TRCSP) to pre-process the EEG signal, and followed by the application of BMFLC for feature extraction provides a better performance as compared to the common average referencing or variant of Laplacian based spatial filters. As BMFLC decomposes the scalar measurement of each EEG channel into a multi-dimensional frequency estimation, the spatial filter that is optimized with respect to the bandpass filtered EEG signal may not be optimal for all frequency components.

In general, the above mentioned problems can be considered as a feature optimization problem (Chen et al., 2009; Estévez et al., 2009). However, the popular greedy search-based feature selection algorithm are not ideal to deal with multi-EEG channels due to its computational complexity. In the worst case scenario, to identify a optimal subset of a n-dimension feature vector, it would require all possible combinations of the n-dimension vector and that results in repetition of the cost function evaluation by P<sup>n</sup> <sup>r</sup> <sup>=</sup> <sup>1</sup> C n r times. With increase in n, it will be no longer feasible due to computational load. As an alternative, the EA-based feature optimization, which encodes a feature subset in its solution and then optimizes it with respect to the performance index of a classifier, could be employed (Muni et al., 2006; Aberg and Wessberg, 2007; Aler et al., 2012). The computation load of the EA-based algorithm is limited by the number of maximum generations or function evaluations. Further, it is less intensive in computation demand as compared to the greedy-search based algorithms. A genetic type EA-based feature optimization algorithm is popular for the binarized solution to find the optimal feature subset (Banerjee et al., 2007). Due to this reason, it is not suitable for optimizing real-valued function. With the recent development of real-valued EA, the classifier optimization and the feature selection can be performed in one setting (Aberg and Wessberg, 2007).

The application of EA also provides us with a new way of estimating the spatial filter. As the common spatial filter estimation is a maximization process, the cost function is generally limited to a convex form for efficient application of the existing optimization algorithms (Lotte and Guan, 2011). With the help of real-valued EA algorithms, direct optimization of the spatial filter with respect to the classification accuracy can be performed where the convexity may not be preserved in the cost function. In Aler et al. (2012), the spatial filter was estimated by using the covariance matrix adaptation evolution strategy (CMA-ES), a type of EA.

In this work, we employ two real-valued EAs for optimizing the features that are produced by BMFLC in a multi-channel EEG. Both spatial filter and feature selection are encoded in the solution vector and optimized simultaneously. Two realvalued EA's, namely global and local real-coded genetic algorithm (GLGA) and CMA-ES, are employed for optimization. The above two algorithms are selected due to their stable performance on various real-valued practical problems (García-Martínez et al., 2008; Hansen et al., 2009). The result of the proposed EA-based feature optimization (BMFLC-EA) is compared with the BMFLC with CSP, and BMFLC. Our results show that the BMFLC with EA outperforms the rest. The preliminary results of this work has been published in Wang et al. (2015). In the current version, we have modified the optimal number of spatial filters selection procedure, and provide a comprehensive comparison with the existing methods.

The remaining paper is organized as follows. Three configurations of BMFLC based classifiers are proposed in Section 2, where the necessary details of the algorithm are provided. The performance of the proposed method and comparison to the existing methods are presented in Section 3. Section 4 concludes this paper.

# 2. METHODS

In this section, three different configurations are proposed that are based on BMFLC for feature extraction. Various methods employed for this purpose are described in brief.

# 2.1. Proposed BCI Architecture

An overview of the EEG signal processing employed in this work is shown in **Figure 1**. The BCI system can be divided into a training phase and a testing phase. The feature optimization and classifier training is executed during the training phase. Once the optimal feature set and classifier has been obtained, the system can shift to the testing phase where feature optimization is no longer required. The reported classification accuracy is estimated during the testing phase of the BCI system. Before application to any feature extraction and optimization algorithm, the multiple channel EEG signals are first band-pass filtered with a Butterworth fifth order filter to the desired frequency band. To analyze the motor imagery, the µ rhythm is employed in the current study.

Three configurations that are based on BMFLC are discussed in this work. The major differences between three BCI configurations lie in how the spatial filter is employed in combination with the BMFLC. The BMFLC is employed to obtain time-frequency mapping of the EEG signal in each configuration. The spatial filter is employed to reduce the number of electrodes to a smaller subset. It can substantially reduce the dimension in the feature vector.

The work flow of the three configurations is shown in **Figure 2**. Configuration 1 shown in **Figure 2** is the simplest of the three configurations. As it is well-known that channels C3 and C4 in 10/20 systems have distinct patterns during motor imagery tasks, EEG signal from these two channels are selected. Thus, the spatial filter is formulated by a manual selection of channels in this configuration. The BMFLC is directly applied to the EEG data of C3 and C4.

Second configuration uses the Tikhonov regularized common spatial filter (TRCSP) to reduce the number of electrodes. Prior to the application of TRCSP, the EEG signal of each channel is band-pass filtered into the frequency band of interest, i.e., µ band. TRCSP then uses the band-pass filtered EEG to estimate spatial filters by maximizing the signal variance among classes. The number of spatial filters that are finally employed in the testing phase depends on the end application. The regularization parameter "α" in TRCSP is optimized based on the training data using the procedure developed in Lotte and Guan (2011). Later, the raw EEG signal in the testing set is multiplied by the selected spatial filters. The BMFLC is then employed to decompose the spatial filtered EEG signal to form the feature vector for the classifier.

Configuration 3 incorporates EA for estimating the spatial filter. In comparison with the earlier two configurations, the BMFLC is applied ahead of the spatial filter estimation. The dimension of the decomposed signal from BMFLC contains the full spectrum information with high dimensionality. EA is thus employed to find spatial filter in order to reduce the dimensionality of the decomposed signal. Compared to

the optimization of the spatial filter in configuration 2, the optimization criterion in EA is the classification performance rather than the variance of the EEG signal in different classes. The importance of this difference will be more clear in the Results Section. Moreover, to further reduce the dimensionality of the feature vector, feature selection is also conducted by finding a subset of frequency components in the whole band that is provided by the BMFLC with EA.

### 2.2. BMFLC-KF Based EEG Feature Extraction

Common to all three configurations, the BMFLC is applied to decompose the EEG signal to obtain time-frequency mapping of the α band. The amplitude information of each frequency component at each time instant or averaged over a fixed moving window can be employed to form the feature vector for the classifier.

BMFLC divides a pre-defined frequency band [ω1, · · · , ωn] into n equally distributed divisions with frequency spacing 1<sup>f</sup> , and estimates the amplitude of each frequency component by using the Kalman filter as shown below (Wang et al., 2012b):

$$\mathbf{y}\_k = \mathbf{x}\_k^T \mathbf{w}\_k + \boldsymbol{\nu}\_k \tag{1}$$

$$\mathbf{w}\_{k+1} = \mathbf{w}\_k + \eta\_k \tag{2}$$

where **x**<sup>k</sup> and **w**<sup>k</sup> are defined as

$$\mathbf{x}\_{k} = \begin{Bmatrix} \begin{bmatrix} \sin(\omega\_{1}k)\sin(\omega\_{2}k) \cdots \sin(\omega\_{n}k) \end{bmatrix}^{T} \\\\ \begin{bmatrix} \cos(\omega\_{1}k)\cos(\omega\_{2}k) \cdots \cos(\omega\_{n}k) \end{bmatrix}^{T} \end{Bmatrix} \tag{3}$$

$$\mathbf{w}\_k = \begin{cases} \begin{bmatrix} a\_{1k} \ a\_{2k} \ \cdots \ a\_{nk} \end{bmatrix}^T \\\\ \begin{bmatrix} b\_{1k} \ b\_{2k} \ \cdots \ b\_{nk} \end{bmatrix}^T \end{cases} \tag{4}$$

Assume that v<sup>k</sup> and η<sup>k</sup> are independent Gaussian processes with 0 mean and covariances of R and **Q**, respectively. The Kalman filter is employed for the adaptation of weights estimate **w**ˆ <sup>k</sup> = **E**[**w**<sup>k</sup> |**y**k−<sup>1</sup> ]. The detailed implementation of BMFLC-KF is provided in the Supplementary Material (S.2). The weight vectors of BMFLC represents the Fourier coefficients of the band-limited signal. The estimated weights are further combined as follows:

$$\mathbf{W}\_k = \left[ \sqrt{a\_{1k}^2 + b\_{1k}^2} \cdot \cdots \sqrt{a\_{nk}^2 + b\_{nk}^2} \right]^T \tag{5}$$

where **W**<sup>k</sup> is the absolute weight vector of the frequency components at time instant k. The time-frequency mapping of a given signal is obtained by cascading the weights in **D** = [**W**1, . . . , **W**n].

#### 2.3. EA-Based Spatial Filter and Feature Selection Optimization

As the usage and implementation of TRCSP in configuration 2 is straightforward, for the sake of brevity, the details are provided in Supplementary Material (S.1). In this subsection, we focus on developing EA-based spatial filter and feature selection optimization.

As EA has no limitations in the formation of solution vector, we need to encode the spatial filter and feature selection into the solution vector and then employ EA to optimize the solution vector with respect to the same cost function. Since the final objective of applying both spatial filter and feature selection is to improve the performance of the classifier, the cost function is selected to be the error rate of the classifier.

In this work, the covariance matrix adaptation evolution strategy (CMA-ES) and, global and local real-coded genetic algorithm (GLGA) are used due their superior performance in real-valued optimization (García-Martínez et al., 2008; Hansen et al., 2009). The algorithms for GLGA and CMA-ES are discussed in the following subsection. For more detailed information, please refer to S.3 and S.4 in Supplementary Material, respectively.

#### 2.3.1. Global and Local Real-Coded Genetic Algorithm (GLGA)

Although we can encode a real-valued quantity into a binary string via some transformation and using traditional genetic algorithm (GA) (Hansen et al., 2009), the dimension of the solution becomes large as the number of required spatial filter grows. In this work, the GLGA, a real-valued version of GA, is employed (García-Martínez et al., 2008).

GLGA uses several operators to balance the exploration and exploitation. First, the solution vector in the current population is divided into male and female group by using female and male differentiation (FMD) operator. GLGA then uses parentcentric BLX-α (PBX-α) crossover operator to generate offspring. The PBX-α takes one solution vector from each group, the generated offspring that lies in the vicinity of the female solution. The distance between the offspring and the female solution is governed by the selected male solution and a parameter α. The parameter α determines the spread of offspring to its parents.

The uniform fertility selection (UFS) and the negative associative mating (NAS) are employed for the selection of female and male parents, respectively(García-Martínez et al., 2008). To apply UFS, a number of elite solutions in the current population are selected as a female parent. The number of times that a solution vector is used as female is tracked during the evolution. The less frequent solution is then selected as female parent. For selection of the male parent, the roulette wheel methods are applied to selected five candidate male parents. The Euclidean distance between the selected female parent and each of the possible male parent is calculated. The one with the highest distance is then selected as the male parent.

GLGA balances the global and local search by tuning the number of female and male parents. Let N<sup>m</sup> and N<sup>f</sup> denote the number of male and female parents, respectively. With a large Nf , GLGA focuses on exploring the solution space, whereas a large N<sup>m</sup> will restrict GLGA to concentrate on local area. GLGA is not operated by generation, rather it uses function evaluation as the criterion to shift from global to local search. GLGA-n% indicates that n% of function evaluations are used for global search and rest are used for local search. After certain number of function evaluations, GLGA assumes that the solutions have already entered into the vicinity of the global optimal, it thus reduces N<sup>f</sup> to fine tune the solution in that region. To maintain the population size unchanged during evolution, the "replace the worst strategy" is adopted. Each time when an offspring is generated, the fitness value of the offspring is compared to the solution in the current population. If it is better than any of the solution in the current population, the worst one is replaced.

#### 2.3.2. CMA-ES

CMA-ES uses the second order statistics estimated from the solution space to find the optimal value of the target function. It is thus well-suited for optimizing the function with a complicated structure. It has been applied to both artificial functions and real practical problems and has shown to be performing better than the contemporary counterparts (Hansen et al., 2009; Elteto et al., 2012).

The (µ, λ) CMA-ES requires λ number of solutions in each generation, and select the best µ number of solutions for updating the value in next step. The CMA-ES algorithm generates the solution s (g) k by sampling from a Gaussian distribution with mean m , step-size δ and covariance estimation **C**, where g and k are indexes for generation and solution, respectively. To find the solution for the next generation, CMA-ES estimates the mean, step-size, and covariance with the solution in the current generation considering the history path of the solution evolution. After each generation, the solutions are ranked based on their fitness value. The update of mean is carried out by a weighted summation of the µ best solutions in the current generation. The value of **C** is determined by the covariance of the estimated evolution path denoted by p<sup>c</sup> and the covariance estimated from the µ number of solutions in the current generation.

#### 2.3.3. Optimization of Spatial Filter and Feature Selection

The key to the application of EA for optimizing the spatial filter and feature selection simultaneously is to develop a proper solution vector. The spatial filter is real-valued by default, unlike the genetic algorithm based feature solution. The binary valued feature selection is replaced with real-valued problem. It was shown in earlier studies that the subject-specific band has a maximum bandwidth of 3 Hz (Veluvolu et al., 2012). Therefore, the EA is employed to search for the starting frequency and the optimal bandwidth of the subject-specific band. Moreover, the dimensionality of the solution vector is significantly reduced by encoding the frequency selection in real-value compared to the binary encoding scheme. As the number of function evaluations required for a EA to converge depends on the dimension of the solution vector (Herrera et al., 1998), this approach should be able to reduce the computational complexity and further improve the solution quality when a fixed number of function evaluations is considered.

Thus, the solution vector for GLGA and CMA-ES is constructed as follows:

$$sol = \left[ \overbrace{sf\_{11}, \dots, sf\_{1M}}^{\text{Spatial Filter}} \overbrace{sf\_{21}, \dots, sf\_{2M}, \dots}^{\text{Frequency selection}} \overbrace{\text{FS}, \overbrace{BW}^{\text{S selection}}}^{\text{Frequency selection}} \right]$$

where sfij denotes the jth component in ith spatial filter, FS is the starting frequency, and BW is the bandwidth. M is the total number of EEG channels. The number of resultant spatial filters depend on the application. For motor imagery BCI system, the number of classes is usually chosen as the number for spatial filters selection (Aler et al., 2012). As the EA is applied to the time-frequency mapping of BMFLC, FS has to lie in the range of [f1, f<sup>n</sup> − BW], where f<sup>1</sup> and f<sup>n</sup> are the lower and upper cutoff frequency of BMFLC, respectively. As shown in Veluvolu et al. (2012), BW is constrained to have a maximum 3 Hz band.

The classification error obtained from linear discriminant analysis (LDA) is employed as the optimization criterion for GLGA and CMA-ES. The frequency weights that are estimated from BMFLC-KF are used as the original feature set. To optimize the spatial filter and frequency selection, the frequency weights obtained from each channel is first combined to the following form:

$$\mathbf{F}\_k = \begin{bmatrix} \mathbf{w}\_{k,1}^{fe} & \dots & \mathbf{w}\_{k,M}^{fe} \\ \vdots & \vdots & \vdots \\ \mathbf{w}\_{k,1}^{fe} & \dots & \mathbf{w}\_{k,M}^{fe} \end{bmatrix}^T \tag{6}$$

where **w** fs k,M indicates the weights of frequency fs at time instant k in channel M. Then, the best solution vector in the current generation is separated into two parts, frequency selection and spatial filter. The optimal frequency band is extracted form the original feature space by setting fs = FS and fe = FS + BW. The spatial filter component of the solution vector is re-organized as follows:

$$\mathbf{SF} = \begin{bmatrix} sf\_{11} & \dots & sf\_{1M} \\ sf\_{21} & \dots & sf\_{2M} \end{bmatrix}^T \tag{7}$$

To obtain the optimized feature set, the spatial filter **SF**<sup>T</sup> is multiplied with the **F**<sup>k</sup> , where superscript T denotes matrix transpose. The resultant feature vector as well as the corresponding label are then sent to the classifier. The error rate of the employed classifier is used to guide the EA to search for a better spatial filter and feature set. To avoid over-fitting, only the classification error obtained from the validation data is used. The solution vector obtained at the end of the evolution contains the optimized spatial filter and frequency selection.

#### 2.4. Dataset Description

To test the performance of the proposed architecture, we choose two publically available BCI datasets. The first dataset (denoted as Dataset I) is from Brain Computer Interface Competition IV (Brunner et al., 2007). Dataset I contains EEG data of nine subjects. EEG was recorded from 22 Ag/AgCl electrodes sampled at 250 Hz. All signals were recorded monopolarly with the left mastoid as reference and right mastoid as ground. Four classes of cue-based motor imagery tasks were carried out, namely the imagination of the movement of the left hand, right hand, both feet and tongue. Each subject data was recorded in two sessions on separate days. Each session consisted of six runs separated by short breaks. One run consisted of 48 trials. During the recording, the subjects sat on a comfortable armchair in front of a computer screen. At the beginning of each trial (t = 0s), a fixation cross appeared on the black screen. Two seconds later, a cue in the form of an arrow pointing either to the left, right, down, or up displayed on the screen and lasted 1.25s. The subjects were asked to perform the motor imagery task until the fixation cross disappeared from the screen at t = 6 s. For more details about data collection, see Brunner et al. (2007).

The second dataset (denoted as Dataset II) was taken from the BCI competition 2003, dataset 2a (McFarland et al., 1997). The dataset contains motor imagery data from three subjects. All subjects were informed to modulate their sensorimotor rhythm to move a cursor from the left edge of the screen to one of the four designated locations appeared on the right edge of the screen. In this paper, we chose the top and bottom target out of totally four available targets. Totally, 64 channels EEG signal were recorded according to the international 10/20 system with a sampling frequency of 160 Hz. Each subjects participated 10 sessions of the experiment. In this work, the top and bottom target in the training session (session 1–6) was employed.

#### 3. RESULTS

#### 3.1. Experiment Settings

As the EEG α band is of interest, the frequency range for BMFLC-KF is set to f<sup>1</sup> = 6 Hz and f<sup>n</sup> = 14 Hz. 1f is set to 0.5 Hz as it has been shown to offer better results in EEG signal decomposition (Wang et al., 2012b). For applying CMA-ES algorithm, the population size λ is set to be five times the solution dimension. The number of spatial filters are selected to be equal to the number of classes in the dataset. The reason for this selection will be discussed in the later section.

The CMA-ES starts by setting the initial value to s (0) , δ (0) , **C** (0) , and updating the solution and is similar to the implementation provided in Hansen and Kern (2004). The parameters provided in Hansen et al. (2009) are employed as they are shown to offer robust performance.

For GLGA, the parameters are chosen from García-Martínez et al. (2008). α is equal to 0.5 and 25% of function evaluations are employed for global search. For a fair comparison between CMA-ES and GLGA, the functional evaluation of GLGA is set to 10,000, as it approximately matches to the function evaluations used by CMA-ES. N<sup>m</sup> = N and N<sup>f</sup> = N/2 are used for global searching and N<sup>m</sup> = 100 and N<sup>f</sup> = 5 are employed for local searching, where N is the population size. The algorithm terminates when the stopping criterion is met.

For performance analysis, the EEG dataset is divided into training set and testing set according to 10-times cross validation scheme. Feature optimization including CSP and EA based methods are performed on the training set. The same training set is employed to train the final classifier. The classification accuracy which is obtained on the testing data is employed for performance comparison. It is important to note that, since the GLGA and CMA-ES relies on classification error as the cost function, a separate 10-times cross validation is also applied to the training set.

Another important factor that affects the performance is the number of spatial filter pairs employed for each algorithm/configuration. In this paper, the number of spatial filter pairs is selected for each algorithm separately to ensure optimal performance for each configuration. The rationale and procedure for this selection will be discussed in the later section.

For the sake of comparison, we consider other feature extraction and optimization algorithms that have been reported to have superior performance on BCI motor imagery tasks. The continuous wavelet transform (CWT) with was shown to have accurate time-frequency resolution in analyzing the EEG signal (Hsu, 2015). Thus, it has been included in our comparison analysis. We have also chosen Daubechies wavelets of order 4 (denoted as Db4) as suggested in Hsu (2015). The obtained wavelet features undergoes the same feature optimization procedure with CMA-ES (denoted as Db4-CMAES). Further, the filter bank common spatial pattern (FBCSP) is included as it was shown to have superior performance on the BCI competition dataset (Ang et al., 2012). In this work, we have implemented a BMFLC based FBCSP. In summary, the EEG signal is first decomposed by the BMFLC. Then, CSP filter was employed for each frequency estimate and the features were constructed as the logarithm power. The obtained features were then optimized by employing the mutual information criterion (Ang et al., 2012). Further, the algorithm developed in Aler et al. (2012) is also implemented (denoted as FFT-CMAES).

# 3.2. Performance of GLGA and CMA-ES Based Feature Optimization

The value of the cost function of CMA-ES for each generation is shown in **Figure 3** for all subjects. The shaded area indicates the standard deviation of the classification error obtained from 10-times cross validation scheme. For all the subjects, we observe that the classification error converges to a stable bound at the end of the evolution process. It can be noted that with CMA-ES, the error converges at a faster rate in the first 20 generations. Later on, the convergence rate is slowed up to 100 generations. After 100 generations, the classification error becomes stable. The simulations for 500 generations also yielded similar results.

Another observation from **Figure 3** is that the standard deviation varies significantly across subjects. A small standard deviations can be identified in subject-1, -7, and -9. Whereas, large standard deviation can be observed in the rest of the subjects. The large standard deviation implies that CMA-ES converges to different values at cross validation runs. Further, it may be also due to the complexity of the cost function employed. However, this divergence in the evolution process does not affect the performance of the final classifier with CMA-ES as shown in **Figure 7**. This result indicates the superiority of CMA-ES in the optimization of a complex function as compared to GLGA.

Similarly, the cost function evolution of GLGA is shown in **Figure 4**. The vertical line in **Figure 4** indicates the transition from global search to local search. The first observation is the slow error decay during the global search phase. After GLGA

moves to local search, initially, a fast error decay followed by a stable phase can be observed. The standard deviation is generally small during global search and the value increases when the algorithm approaches the end of the execution. The results suggest that GLGA tends to converge to different area after each global search during each cross validation run.

Comparing the error evolution of GLGA to CMA-ES, we first note that the GLGA has a higher standard deviation over all subjects, whereas the error evolution in CMA-ES varies with the subject with a smaller volatility. Moreover, CMA-ES converges to a small error bound at the end of the evolution process as compared to GLGA.

To further evaluate the performances of GLGA and CMA-ES, the overall error improvement of each algorithm is obtained for all subjects. The error improvement is calculated as the difference between the maximum error and the minimum error obtained during evolutionary process. It shows how well the employed EA explores the error surface. For comparison, the algorithm proposed in Aler et al. (2012) (named as FFT-CMAES) is also implemented and the results are shown in **Figure 5**. We observe CMA-ES based feature optimization outperforms the GLGA in terms of performance improvement. The BMFLC-CMAES shows superiority in performance compared to FFT-CMAES. BMFLC-GA has the lowest improvement during training. The error improvement is stable for all the subjects with FFT-CMAES, as shown by small standard deviation.

# 3.3. Performance of the Final Classifier for All Configurations

To perform the comparison for all different configurations, the number of spatial filters required for each configuration needs to be identified. As the dataset employed in this work consists of 22 EEG recordings for motor imagery tasks, we can therefore obtain up to 11 pairs of spatial filters for each subject. In this work, we select the number of spatial filters for each configuration based on the generalization performance index. This index is used to quantify the performance for the selected number of spatial filters over all subjects. To calculate this index for a configuration, we first obtain the classification accuracy for all subjects and all spatial filter pairs in the training data set. Then for a particular subject, the classification accuracy for all spatial filter pairs is normalized to the range of [0 1]. The employed unity-based normalization ensures that the optimal spatial filter pair for each subject has the performance index of 1, so that the difference in

classification accuracy among subjects will not bias the spatial filter selection. Then, the generalization performance index for each spatial filter pair is the normalized classification accuracy averaged over all subjects. The results of generation performance index for all configurations is shown in **Figure 6**.

From **Figure 6**, we can observe that all configurations tends to have better generalization performance for fewer number of spatial filter pairs. The CSP and FFT-CMAES display maximum generalization performance when one pair of spatial filters is employed. BMFLC-CMAES peaks at two pair spatial filters, whereas BMFLC-GA requires three pairs of spatial filters. Thus, the obtained number of spatial filter pair is set to each configuration for comparison. After optimizing the features by employing CSP, CMA-ES, and GLGA with pre-selected parameters and number of spatial filter pairs, a final classifier is then built with the same training set and then finally evaluated on the testing set.

The performance of all subjects in Dataset I averaged over 10-times cross validation is shown in **Figure 7**. Configuration 1 is denoted as BMFLC where no feature optimization has been employed. Configuration 2, where CSP is applied prior to BMFLC is denoted by CSP-BMFLC. The two variants of EA based feature optimization procedures are named as BMFLC-CMAES and BMFLC-GLGA, respectively.

In **Figure 7**, we observe that BMFLC-CMAES outperforms all other algorithms in seven out of nine subjects. Further, it has also the best performance when averaged over all subjects, and passes the Friedman test with α = 0.05. We also observe that the features obtained from BMFLC and later optimized by CMA-ES and GLGA display better performance compared to FFT-CMAES on average. The BMFLC-FBCSP has better performance as compared to BMFLC and CSP-BMFLC. The performance varies across subject in CSP-BMFLC and is subject-dependent. It is also clear that feature optimization improves the overall performance.

The classification performance with Dataset II is shown in **Table 1**. We observe that the BMFLC-CMAES and Db4- CMAES outperformed all other methods in mean accuracy. This

observation clearly demonstrates the superiority of employing time-frequency decomposition for feature extraction. Further, we notice that the BMFLC-GA is inferior to the BMFLC-FBCSP method.

We also compared the number of occasions of one algorithm outperforming another algorithm by accumulating the results from the cross validation runs of all subjects in Dataset I. Since 10 cross validation were employed, a total of 90 (9 subject × 10 cross validations runs) accuracy estimates per configuration were obtained. The results are shown in **Figure 8**. The total number and its corresponding percentage value by which one algorithm outperforms the other algorithm is provided in the middle of each block.

From **Figure 8**, it clearly demonstrates the necessity of feature optimization. The configurations that are combined with feature optimization have better classification accuracy when BMFLC based features are employed. The results in the first and second row when compared to the third row of **Figure 8** further demonstrate the superiority of BMFLC and Db4 in extracting information from EEG as compared to FFT.



Moreover, we also observe that all configurations that involve feature optimization outperform the traditional methods. The high standard deviation obtained with GLGA (**Figure 4**) is also reflected in the classification results as its overall performance is lower compared to BMFLC-CMAES.

To further consolidate our observed results, we have performed a repeated measure of ANOVA by collecting all accuracy estimates from all methods on all subjects. As the obtained accuracy estimates violated the sphericity assumption, Greenhouse-Geisser correction has been employed. Results show that the mean accuracy differences are statistically significant among all methods [F(4.037,359.252) = 27.904, p < 0.01]. Post-hoc tests using the Bonferroni correction revealed that the proposed BMFLC-CMAES outperforms all comparison methods with p < 0.01. All methods with feature optimization show superior performance as compared to BMFLC (p < 0.05). It highlights the necessity of employing feature optimization in BCI applications. Furthermore, the BMFLC-CMAES and Db4- CMAES show statistically significant accuracy improvement over CSP-BMFLC and FFT-CMAES (p < 0.01), which indicates the merit of using time-frequency decomposition to expand the feature space.

#### 3.4. Runtime Complexity

To show the applicability of the proposed methods to the realtime BCI applications, we conducted a runtime computational complexity analysis. The reported runtime estimates for each algorithm are obtained on a system with Intel(R) Core i5-6500 CPU and 8GB RAM. The time duration was measured using Matlab (Mathworks, USA) inbuilt function. Here, we reported the average time duration to finish one validation run. The obtained runtime estimates for all feature optimization methods are given in **Table 2**.

The runtime requirement of the proposed feature optimization procedure consists of the feature extraction, optimization, and classification. The feature extraction method BMFLC has been shown to have superior computational complexity as compared to STFT and CWT (Wang et al., 2013b). In this work, we estimated that the time required for BMFLC to decompose a 2 s EEG epoch is 0.0349s ± 0.0037. Also, the computational demand of LDA is negligible as only linear operation is involved. Therefore, the main contributor to the computational complexity of the proposed algorithm is the time taken by the employed optimization algorithm.

All EA based feature optimization algorithms require few minutes of run-time, as shown in **Table 2**, to attain optimized features. Compared to the CSP based methods, the EA based feature optimization has much higher computational demands. However, the feature optimization is usually conducted during the training phase of the BCI application. The feature optimization only needs to be finished in a reasonable time. We estimated that the BMFLC-CMAES requires 7 min to find the



optimized feature subspace. Once the optimized features have been identified during training, the system only needs to perform feature extraction and classification in real-time.

# 4. DISCUSSION

In this work, different BCI configurations which employ BMFLC based features for classification have been developed. As our results suggest, BMFLC-KF with manually selected channels has the lowest classification performance. However, without any optimization, the BMFLC feature offers the most stable performance. There are two possible reasons for this. First, the BMFLC-KF decomposes the EEG signal with high accuracy and this provides the classifier with a stable feature set. Secondly, the stability in the classification results may also indicate the lack of diversity and limited information available within the two channels. This result also highlights the necessity for employing the optimization for feature extraction.

It is evident that the performance of CSP-BMFLC is subject dependent. For three out of nine subjects in Dataset I, the CSP-BMFLC has good testing accuracy. However, for the rest subjects, the CSP-BMFLC performance is similar to or even worse than applying BMFLC alone. As CSP algorithm relies on the training data to find the spatial filter, the unsatisfactory performance in test data may be due to over-fitting of the training data.

Comparing the results from the configurations which involve feature optimization, the BMFLC-CMAES outperforms Db4-CMAES, BMFLC-GA, FFT-CMAES, and BMFLC-FBCSP. In fact, BMFLC-CMAES, and Db4-CMAES have the best performance among all configurations considered in this work. Our results indicate that the feature optimization is critical in improving the testing performance for a given feature set.

We also observe that the performance of each EA involved configuration differs. BMFLC based system is degraded when GLGA is employed for feature optimization. As GLGA divides the evolution into two distinct phases, global, and local searches, the overall performance of GLGA depends on the quality of the global search phase. If the global search fails in finding the promising area, then the subsequent local search may not improve the results and this may introduce severe overfitting. However, if the global search successfully enters into the promising area, then the local search can fine-tune the solution and this results in good performance as shown in subject 3 and subject 8 in **Figure 7**.

Among all feature optimization methods that employ BMFLC, the BMFLC-CMAES has the best performance. Whereas, the BMFLC-GA provides inferior performance as compared to BMFLC-FBCSP. This result indicates that the careful selection of optimization algorithm is also critical to identify the suitable feature subset. Our results suggests that the CMAES is preferable for feature optimization.

BMFLC-CMAES, Db4-CMAES, and FFT-CMAES differ only in the feature extraction algorithm. The superior performance of BMFLC-CMAES and Db4-CMAES as compared to FFT-CMAES highlights the ability of time-frequency decomposition in extracting the information from motion induced EEG signal as compared to the traditional FFT. Moreover, the BMFLC-CMAES provides better performance as compared to Db4- CMAES. However, the improvement failed to attain statistical significant level (p < 0.01). The current results are in line with our previous work that BMFLC has similar performance in terms of time-frequency decomposition of EEG signal (Wang et al., 2013b). However, as shown in Wang et al. (2013b), the BMFLC has less computational complexity as compared to CWT. Thus, it is more suitable for the real-time BCI applications.

The superiority in classification performance obtained from EA based feature optimization comes with a price. All EA based configurations require much higher computational resources compared to the rest of the configurations employed in this work. The computational complexity of the EA based feature optimization grows with the increasing number of EEG channels and the frequency band of interest. However, the frequency band for a BCI system is usually limited and the feature optimization is only required during training phase. We show that the required training time for BMFLC-CMAES is around 7 min, which is tolerable in the current BCI applications. Thus, the computational burden posed by the EA has little impact to the real-time applicability of the proposed feature optimization algorithm.

Feature optimization is mainly intended to identify a suitable subset of features from the original feature space which can improve the classification accuracy and mitigate the requirement in computational demand. The most reliable method to solve the feature optimization problem is to enumerate all possible feature combinations from the the original feature space. However, such enumeration is only possible when the number of features in the original feature space is small. It has been shown in a fNIRS based BCI application that by selecting two out of six features, the testing accuracy is maximized (Naseer et al., 2016). However, the feature dimension in the current work is 17 per channel. With a 22 channels EEG recording, it results in a 374 dimensional feature vector. To enumerate on all combinations on such high feature space requires high computational requirements. The case is even worse when the number of available channels increases. Therefore, an evolution algorithm based approach was desirable to optimize the features in such high dimensional space.

Classification performance of the BCI applications can be improved with support vector machine (SVM) as the classifier (Schlögl et al., 2005; Naseer, 2014; Khan and Hong, 2015). The training of SVM is however computationally extensive when compared to a LDA classifier (Khan and Hong, 2015). Furthermore, to obtain better performance with SVM, its hyperparameters need to be fine-tuned (Vapnik, 1995; Suykens et al., 2002), which increases the computational cost for training a BCI system considerably. The main aim of the current paper is to show the impact of the feature exaction and feature optimization on the performance of BCI system. Therefore, for the illustration of classification performance with proposed optimal feature selection technique, we chose LDA as the classifier in this work. For completeness, we report that one can chose any advanced machine learning techniques as a classifier for the proposed feature extraction and optimization scheme.

# 5. CONCLUSION

In this work, the performance of BMFLC-KF with various combinations of FBCSP,CSP, CMA-ES, and GLGA for feature optimization is quantified with a two-class BCI application. Two publically available BCI competition datasets are employed to study the performance of the proposed feature extraction and feature optimization algorithm. The evolutionary algorithms (namely CMA-ES and GLGA) are employed to optimize the spatial filter and feature selection simultaneously. It is clear from our results that the performance of the final classifier can be improved by employing any of the above mentioned feature optimizing algorithms. Statistical analysis reveals that the proposed BMFLC-CMAES based feature optimization algorithm outperforms its counterparts. Moreover, the superior performance of time-frequency features obtained from BMFLC and CWT also highlights the necessity of employing time-frequency decomposition for feature extraction in BCI applications. The small execution time of CMA-ES indicates that it is more suitable for feature optimization as compared to GLGA.

# AUTHOR CONTRIBUTIONS

This work was carried out by YW under the supervision of KV.

## FUNDING

This research was supported by the Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education, Science and Technology under the Grant NRF-2014R1A1A2A10056145 and in part by the BK21 Plus project funded by the Ministry of Education, Korea (21A20131600011).

## SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: http://journal.frontiersin.org/article/10.3389/fnins. 2017.00028/full#supplementary-material

#### REFERENCES


BCI applications. J. NeuroEng. Rehab. 10:109. doi: 10.1186/1743-0003- 10-109


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2017 Wang and Veluvolu. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Decoding Lower Limb Muscle Activity and Kinematics from Cortical Neural Spike Trains during Monkey Performing Stand and Squat Movements

Xuan Ma<sup>1</sup> , Chaolin Ma2, 3 \*, Jian Huang<sup>1</sup> \*, Peng Zhang<sup>1</sup> , Jiang Xu<sup>4</sup> and Jiping He1, 3, 5, 6 \*

*<sup>1</sup> Neural Interface and Rehabilitation Technology Research Center, School of Automation, Huazhong University of Science and Technology, Wuhan, China, <sup>2</sup> Center for Neuropsychiatric Disorders, Institute of Life Science, Nanchang University, Nanchang, China, <sup>3</sup> Center for Neural Interface Design, School of Biological and Health Systems Engineering, Arizona State University, Tempe, AZ, USA, <sup>4</sup> Department of Rehabilitation Medicine, Tongji Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, China, <sup>5</sup> Collaborative Innovation Center for Brain Science, Huazhong University of Science and Technology, Wuhan, China, <sup>6</sup> Advanced Innovation Center for Intelligent Robots and Systems, Beijing Institute of Technology, Beijing, China*

#### Edited by:

*Dingguo Zhang, Shanghai Jiao Tong University, China*

#### Reviewed by:

*Vassiliy Tsytsarev, University of Maryland, Baltimore, USA Victor Manuel Pulgar, Wake Forest School of Medicine, USA GuangYe Li, Shanghai Jiao Tong University, China*

#### \*Correspondence:

*Chaolin Ma chaolinma@ncu.edu.cn Jian Huang huang\_jan@mail.hust.edu.cn Jiping He jiping.he@asu.edu*

#### Specialty section:

*This article was submitted to Neural Technology, a section of the journal Frontiers in Neuroscience*

Received: *01 October 2016* Accepted: *20 January 2017* Published: *07 February 2017*

#### Citation:

*Ma X, Ma C, Huang J, Zhang P, Xu J and He J (2017) Decoding Lower Limb Muscle Activity and Kinematics from Cortical Neural Spike Trains during Monkey Performing Stand and Squat Movements. Front. Neurosci. 11:44. doi: 10.3389/fnins.2017.00044*

Extensive literatures have shown approaches for decoding upper limb kinematics or muscle activity using multichannel cortical spike recordings toward brain machine interface (BMI) applications. However, similar topics regarding lower limb remain relatively scarce. We previously reported a system for training monkeys to perform visually guided stand and squat tasks. The current study, as a follow-up extension, investigates whether lower limb kinematics and muscle activity characterized by electromyography (EMG) signals during monkey performing stand/squat movements can be accurately decoded from neural spike trains in primary motor cortex (M1). Two monkeys were used in this study. Subdermal intramuscular EMG electrodes were implanted to 8 right leg/thigh muscles. With ample data collected from neurons from a large brain area, we performed a spike triggered average (SpTA) analysis and got a series of density contours which revealed the spatial distributions of different muscle-innervating neurons corresponding to each given muscle. Based on the guidance of these results, we identified the locations optimal for chronic electrode implantation and subsequently carried on chronic neural data recordings. A recursive Bayesian estimation framework was proposed for decoding EMG signals together with kinematics from M1 spike trains. Two specific algorithms were implemented: a standard Kalman filter and an unscented Kalman filter. For the latter one, an artificial neural network was incorporated to deal with the nonlinearity in neural tuning. High correlation coefficient and signal to noise ratio between the predicted and the actual data were achieved for both EMG signals and kinematics on both monkeys. Higher decoding accuracy and faster convergence rate could be achieved with the unscented Kalman filter. These results demonstrate that lower limb EMG signals and kinematics during monkey stand/squat can be accurately decoded from a group of M1 neurons with the proposed algorithms. Our findings provide new insights for extending current BMI design concepts and techniques on upper limbs to lower limb circumstances. Brain controlled exoskeleton, prostheses or neuromuscular electrical stimulators for lower limbs are expected to be developed, which enables the subject to manipulate complex biomechatronic devices with mind in more harmonized manner.

Keywords: lower limb motor control, primary motor cortex, cortical neuronal encoding, electromyography (EMG), Kalman filter, nonhuman primates

## 1. INTRODUCTION

Brain machine interface (BMI), which translates the activity of cortical neurons into specific motor output commands, has been assumed as a promising approach to help patients who have lost normal motor functions due to spinal cord injury or other neural impairments to restore fundamental skills in daily living (Serruya et al., 2002; Andersen et al., 2004; Schwartz, 2004; Lebedev and Nicolelis, 2006; Hatsopoulos and Donoghue, 2009). Extensive literatures have shown successful realization of the concept of BMI on both nonhuman primates (NHP) and human subjects (Wessberg et al., 2000; Taylor et al., 2002; Carmena et al., 2003; Hochberg et al., 2006; Kim et al., 2008; Velliste et al., 2008; Hwang and Andersen, 2009; Carpaneto et al., 2011; Hochberg et al., 2012; Wodlinger et al., 2014). Among the existing BMI studies, the majority is concerned with using motor cortical activity to decode the movements of upper limbs. Previous literatures have shown two-dimensional or three-dimensional hand/arm trajectories when monkey performing cursor tracking, joystick playing, target-pursuit or center-out tasks can be extracted from neural spike trains using linear regression (Moran and Schwartz, 1999; Gao et al., 2003; Hatsopoulos et al., 2004; Kemere et al., 2004; Paninski et al., 2004; Linderman et al., 2008), artificial neural networks (ANN) (Fang et al., 2010; Huang et al., 2011; Wang et al., 2014), or different variants of Bayesian filtering algorithms (Wu et al., 2002; Brockwell et al., 2004; Wu et al., 2006; Li et al., 2009; Wu et al., 2009; Wang and Principe, 2010; Kang et al., 2015; Menz et al., 2015; Hotson et al., 2016). Analogous approaches were also proposed for predicting arm electromyograph (EMG) signals from the discharge activity of an ensemble of motor neurons or multi-channel local field potentials (Morrow and Miller, 2003; Pohlmeyer et al., 2007; Nazarpour et al., 2012; Shin et al., 2012; Liao et al., 2015). In fact, among the existing population of neural trauma victims, a large portion suffer from the inability of standing and walking (Qin et al., 2010). With the expectation of helping them, researchers have begun to seek possible realizations of BMI systems for lower limb motor function restoration. Some studies have tried to unscramble the information correlating with user intentions (Kilicarslan et al., 2013) or lower limb kinematics (Presacco et al., 2011, 2012; He et al., 2014; Luu et al., 2016) or surface EMG signals (Paek et al., 2013) during treadmill walking or sitting/standing (Bulea et al., 2014) from noninvasive scalp electroencephalographic (EEG) signals. To gain more specific information about how cortical neurons modulate their activities to initiate and control lower limb movements, it is expected to directly collect neural activity from the cerebral cortical areas related to lower limbs using invasive electrodes. However, there are relatively scarce reports about these due to the lack of proper experimental paradigms or apparatus and the difficulties in locating the exact functional cortical areas related to lower limb movements compared to the cases in upper limb. Nicolelis et al. (Fitzsimmons et al., 2009) extracted kinematic parameters for monkey bipedal walking from cortical neuronal ensemble activity and proposed possible approaches to control artificial actuators that reproduce walking patterns.

The construction of BMI lies on the detection of cortical neuronal representations of limb movements. Generally speaking, the control and organization of movements are often considered different for upper limb and lower limb (Winters and Woo, 2012). For example, bipedal locomotion of human is dynamically controlled by the skillfully coordinated activations of the redundant lower limb muscles. Under normal conditions, the activations of these muscles are controlled by both specialized neural circuits in spinal cord known as central pattern generator (CPG) and command signals directly from the brain (Ogihara and Yamazaki, 2001; Vogelstein et al., 2006). Here we concentrate a fundamental lower limb voluntary movement: stand and squat. We previously reported a system for training monkeys to perform visually guided stand and squat tasks (Ma and He, 2010). Neuronal ensemble activity from the representation area of the lower limbs in the primary motor cortex (M1), subdermal intramuscular EMG signals in lower limb muscles, and motion trajectories were recorded synchronously with the task. In an acute data recording session, we have detected many M1 neurons tuned to certain task events (Ma et al., 2015).

In this study, we immediately followed our previous work to investigate issues related to neural decoding toward BMI applications. To what extent do neurons in M1 participate the initiation or maintenance of lower limb voluntary movements? Are the neurons showing correlation with certain task events we have observed sufficient for decoding both lower limb muscular activity and kinematics? Which computational models are better for describing the neuronal representation of such visual guided stand/squat movements? We were also curious about whether analogous methods which have been proven practical in upper limb scenario as mentioned above could achieve the same performance in lower limb cases. With the guidance provided by the data collected in acute recording sessions, we identified those locations where more task related neurons clustered, and subsequently carried out chronic electrode array implantation. A recursive Bayesian estimation framework was proposed for decoding EMG signals together with kinematics from cortical spike trains. Two specific algorithms were implemented and compared: a standard Kalman filter and an unscented Kalman filter. For the latter one, an artificial neural network was

**138**

incorporated to deal with the nonlinearity in neural tuning. Our results indicate that lower limb EMG signals and kinematics during monkey stand/squat can be accurately decoded from the activity of M1 neurons.

## 2. MATERIALS AND METHODS

## 2.1. Overview

The data used in this study were collected from two male Rhesus monkeys (monkey 1 and monkey 2). During earlier experimental stage, acute neural recordings were performed for locating the exact brain areas corresponding to the stand/squat movements. After that, chronic electrode array implantation and data recordings were carried out subsequently. In both acute and chronic data recording sessions, subdermal intramuscular EMG signals and limb trajectories were synchronously recorded. Details about animal behavioral training, surgery and some preliminary results of the acute neural data recordings can be found in our previous papers (Ma and He, 2010; Ma et al., 2015). The animal behavioral training and the actual experimental protocols was designed and implemented at Arizona State University (ASU). The experiments were complied with NIH policy on Humane Care and Use of laboratory animals, and were approved by the institutional Animal Care and Use Committee of ASU.

# 2.2. Behavioral Tasks

The monkey performed stand/squat tasks following a series of visual cues in a virtual reality environment. Restrained in a special designed primate chair shown as **Figure 1A**, with the head and both arms fixed and both legs staying on the movable pedal, the monkey could stand upright by pushing down the movable pedal or squat down by contracting legs back. A marker for optical tracking was placed at the animal's right ankle, corresponding to a red ball in the virtual environment, which was defined as the ankle cursor.

The procedures of the behavioral task in a typical successful trial are shown in **Figure 1B**. The emerging of a green box at the bottom of the screen indicated the start of a trial (Center On). To proceed, the monkey must squat down and make the ankle cursor touch the green box (Center Hit). Once the box was touched, its color turned to red immediately. A short time later, a green ball (Target) emerged at the top of the screen (Target On). When perceiving this, the monkey should stand up and push down the pedal to move the ankle cursor toward the target until their positions were matched (Target Hit). The time point when the ankle cursor left the bottom box was defined as Center Release, at which the bottom box turned back to green. The monkey was trained to fully extend the legs to hit the target green ball and hold this standing posture up to about 400 ms. After that, it was required to squat by retracting the legs and make the ankle cursor move toward the starting position (Target Release). When the ankle cursor touched the bottom box again (Center Hit Again), the phase of squatting was finished. About 100 ms later, a trial was completed, and the monkey will receive several drops of water as the reward. Each trial was preceded by an inter-trial interval (varied randomly from 5 to 10 s) when the screen was illuminated with bright blank scene to prevent dark adaptation since the room was in low light condition.

#### 2.3. Data Collection and Preprocessing 2.3.1. Acute Neural Data Recording

For acute neural data recordings, five independently controllable electrodes were transdurally insert into the target brain area (**Figure 1C)** in the left hemisphere with a multi-electrode microdrive (Thomas Recording, Germany). Extra-cellular neural signals were amplified, filtered and recorded with a 64 -channel Plexon MAP system (Plexon Inc., Dallas, TX) at 40 kHz/channel sampling rate. Every 40 successful trials were grouped to form a recording dataset. When a dataset was finished, the locations or penetrating depth of the electrodes will be changed. The locations for electrode penetrating can be found in **Figures 1D,E** for each monkey. Putative neuron units were isolated through spike sorting based on the clustering of detected action potential waveforms in principal components (PC) feature spaces using the Offline Sorter (Plexon Inc., Dallas, TX) software. For each putative neuron, the coordinates corresponding to the electrode were recorded as its location. By calculating the spike counts in consecutive 30 ms nonoverlap bins, discrete spike times were transformed into a firing rate sequences for further analysis.

#### 2.3.2. EMG Data Collection

Separate surgical procedures were performed to implant Teflon coated stainless steel wire electrodes into 8 muscles on each leg, as listed below: right and left soleus (RS and LS), right and left tibialis anterior (RTA and LTA), right and left semitendinosus (RST and LST), right and left recutus femoris (RRF and LRF), right and left extensor digitorum longus (REDL and LEDL), right and left flexor digitorum longus (RFDL and LFDL), right and left adductor magnus (RMG and LMG), right and left flexor hallucis longus (RFHL and LFHL). The EMG signals were recorded at a sampling rate of 1 kHz. Only signals from the right leg were used in further analysis. Raw EMG signals were full-wave rectified and filtered with a 4-pole 10 Hz low-pass Butterworth filter to get nonnegative envelops.

#### 2.3.3. Kinematic Data Collection

The position of the ankle LED marker were captured by a VZ 2000 tracker (PTI, Canada) with a sampling interval of 15 ms and recorded synchronously with the neural data. The velocity of the ankle was approximated by calculating the difference between the position of the marker at current time and that at the previous time, while the acceleration from the velocity in the same way. Since the monkey was restrained on the primate chair when performing repetitive stand/squat behaviors, it is obvious that only the dimension corresponding to the axis perpendicular to the transverse plane (z axis) changed significantly in different movement stages. In fact, motion data on the other two dimensions remained almost invariant during the whole movement. Therefore, we defined the kinematics vector as the instantaneous position, velocity and acceleration of the ankle marker at the z axis. The EMG signals and kinematics were also down-sampled to match the bin size of the neural firing rate sequences.

#### 2.3.4. Identifying Neural Facilitation of Muscle Activation

During the acute data recording session, we have explored a wide region in M1. To determine the optimal locations for chronic electrode implantation, it was necessary to identify those areas which clustered more neurons with high predictive power on EMG signals or motion trajectories. Here we calculated the spike triggered average (SpTA) of each neuron-muscle pair to examine if a given M1 neuron facilitated certain lower limb muscle activation.

Previous studies have demonstrated how the SpTA of EMG signals can be used to identify the synaptic connectivity between cortical neurons and the motor units of activated limb muscles (Fetz and Cheney, 1980; McKiernan et al., 1998; Griffin et al., 2008). By taking into account post-spike characteristics of the averaged EMG such as the onset latency, peak magnitude and duration, SpTA can be used as a predictor of the quality and strength of facilitative contribution that a detected cortical neuron may have in relation to a given muscle (Baker and Lemon, 1998; Aguayo et al., 2009).

For each neuron-muscle pair, the SpTA was calculated by averaging full-wave rectified EMG segments for all successful trials starting 20 ms prior to until 60 ms after a spike event. For each SpTA, a baseline mean and standard deviation (SD) were calculated using the 20–10 ms of the SpTA prior to the spike event. The onset latency is defined as the point that the SpTA crosses a threshold defined by the baseline mean + 2 SD after the spiking events for the first time. Similarly, offset latency is defined as the point that the SpTA falls below a threshold (baseline mean − 2 SD). The peak magnitude is defined as the peak value of an SpTA after the spiking events minus the baseline mean. The peak width at half magnitude (PWHM) is defined as the width of the SpTA at a level that is half the measured peak. A neuron-muscle pair will be considered to have no dependency relationship and eliminated from statistics if the onset latency of the SpTA is less than 5 ms or greater than 15 ms, or the PWHM of the SpTA is less than 0.75 ms or greater than 9 ms.

After scanning enough datasets, a number of neurons which were considered to facilitate the activation of each given muscle were identified from the ensemble. The densities of different muscle-innervating neurons were calculated with regard to each muscle as the number of identified neurons divided by the total number of neurons recorded in an unit brain area. By plotting a series of density contours, the spatial distributions of these muscle-innervating neurons can be directly perceived and utilized to determine the locations for chronic implantation.

#### 2.3.5. Chronic Neural Data Recording

About 1 month after the acute recording session, new surgeries were performed to implant 4 microelectrode arrays chronically into the selected brain areas for each monkey (**Figures 1D,E**). There were 16 polyimide insulated tungsten electrodes on each array as the recording channels, which were arranged with two rows. The diameter of each electrode was 50 microns. The spacing between electrodes in each row was 250 microns and the spacing between each row was 500 microns. In the following chronic neural data recording session, behavior tasks for monkey training and protocols for data recordings remained consistent.

#### 2.4. Neural Decoding Algorithms

The purpose of the neural decoding algorithm is to use cortical spike trains to estimate the lower limb EMG signals together with the kinematics during the stand/squat tasks. We formulated the problem in a recursive Bayesian estimation framework (Bergman, 1999), where the activity of each muscle, characterized by corresponding EMG signals, or the motion of the limb, represented by the kinematics, were viewed as hidden states of a dynamic system, while cortical spiking data acquired by the implanted microelectrodes as constantly updated measurements to the system. In the following sections, we first introduce the general basics of the recursive Bayesian estimation theory we count on, and then describe the two specific algorithms for practical implementation in this study.

#### 2.4.1. Recursive Bayesian Estimation

Here **x**<sup>k</sup> denotes the system states at the k-th time instant t<sup>k</sup> = k1t. For EMG decoding, **x**<sup>k</sup> is specified as **x**<sup>k</sup> = [u1(k), u2(k), · · · , uM(k)]<sup>T</sup> , where um(k), m = 1, · · · , M is the EMG envelop amplitude of the m th muscle at time t<sup>k</sup> , and M is the total number of muscles. For kinematics decoding, **x**k is specified as **x**<sup>k</sup> = [z(k), v(k), a(k)]<sup>T</sup> , representing the zposition, z-velocity and z-acceleration of the ankle marker at tk . Meanwhile, **y**<sup>k</sup> denotes the measurement to the system at tk , which is in fact a vector containing the spike counts of C neurons in the k th time bin. The estimation consists of two steps: prediction and update. The aim is to construct the a-posterior probability density function p(**x**<sup>k</sup> |**y**1:<sup>k</sup> ) for **x**<sup>k</sup> conditioned on all available measurements up to t<sup>k</sup> (**y**1:<sup>k</sup> ).

In the prediction step, the a-priori distribution of **x**<sup>k</sup> is estimated from the states at previous time instants. In terms of an usually made assumption that **x**<sup>k</sup> is generated by a Markov process, we have

$$p(\mathbf{x}\_k|\mathbf{y}\_{1:k-1}) = \int p(\mathbf{x}\_k|\mathbf{x}\_{k-1}) p(\mathbf{x}\_{k-1}|\mathbf{y}\_{k-1}) d\mathbf{x}\_{k-1} \tag{1}$$

where p(**x**<sup>k</sup> |**x**k−<sup>1</sup> ) models how the system state evolves from current time to the next, and p(**x**k−<sup>1</sup> |**y**k−<sup>1</sup> ) corresponds to the a-posterior probability for the previous time instant.

In the update step, the a-posterior prediction is obtained by correcting the a-prior prediction with the information provided by the new measurement data **y**<sup>k</sup> :

$$p(\mathbf{x}\_k|\mathbf{y}\_{1:k}) = \frac{p(\mathbf{y}\_k|\mathbf{x}\_k)p(\mathbf{x}\_k|\mathbf{y}\_{1:k-1})}{p(\mathbf{y}\_k|\mathbf{y}\_{1:k-1})} = \alpha p(\mathbf{y}\_k|\mathbf{x}\_k)p(\mathbf{x}\_k|\mathbf{y}\_{1:k-1}) \tag{2}$$

where p(**y**<sup>k</sup> |**x**k ) reflects the mapping between the system states and the measurements, called the likelihood term, and p(**y**<sup>k</sup> |**y**1:k−<sup>1</sup> ) is a normalization constant α that can usually be ignored.

Following Equations (1) and (2), p(**x**<sup>k</sup> |**y**1:<sup>k</sup> ) can be calculate recursively. In practical implementation, it requires to have closed form solutions to these general equations. Here we explored two specific algorithms, the Kalman filter (Grewal, 2011) and the unscented Kalman filter (UKF) (Julier et al., 1995) for estimating the EMG signals and kinematics recursively from the cortical signals, and details are described as follows.

#### 2.4.2. Kalman Filter

For the implementation of the Kalman filter, the state transition and the measurement model were both assumed linear and with Gaussian noise. Supposing model parameters are time invariant, these models can be written as:

$$\mathbf{x}\_k = \mathbf{A}\mathbf{x}\_{k-1} + \mathbf{w}\_k \tag{3}$$

$$\mathbf{y}\_k = \mathbf{H}\mathbf{x}\_k + \mathbf{q}\_k \tag{4}$$

where **A** is the state transition matrix, **H** is the measurement matrix. For EMG decoding, **A**, **W** ∈ R <sup>M</sup>×M, **H** ∈ R <sup>C</sup>×M, and for kinematics decoding, **A**, **W** ∈ R 3×3 , **H** ∈ R C×3 . The random variables **w**<sup>k</sup> ∼ N (0, **W**) and **q**<sup>k</sup> ∼ N (0, **Q**) denote the process and the measurement noise respectively. **A** and **H** are fitted from the training data via a least square linear regression approach, while **Q** and **W** are regression residuals. Let **x**ˆ − k and **x**ˆ<sup>k</sup> represent the a-prior estimation and the a-posterior estimation of the system state, the error covariance matrices can be defined as

$$\mathbf{P}\_k^- = E[(\mathbf{x}\_k - \hat{\mathbf{x}}\_k^-)(\mathbf{x}\_k - \hat{\mathbf{x}}\_k^-)^T]^-$$

for the a-prior estimation, and

$$\mathbf{P}\_k = E[(\mathbf{x}\_k - \hat{\mathbf{x}}\_k)(\mathbf{x}\_k - \hat{\mathbf{x}}\_k)^T]$$

for the a-posterior estimation.

The estimation started from a prediction step:

$$\begin{aligned} \hat{\mathbf{x}}\_k^- &= \mathbf{A}\hat{\mathbf{x}}\_{k-1} \\ \mathbf{P}\_k^- &= \mathbf{A}\mathbf{P}\_{k-1}\mathbf{A}^T + \mathbf{W} \end{aligned} \tag{5}$$

Subsequently, **x**ˆ<sup>k</sup> was obtained via an update step with the new measurement data at **t**<sup>k</sup> :

$$\begin{aligned} \hat{\mathbf{x}}\_k &= \hat{\mathbf{x}}\_k^- + \mathbf{K}\_k (\mathbf{y}\_k - \mathbf{H}\hat{\mathbf{x}}\_k^-) \\ \mathbf{P}\_k &= (\mathbf{I} - \mathbf{K}\_k \mathbf{H}) \mathbf{P}\_k^- \end{aligned} \tag{6}$$

where **K**<sup>k</sup> = **P** − <sup>k</sup> **<sup>H</sup>**<sup>T</sup> (**hHP**<sup>−</sup> <sup>k</sup> **<sup>H</sup>**<sup>T</sup> <sup>+</sup> **<sup>Q</sup>**) −1 is known as the Kalman gain.

#### 2.4.3. ANN Based Neural Tuning Model

For the update procedure of recursive Bayesian estimation indicated by Equation (2), knowledge about the mapping from system states to the measurements is required. Here it means a more appropriate model describing the neuronal modulation of muscular activity and limb movements should be selected. Previous studies have indicated that different hierarchically organized components, including the motor cortex and several subcortical structures and also circuits in the spinal cord, are involved in the neural motor control process (Kandel et al., 1995; Harel et al., 2008). Therefore, much nonlinearity exists for the neuronal modulations to movements, which is difficult for modeling with classical parametric techniques. The linear model (4) is actually a straightforward approximation for the neuronal modulations. Although easy for implementation, it may not be consistent with the actual neural system. It is expected to have a model which can reflect the intrinsic nonlinearity of the neural control process.

Here we utilized a feed-forward single hidden layer ANN to construct the model for neuronal modulations to EMG and kinematics. With the ability of approximating any complex nonlinear mappings directly from the input samples, ANN has been widely applied in many fields (Suykens et al., 2012), and is assumed to be efficient for the modeling of the neuronal modulations. The proposed feed-forward network included an input layer, a hidden layer and an output layer. Each neuron in one layer had directed weighted connections to those in the subsequent layer. Sigmoid functions were applied as the activation functions for the hidden layer neurons. Parameters of the network, including the connection weights and activation thresholds, were determined through a learning process. The modeling errors were fed back through the network via a back-propagation mechanism, and a gradient descent method was adopted for parameters adjusting in every learning epoch (Graupe, 2013). The ANN based measurement model can be written as:

$$\mathbf{y}\_k = \text{net}(\mathbf{x}\_k) \tag{7}$$

where net(·) represents the neural network. For EMG decoding, net(·):R <sup>M</sup> → R C and for kinematics decoding, net(·):R <sup>3</sup> → R C .

#### 2.4.4. Unscented Kalman Filter

As an improved algorithm to the Kalman filter, the unscented Kalman filter (UKF) incorporates nonlinear models in estimation, and uses a nonstochastic simulation to approximate the nonlinearity (Julier et al., 1995). The basic procedure for the UKF is the unscented transform, a deterministic sampling technique for approximating a given probability distribution with a series of weighted sample points (Sigma points), which has been proven superior to other ordinary linearization or approximation methods (Wan and Van Der Merwe, 2000).

In our implementation, the system states were still assumed to evolve according to a linear function as Equation (3) in the prediction step, while the mapping from the system states **x**<sup>k</sup> to the measurements **y**<sup>k</sup> was computed according to the ANN based model (7) in the update step.

Let d denote the dimension of the state variable. A set of 2d + 1 Sigma points were generated from **x**ˆ − k and **P** − k through the unscented transform:

$$\begin{aligned} \mathbf{X}\_k^{-(0)} &= \hat{\mathbf{x}}\_k^- \\ \mathbf{X}\_k^{-(i)} &= \hat{\mathbf{x}}\_k^- + \sqrt{d+\lambda} [\sqrt{\mathbf{P}\_k^-}]\_i \\ \mathbf{X}\_k^{-(i+d)} &= \hat{\mathbf{x}}\_k^- - \sqrt{d+\lambda} [\sqrt{\mathbf{P}\_k^-}]\_i, i = 1, 2, \dots, d \end{aligned} \tag{8}$$

where [·]<sup>i</sup> denotes the i-th row of the matrix, λ = α 2 (d + κ) − d is a scaling parameter and α and κ determine the spread of the sigma points around **x**ˆ − k . The square root of the matrix **P** − k was obtained by calculating the Cholesky decomposition. These Sigma points were propagated through Equation (7) to get the transformed results:

$$\hat{\mathbf{Y}}\_k^i = net(\mathbf{X}\_k^{-(i)}), i = 0, 1, \dots, d, \dots, 2d \tag{9}$$

The predicted mean and covariance of the measurement, and the cross-covariance between the state and the measurement were calculated from these transformed Sigma points:

$$\begin{aligned} \hat{\mathbf{Y}}\_{k} &= \sum\_{i=0}^{2d} w\_{i} \hat{\mathbf{Y}}\_{k}^{(i)} \\ \mathbf{P}\_{k}^{YY} &= \sum\_{i=0}^{2d} w\_{i} (\hat{\mathbf{Y}}\_{k}^{(i)} - \hat{\mathbf{y}}\_{k}) (\hat{\mathbf{Y}}\_{k}^{(i)} - \hat{\mathbf{y}}\_{k})^{T} + \mathbf{Q} \\ \mathbf{P}\_{k}^{XY} &= \sum\_{i=0}^{2d} w\_{i} (\mathbf{X}\_{k}^{-(i)} - \hat{\mathbf{x}}\_{k}^{-}) (\hat{\mathbf{Y}}\_{k}^{(i)} - \hat{\mathbf{y}}\_{k})^{T} + \mathbf{Q} \end{aligned} \tag{10}$$

where w<sup>i</sup> are a series of constant weights:

$$\begin{aligned} \omega\_0 &= \frac{\lambda}{d+\lambda} \\ \omega\_i &= \frac{1}{2(d+\lambda)}, i = 1, 2, \dots, 2d \end{aligned}$$

The a-posterior estimate **x**ˆ<sup>k</sup> and corresponding error covariance matrix **P**<sup>k</sup> were finally obtained by:

$$\begin{aligned} \hat{\mathbf{x}}\_k &= \hat{\mathbf{x}}\_k^- + \mathbf{K}\_k (\mathbf{y}\_k - \hat{\mathbf{y}}\_k) \\ \mathbf{P}\_k &= \mathbf{P}\_k^- - \mathbf{K}\_k \mathbf{P}\_k^{YY} \mathbf{K}\_k^T \end{aligned} \tag{11}$$

where the Kalman gain was calculated as **K**<sup>k</sup> = **P** XY k (**P** YY k ) −1 . The initial value for the error covariance matrix **P**<sup>0</sup> was set to the unit matrix. **P**<sup>k</sup> will show a convergent feature as the calculation of Equations (8)–(11) at each time instant.

#### 2.4.5. Performance Evaluation

Two metrics, Pearson's correlation coefficient (CC) and the signal to noise ratio (SNR) were used for the performance evaluation of the algorithms.

CC is calculated from the actual signal x and the estimated value xˆ for each channel by:

$$CC = \frac{\sum\_{k=1}^{K} (\mathbf{x}\_k - \bar{\mathbf{x}})(\hat{\mathbf{x}}\_k - \bar{\hat{\mathbf{x}}})}{\sqrt{\sum\_{k=1}^{K} (\mathbf{x}\_k - \bar{\mathbf{x}})^2} \sqrt{\sum\_{k=1}^{K} (\hat{\mathbf{x}}\_k - \bar{\hat{\mathbf{x}}})^2}}$$

where K is the length of the signal. An CC value close to 1 indicates better fit between the real and estimated data.

SNR is defined as the ratio between the variance of the actual data and the mean square error (MSE) of the estimates, and is calculated as:

$$SNR = 10\log\_{10}\frac{\sum\_{k=1}^{K}(\mathbf{x}\_k - \bar{\mathbf{x}})^2}{\sum\_{k=1}^{K}(\hat{\mathbf{x}}\_k - \mathbf{x}\_k)^2}$$

for each channel. It is quantified in decibels (dB) scale, where 0 dB means the signal and the noise are in equal proportion, and larger positive value means more useful signals are extracted. The SNR is sensitive to the errors introduced by amplitude scaling and offsets, and has also been adopted in Fitzsimmons et al. (2009).

## 3. RESULTS

#### 3.1. Synchronized Multiple Data Recordings

Synchronized multiple data recorded during 2 successive trials from monkey 1 is shown in **Figure 2**, including the spike trains of 16 task-related neurons (**Figure 2A**), the raw EMG signals of 5 right lower limb muscles (**Figure 2B**), and the trajectories of the marker attaching on the monkey's right ankle (**Figure 2C**). The occurring time instants for certain behavior events are indicated by short bars at the bottom. Data within the inter-trial interval (about 5 s) are omitted for displaying. Notably, the firing rates of most cortical neurons kept low when resting (quiet squatting, Center Hit to Target On). An ensemble of neurons began to discharge more frequently after target presenting (Target On), while their firing rates have been on a significant increase during such pre-movement epoch (Target On to Center Release). Later, amplitude oscillations were observed for some EMG channels around the initiation of the standing (Center Release), indicating the activation of corresponding muscles. For the subsequent epochs involving standing up (Center Release to Target Hit), standing still (Target Hit to Target Release), and squatting down (Target Release to Center Hit Again), some obvious event-related neural discharging patterns can be revealed via a series of perievent histogram analyses, which have been reported in our previous study (Ma et al., 2015). Compared to the previous work, here we may go a further step to explore the neuronmuscle associations more quantitatively for the neural decoding problems based on these observational or statistical results.

#### 3.2. Spatial Distributions of Different Muscle-Innervating Neurons

In the acute neural recording session, we explored the M1 area in the left hemisphere by changing the locations and penetrating depths of independent movable electrodes at a distance of 1 mm. The exact range of the region was in the coordinate system determined by the stereotaxic apparatus: 8.00–22.00 mm from anterior to posterior (PA), 0.00–16.00 mm from dorsal to lateral (DL), and 0.00–5.00 mm in the depth. A total number of 96 datasets were collected, and 1598 task-related neurons were isolated (909 from Monkey 1, 689 from Monkey 2).

Based on the SpTA analyses mentioned above, the neurons which may facilitate the activation of each given muscle were identified from the ensemble. **Figure 3** shows the density contours of the identified neurons with regard to 8 muscles in the horizontal (**Figure 3A**) and the coronal (**Figure 3B**) view using the data from monkey 1. The central sulcus (CS), as a prominent landmark, is also indicated designedly in the plots. All density values are normalized to [0, 1] and smoothed by a median filter. The hotter color indicates the higher density. These plots are to some extent "maps" for finding the neurons which may contribute to improving the accuracy of the EMG decoding. In the horizontal view, several concentration centers of muscleinnervating neurons for the 8 muscles can be clearly identified (**Figure 3A**). In the coronal view, we noticed that the depth between 0.5 ∼2.5 mm was where muscle-innervating neurons aggregated most densely (**Figure 3B**).

It is notable that some density contours are quite analogous, like those for the RRF-innervating neurons and the RTAinnervating neurons. But clear divergence exists for some other groups, like the RRF and the REDL. Here we evaluated the similarities of the spatial distributions of different muscleinnervating neurons via a distance measure based cluster method. To be specific, the density contours for different types of muscle-innervating neurons can be viewed essentially a series of matrices. The Euclidean distances between every two matrices were calculated for a cluster analysis. The results of the cluster are shown in **Figure 4**, for the horizontal (**Figure 4A**) and the coronal view (**Figure 4B**) respectively, using the data from Monkey 1. Several groups of muscles could be identified from the cluster results: RRF and RMG, RFHL and RFDL, RTA and REDL, either in the horizontal or the coronal view. Dramatically, the results using the data from Monkey 2 are almost the same. We seek possible illustrations for these results with some knowledge in anatomy. In fact, RMG and RRF are both thigh muscles. RFHL and RFDL are leg muscles, and are both innervated by the Tibial Nerve. RTA and REDL are also leg muscles, but are innervated by the Deep Fibular Nerve (Moore et al., 2011). These results suggest that, for muscles sharing larger similarities in anatomy, the corresponding M1 neurons have closer spatial distributions.

#### 3.3. EMG Decoding

The chronic neural recording session lasted 2 weeks and 13 datasets were collected for each monkey. In each dataset, 40 ± 5 successful stand/squat trials were recorded, and an average of 34 task-related neurons can be identified from all detected units. A 4-fold cross-validation was employed for model parameter estimation, in which 3-folds were used for training and 1-fold for testing.

**Figure 5** shows typical plots of the predicted EMG envelops(blue dashed line, by the standard Kalman filter; red dash dot line, by the UKF) from a test subset in comparison with the actual EMG signals(dark gray solid line) during 4 successive trials conducted by Monkey 1. The amplitude of all signals are normalized to the interval [0, 1]. The exact time of 5 important behavior events are indicated in each subplot (light gray dashed line). It is perceived from the plots that both

of the proposed decoding methods were capable to accurately predict the overall trends of the EMG signals during the standing and squatting task, as well as some significant waveform features such as the quiescence before Target On, the gradual amplitude increase during the pre-movement epoch (Target On to Center Release), and the transient changes during the movement stages. Compared to the standard Kalman filter, the UKF method could produce more approximate EMG envelop values. These results clearly imply that not only the timings for muscle activation/relaxation, but also the strength of lower limb muscular activity can be extracted from the cortical ensemble signals.

**Figure 6** shows the CC and SNR between the predicted and real EMG signals in each channel for the 2 monkeys, demonstrating the mean and the standard error of the mean (SEM). It is suggested that the UKF based decoding method can achieve higher CC and SNR values. For Monkey 1, the mean CC values for 7 channels of EMG signals (RFDL, RRF, RST, RTA, RMG, RS, REDL) were greater than 0.75, and the mean SNR values for 6 channels (RFDL, RRF, RST, RTA, RMG, RS) exceeded 3.5, when using the UKF as the decoding algorithm. In contrast, when using the standard Kalman filter to predict the EMG signals, there were only 4 channels (RFDL, RRF, RTA, and RMG) of which the mean CC values exceeded 0.75, and only 3 channels (RRF, RTA, and RMG) of which the mean SNR value was greater than 3.5. Similar results can be perceived for Monkey 2. Afterwards, we performed a right tail t-test with 416 observations (13 datasets, 4-folds, 8 channels of EMG signals) and significance level α = 0.05 for each monkey to test whether the UKF based method improved the EMG decoding performance significantly. The t-test verified that the predicted signals with the UKF had higher CC and SNR values compared to those with the standard Kalman filter for all EMG channels (p < 0.001). In fact, the UKF improved the CC for 13.6% and the SNR for 17.5%.

#### 3.4. Kinematics Decoding

The same procedures for model training and testing as in EMG decoding were adopted for the kinematics decoding. **Figure 7** presents the predicted trajectories for the position, velocity and acceleration of the ankle marker in comparison with the real data (dark gray solid line). It is shown that both methods can provide reliable tracking for the trajectories of the position when the legs of the monkey were moving either downward or upward. For the velocity, the predictions can approximate the overall trend but discrepancies were found within some time intervals. However, both methods cannot achieve satisfactory predictions for the acceleration. **Table 1**

summarizes the CC and SNR between the predicted and real trajectories (mean ± SEM). For tracking the position, high mean CC and SNR values were achieved. A right tail t-test with 52 observations (13 datasets, 4-folds) and significance level α = 0.05 for each monkey verified the superiority of the UKF based method (p < 0.001). For approximating the velocity, the CC and SNR values were relatively low, but higher than 0.5. For decoding the acceleration, both algorithms encountered not good enough performance, with the mean of the CC < 0.5, and SNR < 1.

## 3.5. Stability Analysis of the Neural Decoding Algorithms

We examined the stability of the neural decoding algorithms by investigating the convergence of the a-posteriori estimate error covariance matrix **P**<sup>k</sup> and the Kalman gain **K**<sup>k</sup> . The matrix **P**<sup>k</sup> defines the estimation error at each time step, while **K**<sup>k</sup> represents the relative importance of the measurement residual with respect to the a-prior estimation **x**ˆ − k . For reliable estimations, it is expected that the errors converge to considerable small values as the recursive iteration increases. As **P**<sup>k</sup> gets small, it means there is no requirement to alter the estimation too much at each time instant, therefore, **K**<sup>k</sup> will also get small. So the numerical stability of **P**<sup>k</sup> and **K**<sup>k</sup> are appropriate indexes for evaluating the stability of the neural decoding algorithms. **Figure 8** plots the L <sup>2</sup> norm of **P**<sup>k</sup> and **K**<sup>k</sup> across iterations. It is shown that the L <sup>2</sup> norm of both **P**<sup>k</sup> and **K**<sup>k</sup> converge to a small value after about 20–30 iterations. It is also suggested that the UKF can achieve faster convergence rate for **P**<sup>k</sup> and **K**<sup>k</sup> .


TABLE 1 | The CC and SNR values between the predicted and actual data for kinematics decoding (mean ± SEM).

## 4. DISCUSSION

In this study, we demonstrate feasible approaches for decoding lower limb muscular activity and kinematics from cortical neural signals during monkey performing a series of visual cued stand/squat tasks in a virtual environment. This study provides important insight regarding the cortical innervation of voluntary lower limb movements, while previous work have paid ample attentions to the upper limbs. The results clearly demonstrate that lower limb EMG signals and kinematics can be predicted from the firing rate sequences of multiple cortical neurons, which suggests the possibility for more comprehensive lower limb BMI systems.

The work in Bulea et al. (2014) has demonstrated sitting and standing intention can be discriminated from scalp EEG signals. However, more details about the timing and strength of multiple muscle activity and kinematics are still difficult to be extracted from the general noninvasive signals. There are not too many literatures concerning lower limb motor control using invasive methods due to the lack of systematic experimental protocols like Center-Out in upper limb circumstance and the uncertainty about brain areas for electrode implantation. Similar to the work in Fitzsimmons et al. (2009), we performed invasive neural recordings, but we concentrated voluntary stand/squat instead of the cyclic bipedal walking. By exploring a wide region in M1 with movable electrodes, we have identified the spatial distributions of different muscle-innervating neurons corresponding to each given muscle, which can provide important guidance for implanting chronic electrodes. The recursive Bayesian estimation based methods proposed here are proved effective for implementation, and could produce accurate EMG and kinematic estimation.

In Pohlmeyer et al. (2007), the prediction of EMG signals of several arm and hand muscles in button pressing or grasp movements captured most EMG features, and got R 2 values between 0.60∼0.70. In Li et al. (2009), the moving trajectories of the hand of the monkey can be reconstructed off-line with SNR between 2 and 5 dB, or CC between 0.6 and 0.9. In our study, the neural decoding accuracies of the 8 lower limb muscles and the ankle marker position are comparable to these studies (**Figure 6**, **Table 1**). Such results suggest that current upper limb BMI design concepts can be extended to lower limb occasions.

## 4.1. Implications for the Cortical Neural Encodings of Voluntary Lower Limb Movements

Previous upper limb BMI studies found neurons in M1 (Moran and Schwartz, 1999; Paninski et al., 2004; Wahnoun et al., 2006; Truccolo et al., 2008), ventral premotor cortex (Prabhu et al., 2009; Carpaneto et al., 2011), dorsal premotor cortex (Hao et al., 2013) encode various task related parameters, like reaching direction, joint orientation, or hand gestures, and have developed different neural decoding algorithms based on such observations (Hatsopoulos et al., 2004; Hao et al., 2013, 2014). As indicated in Winters and Woo (2012), motor control process of lower limbs is often considered different from that of upper limbs. We wonder either abstract representation of movements or explicit encodings of muscular activity and limb states can be extracted from the neural responses in M1 lower limb area, which determines the feasibility for constructing lower limb BMI systems.

Our findings suggest M1 neural ensemble may provide specific encodings to the execution of such visual guided standing/squatting task. When the visual cue was launched at Target On, the animal commenced to plan its movement in order to get success (**Figure 1B**). At this moment, the firing rates of a large portion of neurons have begun to increase. After the movement was initiated (at Center Hit), a good control about the position of the legs must be maintained since the monkey was required to match the ankle marker to the figures on the screen in right sequence, while most neurons were observed keeping relatively high firing rate. We think this fact reflects the sustained activity of task-related neurons from planning to executing in voluntary movement. In contrast, most recorded M1 neurons showed high variant firing patterns and exhibited weak modulation to the movement in a treadmill bipedal walking task (Huo et al., 2010). The execution of walking can be attributed to sort of cyclic movements, during which low level centers, like the CPG, take over more control details. Thus, the synchronization of M1 neural activity were absent. However, also in Huo et al. (2010), almost all recorded neurons became increasingly synchronized in firing patterns to each step, especially before and during stance phase, when the monkey walked on the treadmill after a spinal cord hemi-section surgery. These observations imply that M1 neurons are intensively involved into the control of walking when spinal cord is impaired. For the visual guided standing/squatting tasks, accurate responses to certain cues must be produced in time, of which the complexity is far more than the cyclic walking, and beyond the spinal cord circuits can handle. To sum up, M1 neurons participate the specific control of lower limb movements more in those circumstances during which low level centers are damaged or unable to produce accurate responses for purposeful tasks. Compared to another bipedal walking study (Fitzsimmons et al., 2009), the SNR for the predictions of EMG signals on the right leg musculature was 1.55 ± 0.39, which was lower than ours (**Figure 6**). This fact indicates the neural activity patterns during the visual cued standing/squatting is sufficient for achieving better decoding performance than the cyclic walking.

## 4.2. Functional Division of M1 Neurons Corresponding to Muscles

To determine the best locations for chronic electrode implantation, we investigated the contribution of a number of M1 neurons to facilitate given muscles. In fact, the elaborate organization of M1 neurons in lower limb control process can be revealed by plotting the density contours of neurons corresponding to each given muscle (**Figure 3**). The detected neurons appear sort of functional division corresponding to muscles or muscle groups. On one hand, there are significant discrepancies among the spatial distributions of different muscle-innervating neurons. On the other hand, analogous or overlapping spatial distributions of neurons can be identified for muscles innervated by the same nerve (**Figure 4**). The spatial range of the lower limb representative area in M1 has been repeatedly confirmed in the last decades using imaging techniques such as functional magnetic resonance imaging (fMRI), but the exact muscle-neuron associations and the specific locations of different muscle-innervating neurons are still not clear. Compared to these methods, the simultaneous multiple data recordings in this study achieved better time and spatial resolutions. Since the actual number of neurons is huge, we cannot yet declare we have constructed an exact map between M1 neurons and lower limb muscles. The most direct benefit is that we are more clear about which locations are optimal for the chronic electrode implantation. Meanwhile, these findings are helpful for understanding the organizations of the motor control pathway for lower limbs.

# 4.3. Nonlinear Method Achieves Better Decoding Performance

We try to seek better computational models to relate the multidimensional neural firing rate sequence to the EMG signals or kinematics. It is noticeable that higher decoding accuracy was achieved when using the UKF with an ANN based neural tuning model to estimate the EMG signals and the trajectories of the ankle position. Considering the intrinsic nonlinearity in the neural system as described before, a linear model as 4 may not fit the real neural data very well, though it is easy for implementation. The superiority of incorporating nonlinear models for neural decoding has been revealed by some previous studies. In Li et al. (2009), a quadratic neural tuning model relating position, velocity with the binned neural firing rates can considerably improve the decoding accuracy of the 2D hand moving trajectory. In Pohlmeyer et al. (2007), the R 2 between the real and predicted upper limb EMG signals was 0.69 when using linear estimation method, and increased to 0.75 when using nonlinear estimation method. The results in these reports are consistent with ours as mentioned above. The ANN reflected the nonlinearities between the system states to be estimated and the observed multi-dimensional neural firing rate sequences, and improved the decoding accuracy significantly when it was incorporated as the measurement model in the recursive estimation process.

# 4.4. EMG Decoding vs. Kinematics Decoding

We notice that the decoding accuracies for the EMG signals and kinematics are different. As shown in **Figure 6**, the mean CC values for the 8 channels of decoded EMG signals are all above 0.6, and the mean SNR values are all above 2 dB. For the trajectories of the ankle marker position, the prediction is accurate (mean SNR>5 and mean CC > 0.75). For velocity, the prediction captures the overall trend (SNR around 2 and CC around 0.6). However, the estimation of the acceleration is not satisfactory. In fact, the same situation have been noticed in previous studies. In Wu et al. (2002), the reconstruction for hand position and velocity are satisfactory while for acceleration is poor. They thought the acceleration was a second order difference of position, thus the measurements of it tended to be very noisy in real data. They also indicated the acceleration has a weak effect on the model relative to position and velocity.

After Target On, the monkey was in a plan stage, but actual movement has not been initiated until Center Release. The actual position, velocity and acceleration during the plan stage retained still (**Figure 7**, real data). Although the overall prediction accuracy for the ankle position was very high, the estimated values often failed to capture such quiescence (**Figure 7**, upper panel, predicted data). However, many neurons have begun to fire intensively and the amplitudes of the EMG signals have shown an analogous activated profile at the same time (**Figure 7**). In contrast to the ankle position, the estimated EMG values in this stage captured the dramatic increase of the real signals. Generally speaking, EMG signals are stochastic and noisy, and have greater bandwidth, which suggests more difficulties for the neural decoding algorithms. So the mean CC and SNR for the estimated ankle position trajectories are higher than those for the estimated EMG signals. But compared to the kinematics, EMG signals are closer to the cortical neural activity. When initiating and maintaining a movement, M1 neurons modulate their firing patterns to facilitate the contractile force in the skeletal muscles via a hierarchic downstream pathway. The planned movement are produced by the synergic activity of multiple muscles, and is more indirect with the cortical neurons. So the estimated EMG signals from the cortical activity can capture the trend of the real signals well even in the pre-movement epoch.

# 4.5. Potential Applications for FES and Mind Controlled Prostheses

The BMI technology constructs a bridge between the subject's mind and the external actuators. It has been assumed as the most promising technique for neuro-prostheses control and neurorehabilitation implementation. Here we expect the findings in our explorations on monkey stand/squat behaviors could be helpful for transforming BMI design concepts to potential clinical applications.

One of the possible applications is using the predicted EMG signals as the inputs of the functional electric stimulation (FES) to activate specific lower limb muscles of the paralyzed patients. FES is an effective method for preventing the muscle mass loss and promoting the functional recovery after nervous system lesion. In Moritz et al. (2008), a monkey can directly control stimulation of muscles using the activity of neurons in M1, thereby restoring goal-directed movements to a transiently paralyzed arm. Here we have demonstrated the timing and strength of the activity of multiple lower limb muscles can be predicted from cortical neural spike trains. With the predicted data, the timing and intensity of the FES for target muscles can be tuned according to the subject's mind to conduct more purposeful stimulation. Such approaches are expected to achieve similar results as in Moritz et al. (2008) for restoring various voluntary lower limb movements, like standing and squatting.

The predicted lower limb kinematics could be incorporated as the control source to drive artificial legs, exoskeleton or walking assistant robots designed for lower limb paralyzed patients or amyotrophic lateral sclerosis (ALS) patients. The actuators can be controlled to track the estimations of position and velocity extracted from cortical neural signals. Compared to traditional methods using accelerometer or other sensors (Reza et al., 2014), brain control enables the patients to manipulate complex biomechatronic devices in a more harmonized manner.

Meanwhile, the proposed algorithms are easy for implementation in micro-chip or real time embedded system. The Kalman filters does not require long windows to collect data as the multiple linear regression method and provides an explicit generative model, which is propitious to real time implementations for capturing rapid motions. The realization of the ANN and the unscented transform have also been proven feasible on current programmable micro controllers.

# 5. CONCLUSIONS

In this study, we focus on neural signals collected during monkey performing visual guided stand/squat tasks. We have demonstrated lower limb EMG signals and ankle moving kinematics during stand/squat could be accurately decoded from a group of M1 neurons with the proposed algorithms. Our findings provide new insights for extending current BMI design concepts on upper limbs to lower limb circumstances. Brain controlled exoskeleton, prostheses or neuromuscular electrical stimulators for lower limbs are expected to be developed for the future extension of this work.

# AUTHOR CONTRIBUTIONS

Conceived the project and designed the technical detail: JH (Jiping He). Performed the experiments: CM, JX. Analyzed the data: XM, CM, PZ, JH (Jian Huang). Drafted the manuscript: XM, CM. Revised the manuscript: XM, CM, JH (Jian Huang), JH (Jiping He).

# FUNDING

This work is supported by the National Natural Science Foundation of China (31460263, 61233015), 973 Project of China (2013CB329506), and Jiangxi Natural Science Foundation (12004630).

# ACKNOWLEDGMENTS

Authors wish to express their appreciation for the technical helps provided by Dr. Hang Zhang, Dr. Xinying Cai, Mr. Tedd Brandon and other members of the Neural Control Laboratory in Arizona State University.

# REFERENCES


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

The reviewer GL and handling Editor declared their shared affiliation, and the handling Editor states that the process nevertheless met the standards of a fair and objective review.

Copyright © 2017 Ma, Ma, Huang, Zhang, Xu and He. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# A 3D Computational Model of Transcutaneous Electrical Nerve Stimulation for Estimating Aβ Tactile Nerve Fiber Excitability

Kaihua Zhu † , Liming Li † , Xuyong Wei and Xiaohong Sui\*

*School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai, China*

Tactile sensory feedback plays an important role in our daily life. Transcutaneous electrical nerve stimulation (TENS) is widely accepted to produce artificial tactile sensation. To explore the underlying mechanism of tactile sensation under TENS, this paper presented a novel 3D TENS computational model including an active Aβ tactile nerve fiber (TNF) model and a forearm finite element model with the fine-layered skin structure. The conduction velocity vs. fiber diameter and strength-duration relationships in this combined TENS model matched well with experimental data. Based on this validated TENS model, threshold current variation were further investigated under different stimulating electrode sizes with varied fiber diameters. The computational results showed that the threshold current intensity increased with electrode size, and larger nerve fibers were recruited at lower current intensities. These results were comparable to our psychophysical experimental data from six healthy subjects. This novel 3D TENS model would further guide the floorplan of the surface electrodes, and the stimulating paradigms for tactile sensory feedback.

Keywords: 3D computational modeling, tactile sensory feedback, transcutaneous electrical nerve stimulation, Aβ tactile nerve fiber, double-cable model, psychophysical experiments

## INTRODUCTION

Tactile sense is important for us to gather information about the world, and tactile sensory feedback plays a great role in our daily life. Both tactile sensation and motor function are important for reali zing dexterous manipulation. In recent years, incorporation of artificial tactile sensory feedback has obtained great interest worldwide to help achieve the fine closed-loop control for the prosthetic hand (Kim et al., 2014; Tan et al., 2014; Tee et al., 2015).

For stroke patients with severe loss of hand sensation, their grasp function could be greatly improved by incorporation of the fingers' artificial tactile sensation based on mechanical stimulation of other normal skin area such as shoulder (Kita et al., 2013). By combining both tactile and visual feedback, the virtual reality-based training technology made a significant role in rehabilitation for patients with stroke or Parkinson disease (Saposnik et al., 2010; Liao et al., 2014). For the amputees, accomplishing tactile sensory feedback of prosthetics will improve their acceptance for the artificial limb (Antfolk et al., 2013).

Normally, our ability to feel the richness of our tactile environment relies on the cutaneous mechanoreceptors. These receptors detect and convert mechanical stimuli to corresponding electrical signals in the cutaneous nerve fibers. These signals can be delivered to the central nervous

#### Edited by:

*Wenwei Yu, Chiba University, Japan*

#### Reviewed by:

*Stefano Vassanelli, University of Padua, Italy Erika Kristine Ross, Mayo Clinic, USA*

#### \*Correspondence:

*Xiaohong Sui suixhong@sjtu.edu.cn † These authors have contributed equally to this work.*

#### Specialty section:

*This article was submitted to Neural Technology, a section of the journal Frontiers in Neuroscience*

Received: *15 December 2016* Accepted: *18 April 2017* Published: *16 May 2017*

#### Citation:

*Zhu K, Li L, Wei X and Sui X (2017) A 3D Computational Model of Transcutaneous Electrical Nerve Stimulation for Estimating A*β *Tactile Nerve Fiber Excitability. Front. Neurosci. 11:250. doi: 10.3389/fnins.2017.00250* system, where they are further processed and interpreted as our tactile sensation (Gardner, 2010; Woo et al., 2015). Usually, thin-myelinated Aδ- or unmyelinated C- fibers mediate thermal and noxious stimuli which can produce pain sensation. The myelinated Aβ sensory afferents mostly innervate cutaneous mechanoreceptors, and are thought to produce an innocuous tactile sensation (Matsumoto et al., 2008). On the other hand, it was reported that some of Aβ fibers could innervate with the nociceptors (Djouhri and Lawson, 2004), and Aβfiber perception could be transmitted to spinal neurons, which originally received only Aδ- and C-fiber-mediated pain transmission, in animals with nerve injury (Matsumoto et al., 2008). Besides, Aβ fibers can activate a significant proportion of superficial dorsal horn GABAergic neurons for maintaining separation of touch and pain modalities (Daniele and MacDermott, 2009), and the electrical stimulation of the lowthreshold Aβ afferent fibers may treat chronic neuropathic pain (Sdrulla et al., 2015). Then the Aβ fibers play an important role in tactile and pain sensation.

There are various kinds of ways to realize artificial tactile feedback. The mechanical tactile feedback can be obtained by using a skin pusher to convey tactile information. The typical example is the pneumatic system which was composed of pressure pads and a tube exerting pressure to the residual limb from the prosthetic fingers (Childress, 1980). The tactile feedback can also be achieved by electrical stimulation. By implanting microelectrode arrays in the somatosensory cortex, the electrodes can record the action potentials related to tactile sensation (Schwartz, 2004). Direct stimulation of the peripheral nerves is another way to elicit tactile feedback (Dhillon and Horch, 2005; Raspopovic et al., 2014). With a flat interface nerve electrode, the common nerve trunk can be selectively recruited (Tyler and Durand, 2002). Recently, the transcutaneous electrical nerve stimulation (TENS) was applied to accomplish tactile sensation for the amputee (Chai et al., 2015) and healthy subjects (Forst et al., 2015). Due to the non-invasiveness, the TENS could be promising in wide clinical applications to elicit tactile feedback. By attaching a surface electrode to the target skin area, various sensory afferents including Aβ, Aδ, or C fibers can be directly recruited without the activation of mechanoreceptors to produce artificial tactile or nociceptive sensation (Frahm et al., 2013). The excitatory state of sensory afferents such as Aβ nerve fibers can be modulated by different kinds of stimulating waveform parameters (Mahmud and Vassanelli, 2016).

To describe how the TENS evokes tactile neural activities and leads to psychophysical responses, the computational modeling is a feasible and efficient way, and a solid tactile nerve fiber model is the key to investigate the fiber excitation patterns. Although part of the Aβ nerve fibers contribute to pain sensation, they mostly play a significant role in feeling the pressure/light touch (Gardner, 2010). Most of the previous computational work about TENS focused on nociceptive Aδ- or C-type fibers (Yang et al., 2015), or motor nerve fibers (Kuhn et al., 2010; Goffredo et al., 2014). Although some papers established TENS model considering tactile sensory nerve fiber (Kajimoto et al., 2004; Mørch et al., 2011), the fiber model is passive without explicit ion channel distribution. The active tactile sensory nerve fiber model is more comparable to neurophysiological properties and important for TENS modeling. McNeal et al. established the first active multi-compartment cable model in 1976 to study the response patterns of action potentials. The model assumed the myelin sheath as a perfect insulator, and contained a few voltage-gated sodium and potassium channels at the nodes of Ranvier (McNeal, 1976). In 1995, Schwartz et al. recorded the action potentials and membrane currents in single human myelinated nerve fibers by means of the current and voltageclamp techniques. They applied the obtained experimental data to the mathematical model derived from the Frankenhaeuser-Huxley equations (Schwarz et al., 1995). In 1999, Wesselink et al. implemented a human myelinated sensory nerve fiber model for direct spinal cord electrical stimulation (Wesselink et al., 1999). McIntyre et al. (2002) built a McIntyre-Richard-Grill (MRG) neuron model for mammalian motor nerve fibers on the foundation of Hodgkin-Huxley (H-H) model. In recent years, some researches adopted the MRG model as the sensory nerve fiber model in their computational modeling work (Bourbeau et al., 2011; Åström et al., 2015). However, the MRG model was derived from mammalian motor nerve fibers, and motor and sensory fibers were different in recovery cycle, action potential shape, and the properties of excitability (Howells et al., 2012). Consequently, a solid active tactile Aβ sensory nerve fiber is necessary for TENS modeling.

In our present study, we developed a novel three-dimensional (3D) computational model of TENS on the forearm to investigate the human subcutaneous Aβ tactile nerve fiber (TNF) excitation. The 3D model establishment was achieved by combining finite element model of the whole forearm, and the active TNF counterpart. The solid active tactile sensory nerve fiber model was optimized with integral ionic and morphological parameters based on the H-H model. In order to clearly figure out the excitability of TNFs under the TENS, the skin model was delaminated into multiple layers in terms of physiological structures. In addition, the effects of electrode size on the tactile sensation threshold were further compared between computational and psychophysical experimental results on healthy subjects. This novel 3D TENS model would lay a solid basis for optimization of surface electrodes in the transcutaneous electrical nerve stimulation.

# METHODS

# The 3D Finite Element Model

A 3D finite element model (FEM) of the forearm was created using COMSOL software (COMSOL Multiphysics 4.2a, Sweden). The FEM model was established to calculate the electrical potential distribution in the subcutaneous area during electrical stimulation. The stimulating waveform was biphasic chargebalanced cathodic-first current pulse. The FEM models were computed in the AC/DC module with a stationary electric current. According to the charge relaxation theory, the charge relaxation time τ is described as follows:

$$\mathfrak{a} = \frac{\mathfrak{e}}{\mathfrak{a}} \tag{1}$$

where ε and σ are respectively the permittivity and conductivity of the tissues. In neural tissue, the values of these parameters are ε = 10−7 F/m and σ = 0.1 S/m (Gabriel et al., 1996a). As a result, the τ = 10−6 s, which is much smaller than the external time scale such as the stimulation pulse duration on the order of hundreds of microseconds. When applying current-regulated TENS, the influence of capacitance in the intermediate tissues can be neglected (Kuhn et al., 2009a).

Mimicking the circumstance of psychophysical experiments on the forearm, a 150 mm-long cylindrical geometric structure with stimulating and return electrodes was modeled as shown in **Figure 1**. The computational model consisted of seven concentric layers: stratum corneum, epidermis, dermis, fat, muscle, cortical bone, and bone marrow. The detailed electrical properties and thicknesses were listed in **Table 1**.

In this model, the epidermis, dermis, and muscle layers are anisotropic, while the other tissues are regarded as isotropic. The electric scalar potential (VFE) in the FEM was described by Laplace's equation.

$$-\nabla \cdot \left(\sigma \nabla V\_{FE}\right) = 0\tag{2}$$

The σ is conductivity. By free tetrahedral meshing, there are more than 10 million elements in total with extremely tiny meshes due to the thin layers of epidermis and dermis. The computation was carried out using a computer workstation with an Intel Xeon CPU E5 and 64 GB of RAM.

## Subcutaneous Aβ Tactile Nerve Fiber Model

#### Morphological Parameters

In our model, we developed Aβ nerve fibers with distribution density of approximate 2/mm<sup>2</sup> under the skin (Lacour et al., 1991; Lesniak et al., 2014). As shown in **Figure 1B**, Aβ nerve

#### TABLE 1 | Morphological and electric parameters in the FEM.


[*a*]*Yamamoto and Yamamoto (1976);* [*b*]*Gabriel et al. (1996b);* [*c*]*Tavernier et al. (1993);* [*d*]*Kuhn et al. (2009b);* [*e*]*Neerken et al. (2004);* [*f*]*Sandby-Moller et al. (2003) and the* \**notation denotes the values are adjusted for the sensitivity analysis. Through the thickness of the skin varied with many conditions, but the generally the total thickness is thought to be 1.5 mm (Kuhn et al., 2009a). Consequently the thickness of the dermis is determined by the stratum corneum and epidermis thicknesses and the whole skin thickness.*

the 3D model. (D,E) Trimetric and front views of the model in extreme tiny tetrahedron mesh. The blue circle area denotes the two electrodes.

fibers protruded from the muscle layer, and then aroused vertically to the skin which terminated in the dermis. This fiber length was 7 mm long, and was perpendicular to the skin except for the distal branching endings.

The TNFs were developed including parallel double cable models with nodes of Ranvier, paranodal, juxtaparanodal, and internodal sections, and the myelin sheath, etc. based on previous studies (McIntyre et al., 2002; Li et al., 2013). These fiber models have similar structures with the same kinds of ionic channels but with different spatial distribution of ion channels as shown in **Figure 2**.

The internodal section consists of two paranodal myelin attachment segments, juxtaparanodal segments and six internodal segments such that relationship remains valid for the compartments in all models. The Node-to-node length represents the distance between two adjacent Ranvier node centers, as shown in Equation (3).

Node − to − node length = nodal length + 2 × paranodal length + 2 × juxtaparanodal length + 6 × internodal length (3)

The models were built in NEURON v7.3 environment, and solved using backward Euler implicit integration with a time step of 0.001 ms.

It was reported that the axonal diameter of Aβ fibers was about 10 µm (McGlone and Reilly, 2010), while such big diameter was always measured on the ulnar or median nerve trunk. It was found that some mechanoreceptor-innervated nerve fibers such as Ruffini corpuscle afferent axons measured 2–4 µm in diameter (Halata et al., 1985). And, the 3 µm was most common diameter in the superficial cutaneous layer (Provitera et al., 2007). So the diameter of cutaneous Aβ nerve fiber is much compatible to the sural nerve fiber with 3.11 µm (Van Veen et al., 1995). The number of myelin lamellae and axon diameter was not constant, but decreased toward the distal end. Caruso G et al. found that fiber diameters in the skin were at least 3 times thinner than the respective proximal segments in the nerve trunk (Caruso et al., 1992). Besides, Provitera denoted that the subcutaneous fiber diameter should be about 2–8 µm (Provitera et al., 2007). As a result, under TENS of the forearm, the tactile Aβ afferents were selected as small myelinated nerve fibers in our present work. However, no detailed morphological parameters were reported adaptable to small Aβ fibers. The morphological parameters for the fibers were listed in **Table 2**. Some parameters varied with diameters, and the others were fixed according to the experimental data. So we used numeric interpolation and extrapolation to estimate the morphological parameters of Aβ tactile fibers with different diameters as shown in **Figure 3**. Then, the varied morphological properties can be determined including node-to-node length (X1), the number of myelin lamella (X2),

#### TABLE 2 | Geometric parameters for Aβ TNF.


*These fixed values are taken from previous studies. (McIntyre et al., 2002, 2004; Sotiropoulos and Steinmetz, 2007).*

*<sup>a</sup>Values of these parameters are all dependent variables of fiber diameters (*Figure 3*).*

*<sup>b</sup>psw: periaxonal space width.*

*<sup>c</sup>Derived from Equation (3).*

node diameter (X3), juxtaparanode diameter (X4), internode length (X5).

#### The Ion Channel and Membrane Dynamics in the Aβ TNF Fiber Model

To simulate the behavior of the axon under electrical stimulation, the model we implemented contains both linear and nonlinear membrane dynamics. There are many kinds of voltage-gated ion channels in the axon, and the ion channels located in the axonal membrane carry out action potential initiation and conduction by governing the amplitude and shape of the unitary spike and pattern of repetitive firing (Debanne et al., 2011). We developed a kind of human peripheral TNF model based on the double cable model structure (McIntyre et al., 2002; Li et al., 2013). The node Ranviers consisted of slow K+, fast K+, fast Na+, and persistent Na<sup>+</sup> conductance. The juxtaparanodal consisted of fast K<sup>+</sup> conductance in parallel with the membrane and leakage conductance. The dynamics of the nodal and internodal ion channels (Equations in Appendix) were based on the experimental data of sensory nerve fibers (Howells et al., 2012). The maximum density of fast K<sup>+</sup> in the sensory nerve fiber (Howells et al., 2012) is far more than that in the motor fiber (McIntyre et al., 2002), so it was incorporated to our TNF model. Then the novel Aβ tactile sensory nerve fiber model was achieved. All of the parameter values were derived from the reported experimental data, and dynamic equations form for sodium ionic channel was based on ionic conductance. The electrical parameters of the Aβ TNF model are shown in **Table 3**.

## The Combined 3D TENS Modeling

This combined 3D TENS model is accomplished by integration of the forearm FEM model implemented in COMSOL and TNF model developed in NEURON. Stimulating currents were applied to the surface electrode on the forearm FEM model in COMSOL, and then the subcutaneous spatially interpolated

electric potentials VFE in COMSOL were exported to NEURON as the corresponding extracellular voltages Ve (Ve = VFE) of the TNFs. Thus, the excitability of the TNFs under TENS can be quantitatively determined, and the excitation threshold was defined as the minimum current intensity to recruit one fiber.

#### Model Validation

By adopting typical FEM model parameters in **Table 1** and TNF properties in **Tables 2**, **3**, the 3D TENS model was validated in terms of conduction velocity of the TNFs and the strength-duration relationship of the surface electrical stimulation. The conduction velocities were computed in NEURON corresponding to different fiber diameters ranging from 2 to 8 µm. The curves of conduction velocity vs. fiber diameters were compared with previously reported sensory fiber modeling results (Wesselink et al., 1999) and experimental data (Van Veen et al., 1995). In addition, with the stimulating frequency of 50 Hz and pulse duration of 200 µs, the strengthduration relationship in our 3D TENS model were achieved for a given 8-9 mm electrode and an 8-3 µm Aβ nerve fiber with typical electrical properties. The stimulating pulse durations were selected increasing from 10 µs to 1 ms, and the corresponding


[*a*]*Wesselink et al. (1999);* [*b*]*McIntyre et al. (2002);* [*c*]*Schwarz et al. (1995);* [*d*]*Howells et al. (2012).*

threshold current intensities were obtained. Strength-duration curves were further compared among our computational, psychophysical experimental and reported modeling results.

#### Forearm Model Parameter Sensitivity Analyses

The tactile sensation threshold varied between different individuals under TENS. To potentially explore what caused the threshold variance between individuals, we made several sensitivity analyses on the tissue properties. The thickness and conductivity in fat, muscle and dermis layers were changed to explore what tissue properties would contribute most to the TENS difference. As shown in **Table 1**, the fat conductivity changed between 0.0017 and 0.1 S/m, the muscle conductivity (axial) between 0.2 and 0.5 S/m. The muscle conductivity (radial) between 0.0667 and 0.1667 S/m, the dermis conductivity (axial) between 2 and 3.8 S/m, and the dermis conductivity (radial) between 1 and 2.8 S/m. The corresponding thickness variation ranges were as follows: 1.5–4 mm for Fat layer, 1.2–1.7 mm for Dermis layer, and 8–18 mm for Muscle layer. Many combinations can be obtained with the ranges from different parameters. In order to get the threshold current relationship with different conductivities or thicknesses, the specific parameter changes within a variation range, and the remaining geometric and electrical properties were kept constant as the typical values in **Table 1**.

## Threshold Currents under the 3D TENS Modeling

Based on this validated 3D TENS model, the effects of electrode size and fiber diameters on the threshold current intensity were investigated. Fiber diameters were selected from 2 to 8 µm with a step of 1 µm, the surface electrode size from 5 to 12 mm, pulse duration from 10 µs to 1 ms. With the 8-9 mm surface electrode and 8-3 µm fiber, the spatial distribution of threshold current intensity below the surface electrode was obtained, and the fiber excitation characteristics were further observed under different stimulating currents.

## Psychophysical Experiments

The psychophysical experiments were carried out for five consecutive days on six healthy subjects (aged 22–30 years, 3 males, 3 females) without clinical or neuro-physiological disorders. All subjects were provided the informed written consent prior to commencement of the experiment. All experiments were conducted in accordance with the Declaration of Helsinki and approved by the Ethics Committee of Human and Animal Experiments of School of Biomedical Engineering at Shanghai Jiao Tong University (No. 2016012). In our experiments, we used Master-9 Pulse Stimulator with two isolators (Iso-Flex, A.M.P.I. Company, Israel) to generate biphasic charge-balanced cathodic-first stimulating current pulses. The gold stimulating electrode was placed on the volar part of forearm 2 cm from the proximal wrist crease and the return electrode approximately 12 cm apart from the stimulating electrode on the same side. The same-size thinlayer conductive gel was adhered to the stimulating electrode during the TENS. An 8-25 mm non-woven surface electrode (Shanghai Kongren Medical, Inc.) was used as the return electrode. The subjects sit in a chair with a comfortable position, and the small current intensities were adopted to electrically stimulate the forearm skin through the electrode to produce the pressure sensation. The experiments started only when the room temperature was stable at 27◦C. During all the experiments, the subjects were given 5-min relaxation every 20 min to assure the comfort during the whole psychophysical experiments.

The tactile perception thresholds were defined as the least current pulse intensity or amplitude that a subject could detect, and was determined by using a staircase paradigm. The pulse amplitude was first increased with a step of 0.1 mA, and then a step of 0.01 mA to estimate the potential perception threshold current. Immediately after each stimulation, the subjects were required whether or not the stimulus was perceived. Tens of stimulating trials were conducted near the potential threshold currents, and the exact perception threshold was determined in terms of 50 percent level of the trails. In the psychophysical experiments, the effects of diameters of the gold stimulating electrode (5–12 mm) and the pulse duration (50–400 µs) on the tactile perception threshold were also investigated.

The effects of individual subject difference on the perception threshold were also statistically evaluated in the psychophysical experiments. The independent variable was "individual subject," and the dependent variable was "perception threshold." A oneway analysis of variance (ANOVA) was performed and the F-test was used to find out the significance of the independent variable. The significance level was chosen as 0.05.

## RESULTS

#### Model Validation

The typical propagating characteristics of action potentials in an 8-3 µm Aβ nerve fiber were shown in **Figure 4** under biphasic charge-balanced cathodic-first current pulse stimulation. The pulse duration was 200 µs, and the stimulating intensity was 2.84 mA which is 1.2 times the stimulating threshold. The action potential was excited at node S, and then propagated along both directions of the fiber. The other nodes were first hyperpolarized, and then depolarized when the propagating action potential arrived. The conduction velocity of this typical TNF was 8.27 m/s in terms of **Figure 4**. The conduction velocities corresponding to varied fiber diameter were also shown in **Figure 5**. With the fiber diameter increasing from 2 to 8 µm, the corresponding conduction velocity increased from 5.68 to 33.14 m/s, which matched well with experimental data (Wesselink et al., 1999), and previously published modeling results in terms of smalldiameter tactile sensory nerve fibers (Van Veen et al., 1995). The conduction velocity is dependent on a host of factors such as myelin sheath thickness and internodal length (Waxman, 1980). These morphological parameters were interpolated from various existing models as shown in **Figure 3**. Other parameters such as axoplasmic resistivity which have various alternatives were also optimized to match the experimental results. The ultimate optimal parameters as for the morphological parameters and electrical properties were denoted in **Tables 2, 3**, respectively.

With standard 3D TENS model parameters selected, the strength-duration relationship was illustrated in **Figure 6**, where the 8-3 µm Aβ nerve fibers and 8-9 mm stimulating electrodes were adopted. As illustrated in **Figure 6**, multiples of the rheobase current was selected to describe the stimulus threshold. With pulse duration enhancing, the threshold decreased sharply within short pulse durations and maintained stable at durations larger

FIGURE 4 | Conditions for action potential propagating under biphasic charge-balanced cathodic-first current pulse stimulation. The 1.2× means the stimulation current amplitude was 1.2 times the threshold current.

The red point S is the initial node for the action potential. The left panel shows the corresponding position for each Ranvier node in an Aβ tactile fiber.

than 500 µs. The threshold currents for tactile perception were obtained based on psychophysical experiments from 6 healthy volunteers. **Figure 6** showed that the strength-duration tendency was closely consistent with our psychophysical experimental results.

# Forearm Model Parameter Sensitivity Analyses

**Figure 7** showed the typical subcutaneous distribution of current density and electrical field strength during the TENS. The current density reached the maximum in the dermis layer, especially near the electrode edge. On the contrary, the strongest electrical field occurred in the fat layer due to the lower conductivity. The

deviations of experimental results.

properties in dermis and fat layers may play an important role in the tactile sensation threshold. **Figure 8A** showed that the threshold current rose with the increasing electrical conductivity of fat and dermis layers while descending with the increasing electrical conductivity of the muscle layer. Within the given variation range shown in **Table 1**, the threshold current was most sensitive to the electrical conductivity of fat tissue. **Figure 8B** showed that the threshold current rose with the increasing thickness of fat and dermis layers, while descended a bit when increasing the muscle thickness during the given range. The thickness of fat and dermis layers showed obvious effect on the threshold current.

# Threshold Currents under the TENS

The threshold currents were further investigated by means of both the 3D TENS computation and psychophysical experimental work. The threshold currents in terms of 6 subjects were shown in **Figure 9**. The one-way ANOVA results indicated that the different subject showed a significant variance on the tactile perception threshold [F(5, 24) = 24.565, p < 0.0001]. The tactile sensation threshold were between 0.5 and 1.5 mA in our psychophysical experiments, and the corresponding strength-duration relationship was illustrated in **Figure 6**, which showed similar tendency to that of our modeling results.

FIGURE 7 | (A) The current density (0–10 A/m<sup>2</sup> ) distribution under the TENS with stimulating current of 0.5 mA. (B) The corresponding electrical field strength distribution (0–10 V/m).

**Figure 10A** showed the spatial distribution of threshold current in terms of 8-3 µm axons under the 8-9 mm stimulating electrode. It was clear that the least excitation threshold occurred axially below the stimulating electrode corresponding to one fiber excitation. The threshold increased apparently with a further distance from the electrode center. The threshold current below the edge of the electrode almost two times than that of the electrode center. **Figure 10B** showed that when stimulating current increased a little more than the threshold, the fiber excitation area extended greatly. When applying current intensity 1.4 times the threshold, the excitation area was close to size of the 8-9 mm stimulating electrode.

**Figure 10C** illustrated the nerve excitation threshold during 3D TENS computation for different axon diameters and pulse durations. Larger nerve fibers were recruited at lower current amplitudes. The threshold current intensity decreased with fiber diameter from 2 to 8 µm, and varied sharply with small pulse duration stimulation. For instance, with pulse duration of 50 µs for 8-3 µm fibers, the threshold was 1/3 compared with that at 10 µs duration. While, with tenfold variation of pulse duration from 100 to 1,000 µs, the threshold current at 1,000 µs only decreased to be about 1/2 at 100 µs. Besides, at short durations, the threshold difference among different-size fibers was more obvious than that of long durations. For example, the threshold currents at 10 µs for 8-2 µm fibers were 22 mA bigger than that of the 8-8 µm fibers, and the difference decreased to be 3.3 mA at 200 µs correspondingly.

In addition, under typical pulse duration of 200 µs and frequency of 50 Hz, the effects of different fiber diameters and stimulating electrode sizes on the threshold current were further compared between 3D TENS computational results and the psychological experimental data in **Figure 10D**. Computational results showed that the threshold current amplitude increased gradually with electrode size enlarging from 5 to 12 mm. With fiber diameter increasing from 3 to 8 µm, the threshold current decreased. The black square line depicted the psychophysical experimental results averaged from 6 healthy volunteers with similar changing tendency. This line fitted closely to the changing tendency of the 8-7 µm fiber in the computational results.

# DISCUSSION

In this study we developed a novel general 3D TENS computational model which could guide theoretical estimation of the recruitment of tactile afferent fibers. Although sensory nerve model had been studied for many years (Schwarz et al., 1995; Wesselink et al., 1999; Smit et al., 2009; Howells et al., 2012), the specific active model of tactile sensory fibers had not been studied in detail during the TENS. Our tactile nerve fiber model adopted the widely-accepted double-cable-layer model structure. The ionic types and kinetics parameters in this TNF model were derived from the recent study results (Howells et al., 2012), which were different from the previous parameters associated with the motor fiber model (McIntyre et al., 2002). The slow potassium channels in the internode were removed due to the low density distribution (Howells et al., 2012). The other electrical parameters were optimized by mathematical fitting based on the previous separate physiological experiments. This novel fiber model was validated by comparing the velocity vs. diameter and strength vs. duration relationships between our modeling and experimental results. As for the finite element modeling work, the 3D forearm was modeled, and the skin structure was finely layered into stratum, epidermis and dermis layers. This fine structure would be more practicable in comparison with the non-layered skin structure in some previous work (Kajimoto et al., 2004; Kuhn et al., 2009a). Based on the combined 3D TENS model consisting of finite element model of forearm and tactile nerve fiber model, the threshold current variation under TENS in terms of different electrode size and fiber diameters were investigated. The TENS computational results showed high consistency with psychophysical experimental results among 6 healthy volunteers.

The 3D TENS model were validated based on the action potential propagation, conduction velocity vs. fiber diameter relationship, and the strength-duration relationship. The action potential was excited in the Ranvier node S, and then spread along both directions of the axon. The node S is the closest Ranvier node to the dermis-fat junction where the electrical field intensity was enhanced due to the fat layers. The action potentials showed similar propagating characteristics under cathodic current stimulation with the reported computational result (Rattay, 1999). The conduction velocity matched well with reported experimental (Van Veen et al., 1995) and other sensory nerve fiber modeling work (Wesselink et al., 1999).

In strength-duration relationship, the chronaxie was not adopted in our model validation for the reason that many factors can affect the accuracy of the chronaxie measurement (Geddes, 2004), and then we used multiples of rheobase as our comparison standard between modeling and psychophysical experimental work in **Figure 6**. It is more meaningful to have the specific model parameters for each subject which provided similar trend

between the threshold current vs. duration. While, **Figure 6** does not show the relationship between the threshold current and duration, and the Y-axis represents the threshold strength with the unit of multiples of rheobase, which was equal to the threshold current divided by the respective rheobase current. From the sensitivity analyses results, it is clear that conductivity and thickness of the fat layer play a key role in the variation of threshold current during TENS. So the specific fat properties for each subject should be incorporated into the model parameters to have similar trends between the threshold current and duration among different subject. This work will be carried out in the future work.

Threshold current intensities showed significant difference between individuals in our psychophysical experiments, which was consistent with previous studies (Larkin et al., 1986; Ara et al., 2012). The sensitivity analyses of the electrical properties in the forearm tissue clearly showed that both the thickness and electric conductivity in the fat layer greatly affected the variation of threshold current intensities. It could be inferred that fat would play an important role in the tactile perception threshold in the psychophysical experiments.

As shown in **Figure 10D**, the relationship of threshold currents vs. electrode sizes in the psychophysical experiments was most comparable to the 8-7 µm Aβ fibers in the computational modeling results. It was indicated that Aβ nerve fibers with diameters of no less than 7 µm would be more associated with the comfortable tactile perception for the subjects. Nevertheless, there still exists modeling limitation. Although the strengthduration trend matches well in **Figure 6**, the threshold strength only represented multiples of rheobase, and the threshold current for a 3 µm diameter fiber is far more than the experimental values in **Figure 10D**. As a result, in our forthcoming work, a coefficient might be included to the computation of the threshold current, and to achieve similar threshold currentduration trends between computational and experimental results.

By establishing this novel 3D TENS model, we would directly observe the natural tactile sensation activities under the electrical stimulation. Some other issues can also be solved based on this model such as the relationship of nerve fibers activation with the two point discrimination under electrical stimulation (Solomonow et al., 1977) and the influence of nerve fiber regeneration or degeneration on the electrical stimulation (Simpson et al., 2013; Hebert et al., 2014).

#### CONCLUSION

In this paper, we introduced a novel Aβ TNF cable model used for TENS. The model is validated with previous studies and psychological experiments. At the same time, by combining 3D FEM and TNF modeling methods, we developed a novel 3D TENS model to explore the Aβ TNF excitatory threshold currents under TENS. This novel 3D TENS model was validated by comparing computational and experimental results in terms of conduction velocity vs. fiber diameter and strength-duration relationships. Then we investigated the effects of electrode size and fiber diameters on the threshold current intensities. The computational results illustrated that the larger the nerve fibers, the lower the threshold current intensities. These computational results corresponded to our psychophysical experimental data on six healthy subjects. This presented novel 3D TENS model would further guide the generation of artificial tactile sensation based on the surface electrodes.

#### ETHICS STATEMENT

This study was carried out in accordance with the recommendations of the Ethics Committee of Human and Animal Experiments of the Med-X Research Institute at Shanghai Jiao Tong University with written informed consent from all subjects. All subjects gave written informed consent

#### REFERENCES


in accordance with the Declaration of Helsinki. The protocol was approved by the Ethics Committee of Human and Animal Experiments of the School of Biomedical Engineering at Shanghai Jiao Tong University.

## AUTHOR CONTRIBUTIONS

XS contributed to the design of the overall experiment and simulation approaches; KZ conducted the simulation and psychophysical experiments. KZ wrote the first draft and XS also contributed to the whole manuscript revision. LL contributed to fiber model programming, and the manuscript revision. XW contributed to fiber model programming. All authors were active in the editing and revising processes of the manuscript. All authors read and approved the final manuscript.

## FUNDING

This research is supported by the National Natural Science Foundation of China (81671801, 61671300, 61472247), and the SJTU SMC-Morning Star Excellent Young Scholar-B (14X100010047).

#### ACKNOWLEDGMENTS

The authors would like to thank Dr. Menghui Li and Si Li for the modeling advice, Mengnan Li and Ying Chen for help in the psychophysical experiments.

## SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: http://journal.frontiersin.org/article/10.3389/fnins. 2017.00250/full#supplementary-material

nerve stimulation. J. Neural Eng. 12:066002. doi: 10.1088/1741-2560/12/6/ 066002


Yang, H., Meijer, H. G., Doll, R. J., Buitenweg, J. R., and van Gils, S. A. (2015). Computational modeling of Adelta-fiber-mediated nociceptive detection of electrocutaneous stimulation. Biol. Cybern. 109, 479–491. doi: 10.1007/s00422-015-0656-4

**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2017 Zhu, Li, Wei and Sui. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Improving Challenge/Skill Ratio in a Multimodal Interface by Simultaneously Adapting Game Difficulty and Haptic Assistance through Psychophysiological and Performance Feedback

#### Edited by:

Dingguo Zhang, Shanghai Jiao Tong University, China

#### Reviewed by:

Domen Novak, University of Wyoming, USA Nicolas Garcia-Aracil, Universidad Miguel Hernández de Elche, Spain

\*Correspondence:

Carlos Rodriguez-Guerrero carodrig@vub.ac.be

#### Specialty section:

This article was submitted to Neural Technology, a section of the journal Frontiers in Neuroscience

Received: 30 December 2016 Accepted: 12 April 2017 Published: 01 May 2017

#### Citation:

Rodriguez-Guerrero C, Knaepen K, Fraile-Marinero JC, Perez-Turiel J, Gonzalez-de-Garibay V and Lefeber D (2017) Improving Challenge/Skill Ratio in a Multimodal Interface by Simultaneously Adapting Game Difficulty and Haptic Assistance through Psychophysiological and Performance Feedback. Front. Neurosci. 11:242. doi: 10.3389/fnins.2017.00242 Carlos Rodriguez-Guerrero<sup>1</sup> \*, Kristel Knaepen2, 3, Juan C. Fraile-Marinero<sup>4</sup> , Javier Perez-Turiel <sup>4</sup> , Valentin Gonzalez-de-Garibay <sup>5</sup> and Dirk Lefeber <sup>1</sup>

<sup>1</sup> Robotics and Multibody Mechanics, Flanders Make, Vrije Universiteit Brussel, Brussels, Belgium, <sup>2</sup> Institute for Movement and Neurosciences, German Sport University Cologne, Cologne, Germany, <sup>3</sup> Human Physiology Research Group, Vrije Universiteit Brussel, Brussels, Belgium, <sup>4</sup> Biomedical Engineering, Fundacion CARTIF, Centro Tecnologico de Boecillo, Valladolid, Spain, <sup>5</sup> Department of Statistics and Operative Research, Universidad de Valladolid, Valladolid, Spain

In order to harmonize robotic devices with human beings, the robots should be able to perceive important psychosomatic impact triggered by emotional states such as frustration or boredom. This paper presents a new type of biocooperative control architecture, which acts toward improving the challenge/skill relation perceived by the user when interacting with a robotic multimodal interface in a cooperative scenario. In the first part of the paper, open-loop experiments revealed which physiological signals were optimal for inclusion in the feedback loop. These were heart rate, skin conductance level, and skin conductance response frequency. In the second part of the paper, the proposed controller, consisting of a biocooperative architecture with two degrees of freedom, simultaneously modulating game difficulty and haptic assistance through performance and psychophysiological feedback, is presented. With this setup, the perceived challenge can be modulated by means of the game difficulty and the perceived skill by means of the haptic assistance. A new metric (FlowIndex) is proposed to numerically quantify and visualize the challenge/skill relation. The results are contrasted with comparable previously published work and show that the new method afforded a higher FlowIndex (i.e., a superior challenge/skill relation) and an improved balance between augmented performance and user satisfaction (higher level of valence, i.e., a more enjoyable and satisfactory experience).

Keywords: psychophysiology, human–robot interaction, biocooperative, rehabilitation robotics, haptics, multimodal interfaces, biomechatronics

# INTRODUCTION

#### Human–Robot Interaction

In recent years, human–robot interaction (HRI) research has mostly focused on a master–slave relationship. However, as a natural step forward, robotics designers have found new and exciting frontiers to expand the existing research toward human-centered fields. Robots are now physically interacting with humans, especially in the medical domain (Colombo et al., 2001; Hochberg et al., 2012; Bonatti et al., 2014). This whole new field of robotics is evolving toward closing the gap between mechatronic systems and humans. In particular, robots have proved to be an excellent research tool in areas of neuroscience such as human motor control (Reinkensmeyer et al., 2004; Huang and Krakauer, 2009) and neurorehabilitation (Daly and Wolpaw, 2008; Hidler and Sainburg, 2011). Besides robots, other emerging technologies, such as virtual reality (Lewis and Rosie, 2012) and haptics (Okamura, 2009), are nowadays in the spotlight of neuroscientists. These new tools allow hacking and augmenting the human visual, auditory, or vibrotactile systems in a controlled way, thus opening up new routes for groundbreaking research.

# Patient-Cooperative Control

In neurorehabilitation, there are many examples of mechatronic systems designed for human–robot cooperative tasks. The main idea behind these systems is to assist the subject in completing a given task (e.g., reaching or grasping), typically by using a haptic controlled robotic device to deliver the forces needed for the correct completion of the objective (Liu et al., 2006; Nef et al., 2009; Song et al., 2013; Squeri et al., 2014; Pignolo et al., 2016). The intended output of these therapeutic robots is to promote engagement, human effort, and intention of movement and, therefore, to accelerate motor learning and induce neural plasticity (Reinkensmeyer et al., 2004; Edgerton et al., 2008; Huang and Krakauer, 2009; Pennycott et al., 2012). The control strategies of interactive robotic devices are typically designed based on the paradigm "assistance-as-needed," where human and robot cooperate to successfully complete a task, minimizing the intervention from the robotic device and maximizing that of the human. This patient-cooperative control is mostly based on performance measures such as kinematics, kinetics, muscle activity, etc.

Yet, in spite of human–robot cooperative tasks being "performed" accurately and successfully through these patientcooperative control paradigms, patient outcome and recovery did not improve significantly with the introduction of "patient-incharge" robotic rehabilitation devices. Issues such as emotional and cognitive stress, excessive physical work demand, discomfort, pain, boringness, and lack of motivation are rarely taken into account, yet play a key role in human motor control performance (Guadagnoli and Lee, 2004). For example, emotions such as anxiety, frustration, or stress can have a large impact on motor performance, speed, and variability (Coombes et al., 2006). As early as 1908, Yerkes and Dodson (1908) showed that human performance changes in relation to the level of arousal depending on the difficulty of the task: for a difficult task, high levels of arousal can be counterproductive, but if the task lies within the subject's capabilities, high levels of arousal can be beneficial instead (Yerkes and Dodson, 1908; Diamond et al., 2007).

According to Mihály Csíkszentmihályi (1992), cognitive states (e.g., focused, bored, motivated) can also change in relation to the challenge and skill level perceived by a given subject. These cognitive states can either increase or decrease performance accordingly. For instance, a state of boredom will tend to decrease performance, while a focused cognitive state will lead to better results by promoting mental engagement (Guadagnoli and Lee, 2004; Holden, 2005).

#### Assessment of Emotion

Emotions are accompanied by a set of somatic responses associated with autonomic nervous system (ANS) activity. For a specific emotional state, there exists a probable set of somatic and ANS outputs, although it remains unclear which kind of emotion is associated with which exact autonomic signature (Kreibig, 2010). Nevertheless, psychophysiology has been used to indirectly measure ANS-related responses to external stimuli that affect a person's mood and engagement in a variety of interactive scenarios (Kreibig, 2010). Autonomic responses of emotion are present in parameters such as heart rate (HR), skin temperature (SKT), galvanic skin response (GSR), and respiration rate (RR). A complete overview of these parameters has been summarized in a tag cloud based on 134 publications and explained in detail in Kreibig (2010).

#### Biocooperative Control

In an attempt to harmonize existing mechatronic systems with human beings, physiological computing has appeared as an enabling technology to give machines exteroceptive capabilities to measure and record physiological data on the emotional state of the user, allowing proactive and implicit adaptations of HRI in real time (Fairclough, 2009, 2011). This concept includes the human in the control loop and gives birth to a whole new set of possible biomechatronic devices capable of "biocooperative adaptation" (Pope et al., 1995; Prinzel et al., 2000).

In 2008–2009, the first ideas of using ANS-related signals as feedback information to close a biocooperative control loop took shape (Bonarini et al., 2008; Mihelj et al., 2009; Novak et al., 2009; Riener et al., 2009; Rodriguez Guerrero et al., 2009). In 2010, Novak et al. (2010a,b) attempted to determine which psychophysiological responses provide the most reliable information about the subject's psychological state during an upper-limb virtual reality (VR) task with the HapticMaster robot in healthy subjects (Novak et al., 2010a) and stroke patients (Novak et al., 2010b; Goljar et al., 2011). Koenig et al. (2010, 2011b) took this research a step further and determined the most suitable psychophysiological parameters to estimate the psychological state of subjects and neurological patients during walking in a driven gait orthosis Lokomat while completing a cognitively demanding VR task (Koenig et al., 2010, 2011e). Also in 2010, Rodriguez Guerrero et al. (2010) presented a step forward toward a working prototype of a biocooperative closed loop that included HR feedback to modulate a set of assistive forces in a robotic-aided neurorehabilitation scenario. Soon after, Novak et al. (2011a,b), Koenig et al. (2011a), Badesa

et al. (2012, 2014), and Morales et al. (2014) implemented a more complex closed-loop system that included feedback parameters related to HR, RR, GSR, and SKT (Novak et al., 2011a,b; Koenig et al., 2011b; Badesa et al., 2012, 2014; Morales et al., 2014). These studies aimed to change the difficulty level of the VR task depending on the emotional and cognitive load experienced by the subjects (Koenig et al., 2011b; Novak et al., 2011a,b; Badesa et al., 2012, 2014; Morales et al., 2014). In a previous work, Rodriguez Guerrero et al. (2013) developed a closed biocooperative loop with an auto-tuning fuzzy logic classifier to modulate the haptic assistance given to subjects using HR and GSR feedback. Notice that in contrast with the work by Koenig and Novak (Koenig et al., 2011b; Novak et al., 2011a,b), the goal was not to adapt the game difficulty and thus the perceived challenge level, but to adapt the level of assistance given, thereby attempting to affect the perceived skill level instead. Although the conclusions of Rodriguez Guerrero et al. have suggested that psychophysiological feedback alone can help improve the user experience, independent of whether there exists contextual information about the performance or not, they also suggested that the biocooperative system could be further improved by blending the haptic modulation together with adaptations in game difficulty (Rodriguez Guerrero et al., 2013).

## Article Contribution

The Yerkes and Dodson model (Yerkes and Dodson, 1908; Diamond et al., 2007) suggests that arousal levels have a strong impact on performance. It also suggests that high levels of arousal can be counterproductive, especially when the presented task becomes overwhelming to the subject. In contrast, if the task lies within the subject's capabilities, high levels of arousal can be beneficial instead. The contribution of this work expands upon this logic and improves the biocooperative architecture presented by Rodriguez Guerrero et al. (2013) by augmenting the system with contextual performance information that, when blended with the individual's psychophysiological feedback, provides a superior and more individualized continuous adaptation of the system to keep the task difficulty within its user's capabilities, improving subject performance and more importantly the overall user experience. Moreover, as there is still controversy on which psychophysiological parameters are eligible for use in biocooperative enhanced systems, the present study will also take into consideration, as was previously done by Novak et al. (2010a) and Knaepen et al. (2015), which psychophysiological parameters are the most suited for use in a closed biocooperative loop. Finally, a new metric (i.e., FlowIndex) is proposed to numerically quantify and visualize the challenge/skill relation.

# MATERIALS AND METHODS

## Subjects

Eleven subjects participated in this study involving an openloop and a closed-loop experiment. The subjects had no clinical records of neural or motor deficiencies. Six subjects [five males and one female, mean age 30.5 years (SD = 3.83)] participated in the open-loop experiment and 11 subjects [eight males and three females, mean age 35 years (SD = 7.34)] participated in the closed-loop experiment. All experimental procedures were performed according to the standards set by the Declaration of Helsinki for medical research involving human subjects. Upon arrival in the lab, subjects signed a written informed consent form. This research was approved by the medical ethics committee of Fundación CARTIF.

#### Instrumentation Virtual Reality Game

The virtual scenario was based on a popular "catch the falling droplet" reaching game (see Rodriguez Guerrero et al., 2013 for more details), which has also been used by other robotic rehabilitation platforms such as Armeo (Merians et al., 2009; Schwickert et al., 2011; Wittmann et al., 2016). The game interface and its mechanics are very simple and demand basically no explanation or extra human intervention. Droplets were spawned one at a time at a given velocity, vdroplet, defined in pixels/sec. Only one droplet was rendered at a time, to avoid pre-emptive maneuvers from the subject and maintain focus on the actual target. The subject was expected to catch the falling droplet by moving the PHYSIOBOT end effector, which was represented by a cup in the VR environment. Movements of the robot were projected back to the screen, thereby acting as a visual feedback. The initial positions of the droplets were predefined and generated in advance as an array of 100 positions. This array size is sufficiently large to prevent subjects from memorizing the positions, therefore avoiding habituation issues. The difficulty level of the task could be changed by altering either the distance between droplets or the falling speed. However, by keeping the same initial positions, the total amount of power needed for the task to be successfully completed in a certain amount of time was homogeneous among sessions with a defined velocity (i.e., experiments in the open loop). Therefore, the difficulty modulation was reduced to make changes in the velocity of the falling droplets only. The faster the droplets fall, the greater the amount of power needed to catch them in time, due to the permanent velocity-dependent force field rendered by the haptic device.

#### Haptic Assistance

The PHYSIOBOT is capable of rendering a wide range of 3D forces and primitives such as haptic walls, dampers, springs, textures, or a combination thereof (Rodriguez Guerrero et al., 2009). In this setup, a viscoelastic force tunnel was rendered so that it created resistive forces in the sagittal and vertical axes. On the task axis (i.e., transverse axis), a constant-viscosity force field (150 Ns/m) was rendered, such that velocity-dependent resistive forces made the physical task of moving the virtual cup a demanding exercise without the appropriate haptic assistance. The robot could also help the subject by applying assistive forces in the task direction. Forces could be modulated, placed, or removed online at any time.

Optimally, haptic assistance should be computed as the force needed for the robot to take the subject to the target with minimum intervention, and simultaneously promote engagement and self-initiated movements. Therefore, the assistive forces were designed along the task axis, as a critically damped second-order system. A critically damped response allows the system to marginally get to the target in time with the minimum amount of assistance needed to complete the task (see Rodriguez Guerrero et al., 2013 for further details of this calculation). Besides the actuation strategy in terms of magnitude, the timing of the assistive forces needed to be determined. Most of the existing physical human–robot cooperative systems, such as those in rehabilitation robotics, render assistive forces to correct erroneous movements with the help of virtual springs. The problem with this method is that it creates the feeling of being helped rather than the feeling of being more capable/in control of moving the virtual object. The haptic assistance in PHYSIOBOT enhances, rather than corrects, the subject's movements by only giving assistance when the subject has initiated the movement in the target direction. The goal is to "hide" the sensation of being helped and to promote the feeling of being more skillful and capable, a very important feature achieved in this design.

#### Biocooperative Control

In a biocooperative control loop, the human plays a key central role. **Figure 1A** shows the human as a plant in a classical control scheme. The human block is where the final action lies, and also where the primary feedback is taken from. **Figure 1B** shows a greatly simplified human control system that only involves some specific portions of the nervous system directly affected by the multimodal interface: the audiovisual stimuli coming from the virtual environment and the interaction forces coming from the physical human–robot interaction with the PHYSIOBOT.

The biocooperative high-level controller used was a Takagi– Sugeno-type fuzzy inference system (FIS) synthesizing several fuzzy rules (see Appendix A in Supplementary material for a complete list of the rules used). Fuzzy logic has already been used in psychophysiological and emotion modeling (Mandryk and Atkins, 2007; Badesa et al., 2012, 2014; Rodriguez Guerrero et al., 2013; Lledo et al., 2015). This system combines data fusion, classification, and control in one simple and wellknown framework. The system has four inputs and two outputs (**Figure 2**).

The results from the open-loop experiments suggested (Section Open-Loop Results: Open-Loop Results) that the most useful signals to use as inputs for the controller were HR and the tonic (i.e., skin conductance level, SCL) and phasic (i.e., skin conductance response frequency, SCR frequency) components of the GSR signal. Performance was the fourth input of the highlevel controller. The outputs were the haptic assistance and the changes in game difficulty (i.e., changes in the speed at which the droplets fell). The haptic assistance output was a value between 0 and 1, where 0 represents no assistance and 1 represents full critical assistance. This allows the system to act accordingly when either of the two undesired situations (i.e., under- or overchallenge) occurs for the subject. For instance, if the overall state of the subject indicates that he/she is underchallenged, the assistance will cease, and the difficulty of the task will be augmented accordingly. If, on the other hand, the overall state of the subject indicates that he/she is overchallenged, the assistance will be computed and delivered according to his/her ANS activity, and the game difficulty may be reduced as well.

An intrinsic problem in biocooperative controller design is the inter-/intrasubject variability. In Badesa et al. (2012), fuzzy logic was used but no adaptation algorithm was implemented to deal with this effect. This is important as any consumed substance such as β-blockers, antidepressants, coffee, or sugar may drastically impact baseline ANS readings. In order to deal with the problem of the inter-/intrasubject variability, the FIS was tuned to adapt to the immediate state of each subject. Every FIS has two basic tunable sets of parameters, the rules base, and the membership functions (MFs) for each I/O. The MFs were automatically calibrated after an open-loop "calibration task" designed primarily for that purpose. In the 3-min non-hapticassisted VR calibration task, the difficulty level was adapted such that every time the subject caught a droplet, the system augmented the falling speed by 2 pixels/s; on the contrary, if the subject missed a droplet, the system reduced the speed by the same amount. The idea behind the calibration task was that the subject's score would steadily increase over time until the challenge level started to approach his/her skill level. After reaching this tipping point, the subject typically started to get drastically aroused as their skill limit was constantly being challenged by the difficulty of the game. The data obtained at the end of the calibration task can be used to tune the MF of the HR, SCL, and SCR frequency. We refer the reader to Rodriguez Guerrero et al. (2013) for more details regarding MF auto-tuning. Besides these two parameters, the rules base and the MFs, a performance parameter (i.e., based on the subject's performance during the VR game) was used in the high-level controller of the biocooperative feedback loop.

The algorithm shown in **Figure 3** was used to compute the "performance" input to the fuzzy controller based on the trend of the global score. The idea behind this design was that, no matter how well subjects were performing globally (i.e., their total game score), small sustained changes in performance would always influence their emotions. A "trend" or winning/losing streak will often boost or deteriorate our confidence (i.e., dominance value). Therefore, if two droplets were caught/missed in a row, the performance sign changed accordingly in order to take preemptive actions in the controller logic to avoid frustration.

The fuzzy controller worked at a sample rate of 100 Hz, although both of its outputs (i.e., haptic assistance and game difficulty) were updated every epoch (n; the period between two consecutive droplets). Thus, the output of an epoch (n) was the result of the average of the calculations made in the epoch (n−1). As such, the calculations and effects of the inputs that manage the signals were synchronized with the performance input, which was updated every epoch (n). This averaging contributes to filtering the outputs, especially the haptic assistance, so that the subject does not feel as though it is changing continuously, thereby producing a more natural, less noisy feeling of the haptic assistance and avoiding a constantly varying droplet speed.

## Experimental Design

Upon arrival in the lab, the purpose and procedure of the experiment were explained and the subjects signed the written

FIGURE 1 | (A) Macro view of the biocooperative control loop. Three different interacting blocks can be distinguished. From left to right, the red block represents the high-level controller, which makes decisions based on the overall state of the system, including the subject, and sends actions regarding the difficulty of the game and the level of haptic assistance to the multimodal environment represented by the central blue block. The yellow block represents the human, whose responses to the audiovisual and haptic feedback generate ANS responses and interaction forces, which are, in turn, used to close the biocooperative loop. (B) Effects of the multimodal interface (i.e., PHYSIOBOT and VR) on the human control system. External stimuli are captured by the afferent neurons in the peripheral nervous system (PNS) and distributed over different cortical and subcortical levels to the CNS: the thalamus, the limbic system, and the cortex. All external sensory input is received by the thalamus, which sends information simultaneously to the cortex for higher-level processing, and directly to the limbic system (LeDoux, 1995). Part of the information goes to the hypothalamus, which secretes neurohormones that either stimulate or inhibit the secretion of the pituitary hormones that modulate the behavior of the ANS. Simultaneously, the external stimuli make their way to the sensorimotor cortex, which modulates ongoing movements and motor effort, and the prefrontal cortex, which controls the subject's psychophysiological state and turns it into a conscious state and thus a mood or feeling. ANS signals are then fed to the high-level controller and the interaction forces are fed to the multimodal environment as "human in the loop" feedback signals.

informed consent form and were equipped with the Biopac MP150 data acquisition system for measuring ANS responses (i.e., ECG, GSR, and SKT). Next, subjects participated in two different experiments involving a VR task of catching falling droplets by manipulating the PHYSIOBOT end effector (see Rodriguez Guerrero et al., 2013 for a picture of the experimental setup). During each session, electrophysiological data, game performance, and physical performance were measured, and immediately following each session, the emotional status of the subject was registered.

#### Open-Loop Experiment

The first experiment was an open-loop experiment (n = 6) consisting of a warm-up of 3 min followed by four game sessions of 5 min each. The 3-min warmup time (i.e., no physical or mental exercises were performed) was used to stabilize the sensor readings to a baseline. The four experimental sessions contained four different levels of difficulty ranging from very easy to very hard, starting with difficulty level 3, then difficulty 4, followed by difficulty 2, and finally difficulty level 6 (**Table 1**). All difficulty levels were chosen in advance, prior to the experiment.

#### Closed-Loop Experiment

The second experiment was a closed-loop experiment (n = 11) consisting of a control task of 5 min (i.e., the calibration task, hereafter referred to as CT), followed by a 5-min resting period and a 5-min closed-loop trial (hereafter referred to as haptic and difficulty modulation (HDM) controller). The obtained data were compared to the results obtained in Rodriguez Guerrero et al. (2013), where a haptic-assisted-only controller was used (hereafter referred to as HAO).

#### Data Acquisition and Analysis VR Score

The score on the VR task was calculated based on the game score, which added one point for each droplet caught and subtracted one point if a droplet was missed. The score mean value with standard deviation (SD) is presented for each open- and closed-loop experiment in the results.

#### Physical Performance

Physical performance was computed by measuring the interaction force with the PHYSIOBOT haptic device by means of a force sensor mounted between the end effector and the human. The mean absolute force (MAF) expressed in newtons (N) with SD was used as an indicator of physical performance and was computed by extracting the mean of the absolute value of all of the forces over the task axis during a session, sampled at a rate of 1 kHz.

#### Electrophysiological Assessment

For the physiological signal recording, the Biopac MP150 data acquisition system<sup>1</sup> with three amplifiers, ECG100c<sup>2</sup> , GSR100c<sup>3</sup> , and SKT100c<sup>4</sup> , was used. All recordings were collected online during the game time through UDP/IP at a frequency of 100 Hz. All physiological measurements were recorded through noninvasive, ambulatory sensors, using simple, and fast Velcro attachment electrodes for GSR and SKT and sticky disposable electrodes for the ECG. The mean values for the 5-min experimental sessions (i.e., open- and closed-loop experiments) were calculated. From the ECG, the intervals between two heartbeats (NN intervals) were extracted in order to calculate the mean HR value with SD. Two components were extracted from the GSR signal: SCL and SCR frequency. The mean SCL with SD, which is the baseline level of skin conductance, was calculated for each game difficulty level. The SCR frequency represents increases in skin conductance followed by a return to the tonic level. The mean SCR frequency with SD as well as mean SKT with SD are presented in the results.

<sup>1</sup>http://www.biopac.com/data-acquisition-system-mp150-system-glp-win.

<sup>2</sup>http://www.biopac.com/ecg-electrocardiogram-amplifier.

<sup>3</sup>http://www.biopac.com/gsr-eda-galvanic-skin-response-amplifierelectrodermalactivity.

<sup>4</sup>http://www.biopac.com/skin-temperature-amplifier.

#### Emotion Assessment

To subjectively assess emotion, the self-assessment manikin (SAM) test was used (Posner et al., 2005, 2009) (see Appendix A in Supplementary Material). We implemented a computer

TABLE 1 | The four levels of task difficulty in the open-loop experiment (n = 6).


program that presented the SAM test at the end of each session in an automated way, such that the subjects were able to select their choice without any external human intervention that might have otherwise biased their answers. The three emotional dimensions used were arousal, valence (i.e., pleasure), and dominance (i.e., sense of control of the situation). Each dimension was rated on a nine-point scale (i.e., 1–9). For each dimension, the mean value with SD is presented.

#### Vector Representation, FlowIndex, and FlowDir

In this article, a new and simple set of tools is proposed for analyzing, visualizing, and comparing the impact of a biocooperative controller based on arousal and dominance scores: the vector representation, FlowIndex, and FlowDir. These vectors are tools to graphically and numerically evaluate the results of a given control strategy designed to act on the challenge/skill perception of a particular task.

The vector representation (**Figure 4**) is a visual way to present the challenge/skill ratio and compare the [dominance, arousal] results (i.e., from the SAM questionnaire or as a result of the output of an automatic classifier) of one or more tasks (i.e., vector **A**) relative to the maximal/optimal challenge/skill ratio (i.e., reference vector **F**). This metric is a mathematical representation inspired by a theory used in psychology termed "Flow statem," where an optimal experience motivates people to further learn or stay committed to a task (Csikszentmihalyi and LeFevre, 1989; Engeser and Rheinberg, 2008). The challenge level relates to the arousal domain and can be modulated by means of the game difficulty level, and the skill level is related to the dominance domain and can be modulated by means of the haptic assistance; see Equation (Hochberg et al., 2012). Therefore, the equivalence

representation affords a quick glance into the performance of a given session. In this example, the results point toward the relaxation/boredom zone of the figure and therefore it is unlikely that the experience was enjoyable or engaging but rather boring instead.

for their ratios can be defined as:

$$\frac{\text{Challenge}}{\text{Skill}} \cong \frac{\text{Arrosal}}{\text{Dominance}} \tag{1}$$

Since this relation has two components, it can be expressed and plotted as a vector (**Figure 4**).

FlowIndex uses the obtained pair **A**[dominance, arousal] represented in vector form, and quantifies how alike vector **A** (blue) and the ideal vector **F** (solid black) are (**Figure 4**). This form of vector similarity first uses the normalized scalar projection (NSP) (Equation 2) of **A** projected onto **F**. The result of this calculation is a scalar bounded between 0 and 1 that gives information on how **A** projects onto **F**. Then, the angular similarity (AS) (Equation 3) is used to calculate the similarity between the two vector angles and gives again a value bounded between 0 and 1 (i.e., where 0 is completely orthogonal and 1 is perfectly parallel). By multiplying the two obtained results (Equation 4), the AS penalizes vectors that are diverging from **F** (i.e., the angle between them is >0). Notice that both NSP and AS are bounded between 0 and 1 and, therefore, the FlowIndex results will always be bounded in the same way. The FlowIndex is then a scalar value that represents how alike **A** and **F** are in terms of both angle and magnitude.

The higher the FlowIndex value is, the closer we are to being within the flow channel (i.e., the dotted lines delimiting the ideal range of challenge/skill ratio in **Figure 4**) and, therefore, the more pleasant and exciting the experience will be.

$$\text{NSP} = \frac{\text{A.}\hat{F}}{||F||} \tag{2}$$

$$AS = \cos \theta \,\,=\left(\frac{A.F}{||A|| ||F||}\right) \tag{3}$$

$$\text{FlowIndex} = \text{NSP} \ast \text{AS} \tag{4}$$

Since FlowIndex is always positive and bounded between 0 and 1, this metric alone cannot reveal if the resulting vector is pointing toward the frustration or relaxation zone. Nevertheless, the direction of the FlowIndex can be calculated. This computation is straightforward and can be performed as follows:

$$FlowDir = \begin{cases} Frustration & sign \left(A\_\mathcal{Y} - A\_\mathcal{X}\right) = 1 \\ Flow & sign \left(A\_\mathcal{Y} - A\_\mathcal{X}\right) = \mathbf{0} \\ Relation & sign \left(A\_\mathcal{Y} - A\_\mathcal{X}\right) = -1 \end{cases} \quad (5)$$

Equation (5) gives a value of 0 whenever vectors **A** and **F** are parallel (i.e., flow), 1 whenever vector **A** is pointing toward the frustration zone, and −1 whenever it is pointing toward the relaxation zone. For this given example, the FlowIndex of **A** = [8,5] is 0.65 with FlowDir = Relaxation.

#### Statistical Analysis

Statistical analysis was performed using the Statistical Package for Social Sciences 24.0 for Windows statistical software (SPSS Inc., Chicago, IL, USA). Data are presented as mean with SD. Statistical significance was accepted at p < 0.05. As n = 6, nonparametric statistics were applied and effect sizes were calculated. Eight variables were extracted from all recorded data: valence, dominance, and arousal were extracted from the SAM; MAF from the haptic device; and SKT, HR, SCL, and SCR frequency were derived from physiological recordings. To assess differences between the four difficulty levels in the open-loop experiment (n = 6), a Friedman's analysis of variance (i.e., ANOVA) was used. As an estimate of effect size, Kendall's W coefficient was calculated, where a value of W = 0 indicates no relationship and a value of W = 1 indicates a perfect relationship (Tomczak and Tomczak, 2014). The following null hypothesis was tested, for which µ<sup>i</sup> is the mean of the observed variable for each difficulty level (i.e., subindex i corresponds to difficulty levels 2, 3, 4, and 6): no significant differences between the four difficulty levels, H0: µ<sup>2</sup> = µ<sup>3</sup> = µ<sup>4</sup> = µ6.

Furthermore, Spearman's rank correlation coefficient was used to measure the strength of the association between the subjective data from the SAM questionnaire and objective data such as HR, SCL, SCR, SKT, and MAF.

#### RESULTS

#### Open-Loop Experiment

**Figure 5** shows the mean values for all of the eight measured variables (n = 6) in the order of difficulty i.e., 2, 3, 4, and 6 (note that the difficulty levels were presented to the subjects in a different order; see **Table 1**). **Table 2** shows the outcome of all eight variables for the tested hypothesis (H0) with the corresponding effect size. Significant differences (p < 0.05) and large effect sizes between all four difficulty levels were found for arousal, dominance, MAF, SCR frequency, HR, and SCL (**Figure 5** and **Table 2**).

The results of the Spearman's rank correlation analysis between the subjective variables of arousal, dominance, and valence, on the one hand, and objective psychophysiological responses (i.e., HR, SCL, SCR, SKT) and physical effort (i.e., MAF), on the other hand, can be found in **Table 3**.

#### Closed-Loop Experiment

**Table 4** shows the mean values with SD for the total game score, MAF, HR, SCL, SCR frequency, arousal, dominance, and valence during the 5-min calibration task and the 5-min closed-loop task. The percentage change between the results of the closedloop experiment and the baseline, i.e., the calibration task, are presented in **Figure 6**.

#### DISCUSSION

#### Open-Loop Results

For a psychophysiological signal to be useful as feedback information for control purposes, it has to change considerably and coherently in relation to the task difficulty. From the results shown in **Table 2**, we can infer that arousal, dominance, MAF, SCR frequency, HR, and SCL changed significantly across the four difficulty levels (i.e., p < 0.05). These results are coherent with previous studies on psychophysiological changes during mixed physical/psychological HRI tasks (Rodriguez Guerrero et al., 2010; Novak et al., 2010a,b, 2011a,b; Koenig et al., 2011a,c;

TABLE 2 | Significant differences and effect sizes between game difficulty levels for the eight measured variables.


\*Significant difference, p < 0.05.

TABLE 3 | Correlations between subjective and objective measurements.


\*Significant correlation, p < 0.05.

Badesa et al., 2012, 2014; Morales et al., 2014) and show that, in this experiment, HR, SCL, and SCR were the most sensitive physiological parameters to discriminate between the difficulty levels of the VR task. For HR, the observations made went further and showed that this variable exhibited a considerable inertia when preceded by a higher level of physical activity, as can be seen in **Figure 5** (i.e., delayed effect of task presentation order, see **Table 1**; more specifically, a continued effect of level 4 on level 2 can be seen). This delay in HR decrease during recovery from a higher level of physical effort is a well-known effect (Shetler et al., 2001; Vijayalakshmi et al., 2014). In healthy subjects, a decrease of 15–20 beats per minute in the first minute of recovery has been shown to be typical (Shetler et al., 2001). This slower vagal reactivation may have to be taken into account in future research, as such inertial effects may become important when the subject is exposed to a wider range of physical loads over an extended period of time. In such cases, adding an appropriate corrective factor to the HR might be beneficial. The SCL and SCR behaved similarly to HR and increased significantly with increasing level of game difficulty. This is comparable to what has been found in other studies (Novak et al., 2010a; Badesa et al., 2012, 2014; Morales et al., 2014; Knaepen et al., 2015). The SKT was the only physiological variable that did not differ significantly between the four difficulty levels. Although decreases in SKT have been put forward as a marker to detect changes in mental workload (Ohsuga et al., 2001; Koenig et al., 2011e), other studies have found no significant differences for SKT with changes in mental workload (Novak et al., 2010a; Badesa et al., 2012; Knaepen et al., 2015) or found that variations depended on the test subject (Morales et al., 2014). Novak et al. (2010a) proposed that a certain threshold of mental workload should probably be exceeded before the SKT starts to decrease significantly (Novak et al., 2010a). The hardest level (i.e., level 6, see **Table 1**) in this study was designed to be demanding but not frustrating, and as such this threshold might not have been exceeded, which could explain why the SKT did not change significantly. Moreover, the slow response of the SKT, as also shown by Novak et al. (2010a), makes it difficult to use it in a closed-loop biocooperative scenario. Therefore, we have discarded SKT as a possible feedback signal for the proposed closed-loop controller. As a result, HR, SCR, and SCL proved to be the most useful indicators of psychophysiological activity in a mixed physical/psychological load scenario.

To date, it remains unclear which kinds of emotion are associated with which kinds of objective autonomic signature (Kreibig, 2010). Therefore, objectively measuring emotion through physiological signals remains challenging. In our study, a subjective SAM test was used in order to afford a numerical measurement of perceived emotion. However, it is difficult to integrate a subjective measure of emotion into the control loop of assistive robotic devices. Therefore, we also performed correlations between SAM scores and the other psychophysiological and physical parameters. Results from the Spearman correlation analysis (**Table 3**) showed that, while HR did not correlate with any of the subjective emotional scores, arousal was positively correlated (r > 0, p < 0.05) with SCL and SCR. Dominance, on the other hand, was negatively correlated with SCL and SCR (r < 0, p < 0.05), while valence was only positively correlated with SCL (r > 0, p < 0.05). These few significant correlations show that it remains difficult to achieve mapping between physiological signals and the emotional state of a subject based on a few physiological measures. The significant correlations between MAF, on the one hand, and arousal, valence, and dominance, on the other hand, further confirm that additional contextual information and probably more physiological parameters are necessary to accurately predict the emotional states of subjects during HRI. This corresponds to what Kreibig (2010) pointed out: as emotions consist of an integrated variety of physiological responses, it is important to select a sufficient number of response measures to allow for the response pattern and its variations to be accurately identified (Kreibig, 2010). Further research will have to carefully consider which additional physiological parameters should be included in the biocooperative controller in order to correctly identify and influence emotional states of the subjects.

#### Biocooperative Closed-Loop Control

Results from the SAM questionnaires and FlowIndex suggest that the user satisfaction and challenge/skill relation were effectively improved in this work, where haptic and difficulty modulation (HDM) was implemented, compared to the calibration task (CT) and the haptic-assisted-only (HAO) results presented in Rodriguez Guerrero et al. (2013). We invite the reader to take a look at Supplementary Figure 1 (i.e., the SAM scale) in


TABLE 4 | Mean values with SD for the eight measured variables during the 5-min calibration and 5-min closed-loop tasks for the HDM and HAO controller.

CT\_HDM, calibration task in HDM experiment; HDM, experiment with haptic and difficulty modulation controller described in this paper; CT\_HAO, calibration task in HAO experiment; HAO, haptic-assistance-only experiment from 2013 (Rodriguez Guerrero et al., 2013); MAF, mean absolute force; HR, heart rate; SCL, skin conductance level.

Appendix A, to provide visual support during the upcoming discussion.

#### User Satisfaction

The dominance value obtained for HDM (8.09) was superior compared to those obtained for CT (3.72) and for the HAO controller (Hidler and Sainburg, 2011; **Table 4**). An improved dominance values translates into an improved "perceived skill." Besides dominance, the valence scores showed the most interesting and desired improvements. The HDM controller scored 8.18 on the valence scale, while values of 4.09 and 6.27 were obtained for the CT and the HAO, respectively. The differences in standard deviation between HAO and HDM for both dominance and valence can be neglected (i.e., 0.07 and 0.59, respectively). The combination of high dominance (i.e., feeling under control of the situation) and high valence (i.e., pleasure) while still being challenged is likely to influence motivation and, thus, to promote future practice and total duration of training, which, in turn, might influence the success of motor learning (Shetler et al., 2001; Guadagnoli and Lee, 2004; Vijayalakshmi et al., 2014).

#### User Performance

The results for the total game score obtained with HDM (56.72) were greatly improved compared with the scores obtained for the CT (29.72). However, when compared to the total game score for HAO (62.63), the results ended up being lower by a mean of six points per session. Score variability was expected, as not only the skill but also the assistance given to each subject was different. The assistance given depended entirely on the subject's individual performance and ANS activity, which, of course, is highly dependent on how each subject reacts to pleasant/unpleasant situations. As the spawn frequency of the droplets was also being controlled by the game difficulty, the maximum possible number of points varied with each subject's skill. The more skillful a user was, the more challenging the session would likely be and the more droplets would be rendered. Nevertheless, as the HAO difficulty was not controlled, HDM might be better suited for longer training sessions, as it can provide larger/faster adaptation steps and preemptive maneuvers when fatigue starts to appear, by using the extra information from the performance input.

#### Vector Representation, FlowIndex, and FlowDir

Within the framework of this article, a new set of tools was proposed based on arousal and dominance scores, to indirectly evaluate the impact of a biocooperative controller: the vector representation, FlowIndex, and FlowDir. These tools allow the graphical and numerical evaluation of the results of a given control strategy designed to improve the perception of a given task. An optimal flow level, or optimal challenge/skill ratio, results in the high commitment of a person to a given task (Engeser and Rheinberg, 2008) and also stimulates motor learning (Guadagnoli and Lee, 2004). Previous studies have shown that, for optimal motor learning, the motor task should challenge, arouse, and excite the subject, while not being too stressful or boring (Maclean and Pound, 2000; Guadagnoli and Lee, 2004; Koenig et al., 2011d; Rodriguez Guerrero et al., 2013). The arousal results for HDM (6.72) decreased compared to those for the CT (8.45) and HAO (7.45). Since HDM is able to control the game difficulty (i.e., challenge), arousal can also be greatly influenced by its actions. The decrease in arousal obtained by the HDM controller seems to have played a favorable role for this controller, keeping the FlowDir in the relaxation zone (**Figure 7**). This is in contrast with the FlowDir for HAO, which was located in the frustration zone (**Figure 7**). HDM also performed better on the FlowIndex (i.e., 0.82 vs. 0.80 obtained for the HAO controller; see **Table 5**), meaning that it was closer to the optimal challenge/skill ratio. Although this difference in FlowIndex between HAO and HDM is small, HDM also afforded considerably higher values for valence (**Figure 6**). It seems that when subjects are close to the flow zone, they perform better under the relaxation zone rather than under the frustration zone (**Figure 7**). While the evidence for the previous statements is yet to be fully proved, the assumptions are supported by the Yerkes and Dodson model (Yerkes and Dodson, 1908; Diamond et al., 2007), as high levels of arousal (such as those triggered by frustration) can be counterproductive, and the fact that this controller setup led to a more enjoyable experience. Not only did the biocooperative controller presented in this work perform better, it also added greater flexibility to tune the system with

an extra degree of freedom by controlling the difficulty of the game. This allows modulation of the challenge/skill relation, with challenge being controlled by the game difficulty and skill by the haptic assistance. This could certainly be exploited in order to fine-tune the system rules and, thus, to maximize the FlowIndex and obtain a FlowDir closer to the optimal flow zone.

#### Study Limitations

The sample size used here, especially for the open-loop setup (n = 6), was small. Therefore, caution should be taken in attempting to generalize the statistical results. The significant differences and large effect sizes indicate an interesting effect that should be confirmed in future studies with a larger sample size and thus a higher statistical power. Although, the difficulty settings for the open-loop trials were satisfactory for the subjects who tested them prior to the experiments, it seems that we could have added another higher level of game difficulty. In this study, we visualized more the effects of the low difficulty range (i.e., game difficulty levels 2, 3, and 4). Only one level of higher game difficulty was present (level 6). The addition of another session with an even higher level of difficulty would have allowed us to more thoroughly explore the effects of the biocooperative controller, as with the difficulty levels used the subjects never really became frustrated and their dominance levels therefore started to decay only at level 6. It would have been useful to explore these effects over longer periods of time to better understand the signal dynamics and compensate for phenomena such as signal inertia.

It would also have been useful to investigate and compare the HAO controller to the HDM controller over longer sessions. It is probable that the improved HDM adaptation could have clearer benefits (i.e., higher scores for FlowIndex and valence) over longer sessions, where fatigue, boredom, or frustration would be more frequent.

# CONCLUSION

User satisfaction and augmented human performance are the most important features that any human–machine interface can achieve. The architecture and control strategies presented in this work effectively delivered a well-balanced tradeoff between these key design points. For a human-cooperative controlled interface such as the one presented in this work, augmenting human performance could be as easy as delivering the needed assistance to complete the task. However, this lacks the engagement component that drives motivation and, therefore, it always coincides with the tradeoff of deteriorating user satisfaction. Consequently, a biocooperative interface should ideally deliver individually tailored adaptation based on the user's psychophysiological state and (if available) contextual performance information. Augmenting the system with contextual performance information together with the individual's psychophysiological feedback provides a superior and more individualized adaptation of the system to maintain the task difficulty within its user's capabilities, improving subject performance and more importantly the overall user experience.



CT\_HDM, calibration task in HDM experiment; HDM, experiment with haptic and difficulty modulation controller described in this paper; CT\_HAO, calibration task in HAO experiment; HAO, haptic-assistance-only experiment from 2013 (Novak et al., 2011b).

By modulating both haptic assistance and game difficulty, designers have greater potential to control dominance and arousal levels. This has a direct impact on the capabilities to control the challenge/skill relation of a given task.

Psychophysiological activity under physically demanding scenarios can be measured by using ANS-related signals such as those used in this work. However, the direct mapping against subjective emotional scales like arousal, valence, and dominance seems to remain a challenge that could soon be possible to achieve with modern deep-learning algorithms. Nevertheless, by using psychophysiological feedback in conjunction with additional contextual information (i.e., performance), an overall estimation of the emotional state of a subject can still be made and exploited for control purposes with good results.

Although the given inputs for FlowIndex and FlowDir are based on the SAM scale, future work needs to be carried out to achieve mapping between arousal and dominance, on the one hand, and psychophysiological signals, on the other hand, so that the calculations can be performed online in real time. This could allow using the index for control or optimization purposes, taking actions based on minimizing the error between the expected output (i.e., flow vector) and the given desired input and, thus, maintaining an optimal flow level.

# ETHICS STATEMENT

This study was carried out in accordance with the recommendations of "the medical ethics committee of Fundación CARTIF" with written informed consent from all subjects. All subjects gave written informed consent in accordance with the Declaration of Helsinki. The protocol was approved by "the medical ethics committee of Fundación CARTIF."

# AUTHOR CONTRIBUTIONS

CR, DL, JF, and JP conceived and designed the experiments. CR developed the technology for the methods to build the experimental setup, robotics and VR methods, metrics, algorithms, and data processing. KK contributed to the design and layout of the article, tables, additions to the bibliography, and extensive revisions. DL, JF, and JP helped to obtain the funding for the project that financed this research. VG helped with the formulation of the null hypothesis and data processing for the statistics.

#### FUNDING

This research was funded by the Spanish Ministry of Sciences and Innovation (DPI2009-10658) and a research grant (SAN/126/2009) from the Castilla y Leon Council of Health.

## ACKNOWLEDGMENTS

The authors would like to acknowledge DL from the Vrije Universiteit Brussel for the economical and motivational

## REFERENCES


support. Last but not least, we would also like to acknowledge all of the people who voluntarily participated in this study.

## SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: http://journal.frontiersin.org/article/10.3389/fnins. 2017.00242/full#supplementary-material


development, and psychopathology. Dev. Psychopathol. Summer17, 715–734. doi: 10.1017/s0954579405050340


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2017 Rodriguez-Guerrero, Knaepen, Fraile-Marinero, Perez-Turiel, Gonzalez-de-Garibay and Lefeber. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Physiological Signal-Based Method for Measurement of Pain Intensity

Yaqi Chu<sup>1</sup> , Xingang Zhao<sup>1</sup> \*, Jianda Han<sup>1</sup> and Yang Su<sup>2</sup>

*<sup>1</sup> State Key Laboratory of Robotics, Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang, China, <sup>2</sup> Shengjing Hospital of China Medical University, Shenyang, China*

The standard method for prediction of the absence and presence of pain has long been self-report. However, for patients with major cognitive or communicative impairments, it would be better if clinicians could quantify pain without having to rely on the patient's self-description. Here, we present a newly pain intensity measurement method based on multiple physiological signals, including blood volume pulse (BVP), electrocardiogram (ECG), and skin conductance level (SCL), all of which are induced by external electrical stimulation. The proposed pain prediction system consists of signal acquisition and preprocessing, feature extraction, feature selection and feature reduction, and three types of pattern classifiers. Feature extraction phase is devised to extract pain-related characteristics from short-segment signals. A hybrid procedure of genetic algorithm-based feature selection and principal component analysis-based feature reduction was established to obtain high-quality features combination with significant discriminatory information. Three types of classification algorithms—linear discriminant analysis, *k*-nearest neighbor algorithm, and support vector machine—are adopted during various scenarios, including multi-signal scenario, multi-subject and between-subject scenario, and multi-day scenario. The classifiers gave correct classification ratios much higher than chance probability, with the overall average accuracy of 75% above for four pain intensity. Our experimental results demonstrate that the proposed method can provide an objective and quantitative evaluation of pain intensity. The method might be used to develop a wearable device that is suitable for daily use in clinical settings.

Keywords: feature extraction, feature selection and reduction, pain intensity quantification, physiological signals, pattern classification

# INTRODUCTION

From an evolutionary angle of vision, pain is considered as an unpleasant emotional and sensory experience that may be associated with a real or potential tissue damage (Hudspith et al., 2006; Loeser and Treede, 2008). Nowadays, pain is one of the most significant clinical symptoms that can be utilized to detect the acuteness and degree of a patient's injury. Moreover, pain is considered as a warning to danger, and often indicates the site of a lesion; its intensity serves as an indicator of patient well-being. A precise prediction of pain intensity could provide valuable insights in situations in which it can be utilized effectively to ultimately determine the position of pain and accordingly to formulate a reasonable therapeutic schedule. Therefore, pain prediction could enhance the quality of daily life for patients in the health-related field of rehabilitation, in-home healthcare and medical emergency services.

As a subjective first-person experience (Loeser and Treede, 2008), pain does not only reflect perceptual status but would also be substantially affected by psycho-physiological conditions (e.g.,

#### Edited by:

*Dingguo Zhang, Shanghai Jiao Tong University, China*

#### Reviewed by:

*Jun Xie, Xi'an Jiaotong University, China Long Cheng, Institute of Automation (CAS), China*

#### \*Correspondence:

*Xingang Zhao zhaoxingang@sia.cn*

#### Specialty section:

*This article was submitted to Neural Technology, a section of the journal Frontiers in Neuroscience*

Received: *18 February 2017* Accepted: *01 May 2017* Published: *26 May 2017*

#### Citation:

*Chu Y, Zhao X, Han J and Su Y (2017) Physiological Signal-Based Method for Measurement of Pain Intensity. Front. Neurosci. 11:279. doi: 10.3389/fnins.2017.00279*

**181**

fluctuations in attention or alertness) and even psychosocial contexts (e.g., the age or gender of individuals) (Aslaksen et al., 2007). In clinical practice, self-description is the gold standard approach for the determination of the absence, presence, and intensity of pain (Cruccu et al., 2010; Haanpää et al., 2011), such as Numeric Pain Rating Scales (NPRS), Verbal Rating Scales (VRS), and Visual Analog Scales (VAS) (Frampton and Hughes-Webb, 2011). These self-reported scales are especially well applied and validated in cancer patients (Caraceni et al., 2002). In addition, the McGill Pain Questionnaire (MPQ) and Brief Pain Inventory are also used to assess the wider pain perception in multidimensional scales (Frampton and Hughes-Webb, 2011). While self-descripted pain provides important clinical reference indicators and proves to be a valid method for the adequate therapy of patients suffered from pain in most situations (Brown et al., 2011), it would fail to be applied in certain vulnerable populations. Individuals with communicative impairments or disturbance of consciousness, including older adults with dementia and intensive care unit patients in vegetative state, coma, and minimally conscious state (Herr et al., 2006; Schnakers and Zasler, 2007), may not be able to provide effective and credible self-reports of pain (Li et al., 2008). For those populations, lack or any inaccuracy indicators used to evaluate pain may lead to suboptimal or inappropriate treatment of pain, which may bring about various additional clinical issues, such as the deterioration of chronic pain and psychological distress or depression (Roulin and Ramelet, 2012). Furthermore, the self-reported pain is very subjective and unable to be obtained in real-time.

Recent developments in objective pain assessment have concentrated mainly on recognition and prediction from human behaviors, such as vocalizations (Puntillo et al., 2004), body motions (Young et al., 2006), and facial expressions (Lucey et al., 2011; Kaltwang et al., 2012; Irani et al., 2015). While behavioral methods exist, they also may be inapplicable in individuals with paralysis or other motor disorders affecting behaviors. By observing the face of an individual, a huge number of features related with affective state can be extracted, including pain state. However, facial expression-based pain recognition need track the special facial regions of the users, which can be very cumbersome and tedious in the clinical application. Meanwhile, researchers' effort has been shifted to target toward a physiology-driven pain prediction that does not rely on individual's facial or volitional behaviors (Shankar et al., 2009; Treister et al., 2012). Those studies have focused on diverse bio-physiological signals, such as heart rate variability (De Jonckheere et al., 2010, 2012; Faye et al., 2010; Logier et al., 2010), skin conductance or electrodermal activity (Harrison et al., 2006; Treister et al., 2012), electromyography (Oliveira et al., 2012), electroencephalography (Nir et al., 2010; Huang et al., 2013), and functional magnetic resonance imaging (fMRI) (Marquand et al., 2010; Brown et al., 2011). Recently, pain assessment method implemented by multimodality signals has been confirmed to be highly effective, some even outperforming single-signal mode markedly (Werner et al., 2014; Kächele et al., 2015). However, the most physiology-driven pain researches just provide statistically significant correlations between several bio-physiologic signals and the presence of pain. They are qualitative assessments for discriminating the presence or absence of pain. Few measurements can predict the intensity of pain as a level indicator. Hence, despite many remarkable researches, there is no acceptable pain assessment method based on physiological signals of human.

With the increasing availability of wearable smart devices, many researchers focus on developing a non-invasive integrated system for health monitoring. An appropriate interpretation of the signals recorded by wearable devices equipped with sensors can help monitor, evaluate and eventually solve health problems related pain. In this article, the focused goal is the quantitative measurement of pain intensity from multi-physiological signals obtained by wearable sensors in the customer market. At present, only few studies have been carried out the procedure of automatic recognition of pain intensity from physiological signals. Olugbade et al. applied electromyography (EMG) and body motions in combination with Support Vector Machines (SVM) and Random Forests (RF) as classifiers to recognize three pain intensity (Olugbade et al., 2015). Kachele et al. used EMG, skin conductance level (SCL) and electrocardiogram (ECG) incorporated with unsupervised and semi-supervised learning to establish a personalized system of continuous pain intensity recognition (Kachele et al., 2016). However, these assessment procedures are quite complicated and timeconsuming, especially feature extraction. Moreover, the latter focuses on a personalization scenario where the recognition process may not be widely used in the general population.

In this paper, we proposed a convenient and objective method of pain intensity recognition based on multiple physiological signals. The technique uses a hybrid of genetic algorithm (GA) with principal component analysis (PCA) to obtain optimized feature combination related with pain states. And three kinds of classification methods are compared to establish more appropriate recognition models. The presented system that complements the self-reported pain can thus be adapted to general persons.

The remainder of this paper is structured as follows. In Section Dataset and Experimental Protocol, we provide a short presentation of physiological signals used in pain recognition and experiment of pain induced by electrical stimulation. Section Feature Processing elaborates a thorough feature processing phase, including feature extraction, feature selection, and feature reduction. In Section Pattern Classification, we present the three classifiers for pain intensity recognition system. Thorough experimental results and discussion are described in Section Experimental Results and Discussion. Section Conclusions and Outlook provides our findings and ideas for future study in this field.

## DATASET AND EXPERIMENTAL PROTOCOL

#### Subjects

Six subjects (4 males and 2 females, numbered 1–6) aged 22–25 years (standard deviation [SD] = 3.0) were recruited in this study. All participants were healthy, without any history of medical illness, neurological or psychopathological disorders, and none had a history of chronic pain. All the experimental protocols of this study were approved by the ethics committee of China Medical University. Moreover, all subjects were fully informed about the procedures, risk, and benefits of the study, and written informed consent was obtained from all subjects before the study.

The algorithm design and performance validation in this pain intensity measurement were implemented for the databases constructed from physiological signals of each subject. The signals were obtained with the procedure described in Section Physiological Signals and Preprocessing and Pain Induction Protocol. Moreover, we continuously collected 1-week data from the same participant, thus generating a 7-day dataset. For the database of each subject, we randomly chosen 75% of the samples as the training set, and the rest as the testing set. For the 7 day database, data from six randomly chosen days were used as the training set, and the rest were used as the testing set (leave-one-out method).

#### Physiological Signals and Preprocessing

Acquisition of high-quality physiological data is of utmost importance for the pain intensity recognition system. The selection of available physiological signals that are used as input is the first consideration in the pain detection system. It is expected that the physiological signals reflect the effect of pain on the activity level of the nervous system. However, unlike selfreporting or facial recognition of pain, where the truth class labels of pain intensity for a given sample are self-evident, a highconfidence physiological signal in an underlying pain intensity is not easy to be acquired. Furthermore, it is difficult to determine whether the variability of the physiological signals is specific to pain or whether they represent a general response to factors such as thought, cognition, emotion and environment.

Since we were decided to exploit a practical quantitative and convenient system, there was a limitation on the selection of available signals. Though facial electromyograms and electroencephalograms would be expected to be useful, the attachment of electrodes to the face and scalp is complicated and not suitable for practical application. In our study, the selected physiological signals were blood volume pulse (BVP), electrocardiogram (ECG), and skin conductance level (SCL). These signals reflect the activity level of the autonomic nervous system, which is connected with the secretory activity of cardiac muscles and internal organs. As a wearable device, we use the Infiniti 3000A platform developed by Thought Technology Ltd., Quebec, Canada. The Infiniti 3000A apparatus is a compact sensing platform with 10 isolated channels for recording signals. It is small enough to attach to a portable computer and integrates to commercial sensors via custom cabling. The sampling frequency was fixed at 256 Hz for all the three channels.

The BVP signal is derived from a photoplethysmographic (PPG) sensor that monitors blood volume in capillaries and arteries by emitting an infrared light through the tissues. Vasomotor activity, which controls blood vessel diameter, is regulated by the sympathetic nervous system (Babchenko et al., 2001). Hence, changes in BVP amplitude reflect instantaneous sympathetic activation. Most PPG sensors can be placed anywhere on the body, with the finger as the most common location for recording a BVP signal. In this study, A BVP-Flex/Pro sensor was placed against the palmar surface of the middle finger of the right hand with an elastic strap or a short strip of adhesive tape to acquire BVP signals (**Figure 1A**). In the preprocessing phase, a 4-order Butterworth bandpass filter with gain 3, cutoff frequencies [30, 200] Hz was applied to eliminate the bursts in the BVP signal.

The ECG, an electro-physiological signal that is associated with the electrical activity of the sinuatrial node, reflects the cardiovascular activity. Additionally, ECG responses to external stimuli (such as pain stimuli and stress) can produce large variability in a given subject's physiological signal (Sriram et al., 2009). Therefore, we can employ ECG signal to extract universal information about pain state or intensity. The ECG signal was measured from both surface of upper arms with the twoelectrode method based on lead I in our experiment (**Figure 1A**). For the preprocessing of the ECG trace collected by using noninvasive surface electrodes, a typical high-pass and movingaverage filter (Milanesi et al., 2008; Sidek et al., 2014) were used to remove a low frequency baseline drift due to electrode contact noise, respiratory effects, and motion artifacts. Otherwise, we employed an adaptive linear interpolation algorithm to perform baseline correction before feature extraction.

The SCL is another signal that can easily be acquired from the body surface. It reveals variance in the electrical properties of the skin due to the secretory activity of the perspiratory glands. As it is directly regulated by the sympathetic nervous system, it is a good indicator of psychological or physiological arousal level due to external stimuli. Changes in the SCL signal induced by a painful stimulus were validated to be used as a tool to monitor nociceptive stimulation and pain (Harrison et al., 2006; Storm, 2008). The SCL signal was traced from two Ag/AgCl electrodes attached to the tip of the index and ring fingers on the palm-side of the right hand in this study (**Figure 1A**). In the preprocessing phase, a moving average smoothing was carried out to subtract artifacts. Furthermore, the raw SCL data was down-sampled by a factor of 1/2 to reduce the computational complexity in feature extraction phase.

## Pain Induction Protocol

As in many pain researches, there are variety ways that can be used to induce pain, such as thermal or cold pain stimulation (Appelhans and Luecken, 2008; Kachele et al., 2016), mechanical pain elicitation (Matsunaga et al., 2005; Shankar et al., 2009), and electrical shock (Oliveira et al., 2012; Zhang et al., 2016). In our study, pain was induced by an electrical stimulator (MotionStim8; Medel GmbH, Hamburg, Germany), which can generate a current square wave with a certain pulse width. The amplitude and frequency of the current are adjustable. The value of the pulse amplitude can be used as an objective index of pain intensity. In the experiment, the frequency of MotionStim8 was set to 2 Hz. In order to avoid interference of the stimulator with the sensors, especially the ECG sensor, the stimulation electrodes were placed on the tibialis anterior muscle of the right leg, as far away as possible from the sensors (**Figure 1A**).

Before stimulation phase, the current intensity was calibrated according to the subjects' self-reports. An electrical stimulus with 20 mA induced the sensation of pain was as pain starts (threshold). An electrical stimulus with 40 mA induced intense pain was as barely bearable pain (tolerance). Then, we divided the range between threshold and tolerance into 3 equally spaced intervals, 20-mA stimulus (stim20, L1), 30-mA stimulus (stim30, L2), and 40-mA stimulus (stim40, L3) with additional prestimulus (baseline, L0). During the stimulation experiment, each of the different current intensity was proceed for 1 min followed by a recovery period of 1.5–2 min (**Figure 1B**). To eliminate stimulation adaptability and time correlations, the sequence of stimuli and duration of the recovery period were randomly designed. Pain elicitation was executed 30 sessions for each of the 3 calibrated intensities (L1–L3) in the same day. With the addition of baseline phase, a total number of 120 stimulation trials were obtained for each subject. Each session took approximately 15 min. To minimize motion artifacts, the participants were requested to be as relaxed as possible during the stimulation phase. Moreover, the recording scenario that led to the multi-day dataset was continuously carried out 7 days for one person in the same condition. **Figure 2** shows physiological traces from a participant in the four pain states.

## FEATURE PROCESSING

The overflow of the proposed pain intensity estimation system can be seen in **Figure 3**. The pipeline consists of raw signal input, independent preprocessing according to Section Physiological Signals and Preprocessing, feature processing, and finally pain intensity recognition. In this section, the proposed feature processing phase is introduced, including feature extraction, Section Feature Selection and Feature Reduction.

#### Feature Extraction

From the **Figure 2**, we can see that the amplitude of physiological signals obviously varied according to the level of electrical stimulus. Based on this observation, we gathered a variety of basic statistical features to evaluate the response to the pain-inducing stimuli, such as maximum, minimum, median values of data. Furthermore, these features can easily be calculated in an online way, which makes them more suitable for real-time recognition system of pain intensity.

The statistical features can be calculated for each of the signal channels as follows. Let the three preprocessed signals the digitized BVP, ECG and SCL waveforms from any one of the four pain states segments—be designated by X i , i = 1, 2, 3 respectively. Let X i n represent the value of the n th sample of the i th signal, where n = 1, 2, ..., N with N in range of 3,840 to 7,680. Let X˜ i n refer to the normalized signal (zero mean, unit variance), formulated as:

$$\tilde{X}\_n^i = \frac{X\_n^i - \mu^i}{\sigma^i} \qquad i = 1, 2, 3 \tag{1}$$

where µ i and σ i are the means and standard deviations of X i as presented below. Let f i j represent the j th features of the i th signal. • The means of the signals: f i 1 :

$$
\mu^i = \frac{1}{N} \sum\_{n=1}^N X\_n^i \qquad i = 1, 2, 3 \tag{2}
$$

• The standard deviations of the signals: f i 2 :

$$\sigma^i = \left(\frac{1}{N-1} \sum\_{n=1}^{N} \left(X\_n^i - \mu^i\right)^2\right)^{1/2} i = 1, 2, 3 \tag{3}$$

• The means of the absolute values of the first differences of the signals: f i 3 :

$$\theta\_1^i = \frac{1}{N-1} \sum\_{n=1}^{N-1} \left| X\_{n+1}^i - X\_n^i \right| \qquad i = 1, 2, 3 \tag{4}$$

• The means of the absolute values of the first differences of the normalized signals: f i 4 :

$$\tilde{\theta}\_1^i = \frac{1}{N-1} \sum\_{n=1}^{N-1} \left| \tilde{X}\_{n+1}^i - \tilde{X}\_n^i \right| = \frac{\theta\_1^i}{\sigma^i} \qquad i = 1, 2, 3 \tag{5}$$

electrocardiogram (ECG; microVoltage), and skin conductance (SCL; microSiemens). Each graph shows 10 s of response. The segments shown here are visibly different for the four states.

• The means of the absolute values of the second differences of the signals: f i 5 :

$$\theta\_2^i = \frac{1}{N-2} \sum\_{n=1}^{N-2} \left| X\_{n+2}^i - X\_n^i \right| \qquad i = 1, 2, 3 \tag{6}$$

• The means of the absolute values of the second differences of the normalized signals: f i 6 :

$$\tilde{\theta}\_2^i = \frac{1}{N-2} \sum\_{n=1}^{N-2} |\tilde{X}\_{n+2}^i - \tilde{X}\_n^i| = \frac{\theta\_2^i}{\sigma^i} \qquad i = 1, 2, 3 \tag{7}$$

• The minimum: f i 7 :

$$Min = minimum(\mathbf{X}\_n^i) \qquad i = 1, 2, 3 \tag{8}$$

• The maximum: f i 8 :

$$Max = \operatorname{maximum}(\mathbf{X}\_n^i) \qquad i = 1, 2, 3 \tag{9}$$

• The minimum ratio: f i 9 :

$$
\begin{array}{c}
\min \text{Ratio} = \frac{\text{Min}}{N}
\end{array}
\tag{10}
$$

Here, N represents the length of the signal. • The maximum ratio: f i 10:

$$
gamma = \frac{\text{Max}}{N} \tag{11}
$$

Here, N represents the length of the signal.

• The range of the signal: f i 11:

$$range = Max - Min\tag{12}$$

Besides, we also calculated the median value of the signal as f i <sup>12</sup>. Hence, each channel was characterized by 12 features. Considering the time lag of pain reaction, we selected 30 s signal for feature analysis in the 1-min data. For obtaining the subtle changes of the signals in a trial, we segmented the 30 s signal by a sliding window with 3 s sliding step (the window length was 3 s) to obtain the feature windows. Hence, a sample dataset of 40 × 36 can be obtained for each experiment session consisted of 4 trials. For single subject with 30 sessions, the size of sample dataset was 1,200 with 36 feature dimensions.

The BVP, ECG, and SCL are dependent on each participant's initial physiological level. Even when these signals are measured from the same person, they are likely day-dependent due to

variations in mental state affected by emotion, variations in physiology caused by sleep or diet, or variations in the sensor's connectivity with skin (Sun et al., 2010). To eliminate the intrasubject and day-to-day variation factor, a normalization stage for each feature was applied in feature vector set F. Refer to (13), the first was to subtract the minimum value from each feature. Then, the features are divided by the range to make all the values lie between 0 and 1.

$$F(m)\_{norm} = \frac{F(m) - V\_{min}(m)}{V\_{max}(m) - V\_{min}(m)} \tag{13}$$

where, Vmin(m) = min{Fn(m)}, Vmax(m) = max{Fn(m)}, ∀n ∈ |F|.

#### Feature Selection and Reduction

Note that not all of these features are independent. Some features are a linear combination of other features (e.g., θ˜ i 1 and θ˜ i 2 ) or a non-linear property. To obtain high-quality features with significant discriminatory information, a hybrid combination of genetic algorithm (GA) and principal component analysis (PCA) was proposed. This combination was motivated by the roles of the two algorithms: GA selects from a feature set to remove useless and redundant features, while PCA linearly transforms a feature set. Since feature selection is non-linear, the hybrid of these two ways could provide a more powerful separation than either alone.

The GA is a biologically inspired stochastic algorithm based on some optimization criterion that mimics natural evolution. The GA is naturally applicable to feature selection since the problem has an exponential search space (Oh et al., 2004). Furthermore, the distinct specialty of this algorithm is that it always maintains a set of solutions (called chromosomes or individuals) in a population. To simulate the biological evolution, the GA selects fitter chromosomes at each generation by some genetic operations, such as crossover and mutation. The traditional procedures of steady-state GA are described below.


For the feature selection problem, a finite bit string with d binary digits is usually used to represent chromosomes. A binary digit represents a feature. A value of 1 or 0 in the chromosomes means that the corresponding feature is selected or removed, respectively. For example, a chromosome 10100010 means that the first, third and seventh features are selected. Since we wanted to improve recognition rate, the classification accuracy from the classifiers was directly used as the fitness function to evaluate a chromosome (Lin et al., 2015). Based on the initial fitness evaluation, the chromosome for the next generation was selected by a selection mechanism that ensures fitter chromosomes have a higher probability to survive. In our design, a traditional roulette wheel selection strategy was applied. We also used the standard two-point crossover operator to obtain the offspring and Gaussian mutation operations at probability on each of the offspring produced from crossover. The crossover rate and mutation rate were set to 0.8 and 0.01, respectively. Besides, the size of population and the maximum generation in GA were set to 100 and 30, respectively. These parameters are tunable to be suitable for a specific dataset to improve performance.

The principal component analysis (PCA) is a well-known statistics method of multi-variable analysis and used orthogonal transformation to convert a set of observations of possibly correlated variables into a set of values of linearly uncorrelated variables (Karamizadeh et al., 2013), also known as principal components. Moreover, PCA is a tool to reduce dimensionality by searching a linear projection matrix of the dataset to a feature space with lower dimensions while retaining most of the information. This linear projection matrix is a meaningful basis to filter out the noise and reveal hidden information in the original feature space. The key advantages of the PCA are the decreased requirements for memory and capacity, it low noise sensitivity, and increased efficiency in a lower dimension. By solving the covariance matrix, a series of eigenvalues with the corresponding eigenvectors are computed and rank-ordered descendingly by quantifying how "principal" each variance is. The k largest eigenvalues are selected and the corresponding eigenvectors (k principal components) are used to construct the linear projection matrix. Then, a feature space with N dimensions is transformed to a space of k dimensions by this reduction

<sup>1.</sup> Initialize population P;

matrix. For a detailed description of the PCA algorithm about mathematical derivation, the reader is referred to Bro and Smilde (2014).

A hybrid GA with PCA method is proposed, evaluated and implemented for comparison. As mentioned above, the GA algorithm provides a subset of features selected from raw features. Instead of feeding the PCA algorithm with all possible features, we used the selected subset of features as the input to the PCA. Note that the PCA algorithm is applied here for transforming features into a linearly uncorrelated space.

# PATTERN CLASSIFICATION

The features undergo processing were feeding to classifiers to identify a subject' pain intensity. We used three types of learning methods to train and investigate the performance of classifiers, including linear discriminant analysis (LDA), k-nearest neighbor (KNN) algorithm, and support vector machine (SVM). We evaluated classification performance for single-signal datasets and multi-signal datasets, as well as for multi-subject datasets and multi-day datasets.

The LDA is a commonly used statistical method that uses a linear discriminant function to classify a new data points. In the calculation process of the algorithm, a hyperplane is constructed by finding a linear combination of features to separate or characterize two classes of objects or events. The optimal linear combination or projection in a classical LDA is constructed by minimizing the within-class distance and maximizing the between-class distance simultaneously, thereby achieving maximum class separation (Yan et al., 2014). Since a LDA algorithm is a linear binary classifier, it is not suitable in multi-class situations. In our pain study, the classes are three pain intensity levels and the baseline state. Thus, we need a multi-task method to inference pain states. A one-vs.-rest strategy was used to address multi-class problems. For each class l, an independent LDA classifier was trained to separate data of class l and the rest class. In this paper, four trained LDA classifiers were applied in the test set. For a test sample, the recognition result was determined by selecting the class with the highest output value. Moreover, the classification accuracy of LDA was used as the fitness in the GA.

The KNN is a simple multi-class technique that operates by finding the k objects in the training set which are closest to the test sample by some metrics, such as measures based on distance (Weinberger et al., 2005). In order to classify an instance of the test set into a class, KNN calculates the distance between each instance of the training set and this test sample. Then, a prediction is determined as the most common class among this set of nearest neighbors, with each neighbor's vote being assigned a weight inversely proportional to that neighbor's distance from the test sample. The performance of a KNN classifier is primarily determined by the choice of k as well as the distance metric applied (Xiao and Chaovalitwongse, 2016). By effectively using prior knowledge such as distribution of the features, Euclidean distances were applied as the distance metric in our KNN classifier. In addition, the configurable parameter k of the k-nearest neighbor algorithm was set to 3 by multiple testing.

The SVM is a supervised learning algorithm by finding a highdimensional discriminant hyperplane to separate observations into two class with a maximum margin, generally a soft margin (Vapnik, 2005). A regularization parameter was applied in the soft margin to penalize misclassification. By using the "kernel trick" to map the testing data points into a higher dimensional space, a support vector machine can construct an optimized separation hyperplane and make the mapped data easier to be separated. For pain recognition, randomization was executed using a computer-based list randomizer to construct the training data set for each subject. The preferential kernel function was the Gaussian or Radial Basis Function (RBF) kernel in this study. The SVM model was trained by cross-validation method with a grid search to find the optimal parameters of regularization and kernel. In this study, the SVM is optimal with a regularization of 5 and a Radial Basis Function kernel with width of 2.58.

In both of the KNN and SVM methods applied in pain recognition, the processed features were uniformed to zero mean and unit variance. Accordingly, the effect of feature variations in distance-based inference algorithms was reduced.

# EXPERIMENTAL RESULTS AND DISCUSSION

#### Initial Results—Single-Signal Analysis

Previous studies have performed qualitative analyses of pain based on a single physiological signal. For example, Harrison et al. revealed SCL as a promising indicator to predict the presence or absence of pain in hospitalized infants during painful and non-painful medical procedures (Harrison et al., 2006). However, in this study, we applied pattern recognition techniques to quantify pain intensity based on a set of physiological signals. Since there was not yet a priori knowledge of pain recognition, we originally established a quantitative model for pain intensity from individual's single physiological signal. For each signal, we extracted 12 statistical features to construct three types of feature set, including BVP-features, ECG-features, and SCLfeatures. The classification accuracy of the LDA algorithm was evaluated by using these three types of feature set. **Figure 4** presents the recognition results gained with LDA algorithm for each physiological signal in each of the six study subjects.

From the **Figure 4**, the experimental results provide evidence to support our hypothesis that a single signal can be used to quantify pain intensity in some degree, with the average classification accuracy of 50% above. Furthermore, since the BVP and SCL signal were highly affected by the intensity of pain stimuli in our experiment, the classification accuracies were even higher than that of ECG signal alone. Especially, the average classification accuracy of SCL signal was 68.39% (±2.32%), and that of ECG signal was 53.01% (±3.40%). Unlike ECG, it can be observed that even the original SCL signal already presents an obvious variance with the pain stimulation in **Figure 2**. As the experimental results suggest, the extracted SCL features are a good indicator to identify the pain intensity.

This is particularly suitable when it comes to integrate the proposed methods into wearable device. Since the SCL sensors are less intrusive and more convenient attached than BVP or ECG sensors, it can be easily embedded into wearable devices. Although the performance of LDA model for a single signal is considerable, the correct classification rate is relatively low. For improving the classification performance and obtaining better models, we considered applying multiple physiological signals with all-feature combination to quantify pain intensity. The idea was inspired to take full advantages of pain information contained in various signals.

#### Results of the Multi-Signal Model

Since we used multiple signals (BVP, ECG, and SCL), the total number of extracted features per sample was 36 (12 features per signal). We also selected the LDA algorithm as the classifier. The recognition accuracies of running 10 trials LDA are shown in **Figure 5**. The average accuracy of the LDA model was 42.49% (±14.31%) for subject 1, 53.78% (±20.84%) for subject 2, 65.12% (±16.57%) for subject 3, 51.07% (±20.52%) for subject 4, 54.30% (±13.62%) for subject 5, and 55.52% (±11.43%) for subject 6, respectively. The maximum classification accuracy of 10 trials LDA algorithm in the condition of multiple signals was approximately 90% (such as subjects 2 and 3). However, the variation of classification accuracy was large, ranging from 19.64% (subject 1) to 87.61% (subject 3). This finding indicated that the robustness of the multi-signal model established by LDA was insufficient. This may be because some extracted features are not independent and may be correlated. Furthermore, some features do not contribute or contribute little to the performance of the classification. A poor feature combination may greatly degrade the performance of the classifier. Hence, feature selection was needed to select a subset of good features.

In this study, we applied the genetic algorithm technique to seek and identify the potential informative features combinations, which contribute most to the performance of the classification. From other researches, it is obvious that the best accuracy of the LDA by using cross-validation method is considered as the indicator of the fitness function. **Table 1** provide the experimental results of GA selection. The GA was run 15 times; each row lists the results from the corresponding run. In **Table 1**, the column "Best generation" indicates the generation number in which the optimal set of features was selected. And the average number of selected features was roughly 18. The performance of LDA was significantly improved with average accuracy of 72% above. In detail, the feature f i 2 , f i 3 , f i 4 , f i 5 , f i 6 , and f i 7 computed from the three physiological signals were almost repeatedly selected by each running of the GA, yielding 18 features: µ i , σ i , θ i 1 , θ˜ i 1 , θ i 2 , θ˜ i 2 , i ∈ (BVP, ECG, SCL). It means that these features are the more robust feature combination than those that are rarely selected in the context of this pain experiment.

Although the trend of the feature combination proves their efficacy in distinguishing pain intensity, the 18 features selected by GA are not the best informative features for classification. This is because the genetic algorithm cannot easily eliminate the linear features. To solve this problem, we applied the feature reduction technique. **Figure 6** shows samples of 18-dimension feature space projected onto the first three features from subject 1. The selected feature vectors from the same class aggregated a cluster with a large amount of variation, whereas the feature vectors from the different classes overlapped dramatically. Then, a PCA method was applied to reduce the redundant linear features. After data transformation, the feature vector samples within same class were clustered more compactly than did samples before transformation, and the feature vector samples from different classes are farther away from one another (see in **Figure 7**).

After the feature processing stage by using the hybrid GA with PCA, the samples were fed to the LDA classifier to investigate the performance. In addition, we also tested the KNN and SVM algorithms for comparison. The corresponding recognition accuracy is presented in **Figure 8**. Using the optimized feature set, the average classification accuracy of three classifiers reached 80% or higher. Particularly, the performance of LDA algorithm was better than the other two algorithms, with 98% average accuracy. Those results prove that the proposed feature processing method is effective. And, the LDA classifier is more suitable in physiology-based pain intensity recognition for a single-subject scenario.

# Results of the Multi-Subject Model

In addition to discussing the recognition model of a single subject, we also analyzed samples constructed from multiple subjects. A sample set of multiple subjects was generated based on the six participants. Similarly, GA-based feature selection and PCA-based dimension reduction technique were carried out on the sample set comprising multiple subjects. The LDA, KNN, and SVM algorithms were implemented to train the corresponding models. **Figure 9** shows the pain intensity recognition accuracy of running 10 times for the multi-subject model. The average categorize accuracy was 84.28, 83.94, and 96.47% for LDA,



KNN and SVM, respectively. Overall, these three algorithms effectively identified pain intensity. The relevant multi-subject models established by LDA, KNN and SVM are feasible and appropriate. Besides, the multi-subjects model gained from SVM algorithm was better than the other two models. Every time we used the training SVM model to test the rest samples, the predicted accuracy was 92% or higher. Therefore, the trained SVM model can be used as a general model to quantify pain intensity for a multiple-subject scenario.

Meanwhile, between-subjects' experimental validation was conducted by using the LDA, KNN, and SVM in a leave-onesubject-out cross-validation method, which is standard when dealing with multiple subjects. Here, the recognition modes were trained with dataset from randomly selected six subjects

reduction projected onto the first three features.

to predict the pain intensity of the remaining subject. It was repeated until all of the subjects had been part of the testing dataset. For a given between-subject scenario, we repeated 20 times and calculated the average accuracy for each pain intensity, respectively (see **Figure 10**). In detail, the average recognition accuracy of pain intensity with baseline, stim20, stim30, stim40 was 78.01, 90.10, 82.35, 82.90%, respectively. This is a more realistic condition in which the distribution of the training dataset and the testing dataset are dissimilar due to inter-individual variability. Moreover, the SVM algorithm was performed better than the KNN and LDA, with average accuracy of 91.18, 76.14, and 82.83%, respectively. These results demonstrate that the SVM classifier outperforms the other two classifiers for a between-subject scenario. Furthermore, the

proposed system in the context of pain prediction would mitigate the inter-personal variance while preserving precise scalability.

#### Results of the Multi-Day Model

Furthermore, we also conducted experimental evaluation by using the three types of classifiers in a multi-day scenario. For one person, we continuously carried out the pain induction experiment of 7 days and collected a seven-day dataset. A leaveone-day-out cross-validation method was applied to obtain the pain intensity of one person. The results of multi-day model are given in **Figure 11**. The average recognition rate was 71.02, 81.39, and 76.93% for KNN, SVM, and LDA, respectively. It implied that the trained SVM model seemed to be more suitable as a special model to quantify pain intensity of multiple days for one subject. In addition, the overall accuracy was roughly reduced from 87% of 1-day scenario to 70% of multi-day scenario. This may be mainly because that the signals measured from the same person are likely day-dependent. The quality of signals

is affected by the intro-individual variance in mental state and environmental factor. However, our proposed pain intensity system is relatively fitted for practical applications of continuous and long-term pain monitoring.

## CONCLUSIONS AND OUTLOOK

In this paper, we proposed a physiological signal-based method for convenient quantification of pain intensity induced by the external electrical stimulation. The proposed measurement provides a valid, physiology-based substitution of self-reported pain. The newly developed pain intensity measurement system consists of signal acquisition and preprocessing, feature extraction, GA-based feature selection and PCA-based feature reduction, and three types of pattern classifiers, including LDA, KNN, and SVM. Using the hybrid feature processing of GA with PCA, the training samples can be optimized to obtain a higher recognition accuracy while simultaneously reducing computational complexity as unrelated features are removed, such as maximum, minimum, median, et al.

In various experimental validations, our method was tested and the performance of three classifiers were evaluated, including single-signal analysis, multi-signal model, multi-subject scenario, and multi-day scenario. For a LDA model trained by a single signal, the SCL signal seems to provide a better result than that of the BVP or ECG signal. Moreover, after a series of feature process, the performance of LDA classifier was significantly improved for multi-signal model. And, the LDA classifier is more suitable in physiology-based pain intensity recognition for a single-subject scenario compared to the KNN and SVM. For the samples constructed from multiple subjects, the KNN, LDA, and SVM could categorize pain intensity with 83.94, 84.28, and 96.47% accuracy, respectively. We also compared the performance of the three classifiers for a between-subject scenario. It seems that the performance of SVM algorithm was slightly better. Therefore, the trained SVM model can be used as a general model to quantify pain intensity for other unseen population. Furthermore, the three classifiers could also reach high accuracy for a multi-day scenario. These findings suggest that the proposed pain intensity measurement method can be suitable for practical applications of continuous pain monitoring. From **Figures 10**, **11**, we can see that the recognition accuracy of different pain intensities was different. Especially, the recognition rate at baseline was relatively low. This could be caused by the large amount of variation of physiological signals for various subjects and various days at baseline. There are numerous causes that may influence the physiological baseline signals, such as cognition, physical movement, and even physical or psychological status. These factors may also affect the model selection and even deteriorate the recognition ratio. In summary, our system was able to predict well above chance level between three levels of pain intensity with respect to previous researches (Olugbade et al., 2015; Kachele et al., 2016).

For our future work, an online recognition experiment of pain intensity based on multi-physiological signals will be conducted. Then, a system of continuously pain monitor will be developed. Eventually, we expect that the sensors of our proposed system would be executed as a wearable device that is suitable for daily use in clinical settings. In the case of online recognition, the input signal cannot be preprocessed and standardized in advance. A large amount of signal from previous offline experiments in various population must be collected before using to deal with the online prediction of unseen samples. Besides, the latency of the online system must be considered. The final prediction system is to provide a trade-off between lower latency and higher accuracy. Since the dataset generated in the current study was obtained by artificial electrical stimulation to induce different intensities of pain, this procedure is not appropriate for an online prediction system. Hence, more ways should be investigated in order to predict and quantify pain intensity perceived by a person. We furthermore plan to research various methods of feature selection to access to a comparison result.

### AUTHOR CONTRIBUTIONS

YC, XZ, JH, and YS conceived the conception and design of this research. YC executed the experiments, including acquisition and analysis of data for the work. YC, XZ, JH, and YS interpreted the experimental data. YC drafted the manuscript. XZ, JH, and YS revised the manuscript.

#### FUNDING

This research was partially supported by the National Nature Science Foundation of China (Grants nos. 61273355, 61573340),

#### REFERENCES


and in part by the National High Technology Research and Development Program of China (863 Program) under Grant 2015AA042301 and Youth Innovation Promotion Association CAS.

## ACKNOWLEDGMENTS

The authors gratefully acknowledge the support by State Key Laboratory of Robotics and China Medical University for providing us with the acquisition devices. Also, we appreciate the assistance of Lingguo Kong in setting up the experimental system and Lipeng An in data analysis.

from single-trial laser-evoked potentials. Neuroimage 81, 283–293. doi: 10.1016/j.neuroimage.2013.05.017


and peak frequency of alpha oscillations during stimulation and at rest. Brain Res. 1344, 77–86. doi: 10.1016/j.brainres.2010.05.004


International Conference on Mobile Computing, Application, and Services (Santa Clara).


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2017 Chu, Zhao, Han and Su. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Phase-II Clinical Validation of a Powered Exoskeleton for the Treatment of Elbow Spasticity

Simona Crea<sup>1</sup> \*, Marco Cempini 1, 2, 3, Stefano Mazzoleni 1, 4, Maria Chiara Carrozza<sup>1</sup> , Federico Posteraro4, 5 † and Nicola Vitiello1, 6 †

*<sup>1</sup> The BioRobotics Institute, Scuola Superiore Sant'Anna, Pisa, Italy, <sup>2</sup> Shirley Ryan AbilityLab, Chicago, IL, USA, <sup>3</sup> Feinberg School of Medicine, Northwestern University, Chicago, IL, USA, <sup>4</sup> Rehabilitation Bioengineering Laboratory, Volterra, Italy, <sup>5</sup> Rehabilitation Department, Versilia Hospital, Azienda USL Toscana Nord Ovest, Viareggio, Italy, <sup>6</sup> Fondazione Don Carlo Gnocchi, Firenze, Italy*

Introduction: Spasticity is a typical motor disorder in patients affected by stroke. Typically post-stroke rehabilitation consists of repetition of mobilization exercises on impaired limbs, aimed to reduce muscle hypertonia and mitigate spastic reflexes. It is currently strongly debated if the treatment's effectiveness improves with the timeliness of its adoption; in particular, starting intensive rehabilitation as close as possible to the stroke event may counteract the growth and postpone the onset of spasticity. In this paper we present a phase-II clinical validation of a robotic exoskeleton in treating subacute post-stroke patients.

#### Edited by:

*Dingguo Zhang, Shanghai Jiao Tong University, China*

#### Reviewed by:

*Ranu Jung, Florida International University, USA Rong Song, Sun Yat-sen University, China*

> \*Correspondence: *Simona Crea simona.crea@santannapisa.it*

*† These authors have contributed equally to this work.*

#### Specialty section:

*This article was submitted to Neural Technology, a section of the journal Frontiers in Neuroscience*

Received: *24 September 2016* Accepted: *21 April 2017* Published: *12 May 2017*

#### Citation:

*Crea S, Cempini M, Mazzoleni S, Carrozza MC, Posteraro F and Vitiello N (2017) Phase-II Clinical Validation of a Powered Exoskeleton for the Treatment of Elbow Spasticity. Front. Neurosci. 11:261. doi: 10.3389/fnins.2017.00261* Methods: Seventeen post-stroke patients participated in 10 daily rehabilitation sessions using the NEUROExos Elbow Module exoskeleton, each one lasting 45 min: the exercises consisted of *isokinetic passive mobilization* of the elbow, *with torque threshold* to detect excessive user's resistance to the movement. We investigated the *safety* by reporting possible adverse events, such as mechanical, electrical or software failures of the device or injuries or pain experienced by the patient. As regards the *efficacy,* the Modified Ashworth Scale, was identified as primary outcome measure and the NEEM metrics describing elbow joint resistance to passive extension (i.e., maximum extension torque and zero-torque angle) as secondary outcomes.

Results: During the entire duration of the treatments no failures or adverse events for the patients were reported. No statistically significant differences were found in the Modified Ashworth Scale scores, between pre-treatment and post-treatment and between post-treatment and follow-up sessions, indicating the absence of spasticity increase throughout (14 days) and after (3–4 months follow-up) the treatment. Exoskeleton metrics confirmed the absence of significant difference in between pre- and post-treatment data, whereas intra-session data highlighted significant differences in the secondary outcomes, toward a decrease of the subject's joint resistance.

Conclusions: The results show that our robotic exoskeleton can be safely used for prolonged sessions in post-stroke and suggest that intensive early rehabilitation

**194**

treatment may prevent the occurrence of spasticity at a later stage. Moreover, the NEEM metrics were found to be reliable compared to the Modified Ashworth Scale and sensitive to revealing intra-session changes of elbow resistance to passive extension, in agreement with clinical evidences.

Keywords: robotics, rehabilitation, stroke, upper limb, exoskeleton, spasticity

# INTRODUCTION

Stroke has been recognized as one of the major causes of long-term movement disabilities in the elderly, and in the last years its contingency is continuously increasing (Feigin et al., 2003). A huge number of stroke-related disabilities dramatically influence the lives of stroke survivors, of which spasticity is a significant component (O'Brien et al., 1996). Clinically, spasticity is defined as the occurrence of increased involuntary resistance to passive movements. However, this phenomenon reflects a wider spectrum of clinical problems, including abnormal limb posture and hyperactive cutaneous and tendon reflexes. After stroke, the initial usual occurrence of flaccidity and hypotonia is followed by spasticity in about 20% of patients, after a time interval that is highly variable and may occur in the short-, medium-, or longterm post-stroke period depending on a large number of factors (Ward, 2012). Clinical studies have emphasized that preventing spasticity and treating emerging spasticity in a timely manner is essential to improve the neurologic and articular pattern of the impaired limb (Ottenbacher and Jannell, 1993).

Typical rehabilitation therapy for stroke survivors helps them relearn skills that are lost due to brain damage; in particular, patients are prompted to engage in passive or active rangeof-motion exercises to strengthen their impaired limbs, as an integral part of daily management (Langhorne et al., 2011). Indeed, while they tend to avoid using paretic limbs, it has been widely demonstrated that their repetitive mobilization (either passive or active) can reduce muscle hypertonia, thus lowering joint spasticity and encouraging brain plasticity (Lum P. S. et al., 2002; Colombo et al., 2005; Masiero et al., 2007). Among the different rehabilitation exercises (e.g., passive, active-assisted, active as needed, active-resisted), the passive movement of the patient's joints has been demonstrated to maintain the range of motion (ROM) at the joints and flexibility in the muscles and connective tissue, as well as reduce muscle hypertonia and resistance to passive movement (Schmit et al., 2000; Lum P. et al., 2002; Nuyens et al., 2002). One of the key components of effective post-stroke rehabilitation is also the early mobilization of the paretic limb (Langhorne and Pollock, 2002).

When treating muscle contractures, the mobilization of contracted articulations involves close physical contact between the therapist and the patient: the therapist needs to regulate the strength and speed of the motion of the patient's articulations, in order to correct any compensatory strategies and comply with sudden spasticity. In this scenario, using robotic platforms that are capable of automating the repetitive motions usually performed by the physical therapist (Kwakkel et al., 2007) and of complying with sudden spasticity may facilitate high-intensity rehabilitation.

Many devices have been specifically developed for the neurorehabilitation of stroke patients, allowing patients to perform specific movements of the limbs. Extensive reviews of robotic devices for upper-limb neuro-rehabilitation have been carried out (Pizzi et al., 2005; Turchetti et al., 2014): many upperlimb robotic devices have been applied in stroke rehabilitation and their effectiveness has been evaluated at the start and end of treatment by means of widely used clinical assessment scales (Volpe et al., 2000; Masiero et al., 2007; Mazzoleni et al., 2013a,b) and sometimes even by measurements performed using robotic systems (Colombo et al., 2005; Mazzoleni et al., 2011).

Within the above framework, at The BioRobotics Institute of Scuola Superiore Sant'Anna we recently developed the NEUROExos, a powered elbow exoskeleton, designed to perform both passive and active elbow joint mobilization in post-stroke patients (Vitiello et al., 2013). A modified version of the device, the NEUROExos Elbow Module (NEEM) holds certification as a class-II medical device (compliant with EU regulations for medical devices, i.e., IEC EN 60601-1:2007 and EN ISO 14971:2012) and is approved for in-clinic use (Vitiello et al., 2016).

In this paper, a group of 17 post-stroke patients in their sub-acute phase were treated with the NEEM system on a daily basis, in addition to traditional physical therapy. The outcome was assessed in terms of system safety and efficacy. Safety was investigated by reporting possible adverse events, such as mechanical, electrical or software failures of the device or injuries or pain experienced by the patient. To test the treatment efficacy, the Modified Ashworth Scale (MAS, Ashworth, 1964; Bohannon and Smith, 1987; Pandyan et al., 1999) was identified as primary outcome measure and the NEEM metrics describing elbow joint resistance to passive extension (i.e., maximum extension torque and zero-torque angle) as secondary outcomes. We expected that intense therapy performed in the sub-acute phase following the stroke would be effective in preventing elbow spasticity from occurring at a later stage (i.e., 3–4 months after the stroke). Similar to other phase-II studies in the field of poststroke rehabilitation (van Heugten et al., 1998), the present study included only one group of patients. A non-controlled group of study was essential to understand whether the proposed therapy could provide positive results, and was intended to be preparatory for a phase-III randomized control trial, where the efficacy of the proposed therapy will be compared with other conventional therapies. As a second objective, similarly to other studies with robot-assisted therapy (Colombo et al., 2005), the results from the robot metrics were validated against the outcomes from the MAS and clinical evidence from the state-of-the-art.

# METHODS The NEEM System

While a detailed description of the mechatronic modules of the NEEM platform, along with its performance, is given in Vitiello et al. (2016), for the sake of clarity we will summarize its main features below. The NEEM is a powered elbow exoskeleton designed to ensure maximum comfort and safety to the patient (**Figure 1**). It embodies many key design solutions for enhanced ergonomics. Firstly, a compact and light-weight mechanical structure, with double-shelled links and a wide physical humanrobot interaction area, optimizes interaction comfort. Secondly, a four-degree-of-freedom passive mechanism relieves the elbow articulation from loads other than the actuated flexion/extension torque, and at the same time ensures appropriate alignment between human and robot joint axes. Thirdly, a series elastic actuation (SEA) unit (Giovacchini et al., 2014) allows compliant human-robot interaction as well as reliable measurement of the powered torque. The actuation unit is housed remotely in a box on the exoskeleton stand, in order to minimize the weight of the moving part. The actuation unit employs a 400 W DC brushless motor (EC motor EC60, Maxon Motor <sup>R</sup> , Sachseln, Switzerland), a Harmonic Drive reduction stage (transmission ratio 1:80), a torque limiter acting as safety clutch on the gear output, which disengages the actuation unit from the transmission means in case of torque values higher than a pre-defined threshold (set to 35 Nm in this study), and a grooved pulley. Torque is transmitted from this pulley to the exoskeleton actuated joint through a tendon-sheath system: two antagonist tendons, routed by flexible Bowden hoses, transmit rotation from the actuation unit pulley to the exoskeleton joint. A torsion spring is deployed between the actuated pulley and the exoskeleton. This element provides series elasticity, ensuring compliant interaction and accurate force transduction. Two 32-bit absolute encoders (RESOLUTETM, RESA30USA052B ring plus RA32BAA052B30F readhead, Renishaw <sup>R</sup> , Wotton-under-Edge, UK) sensorize the spring, one for each extremity. The encoder more proximal to the human joint (i.e., the joint position encoder) provides the elbow joint angular value, namely the measured joint angle, while the difference between the two encoder readings gives torsion spring deformation and, indirectly, the value of the transmitted torque. The wearable and moving part of the system (the powered orthosis) hangs from an adjustable spherical gimbal, which is mounted on an extensible horizontal cantilever bar supported by the main vertical stand. It allows manual adjustments to best fit the orthosis to the patient's body. Finally, the entire system was assembled onto a wheeled platform, in order to allow easy transportation within the clinical setting.

The NEEM control system is based on a two-layered hierarchical architecture. The NEEM low-level control implements two basic working modalities: (a) position control and (b) torque control. Under the position control mode, the robot joint is displaced along pre-defined angular trajectories (i.e., the desired angle) thanks to the action of a closed-loop proportional-integrative (PI) regulator acting on the error between the desired and measured joint angle. The output of the PI regulator is a desired current which is supplied to the DC motor by means of a commercial servo amplifier (EPOS2 70/10, Maxon Motor <sup>R</sup> , Sachseln, Switzerland). Under the torque control mode, a PI closed-loop compensator is used to control torsional spring deformation and so apply a desired torque to the user joint. Under this control mode, when the reference torque is null, the exoskeleton joint becomes "transparent" and the user can freely move.

The high-level control layer has the form of a finite-state machine, which implements a warm-up routine and a set of pre-defined rehabilitation exercises (i.e., passive, active-assisted, active-resisted, similar to those described in Prange et al., 2006). Once the system is powered on, the finite-state machine enters the so-called "Initialization" state: the exoskeleton holds the current angular position, under the position control mode action. At this point, the physical therapist (namely the experimenter) can switch the machine to the so-called "Wear" state: the system smoothly switches to the torque control mode with null torque as a desired reference, i.e., the exoskeleton joint is free to move. This way the therapist can help the patient wear the exoskeleton. Once

the exoskeleton is fastened to the patient's limb, the therapist can select and start any of the desired rehabilitation exercises (**Figure 1**).

## Rehabilitation Exercise

The rehabilitation exercise that was selected for this work was the so-called isokinetic passive mobilization with torque threshold. The exoskeleton was programmed to displace –under the position control mode– the elbow joint of the patient along a predetermined flexion/extension trajectory (henceforth named "cycle"). In particular, the exoskeleton moved the elbow joint from an initial flexed configuration (i.e., θflx = 120◦ , θ being the patient's elbow angle measured by the exoskeleto) to a full-extended configuration (i.e., θext = 10◦ ) at a constant velocity. For safety purposes, threshold values for the maximum allowed torque, in flexion, τ th flx, and extension, <sup>τ</sup> th ext, applied by the exoskeleton were set for each subject: when the transmitted torque exceeded one of these thresholds, as in the case of any abnormal/spastic muscle activity, the system switched to the torque-control mode with the desired torque set to zero. This way, the robot could smoothly comply with the user's spontaneous movement and prevent spastic muscle strength from increasing, with the consequent worsening of stretch reflexes and hypertonia. Each flexion-extension cycle can be divided into four phases (**Figure 2**), namely: (i) "elbow extension," the exoskeleton moved at a speed of 20◦ /s from θflx to θext; (ii) "rest," the exoskeleton held the fully extended position for 2 s; (iii) "elbow flexion," the exoskeleton moved at the same speed from θext to θflx; and (iv) "rest," the exoskeleton kept the fully flexed position for 2 s. Exoskeleton speed was kept at 20◦ /s to reduce the probability to evoke stretch reflexes during execution of the movement (Levin et al., 2000). Speed, as well as the values of θext and θflx were constant across the patients. The threshold values for the applied torque were set as follows. For the flexion (positive) torque, the threshold was equal for all patients and set to τ th flx <sup>=</sup> 7 N·m. For the extension (negative) torque, the threshold changed among subjects in the range τ th ext = −5 ± 1 N·m.

# Experimental Protocol

The experimental protocol was approved by the local ethical committee (Azienda Ospedaliera Universitaria of Pisa, reference number 3919, year 2014) and carried out in accordance with the principles stated by the Declaration of Helsinki. The trial was also registered on ClinicalTrials.gov (Identifier is NCT02934646). All participants signed a written informed consent before starting the experiment. A physical therapist assessed the participants' elbow ROM, Chedoke-McMaster Stroke Assessment Measure (CMSA) (Gowland et al., 1993), and MAS before the beginning of the treatment. After the first assessment was completed, the subjects were seated next to the exoskeleton, on either an armless chair or their own wheelchair, as depicted in **Figure 1**. When using the wheelchair, the right armrest was removed for the entire duration of the exercise. A physical therapist helped the participant place his/her forearm on the lower link of the exoskeleton, and his/her upper arm on the upper link. The exoskeleton vertical and horizontal position and rotation on the main stand were adjusted to best adapt the patient's arm to the powered orthosis: the upper and lower back shells of the exoskeleton were adjusted so that they adhered completely to the limbs and so that the height of the patient's right shoulder was symmetrical to the left side. After adjusting the exoskeleton, the device was fastened to the user's arm by means of two belts. Subjects participated in 10 rehabilitation sessions, on consecutive working days, for 45 min, i.e., 90 flexion/extension cycles per session. When the subjects showed any discomfort in maintaining the seated position for 45 min, the duration of the treatment was set to 15 min. Subjects were allowed to skip up to two consecutive sessions. Subjects who skipped three consecutive sessions were excluded from the protocol. It is worth noting that after the first session, the subjectspecific settings of the exoskeleton were not changed for the entire duration of treatment. After completing the last session, the subjects were evaluated once more by a physical therapist who quantified the post-treatment elbow ROM values and MAS scores. After 3 months from the stroke, the subjects were asked to participate in a follow-up screening to assess MAS and ROM.

The first assessment of the subject's ROM was used to set the limits on θext and θflx and the torque threshold τ th ext so that the applied torque allowed to fully extend the patient's elbow without causing pain.

#### Participants

Subjects enrolled in the study ranged in the 15th to the 35th day after stroke, at the beginning of the therapy. Plegia on the right upper limbs due to stroke was considered an inclusion criteria, while subjects with orthopedic pathologies at the elbow, e.g., articular blocks, were excluded from the study. Seventeen poststroke subjects completed the protocol (aged 71 ± 14 years); 12 of them suffered from ischemic stroke, and the rest from hemorrhagic stroke. On average, subjects started the robotassisted treatment in their subacute phase, i.e., the 25th day after the stroke (±5 days). Two additional subjects initially started the protocol but were then excluded since they skipped more than two sessions. Two out of 17 subjects performed treatment 15 min a day instead of 45 min because they could not maintain a sitting position for prolonged sessions. Out of the 17 participants, only 13 were able to attend the follow-up assessment.

All subjects fit into the maximum ROM of the NEEM, i.e., θflx = 120◦ and θext = 10◦ and could perform the movement at the pre-defined constant speed of 20◦ /s.

## NEEM Data Collection and Analysis

We collected the following variables from the exoskeleton sensing apparatus: the applied torque τ , and the elbow desired θdes and measured θ angle. All variables were sampled and stored at 100 Hz. After completion of each session, data were downloaded from the NEEM control unit and stored for offline data analysis. We extracted the following metrics from the variables of each flexion-extension cycle, as indicators of the resistance to the elbow extension (**Figure 2**):

• Maximum Extension Torque (MET): the torque exerted by the NEEM at the fully extended position;

• Zero-Torque Angle (ZTA): the angle at which the applied torque was zero during the elbow extension phase of the cycle.

Notably, the corresponding flexion parameters were not considered in this study as all the participants had spasticity on the elbow flexors, thus the resistance to elbow extension is more informative about the status of the spasticity.

## Statistical Analysis

The overall effect of the robot-mediated treatment was quantified using two approaches.


In addition, we analyzed the NEEM metrics to assess the intrasession effect of the treatment, by comparing the initial and final cycles of each session. For each session, median values of the first and last 5 flexion/extension cycles were extracted. The median values of all the indices extracted from the initial and final cycles of each session (from Days 1 to 10) were computed. MET and ZTA data from all participants were thus grouped in four vectors, namely METInit Cycle, METFinal Cycle, ZTAInit Cycle , and ZTAFinal Cycle. A Wilcoxon signed-rank test was used for comparison of initial and final cycles values.

Statistical significance was set at p < 0.05. SigmaStat software (version 3.5) (Systat Software, San Jose, CA, USA) was used for statistical analysis.

# RESULTS

#### Safety

All participants easily wore the exoskeleton without reporting any discomfort. Subjects had different weights, arm sizes and upper-arm circumferences, so different inner-shell sizes were used to provide a comfortable wearability of the exoskeleton. Moreover, subjects had different levels of upper-limb motor impairments (CMSA scores ranging from 1 to 6). In addition, some of them could walk easily and sit on a normal armless chair while most of them needed to sit on their own wheelchair. All participants could easily hold a correct sitting posture next to the exoskeleton to effectively complete the rehabilitation sessions.

The device was tested for a total of more than 120 h. During the entire duration of the treatment no mechanical or electrical failures or reliability-related issues were reported; the clinical validation was completed without any damages to the device (or any of its components) or any undesired effects on the patients.

# Efficacy of the Treatment Overall Assessment Based on MAS Scores

**Table 1** reports the MAS scores and the ROM values for all the participants assessed in the pre-treatment, post-treatment and follow-up sessions. The CMSA scores are reported only for the pre-treatment evaluation. Friedman test revealed no changes in the MAS scores in pre-treatment, post-treatment and follow-up session [χ 2 (2, 17) = 0.5, p = 0.779]<sup>1</sup> , thus post-hoc pairwise comparison was not performed.

At the beginning of the treatment all patients showed a MAS score 0 or 1 and a full elbow ROM as they were recruited during the subacute phase when spasticity is not usually arisen.

Fifteen out of 17 participants presented the same value of MAS score and ROM in pre- and post-treatment. A decrease of the MAS score from 1 to 0 was observed in one subject (#16), and in another (#6) an increase from 1 to 1+. Ten out of 13 subjects who participated in the follow-up, showed the same MAS score in the follow-up and the post-treatment sessions; two subjects (#7 and #15) showed a decrease from 1 to 0, whereas an increase from 0 to 1 was observed in one subject (#16). In the follow-up session, one participant revealed moderate spasticity (MAS = 1+).

#### Overall Assessment Based on NEEM Metrics

**Figure 3** reports the results of the two indices extracted from the NEEM measurements, MET and ZTA. Data are presented both for each subject and as aggregated form. Comparison between the initial and final treatment sessions did not show any statistical difference in MET (p = 0.09) and ZTA (p = 0.28), in accordance with the MAS results. **Table S1** (in Supplementary Materials) reports the numerical data.

Subject #6, who presented an increased MAS score between pre- and post-treatment, also showed an increased MET (+27%, absolute value) and an increased ZTA (+28%); subject #16, who showed a decreased MAS score in the post-treatment, presented a decreased MET (−27%, absolute value) and a decreased ZTA (−25%). **Figure 4** shows MET and ZTA values from subject #6

<sup>1</sup>The software is able to consider missing data in the computation of statistical tests

and subject #16 over all the treatment sessions. The regression line is calculated using the least-square technique and confirms the trend over the whole treatment.

## Intra-Session Treatment Assessment Based on NEEM Metrics

**Figure 5** shows the comparison between MET and ZTA values referred to the first and last cycles of the rehabilitative sessions. Data are presented both for each participant and as aggregated form. Comparison between the initial and final values revealed a statistical difference in MET (p = 0.020) and ZTA (p = 0.003).

In 10 subjects, when comparing the initial and final cycles of the treatment sessions, both MET and ZTA values substantially decreased (at least −10% from their initial values on both parameters). One subject (#7) showed increased MET and ZTA values (over +10% on both parameters), whereas the others showed variations in the range of ±10% of the initial values (#3, #4, #5, #8, #13, #14).

# DISCUSSION

#### Overall Usability and Safety

The use of robotic devices for rehabilitation can provide numerous advantages in terms of intensity and repeatability of the treatment for the patient and decrease of physical effort for the physiotherapists (Turchetti et al., 2014). Moreover, robot-assisted rehabilitation treatments may provide better optimization of healthcare resources (e.g., therapists can supervise more than one patient at a time, without reducing the effort dedicated to each patient). In addition robotic devices are able to provide quantitative evaluation of motor recovery by means of specific parameters that can characterize the rate of improvement of the patients (Colombo et al., 2005). However, for the treatment of upper-limb spasticity, studies using the MIT-Manus (Krebs et al., 2004), Bi-Manu Track (Hesse et al., 2003), and REHAROB (Fazekas et al., 2007) robots proved that passive repetitive exercises could improve spasticity in chronic stroke patients, but few of them correlated the treatments with the rising of spasticity at a later stage in sub-acute patients. Moreover, just recently it has been demonstrated that from a functional perspective, there is significant effect in favor of robot-assisted therapy with respect to traditional therapies (Staubli et al., 2009;


*NA, Not Available.*

FIGURE 3 | Overall therapy effectiveness. (A) Maximum extension torque (*MET*) in the first (blue) and last (red) day of treatment. (B) Zero-Torque Angle (*ZTA*) in the first (blue) and last (red) day of treatment. The left panels show the median values and 90% confidence interval for each subject. The right panel reports the inter-subject median value and confidence interval.

Mehrholz et al., 2012), whereas previous review studies could not prove it (Prange et al., 2006; Kwakkel et al., 2007).

Subjects having different levels of motor impairments were able to complete the rehabilitation sessions characterized by the same flexion and extension angle and speed values.

This study shows the safety of the device for treating patients with different body sizes and motor impairments, and for prolonged sessions.

Moreover, it contributes to the investigation of efficacy-related aspects in robotic post-stroke rehabilitation.

# Efficacy of the Therapy: Early Mobilization of the Paretic Limb Could Contribute to Reduced Risk of Developing Spasticity

Although a growing number of studies evaluated the prevalence of spasticity among the post-stroke population, the huge variability in time of onset of spasticity still impedes the determination of a universal prevalence rate (Ward, 2012).

One of the most recent studies on this topic reported that within 2 weeks after stroke, 24.5% of patients develop increased muscle tone (i.e., MAS ≥ 1), while after 16 weeks spasticity increases only slightly, up to 26.7%. Results highlight that in

98% of the patients who developed spasticity within the first 2 weeks, this is still present after 16 weeks (Wissel et al., 2010). Another study reports a similar finding: patients who develop spasticity after 3 months from stroke (i.e., 19% of patients), in general continue to report spasticity one and a half years later (Sommerfeld et al., 2004). At the same time, some patients stopped reporting spasticity, even though this outcome is rather rare. In line with the above mentioned studies (Sommerfeld et al., 2004; Wissel et al., 2010), in our population all the patients, but one, reporting spasticity in the pre-evaluation session (i.e., MAS ≥ 1), reported the same after treatment and in the follow-up.

Recently, Kong and colleagues (Kong et al., 2012) reported that 3 months after stroke, 33% of patients present spasticity, 65% of which moderate to severe spasticity (i.e., MAS ≥ 1+). Strikingly, our results show that among the 13 subjects who reported spasticity in the follow-up (MAS >0), only one participant showed moderate spasticity (with MAS = 1+), and none reported severe spasticity (i.e., MAS > 1+). The possible reasons to explain this positive outcome can be two-fold.

On the one hand the spasticity treatment we provided through the NEEM in the early post-stroke phase, prior to the progression of moderate or severe spasticity (Paolucci et al., 2000; Leathley et al., 2004), could effectively prevent the rising of spasticity in the later stage. This hypothesis is grounded on results reported by several studies. In particular, O'Dwyer and colleagues examined 24 hemiparetic patients recruited within 13 months of stroke and demonstrated how preventing or treating contractures in the early rehabilitation phase, is paramount to reduce the occurrence of spasticity (O'Dwyer et al., 1996). The intense rehabilitation treatment provided in this study by means of the NEEM can doubtfully be replicated by physical therapists, due to the huge amount of time and effort, and such intense early mobilization could have contributed to preventing the rising of severe spasticity in the following months.

On the other hand, it might be possible that prevalence rates calculated on our relatively-small group of patients, despite being similar to many similar studies on robot-assisted robotic treatment, can be slightly different from the statistics reported by the studies carried out on larger populations. It is worth noting that in this study, the absence of a control group prevents us from drawing conclusions about the relative efficacy of robot-assisted therapy with respect to other conventional physical therapies or spontaneous improvements: in the future, a randomized control trial will be necessary to evaluate the effects of different rehabilitation approaches to treating emerging spasticity.

#### NEEM Metrics Could Reliably Assess the Outcome of the Rehabilitation

NEEM metrics has been validated against two benchmarks, i.e., the MAS scores and evidence from the state-of-the-art.

First of all, NEEM metrics was used to assess the overall treatment effects and outcome compared to the MAS scores. In agreement with results obtained from the MAS scores, both MET and ZTA values showed no statistical differences between preand post-treatment values. The qualitative comparison between device measurements and traditional clinical scales revealed that the robot measurements could detect changes in patients who reported a change in the MAS score (see subject #6 and #16). At the same time, the device revealed to be more sensitive in detecting small changes in joint resistance to passive extension since some participants who did not reported any MAS changes from pre- to post-treatment, showed visible differences in MET and ZTA.

Second, we validated intra-session results from NEEM metrics with evidence from the state of the art. It has been widely demonstrated that within a rehabilitation session with passive mobilization of the joint, its resistance to extension decreases (Nuyens et al., 2002). Indeed, muscle stretching and joint mobilization is a common intervention for treatment of spastic hypertonia or muscle contractures, which is based on the clinical experience that resistance decreases during repeated passive movements. Thus, we expected this result in almost all the participants and NEEM indices confirmed a statistical reduction of the elbow resistance, highlighting at the same time the reliability of the measurements extracted from the variable collected by the device.

The benchmark with the MAS scores, that is a golden standard for the assessment of joint spasticity, and evidences from the state of the art proved that the robot measurements could reliably describe the overall outcome of a rehabilitation treatment and suggests the possible use of the device not only for providing the treatment, but also as a tool to assess the outcome of a therapeutic treatment in an objective manner.

# CONCLUSIONS

The results of this study proved the usability and safety of the NEEM in rehabilitation; moreover, the results suggested that intense early rehabilitation could contribute to preventing elbow spasticity from occurring at a later stage (3–4 months after stroke); however the small sample size does not allow deriving straightforward conclusions applicable to larger populations.

The metrics extracted from the robot measurements were consistent with the MAS scale suggesting that the device could be potentially used as a tool to assess the outcome of rehabilitation. Notably, other features can be extracted from the NEEM measures (e.g., positive/negative delivered energy or power, active ROM, zero-torque angle and maximum torque in the

### REFERENCES


flexion phase) and it can be easily synchronized with commercial electromiographic devices and allow measuring muscle activity.

The positive findings of this work invite further investigations in order to test the effectiveness of the proposed approach in a large-scale clinical study, where different therapy modalities, such as active-assisted or active-resisted exercises, can be proved to be able to prevent the occurrence of spasticity.

# AUTHOR CONTRIBUTIONS

Study concept and design: SC, MC, NV, MCC, FP. Acquisition of data: SC. Statistical analysis and interpretation of data: SC, MC, NV, FP, SM, MCC. Drafting of manuscript: SC, MC, NV, FP, SM, MCC. Study supervision: NV, FP.

# FUNDING

This study was partly supported by: the European Commission under the H2020 AIDE project (GA 645322) and by Regione Toscana under the RONDA project (Bando PAR FAS 2007– 2013).

# ACKNOWLEDGMENTS

The authors would like to thank Francesco Giovacchini, Matteo Moisè, and Mario Cortese for their support in the development of the NEEM, and Stefano Romiti for his help with the platform installation and data acquisition.

## SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: http://journal.frontiersin.org/article/10.3389/fnins. 2017.00261/full#supplementary-material

Table S1 | Absolute value of MET and ZTA values for each subject. Medians and 50% confidence intervals are reported.


study. Arch. Phys. Med. Rehabil. 86, 410–415. doi: 10.1016/j.apmr.2004. 10.022


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

SC, MCC, and NV have interests in the commercial exploitation of the patent application protecting the design of the NEEM torsional spring.

Copyright © 2017 Crea, Cempini, Mazzoleni, Carrozza, Posteraro and Vitiello. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# On the Value of Estimating Human Arm Stiffness during Virtual Teleoperation with Robotic Manipulators

#### Jacopo Buzzi <sup>1</sup> \*, Giancarlo Ferrigno<sup>1</sup> , Joost M. Jansma<sup>2</sup> and Elena De Momi <sup>1</sup>

*<sup>1</sup> Department of Electronics, Information and Bioengineering, Politecnico of Milan, Milan, Italy, <sup>2</sup> Mechanical Engineering Department, Delft University of Technology, Delft, Netherlands*

Teleoperated robotic systems are widely spreading in multiple different fields, from hazardous environments exploration to surgery. In teleoperation, users directly manipulate a master device to achieve task execution at the slave robot side; this interaction is fundamental to guarantee both system stability and task execution performance. In this work, we propose a non-disruptive method to study the arm endpoint stiffness. We evaluate how users exploit the kinetic redundancy of the arm to achieve stability and precision during the execution of different tasks with different master devices. Four users were asked to perform two planar trajectories following virtual tasks using both a serial and a parallel link master device. Users' arm kinematics and muscular activation were acquired and combined with a user-specific musculoskeletal model to estimate the joint stiffness. Using the arm kinematic Jacobian, the arm end-point stiffness was derived. The proposed non-disruptive method is capable of estimating the arm endpoint stiffness during the execution of virtual teleoperated tasks. The obtained results are in accordance with the existing literature in human motor control and show, throughout the tested trajectory, a modulation of the arm endpoint stiffness that is affected by task characteristics and hand speed and acceleration.

Keywords: human-robot interaction, surgical teleoperation, arm end-point impedance, musculoskeletal model, master devices comparison

# 1. INTRODUCTION

Teleoperated robotic systems are widely used in several long-short range application fields, from plant decommissioning (Cragg and Hu, 2003), environment exploration (Mitsou et al., 2006), hazardous material handling and surgery (Johnson and Somu, 2016). In order to control the remotely operated system, the user interacts with a master interface, a robot whose precise and accurate manipulation allows task execution at the slave side. The design of master interfaces is a fundamental aspect in teleoperation, indeed several studies focused on the master devices physical and control characteristics to achieve control transparency while assuring system stability (Fischer et al., 1990; Conti et al., 2014; Enayati et al., 2016). In teleoperation, stability of humanrobot interaction (Colgate, 1994) is guaranteed by closing the control loop with the user sensory and motor systems and, in this context, both the robot and the user are considered as passive elements (Lee and Spong, 2006). Therefore, although the design of master devices architecture

#### Edited by:

*Dingguo Zhang, Shanghai Jiao Tong University, China*

#### Reviewed by:

*Jason Scott Metcalfe, US Army Research Laboratory Human Research and Engineering Directorate, United States Victor Manuel Pulgar, Wake Forest School of Medicine, United States*

> \*Correspondence: *Jacopo Buzzi jacopo.buzzi@polimi.it*

#### Specialty section:

*This article was submitted to Neural Technology, a section of the journal Frontiers in Neuroscience*

Received: *10 January 2017* Accepted: *11 September 2017* Published: *26 September 2017*

#### Citation:

*Buzzi J, Ferrigno G, Jansma JM and De Momi E (2017) On the Value of Estimating Human Arm Stiffness during Virtual Teleoperation with Robotic Manipulators. Front. Neurosci. 11:528. doi: 10.3389/fnins.2017.00528*

**204**

relies on the human user to achieve stability, the high variability that characterizes human control strategies are usually neglected. In fact, in human motor control, task stability is achieved by regulating the dynamic proprieties of the limbs through muscular activation. The arm impedance and its components (viscosity, inertia and stiffness) can be modified to adapt to different tasks and desired interactions by tuning muscle contractions and varying joint angular position, . As the most predominant component, the arm stiffness also directly depends on the joint angular velocity, reflex modulation and the presence of expected perturbations (Flash and Mussa-Ivaldi, 1990). Previous studies (Burdet et al., 2001) demonstrated the existence of control strategies employed to achieve stability through the regulation of the arm endpoint stiffness in terms of maximal value and orientation. Other studies showed the importance of stiffness regulation during complex tasks execution and motor learning (Osu et al., 2002; Gribble et al., 2003; Buzzi et al., 2017).

Up to now, the dynamic proprieties of the human arm have not been fully considered in the control and optimization design of the master interfaces (Hadavand et al., 2014; Qiu et al., 2014). Through robust and continuous estimation of arm stiffness, it would be possible to implement a master controller able to adapt and regulate the physical interaction between the robot and the human (Tsumugiwa et al., 2002; Ajoudani et al., 2012) in order to achieve better performance, higher resistance to external perturbations, and possibly reducing muscular fatigue (Wiker et al., 1989).

In order to estimate the human arm stiffness, several methods and devices have been proposed in literature: Flash et al. (Flash and Mussa-Ivaldi, 1990) as well as Gomi et al. (Gomi and Kawato, 1997) used planar robotized handles fitted with force sensors to record the interaction forces between the subjects hand and the robot when known displacements were applied. Force and displacement variations were used to compute hand stiffness in multiple directions. While these methods produce a measurement of the stiffness during postural maintenance, they cannot be applied to the study of stiffness during movement and task execution without interfering with arm kinematics. To overcome this limitation, microscopic displacements combined with a time frequency analysis were used to estimate the mechanical proprieties of the arm during a single reaching movement (Piovesan et al., 2013). Following an electromyography (EMG) based approach, recent works (Darainy et al., 2004; Shin et al., 2009; Ajoudani et al., 2015), estimated the arm endpoint stiffness using simplified planar musculoskeletal models and recorded surface muscle activations from couples of shoulder and arm antagonist muscles. Muscular models were used to estimate the force direction and arm of muscular units, while force intensities were obtained through calibration from the recorded EMG signals and the maximal voluntary contractions. Although the simplified musculoskeletal models showed the ability to estimate the arm endpoint stiffness during task execution and without applying perturbations to the user kinematics, stiffness computation was limited to specific planes and directions thus neglecting the effects of other couples of muscles, such as the wrist flexor/ extensors.

In this work, we present a non-disruptive method for the computation of the arm endpoint stiffness based on a user specific 7 degrees of freedom (DoF) musculoskeletal model of the upper limb (Delp et al., 2007). The dynamic characteristics of the model as well as the activation dynamics of muscle units are used in conjunction with joint kinematics and EMG signals to obtain a continuous estimation of the arm stiffness (Pizzolato et al., 2015).

The aim of this work is to evaluate how different master devices and tasks influence the regulation of arm endpoint stiffness and its relation with task performance, hand speed and acceleration. We developed two planar tasks consisting in the position and orientation control of a virtual tool. The two task variations were intended to trigger the use of different biomechanical DoFs to demonstrate the influence of such constraints on the estimation of end-effector stiffness. We compared a parallel link (PL) and a serial link (SL) master device, to test how the differences between the two manipulators would elicit different levels of arm end-point stiffness during virtual teleoperation. During task performance, arm kinematics and EMG signals were acquired using an optical and a magnetic tracking systems).

Our primary hypothesis is that end-effector stiffness would be modulated according to both the mechanical features of the master device and the task characteristics. In particular, we expect the users to generate higher overall end-point stiffness when teleoperating with the serial link master device, characterized by lower structural stiffness and apparent mass. We also expect the users to increase arm stiffness while performing the second task; due to the increased complexity of simultaneously controlling position and orientation of the virtual tool. Our second hypothesis is that end-point stiffness would be modulated accordingly to curvature variations throughout the trajectory. More specifically, we believe that high curvature can be associated with higher complexity in executing the task, leading to higher values of arm end point stiffness. As third and final hypothesis, we expect a correlation between the hand speed and acceleration and arm end-point stiffness so that the users would generate the maximal levels of stiffness during slow and non accelerated movements.

# 2. MATERIALS AND METHODS

In order to analyze the stiffness regulation during virtual teleoperated tasks, we created a simple teleoperation scenario in which the users interacted with master devices to control virtual tools used to perform specific tasks.

# 2.1. Tasks Design

Simple repeatable and cyclical planar virtual tasks were developed to challenge the users with different levels of complexity without requiring any surgical expertise.

1. Half Cloverleaf (HC)

**Figure 1**, 1 shows the first virtual trajectory which represents half the cloverleaf motion as presented by Levit-Binnun et al. (2006). A virtual stylus shaped tool was manipulated in the simulated environment to follow the trajectory, starting from

FIGURE 1 | The figure represents the two trajectories designed. Task 1. A line following task shaped as an half cloverleaf (HC). The aim to to finely follow the trajectory starting from the green dot and moving anti-clockwise. Task 2. In the task, shaped as a thicker half cloverleaf (SFHC), the users have to orient the tool's cylindrical end-effector along the trajectory (2.1–2.4).

the initial position (green dot in **Figure 1**, 1) and moving first counter-clockwise, then straight crossing the intersection in the middle, finally in a clock-wise direction to return to the starting position. In order to make the subject perform the task approximately in the x-y plane of a three-dimensional Cartesian reference frame, a visual cue was provided in the form of the path color which turned green when the tool-tip of the stylus was in the said plane (−1mm < z < 1mm), red otherwise.

#### 2. Shape Fitted Half Cloverleaf (SFHC)

The second task was designed to include wrist rotation as the subjects were asked to navigate a virtual cylindrical shaped tool tip along the path, as shown in **Figure 1**, 2. The trajectory that the users were requested to follow was identical to the one of the previous task, with the added complexity of following the trajectory silhouette using the tool end-effector. Starting from the green dot (**Figure 1**, 2), the users were asked to slide the tool end-effector over the shape fitted trajectory (see **Figure 1**, 2.1) . In order to follow the first portion of the trajectory, the users needed to flex the wrist until the central intersection was reached (see **Figure 1**, 2.2) and then to extend it to reach the top left. At the trajectory upper end (red dot in **Figure 1**, 2.2), the users had to flex the wrist and abduct the shoulder to reposition the tool end-effector on the shape fitted trajectory (see **Figure 1**, 2.3). Similarly, the users had to reach the lower open end (see **Figure 1**, 2.4) to complete the task. As in the previous task, the visual feedback consisted in the change of the path color to green when the tool tip lied in the desired plane.

## 2.2. Experimental Setup

#### 2.2.1. Master Devices

The subjects were controlling the virtual tools position and orientation using two different master devices: a 7 DoFs parallel links haptic interface (PL) and a 6 DoFs serial links haptic interface (SL).

FIGURE 2 | Experimental setup: the user performs the task looking at a monitor laid flat on a table (1) while teleoperating using either a parallel link haptic device PL (2) or a serial link haptic device SL (3) while the thorax and arm kinematic are acquired with an optical camera (4) and reflective markers (5) and with electromagnetic tracker (6) and markers (7). The EMG activation is also acquired with bipolar electrodes (8).The virtual reference frame (9) and the shoulder reference frame (10) are grossly aligned in the setup phase.

A Force Dimension Sigma7 (Force Dimension, Nyon, Switzerland) was used as PL (see **Figure 2**, 2). The master device, gravity compensated, is characterized by 6 DoFs plus a grip control, has a resolution of 0.0015 mm and 0.013◦ and an elliptical dome workspace with radiuses of approximately 190 × 130 mm. Thanks to its design, the translational and rotational degrees of freedom are completely decoupled, and the grasping unit has an apparent mass of 259 g (Tobergte and Helmer, 2013). The parallel-link structure also contributes to produce a system stiffness of approximately 14 N/mm.

A Phantom Omni (3D Systems, South Carolina, USA) was used as SL (see **Figure 2**, 3). The device is characterized by a 0.055 mm resolution, a 160 × 120 × 70 mm workspace and its controlled through a stylus end-effector. Even though the device is not gravity compensated, which means that the tele-operator had to sustain the stylus and part of the links weight when manipulating it, it is characterized by an apparent mass at the tip of 45 g. Due to its design, the system stiffness components are not constant in the three axes, ranging from 1.02 N/mm to 2.31 N/mm.

Such differences in the master devices dynamic proprieties, both in terms of apparent mass of the end-effector and system stiffness, contribute to create a higher sense of stability while teleoperating with PL.

For both the SL and PL master devices custom impedance controllers were developed using the proprietary API. In both master devices, the users hand movements were downscaled with a factor of 2. The described tasks were designed to fit within the workspaces of both devices in order to avoid the necessity of using the devices clutching option. If used, the clutching would allow to decouple the virtual tool position from the master device end-effector position, allowing to reconfigure the arm when hitting the workspace limits. This option was excluded from the experiments since it would have caused significant kinematic variability during task execution.

#### 2.2.2. Acquisition Framework

The users thorax and arm position and configuration were acquired using two localization devices calibrated to the same reference frame with a hand eye calibration approach (Horaud and Dornaika, 1995). The thorax position was acquired using an optical localization system (see **Figure 2**, 4) [Vicra, Northern Digital, Ontario, Canada, 20 Hz sampling rate, 0.25 mm position Root Mean Squared Error (RMSE)] using three passive retroreflective markers attached to the right and left acromions and next to the jugular notch.

The arm configuration was measured using an electromagnetic localization system (see **Figure 2**, 6) (Aurora, Northern Digital, Ontario, Canada, 30 Hz sampling rate, 0.48 mm and 0.3◦ position and orientation RMSE, dome shaped field with a radius of approximately 500 mm) and three 6-DoF 1.8 × 9 mm electromagnetic sensors (see **Figure 2**, 7) that were used to generate 6 corresponding virtual markers calibrated on the users recognizable bony landmark on elbow, wrist and hand.

EMG signals were recorded through a TMSi Porti device (Twente Medical Systems International, Oldenzaal, Nederland, 32 channel acquisition system, 2,048 Hz sampling rate) using 10 bi-polar electrodes (see **Figure 2**, 8). Three couples of electrodes were used to acquire the electromyographic signals from the anterior, lateral and posterior deltoid fiber bands (**Figure 3**, 1- 2-3). Two couples of electrodes were used to acquire the long and lateral triceps brachii heads (**Figure 3**, 4-5) and a single couple was used to acquire the biceps muscle (**Figure 3**, 6). Four electrode couples were used on the forearm to measure the activation of the brachioradialis, flexor carpi ulnaris and radialis and extensor digitorum (**Figure 3**, 7-8-9-10). The monopolar electrode used as reference was attached to the users' left hand. The muscular maximal voluntary contraction (MVC) was recorded right before the experiments: each subject was asked to perform a set of different isometric contractions against

static resistance with different joint configurations (Lehman and McGill, 1999). The different movements were designed to elicit the activation of the muscles responsible for the same kinematic function: for instance, the biceps' MVC was recorded by asking the users to push the hand palm against a tabletop bottom while flexing the elbow; similarly, the forearm flexors' MVC were acquired during isometric wrist flexions against static objects. The contractions lasted for about 5 s and were followed by a second trial after 30 s, to avoid muscular fatigue. The mean of the two maximal absolute values registered during the two repeated movements was extracted from the conditioned signals for each muscle. The MVC signals were processed with the same procedure as the EMG signals recorded during the experiments.

Retroreflective and electromagnetic markers movements and EMG signals were acquired, recorded and synchronized using custom developed software based on the Robotic Operating System (ROS, http://www.ros.org/).

#### 2.2.3. Experimental Protocol

ulnaris respectively and 10 for extensor digitorum.

We recruited 4 healthy subjects (2 female and 2 male, mean age 23 ± 1.5) who provided informed written consent, in accordance with the recommendations of Politecnico di Milano Ethical committee Board. All subjects gave written informed consent in accordance with the Declaration of Helsinki.

The subjects were sitting in a comfortable chair without arm rests, in front of a 2D monitor screen where the virtual tasks were displayed (**Figure 2**, 1) . The chair position was adjusted in order to allow an easy interfacing between the subjects and the tasks. The monitor was laid as flat as possible on the table to be approximately parallel to the plane in which the tasks were performed, allowing for the most intuitive control, in which the users hand movement directly corresponded to the virtual tool movement.

TABLE 1 | Experiments list and description.


Each user was asked to perform ten trials of every task while trying to maintain the hand movement on a constant plane, exploiting visual color feedback as explained in section 2.1. Subjects were instructed to perform HC and SFHC tasks finding a personal trade-off between precision and execution speed. **Table 1** describes the four experiments performed: for each subject the experiment order was randomized so that users performed the 10 repetitions of a randomly selected condition before moving on the next randomly selected one.

#### 2.3. Stiffness Computation

The kinematic and dynamic characteristics of the user arm movements, as well as their muscular activations were obtained and elaborated to compute the arm end-point stiffness during task execution.

#### 2.3.1. Kinematic Analysis

The arm and thorax spatial configuration were reconstructed using the experimental markers data and the musculoskeletal model implemented in OpenSim (Delp et al., 2007). The model, whose capabilities in predicting the arm dynamics were previously assessed (Saul et al., 2015), is derived from Holzbaur's model (Holzbaur et al., 2005) and composed by seven DoFs activated by 32 muscle compartments. The model was scaled to fit the anthropometric characteristics of each subject and its virtual markers were displaced to the average real markers position acquired during a static pose.

The markers position data were filtered by applying an IIR second order Butterworth filter twice, reversing the time in the second filtering in order to cancel the non linear phase shift. Filter cutoff frequency was set to 4 Hz (-6 dB) and the filtered data were used as input for the inverse kinematics. The inverse kinematic algorithm solves the minimization problem in Equation (1) for each set of marker coordinates.

$$\min\_{\mathbf{q}} \left[ \sum\_{k \in N = \text{number of markers}} \boldsymbol{w}\_k \| \mathbf{x}\_k^{\text{exp}} - \boldsymbol{x}\_k(\mathbf{q}) \|^2 \right] \tag{1}$$

Where **q** is the vector of joint angles, **x** exp k is the experimental 3D position of the k th marker, acquired using the optical or the magnetic tracking device, **x**<sup>k</sup> (**q**) is the position of the corresponding virtual marker on the model that depends on the vector of joint angles **q**, and w<sup>k</sup> is the k th marker weight thorough which is possible to account for the uncertainty related to each marker position detection. For each set of marker coordinates, the algorithm finds the best joint vector **q** that minimizes the error between the experimental marker position vector **x** exp and the virtual markers position vector x(**q**), which is a function of the fixed model geometry and joint vector, **q.**

In order to define the correct weighting vector **w** = [w1, . . . ,w<sup>k</sup> , . . . ,wN] with N = number of markers, we analyzed the markers positions during a static recording. The ratio between the standard deviations from optical markers and electromagnetic markers is comparable with the ratio between the RMSE characteristics of each device. We therefore weighted the thorax optical markers with a coefficient of 1, while the arm, forearm and hand electromagnetic markers with a coefficient of 0.52 (obtained from the ratio between the nominal RMSE for the optical tracker, 0.25 mm and the electromagnetic tracking device RMSE, 0.48 mm).

#### 2.3.2. Dynamic Analysis

The obtained joint angles were filtered with an IIR second order Butterworth filter (cutoff frequency 4 Hz) twice, compensating the nonlinear phase shift. The filtered data were used as input for the inverse dynamic reconstruction, along with the model dynamic characteristics (masses and inertial proprieties of each model segment). Equation (2) shows the dynamic equation when no external forces are applied to the arm; τ is the unknown (7 × 1) vector of joint torques, **M** is the system mass matrix, **C** is the vector of Coriolis and centrifugal forces, **G** is the vector of gravitational forces, while **q**, **q**˙, **q**¨ are the vectors of joint position, velocities and accelerations, respectively.

$$\mathbf{r} = M(\mathbf{q})\ddot{\mathbf{q}} - C(\mathbf{q}, \dot{\mathbf{q}}) - G(\mathbf{q}) \tag{2}$$

#### 2.3.3. EMG-Informed Torque Estimation

A refined estimation of the joint torques was obtained using the Calibrated EMG-informed Neuro-Musculoskeletal (CEINMS) modeling toolbox in which the musculotendon unit (MTU) force arms obtained with the inverse kinematics are used with the joint torques estimated by the inverse dynamics and the recorded EMG signals.

The tool firstly calibrates the experimental muscle excitations derived from the surface EMG acquisition with the muscle activation patters, obtained from the inverse dynamic algorithm (Lloyd and Besier, 2003; Sartori et al., 2012). The resultant calibrated model is then used to predict MTU forces, joint moments, and muscle activations solving a set of differential equations that relates the muscle excitation with the electrical activity recorded (Shao et al., 2009). To be used as an input to the EMG-informed inverse dynamic algorithm, the EMG data was previously high pass filtered at 30 Hz, full wave rectified and filtered with a zero-lag second order Butterworth filter (6 Hz cutoff frequency) (Lloyd and Besier, 2003). The experimental muscle excitations were normalized using each users maximal voluntary contractions (MVC).

#### 2.3.4. Stiffness Computation

For each model DoF, the joint stiffness (7 × 7) matrix (**Kj**) was computed using Equation (3), where τ (i) is the joint torque vector that is associated with the **q**(i) arm configuration at the i th time instant (McIntyre et al., 1996).

$$\mathbf{K}\_{\dot{l}}(i) = \frac{d\mathbf{r}(i)}{d\mathbf{q}(i)}\tag{3}$$

In order to obtain the Cartesian representation of the arm stiffness, the arm end-point Cartesian stiffness (**Ke**) has been obtained using the arm kinematic Jacobian **J** (7 × 6), as follows:

$$\mathbf{K\_{e}}(i) = \mathbf{J}^{T^{-1}}(i)\mathbf{K\_{j}}(i)\mathbf{J}^{-1}(i) \tag{4}$$

Through the Singular Value Decomposition (SVD), the left singular vectors and non-zero singular values of **K<sup>e</sup>** were obtained to draw the end-point stiffness ellipsoid orientation and dimensions that are used as a way of visually represent the stiffness (McIntyre et al., 1996). The maximal singular value of **K<sup>e</sup>** is the maximal stiffness value (Kmax(i)) in the i th time instant.

#### 2.4. Metrics and Statistical Analysis

The following performance-related parameters were evaluated:

• 2D distance from the trajectory at the ith time frame, d<sup>i</sup> : the distance is computed from the tool tip to the closest point on the trajectory center on the task plane (see **Figure 1**); it is calculated as shown in Equation (5).

$$d(i) = \sqrt[2]{(\chi\_{target}(i) - \chi\_{total}(i))^2 + (\chi\_{target}(i) - \chi\_{total}(i))^2} \tag{5}$$

Where (x, y)target are the tool tip's 2D coordinates and (x, y)tool are the 2D coordinates of the closest point on the trajectory. The z-component of the displacement from the trajectory was excluded from the distance metric computation due to the significant differences that the color visual feedback could introduce with respect to the other two dimensions.

• Maximal stiffness **K**max: the end-point stiffness ellipsoid main axis was computed in 29 (for HC) and 23 (for SFHC) equally spaced points along the task trajectory (for SFHC, the points corresponding to the open ends were excluded from the analysis). For each task trial and for each point selected, the three closest virtual tool positions were searched and the corresponding maximal stiffness (**K**max(j), **K**max(j + 1) and **K**max(j − 1)) were averaged to obtain the mean maximal stiffness estimation as in Equation (6)

$$\overline{K}\_{\text{max}} = \frac{1}{3} (K\_{\text{max}}(j-1) + K\_{\text{max}}(j) + K\_{\text{max}}(j+1)) \quad \text{(6)}$$

With j = 1...M − 1 number of points along the trajectory.

• Curvature C: In both tasks, the trajectory was created to fit the same Bernulli's Lemniscate function. The generic function of a Bernulli Lemniscate curve in a planar x-y plane with the main axis oriented along the x axis is

$$(\chi^2 + \chi^2)^2 = 2a^2(\chi^2 - \chi^2) \tag{7}$$

Where a is the parameter that defines the position of the curve foci. The task trajectory curvature (C) can be computed in polar coordinates using:

$$C(t) = \frac{3 \ast \sqrt{2} \ast \cos(t)}{\sqrt{3 - \cos(2t)}}\tag{8}$$

with t that spans from 0 to 2π. For each 3D point of the trajectory selected for the stiffness analysis, the corresponding curvature was computed.

• Hand speed and acceleration: the position and orientation of the hand in the shoulder reference frame (see **Figure 2**, 10) was computed. The speed was obtained performing numerical differentiation on the x-y-z coordinates, smoothing the signal with a second order Butterworth filter with cutoff frequency of 4 Hz twice forward and backward. Acceleration was computed and filtered from hand speed following the same procedure previously described.

For all the metrics, the statistical distribution normality was tested for each user in the four experiments PL1, PL2, SL1, SL2 (see **Table 1**) using a one-sample Kolmogorov-Smirnov test with 1% significance level. The arm stiffness data resulted to be non-normal, therefore, using the natural logarithm function (as in Wilson and Worcester, 1945), the data was normalized: the same Kolmogorov-Smirnov test was performed for each user, repetition and experiment, identifying as normal the 97% of the distributions. The distance metric, as well as the hand speed and acceleration had a normal distribution (Kolmogorov-Smirnov α = 0.01).

#### 2.4.1. Analysis Performed and Hypothesis Tested

All the inferential statistic analysis that will be presented were conducted with the Statistics and Machine Learning Toolbox for Matlab 2016b (Mathworks, Natick, Massachusetts, US).

1. Maximal stiffness through the trajectory - The stiffness maximal values Kmax were first of all studied as a function of the task type and controller. The distributions of the normalized stiffness data were compared between users and repetition, since no significant differences were found, the users data were grouped together for each experiment

Hypothesis 1 & 2: To analyze whether the task and master device types, as well as level of trajectory curvature affected the maximal values of stiffness, the data from the 29 (HC) and 23 (SFHC) points on the trajectory for each experiment were grouped. The mean values of maximal stiffness for each region of curvature and repetition were extracted and a threeway ANOVA was performed on the log-normalized data (task, master device type and curvature region as fixed factors, user number as random-effect factor). To further test the effects of the different factors on the end-point stiffness, the same ANOVA model was also applied to the stiffness variability in the various regions and repetitions. To test the presence of possible correlations between stiffness and curvature the Pearson rank test (α = 0.05) was adopted.

2. Stiffness against hand speed and acceleration - Using two separate two-ways ANOVAs (task and master device type as fixed factors, user number as random-effect factor) the hand speed and acceleration distributions in the four experiments were analyzed. Hypothesis 3: we tested the hypothesis that a relation between hand speed and acceleration with respect to maximal stiffness could exist. For each of the ten trajectory repetitions, 4 levels of hand speed and acceleration were obtained in the four experiments and for the 4 users. Two separated three-way ANOVAs were performed on the corresponding values of stiffness (task, master device type and level of speed/acceleration as fixed factors and user number as random effect factor).

3. Distance metric - The difference between different trials of the same experiment for each user was tested using a one-way ANOVA test (α = 0.05). Since no statistical difference emerged, the mean values of distance throughout each repetition and user were extracted. The four experiments' performance distributions were then modeled with a twoway ANOVA statistical model, where task and master device type were considered as fixed factors, while user number was considered as random effect factor.

## 3. RESULTS

# 3.1. Maximal Stiffness through the Trajectory

**Figure 4** shows the results obtained when the mean stiffness values through the trajectory among users and trials for each experiment are grouped together. The stiffness values below a threshold of 0.1 N/m were removed from the analysis: the threshold was obtained from previous arm end-point stiffness estimations (Gomi and Kawato, 1997; Tsumugiwa et al., 2002) and the percentage of values discarded was less than 1% of the total dataset. The results of the three-way ANOVA analysis showed no significant interaction between the factors, and a statistical difference between tasks (HC and SFHC) [F(1, 624) = 10.06, p < 0.005] while no significant difference was found between the master devices [F(1, 624) = 0.14, p = 0.70]. Overall, users elicited significantly higher arm stiffness when performing the HC task with respect to SFHC. **Figure 5** shows the endpoint stiffness variability in the four experiments. The three-way ANOVA test showed no significant interactions among factors and statistical difference between both tasks [F(1, 624) = 7.75, p < 0.01] and master devices [F(1, 624) = 7.64, p < 0.01]. Interestingly while the stiffness variance increases in the SFHC task when teleoperating with PL, when teleoperating with the SL master device, instead, users decreased the arm stiffness variance when performing SFHC with respect to HC. Overall, users explored the highest range of end-point stiffness in HC when teleoperating with SL and showed the lowest variance with the same master device in the SFHC task.

To further analyze stiffness modulation through the 29 (for HC) and 23 (for SFHC) points along the trajectory while teleoperating with the PL and SL master devices, **Figure 6** shows the median values obtained for each experiment. The results are presented as a tri-dimensional graph where each point mean Kmax value is represented as a colored column (see **Figure 6**). No evident trends can be found in stiffness maximal values modulation through the trajectory points.

Regarding the end-point stiffness mean values relation with curvature, the three-way ANOVA showed significant difference [F(3, 624) = 28.19, p < 0.0001] between the four different levels of curvature. The box-plots in **Figure 7**, 3 show the maximal

the four increasing levels of curvature. The results are presented using boxplot indicating median, first and third quartile, minimal and maximal values. Horizontal lines over the boxes indicate statistical difference while the number of stars indicate different levels of significance (\*\**p* < 0.01).

FIGURE 5 | Variance of maximal stiffness (*Kmax*) in the four experiments for the four increasing levels of curvature. Horizontal lines over the boxes indicate statistical difference while the number of stars indicate different levels of significance (\*\**p* < 0.01).

stiffness distribution as function of four levels of trajectory curvature for PL and for SL. Although characterized by high interquartile distances, the similar behavior can be observed for PL and SL and in the two tasks: the smallest Kmax mean values are registered where the normalized curvature is <0.25 while the maximal values appear in the range between 0.75 and 1. Interestingly, while for the hybrid parallel-serial link master device (PL) the end-point stiffness behavior in HC and SFCH shows the same trend, with an apparent plateau in the middle curvature zones, different behaviors can be seen with SL. In this case, HC shows a clear increase in the mean value of maximal stiffness with increasing levels of curvature, while in SFHC users were eliciting very small variations of arm end-point stiffness. Due to the high variance in the data, Pearson correlation analysis showed non-significant (p > 0.05) correlations for all the experiments. Furthermore, regarding the stiffness variability in the four regions of curvature, the three-way ANOVA test showed no significant differences [F(3, 624) = 1.54, p = 0.20].

## 3.2. Stiffness against Hand Speed and Acceleration

**Figure 8** shows the speed and acceleration distribution in the four experiments; the two separated two-ways ANOVAs showed significant differences in tasks and master devices in both speed [F(1, 156) = 998 p < 0.0001 and F(1, 156) = 36 p < 0.0001, respectively] and acceleration [F(1, 156) = 1,300 p < 0.0001, F(1, 624) = 180 p < 0.0001, respectively]. The two separate threeway ANOVAs showed no significant interaction among task, master device type and level of speed or acceleration; a main significant effect was found for the task type [F(1, 624) = 10.23, p < 0.005 when the data was grouped in levels of speed and F(1, 624) = 10.23, p < 0.005 for levels of acceleration] while no significant effects were found for the master device type and the levels of speed or acceleration. **Figure 9** shows the endpoint stiffness distribution with respect to increasing levels of speed and acceleration for the four experiments. To emphasize the differences in the stiffness distribution, the non-normalized stiffness values are presented. The Pearson correlation tests between the log-normalized data and the hand speed and acceleration in the four experiments showed low (|p| ranging from 0.1 to 0.3) non significant (ρ > 0.1) negative correlations.

#### 3.3. Distance Metric

The mean distance value for each repetition was obtained for each of the 10 repetitions and users and the obtained data sets were grouped in the four experiments. The results of the twoway ANOVA with subjects' number as random-effect parameter showed a difference between HC and SFHC [F(1, 156) = 548 p < 0.0001] and no difference between PL and SL [F(1, 156) = 2.43 p = 0.1214]. The users were capable of achieving the best performances, testified by lower distances, with the parallel link master device (PL) and in the first task (HC). Similarly as seen with the maximal stiffness values (see **Figure 4**), the interquartile distances for SFHC are significantly higher than in HC.

#### 4. DISCUSSION

We evaluated the arm end-point stiffness modulation adopted by novice teleoperators in performing two tasks with two

different master devices as a reflection of the control strategies adopted by the central nervous system to increase the hand resistance to internal and external noise. Using kinematic and muscular parameters, we estimated the stiffness and its relation with the trajectory characteristics and the hand speed and acceleration.

The difference in maximal stiffness values between the different task types (as presented in **Figure 4**) proves that the users were adapting their stiffness modulation strategies to the different task characteristics. We were expecting higher values of end-point stiffness during the execution of the SFHC task, due to the increased complexity added by the requests to orient the tool; instead, the stiffness is significantly higher in the HC tasks. A possible explanation to this result may be that the users were discouraged to increase the arm stiffness in order to comfortably activate the wrist joint. In fact, to increase the overall arm stiffness, it would be necessary to increase the level of co-contraction also for the wrist flexor-extensors, potentially impairing the free rotation of the wrist. Therefore, this significant difference in stiffness values between HC and SFHC may be explained by the different levels of wrist flexionextension activations that were registered in the two tasks. As an example, **Figure 10** shows the wrist flexion-extension patterns found during the execution of HC and SFHC for a single user

with the PL master device. The same significant difference can be seen in all the acquired users.

The fact that users were eliciting similar values of stiffness when teleoperating with the two master devices seems to suggest that users can compensate for the significant differences in the master devices mechanical proprieties assuring the same overall dynamic performances.

Users showed the highest end-point stiffness variability when teleoperating with the serial link robot during the execution of the HC task, possibly hinting that for that specific task, the SL master device's characteristics offered a less stable and reliable interaction with respect to PL. Interestingly, while for the PL master device the request to orient the tool along the trajectory introduced higher variability in the elicited stiffness, the opposite can be seen for SL. This discrepancy could be caused by the significant differences in the master device structural construction: in PL the hand rotations are completely decoupled from translations while in SL rotations are less hand-centered. This finding would therefore endorse the hypothesis that different kind of tasks may require different master device mechanical characteristics.

The statistical difference in arm stiffness mean values between different regions of curvature could suggest a relation between trajectory curvature and stiffness maximal values: in fact, the users were generally generating the lowest stiffness in the lowcurvature portion of the trajectory. Under the hypothesis that straight trajectories are easier to follow than highly curved ones, it is possible that the users were relying on higher stiffness to increase their performance in the most difficult parts of the trajectory. On the other hand, this modulation seems particularly affected by the task and master device characteristics. By looking

at the stiffness distributions in **Figure 7**, it seems that while the PL master device offers a rather consistent interface for both tasks, allowing more or less the same modulation in HC and SFHC, the SL master device showed different behaviors in the two tasks. Specifically, while an increase in the mean stiffness at high curvature can be seen for the HC task, arm end-point stiffness was almost constant for SFHC. This discrepancy may suggest that, during the SFHC task performance, when teleoperating with the SL master device, users weren't able or didn't felt the necessity to adopt the same kinetic strategy that they used otherwise.

The disparity in the speed and acceleration distributions between the two tasks could lie in the differences in the task graphical representation: in order to make the SFHC task more clear, the trajectory thickness is increased with respect to the HC. In the same way, the tool-tip changes from a small point (in HC) to a cylinder (in SFHC). These differences could have decreased the users perception of the distance from the target trajectory thus reducing the feedback on their actual performance. Consequently the users performed the task with higher speed. This explanation seems to be endorsed by the statistically higher mean distances

recorded in SFHC with both master devices. The differences in mean speed and acceleration between the master devices are affected by the task request: during HC users where slower using PL while significantly faster using SL. On the contrary, the maximal speed for SFHC was obtained with PL, showing how different tasks may require different types of master devices in order to be efficiently executed. No statistical difference was found between the arm stiffness at different levels of speed and acceleration, but the higher end-point stiffness values and variability that were seen for some experiments, suggest that, in those experiments, high stiffness is more likely to occur at lower speed and accelerations. On the other hand, it appears that, even though joint speed and acceleration are fundamental components in the joint stiffness computation, their relation with arm end-point stiffness can't be modeled with simple correlations.

A limitation of this work can be found, first of all, in the small number of subjects; the number of trials for each task may not have been enough to account for the high variability that characterizes human motor control. Another limitation lies in the described differences between the two tasks; the SFHC task, in fact, differs from HC not only for the necessity or re-orienting the tool end-effector, but also in terms of trajectory thickness and tool-tip dimension.

In conclusion, the results obtained suggest that the users tend to modulate the arm endpoint stiffness with respect to different

#### REFERENCES

Ajoudani, A., Fang, C., Tsagarakis, N. G., and Bicchi, A. (2015). "A reducedcomplexity description of arm endpoint stiffness with applications to teleimpedance control," in Intelligent Robots and Systems (IROS), 2015 IEEE/RSJ International Conference on (Hamburg: IEEE), 1017–1023.

tasks and interfaces and that this modulation is influenced by both the trajectory characteristics and the users' hand kinematics. The users were coping with the difference in task design and master device by adapting their arm stiffness modulation both in terms of central tendencies and variability. Although affected by some limitations, these findings prove that the arm dynamic proprieties are highly variable and that there could be significant benefits from the estimation of the kinetic proprieties of the users' arm during teleoperation. For instance, knowing how the users modulate their stiffness would allow to develop master devices able to match and to enhance this modulation, potentially reducing the energetic cost of the teleoperators while maintaining high precision. The results obtained could also be used to improve human-robot interactions during cooperative tasks, as in Beretta et al. (2015a,b).

#### 4.1. Future Developments

An interesting possible future development would be the inclusion in the research of a higher number of subjects as well as expert teleoperators, in order to study and compare the stiffness modulation strategies that they may have developed with expertise. To further understand which are the characteristics and parameters that play a fundamental role in the humanrobot interaction, it would also be advisable to study how stiffness relates to other dynamic parameters such as damping and inertia whose relation to human-robot system stability have been recently investigated (Dyck et al., 2013). These studies proved that the human arm behavior can change from passive to active based on the task performed and on the magnitude of the force perturbation applied.

#### AUTHOR CONTRIBUTIONS

JB developed the acquisition framework, system synchronization software, performed the experiments, analyzed data and wrote the paper; with ED, he conceived the methods and analysis. JJ helped in designing and performing the experiments, also writing the corresponding portion of the manuscript. GF and ED supervised, reviewed and approved the work.

#### ACKNOWLEDGMENTS

This project has received funding from the European Unions Horizon 2020 research and innovation programme under grant agreement No. H2020-ICT-2016-732515. The authors would like to thank Alexandra Stefanescova for the precious help with the Scaling and Inverse Kinematics.

Ajoudani, A., Tsagarakis, N., and Bicchi, A. (2012). Tele-impedance: teleoperation with impedance regulation using a body–machine interface. Int. J. Robot. Res. 31, 1642–1656. doi: 10.1177/0278364912464668

Beretta, E., De Momi, E., Rodriguez y Baena, F., and Ferrigno, G. (2015a). Adaptive hands-on control for reaching and targeting tasks in surgery. Int. J. Adv. Robot. Syst. 12:50. doi: 10.5772/60130


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2017 Buzzi, Ferrigno, Jansma and De Momi. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Generation of a Movement Scheme for Positive Training

Lin Liu<sup>1</sup> , Le Xie1, 2 \*, Yun-Yong Shi <sup>2</sup> and Bing-Chen An<sup>3</sup>

1 Institute of Forming Technology and Equipment, Shanghai Jiao Tong University, Shanghai, China, <sup>2</sup> School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai, China, <sup>3</sup> HuaDong Hospital Affiliated with Fudan University, Shanghai, China

Rehabilitation robots have been demonstrated to be an efficient tool in the field of rehabilitation training. Meanwhile, there are varieties of tasks designed for motion training. These tasks need to be transmitted to motion data for rehabilitation robots. In this paper, we designed a drinking task and captured the motion data as the ground truth, through sensors of an exoskeleton device named Neo-Arm. To verify the effectiveness of Neo-Arm, we used a Vicon system to capture the same motion task without Neo-Arm for comparison. Eight subjects participated in the experiment. The motion data of the drinking task, including the range of motion (ROM) and the velocity of each joint, are obtained. The result shows that the Neo-Arm can achieve the suitable precision and be fit for other kinds of upper limb motion tasks.

#### Edited by:

Wenwei Yu, Chiba University, Japan

#### Reviewed by:

Giuseppe D'Avenio, Istituto Superiore di Sanità, Italy Waldemar Karwowski, University of Central Florida, USA

> \*Correspondence: Le Xie lexie@sjtu.edu.cn

#### Specialty section:

This article was submitted to Neural Technology, a section of the journal Frontiers in Neuroscience

Received: 28 September 2016 Accepted: 14 February 2017 Published: 01 March 2017

#### Citation:

Liu L, Xie L, Shi Y-Y and An B-C (2017) Generation of a Movement Scheme for Positive Training. Front. Neurosci. 11:96. doi: 10.3389/fnins.2017.00096 Keywords: rehabilitation, training, upper limb, robot, trajectory

# INTRODUCTION

A stroke, also called a cerebral vascular accident (CVA), is one kind of acute disease in the blood vessels in the brain characterized by varying degrees of neurological deficit, which leads to series of sequelae and symptoms. Among all the symptoms, hemiplegia is the most common. However, it is quite difficult for hemiplegic patients to regain their motion function of the upper limb, which leads to a difficult life with a constant reliance on family assistance and medical care. Hence, a variety of rehabilitation robots have been developed to help hemiplegic patients receive better rehabilitation training.

Early studies on rehabilitation robots are mainly based on the design using end-effecter mechanisms. For example, Stanford university developed an upper limb rehabilitation robot called MIME (Lum et al., 2004, 2006), based on the PuMA550 robot. Meanwhile, MIT-MANUS (Krebs et al., 2004), GENTLE/S (Coote et al., 2008), and iPAM (Holt et al., 2007) were developed. Although, these robots can lead the arm to move in space with multiple degree of freedom (multi-DOF) movements, they are not capable of controlling each joint of the upper limb during each period of motion. To control the upper limb from each joint, a variety of rehabilitation robots were designed as exoskeleton mechanisms, such as T-WREX (Housman et al., 2007), Intelliarm (Park et al., 2008), MEDARM (Ball et al., 2007), RUPERT (He et al., 2005), Dampace (Stienen et al., 2009), SUEFUL-7 (Gopura et al., 2009), and Armin I, II, and III (Nef and Riener, 2005; Mihelj et al., 2007; Nef et al., 2009). Nevertheless, they are not suitable for capturing motion information from active movement. Because these robots are heavy for subjects, motors, and corresponding drive mechanisms add more weight to the robots. In this case, subjects have to use more strength and change how they move to offset the weight and the inertia of the mechanism.

To capture movement, optical measurement equipment has been widely used, which can record the trajectory of the marked point in the skin of the subject, such as Kinect (Chang et al., 2013; Uttarwar and Mishra, 2015) and Vicon (Chung et al., 2011; Henmi et al., 2014).

However, there is relative motion between the skin and inner bones during the upper limb movement, especially for internal/external rotation, which can lead to angular errors. Meanwhile, upper limb movement is complicated with multi-DOFs in 3-D space, which can lead to a blocking problem when optical equipment is used to capture movement trajectories.

The objective in this research is to capture healthy people's trajectory in a motion task using an upper limb exoskeleton device. The tasks was designed as a series of movements for drinking. Eight healthy subjects participated in the experiment. They were instructed to wear Neo-Arm to perform the drinking task and to perform the same task using the Vicon system for comparison. During the task, angular information was recorded by both Neo-Arm and the Vicon system. Finally, the 3D trajectory of the upper limb movement was obtained.

#### MATERIALS AND METHODS

# Device Description

#### Neo-Arm

A 7-DOF rehabilitation device for active training was developed in our earlier research (Yu et al., 2015), called the Neo-Arm device, as shown in **Figure 1**. The device is equipped with an angular displacement transducer at each DOF, which can capture the position information of real-time movement. Neo-Arm can be used to capture the 3D trajectory of the rehabilitation movement in positive training.

At each joint of Neo-Arm, an angular displacement sensor is equipped through the axis of the rotation, by which the angular information can be captured and transformed from electrical signals to angular data. Through the training system, data are saved as a file for the subject. The coordinate system of Neo-Arm is shown in **Figure 2**. Through the coordinate system and the data captured by sensors, we can calculate the 3D trajectory of the endpoint of the upper limb.

The Denavit–Hartenberg (D–H) matrix is adopted as Equation (1):

$$\begin{aligned} \;^0\_6T = \;^0\_1T\_2^1T\_3^2T\_4^3T\_5^4T\_6^5T = \begin{bmatrix} n\_x & o\_x & a\_x & p\_x\\ n\_y & o\_y & a\_y & p\_y\\ n\_z & o\_z & a\_z & p\_z\\ 0 & 0 & 0 & 1 \end{bmatrix}. \end{aligned} \tag{1}$$

where, { p<sup>x</sup> p<sup>y</sup> p<sup>z</sup> } T is the location of the endpoint of the robot.

**Table 1** is also needed as the demand of D–H matrix to calculate the endpoint position of the robot in 3D space.

In **Table 1**, the values of d<sup>i</sup> and ai−<sup>1</sup> are the length parameters of Neo-Arm. The length parameters for upper limb and Neo-Arm are shown in **Table 2**.

In **Table 1**, the values of θi are the angle parameters of the DOF of the human upper limb. The values of J<sup>i</sup> can be captured by an

angular displacement sensor. The angle parameters for the upper limb and Neo-Arm are shown in **Table 3**.

#### Vicon System

The optional motion capture system used in the experiment is a Vicon system (Oxford Metrics Ltd., Oxford, UK), in Ruijin Hospital, Shanghai Institute of Orthopedics and Traumatology, Shanghai, China, see **Figure 3A**. It is composed of 10 cameras and an MX13 motion capture system. The body of the subject is marked with 39 markers (14 mm in diameter), which are used to record trajectories in a sampling frequency of 100 Hz. The motion data are processed in Vicon polygon software, see **Figure 3B**.

#### Subjects

Eight male subjects were selected to participate in the experiment. This study was carried out in accordance with the recommendations of Ethics Committee at Shanghai Ninth People's Hospital affiliated to Shanghai Jiao Tong University School of Medicine (No. 2013011). All subjects provided written informed consent prior to participation. Subjects are similar in age and sitting height, as well as their upper limb dimensions, as described in **Table 4**.

Meanwhile, the dimensions of the Neo-Arm device can be adjusted to accommodate the distinctive dimensions of the upper limb for different subjects, by which means each joint of the whole mechanism can be correlated with the corresponding joint of the subject's limb joint. The dimension information can be

#### TABLE 1 | D-H table.



saved as a record. The basic parameters of the device are shown in **Table 5**.

#### Task Design Requirement

The activities of daily living (ADL) include various self-care activities people perform in everyday lives, such as bathing, feeding, drinking, and dressing. Since the assessment of ADL has

#### TABLE 3 | Angle parameters of the upper limb and Neo-Arm.


played an important role in rehabilitation training, it is significant that in this research subjects should follow the mode of motion in ADL. Given that drinking is a basic movement performed repeatedly in everyday life, drinking was proposed and discussed as a task in this paper.

To simulate drinking in daily life, the task was designed based on three principles. First, each time subjects should perform a different movement. Second, the movement that subjects perform should be a motion combination including picking up a cup, fetching water, and drinking. Third, the ROM of the device should be within the ROM demanded in the task. Different movement trajectories combined with motion combinations can avoid the simple advance and return movement in rehabilitation training. In this way, subjects are led to finish the task by actively thinking.

#### TABLE 4 | Subjects.


#### TABLE 5 | Neo-Arm device parameters.


# Equipment Layout

According to the first principle, three cup points are set up for P2, which leads to different movement trajectories during A2 and A3 actions. Three cup points were set up, as shown in **Figure 4**, including the left point (Point L), right point (Point R), and bottom point (Point B). The locations of the chair, the desk, the bottle, and J1 DOF are also fixed.

In the experiment, movement parameters for different cup points were recorded. At first, subjects were instructed to perform the task for one cup point without any practice. After practicing, they were instructed to perform it at the same point again. This point was chosen as the left point. Then, subjects proceeded to complete the task with random cup points assigned by the researcher.

#### Task Description

According to the second and the third principles, a series of movements for drinking was proposed in the experiment. The subject had to use two hands coordinating with each other. Neo-Arm I was worn on the left upper limb to perform a series of movements, including five actions and five hovering postures,

as shown in **Figure 5**: zero position posture (P0), ready action (A1), ready posture (P1), fetching cup action (A2), holding cup posture (P2), getting water action (A3), getting water posture (P3), drinking action (A4), drinking posture (P4), homing action (A5), and homing posture (P5).

The first posture (P0) is zero position posture, as shown in **Figure 5A**, which is a fixed initial posture when the subjects put on the device in the task. P1 is the position when subjects get ready to fetch the cup, which is shown in **Figure 5B**. A1 is the action in which the upper limb is moved from P0 to P1. A2 is the action in which subjects move the upper limb to fetch the cup. Although, it is essential that the subject completes the action of picking up the cup in real-world situations, hand function training is not the focus of this paper. Therefore, the catching action is simplified as moving the handle of the device and reaching the cup point. The posture where the upper limb hovers at the cup point is P2, as shown in **Figure 5C**. Then, the subject should move the left upper limb from the cup point to the bottle through A3 action. At P3, although the right upper limb is not tested in this research, the subject has to use his/her right hand to press the button to coordinate with the motion of the left upper limb, shown as in **Figure 5D**. After that, the subject should move the handle from the bottle to the side of his/her mouth and perform the drinking movement through the A4 action. Meanwhile, the right side of the upper limb must help to hold the cup. Finally, the upper limb should be moved to the ready posture.

# RESULTS

# ROM

According to the different periods of the whole movement, the movement in the task was divided into five actions and six hovering postures. Since the reaching cup motion, the getting water motion, and the drinking motion play important roles in

P1; A2 action is the motion from P1 to P2; A3 action is the motion from P2 to P3; A4 action is the motion from P4; A5 action is the motion from P4 to P5.

the task, the angle of each DOF from all subjects at the P2, P3, P4, and P5 postures are listed in **Figures 6**–**10**.

The data for P2 at the L point are shown in **Figure 6**. The mean angles of J1 from Neo-arm and Vicon are −66.28 ± 1.42 and −63.26 ± 1.95 mm (p = 0.038); the mean angles of J2 are 32.96 ± 0.49, 28.50 ± 1.31 mm(p = 0.046); the mean angles of J3 are −25.14 ± 0.27, −28.48 ± 1.28 mm(p = 0.031); the mean angles of J4 are 47.84 ± 2.74, 52.65 ± 2.36 mm(p = 0.042); the mean angles of J5 are 6.21 ± 1.03, 10.56 ± 3.48 mm(p = 0.11); the mean angles of J6 are 27.78 ± 2.4, 22.65 ± 4.65 mm(p = 0.12).

The data for P2 at B point are shown in **Figure 7**. The mean angles of J1 from Neo-arm and Vicon are −84.86 ± 4.8, −80.26 ± 5.31 mm (p = 0.062), the mean angles of J2 are 57.16 ± 6.1, 54.19 ± 7.49 mm (p = 0.053), the mean angles of J3 are −14.55 ± 1.46, −12.33 ± 1.66 mm (p = 0.037), the mean angles of J4 are 83.89 ± 1.39, 79.67 ± 1.88 mm (p = 0.046), the mean angles of J5 are 21.06 ± 3.1, 18.35 ± 5.46 mm (p = 0.032), the mean angles of J6 are 22.04 ± 3.16, 18.61 ± 7.62 mm (p = 0.067).

The data for P2 at the R point are shown in **Figure 8**. The mean angles of J1 from Neo-arm and Vicon are −50.86 ± 4.52, −45.64 ± 3.88 mm (p = 0.055), the mean angles of J2 are 27.26

± 2.78, 25.37 ± 3.37 mm (p = 0.036), the mean angles of J3 are −8.86 ± 3.29, −10.50 ± 5.43 mm (p = 0.044), the mean angles of J4 are 58.30 ± 3.11, 55.16 ± 2.34 mm (p = 0.036), the mean angles of J5 are 3.37 ± 3.70, 5.35 ± 5.86 mm (p = 0.067), the mean angles of J6 are 28.16 ± 6.03, 31.00 ± 3.44 mm (p = 0.072).

The data for P3 posture are shown in **Figure 9**. The mean angles of J1 from Neo-arm and Vicon are 24.36 ± 3.17, 22.67 ± 3.03 mm (p = 0.028), the mean angles of J2 are 11.91 ± 4.86,

Bottom task.

13.38 ± 6.71 mm (p = 0.043), the mean angles of J3 are 1.07 ± 3.52, 2.55 ± 2. 88 mm (p = 0.035), the mean angles of J4 are 21.63 ± 2.96, 19.57 ± 1.97 mm (p = 0.038), the mean angles of J5 are −42.28 ± 4.69, −35.38 ± 5.68 mm (p = 0.24), the mean angles of J6 are −85.20 ± 3.37, −80.36 ± 8.35 mm (p = 0.14).

The data for P3 are shown in **Figure 10**. The mean angles of J1 from Neo-arm and Vicon are −19.20 ± 2.28, −22.13 ± 2.18 mm (p = 0.041), the mean angles of J2 are 34.84 ± 4.34, −22.13 ± 2.18 mm (p = 0.032), the mean angles of J3 are −46.32 ± 7.35, −40.35 ± 9.3 mm (p = 0.058), the mean angles of J4 are 123.28 ± 3.73, 120.89 ± 1.37 mm (p = 0.044), the mean angles of J5 are −44.39 ± 3.62, −48.32 ± 7.86 mm (p = 0.13), the mean angles of J6 are 25.56 ± 3.28, 22.13 ± 8.97 mm (p = 0.17).

Considering the differences among subjects, the average and the median angles of each DOF are listed in **Table 6**.

#### Velocity

In the experiment, during each action, the average angular velocities are listed in **Table 7**. During the A2 and A3 periods, since different cup points lead to different actions, they are listed with v\_L, v\_R, and v\_B, which represent the velocity of different actions for A2 and A3.

#### Measured ROM

**Figure 11** shows the motion of J1 from eight subjects in the Point Left task. J1 represents the shoulder abduction/adduction. To compare all curves, the suspending period has been adjusted within 2 s. Besides, the other five DOFs were solved with the same manner as the J1 DOF.

#### Movement Scheme

Since the average angular and the average velocity have been obtained, the movement scheme is shown in **Figure 12**. It is shown that the angle of each joint, which is displayed as a distinctive color, varies with time. Meanwhile, since the velocity of the endpoint cannot reflect the whole movement comprehensively, the velocity scheme is proposed, as represented by the gradient of each line.

#### 3D trajectory

According to the movement scheme, the 3D trajectory is shown in **Figures 13**, **14**. Considering that the movement of the wrist would reflect the whole movement trajectory with interference, the intersection of the forearm pronation/supination axis and wrist flexion/extension axis was chosen as the endpoint. The y-z plane represents the horizontal plane. The zero point represents the ending point of the upper limb when it is at the zero posture position, the cup point represents the position where the cup is to be placed, the bottle point represents the position where the water bottle is to be placed. The segment of line 1 represents the movement of reaching for the cup, the segment of line 2 represent the movement of moving the cup to the water bottle, and the segment of line 3 represents the movement of drinking.

## DISCUSSION

#### Comparison between Neo-Arm and Vicon

When the hand reached the P2 posture for the cup and P3 for the bottle, the marks on the wrist and the hand were blocked by the cup and the bottle. In this situation, the marks in the Vicon software are lost in these zones, see **Figure 15A**. To generate a continuous trajectory, the Vicon polygon software simulated the motions of the lost mark based on the trajectory captured previously and afterward. Meanwhile, when one mark is blocked, the Vicon polygon system can still calculate the results from the motion data of adjacent marks. For example, in the two simulated trajectories, J6 DOF varied in two ways, see **Figure 15B**. According to the simulation, both ways are reasonable. However, during this process, the simulated values of the angles varied for different trajectories, which resulted in a larger standard deviation and p > 0.05 for three DOF of the wrist in the Vicon system than in Neo-Arm.

#### Motion Coordination and ROM for the Point Left Task

According to **Figure 12**, there are similar patterns in three points tasks. Especially in the shoulder joints and the elbow, four DOFs change with time in a similar pattern.



TABLE 7 | Velocity of each joint during each motion.

During the A2 period, in the first half, J1 rotated from the zero position posture to the left side, with a maximum of −66.57◦ on average, while J3 started to increase from 0◦ at a low velocity. In the second half of A2, the angle displacement of J3 increased at a high velocity, while J4 rotated from the maximum to the minimum. The changes in this period mean that when subjects moved their left arm to reach the cup, which was placed on the left side, they were inclined to perform shoulder abduction first, accompanied by slight shoulder external rotation synchronously. After that, subjects tried to press their elbow by the way of shoulder extension, while the elbow extended at a maximum angle to open their arm. However, the motion of J5 exhibited two patterns. Subjects tended to pronate the forearm to 4.3◦ on average. By this way, the wrist is adjusted to perform radial deviation, so the subjects could move J5 in the tangential direction of the space trajectory.

During A3, the angular variation of J2 and J3 led the arm move in advance. The two joints started to rotated back to ∼0 ◦ . During the second half of this process, J1 started to increase to a maximum angle of 23.82◦ on average, while J4 rotated to 22.12◦ on average. J5 moved to an angle of −42.16 on average. To perform this action, subjects first lifted their elbow using shoulder flexion and internal rotation. Before they finished the lifting, the shoulder began to adduct, accompanied by elbow flexion. All these variations stopped simultaneously when the hand reached the water bottle position. The forearm and the wrist remained still in this process.

During A4, J1 decreased at a high velocity while J4 increased at a high velocity. In this process, J3, J5, and J6 increased slightly. In this process, the elbow flexed while the shoulder abducted. By this movement, subjects moved the hand to the mouth position. At the same time, the forearm pronated to maintain a drinking gesture.

# Comparison between Two Patterns of variations of velocity

During A2 and A3, two major modes of angular velocity curves were presented for J1, referred to as the M-curve and N-curve, as shown in **Figure 16**. The major difference between these two types of curves is that there are two wave crests in the M-curve,

between which there is a wave trough. Meanwhile there is only one wave crest in the N-curve.

In the experiment, this difference involves three kinds of conditions. The first is whether the subjects have practiced with the device and the task. When subjects performed the fixed cup point task without any practice, the angular velocity varied in the M-curve mode. Meanwhile, the angular velocity varied in the N-curve mode after subjects experienced the practice. The

reason for this phenomenon is that the practice made subjects more skillful in the task. The more skilled the subjects became, the more smoothly they moved.

Second, even after practice, when the testing mode was changed from a fixed cup point to random cup points, the angular velocity curve appeared to be in the M-curve mode. This is

FIGURE 15 | (A) The lost points of J6 DOF in the movement before simulation from Vicon, (B) the two results of the simulation motion data of J6 from Vicon.

because compared with the fixed cup point task, mechanical repetition was avoided. And the more familiar they became with the task, the less their conscious thought process was involved. However, when performing the random cup point task, subjects had to focus on the task goal and made corresponding movement adjustments. As such, the random cup point task motivated their conscious thought process more than the fixed cup point.

Third, this difference occurred obviously in J1 and J4 during A2 and A3. This is because horizontal movements played a main role in the two actions. The range of abduction and adduction of J1 for the shoulder was large. Other movements of joints were also accompanied during the same period, which lead to a reduction in the movement smoothness.

#### Comparison between the Three Points

The functional movement in daily life is not limited in fixed posture and motion. In order to provide a task with different range of motion (ROM), we proposed three cup points in this research. Through the three points, the subject need to move their upper limb in different ROM. At each point, the posture of the upper limb is distinctive. In the task, according to the different points, it is shown that the angle of each DOF is distinctive at each posture, see **Figures 6**–**10**.

Meanwhile, when the task is set with one cup point, the subject can be familiar with the task soon, and form a mechanical movement, without the active consciousness. But in the task of random points mode, the velocity of DOF varied in a M-curve type, see **Figure 16**. It is considered that this velocity variation mode is caused by the active consciousness from the subject. So, the three points task is one solution for the upper limb active training. In this case, the three points movement scheme should be obtained from the research, see **Figure 12**.

#### Movement Scheme

For Point Left task, the time length of the period of hovering is set at two seconds. During this time interval, all the joints should be kept still. This period can be prolonged as required. But it is still necessary that each interval last for at least 2 s. This time period is set for the subject to adjust his/her movement. Since A1 is for reparation and A5 is for backing, which includes simple rotation for DOF, it is detailed discussed on A2, A3, and A4 as below.

In the scheme, A2 lasts for 2 s. The objective is to open the upper limb to reach a cup placed on the left side. This action is designed in two parts. For 4–6 s, the shoulder extends slightly at a low velocity. The shoulder abduction is accomplished within the first 1 s, and the forearm pronation is accomplished within the first 0.5 s. This forearm pronation is needed to adjust the wrist at an angle by which the fist can move along the tangential direction of the space trajectory. In this way, the forearm can move more easily and fluently. The elbow merely extends slightly, while the forearm follows the shoulder extension. For 6–8 s, the shoulder abduction stops at the maximum angle. The elbow is pressed down as a result of shoulder external rotation and extension, which is accompanied by wrist radial deviation. At the same time, the elbow extends with vast angular displacement at a high velocity. It is obvious that during the A2 period, shoulder external

rotation and elbow extension occur near the end of shoulder abduction.

A3 lasts for 2 s. First, for 8–10 s, the elbow moves up with the assistance of the gravity compensation mechanism by means of shoulder flexion and internal rotation. Meanwhile, the elbow has slight flexion. Then, for 9–10 s, the shoulder accomplishes the adduction swiftly while the elbow finishes the remaining part of the flexion. A4 lasts for 12–14 s. During this period, the shoulder abducts greatly and the elbow flexes to move the hand to the mouth position to maintain a drinking gesture. The forearm pronates at an angle to perform the gesture of maintaining the cup.

This movement scheme can be stored as a training task on a computer and applied for upper limb rehabilitation training with exoskeleton robots.

#### CONCLUSION

In the research, subjects were instructed to move the left upper limb from the center at the front of the chest to the left side, and then from the left side to the right side, and then back to the body. During these movements, subjects were inclined to adjust the upper arm to lead the distal joints when the task was to move the upper limb from a proximal to a distal location. Meanwhile, subjects were inclined to adjust the distal joints when the task was to move the upper limb from a distal to a proximal location.

Motion data were obtained using two systems, Neo-Arm and Vicon. The motion of each DOF of the upper arm was directly

#### REFERENCES


captured by Neo-Arm, while with Vicon, it was calculated by the trajectories of reflective marks. The results indicate that the Neo-Arm can achieve the suitable precision and be fit for other kinds of upper limb motion tasks. The motion data of three points in task are captured and stored as a scheme in the task library on a computer.

#### AUTHOR CONTRIBUTIONS

LL: main author who have did this research and wrote the manuscript. LX: the doctoral supervisor of the LL and YS, who proposed the research and direct us in the experiment. YS: assist in the experiment and part of work in data processing. BA: proposed the opinions from the medicine, and proposed the control experiment.

#### ACKNOWLEDGMENTS

The work described in this paper was supported by The project of National Natural Science Foundation of China (No. 61672341), The Major Program of National Natural Science Foundation of China (No. 61190124, 61190120), National High Technology Research and Development Program of China (No. 2015AA043203), The project of the Science and Technology Commission of Shanghai Municipality (14441900800, 14DZ1941100, 14DZ1941103, 15DZ1942103), National Key Technology Support Program (No. 2009BAI71B06).


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2017 Liu, Xie, Shi and An. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Virtual Normalization of Physical Impairment: A Pilot Study to Evaluate Motor Learning in Presence of Physical Impairment

#### Christopher Jarrett\* and Andrew McDaid

*Department of Mechanical Engineering, Faculty of Engineering, University of Auckland, Auckland, New Zealand*

Motor learning is a critical component of the rehabilitation process; however, it can be difficult to separate the fundamental causes of a learning deficit when physical impairment is a confounding factor. In this paper, a new technique is proposed to augment the residual ability of physically impaired patients with a robotic rehabilitation exoskeleton, such that motor learning can be studied independently of physical impairment. The proposed technique augments the velocity of an on-screen cursor relative to the restricted physical motion. Radial Basis Functions (RBFs) are used to both model velocity and derive a function to scale velocity as a function of workspace position. Two variations of the algorithm are presented for comparison. In a cross-over pilot study, healthy participants were recruited and subjected to a simulated impairment to constrain their motion, imposed by the cable-driven wrist exoskeleton. Participants then completed a sinusoidal tracking task, in which the algorithms were statistically shown to augment the cursor velocity in the constrained state such that it matched position-dependent velocities recorded in the healthy state. A kinematic task was then designed as a motor-learning case study where the algorithms were statistically shown to allow participants to achieve the same performance when their motion was constrained as when unconstrained. The results of the pilot study provide motivation for further research into the use of this technique, thus providing a tool with which motor-learning can be studied in neurologically impaired populations. This could be used to give physiotherapists greater insight into underlying causes of motor learning deficits, consequently facilitating and enhancing subject-specific therapy regimes.

Keywords: virtual normalization, biomechatronics, robotic assessment, motor learning, rehabilitation robotics

## INTRODUCTION

A major neuroscience and clinical question is how best to invoke motor learning, a critical component of the rehabilitation process (Krakauer, 2006). It is difficult to completely understand why neurologically impaired patients cannot learn to perform certain tasks, as there are many underlying factors that contribute to motor learning deficits. Often the presence of physical impairment, such as issues with strength or range of motion (ROM), can confound results. This can make it unclear whether the inability to perform aspects of a task is physical, a neurological motor learning deficit or a combination of both.

#### Edited by:

*Wenwei Yu, Chiba University, Japan*

#### Reviewed by:

*Giancarlo Ferrigno, Polytechnic University of Milan, Italy Waldemar Karwowski, University of Central Florida, USA*

> \*Correspondence: *Christopher Jarrett cjar160@aucklanduni.ac.nz*

#### Specialty section:

*This article was submitted to Neural Technology, a section of the journal Frontiers in Neuroscience*

Received: *22 November 2016* Accepted: *15 February 2017* Published: *14 March 2017*

#### Citation:

*Jarrett C and McDaid A (2017) Virtual Normalization of Physical Impairment: A Pilot Study to Evaluate Motor Learning in Presence of Physical Impairment. Front. Neurosci. 11:101. doi: 10.3389/fnins.2017.00101*

This is illustrated in multiple domains. One example is the administration of botulinum toxin type A (BoNT-A), commonly used to reduce spasticity in children with cerebral palsy (Boyd and Graham, 1999; Wissel et al., 1999; Wallen et al., 2004; Fattal-Valevski et al., 2008, 2010; Molenaers et al., 2009, 2010; Tedroff et al., 2009). BoNT-A addresses issues due to neural spasticity and in cases where patients exhibit motion impairment after injections, it can be unclear whether this is a result of dosage, issues with the structural properties of muscle, or a fundamental incapacity for neural plasticity and motor learning. Similarly confounding issues have been reported in other studies, alluding to the uncertainty of drawing conclusions when motor learning appears poor in children with CP (van Abswoude et al., 2015). Self-exploration and problem solving are critical to the motor learning process, but these can be impacted by motor difficulties, preventing optimal learning (Valvano and Rapport, 2006).

The emergence of bio-mechatronics systems, specifically rehabilitation robotics, has provided researchers with a new suite of tools for interacting with humans and hence the ability to investigate the fundamental mechanisms of rehabilitation and motor learning (Boudreau et al., 2010; Turner et al., 2013). Robots have served as a useful research platform, igniting debate about the most efficient way to stimulate motor learning. Some authors promote the well-developed "assist-as-needed" (AAN) paradigm (Jezernik et al., 2004) while others suggest that erroraugmentation (EA) is more effective (Patton et al., 2006). The most recent evidence suggests that the answer depends on a range of factors, from frequency of feedback to initial skill level (Wulf et al., 2010; Burtner et al., 2014; Marchal-Crespo et al., 2015; Fujii et al., 2016). In other words, the potential for motor learning is largely subject-dependent. Moreover, recent reviews indicate that a gap exists between current technology and true clinical needs (Cordella et al., 2016).

In order to optimize rehabilitation a therapist must better comprehend the fundamental subject-specific causes of motion deficit (Lipp and Tomassini, 2015). Recent studies have investigated the separation of neural components of torque such as spasticity, from non-neural components such as muscular stiffness (Bar-On et al., 2014, 2015; Lin et al., 2016). While this provides a valuable analysis of motion, and enables the effectiveness of various treatments to be analyzed, in these studies the cognitive ability to motor learn was not considered as a component of observed motion. Given the importance of motor learning in rehabilitation and to provide further insight into the causes of motion issues, it would be useful to evaluate motor learning independently of confounding physical impairment.

Melendez-Calderon et al. (2015) conducted a study which demonstrated motor learning could be achieved without physical motion. This built on studies focused on adaptation to environments after virtual training with visual feedback (Sarlegna et al., 2010; Melendez-Calderon et al., 2011; Rotella et al., 2014). In the case of physical impairment force production might be limited, which could affect both virtual exploration and conclusions regarding motor learning. Participants' motion could also be scaled in a purely kinematic sense and with no regard to dynamic models, as was previously used to investigate transfer of knowledge between scaling conditions (Ojakangas and Ebner, 1991; Smiley-Oyen et al., 2003; Paz et al., 2005). To the best of our knowledge no conclusive method has been developed to specifically study motor learning independently of physical impairment.

In this study, we ask whether it is possible to use a robotic exoskeleton as a tool to evaluate the motor learning capacity of an impaired individual by normalizing their physical abilities in a virtual environment and investigating their ability to learn a task in this environment. The robotic exoskeleton rehabilitation device is used to simulate a physical impairment on healthy participants and visually augment the residual motion on a display; thus "normalizing" the "impairment". The "impairment" is manifested as a set of constraints on motion. A case study is presented, where we hypothesize that the virtual normalization of participants' abilities can be used to evaluate the cognitive ability to motor learn irrespective of physical impairment. The exoskeleton will therefore be used as a tool to diagnose sources of motion deficit and evaluate the motor learning capacity of impaired individuals, serving as a valuable aid for clinicians to create optimized, subject-specific therapy routines. This also has potential to lead to enhanced and harmonized human-robotic interaction for people with impairment. Moreover, it provides a platform for further research into motor learning in the presence of impairment.

# MATERIALS AND METHODS

System Architecture and Model of Learning The architecture of the entire human-exoskeleton system proposed in this paper is described in **Figure 1**, with the human

represented by the blocks encompassed by the dashed lines. A movement is planned, resulting a motor command which is translated into a desired Cartesian coordinate, x<sup>d</sup> which is converted via inverse kinematics to a desired joint angle(s), <sup>θ</sup> <sup>d</sup> . The Neuromuscular System block encompasses the neuromuscular dynamics of the human, the output of which is a wrist torque, <sup>τ</sup><sup>w</sup> . This human torque along with the external torque exerted on the wrist by the robotic exoskeleton, <sup>τ</sup> ex, determine the wrist kinematics. The human and robot torques are fed back internally to inform the human's motor planning. The exoskeleton torque is comprised of the resistive torque due to the constraints imposed by the simulated impairment, while the Neuromuscular System block encompasses any real impairment, which then affects the generated wrist torque.

As the wrist is coupled to the exoskeleton, the angle of the exoskeleton is equal to the angle of the wrist, <sup>θ</sup><sup>w</sup> . This kinematic output, <sup>θ</sup><sup>w</sup> , is converted into Cartesian coordinates, xw, then passed through the normalization (which is unity in the healthy state) to be presented visually on screen as feedback (in the form of the cursor) to the participant, x<sup>c</sup> . The behavior of the dynamic target, x<sup>t</sup> , is related to the normalized wrist kinematics by a simulated model, in this case a viscous damper. The target and cursor locations are viewed by the human to inform motor planning for the subsequent motion. This has been intentionally modeled as a black-box because, while it is understood the relationship between wrist movement and corresponding target movement can be learnt, literature

documents multiple mechanisms by which learning occurs (Haith and Krakauer, 2013) and this is not of direct relevance to this work.

## Participants

A convenience sample of 10 participants, 8 male and 2 female, with no cognitive or physical impairments were recruited for the study. The study was approved under the guidelines of the University of Auckland Human Participants Ethics Committee (UAHPEC), reference number #013729 and written and informed consent was obtained from all participants in accordance with these guidelines. Study size was considered sufficient, given the similarity of the results in the healthy population and exploratory nature of the study, where the aim was to technically and experimentally prove the concept of virtual normalization. All participants were coincidentally dominant in the right hand.

## Exoskeleton Device

The 3 degree-of-freedom (DOF) AW-TEX robotic wrist exoskeleton was used in this study (McDaid, 2015). Flexion/extension (FE) and radial/ulnar deviation (RU) are actively controlled. The third DOF is passive, designed to accommodate forearms of differing lengths. The exoskeleton is shown in **Figure 2**.

Series Elastic Actuators (SEAs) are used at the active joints and are driven by remotely located motors with Bowden cables. An inner controller based on sliding mode control regulates the relative displacement of the SEA components, thus controlling the torque in an impedance-based architecture (Jarrett and Mc Daid, 2017).

## Implementation of Simulated Impairment

The present study is performed with healthy participants with a simulated CP-like impairment so that we can use each person as their own control in a cross-over study. CP describes a group of disorders with a variety of phenotypic manifestations and studies have varied in their attempts to describe the "average" kinematics of people with cerebral palsy (Barroso et al., 2011; Klingels et al., 2012). In this study, the impairment is modeled with two components: (i) a simulated contraction, implemented by a linear torque spring restricting motion in the extension direction from 5◦ ; (ii) a velocity-dependent torque, simulating restricted torque production and applied throughout the workspace. The simulated impairment is manifested as a set of "constraints," and is mathematically described in Equations (1) and (2).

$$\tau\_{\acute{e}e} = \begin{cases} -\text{sgn}\left(\dot{\theta}\_{\acute{f}e}\right) \left[K\_{\mathbb{P}}\left(\theta\_{\acute{f}e} - 5\right) + K\_{d}\dot{\theta}\_{\acute{f}e} + K\_{\mathbb{V}}\dot{\theta}\_{\acute{f}e}\right], & \theta\_{\acute{f}e} > 5^{o} \\\ -\text{sgn}\left(\dot{\theta}\_{\acute{f}e}\right) \left[K\_{\mathbb{V}}\dot{\theta}\_{\acute{f}e}\right], & \theta\_{\acute{f}e} < 5^{o} \end{cases} \tag{1}$$
 
$$\tau\_{ru} = -\text{sgn}\left(\dot{\theta}\_{ru}\right) \left[K\_{\mathbb{V}}\dot{\theta}\_{ru}\right], \; \forall \; \theta\_{ru} \tag{2}$$

where τfe and τru are the torques applied to the wrist in the flexion/extension and radial/ulnar deviation axes, θfe and θru are the flexion/extension and radial/ulnar deviation angles of the wrist, θ˙ fe and <sup>θ</sup>˙ ru are the angular velocities of the wrist in the flexion/extension and radial/ulnar deviation axes, K<sup>p</sup> and K<sup>d</sup> are the stiffness and damping of the wall that restricts the ROM; set to 2Nmdeg−<sup>1</sup> and 0.1Nmsdeg−<sup>1</sup> respectively, K<sup>v</sup> is the gain applied to implement the velocity-based resistance; set to 0.02Nmsdeg−<sup>1</sup> and sgn is the sign function. Operation of the exoskeleton in this mode is hereafter referred to as the "constrained" state.

#### Impairment Assessment

A calibration routine was developed to evaluate the taskindependent physical ability of patients in the healthy (exoskeleton in backdriven mode) and constrained states. This extracted three subject-specific measurements:


(3) Position-dependent velocity, this was recorded by asking participants to perform eight blocks of five linear movements, alternating between the flexion/extension and radial/ulnar deviation axes, (a total of four blocks per axis). **Figure 3** illustrates the visual representation of the workspace shown to participants, displayed as nodes placed at 10◦ intervals in the workspace measured by the preceding test. They were the locations at which average velocities were measured. One minute rest periods were enforced after four blocks of movement to eliminate effects of fatigue. The routine was completed twice: first in the healthy state, then with the constraints applied and separated by a 2 min break.

#### Kinematic Normalization

To normalize the ability of the participants in their constrained states, two steps were required. The first is to shift the position of the cursor such that the constrained ROM centers on (0, 0). The second step is to apply a position-dependent scaling factor to the velocity of the cursor such that it matches the position-dependent velocity recorded in the healthy state. This was achieved with the aid of Gaussian Radial Basis Functions (GRBFs). GRBFs are advantageous due to their ability to describe any continuous function; and have been used in other robotic rehabilitation research to model patient torque as a function of workspace position (Wolbrecht et al., 2008; Oboe and Pilastro, 2014; Pehlivan et al., 2015). The nodes at which velocities were recorded during the calibration phase served as the locations of the RBFs.

The first step was to compress and shift the node coordinates in the healthy workspace into the constrained workspace, illustrated by the green plot in **Figure 4**. The compressed healthy RBF was then evaluated at the nodes describing the constrained workspace. This allows the calculation of the ratio of healthy to constrained velocity at each of the constrained velocity nodes, as described by Equation (3).

$$SF\_n = \frac{\nu\_{hn}}{\nu\_{in}}\tag{3}$$

where SF<sup>n</sup> is the scaling factor at the n th constrained node, vhn is the estimated healthy velocity at the n th constrained node

FIGURE 3 | Presentation of nodes in FE workspace during velocity calibration.

and vin is the constrained velocity at the n th constrained node. This creates a discrete set of scaling factors which are used to create a second RBF that normalizes velocity in the constrained workspace. The RBF allows the algorithm to capture velocity characteristics that vary with wrist position.

Equation (4) describes the RBF that was used to derive the scaling factor:

$$\text{SF}\_{\text{j}}\left(\theta\_{\text{j}}\right) = \text{Ya} + \text{w} \tag{4}$$

where SFj(θj) is the scaling factor as a function of motion at joint j, θ<sup>j</sup> , Y is the regressor matrix, composed of Gaussian basis functions as illustrated in Equation (5), a is a set of weightings applied to each basis function and w is a linear bias term applied to the output layer, calculated by multiplying an additional weighting coefficient by the current position.

$$Y = \begin{bmatrix} \mathbb{g}\_1 & 0\\ 0 & \mathbb{g}\_n \end{bmatrix} \tag{5}$$

where n is equivalent to the number of nodes that were obtained in the ROM test and g<sup>n</sup> refers to the Gaussian radial basis function, defined in Equation (6).

$$g\_n = e^{-\frac{d\_n^2}{2\sigma^2}}\tag{6}$$

where e is the exponential function, d<sup>n</sup> is the distance of the cursor from the nth node and σ is the width of each basis function; defined as the average distance between nodes.

Equation (4) contains a linear bias term, the purpose of which is to reproduce global behavior of the RBF function (Li and Verma, 2015). In previous work in rehabilitation robotics involving RBFs, the linear bias term was omitted (Wolbrecht et al., 2008; Oboe and Pilastro, 2014; Pehlivan et al., 2015). In this research, we set the value of w to 0 in the RU axis, to mirror the previous studies and investigate if the linear bias term can significantly improve the normalization algorithm.

An example of the resulting scaling factor is illustrated in **Figure 5** for both axes.

#### Validation of Normalization Algorithms

To validate the normalization, participants were subjected to the constraints with their cursor velocity normalized by the subjectspecific scaling factor RBF. They were asked to try to track the moving target, which was a recording of their movement during the healthy calibration. The velocity of the cursor was recorded for comparison with the velocity of the healthy movement. Four movement blocks were performed, in the same structure as the calibration routine.

## Case Study: Motor Learning Dynamic Cursor Model

To evaluate the use of the algorithm in conducting kinematic motor learning experiments a task where the target has a massdamper relationship with the cursor is implemented. The aim was to prove that when their constrained velocity was normalized, participants could perform the task to the same standard with the same amount of learning as they did in the healthy state. Eight participants, six males, and two females, continued onto this phase of the study.

#### Study Design

The participants were randomly separated into two groups, to conduct a cross-over study. Each group was tested for performance in the task across two sessions; each conducted in one of the healthy and constrained states. Each session consisted of a familiarization phase, followed by three blocks of movement in an acquisition phase, details below. Group A conducted the first session in the healthy state, followed by the constrained state 1 week later and vice versa for Group B. The design is illustrated in **Figure 6**. In the constrained state, participants' residual motion was augmented to allow them to reach the same velocity in task space as they could under "healthy" states. The 1 week break between experimental states served as a washout period, similar to comparable motor learning crossover studies (Abdollahi et al., 2014).

#### Task Description

Participants were seated in front of a computer screen, upon which their wrist position was represented by a blue cursor and a target was illustrated in green, **Figure 7**. During task execution, the yellow origin was always visible. The goal of the task was to reach the target, with "success" occurring when participants had been within a 2◦ radius for 750 ms. Only one target appeared at a time. Targets were initially placed at a distance equal to 20% of the smallest ROM measurement across both axes. One participant's targets were placed at 25% of this distance, due to a small ROM (≈10◦ in the radial direction) which would have placed the initial location of the targets within the 2◦ threshold for target achievement. To make the task non-trivial and require some level of adaptation, the targets were programmed to be virtually connected to the participants' movement as a function of wrist velocity, according to Equation (5):

$$
\ddot{\mathbf{x}}\_t = \frac{c\_t \ (\dot{\mathbf{x}}\_t - \dot{\mathbf{x}}\_t)}{m\_t} \tag{7}
$$

where x¨<sup>t</sup> is the acceleration of the virtual target, m<sup>t</sup> is the virtual mass of the target, c<sup>t</sup> is the damping coefficient, x˙<sup>c</sup> is the velocity of the cursor and x˙<sup>t</sup> is the velocity of the target. The movement of the target is thus related to the kinematics of the participants' wrists and the difficulty can be varied. Based on pilot trials the mass and damping were set to 5 kg and 18 kgms−<sup>1</sup> respectively, to ensure the task was non-trivial and requires some learning and adaptation. No external torque field was applied (except the constraints) and consequently, the task requires pure kinematic adaptation.

Participants were instructed to: "Try and catch each target by moving toward it. The target will disappear if you have been successful and turn red if you fail to achieve it." They were further informed that a successful reach required that they were over the target for approximately 1 s and they had 10 s to reach each target. Targets were considered failed if their position exceeded the outer boundary (illustrated as the red circle in **Figure 7**) or if the participant had not achieved it within 10 s of it appearing.

#### Experimental Procedure

#### **Familiarization phase**

Prior to conducting the task, participants underwent a familiarization phase, consisting of reaching 16 randomly ordered, stationary targets in both the positive and negative directions for each axis. Targets were placed at the same distance as for the main task.

#### **Acquisition phase**

Following familiarization, participants performed three blocks of the main task, separated by a 1–3 min break. Both targets were

visited 15 times each, in a randomized order, after which the RU target was made to reappear as a stationary target, serving as a catch trial to test for after-effects. Ideally, there would have been a subsequent catch trial in the FE axis as well; however, this would have been predictable, affected by a prior catch trial in the RU axis and generally confounded results (Focke et al., 2013; Stockinger et al., 2014).

# RESULTS

# Validation of Normalization Algorithm from Calibration

A representative response from the validation of the normalization algorithms is presented in **Figure 8**. Three metrics from the healthy and constrained velocity profiles are

extracted and presented in **Figure 9**. In **Tables 1**, **2**, the median values for all participants' metrics across the four trials from the validation are presented, in addition to the RMS error normalized as a percentage of the healthy velocity. All statistical analyses used an α level of 0.05.

The dominant frequency was measured by a Fast Fourier Transform (FFT). When the dominant frequency data was tested with a Wilcoxon signed rank test, there was no evidence of a difference between the healthy and constrained velocity profiles for either the RU (p = 0.50) or FE (p = 0.75) axes. Also reported are the power components measured at the dominant frequencies, a measure of the amplitude of the signal at that frequency. A Wilcoxon signed rank test on this data revealed no evidence of a statistical difference in the FE axis (p = 0.19) and strong evidence of a difference in the RU axis (p = 0.0098).

The absolute peak velocities of the constrained and healthytarget states were extracted and the median of these was used as an indicator of the peak velocity in each state. A Wilcoxon signed rank test revealed no evidence of a difference in the peak velocities for the FE axis (p = 0.32) but strong evidence of a difference in the RU axis velocities (p = 0.027), with the medians reported in **Tables 1, 2**. This suggests the specific normalization algorithm may have had an effect. The median RMS errors are also reported as a percentage of the maximum healthytarget velocity and were 21.0 and 22.1%for the RU and FE axes respectively.

#### Motor Learning Task

Participant 6's adaptation to achieving the RU Target is shown in **Figure 10**, comparing performance between the first RU Target in Block 1 (a), the last RU target in Block 3 (b) and the catch trial (c). In (a), the participant has failed to achieve the target before it

reached the boundary. In (b), the target was successfully achieved, and (c) illustrates a clear after-effect as the participant overshot before settling on the stationary target.

Two metrics were used to measure task performance; (i) number of successful targets achieved; (ii) sum-of-squared-error (SSE) measured from the time of target appearance to success (to reflect aspects of both settling time and overshoot); in addition to magnitude of after-effects observed on the catch trial (to quantify the amount of learning).

Qualitatively observing participants indicated that they used one of two methods to complete the task. Either they would immediately move toward the target, or initially move away from the target, giving themselves more "run-up" to reach the target. Therefore, for a given participant, the after-effect is either the maximum overshoot of the target (when participants immediately moved toward the target) or the maximum initial error (when participants chose to move away from the target initially). For each participant, the metric was calculated as the median value across the three blocks.

**Figure 11** illustrates one participant's task metrics across the three blocks of trials for the RU Target in the constrained state. An increase in the number of targets achieved is accompanied by a decrease in SSE, representing an improvement in performance.

Both inter-session and inter-state analyses are presented. Inter-session analysis is performed using a Wilcoxon signed rank test, where statistical differences between sessions would imply that 1 week was insufficient time to washout learning. These results are summarized in **Figure 12** and **Table 3**. There was no evidence of a difference between Sessions 1 and 2 for either target (FE, RU) for the number of targets achieved (p = 0.89, p = 0.94) or the SSE (p = 0.15, p = 0.20). There was also no evidence of a difference for the magnitude of after-effect on the catch trial (p = 0.74). The results conclude that 1 week was indeed sufficient to washout learning.

Healthy and constrained states are also compared and the results summarized in **Figure 13** and **Table 4**. Wilcoxon signed rank tests were performed and for the SSE, there was no evidence



of a difference for either the FE or RU target (p = 0.55, p = 0.15). For the number of targets achieved, there was no evidence of a difference for the RU target (p = 0.16) but there was evidence to suggest a difference in the metric for the FE target (p = 0.016). For the magnitude of the after-effect on the catch trial, there was no evidence of a difference between the states (p = 0.74).

# DISCUSSION

#### Validation of Normalization Algorithm in Calibration

Overall, the results from the calibration phase show that the algorithms were able to adequately normalize the velocity of the participants' cursor, despite their motion being constrained. For


the majority of participants, the dominant frequency of motion of the normalized cursor matched that of the healthy-target motion. Where there is a difference, it is on the scale of 0.05 Hz and can be considered negligible. This is reinforced by a Wilcoxon signed rank test, suggesting no evidence of a difference in dominant frequency in either the RU or FE axes. When the power components of the dominant frequencies are considered, there is strong evidence of a difference for the RU axis, (p = 0.0098) and no evidence to suggest that the power components are different for the FE axis (p = 0.19).

The discrepancy in power components for the RU axis is not surprising, given the additional cognitive load associated with target tracking in comparison to free motion; participants constantly adjust to try and stay on the target and thus perfect tracking cannot be expected. This is reflected in the high frequency components of the graph shown in **Figure 8**. Moreover, the median values were similar for the constrained and healthy velocities in the RU axis (37.6 degs−<sup>1</sup> and 34.5 degs−<sup>1</sup> ), indicating that despite being unable to conclude they are statistically similar, the values are reasonably close.

Some further insight can be gained by examining the peak velocities attained in the healthy and normalized states. A Wilcoxon signed rank test on these values revealed no statistically significant differences for the FE axis, but strong evidence of a difference for the RU axis. However, the median values of healthy and constrained velocity in the RU axis are 51.2 degs−<sup>1</sup> and 47.3 degs−<sup>1</sup> respectively, which is a relatively small error of 7.62%. Furthermore, the boxplots in **Figure 9** visually appear very similar. This suggests that the constrained velocity peaks in the RU axis were comparably close to the healthy velocity peaks.

As with the power content of the dominant frequency, some differences between peak velocities are to be expected, due to the difficulties associated with target tracking. While these metrics provide some insight into the performance of the algorithm, they are also strongly linked to individuals' cognitive abilities in target tracking. Considering the frequency components of motion, as opposed to peak velocities, offers an analysis more closely focused on the non-cognitive aspect of the motion.

The normalized RMS error was quite large, albeit similar in both the RU and FE axes, at 21.0 and 22.1% respectively. Less weight should be attached to the RMS values, however, since they are not representative indicators of the algorithms' performance due to phase lag and high-frequency motion components induced by the cognitive load of target tracking. The healthy-target was intended as a guide; thus we are not interested in matching its time-dependent position exactly. The values of the RMS error should therefore be regarded as a measure of comparing the two algorithms to each other, rather than explicitly evaluating their performance individually. From the RMS metric, it is obvious that the two algorithms have performed comparably.

Overall, we can conclude that the velocity in the constrained state was restored reasonably closely to the original healthy-target velocity by both algorithms. This justifies further exploration of this method as a tool to normalize patient physical ability in tasks requiring motor learning.

#### Motor Learning Task

In general, the weight of evidence from this experiment suggests that both algorithms were able to normalize constrained velocities such that learning of the task was not hindered. In terms of task metrics, the only case in which there was evidence of a statistical difference was the number of times the FE target was achieved (p = 0.016). Further information can be gleaned by examining the median number of times the FE target was achieved in the healthy and constrained states: 12.5 and 10 respectively; implying that it was easier to reach in the healthy state.

TABLE 3 | Inter-session summary statistics.


One reason may be due to the differing algorithms used. The normalization algorithm in the FE axis was supplemented with a linear bias. Simultaneously, most participants qualitatively noted that motion with the exoskeleton felt less restrained in the FE axis. This led to greater sensitivity in the FE axis, which could have led to less consistent performance, potentially explaining the non-significant result for the FE target. Additional human biomechanical and/or exoskeleton dynamics may have contributed to differences in each axis.

One of the primary aims of this research was to create a tool with which motor learning can be studied; thus we must attempt to quantify the amount of learning that took place. The fact that task performance was similar when the algorithm was applied is promising. However, this alone does not reflect the amount of learning that occurred. Haith and Krakauer (2013) distinguish between model-based learning, requiring an internal model, and model-free learning, operating on a trial-and-error basis with no internal model formed. From a technical viewpoint, it does not matter whether the learning that occurred was model-based or model free, since both fall under the accepted definition of "motor learning," and thus it suffices to show that the normalization allows equivalent learning across states.

Catch trials are often included in motor learning studies to test for after-effects (Abdollahi et al., 2014; Melendez-Calderon et al., 2015); and are commonly cited as evidence that an internal model was formed, since they are predictive errors (Shadmehr and Brashers-Krug, 1997; Krakauer, 2006; Haith and Krakauer, 2013). In the present study's catch trials, participants displayed substantial after-effects and no evidence of a difference was found between the healthy and constrained states for the magnitude of after-effects (p = 0.74). This provides evidence that the learning was the same despite the physical limitations imposed by the constraints. A limitation of the study is that the catch trial was only conducted in the RU axis and thus conclusions cannot be drawn with regards to the FE axis. This was done so as to avoid subsequent catch trials affecting each other (Focke et al., 2013; Stockinger et al., 2014). A further limitation of the catch trial analysis in this study is that two participants used different strategies to reach the target, initially pulling away from it. Therefore, the absolute values of their metrics are larger than the rest of the participants' data. While this affects the spread of the data presented, the statistical analyses remain valid, as after-effects were consistent for a given participant and paired Wilcoxon tests were used.

Haith and Krakauer (2013) also cite adaptation time as an indicator of the mechanism of motor learning. In the present study, participants required minimal adaptation time, suggesting model-based learning occurred; however, adaptation time is also

TABLE 4 | Inter-state summary statistics (Healthy – Constrained).


influenced by task difficulty. Nevertheless, the presence of aftereffects provides quantifiable evidence that learning did occur, regardless of the mechanism by which it transpired.

One limitation of the study is the simulation of physical impairment; this adds dynamics that participants must compensate for. The familiarization phases in both the impairment assessment and in the motor learning case study were designed to ensure that participants had "learnt" the constraints as much as possible before the tasks were conducted. However, in previous studies, it was found that learning of dynamic and kinematic transformations occur independently of, and in parallel to, each other (Krakauer et al., 1999). Consequently, in this study it may be that "learning" of the constraints did not interfere with learning the kinematics required to complete the task. The lack of a systematic difference in metrics between the healthy and constrained states strengthens this conclusion. However, there may be some interference in learning the transformation between physical and normalized motion with the learning of the task kinematics.

The majority of metrics used to evaluate motor learning and task performance displayed no significant differences between the states. Thus the weight of evidence concludes that the normalization allowed participants to learn and complete the task to the same standard in the constrained state as in the healthy state. This significant because it lends justification to the potential use of the normalization algorithms to conduct studies on patients with physical impairment. Physical impairment has often been cited as a confounding factor in motor learning studies, as researchers are unable to determine whether poor results are due to an inability to motor learn or because of physical barriers (Valvano and Rapport, 2006; Burtner et al., 2014; van Abswoude et al., 2015). Using normalization, therapists may be able to evaluate the cognitive ability of a patient independently of their physical impairment, alleviating the compounding factors surrounding motor learning.

It is acknowledged that the task presented here is quite specific and that it may not be considered as useful for restoring function. However, we argue that it is the concept that is important, rather than the specific task, and the concept is easily generalized to more complex tasks. Furthermore, the purpose of the normalization is not to provide restoration of physical function, but as a tool with which therapists can fairly compare individuals' (with and without impairment) inherent ability to motor learn and provide subject-specific, optimized therapy programs informed by this knowledge. This study is intended as an exploratory investigation of the concept of the normalization and future work will examine the use of this technique in patients with cerebral palsy, which will lead to further understanding of motor learning in impaired participants.

## Comparison of Algorithms

The final matter to address is the comparison of each algorithm. Both algorithms were able to individually restore velocity capabilities to a decent standard. When the validation of the calibration is considered, it was the algorithm used in the FE axis that provided better results. However, it is important to consider that the only metrics where it significantly outperformed the RU algorithm were the measurement of peak velocities and the power component of the dominant frequency, which are affected by cognitive difficulties and are not considered the optimal measures of performance. Furthermore, for both algorithms, the summary statistics for all metrics were comparably close between the healthy and constrained velocity profiles.

The algorithm used in the RU axis overall provided better performance in the motor learning task, where no evidence of a difference between healthy and constrained states was found for any metric. This contrasts with the FE axis, where significantly more targets were achieved in the healthy state.

It is therefore difficult to say conclusively which algorithm was superior. The linear bias added to the algorithm in the FE axis gave no clear advantage over the standard algorithm in this application. In contrast, the algorithm in the RU axis performed sufficiently, despite the lack of a supplementing bias factor, and matches the structure of RBF used in the research that originally inspired our use of RBFs (Wolbrecht et al., 2008; Oboe and Pilastro, 2014; Pehlivan et al., 2015). Consequently, this shall be used in future development of this normalization tool.

#### Limitations

The primary limitation of this study is the method of simulating physical impairment. The torques exerted by patients in the healthy and constrained cases are quite different. While this does simulate the fact that patients with impairment tend to tire more quickly than their healthy counterparts, it is possible that fatigue may be a compounding factor on the results. The focus of this preliminary study, however, was to investigate whether the kinematics of the constrained participants could be normalized and thus kinetics were not considered. In future, EMG could be used to measure torque, and further studies will focus on incorporating kinetic data into the algorithm to give a fairer normalization of the abilities of impaired individuals.

There is also difficulty associated with accurately replicating a physical impairment. In reality, these are not modeled simply. However, the purpose of this study was to show that the kinematics can be normalized in the presence of a torque field that restricts motion. In this respect, the impairment can be treated as a black box, with the only important phenomenon being that it has an impact on kinematics. The use of biomechanical models could be considered for more accurately

## REFERENCES


representing actual impairments; however, this was considered to be out of scope for a preliminary study.

The pilot sample size reduces the strength of the statistical outcomes. However, as a preliminary investigation into the feasibility of the algorithm, a small sample size was considered sufficient and has gleaned significant relevant information to provide interesting results and motivate further investigation into this work.

# CONCLUSIONS

A new technique for investigating motor learning in physically impaired individuals has been proposed in an investigative pilot study focussing on wrist motion. Measurements of velocity as a function of position are used to derive subject-specific scaling factors that normalize the velocity of individuals to a "healthy" benchmark. Healthy participants were subjected to an artificial impairment that restricted both range of motion and velocity in the RU and FE axes of the wrist. The algorithms were statistically proven to restore velocity accurately. In a crossover study participants were able to complete the task with the same performance when the algorithm was applied to their constrained state as they did in the healthy state, with some evidence that motor learning had occurred. Overall, the results show promise for the further investigation of this technique in exploring motor learning in impaired individuals.

## ETHICS STATEMENT

This research was supported by the Marsden Fund Council from Government funding, managed by the Royal Society of New Zealand.

# AUTHOR CONTRIBUTIONS

Both authors contributed to identifying study motivation, study design and experimental protocol. Experiments were conducted by CJ.

## FUNDING

This research was partially supported by the Marsden Fund Council from Government funding, managed by the Royal Society of New Zealand.

in cerebral palsy. Biomed Res. Int. 2015:317047. doi: 10.1155/2015/ 317047


management of children with cerebral palsy. Euro. J. Neurol. 6, s23–s35. doi: 10.1111/j.1468-1331.1999.tb00031.x


children and young adults with cerebral palsy: a randomized, double-blind study of high-dose versus low-dose treatment. Neuropediatrics 30, 120–124. doi: 10.1055/s-2007-973475


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2017 Jarrett and McDaid. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Analysis of Interrelationships among Voluntary and Prosthetic Leg Joint Parameters Using Cyclograms

Farahiyah Jasni 1, 2, Nur Azah Hamzaid<sup>1</sup> \*, Nor Elleeiana Mohd Syah<sup>1</sup> , Tze Y. Chung<sup>3</sup> and Noor Azuan Abu Osman<sup>1</sup>

<sup>1</sup> Department of Biomedical Engineering, Faculty of Engineering, University of Malaya, Kuala Lumpur, Malaysia, <sup>2</sup> Department of Mechatronics Engineering, Kulliyyah of Engineering, International Islamic University Malaysia, Selangor, Malaysia, <sup>3</sup> Department of Rehabilitation Medicine, Faculty of Medicine, University of Malaya, Kuala Lumpur, Malaysia

The walking mechanism of a prosthetic leg user is a tightly coordinated movement of several joints and limb segments. The interaction among the voluntary and mechanical joints and segments requires particular biomechanical insight. This study aims to analyze the inter-relationship between amputees' voluntary and mechanical coupled leg joints variables using cyclograms. From this analysis, the critical gait parameters in each gait phase were determined and analyzed if they contribute to a better powered prosthetic knee control design. To develop the cyclogram model, 20 healthy able-bodied subjects and 25 prosthesis and orthosis users (10 transtibial amputees, 5 transfemoral amputees, and 10 different pathological profiles of orthosis users) walked at their comfortable speed in a 3D motion analysis lab setting. The gait parameters (i.e., angle, moment and power for the ankle, knee and hip joints) were coupled to form 36 cyclograms relationship. The model was validated by quantifying the gait disparities of all the pathological walking by analyzing each cyclograms pairs using feed-forward neural network with backpropagation. Subsequently, the cyclogram pairs that contributed to the highest gait disparity of each gait phase were manipulated by replacing it with normal values and re-analyzed. The manipulated cyclograms relationship that showed highest improvement in terms of gait disparity calculation suggested that they are the most dominant parameters in powered-knee control. In case of transfemoral amputee walking, it was identified using this approach that at each gait sub-phase, the knee variables most responsible for closest to normal walking were: knee power during loading response and mid-stance, knee moment and knee angle during terminal stance phase, knee angle and knee power during pre-swing, knee angle at initial swing, and knee power at terminal swing. No variable was dominant during mid-swing phase implying natural pendulum effect of the lower limb between the initial and terminal swing phases. The outcome of this cyclogram adoption approach proposed an insight into the method of determining the causal effect of manipulating a particular joint's mechanical properties toward the joint behavior in an amputee's gait by determining the curve closeness, C, of the modified cyclogram curve to the normal conventional curve, to enable quantitative judgment of the effect of changing a particular parameter in the prosthetic leg gait.

Keywords: transfemoral, biomechanics, cyclogram, prosthesis

#### Edited by:

Wenwei Yu, Chiba University, Japan

#### Reviewed by:

Giuseppe D'Avenio, Istituto Superiore di Sanità, Italy Hans-Eckhardt Schaefer, University of Stuttgart, Germany

#### \*Correspondence:

Nur Azah Hamzaid azah.hamzaid@um.edu.my

#### Specialty section:

This article was submitted to Neural Technology, a section of the journal Frontiers in Neuroscience

Received: 10 December 2016 Accepted: 05 April 2017 Published: 25 April 2017

#### Citation:

Jasni F, Hamzaid NA, Mohd Syah NE, Chung TY and Abu Osman NA (2017) Analysis of Interrelationships among Voluntary and Prosthetic Leg Joint Parameters Using Cyclograms. Front. Neurosci. 11:230. doi: 10.3389/fnins.2017.00230

# INTRODUCTION

Human walking is typically characterized by plotting kinematics and kinetics curves as a function of time or percentage of gait cycle. These kinematics and kinetics curves became the primary guideline in the prosthetic leg design that would mimic the normal gait. These were done by ensuring that the prosthesis produces user gait performance that is close to the normal singlevariable curve (Engeberg, 2013; Kutilek et al., 2013).

Although it is more common and convenient to use a single variable curve of kinematics or kinetics parameters as guideline in designing the control system for microprocessor based prosthetic leg, literatures proved that there are advantages of analyzing it as a pair. A number of relevant dynamic effects can be identified when pairs of kinematics and kinetic variables are examined together and correlations among them were concurrently assessed (Crenna and Frigo, 2011). These combinations of kinematics and kinetics variables, omitting the time variables from the two signals, create different cyclic trajectories known as cyclograms (Goswami, 1998). From the cyclic trajectories represented by the coupled variables, the dynamic variation of the gait can be easily visualized through the changes in the cyclograms pattern. In addition, the interaction between two co-existing parameter across the joints could be concurrently assessed. This is the foundation of applying the cyclograms concept to determine the most influential knee parameter in producing closest to normal and most efficient gait.

The use of cyclograms over a function of time has been used as a reference for designing a microprocessor-controlled prosthesis (Kutilek et al., 2013, 2014a,b). This is because, the fact that "locomotion is a tightly coordinated movement of several limb segments" can be more naturally grasped as the coupled variables of two or more joints rather than individual joint kinematics or kinetics. The cyclogram pattern was noted to be an extremely stable mechanism to identify gait behavior due to the use of cyclic traces of joint variables (Kutilek et al., 2014a). This was achieved based on the principal that a coordinated motion of a leg is to be perceived as an interaction between two or more limb segments rather than a phenomenon of isolated joint movements over time (Awai and Curt, 2014).

According to Pitkin, designing a better prosthetic leg does not revolve around the integration to mimic close-to-normal limb segments behavior (Pitkin, 2010). The design should take into account the amputees pathological condition (Pitkin, 2010). Therefore, there is a need to outline the basis for the prosthetic design that requires minimal modification to adjust with, if not all, most of the amputees' conditions.

The techniques that had been used to mathematically represent gait data and to analyze it include fuzzy system methods, multivariate statistical analysis, fractal dynamics, artificial intelligence (including neural networks), and wavelet methods (Arjunan and Kumar, 2010). Neural networks had been the most prevalent emerging non-traditional method applied to the analysis of gait data (Su and Wu, 2000). The combination of neural network and cyclograms in a single study is rarely found in the literature, and none has been introduced for identifying the gait disparity in amputees. To date, neural network has been used to perform automated diagnosis of gait patterns represented by angle-angle cyclogram (Barton and Lees, 1997). In one study (Barton and Lees, 1997), once trained, the network can identify three different conditions—normal gait, a gait with unequal leg length and a gait with unequal leg weights at 83.3% success rate. The work of Kutilek and Farkasova that described the method of predicting the motion of lower extremities using neural network and had suggested that the predicted data may be useful for evaluation of human walking in physiotherapy practice based on angle-angle diagram (Kutilek and Farkasova, 2011). Their method was one of the earliest to apply neural network in a clinical practice for study of disorders or characteristics in motion function of human body, and also to be used in new design of lower limb prosthesis (Kutilek and Farkasova, 2011).

This study proposed a method of using the cyclograms as the cyclic representation of locomotion and two-variable interaction, to determine the most influential parameter in each gait subphases for the knee control design in transfemoral prosthesis. Using neural network, we analyzed the interaction between the users' gait coupled parameters holistically (of the joint itself and across two joints) toward the least error production from normal profile. The final parameter at each gait sub-phase was then determined by analyzing the effect of manipulating one parameter onto another parameter of the coupled variables at the most deviated cyclogram for different amputee's profile.

# METHODS

This study consists of 3 phases. The first phase involved gait analysis and cyclogram model generation based on the leg joint parameters. Phase 2 verified the generated cyclogram models by determining the gait deviation at each sub phase throughout the gait cycle of participants with different pathological conditions. In the last phase, the modified cyclogram models were used to identify the most dominant parameter at each gait sub-phase in the case of transfemoral amputees.

# Phase 1: Gait Analysis and Generating Cyclograms

In this section, the data collection procedure was explained in detail. Then, the generation of cyclogram pairs from the voluntary and mechanical joints' parameters was described.

#### Data Collection and Gait Analysis Method

**Table 1** describes the details of the participants involved in this work.

The profile data for each of the transtibial, transfemoral and orthosis subjects and their respective pathological condition are presented in **Table 2**. All the subjects provided their written and informed consent by signing the consent form provided to them, to participate in this study. Approval for the gait analysis procedures was obtained from the University Malaya Medical Centre Ethics Review Board and this study confirmed with the regulatory standards.

The experimental procedures of this study were conducted in a motion analysis lab on a 4 m straight walkway. The lab was equipped with an optical three-dimensional motion capture

#### TABLE 1 | Description of the subjects.




R, Right; L, Left; B, Both; TT, Transtibial; TF, Transfemoral; OT, Orthotic; AFO, Ankle-foot Orthosis.

system, a 6-infrared camera Vicon Nexus 1.5 (Vicon, United Kingdom) mounted strategically around the room, working at 50 Hz and integrated with two force plates (Kistler Instruments, Switzerland) working at 200 Hz, located midway to obtain synchronized kinematic and kinetic data within a capture volume of approximately 4 × 4 × 2 m.

The subjects' bilateral leg length (from greater trochanter to lateral malleolus), ankle width and knee width were measured and recorded to the system software for automated calculation of mass, center of mass and moment of inertia. The knee width of residual limb was considered as similar with the sound limb for transfemoral subjects. The ankle width for both transfemoral and transtibial subject residual limb were also considered to be similar to the sound limb measurement. Ground reaction force was obtained from the force plates for kinetic automated calculation by the Vicon software to obtain joint power and moment. Sixteen reflective markers of 14 mm diameter were attached bilaterally based on Helen Hayes marker placement.

At least 20 trials were recorded for each subject and all the subjects were asked to rest in between the trial to prevent fatigue effect to the data collection. The gait data was analyzed with a conventional model Vicon Plug-In-Gait and filtered using second-order Butterworth filter. The ankle, knee and hip joint kinematics, moments and gait events as well as relevant markers' trajectories, Ground Reaction Force (GRF) and Center of Pressure (COP) data were imported into Matlab for extraction and further analysis. Linear interpolation was applied to the original data points to obtain data points for joint kinematics and kinetics data at every 2% of stride duration. The joint moment was computed by inverse dynamics, using subjects' measurements and anthropometric properties and normalized to body weight (Robertson, 2004).

The ankle dorsi-plantar flexion angles, knee and hip flexionextension angles were the kinematic data required in this study, while for kinetic analysis, all the three joints' sagittal moment and power were extracted.

#### Cyclogram Models Generation

A total of 36 relationships were obtained by pairing the variables of the joint itself and across the two joints pair. The relationships can be classified into 6 groups, namely:


Most of these relationships were not found in the literature except for angle-angle relationship across the two joints pair (Cavanagh and Grieve, 1973; Goswami, 1998) and moment-angle of the ankle joint (Crenna and Frigo, 2011; Wang et al., 2012).

The cyclogram relationships for the normal subjects were obtained by averaging parameters of 200 trials from 20 subjects due to the consistency of the gait sub-phase duration among the normal subjects. On the other hand, this averaging method does not apply to the transtibial, transfemoral, and orthosis subjects because of their irregularities of gait sub-phase duration in each trial. Thus, the relationships were obtained from each of the trial, for 10 trials for these amputee subjects. The amputee subjects' cyclograms in addition to its respective gait sub-phases from the trials would be input into the neural network in Phase 2 to predict the output of subject-based movement profiles and identify the gait deviation at each of the gait sub-phase.

From the normal cyclograms relationship obtained, it was found that the geometric shape of the cyclograms changed as the traveling direction of the gait sub-phases change between both legs. Thus, it was deduced that the interaction between the two variables were different between left and right leg. Primarily, this geometric difference was influenced by the limb dominance effect on the normal walking gait. The dominant limb was the preferred limb (leading limb) that the subjects used for mobilizing or propulsion; while in contrast, the nondominant limb is the non-preferred limb (trailing limb) that is used to support the actions, i.e., stability control. This support

and mobility task performed by each leg was interpreted as gait asymmetry that is normal to occur even to healthy able-bodied subjects. All the normal subjects in this study were right limb dominant, therefore, right side served as the propulsion source during walking. The necessity of defining which limb contributes to propulsion or support task was important in this study in order to select the reference side of cyclograms that was identical to the functional role of the prosthetic and the intact side for amputees. It has been revealed in the study by Carpenter et al. that the transtibial amputees who undergone osteomyoplastic amputation (also known as Ertl) rely on the prosthetic side to support, while the intact side function to propel the body forward (Carpenter et al., 2012). This is further supported by the study that revealed the unilateral amputees experience increased asymmetry in their intact limb during loading and stance time due to the loss of ankle plantar-flexors which contribute to body propulsion and swing initiation (Liu et al., 2006). Therefore, the subsequent phases in this study make use of the cyclograms on the left side (non-dominant) to study the deviation that occur on the prosthetic side of the amputee.

# Phase 2: Using Cyclograms to Quantify Gait Disparity in Terms of Gait Sub-Phases

All 36 cyclograms that were created in previous phase were used to determine the gait disparity of the amputees. **Figure 1** illustrates the example on how the cyclograms was utilized to determine the gait disparity occurrence at each gait sub-phase for each of the subject. The orange-colored area represents the error

C = (1 − ep−c) × 100 (1)

between amputee's cyclogram and the normal cyclogram, i.e., the gait disparity. The higher the error as compared to the normal gait's cyclogram, the higher is the gait deviation of the subject.

At this stage, neural network was adopted as the tool to model the interaction between two parameters in a cyclogram, as well as to calculate the disparity between cyclograms. Hence, the feedforward network with backpropagation algorithm was employed. This network was chosen as it uses supervised learning rule during the training. The supervised learning rule provide an example for the network learning, and the network learn by comparing the predicted output with the desired output, and backpropagate the error to update the network weight/bias (i.e., the constant used to define the strength of the input in determining the output) (Beale et al., 2010). The neural network architecture was constructed in Matlab and the predicted outputs and errors obtained were used to quantify the disparity occurrence at each of the gait sub-phase.

The network consisted of 3 layers; input layer (30 neurons), hidden layer (7 neurons) and output layer (3 neurons). The relationship between the input data and the output data was found linear, thus linear regression functions were employed. The input and target data arrangement was illustrated as in **Figure 2**. The first 10 rows (E1-E10) represent the first cyclogram's variable for 10 trials, the next 10 rows are the second cyclogram's variable for 10 trials, and the last 10 rows are the gait sub-phases which annotated as 1 until 7 corresponding to loading response to terminal-swing for 10 trials. On the other hand, S1-S50 represents the samples of the variables series at an interval of 2% gait cycle and its respective gait sub-phase duration.

The network was first trained with data of 100 trials that was randomly chosen from 10 healthy subjects as network input and average normal data of 200 trials from 20 healthy subjects as the network target in order for the NN to learn the outcome of different paired variables interactions of normal subject as the reference. A total of 36 NN were trained with 36 paired variables, i.e., cyclograms, respectively. During the training, default random division algorithm was used for the NN to divide the data randomly for training (60%), validation (20%), and testing (20%). The network weight was reinitialized if all the following criteria for the network performance were not met, and only network that fulfilled the following criteria were kept for simulation with prosthetic cyclograms: (1) final mean square error <0.9 at the final iteration, (2) the test set error and validation set error has similar characteristics, (3) the regression for all set was > 0.9.

Before simulating the trained network with amputee's cyclograms, the network was first tested and validated with 5 random normal subjects and 25 different cases of amputees and orthosis subjects' pathological conditions. The predicted output of cyclogram's pair from the simulation was split into two variables curve as a function of gait cycle percentage. These two curves were then compared with the curves that used the conventional trials averaging method. The errors between the two curves (ep−c) were calculated for each subject as illustrated in **Figure 3**.

The network model prediction ability was presented in terms of percentage of the closeness of the network predicted output curve as compared to the conventional trials averaging curve. The percentage of closeness was calculated as in Equation (1), with C =100% indicates totally close to normal or a perfect match to normal.

where,

$$\mathcal{C} = \text{curves closest}$$

ep−<sup>c</sup> = Errors between predicted curve points and conventional curve points, with 1 being maximum (100%) error or difference between predicted and conventional curve points, and 0 to be no error at all.

The validation percentage presented in **Table 3** indicates that the model, as it was trained with the normal subjects' data, was as expected to have prediction ability of more than 90% with randomly-selected normal subjects. As for orthosis, transtibial, and transfemoral subjects, the percentage range indicates that the NN used has the ability to segregate the deviated data as a whole series of gait cycle and by the two interacting variables that might have contributed most to the gait deviation. The validation for these "pathological" subjects was expected to be <90%, with transfemoral subject having the greatest error (**Table 3**) due to the least voluntary control of residual limb. This error was used as the quantifying measure in this study.

## Phase 3: Using Manipulated Cyclograms to Identify the Dominant Parameter(s) in Each Gait Sub-Phases

In this phase, the results obtained from the previous phase were used to assess the effect of manipulating one variable toward the other variable of the cyclograms. At each of the gait sub-phase for each amputee, the cyclograms relationship that resulted in the largest deviation was then being input back into the linear network model. A modification of the variable values at the particular sub-phase with the highest mean normalized error was done by inserting the normal value (i.e., from healthy subjects that is averaged using conventional trials averaging method described in Section Cyclogram Models Generation) at one of the paired variables as illustrated in **Figure 4**. The network model was simulated with the modified input data and the target output data was again being supplied. The error between the predicted output and the desired output was calculated again at each of the sub-phase. This process was repeated with the other variable of the pair. The results obtained were then compared to determine which parameter manipulation gives predicted output that was closer-to-normal cyclograms. This was the crucial identification

TABLE 3 | Percentage range of closeness between networks predicted output curve and conventional trials averaging curve.


step in determining the relevant parameter to be controlled at each of the gait sub-phase.

# RESULTS

## Using Cyclograms to Quantify Gait Disparity in Terms of Gait Sub-Phases

The cyclograms relationship that contributed to the highest mean normalized error at each gait sub-phase for each subject was summarized in **Table 4**. At loading response (LR), the kinematics-kinetics relationship shows high mean normalized error for each of the subject. The kinetics variable of the pair indicated in red color contributes to the highest mean normalized error, except for subjects TT3 (ankle angle, Aθ) and TT5 (knee angle, Kθ).

At mid-stance (MSt) phase, the result indicated that the kinematics-kinetics relationship has the highest mean normalized error for each of the subject, except for TT5 (AM-HP), TT6 (AM-HP), and TT8 (AM-HP), which shows the kinetics-kinetics relationship. Nevertheless, the variable of the pairs that contributes the highest mean normalized error at this phase were all originated from the kinetics variable (knee power/hip power/hip moment). Except for TT5, TT6, TF4, and TF5 which shows that kinetics-kinetics relationship has high mean normalized error at terminal-stance (TSt), the rest showed that the kinematics-kinetics pair exhibits high mean normalized error. Similar with the MSt phase, the kinetic variable (knee power/hip power/hip moment) of the pairs has the highest mean normalized error in TSt phase compared to the kinematic variable.

Similar cyclograms' relationship profile as the aforementioned MSt and TSt were found for pre-swing (PSw), initial-swing (ISw) and mid-swing (MSw), whereby most of the relationships that yield the high mean normalized error were of kinematicskinetics pair except for TF4 (AM-HP) in pre-swing and TT2 (AM-HP) in ISw. As for terminal-swing (Tsw), 10 out of the 15 amputee subjects showed unanimously that the kinematics of knee angle variable yield the highest mean normalized error. This contradicted with the results of previous gait sub-phases in which kinetics variable (ankle moment/ankle power/knee moment/knee power/hip moment/hip power) of the pair contributed to the highest mean normalized error.


TABLE 4 | Summary of the paired variables that contribute to the highest mean normalized error at respective gait sub-phase for each amputee subject.

A, Ankle; K, Knee; H, Hip; θ, Angle; M, Moment; P, Power.

Red-colored acronyms represent the variable correspond to yielding the highest mean normalized error. The bold figures in bracket refer to the mean normalized error of one of the paired variable, E.¯

# Using Manipulated Cyclograms to Identify the Most Dominant Parameter in Each Gait Sub-Phase

The results of the manipulated variable that showed the lowest mean normalized error were translated as the variable that induced closest-to-normal cyclograms curve compared to the after-effect of manipulating the other parameters. The overall results were summarized as in **Table 5**. The results indicated that at the LR phase, when the knee power was manipulated, i.e., replaced with normal values, the mean normalized error was reduced at the phase and consequently the other phases along the gait cycle were also improved. At MSt phase, the result showed that manipulating ankle angle and knee power produced the lowest mean normalized error. The results obtained from NN showed that manipulating either one of these two parameters would yield the same result along the whole gait cycle. This indicated that modifying either one of the parameter is sufficient to produce close to normal gait profile during MSt. For TSt, ankle angle was the dominant parameter that demonstrated the lowest mean normalized error. While at PSw and ISw, hip power and hip angle, respectively produced the lowest mean normalized error.

Manipulation of any one of the parameters at MSw does not improve the sub-phase normalized errors. Observation at this phase indicated that the manipulation of parameter at this phase consequently caused the neighboring phases to yield a large mean normalized error. Therefore, no parameter was selected for manipulation at this sub-phase. Finally, for the TSw, knee power parameter was selected as it demonstrated the lowest mean normalized error when being manipulated.

#### Identifying Dominant Knee Parameters for Potential Microcontroller Based Knee Application: A Case Study

All the dominant parameters identified through this method were of either hip, knee or ankle parameters. However, in case of minimizing gait disparity through optimization of the knee parameters only, results related to hip and ankle parameters were revisited to identify the dominant knee parameters in its place.


MSw phase does not require any joint control.

This was done by referring to the values of mean normalized error at Phase 2 between hip and ankle variables paired with knee variables. The knee variables that yielded the least mean normalized error when paired with hip and ankle variables of the result as in **Table 5**(a) was extracted at that particular sub-phase. The revised result, i.e., for knee parameters only, is presented in **Table 5**(b).

#### DISCUSSION AND CONCLUSION

This study modeled all cyclograms of lower limb joint's gait variables holistically and analyzed it to identify the gait disparity among prostheses users. It also identified the most dominant knee parameter at each gait sub-phase for a prosthetic knee design.

The highlight of the finding was the results of MSw gait phase modified behavior. Manipulation of both parameters in the coupled pair that was found to have the largest mean normalized error in MSw phase did not improve the performance at that particular phase. Worse is, the manipulation of either parameter at the MSw phase caused the neighboring subphases (ISw and TSw) yield larger mean normalized error. This indicated biomechanical justification of the momentum and gravity influence during MSw. The prosthetic knee should be left independent to extend by the effect of momentum produced by its weight and the gravity at the mid-swing in order to connect the end of initial-swing phase with the beginning of terminalswing as illustrated in **Figure 5**. This concept was discussed as "ballistic synergy" concept introduced by Pitkin that allowed for the control simplification and reduces the power supplies demand when compared with robots for which all its degrees of freedom in motion are controlled during the entire gait (Pitkin, 2010). This would make the kinematics and kinetics match with normal human ballistic gait synergy.

In this study, the results shown that altering one of the dominant parameter that was identified using manipulate cyclograms method yielded to a better performance in most of the gait sub-phases. However, it might be oversimplified considering the complexity in the dynamics of human gait. Therefore, the findings will need to be further verified by performing experiment using a micro-controller knee on a patient and the effect in the gait of the patient is analyzed.

It has been recommended in a previous study that more prospective intervention studies that take into account the multifactorial nature on the amputees walking ability should be conducted (Van Velzen et al., 2006). Thus, we proposed the use of inter-relationship modification of two-variables interaction that co-exist during a gait cycle, especially variables across twojoints which provided an insight into the effect of controlling one variable onto another variable at other joints during walking for an amputee.

Finally, only the affected limb of each subject was thoroughly investigated and reported and the actual speed of each subject was not considered. Several adaptation strategies in terms of joint power or work in both the amputated and intact leg were demonstrated in patients with transfemoral amputation (Prinsen et al., 2011). Investigation on the bilateral control was only done to determine the difference in leading or trailing limb kinematic behavior but not in more detailed toward its cyclograms changes. Further investigation would offer a more precise understanding of each subject's compensatory movement.

## AUTHOR CONTRIBUTIONS

FJ, NAH, and NMS developed the scientific content and context of the article. NMS performed the data collection of the study participants and analysis of the data. TC referred and pre-screened the study participants and contributed clinical insights into the article. NAO provided the funding, setup and facilities to perform the study and oversees the scientific content of the study. The final article was majorly written by FJ, NAH, and NMS.

## FUNDING

This study was funded by Ministry of Higher Education (MOHE) of Malaysia; grant no: UM.C/HIR/MOHE/ENG/10 and the first author's research work is supported by Ministry of

# REFERENCES


Higher Education, Malaysia and International Islamic University Malaysia under SLAB/SLAI scholarship.

### ACKNOWLEDGMENTS

The authors thank the subjects who participated in this study and the technical support staffs in the Motion Analysis Laboratory, Faculty of Engineering, University of Malaya.


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2017 Jasni, Hamzaid, Mohd Syah, Chung and Abu Osman. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

digital media

of impactful research

article's readership