# APPLICATIONS OF SYNERGIES IN HUMAN MACHINE INTERFACES

EDITED BY : Ramana Vinjamuri, Zhi-Hong Mao and Anil Maybhate PUBLISHED IN : Frontiers in Bioengineering and Biotechnology

#### Frontiers Copyright Statement

© Copyright 2007-2019 Frontiers Media SA. All rights reserved. All content included on this site, such as text, graphics, logos, button icons, images, video/audio clips, downloads, data compilations and software, is the property of or is licensed to Frontiers Media SA ("Frontiers") or its licensees and/or subcontractors. The copyright in the text of individual articles is the property of their respective authors, subject to a license granted to Frontiers.

The compilation of articles constituting this e-book, wherever published, as well as the compilation of all other content on this site, is the exclusive property of Frontiers. For the conditions for downloading and copying of e-books from Frontiers' website, please see the Terms for Website Use. If purchasing Frontiers e-books from other websites or sources, the conditions of the website concerned apply.

Images and graphics not forming part of user-contributed materials may not be downloaded or copied without permission.

Individual articles may be downloaded and reproduced in accordance with the principles of the CC-BY licence subject to any copyright or other notices. They may not be re-sold as an e-book.

As author or other contributor you grant a CC-BY licence to others to reproduce your articles, including any graphics and third-party materials supplied by you, in accordance with the Conditions for Website Use and subject to any copyright notices which you include in connection with your articles and materials.

All copyright, and all rights therein, are protected by national and international copyright laws.

The above represents a summary only. For the full conditions see the Conditions for Authors and the Conditions for Website Use. ISSN 1664-8714 ISBN 978-2-88945-897-4 DOI 10.3389/978-2-88945-897-4

#### About Frontiers

Frontiers is more than just an open-access publisher of scholarly articles: it is a pioneering approach to the world of academia, radically improving the way scholarly research is managed. The grand vision of Frontiers is a world where all people have an equal opportunity to seek, share and generate knowledge. Frontiers provides immediate and permanent online open access to all its publications, but this alone is not enough to realize our grand goals.

#### Frontiers Journal Series

The Frontiers Journal Series is a multi-tier and interdisciplinary set of open-access, online journals, promising a paradigm shift from the current review, selection and dissemination processes in academic publishing. All Frontiers journals are driven by researchers for researchers; therefore, they constitute a service to the scholarly community. At the same time, the Frontiers Journal Series operates on a revolutionary invention, the tiered publishing system, initially addressing specific communities of scholars, and gradually climbing up to broader public understanding, thus serving the interests of the lay society, too.

#### Dedication to Quality

Each Frontiers article is a landmark of the highest quality, thanks to genuinely collaborative interactions between authors and review editors, who include some of the world's best academicians. Research must be certified by peers before entering a stream of knowledge that may eventually reach the public - and shape society; therefore, Frontiers only applies the most rigorous and unbiased reviews.

Frontiers revolutionizes research publishing by freely delivering the most outstanding research, evaluated with no bias from both the academic and social point of view. By applying the most advanced information technologies, Frontiers is catapulting scholarly publishing into a new generation.

#### What are Frontiers Research Topics?

Frontiers Research Topics are very popular trademarks of the Frontiers Journals Series: they are collections of at least ten articles, all centered on a particular subject. With their unique mix of varied contributions from Original Research to Review Articles, Frontiers Research Topics unify the most influential researchers, the latest key findings and historical advances in a hot research area! Find out more on how to host your own Frontiers Research Topic or contribute to one as an author by contacting the Frontiers Editorial Office: researchtopics@frontiersin.org

# APPLICATIONS OF SYNERGIES IN HUMAN MACHINE INTERFACES

Topic Editors:

Ramana Vinjamuri, Stevens Institute of Technology, United States Zhi-Hong Mao, University of Pittsburgh, United States Anil Maybhate, Johns Hopkins University, United States

Citation: Vinjamuri, R., Mao, Z.-H., Maybhate, A., eds. (2019). Applications of Synergies in Human Machine Interfaces. Lausanne: Frontiers Media. doi: 10.3389/978-2-88945-897-4

# Table of Contents

*04 Reducing the Impact of Shoulder Abduction Loading on the Classification of Hand Opening and Grasping in Individuals With Poststroke Flexion Synergy*

Yiyun Lan, Jun Yao and Julius P. A. Dewald


Vrajeshri Patel, Jamie Craig, Michelle Schumacher, Martin K. Burns, Ionut Florescu and Ramana Vinjamuri

*56 Low-Dimensional Synergistic Representation of Bilateral Reaching Movements*

Martin K. Burns, Vrajeshri Patel, Ionut Florescu, Kishore V. Pochiraju and Ramana Vinjamuri

*71 Muscle Synergies Facilitate Computational Prediction of Subject-Specific Walking Motions*

Andrew J. Meyer, Ilan Eskinazi, Jennifer N. Jackson, Anil V. Rao, Carolynn Patten and Benjamin J. Fregly

*97 A Muscle Synergy-Inspired Adaptive Control Scheme for a Hybrid Walking Neuroprosthesis*

Naji A. Alibeji, Nicholas Andrew Kirsch and Nitin Sharma


Jose G. Martinez, Beatriz Berrueco and Toribio F. Otero

# Reducing the Impact of Shoulder Abduction Loading on the Classification of Hand Opening and Grasping in Individuals with Poststroke Flexion Synergy

*Yiyun Lan1,2, Jun Yao1,2 and Julius P. A. Dewald1,2,3,4\**

*<sup>1</sup> Interdepartmental Neuroscience Program, Northwestern University, Chicago, IL, United States, 2Department of Physical Therapy and Human Movement Sciences, Northwestern University, Chicago, IL, United States, 3Department of Biomedical Engineering, Northwestern University, Chicago, IL, United States, 4Department of Physical Medicine and Rehabilitation, Northwestern University, Chicago, IL, United States*

#### *Edited by:*

*Ramana Vinjamuri, Stevens Institute of Technology, United States*

#### *Reviewed by:*

*Ioannis Andrea Ieropoulos, University of the West of England, United Kingdom Ping Zhou, University of Texas Health Science Center at Houston, United States*

> *\*Correspondence: Julius P. A. Dewald*

*j-dewald@northwestern.edu*

#### *Specialty section:*

*This article was submitted to Bionics and Biomimetics, a section of the journal Frontiers in Bioengineering and Biotechnology*

*Received: 01 January 2017 Accepted: 12 June 2017 Published: 30 June 2017*

#### *Citation:*

*Lan Y, Yao J and Dewald JPA (2017) Reducing the Impact of Shoulder Abduction Loading on the Classification of Hand Opening and Grasping in Individuals with Poststroke Flexion Synergy. Front. Bioeng. Biotechnol. 5:39. doi: 10.3389/fbioe.2017.00039*

Application of neural machine interface in individuals with chronic hemiparetic stroke is regarded as a great challenge, especially for classification of the hand opening and grasping during a functional upper extremity movement such as reach-to-grasp. The overall accuracy of classifying hand movements, while actively lifting the paretic arm, is subject to a significant reduction compared to the accuracy when the arm is fully supported. Such a reduction is believed to be due to the expression of flexion synergy, which couples shoulder abduction (SABD) with elbow/wrist and finger flexion, and is common in up to 60% of the stroke population. Little research has been done to develop methods to reduce the impact of flexion synergy on the classification of hand opening and grasping. In this study, we proposed a novel approach to classify hand opening and grasping in the context of the flexion synergy using a wavelet coherence-based filter. We first identified the frequency ranges where the coherence between the SABD muscle and wrist/finger flexion muscles is significant in each participant, and then removed the synergy-induced electromyogram (EMG) component with a subject-specific and muscle-specific coherence-based filter. The new approach was tested in 21 stroke individuals with moderate to severe motor impairments. Employing the filter, 14 participants gained improvement in classification accuracy with a range of 0.1 to 14%, while four showed 0.3 to 1.2% reduction. The remaining three participants were excluded from comparison due to the lack of significant coherence, thus no filters were applied. The improvement in classification accuracy is significant (*p* = 0.017) when the SABD loading equals 50% of the maximal torque. Our findings suggest that the coherence-based filters can reduce the impact of flexion synergy by removing the synergy-induced EMG component and have the potential to improve the overall classification accuracy of hand movements in individuals with poststroke flexion synergy.

Keywords: stroke, flexion synergy, machine learning, hand movements, classification, neural machine interface, coherence

### INTRODUCTION

Functional movements that demand independent joint control of the shoulder, elbow, and wrist/fingers (e.g., reach-to-grasp) are essential to activities of daily living. Unfortunately, most individuals with chronic hemiparetic stroke have lost such ability due to the stereotypical muscle coactivation patterns between shoulder abductor, elbow flexor, and wrist/finger flexors, commonly referred to as the flexion synergy (Dewald et al., 1995; Sukal et al., 2007; Miller and Dewald, 2012; Lan et al., 2014; Ellis et al., 2016). Due to the expression of the flexion synergy, many individuals find it harder or even impossible to open the hand and/or grasp an object when lifting the paretic arm at the same time (Miller and Dewald, 2012; Lan et al., 2014). To overcome this difficulty, past studies have implemented statistical models to learn and translate the electrical biosignals [e.g., electroencephalogram (EEG) or electromyogram (EMG)] into control signals of external devices, such as robotic exoskeletons (Collinger et al., 2013; Hortal et al., 2015) or functional electrical stimulation systems (Moritz et al., 2008; Pohlmeyer et al., 2009; Ethier et al., 2012). While moderate to high accuracies in learning and translating the poststroke EMGs were reported (Sang et al., 2010; Zhang and Zhou, 2012), none of them have given consideration of the deleterious effect of the flexion synergy.

Due to the flexion synergy, poststroke EMG signals recorded at wrist and fingers during functional movements include two components (Miller and Dewald, 2012; Lan et al., 2016). One component is the voluntary EMG signals generated due to the voluntary contraction of wrist and finger muscles; and the other one is the synergy-induced EMG signals generated due to the involuntary contraction of wrist and fingers muscles associated with the activation of shoulder abductor muscles (Miller and Dewald, 2012; Lan et al., 2014). It has been shown that after a stroke synergy-induced EMG signals from the impaired hand can reach to a significant level with increased shoulder abduction (SABD) loading, even when a study participant was instructed to relax the hand (Miller and Dewald, 2012). These synergy-induced EMGs do not represent the intention of hand movements and thus are detrimental to the accurate classification of volitional hand movements (Lan et al., 2011). For example, when the impaired arm was fully supported, i.e., no effect of flexion synergy, the classification of hand movements can reach an overall accuracy of 96% with high-density myoelectric recordings (Zhang and Zhou, 2012), or 95% with bipolar surface EMG recordings (Lan et al., 2011), but when lifting the paretic arm, the overall accuracy drops by 10 to 16% using EMG signals (Lan et al., 2011) or using EEG signals (Yao et al., 2015).

The overall goal of this study is to find out whether it is possible to reduce the impact of the flexion synergy on the classification accuracy of hand movements by removing the synergy-reduced EMG signals from the wrist and fingers muscles. It was noted from earlier findings that voluntary EMGs and synergy-induced EMGs may be generated using different neural pathways. The synergy-induced EMG signals are likely to be delivered *via* slow-conducting, polysynaptic contralesional corticoreticulospinal pathway, resulting in EMG–EMG oscillation in the alpha band (8–13 Hz) between muscles that share the same neural projections (Lan et al., 2016). In contrast, voluntary EMGs are conveyed *via* fast-conducting, monosynaptic corticospinal pathway that produces EMG–EMG oscillation in the beta band (15–30 Hz) (Farmer et al., 1993; Gross et al., 2000; Lan et al., 2016). With coherence analysis of EMGs between the shoulder abductor and wrist/finger muscles, it is possible to differentiate the synergy-induced EMGs from the voluntary EMGs by studying the coherence power during hand movements. It is our hypothesis that the overall classification accuracy in individuals with stroke should be improved after removing the synergy-induced EMGs with a specific filter. Such a filter should be coherence-based and subject-specific due to the expected between-subject variation in the frequency ranges where the coherence of synergy-induced EMGs is significant. Classification accuracies before and after the filtering will be compared and discussed.

### MATERIALS AND METHODS

### Participants

A total of 29 individuals (stroke: 21, control: 8) participated in this study. Participant demographics are listed in **Table 1**. Control participants were age-matched to the stroke participants and reported no history of cerebral vascular accidents. Stroke participants were selected from the Clinical Neuroscience Research Registry that is housed in the Rehabilitation Institute of Chicago, as well as from individuals residing in the Chicago area who wished to participate in the study. Qualified stroke participants met the following inclusion criteria: (1) sustained a unilateral lesion at least 1 year prior to participation in this project; (2) paresis confined to one side; (3) absence of a brainstem and/or cerebellar lesion; (4) absence of severe concurrent medical problems (e.g., cardiorespiratory impairment, changes in management of hypertension); (5) absence of any acute or chronic painful condition in the upper extremities or spine; (6) capacity to provide informed consent; (7) ability to elevate their limb against gravity up to horizontal and to generate some active elbow extension; and (8) Fugl-Meyer Assessment (Fugl-Meyer et al., 1975) within the range of 10–40 out of a possible 66 and 2–5 out of a possible 7 in Chedoke-McMaster Stroke Hand Assessment (Gowland et al., 1993). All subjects gave informed consent for participation in this study, which was approved by the Institutional Review Board of


*M, male; F, female; L, left; R, right; UE, upper extremity; FMA, Fugl-Meyer assessment; CMSAh, Chedoke-McMaster Stroke Assessment (hand).*

*Values are listed as mean* ± *SD (range).*

*a In this experiment, the stroke subjects were tested at the paretic UE, while the control subjects were tested at the dominant UE.*

Northwestern University in accordance with the ethical standards stipulated by the 1964 Declaration of Helsinki for research involving human subjects.

### Equipment and Setup

The experiment was carried out using an arm coordination training 3-D system (ACT3D, **Figure 1A**), which consists of a modified HapticMaster robot (Moog-FCR B. V., the Netherlands), a Biodex chair (Biodex Medical Systems, Shirley, NY, USA), and T-base support system (Biodex Medical Systems, Shirley, NY, USA). The ACT3D allows for free movements in three dimensions and was used to modulate SABD torques applied to the tested arm.

For the experimental setup, each participant was seated in the Biodex chair with the trunk strapped to the back of the chair to prevent unwanted movement of the upper body. The to-be-tested forearm was placed in a forearm orthosis and the fingers/palm rested on a cylinder. The cylinder was rigidly coupled to the end effector of the ACD3D. Surface EMGs were collected using an Avatar physiological recorder (Electrical Geodesics, Inc., Eugene, OR, USA) from intermediate deltoid (mDEL), flexor carpi radialis (FCR), flexor digitorum superficialis (FDS), extensor carpi radialis (ECR), and extensor digitorum communis (EDC). EMGs were sampled at 1,000 Hz and preprocessed with a band-pass filter at a cutoff band of 5–450 Hz.

### Protocol

Prior to the experiment, each participant's maximum SABD torque was measured using a manual dynamometer (Lafayette Instrument Company, Lafayette, IN, USA) placed just proximal to the axis of rotation of the elbow in a limb configuration of 85° SABD, 45° shoulder flexion, and 90° elbow flexion. Participants were presented with a home object and a target object on a monitor in front of them (**Figure 1B**). At the beginning of the task, participants were instructed to find the home object, trigger the trial, and then reach out to the target object. Once the hand arrived at the target, the participant was given 2 s to lift the tested arm and hold the position. After the 2 s, while keeping the arm lifted, the participant was asked to perform one of the following three hand tasks for 5 s in a random order: (1) open the hand with a maximal effort; (2) grasp the cylinder with a maximal effort; (3) no hand movement. All participants performed these three hand tasks with two different SABD loadings equaling to 25 or 50% of the subject's maximum SABD torque. Ten to twelve repetitions of each hand task were performed.

### Data Analysis Coherence

Wavelet coherence was used to examine the linear dependency of two sequences of surface EMGs in the time-frequency domain (Torrence and Compo, 1998) and is considered efficient and reliable in detecting the synchronizing activity between two time series (Daubechies, 1990; Jevrejeva et al., 2003; Grinsted et al., 2004). In this study, the Morlet wavelet was applied for transformation. Monte Carlo simulation methods were used to determine the 5% statistical significance level of the coherence (Grinsted et al., 2004). It is assumed that the EMG time series has a mean power spectrum, which is only considered as significant when it is above the white noise of the background spectrum. And during the holding phase of grasping/opening, the coherence remains relatively steady (Baker et al., 1997; Kilner et al., 1999, 2000). Wavelet coherences were calculated for each of following muscle pairs, i.e., FDS-mDEL, FCR-mDEL, EDC-mDEL, and ECR-mDEL, for each participant and for all three hand tasks.

#### Algorithm

#### *Without Coherence-Based Filter*

All EMG signals were manually segmented and concatenated to exclude the idling EMGs collected between hand tasks. A 250-ms long window was implemented to slide from the beginning to the end of the concatenated EMGs with a 50% increment and a 50% overlap between adjacent moving windows. Within each moving window, features were extracted based on the method proposed by Hudgins (Hudgins et al., 1993). This method proposes four features in the time domain: mean absolute value, zero crossings, slope sign changes, and waveform length (see **Table 2** for the definition of each of these four features).

Linear discriminant analysis (LDA) was used to classify the EMG signals in this study. LDA has been proved to be effective in EMG classification as well as with low computational cost (Scheme and Englehart, 2011). For each of the hand tasks, LDA maximizes the posterior probability of Bayesian equation and assigns the class labels (i.e., hand open, grasp, or relax) with the largest possibility,

$$\operatorname\*{argmax}\_{\boldsymbol{\nu}} p(\boldsymbol{\nu}\_{\boldsymbol{j}} \mid \boldsymbol{\nu}\_{\boldsymbol{j}}) = \frac{p\left(\boldsymbol{\varkappa}\_{\bar{\boldsymbol{y}}} | \boldsymbol{\nu}\_{\boldsymbol{j}}\right) \mathsf{\*} p\left(\boldsymbol{\nu}\_{\boldsymbol{j}}\right)}{\sum\_{i=1, j=1}^{i=n, j=m} p\left(\boldsymbol{\varkappa}\_{\bar{\boldsymbol{y}}} | \boldsymbol{\nu}\_{\boldsymbol{j}}\right) \mathsf{\*} p\left(\boldsymbol{\nu}\_{\boldsymbol{j}}\right)}$$

where *xij* and *n* represent the features and the number of features in the training set, *i* indicated the *i*th feature, *j* indicated the *j*th category, *y* is classification category.

### *With Coherence-Based Filter*

Frequency ranges where coherence was significant were first identified during the three hand tasks. For each participant, significant ranges found in the three hand tasks were merged to determine the cutoff frequency for the band-stop filters (fourth order Butterworth). Separate filters were built specific to each of the two SABD loading levels in this study. Concatenated EMG signals were then preprocessed with these coherence-based filters followed by feature extraction and LDA classification described above.

#### Dataset and Model Evaluation

For each participant, the dataset was split into a training set (75%) and a testing set (25%). The training set was used to train the model to learn and differentiate the EMG patterns of different hand tasks. The model was trained and tested based on a 250-ms long window. Each subject will generate 10–12 trials of 5 s hand movements for three hand tasks (open/grasping/ relax). The total number of the dataset is 1,200–1,440 samples. Tenfold cross-validation was implemented to determine the best model that reported the highest detection accuracy in training

home position and triggered the trial; step 3: to find the target position.



*k, the kth sample; x, the feature;* ∈*, pre-defined threshold; L, window length; |*Δ*xk|, waveform length between two adjacent samples.*

set. The testing set was then used to assess the strength of the model and the extent to which the fitted model could generalize to the future data. The testing set was put aside until the model was complete with training.

Model was evaluated by calculating the classification accuracy within each of hand movement categories, as shown below,

$$\text{accuracy} = \frac{\text{number of correct classification}}{\text{total number of classification}} \text{\%...}$$

The overall classification accuracy is the mean classification accuracy of the three hand tasks.

#### Statistics

A mixed three-way ANOVA with repeated measures was conducted to determine whether loading (SABD25, SABD50), filter (before filtering, after filtering), hand task (relax, open, grasp), and/or their interaction explain the changes in classification accuracies. *Post hoc* comparisons with the Bonferroni adjustment were used to compare within-group differences. Unless specified otherwise, results are presented as mean ± SE. Statistical significance was set at *p* < 0.05. The statistical analysis was performed using the IBM SPSS version 22 software.

### RESULTS

Shoulder abduction loading showed a negative impact on the overall accuracy in the stroke group (**Figure 2**, see *p* values in **Table 3**). Both groups showed high overall accuracy when the tested arm was fully supported, but the accuracy in the stroke group started to drop significantly during conditions where the participants lifted the paretic arm thus generating SABD loads. The control group was not significantly affected by SABD loading.

The result of coherence analysis of EMGs is shown for one stroke participant and one able-bodied individual during grasping while generating a 50% of max SABD load (see **Figure 3**). Significant coherence between wrist/finger flexors and mDEL in the alpha band (8–13 Hz) was found in the stroke participant during the hand grasp task while such activities were mostly absent in the able-bodied individual (**Figure 3**). The increased alpha-band coherence during hand grasping while lifting the arm in the stroke individual suggested a greater level of shared neural drive to both muscles. The global coherence depicted the coherence power aggregated over time as a function of frequency, highlighting the frequency range where coherence power was the most prominent. It is evident from the global coherence figures that this stroke individual had a greater level of EMG–EMG synchronization that centered at around 8 Hz.

Our data also show a significant global coherence between wrist/finger muscles and mDEL for the hand grasping task with SABD loading at 50% of the max torque based on the Monte Carlo simulation approach in each of the participants (see Figure S1 in Supplementary Material). For each participant, the significant frequency range is represented by solid lines whose lengths denote the range and a solid filled circle whose location denotes the peak value. The figure shows that the stroke group has more significant coherence bands in the alpha band than the control group, especially for the more severely impaired individuals. Coherence in the beta band is not evident for the stroke group. Additionally, there is also great variation between subjects and between muscles in the alpha band coherence in the stroke group. For example, the significant coherence in the extensors are either very short (ECR for the severe cases) or very rare (EDC for the severe cases), while coherence in the flexors is generally longer. Across all individuals, no one stroke individual shared the same significant frequency band as the other.

Based on the significant coherence bands found in all three hand tasks, the cutoff frequency for the band-stop filter was determined by the frequency ranges where coherence was significant for each muscle in each participant. This subject-specific, musclespecific coherence-based filter (referred as "coherence filter" below) was applied to the EMGs in each participant to remove the synergy-induced components from the original EMGs. For individuals (*n* = 3) who showed the peak value as the only significant coherence or no significant bands, no coherence filter was applied. Results from these three individuals were not included in the statistic analysis either. **Figure 4** showed the improvement in classification accuracy after applying the coherence filters to the stroke group at SABD50. Overall, 14 subjects showed improvement in the classification accuracy, four subjects showed reduced accuracy after removing the synergy-induced EMGs. A mixed three-way ANOVA found significance in an interaction effect of loading × filters (*p* < 0.05). *Post hoc* analysis found no significant difference in the classification accuracy with the coherence filters applied at SABD25, but significantly greater improvement with the coherence filters at SABD50 (**Table 4**). One subject showed 14% improvement with the filter. It is expected that some individuals may respond very favorably to the filter. Nevertheless, even when eliminating this individual from the analysis, the conclusion still holds as the sample mean at SABD50 without this subject remains significant compared to the one without the filter (*p* = 0.038).

### DISCUSSION

### Novelty and Main Finding

Decoding EMG signals for neural machine interface is a great challenge in the individuals with chronic hemiparetic stroke due to prevalent motor deficits, such as flexion synergy (Lan et al., 2011; Yao et al., 2015). Many studies made great efforts acquiring



better quality of EMGs or features to improve the overall classification accuracy. For example, Zhang and Zhou (2012) have reported that using high-density EMG signals can achieve high classification accuracies in the stroke individuals. Englehart and Hudgins (2003) suggested optimal parameters for feature extraction, such as window length, overlap, increment length to

*P*tb-50 = 1.000

produce models with low bias, and variances that can generalize well to the test data (Scheme and Englehart, 2011). Features in the time domain (Hudgins et al., 1993; Zhou et al., 2007), the frequency domain (Merletti, 1997; Li et al., 2014), and the time-frequency domain (Englehart et al., 1999; Zhou et al., 2007; Nurhazimah et al., 2016) have also been broadly investigated. While improvement of the overall accuracy has been reported, the limitation is that very few studies have given full consideration of the effect of the flexion synergy common in up to 60% of the stroke population nor proposed approaches that can reduce EMG contamination associated with activation of proximal arm muscles (Fougner et al., 2011; Lan et al., 2011).

This study proposed a novel approach to reduce the impact of the flexion synergy on classification of the hand movements in individuals with chronic hemiparetic stroke for future use in neural machine interfaces. A subject-specific and musclespecific coherence-based filter was developed to remove the synergy-induced component in EMG signals collected from the forearm. The subject-specific filter is believed to be more effective in removing individualized synergy-induced EMG component than a filter with a universal cutoff frequency as the expression of flexion synergy on the frequency ranges of significant coherence varies across individuals with different motor impairment severities (see Figure S1 in Supplementary Material). It was found that such filters can significantly improve the classification accuracy during a greater level SABD loading. To our knowledge, this is the first study in the

field that took the effect of the upper-limb flexion synergy during functional reaching, hand opening, and grasping tasks into consideration.

### Impact of Flexion Synergy on Classification Accuracy

It was previously reported that SABD loading had a negative impact on the overall classification accuracy of hand opening using either EMGs (Lan et al., 2011) or EEGs (Yao et al., 2015). Even in the able-bodied individuals, variations in the limb position can have a substantial impact on the robustness of EMG recognition (Fougner et al., 2011). The results in this study confirm these previous findings. It is now understood that the activation of shoulder abductors can result in the involuntary coactivation of the wrist and fingers (Miller and Dewald, 2012), and such involuntary expression of flexion synergy at the hand can be further enhanced by increased SABD loading on the paretic limb. The EMG signals in the flexors increased with SABD loading even when no voluntary hand movement was initiated (Miller and Dewald, 2012). Prior studies suggested that the synergy-induced EMG component might be delivered *via* the contralesional corticoreticulospinal pathway during increasing levels of SABD (Dewald et al., 1995; Miller and Dewald, 2012; Lan et al., 2016) and should not represent the volitional aspect of hand movement, thus resulting in a decrease in classification accuracy. It is worth noting that while the paretic limb was fully supported, the overall classification of the stroke group has an average of 93.4% accuracy, suggesting that the current feature extraction and classifier choice is sufficient to decode myoelectric patterns in the absence of flexion synergy. However, the same feature extraction and classifier choice is less effective in the presence of synergy-induced EMG, such as when lifting the weight of the arm. It also seems that the reduction in classification accuracy, due to increased SABD loading, is not strictly linear since the accuracy at SABD50 only decreased by 1.1% compared with SABD25. However, more and greater SABD loading conditions are needed to confirm the relationship between SABD loading and reduction in classification accuracy.

### Variation in Significant Coherence Frequency Range

For both hand grasp and hand open tasks, there was great between-subject and between-muscle variation in the frequency ranges where the coherence between wrist/finger muscles and mDEL is significant. One explanation is the broad range of stroke severity included in this study. The severity of stroke participants in this study ranges from moderate to severe impairment, as demonstrated by Fugl-Meyer Assessment and Chedoke-McMaster Stroke Hand Assessment (see methods). It is possible that more severe individuals show greater coherence in the alpha band due to the increased reliance on the contralesional corticoreticulospinal pathway, resulting in a greater portion of synergy-induced EMGs at the wrist/fingers during SABD. The between-muscle variation is also evident across individuals. For example, the significant coherence frequency range in the flexors is generally more common than the extensors (Figure S1 in Supplementary Material). It was noted from previous studies that hand muscles receiving projections from the contralesional reticulospinal tract are flexor-facilitated on the impaired side (Davidson and Buford, 2006; Riddle et al., 2009), meaning activation of the flexors using this pathway is much stronger than the extensors. From evidence provided in monkeys which had recovered from a unilateral lesion of the pyramidal tract, it was shown that reticulospinal-induced amplitude and incidence of synaptic inputs to forearm flexors were significantly increased, while inputs to extensors remained unchanged (Baker et al., 2015). For the control group, the significant coherence frequency range is no more than sporadic across all muscles and subjects, indicating the reduced extent of shared neural drive to the shoulder and the hand compared to stroke participants.

### Variation in Classification Improvement

Electromyogram classification from some individuals (e.g., stroke participant 9, see Figure S1 in Supplementary Material) responded more favorably to the filtering process than others (e.g., stroke participant 1). Such difference may be related to the remained volume of ipsilesional corticospinal tract that is

responsible for voluntary hand movements. It is possible that individuals with a more intact ipsilesional corticospinal tract may have smaller room of improvement in classification accuracy. Conversely, individuals with great reliance on the contralesional corticoreticulospinal tract may benefit more from the coherence filter after the synergy-induced EMGs was removed. It is also important to point out that three stroke individuals showed zero improvement. This is because none of them showed significant coherence frequency ranges and, therefore, no filters were applied. Interestingly, four individuals showed reduced classification accuracy after applying the filters. This could be due to the artifact introduced by the filters that may have caused EMG signal attenuation, and EMGs from these four individuals may be particularly sensitive to such an artifact. It could also be due to the fact that these four individuals already had limited voluntary EMGs in the first place thus removing the synergy-induced component brought little change in the overall classification accuracy.

### Scientific Implications and Future Work

A common approach to preparing surface EMGs is to apply a band-pass filter with a cutoff frequency range of 20–450 Hz. This is very much rooted in the previous work by De Luca and colleagues demonstrating that most of the energy related to motion artifacts is in the frequency range from 0 to 20 Hz (De Luca, 2002; De Luca et al., 2010). However, more recent evidence has shown that after stroke the central nervous system might have adopted an alternative motor control strategy that generates neural oscillation in the alpha range (Lan et al., 2014;

#### Table 4 | Mixed three-way ANOVA for improvement in the overall accuracy rate.


Baker et al., 2015). This control strategy may emphasize using neural pathways that produce frequency contents under 20 Hz and hence it was suggested as the potential target for poststroke rehabilitation (Baker et al., 2015). That could imply that the EMG below 20 Hz might contain useful motor information. Thus *selectively* removing the frequency content below 20 Hz might be a more effective approach for pattern recognition of poststroke EMGs and should be studied in more detail. To achieve this goal, future work should first focus on quantifying real-time classification in the stroke group. The present work is an off-line application of the algorithm, and we reported the overall improvement in most stroke individuals. To make this approach more clinically applicable, it is recommended to implement the algorithm online, meaning classification is made while EMGs are generated. We have used a 250ms-long time window for processing the data. This allows for an online classification of the hand movement without creating a sense of delay. Second, the classification platform should be realized with a close-loop connection to an external device (e.g., a robot device or functional electrical stimulator), which receives the classification signal and generates movements or activates relevant muscles in the wrist and fingers. Eventually, such a platform requires a training period which acquires subject-specific data to train the classifier and a real-time testing period.

### REFERENCES


Future work should also extend the current experimental protocol to multiple abduction levels to better understand the benefits of applying filters to functional hand movements. Furthermore, integrating neuroimaging approaches into the current line of research can further help us to gain scientific insight into the possible use of ipsilesional corticospinal and contralesional corticoreticulospinal tracts after stroke. This is likely to bring new perspectives to a more effective subject-specific future application of neural machine interfaces within the context of flexion synergy.

### ETHICS STATEMENT

This study was carried out in accordance with the recommendations of the Institutional Review Board of Northwestern University with written informed consent from all subjects. All subjects gave written informed consent in accordance with the Declaration of Helsinki. The protocol was approved by the "the Institutional Review Board of Northwestern University."

### AUTHOR CONTRIBUTIONS

YL designed the study, developed the methodology, performed the analysis, and wrote the manuscript under the guidance of JY and JD. Both JY and JD contributed extensively to revising and finalizing this manuscript as well as to the development of the original research idea.

### FUNDING

The study was supported by the National Institutes of Health grant: NIH R01HD039343.

### SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at http://journal.frontiersin.org/article/10.3389/fbioe.2017.00039/ full#supplementary-material.


**Conflict of Interest Statement:** The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

*Copyright © 2017 Lan, Yao and Dewald. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

*Vrajeshri Patel <sup>1</sup> , Poojita Thukral <sup>2</sup> , Martin K. Burns <sup>1</sup> , Ionut Florescu1 , Rajarathnam Chandramouli <sup>1</sup> and Ramana Vinjamuri <sup>1</sup> \**

*1Sensorimotor Control Laboratory, Department of Biomedical Engineering, Chemistry, and Biological Sciences, Stevens Institute of Technology, Hoboken, NJ, USA, 2Department of Electrical and Computer Engineering, Carnegie Mellon University, Pittsburgh, PA, USA*

Recently, the need for more secure identity verification systems has driven researchers to explore other sources of biometrics. This includes iris patterns, palm print, hand geo metry, facial recognition, and movement patterns (hand motion, gait, and eye movements). Identity verification systems may benefit from the complexity of human movement that integrates multiple levels of control (neural, muscular, and kinematic). Using principal component analysis, we extracted spatiotemporal hand synergies (movement synergies) from an object grasping dataset to explore their use as a potential biometric. These movement synergies are in the form of joint angular velocity profiles of 10 joints. We explored the effect of joint type, digit, number of objects, and grasp type. In its best configuration, movement synergies achieved an equal error rate of 8.19%. While movement synergies can be integrated into an identity verification system with motion capture ability, we also explored a camera-ready version of hand synergies—postural synergies. In this proof of concept system, postural synergies performed well, but only when specific postures were chosen. Based on these results, hand synergies show promise as a potential biometric that can be combined with other hand-based biometrics for improved security.

Keywords: biometrics, human hand, grasping, synergies, principal component analysis

# INTRODUCTION

Identity theft has become a common crime that affects about 7% of the population each year (Harrell, 2015). Passwords and social security numbers are the most common forms of identity verification. Biometrics, or recordable biological measurements, have also been integrated into identity verification systems (Jain et al., 2007). Although there is still much progress to be made in the field (Jain et al., 2006), biometrics eliminates the need for password memorization and offer heightened security. Researched biometrics include palm prints/fingerprints (Jain et al., 2007), iris or retina scans (Hill, 2002), face images (Heo and Savvides, 2012), and electroencephalography signals (Ruiz-Blondet et al., 2016). Recently, researchers have explored the potential of other hand biometrics, including vein patterns (Wang et al., 2008), hand geometry (de-Santos-Sierra et al., 2011), and palm prints (Kumar et al., 2003). The common factor of these identity verification methods is their basis on statically recorded information, usually in the form of a feature matrix, which is then encrypted on a server. While the complexity of feature matrix derivation, encryption methods, and server safety sets the level of security, the fact remains that information can be stolen and used. Recently, Experian, a company commonly used for credit checks and even identity theft protection, was the target of a hack, resulting in the theft of records for approximately 15 million people (Nasr, 2015). This included encrypted social security numbers, passport numbers,

#### *Edited by:*

*Alessandro Tognetti, University of Pisa, Italy*

#### *Reviewed by:*

*Fu Qiushi, Arizona State University, USA Claudia Casellato, Polytechnic University of Milan, Italy*

*\*Correspondence:*

*Ramana Vinjamuri ramana.vinjamuri@stevens.edu*

#### *Specialty section:*

*This article was submitted to Bionics and Biomimetics, a section of the journal Frontiers in Bioengineering and Biotechnology*

*Received: 25 January 2017 Accepted: 03 April 2017 Published: 02 May 2017*

#### *Citation:*

*Patel V, Thukral P, Burns MK, Florescu I, Chandramouli R and Vinjamuri R (2017) Hand Grasping Synergies As Biometrics. Front. Bioeng. Biotechnol. 5:26. doi: 10.3389/fbioe.2017.00026*

and driver's license numbers. Soon after, a data breach of The United States Office of Personnel Management led to the loss of social security numbers, fingerprints, and other identifiable information, of 21.5 million people (Nakashina, 2015). Moreover, certain biometrics, such as iris scans, can potentially be forged in order to gain entry into biometric-based systems (Ruiz-Albacete et al., 2008). These reports reveal the need for identity verifications systems that do not only rely on static images, scans, or numbers.

Human movement may seem as simple as multiple joints working in parallel to accomplish a task. However, the complete architecture of motor control is still not understood (Scott, 2012). Based on an individual's anatomy, different neural commands are required to complete the same task across individuals. Furthermore, each individual has advanced his/her motor skills over years of learning. This includes acquisition of basic grasps as an infant to more dexterous motor control such as piano playing and typing. Importantly, these characteristics cannot be forgotten or voluntarily reproduced by another individual. Recently, researchers have taken advantage of the complexity in human movement for use in identity verification systems. Keystroke dynamics involves characterizing keyboard inputs, such as keystroke latencies and durations, finger placement, and finger pressure, to determine a user's unique typing characteristics (Monrose and Rubin, 2000). Optimized string inputs (i.e., alphanumeric, unstructured vs structured) and classification algorithms have propelled this field of biometrics to commercial use. However, factors, such as emotional state (Epp et al., 2011), keyboard type, and user position, may affect performance (Banerjee and Woodard, 2012). In an attempt to reduce the lengthy time needed to register and identify a user when using keystroke dynamics, Roth et al. (2014) introduced a typing posture biometric that characterizes the shape and position of hands during typing and later introduced keystroke sound (Roth et al., 2015). As a relatively new biometric, these typing characteristics still need to be optimized in order to reduce equal error rate (EER). Arm movement biometrics may be a more appropriate option for individuals without basic typing skills. In-air signatures captured either by camera (Mendels et al., 2014) and smartphones (Casanova et al., 2010; Blanco-Gonzalo et al., 2014) have shown promising results. In an attempt to leverage the complexity of hand movements, in-air signatures of a person's name (Kamel et al., 2008) or a unique password expressed through American Sign Language (Fong et al., 2013) and touchscreen dynamic (Sae-Bae et al., 2012; Frank et al., 2013) have also been introduced.

Synergy-based movement theory hypothesizes that some commonly used movement patterns are encoded in the central nervous system (CNS). These movement patterns, or synergies, reduce the degrees of freedom that the CNS must control and can be combined to perform more complicated movements. The human hand is one of the most mechanically complex end effectors in the human body and has been researched in relation to synergy-based movement theory for many years. Object grasping is one hand-related activity that is commonly performed throughout the day. It requires coordinated control of four fingers and the thumb to produce postures and force vectors required to grasp and lift objects. It also requires integrating various sensory information (visual, proprioceptive) and planned velocity control (distance and forced dependent) that begins premovement (MacKenzie and Iberall, 1994). It has been found that certain grasping traits maintain high intra-subject similarity and high intersubject variability (Reilmann et al., 2001; Wong and Whishaw, 2004), potentially stemming from different neural and mechanical mechanisms. We and others have previously explored hand synergies and have applied it to motor control models and prosthetics (Santello et al., 2002; Weiss and Flanders, 2004; Vinjamuri et al., 2010; Bicchi et al., 2011). Here, we explore hand synergies' potential role as biometrics. Often, it is found that the first synergy is characterized by flexion in hand joints, mimicking a power grasp (Santello et al., 2002; Vinjamuri et al., 2010). However, as previously mentioned, motor control is affected by an individual's unique experience and anatomy. For these reasons, synergy-based biometrics may offer unique advantages compared to static and hand geometry-based biometrics.

In this study, we explore 10 synergies extracted from grasping data for their potential use as biometrics. Each is tested for specificity and sensitivity. We hypothesize that hand synergies contain identifiable information that is robust enough to be incorporated into identity verification systems. As a proof of concept, we also develop a system that can easily be integrated into a camera phone. Subjects pose the end posture of each movement synergy. These 10 "postural synergies" are photographed and tested as potential biometrics.

### MATERIALS AND METHODS

### Overview

For this study, 10 individuals (5 females, 5 males; 9 right handed, 1 left handed; mean age 21.7 ± 1.95) were recruited under Stevens Institute of Technology Institutional Review Board approval. Subjects performed grasping tasks while wearing a data glove that records hand kinematics. Using principal component analysis (PCA), spatiotemporal synergies were then immediately derived from these data. These spatiotemporal synergies provided us with two forms of synergies that could be used as biometrics: movement synergies and postural synergies. Movement synergies for biometrics were tested using recorded data. Postural synergies were displayed to the subject, who practiced and performed each posture. Postural synergies for biometrics were tested using photographed hand images of these postures. In addition to their own postural synergies, five subjects practiced and performed another subject's postural synergies to be tested as false entries.

Five of the 10 subjects returned 4–8 months later for a followup session of the motion-recording portion of the experiment. This additional dataset was used as additional entry tests for movement synergies.

### Data Capture

Subjects wore a right or left CyberGlove (CyberGlove Systems LLC, San Jose, CA, USA) that records joint angles. For this study, we used 10/18 sensors. These sensors measure the interphalangeal and metacarpophalangeal (MCP) joints of the thumb and MCP and proximal interphalangeal (PIP) joints of the four fingers. Abduction sensors were not used in order to keep replications of a synergy posture simpler. Wrist sensors were not used because they do not pertain to the hand. Data were captured at 125 Hz using a custom-built LabVIEW (National Instruments Corporation, Austin, TX, USA) system. The glove was calibrated for each subject using custom goniometers ranging from −10° to 90°.

An overview of the data capture and synergy testing programs is presented in **Figure 1**. For each subject, the grasping dataset consisted of rapidly grasping 25 objects (3 repetitions) that span 6 types of grasps (power, precision, hook, tripod, lateral key, and spherical). The selected objects were those found in activities of daily living. Each grasp type had four objects associated with it, with the exception of "hook," which had five objects. The object was placed 40 cm away from the midline of the body, and the hand was placed in an initial resting position 20 cm to the right or left (depending on self-reported hand dominancy) of the body midline (**Figure 1A**). The subject was asked to rapidly grasp the object after an audio "go" signal and to hold the grasp until an audio "stop" signal was heard. This concluded the grasping portion of the experiment. After data processing, a subject's postural synergies were shown, after which the subject performed each posture. Images of postural synergies were taken against a green background (for chroma keying) using an 8 megapixel mobile phone camera positioned approximately 38 cm above the hand. During image capture, subjects wore a wrist band to prevent wrist extension/flexion and deviation.

### Synergy Derivation

Synergy-based movement can be generated using a convolutive mixture model (Vinjamuri et al., 2010; Patel et al., 2015). In this model, an impulse originating in higher levels of the CNS passes through a set of linear filters that relate neural and biomechanical structures (possibly cerebellum, basal ganglia, spinal cord, and muscles). Spatiotemporal synergies, represented by joint angular velocity profiles that relate the activity of multiple joints over time, are one form of response from these filters. Here, we consider the rapid grasp condition to inherently contain feedforward direct command signals because they result from minimum or negligible sensory input. Based on previous work, we have found that synergies derived from PCA are able to better capture inherent joint patterns that can reconstruct movements (Patel et al., 2016). Thus, PCA is used to extract the principle components (PCs) of the dataset. Each PC is considered a synergy because it captures a common spatiotemporal pattern that exists across all hand joints.

For each grasp trial, data from movement onset (first time a joint reaches 5% of peak velocity) to grasp completion were extracted (last time 5% of peak velocity is reached). Across all subjects, the maximum time required to complete a grasp was 1.208 s, or 151 samples. Angular velocity profiles were derived from these data to create an angular velocity matrix, *V* [25 × 1,510 (10 joints × 151 samples)]. Singular value decomposition was performed on *V*:

#### svd(*V U* )= ∑*R*

such that *U*'*U* is a 25 × 25 identity matrix, *R* is a 1,510 × 1,510 matrix such that *RR*' is a 25 × 25 identity matrix. Σ is 25 × 25 diagonal matrix: diag{λ1, λ2, …, λ25} with λ<sup>1</sup> ≥ λ<sup>2</sup> ≥ Λ ≥ λ<sup>25</sup> ≥ 0. We then reduce matrix *V* to *V˜* by replacing Σ with Σ*S*, which contains only the *n* largest singular values, λ1, …, λ*n*. All the other singular values are replaced by 0s. The approximation matrix *V˜* can be written as

$$\bar{V} = U\_{\rm s} \operatorname{diag} \{ \lambda\_1, \dots, \lambda\_n \} R\_{\rm s},$$

where *US* is a 25 × *n* matrix containing the first *n* columns of *U* and *RS* is a 25 × 1,510 matrix containing the first *n* columns of *R*. Each column of *RS* is called a PC. For the purpose of dimensionality reduction, we perform our analysis on the first 10 principal components, or synergies, only (*n* = 10).

#### Movement Synergies

Spatiotemporal synergies derived using PCA represent movement patterns over time. In order to test movement synergies for biometrics, we derived three sets of synergies, one from each grasping repetition. Each synergy set contains 10 synergies. The first set was used as the "template" that is stored for user registration (**Figure 1B**). The remaining two sets (entry synergy sets) were used to test the authentication process. The five subjects that were later retested had an additional four entry synergy sets. We used summed correlation across all joints (maximum correlation is equal to 10) to determine the similarity between a template synergy set and entry synergy set. A minimum correlation, *T*m, level is required to enter the system (**Figure 1D**). EER determined by the intersection of false positive rate (false positive/total number of false entries) and false negative rate (false negative/total number of true entries) was used to determine optimum thresholds.

When comparing a template synergy to entry synergy, the following preprocessing steps were performed. Each dataset results in 10 synergies ranked according to the variance they account for. However, this ranking may vary across datasets. Thus, each template synergy was iteratively compared to all synergies from an entry synergy set and then paired with the synergy with highest correlation. After a potential entry synergy is paired with a template synergy, it is removed from the synergy set, so remaining synergies can be paired. Additionally, up to a ±20 sample (160 ms) time shift (zero-padding before or after) was used to account for intersubject time variation. To test correlation by chance, we randomly reshuffled the velocity profile of each joint in a synergy using eight time bins. Each time bin was 20 samples long, with the exception of the last time bin that was 11 samples long. Then, the same comparison approach described above was used to determine correlation by chance.

Each subject's synergies were also used as false entries for all other subjects. Thus, across all subjects, the performance of each synergy was tested under 40 true conditions (5 subjects × 2 attempts + 5 subjects × 6 attempts) and 180 false conditions (9 false synergies × 2 repetitions × 10 subjects). Note that subjects with four attempts are those who attended the follow-up session. To optimize correlation measurements, we explored the effect of different configurations of the synergy template (i.e., using only certain joints). We tested the effect of joint type (MCP vs PIP) and finger (thumb, index, middle, ring, and pinky). For each "configuration," correlation values between the adjusted entry synergy and the adjusted template synergy was measured and averaged across subjects. A one-way analysis of variance test, with Tukey–Kramer *post hoc* was used to determine if any configurations significantly increased (*p* < 0.05) correlation. Although this study explores the use of synergies for biometrics, it is important to consider an appropriate time limit for user registration and entry. Thus, we examined the effect of object type and number of objects for potential reduction in data acquisition. When analyzing number of objects, each grasp was first ranked according to the following procedure. First, 25 new synergy sets were extracted, each set omitting a single grasp. For each new set, we determined the average correlation between false synergy sets and the test synergy. Objects that decreased correlation the most were prioritized (ranked highest). After each of the 25 objects were ranked, new synergies were derived by iteratively omitting grasps.

#### Postural Synergies

Postural synergies represent the final position of each movement synergy (velocity profiles are integrated over time to determine final position of each joint). After grasp data were collected, synergies were immediately extracted from all three repetitions (**Figure 1C**). Each synergy was multiplied by a maximum possible gain under the following criteria: (1) final posture fell within normal range of movement and (2) the majority of a finger did not cross the palm. A virtual hand, built using Simulink 3D Animation toolbox (MathWorks, LLC) was used to display front and side views of the resulting hand posture. Here, we checked whether criterion 2 (from above) was met. Based on preliminary testing, we found that too much flexion in a finger would cause the image analysis procedure to incorrectly omit a finger. Thus, if any of the distal interphalangeal (DIP) joints on the virtual hand cross the upper palm edge, then the weight of the synergy was reduced in 0.01 increments until the DIP joint no longer crossed the palm. Once the synergy postures were finalized by the experimenter, they were shown to the subject. Because subjects were asked to perform these postures for the first time, we allowed an initial practice time for each posture to ensure correct movements (approximately 10 min total). Each synergy was performed and photographed (10 template postures). These were used as template images for each synergy. Then, two more repetitions were taken of each postural synergy (20 entry images). Subjects were encouraged to maintain similarity and "approach" when performing each synergy. "Approach" refers to the order each finger was flexed to achieve the target posture. Additionally, preliminary work showed the thumb to cause excessive variation in images. Thus, subjects were asked to keep the thumb in a natural straight position. In this experimental setting, the hand model in a specific synergy posture was displayed as users attempted to perform each posture. However, in realistic settings, we would expect users to only choose up to three postural synergies to memorize and use for entry.

Image analysis of template and entry postural synergies was performed using the Image Analysis Toolbox in MATLAB. In this preliminary work, we controlled lighting to prevent shadows around the hand. Because the focus of this study is to use synergy-based differences across individuals, the effects of other commonly used hand biometrics (skin color, palm/finger size, and vein/texture attributes) were eliminated with the following steps. Preprocessing image analysis steps include: background removal, conversion to binary image (removes skin color and vein/text attributes), wrist cropping, and image centroid calculation. An example of the resulting image is shown in **Figure 2A** (green dot indicates centroid). Then, the edge of the hand figure that includes only digits portions was taken. Importantly, this outline is a result of MCP and PIP extension/flexion movements, but also reflects natural abduction/adduction movements that occur between fingers. The distance between the palm centroid and each point of the outline is measured and normalized to remove the effect of different hand sizes (**Figure 2B**). This involves finding the shortest distance from the centroid and for each finger.

dividing all other points from this distance. The resulting values describe a relative hand posture. If an imposter was able to reproduce the exact same ratio of flexion across the fingers as another use, abduction tendencies and enslaving magnitudes, which are unique to an individual, would still cause subtle differences. Each finger is then separated from the outline; this finger profile is used as a basis for comparison (**Figure 2C**). It should be noted that while the distances are normalized, the number of data points dedicated to a finger is not set. Thus, an individual with a narrower finger or the same individual with a lesser degree of flexion may decrease the number of data points detected for a finger (and *vice versa*). This can be seen 2 C where the blue line represents the outline of the ring finger from the template posture. In entry Attempts 1 and 2, there are fewer data points, possibly resulting from less abduction or less flexion. The Euclidian error between a template finger outline and an entry finger outline is summed across fingers. A maximum error threshold, *T*p, is used to determine if the entry postural synergy matches the template postural synergy.

Across all subjects, each of the 10 synergies was tested under 20 true conditions (10 subjects × 2 attempts). Five of the subjects practiced and reproduced postural synergies of another subject; thus, each of the 10 synergies was tested under 10 false conditions (5 subjects × 2 attempts). Because certain fingers of the hand have more dexterity (i.e., index) or enslavement (i.e., ring) than others, we examined which fingers are sources of greater error in true (authentic) and false (imposter) condition. One-way ANOVA was used to compare Euclidian errors between the five fingers. For each individual finger, we then examined whether imposter attempts average equal errors as authentic attempts using a Student's *t*-test; significance was set at *p* < 0.05.

### RESULTS

### Hand Synergies

Ten synergies were extracted from grasping data. A movement synergy is defined by velocity profiles for each of the 10 joints. The cumulating fraction of variance accounted for by these synergies is presented in **Figure 3**. Across all subjects, the first synergy accounted for an average of 54% of the variance. An example of synergy 1 from a representative subject is provided in **Figure 4**,

blue. For comparison, the averaged velocity profile across all 25 grasps is provided with the red trace, with SD depicted by the red shaded regions. Both, the averaged profile and the synergy profile capture the overall pattern. Please note that for visual purpose only, the synergy profile in **Figure 4** was multiplied by a gain, such that both the averaged profile and the synergy profile would be on the same scale. As depicted in the figure, there also exists a pattern across and between MCP (left) and PIP (right) joints. The postural synergy representation of this particular movement synergy is shown by the hand image at the bottom right of **Figure 4**. For movement synergies, each of the 10 velocity profiles needs to adequately match. For postural synergies, images of a hand in these hand configurations need to adequately match.

### Biometric System Based on Movement Synergies

To explore the use of movement synergies for biometrics, we first assessed the ability of an individual to reproduce the same synergy profile across repetitions. A maximum of correlation of 100% represents a perfect match. Percent of maximum correlation for different finger configurations (all joints, MCP joints only, PIP joints only, and MCP and PIP joints of each finger removed) across all synergies is shown in **Figure 5**. All synergies

FIGURE 4 | The red line shows the averaged joint movement across all 25 grasps in one repetition. SD is provided by the red shaded region. Metacarpophalangeal (MCP) and proximal interphalangeal (PIP) joints of the thumb (T), index (I), middle (M), ring (R), and pinky (P) are shown on the left and right, respectively. To show how the first synergy compares to the average movement, synergy 1 of this particular subject is overlaid by the blue line. Note that this synergy has been multiplied by a gain so that both the red and blue traces match for visual purposes only. The end posture of this synergy is shown on the bottom right. This is considered a postural synergy.

FIGURE 5 | Across subjects, the mean (bars) and SD (error bars) of correlation values between template and entry movement synergies are shown. Values are shown for different joint configurations: all joints are used, only metacarpophalangeal (MCP) joints are used, only proximal interphalangeal (PIP) joints are used, thumb joints are removed, index joints are removed, middle joints are removed, ring joints are removed, and pinky joints are removed. Black bars show correlation by chance. Green dots show the calculated equal error rate (EER) for each synergy and configuration. Synergy 1 has the highest correlation and lowest EER values. All synergies had correlations above chance level (black bars).

fell above correlation by chance levels (**Figure 5**, black bars). Results show that the first synergy had the highest correlation, indicating that it was most reproducible across repetitions. Within synergies 1 and 2, no significant difference was found between the different finger configurations; however, removing the thumb resulted in slightly higher correlations. For synergies that did show significant differences across different configurations, *p*-values are provided in **Table 1**. Namely, higher order

#### TABLE 1 | Significant differences between different synergy configurations.


*MCP, metacarpophalangeal; PIP, proximal interphalangeal.*

*p Values from one-way ANOVA of correlations presented in Figure 5 are given.* 

*Only synergies that expressed a significant difference in at least one comparison are provided. Bolded values indicate a p-value* < *0.05.*

synergies (synergies 6–10) had significantly lower correlation scores when more joints were used compared to MCP and PIP configurations that only used five joints each. Green dots in **Figure 5** show EER for each synergy and finger configuration. Only synergy 1 fell in an acceptable EER range for biometrics, performing best when pinky MCP and PIP joints were removed (EER = 8.19%) and when all joints were used (EER = 10%). False acceptance rate (FAR) and false rejection rates (FRR) for synergy 1, using all joints is shown in **Figure 6**. The threshold at the EER point is 70%. Using both synergies 1 and 2 as part of the biometric key did not significantly improve results.

In this experimental setup, 25 objects were used to elicit varying grasping patterns. However, in an actual biometrics application, an individual would not be expected to grasp 25 objects. Using EER, we determined the minimum number of objects required to extract and match the first synergy. The overall trend seen in **Figure 7A** shows increasing the number of objects does not significantly decrease EER. We found that removing the pinky MCP and PIP joints (red) further reduced EER compared to when all joints are used (blue). Synergies extracted from only six of the highest ranking objects produced an EER of 10%. These objects were: screwdriver, water bottle, CD, petri dish, bag handle, and bracelet.

In an attempt to determine if certain grasp types elicit unique hand patterns more robustly, we then extracted synergies from tasks that used certain grasp types. Using these synergies, EER of cylindrical, precision, hook, lateral key, spherical, and tripod grasps is shown in **Figure 7B**. Synergies extracted from lateral key grasps and tripod grasps showed the lowest EER of 9.17 and 10%, respectively, when the pinky was removed.

### Proof of Concept: Biometric System Based on Postural Synergies

For each subject, 10 postural synergies were determined by the end posture of each movement synergy. **Figure 8** shows front and side views of these synergies for subjects 2, 6, and 10. Similar to previous research in kinematic hand synergies, the first synergy in all subjects was characterized by flexion in MCP and PIP joints, analogous to a power grasp. Synergy 2 was also characterized by

FIGURE 6 | False rejection rate (FRR) and false acceptance rate (FAR) of synergy 1, when using all 10 joints. Equal error rate (EER) is calculated at the intersection of these plots. EER is 10% at a threshold of 70% correlation.

FIGURE 7 | (A) Equal error rate (EER) when increasing number of objects are used. Using the first six highest ranked objects produces an EER of 10% when pinky is removed. (B) Using only five lateral key grasps produces an EER of 9.17% when the pinky is removed.

flexion in MCP and PIP joints, but the magnitude at each finger was less consistent across subjects. Synergies 3–9 did not show consistent patterns across fingers and subjects. However, synergy 10 consisted of MCP extension and PIP flexion in all subjects.

Although each posture is characterized by MCP and PIP angles for each finger, the imaged posture is evaluated by finger endpoint (i.e., distance of finger peak to palm center). We evaluated the error of each finger across all synergy postures for both authentic (user replicates his/her own posture) and imposter (imposter replicates another user's posture) cases. Individual finger results are provided in **Figure 9**. For authentic attempts, no significant differences were found across fingers (one-way ANOVA, *p* > 0.05). For imposter attempts, the pinky (indicated by red star) showed significant differences (one-way ANOVA followed by multiple comparisons using Tukey–Kramer test) compared to the thumb (*p* = 0.0459), index (*p* = 0.0009), middle (*p* = 0.0001), and ring (*p* = 0.0401) fingers. In all fingers (black star), imposter attempts averaged significantly greater errors than authentic attempts: thumb (*p* = 2e−5), index (*p* = 4e−5), middle (*p* = 1.5e−4), ring (*p* = 0.0017), and pinky (*p* = 0.0018).

The performance of using each postural synergy as a biometric key is presented in **Table 2**. The best performance was achieved by synergy 2 with an EER of 2.5%. We further tested the system by using the first, second, and third best postures (least replication error) from each subject. This resulted in EER of 0, 2.5, 10, and 10% for one, two, three, and four postures, respectively.

### DISCUSSION

### Movement Synergies As Biometrics

All 10 movement synergies were tested as a potential biometric. Results indicate that only synergy 1 is similar enough across repetitions to qualify as a biometric. Synergy 1, accounting for a mean variance of ~54%, characterizes the most general pattern from the grasping dataset. Although it describes a hand-closing pattern for all subjects, results show that each fingers' flexion rate differs across subjects. Thus, it holds characteristic unique to an

FIGURE 9 | For each finger, the mean Euclidian error between template and entry postural synergies is shown. Blue bars represent mean taken across all authentic attempts, and red bars represent mean taken across all imposter attempts. Error bars represent 1 SD. For authentic attempts, no significant differences are seen across fingers. For imposter attempts, the pinky showed significantly less error (indicated by the red star, *p* < 0.05) than all other fingers. In every finger, imposter attempts averaged significantly greater (indicated by the black star, *p* < 0.01) error than authentic attempts.

TABLE 2 | Equal error rate (EER) for each postural synergy.


*EER when each of the 10 postural synergies are used alone as a biometric. Synergy 2 has the best performance with an EER of 2.5%. When the best postural synergy (subject had least error reproducing the posture) was chosen as the biometric key, the system had an EER of 0%.*

individual that can only be determined by grasping everyday objects. Synergy 2 showed some consistency across repetitions but did not perform well enough to be used as a biometric itself. When combining synergies 1 and 2, EER dropped to 9.72%. Further, testing should be done to determine the benefits of including two synergies. Synergies 3–10 were not reproducible enough to be considered for biometrics.

Five of the 10 subjects from this dataset were tested 4–8 months after initial evaluation. The present analysis incorporated all attempts when calculating EER, but it is important to consider the effects of short-term stability (or instability) on these values. We used correlation values to measure short-term stability of the five subjects who participated in the follow-up session. **Figure 10** shows that subjects 4 and 9 were not able to perform

with enough similarity in the follow-up session, while subjects 1, 3, and 7 have consistent performance. Further analysis of these subject's synergy profiles will help us determine which movement parameters were not reproducible (i.e., peak velocity amplitudes or time domain features).

We expected that eliminating highly individuated digits, such as the thumb or index (Ingram et al., 2008), would decrease EER. However, results showed an increase in EER for synergy 1 (red points in **Figure 5**) when these digits were not included. Further analysis showed that eliminating the thumb decreased FRR and increased FAR. This indicates that the thumb's movement is difficult to reproduce within a subject but contributes to the synergy's uniqueness. Eliminating the index finger slightly increased FRR but did not change FAR. This indicates the index finger is more easily reproducible within a subject. Surprisingly, removing the pinky decreased EER by decreasing FAR. This indicates that imposter attempts were able to reproduce the pinky movement. It has been shown that order of digit contact is often radial to ulnar (Schettino et al., 2013) unless all digits are required for successful grasp (i.e., in cylindrical grasp). In the majority of tasks, the pinky may have followed as a supportive digit, resulting in less complex control.

In terms of choosing optimal objects to decrease system access time, we found that using the six highest ranked objects achieved an EER of 10% (when pinky is removed). As seen in **Figure 7**, the ranking procedure does not show a clear trend when using one to six objects. The ranking order used in this analysis may not been optimal for a low number of objects because PCA depends on multiple objects. We also found that using the five lateral key grasp objects achieved an EER of 9.17%. Compared to all other grasp types, using lateral key grasps decreased FRR, but increased FAR. This indicates that lateral key grasps were more reproducible within subjects as well as across subjects when compared to the other grasp types.

Our results indicate movement synergies hold unique properties that cannot be reproduced by imposter attempts. In most of the above mentioned scenarios, FAR values were minimal, while FRR values were high; this indicates the sensitivity of the system needs to improve. We believe including additional features that are often maintained in an individual (reaction time, movement time) as well as individualized threshold values will increase performance of the system. Other movement-based biometric systems have used dynamic time warping (DTW) to compare two time profiles. In these studies, testing a pre-determined set of gestures resulted in EER of l.89% (Wu et al., 2013) and 2.58% (Scott, 2012). DTW is currently being evaluated as a potential synergy comparator.

### Postural Synergies As Biometrics

In the postural synergies biometric system, subjects would be required to perform up to three memorized postures in order to gain access to the system. From the 10 postural synergies analyzed, no obvious optimal postural synergy(s) could be determined. EER rates as low as 2.5% show postural synergies have potential as a biometric.

Postural synergies have two layers of uniqueness. First, the movement synergies derived from the grasp data extracts motion patterns that exemplify joint relationship during grasping. Although synergies 3–10 were not reproducible within subjects, they greatly varied across subjects. Second, the ability to conform the hand to the resulting posture, or postural synergy, further exposes patterns in the hand (i.e., unintentional abduction angles resulting from joint flexion). Other posture-based biometric systems employ commonly used postures, such as ASL postures, to spell out a password. In one such study, Fong et al. (2013)

concluded that recognizing behavioral patterns is more difficult than recognizing the actual hand shape. If these behavioral patterns are already extracted by the synergies, then further improving the hand shape recognition methods may greatly improve the system. Hand geometry-based studies have used finger width values, variance corrections (de-Santos-Sierra et al., 2011), and hand rotation corrections (Yoruk et al., 2006) to optimize feature extraction and matching performance. Combining traditional geometry-based and intensity-based (hand color) hand biometric methods, we can further improve the postural synergy biometric system.

### Hand Synergies As Biometrics

This study aimed to test the use of movement and postural synergies for use in biometrics. Results are comparable to other movement-based biometric systems that do not use subjectspecific passwords (Matsuo et al., 2007; Liu et al., 2009; Bailador et al., 2011) but still below keystroke dynamic performance values (review in Banerjee and Woodard, 2012). Movement synergies reached its best performance at an EER of 8.19%. Postural synergies reached its best performance at an EER of 0 and 2.5%. However, FAR of the system still needs to improve in order to meet adequate performance for biometric use.

Using movement synergies as a biometric requires the use of a hand data glove that would need to be calibrated for all hand sizes. In our procedure, we performed an exhaustive calibration, which would not be ideal for real-world use. We are currently developing a low cost, easy-to-deploy hand data glove that may be more suitable. It is important to keep in mind that this type of biometric cannot be used for rapid user verification. Instead, the user must perform an action in order to gain access in an environment equipped with a computer, data glove, specific grasp objects, etc. Thus, it may only be warranted for higher-security applications that combine multiple biometrics. Like keystroke dynamics and typing posture biometrics, movement synergies do not require the user to memorize a password that has the potential to be forgotten or stolen. However, all three systems require a rather structured setup (i.e., computer to capture data, same keyboard, and similar user positions). Finally, as with all movement related biometrics, template passwords need to be updated due to the effects of aging on motor skills. The postural synergy proof of

### REFERENCES


concept showed that performing postures in front of camera may offer a quicker and more mobile platform. This type of hand biometric can potentially be applied to computers/phones with a camera. Similar to performing in-air signatures, the postures would have to be memorized; however, because of the complexity of synergy hand postures, it may not be necessary to have a private area in order to perform the postures.

Future work includes exhaustively testing both systems with a greater sample size and testing short-term stability and incorporating other commonly used biometric methods such as using averaged time velocity profiles as a synergy, DTW for synergy comparisons, and hand geometry measurements for postural synergies. Finally, we will test whether using the source of hand synergies, i.e., neural signals collected from electroencephalography, can improve performance.

### ETHICS STATEMENT

The study was approved by IRB at Stevens Institute of Technology.

### AUTHOR CONTRIBUTIONS

VP helped design the experiment and recruit subjects, collected and analyzed data, and participated in writing the manuscript. PT helped design the experiment, collected and analyzed data, and participated in writing the manuscript. MB helped create the data collection software and contributed toward revising the manuscript. IF contributed toward statistical analysis. RC contributed toward the design of methods and participated in writing the manuscript. RV directed subject recruitment procedures, developed experiment framework and procedures, and participated in writing and revising the manuscript.

### ACKNOWLEDGMENTS

The authors would like to thank the Office of Innovation and Entrepreneurship and Department of Biomedical Engineering, Chemistry, and Biological Sciences of the Stevens Institute of Technology for their continued support of ongoing research.


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Copyright © 2017 Patel, Thukral, Burns, Florescu, Chandramouli and Vinjamuri. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# **Synergy-Based Bilateral Port: A Universal Control Module for Tele-Manipulation Frameworks Using Asymmetric Master–Slave Systems**

*Anais Brygo<sup>1</sup> \*, Ioannis Sarakoglou<sup>1</sup> , Giorgio Grioli <sup>2</sup> and Nikos Tsagarakis <sup>1</sup>*

*<sup>1</sup> Department of Advanced Robotics (ADVR), Istituto Italiano di Tecnologia (IIT), Genova, Italy, <sup>2</sup> Interdepartmental Research Center "E. Piaggio", Faculty of Engineering, University of Pisa, Pisa, Italy*

Endowing tele-manipulation frameworks with the capability to accommodate a variety of robotic hands is key to achieving high performances through permitting to flexibly interchange the end-effector according to the task considered. This requires the development of control policies that not only cope with asymmetric master–slave systems but also whose high-level components are designed in a unified space in abstraction from the devices specifics. To address this dual challenge, a novel synergy port is developed that resolves the kinematic, sensing, and actuation asymmetries of the considered system through generating motion and force feedback references in the hardware-independent hand postural synergy space. It builds upon the concept of the Cartesian-based synergy matrix, which is introduced as a tool mapping the fingertips Cartesian space to the directions oriented along the grasp principal components. To assess the effectiveness of the proposed approach, the synergy port has been integrated into the control system of a highly asymmetric tele-manipulation framework, in which the 3-finger hand exoskeleton HEXOTRAC is used as a master device to control the SoftHand, a robotic hand whose transmission system relies on a single motor to drive all joints along a soft synergistic path. The platform is further enriched with the vision-based motion capture system Optitrack to monitor the 6D trajectory of the user's wrist, which is used to control the robotic arm on which the SoftHand is mounted. Experiments have been conducted with the humanoid robot COMAN and the KUKA LWR robotic manipulator. Results indicate that this bilateral interface is highly intuitive and allows users with no prior experience to reach, grasp, and transport a variety of objects exhibiting very different shapes and impedances. In addition, the hardware and control solutions proved capable of accommodating users with different hand kinematics. Finally, the proposed control framework offers a universal, flexible, and intuitive interface allowing for the performance of effective tele-manipulations.

#### *Edited by:*

*Ramana Vinjamuri, Stevens Institute of Technology, USA*

#### *Reviewed by:*

*Dimitris Tsakiris, Institute for Computer Science – FORTH, Greece Shigeru Kuchii, National Institute of Technology, Japan*

*\*Correspondence: Anais Brygo anais.brygo@gmail.com*

#### *Specialty section:*

*This article was submitted to Bionics and Biomimetics, a section of the journal Frontiers in Bioengineering and Biotechnology*

> *Received: 01 January 2017 Accepted: 10 March 2017 Published: 03 April 2017*

#### *Citation:*

*Brygo A, Sarakoglou I, Grioli G and Tsagarakis N (2017) Synergy-Based Bilateral Port: A Universal Control Module for Tele-Manipulation Frameworks Using Asymmetric Master–Slave Systems. Front. Bioeng. Biotechnol. 5:19. doi: 10.3389/fbioe.2017.00019* **Keywords: teleoperation, tele-manipulation, synergy, asymmetric systems, exoskeleton, underactuation**

### **1. INTRODUCTION**

Performing a task using a teleoperated robot avatar makes possible to safely handle hazardous material as well as to take advantage of the machines' superior sensing and actuation capabilities, enabling the execution of tasks that require high precision and large wrenches. Furthermore, the introduction of a digital control layer between the operator's commands and the actuator's reference output allows to resort to signal processing techniques such as scaling or filtering in order to rectify the user's input. Additionally, preoperative automated task supervision features can be implemented to improve safety with, for instance, the introduction of motion constraints and safeguards to contain the risks stemming from accidental commands or external disturbances. As such, teleoperation not only enables to project the skills of an operator into a remote environment but it also permits to extend and enhance the human's manipulation capabilities with higher power, higher motion precision, and with greater safety. Finally, this powerful tool allows merging the strengths of men and machines so as to achieve enhanced performances. However, to take advantage of the potential benefits offered by tele-manipulation, a number of considerations need to be addressed both at the hardware and software levels.

### **1.1. Coping with the Master–Slave Asymmetry**

Of paramount importance in the realization of an intuitive tele-manipulation platform is the design of a bilateral control module capable of efficiently coping with diversely asymmetric master–slave systems.

To better comprehend this matter, one should bear in mind that the inherent capabilities of a teleoperator are conditioned by those of its master and slave devices, whose choice is guided by distinct functional requirements. On the one hand, the slave robot must incorporate an end-effector that is suitable for the task, i.e., that achieves the most adequate versatility–robustness trade-off to effectively manipulate its environment. Depending on the manipulation scenario considered, the end-effector may be chosen to be anything from a highly anthropomorphic dexterous hand to a single DoF industrial gripper. On the other hand, the master device should allow for unconstrained motions in the entire user's hand workspace and provide an accurate posture tracking as well as a rich haptic guidance while maintaining the degree of portability required by the application.

These criteria often lead to the selection of highly asymmetric systems at the kinematics, sensing, and actuation levels. Such asymmetry, if not carefully handled, might greatly decrease the intuitiveness of the framework and consequently jeopardize its effectiveness. The core strategy to make the operator unaware of the system's asymmetry and allow him/her to command manipulation actions as if directly performing the manual task is to capture his/her intent during natural motions and map this input signal from the master sensing space to the slave's actuator space so as to generate motion references translating the desired actions. Conversely, an adequate algorithm is needed to synthesize, from the data sensed at the slave side, a force feedback reference for the master's actuators in order to display an effective haptic guidance capable of enabling the operator to immerse in the task. Those policies play a major role in isolating the user from the complexity of the asymmetric system so as to create an efficient human–machine interface, where the natural coupling with the slave makes possible the execution of complex manipulation tasks while maintaining low the cognitive load of the operator.

Finally, tele-manipulators belong to the larger class of Human-In-the-Loop (HIL) systems. As such, their overall performance not only depends on the capabilities of the master and slave devices in isolation but also on how well these two entities are interconnected. It is therefore important to develop bilateral control strategies that effectively cope with the master–slave asymmetries.

### **1.2. On the Need of Unified Control Frameworks**

Besides handling the master–slave asymmetry, it is desirable to design flexible and universal control frameworks.

Indeed, in an attempt to achieve the versatility required to accommodate the wide range of shapes and sizes of the objects populating our workspaces, artificial hands exhibiting increasingly sophisticated designs have been developed during the last decade. However, the effective use of such highly articulated devices has been hindered by their inherent control complexity. A promising approach to tackle this difficulty lies in the development of unified frameworks promoting the synthesis of hardware-independent algorithms that can be indifferently used to control a variety of end-effectors. This can be achieved through adopting a two-layer architecture, where the low-level layer is used to encapsulate the kinematics, sensing, and actuation specifics of the device, while the high-level layer focuses on the resolution of the manipulation problem. This latter entity, which may be seen as equivalent to the concept of middleware in software engineering, can be based on universal, task-oriented control policies designed in complete abstraction from the hardware to be used. Provided the use of a generic interlayer interface, such platform permits to flexibly interchange not only the robotic hand used but also the control layer, making possible to easily implement a diversity of manipulation algorithms proposed in the literature, from autonomous or semi-autonomous grasp planners, as the one presented in Ciocarlie et al. (2007), to tele-manipulation strategies designed in the object domain, as proposed in Gioioso et al. (2013), or any other mapping that best suits the application considered.

The implementation of interfaces as generic as possible would benefit from the selection of a common expression space. As such, the concept of postural synergies has been proposed as the alphabet of the universal language for such unified control frameworks. This bio-inspired approach rests on neuroscience studies that have shown the existence of consistent spatiotemporal coordination patterns of the human hand joints during the execution of a variety of manipulation tasks (Santello et al., 2016). Among other possible projection spaces, the synergistic one appears very attractive for two main reasons. First, it seems sensible to draw inspiration from the CNS's<sup>1</sup> strategy to control the human hand that remains to date and by far the most complex and versatile manipulation tool existing. Second, considering the postural synergies as motion primitives that can be combined to design manipulation strategies can significantly decrease the control complexity. Indeed, analysis has revealed that a few variables, corresponding to the coordinates along the synergistic directions, can describe most of the variance of the hand posture during grasping motions (Santello et al., 1998).

<sup>1</sup> central nervous system.

Hence, deriving control algorithms in the synergy space can substantially reduce the dimensionality of the manipulation problem and therefore the associated computational cost.

To complete such architecture, low-level modules are required to translate the references described in synergy space into equivalent patterns of coordinated motion of the artificial hands' joints. While this mapping is quite straightforward when considering anthropomorphic robotic hands, the derivation of optimal posture subspaces for end-effectors with kinematics considerably different to those of the human hand remains an open research question. A first contribution in this direction has been proposed in Ciocarlie and Allen (2010), where the authors empirically define a set of basis vectors, referred to as eigengrasps, which describe a low-dimensional posture subspace for 5 artificial hands. This very promising work demonstrates the applicability of such approach and paves the way to the generalization of robotic hands control within a synergy-based unified framework.

### **1.3. The Approach Proposed**

This present work introduces a novel bilateral tele-manipulation control strategy that resolves the asymmetry of the master–slave system and generates motion and force feedback references in the hardware-independent hand postural synergy space.

In this tele-manipulation platform, the 3-finger hand exoskeleton HEXOTRAC (Sarakoglou et al., 2016) is used at the master station of the proposed platform. Its sensory and actuation systems permit to monitor the position and orientation of the operator's fingertips and to display kinesthetic feedback reflecting some mechanical properties of the remote environment. This exoskeleton is used to control the Pisa/IIT SoftHand (SH), an anthropomorphic robotic hand, the as slave device. This device implements the concept of soft and adaptive synergies mechanically, such that its 19 joints are driven by a single motor along the first synergy in free space, while its shape adaptation capabilities stemming from underactuation let it conform to the shape of objects during grasping. With a single actuator and a sensory system limited to a position encoder and a current sensor, the SoftHand's minimalistic design promotes robustness but challenges traditional teleoperation strategies. Indeed, the absence of joint encoders and individual actuators prevents the control of the slave's joints or fingertips trajectories as a straightforward mapping of the operator's motions. Similarly, the absence of per-digit haptic sensors calls for a novel force feedback control strategy.

To tackle this challenge and achieve an intuitive framework capable of efficiently handling the large asymmetry between the master and the slave devices, the concept of Cartesian-based hand synergies is introduced as a set of independent vectors oriented along the grasp principal components and described in the fingertip Cartesian space. This base is used to develop a novel bilateral synergy port, schematized on **Figure 1**, which is proposed as a mapping tool for performing bilateral tele-manipulations.

In the proposed approach, the operator's hand posture is projected on this space to extract the corresponding coordinate along the first synergy, which is then used as the SoftHand's motor position reference. This algorithm elegantly scales the human hand's many degrees of freedom to the slave's single Degree Of Actuation (DOA) by extracting from natural motions the relevant information to be used as input for operating the slave—here the desired degree of hand closure. Conversely, the proposed synergy port is used to generate force feedback references for the exoskeleton to display perceptually meaningful haptic stimulations that characterize the remote interaction. The 1-dimensional grasping force applied by the SoftHand's actuator on its environment along the first synergy is estimated using a torque observer. It is then re-projected on the Cartesian-based synergy space in order to synthesize a 9-dimensional force reference in the user's fingertip Cartesian space. These finger-individualized forces, which reflect the grasping effort developed by the SoftHand during its interactions with the objects manipulated, are finally projected on the exoskeleton's joint space in order to generate the reference for its actuated DOFs. To complete the tele-manipulation interface, the framework is subsequently enriched with the vision-based motion capture system Optitrack, which is used to monitor the position and orientation of the user's wrist. This input is used to generate in real time the 6D position reference trajectory for the wrist of the robot arm on which the SoftHand is mounted. This control scheme has been implemented to control the IIT humanoid robot COMAN as well as the KUKA LWR manipulator.

This work builds upon the proof-of-concept study presented in Brygo et al. (2016), which demonstrated the effectiveness of the

proposed synergy-based interface in allowing a single trained user to perform the desired tele-manipulation. This preliminary work is developed and extended here in order to achieve a universal platform. In particular, the theoretical and practical tools needed to endow the framework with the capability to accommodate very different users' hand sizes are implemented, and the usability of the interface by novice, untrained operators is experimentally assessed.

The rest of this paper is organized as follows. In Section 2, a description of the hand exoskeleton and the SoftHand is presented along with the principal characteristics that motivated the selection of these devices. A brief review of the strategies proposed in the literature to control robotic hands using asymmetric teleoperators is then presented, and the need for a novel mapping strategy suiting the proposed system is explained. Section 3 addresses the generation of the SoftHand's motor position reference as a mapping of the user's hand posture. In particular, the concept of Cartesian-based hand synergy is introduced together with the user study that has been conducted to analyze the possibility to describe the human hand posture during grasping motions using a low-dimension set of variables constructed as a linear combination of the fingertip Cartesian trajectories. Subsequently, the theoretical tools used to build the synergy-based teleoperation port are derived. The effectiveness of this approach to synthesize the artificial hand motion reference realizing the operator's intentions is then experimentally demonstrated. Section 4 describes how the force feedback references are generated through inverse projection on the synergy space, and a finger-individualized force scaling procedure is proposed to homogenize the force feedback amplitudes across users. The closed-loop performance of the system is then experimentally characterized. Section 5 discusses the possibility of relying on a simplified calibration procedure and analyzes how the use of a user-independent Cartesian-based synergy matrix affects the position and force feedback references. Section 6 presents the experiments conducted with both robotic slaves in order to evaluate the effectiveness of the proposed framework. Finally, conclusions of this work are drawn in Section 7.

### **2. A HIGHLY ASYMMETRIC MASTER–SLAVE SYSTEM**

### **2.1. A Hand Exoskeleton at the Master Station**

A fundamental aspect to consider when designing telemanipulation interfaces concerns the choice of the master device. One option to monitor the user's hand posture consists in tracking the fingers' joints' angular excursion. Vision-based motion capture systems can be used to this end. While flexibly adjusting to different hand sizes, their main drawback regards the need to handle occlusions, an issue made particularly salient by the large number of markers in a small workspace. Furthermore, the installation of a large number of markers on the skin is a time-consuming procedure. Alternatively, data gloves such as the VPL data glove (Hong and Tan, 1989) or the CyberGlove from Immersion Corporation (Peer et al., 2008) represent a popular solution that does not suffer from this limitation. However,

reservations have been expressed about their tracking accuracy. Indeed, not only it can be affected by physical factors such as sweat and temperature but it is also largely dependent on the fit of the glove. Accommodating diverse hand sizes is a critical requirement when aiming at designing a universal interface, as it is the case of this work. Calibration procedures have been developed to account for the different hand kinematics (Griffin et al., 2000), but besides consisting in tedious processes that can be time consuming, their results are often not ideal (Dipietro et al., 2008). Finally, additional solutions need to be added in order to enhance the master station with force feedback capabilities, which is desirable to enable remote haptic explorations and is believed to assist the operator in regulating the manipulation forces.

These considerations motivated our choice to use the hand exoskeleton HEXOTRAC (Sarakoglou et al., 2016), pictured in **Figure 2**, at the master station of our tele-manipulation framework. It consists of a three 6-DOF linkages mounted on a common base, which is attached to the dorsal side of the hand. The other end of the linkages is fastened to the user's thumb, index, and middle fingertips through interchangeable 3D-printed thimbles. This attachment system not only allows for fast donning and removal of the device but it also permits to suit a large range of hand sizes. The overall kinematics design is such that the reachable workspace of the linkages endpoint completely covers the human hand workspace. This makes possible the execution of any type of motion and permits in particular the rotation of the fingertips, which provides the freedom to perform natural manipulations and is the key to accommodating a very large range of hand sizes without any mechanical adjustment. High resolution magnetic encoders monitor the position of all the linkages' joints, permitting the 6-DoF tracking of the operator's fingertips.

The hand exoskeleton is fitted with three low-gear DC motors whose output torques are transmitted to the first joint of each finger linkage through a pair of anti-backlash bevel gears. Pairs of bilaterally bonded strain gages provide measurement of the torque applied at the proximal joints. This measurement is used as feedback signal by a PD controller that tracks the reference torque trajectory. *τint des ∈* R 3*n<sup>a</sup>* is the joint torque vector corresponding to the desired fingertip interaction force projected on the actuated joint space. Since the strain gages reading *τstr g ∈* R 3*n<sup>a</sup>* encompasses both the interaction torque and the gravity torque, a gravity compensator is introduced to compute *τ<sup>g</sup> ∈* R 3*n<sup>a</sup>* , the model-estimated gravity joint torque projected on the actuated joint space. Note that *n<sup>a</sup>* is the exoskeleton finger-actuated joint space dimension.

Finally, this hand exoskeleton is used as master system of the teleoperation system. The following section describes the robotic hand used as slave device.

### **2.2. The SoftHand at the Slave Station**

The choice of the slave device is of paramount importance, since its intrinsic characteristics greatly influence the capabilities of the tele-manipulator as well as its intuitiveness. After introducing in Sect. 2.2.1 the efforts of the robotic community toward the development of robotic hands equipped with the technical capability to skillfully manipulate the many different objects populating

our workspaces, the SoftHand's characteristics that motivated its selection as the slave hand of the proposed tele-manipulation framework are presented in Sect. 2.2.2.

### 2.2.1. Achieving Stable Grasp with Robotic Hands, a Long-lasting Challenge

Versatile robotic manipulation in unstructured environments is a challenging topic for the robotics community and a very active area of research during the last decade (Shimoga, 1996; Kemp et al., 2007). Of particular importance is the ability to manipulate objects with diverse geometries while precisely controlling the contact locations and force distribution in order to achieve force closure grasps, also referred to as stable grasps (Bicchi and Kumar, 2000; Okamura et al., 2000; Rosales et al., 2012; Prattichizzo et al., 2013). Considering the diverse shapes and sizes of the objects populating human workspaces, a considerable effort has been recently devoted to the development of highly dexterous artificial hands. The Utah/MIT hand (Jacobsen et al., 1984), the Anthrobot hand (Kyriakopoulos et al., 1997), the Shadow hand (Shadow Robot Company Ltd, 2017), or the DLR hands (Butterfass et al., 2001) are a few examples of such attempts to approach the fine manipulation capabilities of the human hands. However, their actual use has been largely hindered by their limited power capabilities and poor robustness, resulting from the implementation of many small and usually delicate actuators, as well as by their increased control complexity. Aware of these limitation, an innovative design has been proposed and implemented in the Pisa/IIT SoftHand (Catalano et al., 2012) used as slave device of the proposed tele-manipulation platform. The intrinsic properties of this hand, which conditioned the development of the control framework, are described in the following section.

### 2.2.2. SoftHand, a Powerful Grasping Tool

The SoftHand has been selected because it elegantly combines versatility and robustness within a unique design that turns it into a powerful tool to grasp and manipulate objects of diverse shapes in unstructured environments.

**FIGURE 3 | Left: picture of the Softhand; right: routing of its tendon**.

Its transmission mechanism uses a system of pulleys and a tendon routed through all fingers to transmit the torque of a single motor to the 19 DOFs of the hand, as shown on **Figure 3**. Its differential system has been designed to drive all joints along the first synergy in free space motions, while during contact its shapeadaptive property deriving from underactuation lets it conform to the geometry of the object grasped. Such behavior is achieved through combining two main theoretical tools, such that the soft synergy concept is realized with a shape-adaptive underactuated mechanism. Indeed, while the neuroscience-based concept of the first postural synergy appears attractive since the manipulation of a single coordinate allows to describe a large percentage of the posture variance during natural grasps, a purely kinematic implementation of this synergy leads to an inconsistent grasp force distribution. The concept of soft synergies was proposed to tackle this limitation and provides a model generating suitable internal forces that permit to robustly hold an object (Bicchi et al., 2011).

In addition, an innovative design employing rolling contact articulations assembled with rubber ligaments allows the joints to disarticulate during impacts before passively returning to their initial configuration. These features endow the SoftHand with an exceptional robustness through ensuring a soft and safe behavior during unexpected collisions, making it suitable for manipulation tasks in unstructured environments.

Finally, this design embeds part of the control intelligence in the physical system itself. Indeed, while the actuation space is reduced to a single dimension, the SoftHand's multiple DOFs move in a principled way and adapt to the environment manipulated, achieving high versatility at a reduced control complexity cost. However, these same characteristics challenge traditional teleoperation techniques, as discussed in the following section.

### **2.3. A Design That Challenges Traditional Teleoperation Strategies**

With a single electric motor, one position encoder, and one current sensor, the SoftHand's overall design aims to a low cost and robust device. However, such underactuation and minimum sensing represent a major challenge when considering the SoftHand as a slave device to teleoperate using a hand exoskeleton and prevents from relying on traditional teleoperation strategies. For a better understanding of the problematic, below are presented the different strategies that have been proposed in the literature to teleoperate robotic hand and cope with asymmetric master–slave systems.

There are three typical control strategies for teleoperating robotic hands: joint-to-joint, point-to-point, and pose mapping. Alternatively, an object-centered approach has been recently proposed. Following is a brief overview of these techniques.

Direct joint-to-joint mapping has been proposed as control strategy to teleoperate robotic hands (Wright and Stanisic, 1990; Bouzit, 1996; Yokokohji et al., 2003). This method, which appears suitable to drive anthropomorphic hands from recording the trajectory of the operator's hand joints using motion tracking systems such as data gloves, presents some limitations. Indeed, the many DOFs of the human hand endow it with a remarkable dexterity that most robotic hands fail to achieve, thus creating a kinematic mismatch between the two sides. This in turn renders this approach of directly mapping the DoF of the slave hand to motions of the human hand joints unable to efficiently cope with this level of kinematic master–slave asymmetry.

As an alternative, point-to-point mapping has been proposed to control non-anthropomorphic hands or grippers. This approach consists in designing the reference trajectory of particular points of the robotic hand as a function of the Cartesian position trajectory of associated points of the master hand. Typically, the fingertips of the master hand are monitored and used to derive the desired position trajectory of the slave's fingertips. In Rohling et al. (1993), the Utah/MIT Dextrous Hand is teleoperated using the Utah Dextrous Hand Master. In this work, the control algorithm aims at mapping the position and the orientation of master to the slave device. To account for the slight kinematic dissimilarities between the systems, a priority-based strategy is implemented such that position is considered as the prime goal while orientation comes as second objective. Another implementation reported in Hong and Tan (1989) proposes to teleoperated the Utah/MIT Dextrous hand using the VPL data glove. The position trajectories of the operator's fingertips are computed from the measured joints position using the forward kinematics of the human hand model and then used to derive the reference trajectories of the slave fingertips. However, a direct mapping of the Cartesian trajectories of the fingertips might not be possible when considering larger asymmetries, such that alternative mapping might be needed. In Peer et al. (2008) is presented a method for teleoperating the three-fingered gripper BarrettHand using the CyberGlove as master tracking system. The control space of the slave's hand is quite restricted since each finger counts two coupled DOFs actuated by a single motor; furthermore, two of the three fingers can rotate synchronously and symmetrically around the base joint. To generate references lying within the gripper's reachable space, the workspace of the human fingers is scaled and then vertically projected.

The third approach, referred to as pose mapping, is based on the functional analysis of the human hand during natural actions. A collection of human poses or grasps are defined, and to each of them is associated an equivalent posture of the robotic hand. A transformation matrix is proposed in Pao and Speeter (1989) that maps human hand positions to corresponding positions in the slave space, and an interpolation algorithm is employed to shift from one pose to another. In Wojtara and Nonami (2004), a grasp recognition method relying on neural network is developed together with a custom mapping algorithm. The use of neural networks for grasp identification is also studied in Gorce and Rezzoug (2004). Alternatively, Ekvall and Kragic (2006) use Hidden Markov Models for the recognition of the human grasp, while a neural network is used to define the mapping between the operator's and the slave's hand configuration spaces. A grasp taxonomy is introduced in Liu and Zhang (2004), where a Gaussian Mixture Model based classifier is designed to recognize the type of grasp. Another strategy is used in Kang and Ikeuchi (1997) where a virtual finger mapping method is used after recognizing the human grasp in order to generate the slave's reference. The main limitation of those strategies regards the poor robustness of the identification algorithms. As such, small changes in the operator's hand posture can lead to switches between the reference pose in the target space, resulting in somewhat unpredictable motions that are especially undesirable during tele-manipulations.

A more recent approach has been proposed that focuses on manipulation tasks. The core concept consists in introducing a virtual object at the master station. The operator's finger motions are translated into the rigid-body motion and deformation imparted on the virtual object. The slave hand is then controlled such that the object—either virtual or real—tracks the transformations of the virtual object. This paradigm aims at integrating the operator's manipulation intent in the object domain. A 2D instance of such method is implemented in Griffin et al. (2000) and used to control a symmetric two-fingered planar robot. A non-linear mapping is developed to map the motion of the master virtual object to the motion of the slave virtual object in order to better utilize both station's workspaces. This method is extended to 3D cases in Wang et al. (2005), where a 3-fingered robotic hand is teleoperated. A virtual circle is defined by the three considered fingertips of the operator. The master virtual circle is parameterized, and the parameter's transformations are mapped to similar transformations of the virtual circle defined at the slave side. Finally, the robot's finger motions are computed according to the desired fingertip position. An analogous approach is undertaken in Gioioso et al. (2013) where the smallest sphere enclosing the fingertips of the operator is used as virtual object. The user's input is described by modeling the effects of the fingers' displacement on this virtual sphere's linear and angular velocities as well as it deformation. These transformations are scaled and tracked by a virtual sphere defined as the slave's side. In Salvietti et al. (2013), the virtual object is used to both capture the user's hand motion and to compute the force feedback reference, such that the wrench displayed to the user corresponds to the wrench applied by the robotic hand on the virtual object. Those object-based methods provide a universal interface that abstracts from the kinematics of the master and slave systems and therefore allows to cope with highly asymmetric systems.

However, none of these approaches can be used in the present work. Indeed, the extreme underactuation of the SoftHand and the absence of joint encoders in its fingers prohibit the design of the motion control reference in joint or Cartesian space as a traditional mapping of the operator's motions. Furthermore, the absence of haptic sensors on the slave's digits calls for a novel force feedback strategy in order to generate appropriate references for the hand exoskeleton's actuators. Hence, the need to develop a novel bilateral teleoperation algorithm suitable for the system is considered in this work. This aspect is addressed in the following section.

### **3. POSITION CONTROL: A SYNERGISTIC-BASED TELEOPERATION STRATEGY**

While the large number of DoFs affords the human hand with remarkable dexterity, the SoftHand's underactuated kinematics, with its single actuator, limit control over the postures it can achieve. The purpose of the work presented in this section is to resolve this asymmetry through synthesizing a suitable algorithm that maps the native hand's unconstrained motions into motions achievable by the robotic hand that best translate the user's manipulation intent. To this end, we propose to capitalize on the insights recently gained with regard to the concept of postural synergies, which was revealed to be a key mechanism underlying the human hand motor control and which inspired the design of the SoftHand's transmission system. The main idea of the proposed algorithm is to extract the user's hand first motor synergy's activation level and use this trajectory as position reference to drive the slave's actuator. Indeed, this component not only reflects the high-level intent of the operator but it also corresponds to a coordinate along the synergistic path described by the SoftHand's joints.

To begin with, Sect. 3.1 outlines the neuroscience-based notion of hand synergies, which inspired the robotic community with a radically new control approach that aims at emulating the strategy employed by the CNS to tame the motor control complexity of the hand's many DOFs. While a large body of research has built upon this idea, the totality of these studies—to the best of our knowledge—has considered the synergies as derived from the hand joint trajectories. The present work investigates the possibility of extending this concept toward extracting the synergistic patterns of motion of the hand from its postural descriptions in the Cartesian space of the fingertips instead of the joint space. Section 3.2 elaborates on a user study that has been conducted to investigate this possibility. On the basis of the

experimental results that validated this approach, the notion of Cartesian-based synergy space is introduced. Section 3.3 describes how this paradigm is implemented into the control scheme of the proposed teleoperation framework. A characterization of the asymmetric teleoperator in then presented, where the behavior of the slave is analyzed under different inputs issued by the operator. Specifically, the response of the system to finger adduction–abduction and flexion motions is reported. The results clearly indicate the effectiveness of the approach, which enables to intuitively control the slave robotic hand from natural hand motions.

This Cartesian-based synergy port is proposed as an analogous and simpler alternative over the classical postural synergy that may suit better hand tracking systems, which do not provide direct measurement of the joint angles but are able of high resolution 6-DOF tracking of the fingertips. In such cases, the Cartesianbased synergy analog could be used for extracting from Cartesian tracking data the activation level along a synergy coordinate, alleviating the need of resorting to complex joint tracking systems or computationally expensive and potentially inaccurate inverse kinematic solutions of the hand. In addition, given a different endeffector or robotic hand with different kinematics, actuation, and posture control capabilities, this method could be tuned to extract the user's intent along additional synergistic actions that could be then related to the commanded actions of the slave.

### **3.1. Hand Synergies, a Path toward Simplicity**

The human hand is a fascinating and remarkably complex biological system in which a multitude of muscles and tendons controlling the joints' angular excursions are simultaneously engaged to generate purposeful movements. Understanding how the brain harnesses the motor control complexity of this organ has been extensively studied, and evidences have been put forward to substantiate the idea that the Central Nervous System does not consider it as a collection of independent joints. Instead, various neuroscience analyses postulate the existence of motor primitives defining the coordinated motion of multiple joints along trajectories that are referred to as synergies (Latash, 2008). Interested readers are referred to the thorough reviews presented in Tresch et al. (2006) and Santello et al. (2013). Although the term synergy has been employed to conceptualize coordination at various levels including kinematics, i.e., joints motion coordination (Santello et al., 1998), kinetics, i.e., digit force coordination (Zatsiorsky et al., 2000), neuromuscular, i.e., multi-muscle activation patterns (Bizzi et al., 2008), or neurosensorial, as the perceptual counterpart of the motor synergies (Bicchi et al., 2011), we will use here the definition proposed in Turvey (2007), according to which a synergy is "a collection of relatively independent DOFs that behave as a single functional unit."

This synergistic behavior is believed to result from a combination of biomechanical constraints, related in particular to the muscles and tendons spanning several joints, with neural constraints, stemming from a specific circuitry that distributes high-level commands into multiple inputs traveling along descending pathways to various muscles (Schieber and Santello, 2004). The complex interplay between these hard-wired and soft-wired constraints results in multi-DOFs coordination patterns, such that each synergy can be seen as a vector specifying the relative motion of a collection of joints, while their absolute position is controlled through modulating a single signal. Therefore, they constitute basic building modules that can be combined using a set of timesynchronized activation level trajectories that translate in principled patterns of motion and ultimately allow to produce a large repertoire of movements.

Gaining a deeper insight into the synergistic mechanism underlying the hand motor control is crucial to apprehend the high-level intentions of the user from motion analysis. Indeed, synergies are believed to be closely linked to the functional outputs of a motor behavior (d'Avella and Bizzi, 2005) and "may therefore represent the bottom of a hierarchical neural control structure in which higher neural centers operate on increasingly conceptual variables related to task-level motor performance" (Ting and McKay, 2007).

Among the many implications deriving from the synergistic organization of the hand motor apparatus, a very interesting aspect regards the dimensionality reduction of the control space it implies. Diverse techniques have been proposed to individuate the patterns of joints' covariation that occurs when modulating the hand posture to grasp objects of diverse shapes, including Singular Value Decomposition (Mason et al., 2001) and Principal Components Analysis (PCA) (Santello et al., 1998). The latter study reveals that only two independent variables, corresponding to the levels of activation of the first two postural synergies, account for more than 80% of the variance of a dataset containing the static postures at the end of the grasp motions of a large number of imagined objects. This indicates that the hand molding motions involve principally two synergies. It is worth specifying that these observations do not imply that additional synergies represent random, task-unrelated variability. Instead, they represent effective degrees of freedom that come into play when finer posture adjustments are required, especially in order to conform to the actual geometry of the objects manipulated (Todorov and Ghahramani, 2004).

While a large amount of studies have been dedicated to the analysis of the hand postural synergies, all of them—to the best of our knowledge—have considered the synergies as extracted from the hand's joint space description. In the present work, we propose to investigate the possibility to perform an analogous PCA-based dimension reduction analysis from the hand posture described in the fingertip Cartesian space. Note that the objective is not to discuss whether motor control strategies are encoded at the joint or task-space level by the CNS but rather to explore how they can be adequately monitored. Considering that joints and end-effector Cartesian trajectories are correlated, it is reasonable to hypothesize that the synergistic organization that has been evidentiated in the joint space would be reflected in the Cartesian space.

This approach stems from the hand exoskeleton device used at the master station, which provides a direct tracking of the fingertips 6D position trajectories. These could be used to infer the joints trajectories through implementing an inverse kinematics module. However, not only this would require the introduction of a calibration procedure that can be long and tedious to identify the user's relevant kinematic parameters but it would also and more importantly entail a reduction of the hand posture estimation accuracy. Indeed, the intrinsic complexity of the hand anatomical organization makes accurate modeling challenging. This complexity compels to the development of simplified kinematic models so as to achieve manageable solutions and meet the computational speed requirements associated with real-time applications (Bullock et al., 2012). However, it is important to keep in mind that such simplifying assumptions (e.g., joints axis alignment, number, nature of the joints, etc.) introduce an error in the hand posture estimation. Instead, the extraction of the synergies coordinates directly from the motion data captured permits to avoid this pitfall.

In the following is presented the study that has been conducted to investigate the feasibility of identifying the hand postural synergistic paths and their respective levels of activation from the tracking of the fingertips motion in the Cartesian space.

### **3.2. Experimental Analysis of Human Grasps in Cartesian Space**

An experimental study has been carried out in order to determine whether the postures adopted by the hand during grasping motions could be adequately described by a low-dimensional set of coordinates traveling along motion primitives expressed in the fingertips' Cartesian position space. The aim was to investigate whether the preliminary results presented in Brygo et al. (2016), which considered a single subject, could be generalized to different hands kinematics.

Eight right-handed subjects—3 females and 5 males aged between 25 and 36—took part to this experiment. Ethical approval was not required for this study as per the national and institutional requirements, and written informed consent was obtained from all participants. They were instructed to wear the hand exoskeleton HEXOTRAC and to sit at a table with the upper arm resting along the body, the elbow joint right-angled, the forearm horizontal with the palm-down hand opened. They were then asked to imagine an object placed 40 cm away on the table and enjoined to shape their hand while executing a natural reach-to-grasp motion as if to pick it up, before returning to their initial configuration. They repeated this operation 5 times per object with 57 different objects, for a total number of 5 *×* 57 items per subject. In order to allow for comparisons, the objects used were the same as listed in Santello et al. (1998). These everyday objects require both precision and power grips and exhibit a large range of shapes in order to account for the modulation of the hand posture during natural grasping. Note that subjects were asked to imagine the objects in order to allow for the study of the motor control strategies employed by the CNS while avoiding the mechanical interference and constraints that would result from physical interactions with real objects.

The exoskeleton's fingers' joints position trajectories were recorded at a 1 kHz sample rate and used to compute the Cartesian position trajectory of the users' fingertips using the equation (1) as detailed in Sect. 3.3. Finally, the data were stored in a *n<sup>c</sup> ×* 3*m* matrix, with *n<sup>c</sup> ∈* R the data collection's number of samples and *m* = 3 the dimension of the space used to describe the position of each fingertip.

A PCA analysis has been performed to examine the internal structure of the datasets, and the percentage of the variance explained by each principal component (PC) has been computed for each subject. The averaged values over all subjects of the five largest PCs are presented in **Figure 4**. Additionally, the dispersion of the data from the mean is quantified for each PC by computing the associated SD, which is graphically represented on the same figure by the mean of error bars.

The first PC accounts in average for 71.8% of the variance of the data, while the first two PCs account in average for more than 85% of the variance. Furthermore, it is worth noting the low SD values, which indicate that the percentage of the variance explained by the each PC is fairly consistent among subjects. These results reveal that the 9 variables expressed in the initial base are strongly correlated. Therefore, the hand motion can be more adequately described using a new set of independent variables varying along orthogonal axis oriented in the directions maximizing the variance explained in the dataset. Through discarding the variables corresponding to the large PCs, which contribute very little to the variability of the data, it is possible to describe the hand posture during grasp motions using a low-dimensional set of variables with a minimum information loss.

This change of base can be performed using a projection matrix whose columns are the eigenvectors of the covariance matrix of the mean-centered data. In analogy to the joint space terminology, this matrix will be referred to as the Cartesian-based synergy matrix *Sx*. Its column vectors, which are the orthogonal axes of the new base and represent the motor primitives whose linear combination describe the hand postures, will be referred to as the Cartesian-based synergies. In the rest of this work, if not stated explicitly, the term synergies will be used to refer to the Cartesianbased synergies, although the reader is invited to keep in mind that they are distinct from the joint space-based synergies as defined in the literature.

The obtained synergy matrix *Sx*, computed once from the data collection, is then used to project in real time the user's hand posture on the synergy space so as to extract each posture's coordinate along the first synergy, as detailed in the following section.

### **3.3. First Synergy Position Reference Extraction**

This section explains how the raw fingertips' Cartesian position trajectories monitored by the hand exoskeleton can be mapped onto a new set of scalar variables representing the hand's posture coordinates in the Cartesian-based synergy space. Subsequently, an experimental study is presented that analyzes the effectiveness of the proposed approach in capturing the user's grasping intent during unconstrained motions and in translating it into adequate references to control the amount of closure of the slave hand.

The exoskeleton links' configuration is monitored using the position encoders mounted on all joints. The 6-DOFs Cartesian trajectory of the operator's fingertip *x*(*t*) can be computed straightforwardly using the forward kinematics of the exoskeleton fingers as shown in equation (1).

$$\varkappa(t) = f \kappa(q(t)) \tag{1}$$

with *x*(*t*) *∈* R 6 the fingertip Cartesian trajectory, *q*(*t*) *∈* R *n* the exoskeleton finger position trajectory in joint space, *n* = 6 the number of joints per finger, and *f<sup>K</sup>* the exoskeleton forward kinematics function. The hand posture tracked by the exoskeleton can then be mapped from the Cartesian space to the synergy space using equation (2).

$$
\sigma(t) = \mathcal{S}\_{\mathbf{x}}^{-1} \mathcal{X}(t) \tag{2}
$$

with *m* = 3 the number of fingers, *X*(*t*) *∈* R 3*m* the fingertips position described in Cartesian space stacked in a single vector, *σ*(*t*) *∈* R 3*m* the projection of *X*(*t*) on the synergy space, and *S<sup>x</sup> ∈* R 3*m×*3*m* the hand synergy matrix. The first component *σ*1, which corresponds to the projection of the user's hand posture along the first synergy, is extracted and scaled to the robotic hand's motor range. This position reference is then tracked within inner PI controller.

An experimental study has been conducted to characterize the system's behavior. Two types of motion were considered. During the first phase, the operator performed a fingers abduction motion in the palm plane. During the second phase, the operator was instructed to close the hand as if to grasp a sugar lump. Note that this object does not belong to the list of items used to build the synergy matrix so as to attest that this procedure extracts the grasping intent of the user independently from the exact object that is being grasped. Seven snapshots of the hand during the different phases of motion are presented on **Figure 5**. The extracted coordinate along the first synergy is presented on **Figure 6**.

Results (see also Extension 1) clearly indicate that motions outside of the first synergy do not affect the slave reference, which remains almost constant during the first phase. During the second phase, the grasping motion of the operator is properly mapped to the synergistic coordinate *σ*1. Therefore, the proposed teleoperation strategy allows for the intuitive control of the SH through handling natural, unconstrained motions of the operator; from the hand posture is extracted in real time the first eigengrasp coordinate that can be used to drive the SH.

While this section addressed the extraction of the grasping intent of the operator to control the closure of the SH, the following section analyzes the generation of the force feedback reference in order to render to the operator the impedance of the environment manipulated by the slave hand.

**FIGURE 5 | Snapshots corresponding from left to right to the a, b, c, d, e, f, and g points in Figure 6**.

### **4. FORCE FEEDBACK: MAPPING A 1-DOF GRASPING TORQUE TO A 3-FINGER FORCE REFERENCE**

In order to increase the sense of telepresence of the user, a kinesthetic feedback reflecting the interaction forces between the robotic hand and the manipulated environment is displayed by the exoskeleton.

As the SoftHand's motor current sensing is the only available information about the amount of forces generated by the robotic hand during the manipulation, an interaction torque observer is implemented. This estimated interaction torque represents the grasping effort applied along the SH's actuated dimension, which coincides with the hand's first synergy. This one-dimensional reference is then mapped into a 9-D reference in the fingertips Cartesian space of the user through an inverse projection of the Cartesian-based synergy space introduced in the previous section. As the corresponding Cartesian-based synergy matrix depends on the kinematics of the hand, a novel force scaling procedure is introduced that permits to homogenize the amplitude of the force reference across users in consistency with the aim of designing a universal platform. Finally, the resulting force reference in the fingertips Cartesian space is projected on the exoskeleton's actuated joint space in order to derive the motors torque references.

Note that another possible approach to generate force feedback references consists in mounting force/torque sensors on the fingertips of the SoftHand so as to get a direct measurement of the interaction forces. However, the choice to rely on the Soft-Hand's current measurement to estimate the interaction forces stems from the overall objective of minimizing the complexity of the setup so as to achieve a highly robust and cost-effective interface. Complementarily with this goal, the present study aims at investigating the possibility to compensate for such minimal sensing system by the mean of a control layer capable of resolving the large master–slave asymmetry through mapping a scalar force reference described in the slave's sensing space to a force reference of larger dimension described in the master's actuated space.

After deriving the mathematical foundations underlying the mapping from the synergistic force reference to the fingerindividualized force feedback in Sect. 4.1, the closed-loop performance of the bilateral system is analyzed, and a force feedback characterization is presented in Sect. 4.2.

### **4.1. Mapping a 1-DOF Grasping Torque to a 3-Finger Force Reference**

Considering that the synergy matrix *S<sup>x</sup>* introduced in Sect. 3.2 is a linear, time-constant operator, the equation (2) can be differentiated with respect to time such that

$$
\dot{X}(t) = S\_\mathbf{x} \dot{\sigma}(t) \tag{3}
$$

Provided that *X*˙ represents a velocity, then the power in initial base can be expressed as

$$\boldsymbol{F}(t)^{T}\boldsymbol{\dot{X}}(t) = \boldsymbol{P}(t) \tag{4}$$

with *F*(*t*) *∈* R 3*m* representing the Cartesian space force vectors applied at the fingertips stacked in a single column vector.

Introducing *z*(*t*) *∈* R 3*m ,* the force vector in the synergy space, the power in this base can be defined through

$$\left(z(t)^{T}\dot{\sigma}(t) = P(t)\right) \tag{5}$$

Therefore, applying the power balance equation between the initial and projected bases, the following force space mapping holds:

$$z(t) = \mathbf{S}\_{\mathbf{x}}^{T} \mathbf{f}(t) \tag{6}$$

Considering that *S<sup>x</sup>* is an orthogonal matrix, the Cartesian force vector can be computed from the synergy force vector through

$$F(t) = \mathbf{S}\_{\mathbf{x}}^{-T} \mathbf{z}(t) \tag{7}$$

with *z*(*t*) the force reference in synergy space defined as

$$z(t) = [z\_1(t) \; 0 \; \dots \; 0] \tag{8}$$

such that

$$z\_{\mathbb{I}}(t) = \tau\_{\text{int}(t)} \tag{9}$$

with *τint ∈* R the interaction torque applied by the slave hand on the remote environment. It is estimated from the current and position measurement at the DC motor of the SoftHand using the interaction torque observer presented in Ajoudani et al. (2014). In this module, the total torque generated by the motor is computed, and its external component is isolated from its internal component, which corresponds to the elastic torque generated by the tendons and the frictional torque arising at the joints and pulleys. The resulting interaction torque observed corresponds to the grasping effort applied by the SoftHand on its environment along the first synergy.

The Cartesian force reference is then mapped to the exoskeleton joint space using

$$
\tau\_{\rm expo}(t) = J(q)^T F(t) \tag{10}
$$

where *τexo ∈* R 3*n* denotes the exoskeleton torque vector, *J*(*q*) *∈* R 3*m×*3*n* the exoskeleton Jacobian, and *n* = 6 the dimension of each finger joint space.

The force feedback reference *F*(*t*) corresponding to a given interaction torque varies across users depending on their individual synergy matrix as described by equation (7). To get comparable force feedback amplitudes across all users for the same interaction torque and to use the full torque range of the exoskeleton, user-dependent, finger-individualized force scaling gains *k<sup>i</sup> ∈* R are introduced such that

$$\tau\_{\exp i}(t) = J\_l(q)^T F\_l(t) k\_i \tag{11}$$

with *i ∈* {0, 1, 2} the exoskeleton's finger linkage considered. These gains are computed such that the maximum interaction torque *τint max ∈* R that the system is expected to render is mapped to the maximum torque that the exoskeleton can deliver *τexo max ∈* R when the hand of the user is closed:

$$k\_i = \frac{\tau\_{\text{exo max}}}{J\_{c\text{ i}}(q)^T F\_{i\text{ max}}} \tag{12}$$

with *J<sup>c</sup> <sup>i</sup>*(*q*) *∈* R *m×n* the Jacobian of the *i*-th linkage in closed hand configuration and *Fi max ∈* R *m* computed from

$$F\_{\text{max}} = \text{S}\_{\text{x}}^{-T} z\_{\text{max}} \tag{13}$$

with

$$z\_{\max} = [\tau\_{\text{int }\max} \mathbf{0} \dots \mathbf{0}] \tag{14}$$

The integration of this synergy port within its overall bilateral teleoperation control scheme as illustrated on **Figure 7**.

### **4.2. Closed-Loop Performances: Force Feedback Characterization**

A force feedback characterization has been performed in order to analyze two main aspects. The first objective was to examine whether the direction of the forces, as computed through the proposed method and after projection on the exoskeleton's actuated joint space, was meaningful. The second objective was to check that the amplitude of the force feedback adequately reflects the level of forces applied by the slave hand on its environment.

To this end, an experiment has been conducted during which a user was instructed to control the closure of the SoftHand so as to grasp and squeeze three foam balls (1), (2), and (3) of increasing stiffnesses. The fingertips position trajectories were monitored by the exoskeleton and projected on the synergy space so as to derive the coordinate along the first synergy *σ*1(*t*) used to control the SH. Results are presented in **Figure 8**. The interaction torque *τ int*(*t*) developed during each ball squeezing was estimated by the torque observer. Both *σ*1(*t*) and *τ int*(*t*) are shown on the top left plot. The direction of the forces actually applied at the fingertips by the exoskeleton is shown on the top right plot. To facilitate the comparison of the force amplitudes in the three conditions, the bottom plot presents a projection in the finger's plane of motion of the forces applied on the middle finger during the squeezing of the each ball.

These plots indicate that forces are applied along directions that oppose the user's hand closure along the first synergy, making coherent the synergy-based haptic closed-loop approach. Furthermore, the amplitude of the force applied at each fingertip increases with the grasping force, properly reflecting the interaction at the slave side. This behavior is also shown on Extension 1. After experimentally validating the effectiveness of the force feedback policy proposed, the possibility of introducing a simplified calibration procedure has been analyzed as discussed in the following section.

### **5. A SIMPLIFIED CALIBRATION PROCEDURE**

In the approach presented previously, the Cartesian-based synergy matrix needs to be computed for each user since it depends on the hand's kinematics and in particular on the phalanges' length. However, the associated procedure described in Sect. 3.2 can appear a bit long and wearing. As such, it might hinder the quick and easy use of the system by any new user. Hence, the idea to evaluate the possibility to project the fingertip position trajectory on a unique, user-independent synergy matrix (which will be referred to as the Simplified Calibration Procedure or SCP) rather than on the user's own synergy matrix (which will be referred to as the Full Calibration Procedure or FCP). In order to analyze the feasibility of this approach, the difference stemming from the use of the full versus simplified calibration procedure is examined

both in terms of position control of the slave's hand in Sect. 5.1 and in terms of force feedback in Sect. 5.2.

### **5.1. Position Control: A Comparative Analysis**

To analyze the effect of relying on the simplified rather than full calibration procedure, the fingertips position trajectory *Xuser*(*t*) *∈* R 3*m* of 7 users were recorded using the exoskeleton during a hand closing-opening motion.<sup>2</sup>

These trajectories have then been projected on each user's own synergy matrix [see equation (15)] and on another unique synergy matrix *Sx ref* 3 [see equation (16)].

$$
\sigma\_1(t) = \mathcal{S}\_{\text{uuser}}^{-1} X\_{\text{user}}(t) \tag{15}
$$

$$
\sigma\_1'(t) = \mathcal{S}\_{\text{xref}}^{-1} X\_{\text{user}}(t) \tag{16}
$$

Results indicate that the coordinate along the first synergy obtained through the SCP *σ ′* <sup>1</sup>(*t*) differs from the one obtained through the FCP *σ*1(*t*), as shown for two of the subjects on **Figure 9** (left hand side).

To reduce the intersubject variability as well as to scale the position reference to the slave's range of motion, the raw coordinate along the first synergy is subsequently normalized. This procedure simply requires to record the fingertips' position of each user with the hand fully opened and fully closed so as to compute the associated coordinates *{σ*<sup>1</sup> *min, σ*<sup>1</sup> *max}.* These values are used to scale in real time the coordinate along the first synergy, reflecting the user's current degree of hand closure, to a value between 0 and 1. The normalized coordinates obtained from the two calibration procedures are then compared. They present a high degree of similarity, as shown for two of the subjects on **Figure 9** (right hand side). To quantify the error introduced by the simplified calibration procedure with respect to the full one, the average Normalized Root-Mean-Square Error (NRMSE) between *σ*1(*t*) and *σ ′* <sup>1</sup>(*t*) after normalization is computed for each subject and presented in **Table 1**.

With an averaged NRMSE across all subjects of 2.32%, *σ ′* <sup>1</sup>(*t*) and *σ*1(*t*) are deemed to be similar enough, after normalization, to consider that the SCP does not affect the user's ability to intuitively and precisely control the position of the slave's hand.

In the following sections, the effect of using the simplified rather than the full calibration procedure are examined from the point of view of the force feedback generated.

### **5.2. Force Feedback: A Comparative Analysis**

This section addresses the comparison of the force feedback reference generated through the FCP [*F*(*t*); see equation (17)] in the

<sup>2</sup>These subjects belong to a panel of 5 men and 2 women aged from 25 to 36 years. 3 *Sx ref* is the synergy matrix of an 8*th* user (female, 9-cm index finger).

**FIGURE 9 | Comparison of** *σ***<sup>1</sup> and** *σ ′* **<sup>1</sup> before (left) and after (right) normalization**. The first row corresponds to a typical case (NRMSE = 1.85%), while the second row represents the subject with the largest NRMSE, or worst case (NRMSE = 4.25%).

**TABLE 1 | NRMSE between** *σ***1(t) and** *σ ′* **<sup>1</sup>(t) after normalization for each subject**.


one hand, and through the SCP [*F ′* (*t*); see equation (18)] in the other hand.

$$F(t) = \mathcal{S}\_{\text{xuser}}^{-T} \, k \, z(t) \tag{17}$$

$$F'(t) = \mathcal{S}\_{x \, ref}^{-T} k \, z(t) \tag{18}$$

To this end, an operator wearing the hand exoskeleton remotely controlled the squeezing of a ball by the slave hand. The interaction torque was recorded, and the corresponding force feedback references were computed from equation (17) using the 7 subjects' synergy matrices, as determined in Sect. 3.2, and are presented in **Figure 10** (left hand side).

Results are clearly different across subjects, indicating that the user's synergy matrix does affect the desired force feedback. Therefore, if a master device capable to apply such forces is to be used, relying on the simplified calibration procedure would alter the kinesthetic feedback in a non-negligible way. It would be interesting to analyze whether users feel more comfortable when the force feedback reference is generated using their own synergy matrix rather than a user-independent synergy matrix, i.e., if it does alter the teleoperation performances.

That being said, it is worth noting that the differences across subjects' mainly regard the force direction. However, the feedback device used in the proposed implementation is highly underactuated, such that the direction of the force that can be applied is posture-dependent. Consequently, in order to analyze the influence of the calibration procedure on the force feedback that is actually displayed to the user, the desired forces *F*(*t*) and *F ′* (*t*) need to be compared after projection on the exoskeleton's actuated joint space using equation (10). These forces are computed for all subjects and presented for one of them on **Figure 10** (right hand side).

After projection on the exoskeleton's actuated joint space, the desired forces synthesized from the two calibration procedures are parallel, such that they can be compared through quantifying their norm difference using equation (19).

$$error\ norm\ force = \left|\frac{||F|| - ||F'||}{||F'||}\right|\tag{19}$$

This error has been computed for the 7 subjects, and the averaged values for all data points and all fingers are presented in **Table 2**.

With an average value of the force norm error across all subjects of 4.73%, the influence of the calibration procedure on the force feedback applied by the hand exoskeleton is deemed marginal.

Finally, the analysis presented in this section indicates that the bilateral synergy-based approach proposed is robust enough to handle an approximation on the synergy matrix used. As such, a unique, user-independent synergy matrix can be used for users with different hand kinematics. Such simplified calibration procedure is considered to be adequate to generate both the position reference for the slave hand and the force feedback references for the hand exoskeleton device. In order to validate the hypothesis that users are able to effectively use the resulting interface, an experimental study has been conduced and is presented in the following section.

### **6. EXPERIMENTAL ASSESSMENT OF THE FRAMEWORK PERFORMANCES**

In order to convert the tele-grasping interface into a telemanipulation one, the proposed framework is further enriched

**TABLE 2 | Force norm error for each subject**.


with a robotic arm at the slave side, a tracking system at the master side as well as an associated control system, as described in Sect. 6.1. The effectiveness of the overall framework in enabling an operator to reach, grasp, and transport remote objects is then experimentally evaluated during two experiments presented in Sect. 6.2 and 6.3.

### **6.1. Robotic Arm Teleoperation**

The synergy port developed permits the bilateral teleoperation of the SoftHand from monitoring the user's finger posture using a hand exoskeleton. However, a grasp is completely defined by the combination of the hand's intrinsic and extrinsic DOFs, as underlined in Ciocarlie and Allen (2009). Indeed, the stable grasp of an object located in its environment requires the control of the wrist position and orientation during the reaching phase so as to place the robotic hand in an adequate configuration with respect to the object such that the further closure of the hand during the pre-shaping and grasping stages leads to a force closure grasp.

To address this requirement and enable the operator to manipulate remote objects, the teleoperation framework is further enriched with a robotic arm on which the SoftHand is mounted. Additionally, the vision-based motion capture system Optitrack is integrated at the master station, and a visual tracker is attached to the base of the exoskeleton. This augmented interface enables to monitor the position and orientation of the operator's wrist in the world frame. This signal is processed (filtering, frame transformations) and used as position reference trajectory in Cartesian space for the robotic arm's wrist. The control principle of the overall framework is schematized on **Figure 11**. Two robotic arms have been integrated to the framework: the humanoid robot COMAN in the one hand, and on the other, the KUKA LWR manipulator. While the Cartesian controller of KUKA could be used straightforwardly,

with COMAN a second controller layer has been implemented, as described in Brygo et al. (2014), to derive through inverse kinematics the arm's joints position trajectory from the robot's wrist 6D position trajectory reference.

So finally, the setup consists of a visual tracker (a) attached to the hand exoskeleton (b), a robotic arm (c) on which is mounted the SoftHand (d) as illustrated on **Figures 12** and **13**.

The effectiveness of the overall tele-manipulation framework is then experimentally evaluated, as presented in the following two sections as well as in Extension 1.

### **6.2. Tele-Manipulation with the Humanoid Robot COMAN**

A first experiment has been conducted that focused on evaluating the versatility of the platform through assessing the possibility to execute the stable grasp of a large range of different objects. To this effect, the user was instructed to drive the arm of the humanoid robot COMAN fitted with the SoftHand so as to reach for one of the items placed on a table, securely grasp it, lift it, and drop it

<sup>4</sup> [Right hand side figure] This worst-case plot shows the forces computed for the subject with the largest error as defined in equation (19) for the index finger (*error norm force* = 6.92%).

**FIGURE 12 | An augmented framework for performing remote tele-manipulations using the humanoid robot COMAN**.

**FIGURE 13 | An augmented framework for performing remote tele-manipulations using the KUKA LRW robot**.

off on a case, as shown in **Figure 12**. <sup>5</sup> The operator was invited to repeat this reach–grasp–transport–release cycle with a collection of 11 objects counting a remote control, a screwdriver, a water bottle, a phone charger, a pair of pliers, an emergency button, a chewing-gum box, a noise-canceling headset, as well as a dinosaur, a whale, and a leopard cuddly toys. These articles have been selected upon the diversity of their volumes, their geometries, and the mechanical properties of their materials that reflect the heterogeneity of the objects typically populating human workspaces.

After a short trial period to get used to the setup and the geometry of the objects, the operator successfully completed the task, grasping in a stable way and transporting all the items without dropping them. Besides illustrating the effectiveness of the proposed human–machine interface, this experiment demonstrates that the proposed platform achieves the level of versatility required to manipulate objects exhibiting very different shapes

and impedances. Such endowment confers to the framework a large flexibility, which makes it suitable for a variety of manipulation scenarios in unconstrained environments.

Additional characteristics of interests are examined in a second experiment, as presented in the following section.

### **6.3. Tele-Manipulation with the KUKA LWR Manipulator**

To further evaluate the effectiveness of the proposed bilateral telemanipulation interface, a second experiment has been carried out. With the twofold purpose of increasing the reachable workspace of the slave and of showcasing how the modularity of the framework permits the straightforward interchange of manipulators, the KUKA LWR manipulator has been used as robotic arm.

This experiment focused on assessing the universality of the proposed interface. The term universality refers here to a dual concept, such that the first objective was to analyze the capacity of the framework to accommodate for different users with different hand sizes and to rely on the simplified calibration procedure described in Sect. 5. The second aim was to evaluate the level of intuitiveness of the overall platform.

To this end, 10 subjects aged from 24 to 34 years took part to the experiment. Ethical approval was not required for this study as per the national and institutional requirements, and written informed consent was obtained from all participants. Their index and middle fingers' lengths, respectively, ranged between 8.3–9.9 and 9.3–11 cm. None of them ever used the platform before, and 5 of them never teleoperated a robot before. They were given 5 min to get used to the setup before starting the task, which consisted in picking up one by one the 6 items (a whale and a dinosaur soft toys, a phone charger, a noise-canceling headset, a remote control, and an emergency button) lying on the table and place them in a 21 cm *×* 18 cm *×* 15 cm box, labeled with the letter (e) on **Figure 13**. At the end of the experiment, subjects were invited to fill a form rating their experience on a 1–5 scale.

Results indicate that all the subjects successfully completed the task and managed to place all the objects in the target box in an average time of 3.4 min. To quantify the perceived intuitiveness of the overall platform, subjects were asked to answer to the question "How easy was it to control the slave robot (KUKA + SoftHand)?" on a scale from 0 (very difficult, non-intuitive) to 5 (very easy, very intuitive), which was rated as 4.3/5 in average. The average rating to the question "In order to understand how confident you are in using this platform, rate how stable/predictable you perceived it," was 4.2/5. Finally, subjects were asked to indicate whether they liked using the interface in overall terms, from 1

<sup>5</sup>Note that the right SH is mounted on the left arm because COMAN's right arm was under maintenance at the time of this experiment.

("No, I got frustrated. The slave was not doing what I wanted, or it was too difficult to achieve.") to 5 ("Yes, it is very nice to use"), which received a 4.7 rating in average. The distribution of the ratings for all three questions is shown on **Figure 14**.

Finally, this experiment clearly indicates that (i) the simplified calibration procedure is suitable and provides the bilateral control framework with the robustness and flexibility required to accommodate for different users with different hand sizes, relying on a simple 30 s calibration step; (ii) the platform is highly intuitive and allows naive users with no prior training to perform remote manipulation of objects of diverse shapes and impedances; (iii) the interface is perceived as likable and reliable, enabling to execute the desired operations.

### **7. CONCLUSION**

This work presents a novel bilateral control framework allowing for intuitive tele-manipulations of objects of various shapes and impedances using highly asymmetric master–slave systems. The keystone of the proposed control architecture consists of a bilateral Cartesian-based synergy port, which is introduced as a tool mapping the user's fingertips Cartesian space to the directions oriented along the grasp principal components. This module builds upon the results of an experimental study that revealed the possibility to describe the postures adopted by the human hand during grasping motions using a low-dimensional set of coordinates traveling along motion primitives expressed in the fingertips' Cartesian position space. A 1-min calibration procedure has been developed that enables the platform to accommodate different operators' hand kinematics. Using this module, both the slave's motion and the master's force feedback reference trajectories are designed in the Cartesian synergy space.

As such, the proposed control framework addresses two main challenges associated with the design of tele-manipulation interfaces. First, it effectively handles master–slave systems exhibiting large asymmetries at the kinematic, sensing, and actuation levels. This permits, through accepting natural manipulative motions as input, to abstract the operator from the setup's complexity so as to achieve a highly intuitive interface. Second, the bilateral control policies are synthesized in the hardware-independent hand synergy space. This makes possible the implementation of a 2-layer architecture where the high-level manipulation strategies can be designed in abstraction from the devices' specifics that are encapsulated in the low-level layer. Such modularity not only enables to flexibly interchange end-effectors according to each task but it also promotes the synthesis of universal manipulation algorithms. As such, the present work aims at fostering the development of unified tele-manipulation control frameworks and proposes the synergies as the alphabet of a common language for robotic hands control.

In order to evaluate the effectiveness of the proposed approach, the synergy port has been implemented into the control system of a highly asymmetric tele-manipulation platform. The underactuated 3-finger hand exoskeleton HEXOTRAC permits to monitor the user's hand posture and to display a kinesthetic feedback of the interaction forces, while the synergy-driven robotic hand SoftHand is used as slave end-effector to manipulate the remote environment. In addition, the position and orientation of the user's wrist is tracked by the vision-based motion capture system Optitrack and used as reference trajectory to drive the robotic arm on which the SoftHand is mounted.

Experiments have been performed with the humanoid robot COMAN as well as with the KUKA LWR manipulator. Results indicate that (i) the control framework adequately extracts the grasping intent of the user during natural motions, maps this input into references for the slave hand to perform the desired manipulative task, and reflects the impedance of the manipulated environment to the user; (ii) the platform is highly intuitive and allows users with no prior experience to securely grasp and transport objects exhibiting a large range of shapes and impedances after only a few minutes of familiarization with the system; (iii) not only do the hardware and the control solutions permit to accommodate users with different hand kinematics but they also allow for a fast donning and a short calibration procedure, turning the platform into a universal and fully practical interface.

Finally, this work investigated the possibility to use largely underactuated master and slave devices fitted with minimal sensing systems so as to achieve a simple and highly robust telemanipulation interface and analyzed the possibility to compensate for the associated hardware limitations by the mean of an adequate control strategy. Results indicate that the proposed control framework elegantly resolves the master–slave asymmetries and provides a universal and flexible interface for performing intuitive and effective tele-manipulations.

## **AUTHOR CONTRIBUTIONS**

AB: theoretical contribution: equations formulation of the synergy port, development of the calibration procedure, implementation of the software architecture (control framework, interface with all robotic systems, and experimental tools); experiments: design, realization, and analysis; video; and paper writing. IS: mechanical design of the hand exoskeleton; paper revision. GG: idea of the force feedback synthesis through inverse projection on the synergy space. NT: idea of the Cartesian-based synergy mapping to control the SH.

### **ACKNOWLEDGMENTS**

This work is supported by the Horizon 2020 Research and Innovation Program "SoftPro: Synergy-based Open-source Foundations and Technologies for Prosthetics and Rehabilitation" under Grant Agreement no. 688857 and from the European Union Seventh Framework Programme FP7/2007-2013 under Grant Agreement no. 601165 of the project "WEARHAP—WEARable HAPtics for humans and robots."

### **SUPPLEMENTARY MATERIAL**

The Supplementary Material for this article can be found online at http://journal.frontiersin.org/article/10.3389/fbioe. 2017.00019/full#supplementary-material.

**VIDEO S1 | Synergy-based bilateral port: a universal control module for intuitive tele-manipulations using asymmetric master-slave systems**.

### **REFERENCES**


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Copyright © 2017 Brygo, Sarakoglou, Grioli and Tsagarakis. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

*Vrajeshri Patel, Jamie Craig, Michelle Schumacher, Martin K. Burns, Ionut Florescu and Ramana Vinjamuri\**

*Sensorimotor Control Laboratory, Department of Biomedical Engineering, Chemistry, and Biological Sciences, Stevens Institute of Technology, Hoboken, NJ, USA*

Traditionally, repetitive practice of a task is used to learn a new skill, exhibiting as immediately improved performance. Research suggests, however, that a more experience-based rather than exposure-based training protocol may allow for better transference of the skill to related tasks. In synergy-based motor control theory, fundamental motor skills, such as hand grasping, are represented with a synergy subspace that captures essential motor patterns. In this study, we propose that motor-skill learning through synergy-based mechanisms may provide advantages over traditional task repetition learning. A new task was designed to highlight the range of motion and dexterity of the human hand. Two separate training strategies were tested in healthy subjects: task repetition training and synergy training versus a control. All three groups showed improvements when retested on the same task. When tested on a similar, but different set of tasks, only the synergy group showed improvements in accuracy (9.27% increase) compared to the repetition (3.24% decline) and control (3.22% decline) groups. A kinematic analysis revealed that although joint angular peak velocities decreased, timing benefits stemmed from the initial feed-forward portion of the task (reaction time). Accuracy improvements may have derived from general improved coordination among the four involved fingers. These preliminary results warrant further investigation of synergy-based motor training in healthy individuals, as well as in individuals undergoing hand-based rehabilitative therapy.

Keywords: kinematic synergies, hand, motor learning, principal component analysis, rehabilitation

# INTRODUCTION

From learning to grasp a ball to learning to type at a keyboard, we are continuously tasked with acquiring new motor skills throughout life. There is a dynamic combination of both cognitive (strategy formation and task comprehension) processes and motor processes (feedback integration and motor execution) that allow us to learn and execute these motor skills. In the 1940s, Nicholais Bernstein provided an intriguing definition of a motor skill: "… not a movement formula … [but] an ability to solve one or another type of motor problems" (Latash and Latash, 1994). Subsequently, motor-skill learning can be defined as "a set of processes associated with practice or experience leading to relatively permanent changes in the capability for responding" (Schmidt, 1976). In these definitions, an emphasis is placed on learning a response through practice rather than memorizing motor patterns. Much research has been dedicated to determining what phenomena occur

*Edited by:* 

*Alessandro Tognetti, University of Pisa, Italy*

#### *Reviewed by:*

*Fu Qiushi, Arizona State University, USA Claudia Casellato, Polytechnic University of Milan, Italy*

> *\*Correspondence: Ramana Vinjamuri ramana.vinjamuri@stevens.edu*

#### *Specialty section:*

*This article was submitted to Bionics and Biomimetics, a section of the journal Frontiers in Bioengineering and Biotechnology*

*Received: 20 November 2016 Accepted: 03 February 2017 Published: 27 February 2017*

#### *Citation:*

*Patel V, Craig J, Schumacher M, Burns MK, Florescu I and Vinjamuri R (2017) Synergy Repetition Training versus Task Repetition Training in Acquiring New Skill. Front. Bioeng. Biotechnol. 5:9. doi: 10.3389/fbioe.2017.00009*

during practice and how these phenomena lead to a permanently acquired motor skill.

It has long been known that repetition is essential to learning a new skill, or procedural learning. Skill learning theories suggest that motor skills are initially developed in a fast cognitive-based stage. During this time, explicit learning is based upon declarative knowledge and working memory. At the neuronal level, we see decreased inhibition allowing for increased excitability in the primary motor cortex (M1) (Karni et al., 1995). Simultaneously, structural changes such as increased myelination (Sampaio-Baptista et al., 2013) and clustering of new dendritic spines (Fu et al., 2012) occur in various relevant pathways including multiple M1 layers (Huber et al., 2012), sensorimotor cortex (Sampaio-Baptista et al., 2013), and cerebellum (Cantarero et al., 2015). These changes support the ascension from the cognitive phase to the associative phase, where ineffective actions are filtered out. Individuals become unconsciously sensitive to regularity and other implicit learning mechanisms. After extensive practice, the autonomous stage is reached; retention of the procedural memory of the motor skill is thought to be stored in cortical– striatal circuits (Doyon and Benali, 2005).

Repetition training is often compared to interleaved training, which involves a more varied protocol. Interleaved training may engage more prefrontal and parietal regions because each task requires the individual to reconfigure motor commands (Li and Wright, 2000; Kantak et al., 2010). Repetition training versus interleaved training, however, allows for greater M1 excitability. It has been found that interleaved training performs worse than repetition training in short-term performance, but better in longterm performance transference (Dromerick et al., 2009). The varied presentation may promote implicit learning because it provides greater exposure to correlated variables (Meier and Cock, 2010). It is apparent that both repetition and interleaved practice have advantages and disadvantages; consequently, balancing both learning strategies may allow us to optimize motor-skill learning.

This balance of these learning variables is especially important for individuals in rehabilitative therapy that must relearn essential skills, with limited time and usually, limited muscle strength. Because the individuals are relearning previously acquired motor skills, researchers have turned to natural motor control and motor learning concepts to determine ways of optimizing physical therapy strategies (Krakauer, 2006. Muratori et al., 2013). For example, task-orientated training with spaced practice versus conventional massed practice (learning with short or no intervals) may promote long-term memory of the learned motor skills (Dromerick et al., 2009). The ability for implicit and explicit learning in individuals with stroke has also been studied. Depending on the location of cerebral damage, implicit learning may be compromised (Ackermann et al., 1996; de Guise et al., 1999), and explicit information may be detrimental (Boyd and Winstein, 2003). This suggests that skills normally learned *via* implicit and explicit mechanisms need to be presented in a different format.

In this study, we propose a new mechanism of motor training: synergy training. In its most general definition, synergies represent learned motor primitives that reduce the computational burden of the central nervous system (CNS). For example, throughout life, the hand's ability to dexterously grasp and manipulate objects found in activities of daily living is a skill that requires control over 30 degrees of freedom. It is hypothesized that after years of learning, the motor control system has optimized the "reach and grasp" motor task. This skill may be represented in the CNS as motor synergies that encode simultaneous coordination of the many involved joints versus individual control of each joint. Using dimensionality reduction techniques, we (Vinjamuri et al., 2010; Patel et al., 2015; Burns et al., 2017) and others have characterized this synergy subspace at various hierarchical levels including neural (Saleh et al., 2010), muscle (d'Avella et al., 2011), and kinematic (Santello et al., 2002; Vinjamuri et al., 2014). If synergies represent motor strategies that have been optimized through experience (involving both explicit and implicit learning mechanisms), they may useful during the learning experience itself. Furthermore, continued advancements in robotic technologies will soon allow therapist to implement synergy training in individuals who have altered synergies, poststroke (Cirstea and Levin, 2000; Michaelsen et al., 2001; Roby-Brami et al., 2003; Zackowski et al., 2004; Neckel et al., 2006; Roh et al., 2013; Jarrassé et al., 2014).

In this study, we model the potential effects of synergy-based training using a simplified motor-skill learning framework. In order to keep the study related to hand motor skill, we design a new task that requires users to coordinate control of joints in the four fingers. We compared traditional task repetition training with a new synergy-training protocol to determine the effects of each method. Subjects are tested after a specific training procedure to measure retention. Additionally, they are then tested on a separate set of tasks to measure transference. We hypothesize that the task repetition group will exhibit stronger performance during the retest phase, while the synergy group will exhibit stronger performance during the transference phase.

## MATERIALS AND METHODS

### Overview

For this study, 16 right-handed healthy individuals were recruited (10 males, 6 females, mean age 21.5 ± 1.5) under Stevens Institute of Technology Institutional Review Board approval. Through selfreport, subjects that were mildly skilled musicians were excluded from the study. Subjects were assigned to one of three groups: task repetition training, synergy training, and control. Data from one subject (subject 2) were removed because of data collection complications, resulting in five subjects in each group.

### Experimental Procedure

A user interface was created in LabVIEW 2014 (National Instruments Corporation, Austin, TX, USA) using Touchscreen Toolkit (Aledyne Engineering, Morgan Hill, CA, USA). The interface was displayed on a touchscreen monitor with multi-touch compatibility (Acer, San Jose, CA, USA). As seen in **Figure 1**, a 3 × 4 button grid is displayed on the touchscreen. Each column, from left to right, corresponds to the index, middle, ring, and

pinky. Three rows represent three ranges of extensions that each finger will have to achieve. The alignments of these buttons were subject based. Before testing began, the subject was seated in the experimental setup (wrist strapped down, which is further described below). Starting in a closed fist position, the subject was asked to extend to three comfortable levels. The first level requires that the subject has enough extension so that either the fingertip or finger pad makes contact with the screen. The third level requires the subject to extend as much as possible, while still making contact with the screen. The second level fell between these two levels. Touch points (**Figure 1A**) collected and imported into MATLAB (MathWorks, Natick, MA, USA). Pixel locations of these touchpoints determined button locations (after minor adjustments to account for button size), which were then imported by LabVIEW (**Figure 1B**).

A single touch task requires subjects to touch the four buttons that are lit. Out of a total of 81 possible button combinations; four button combinations were removed due to anatomical difficulty in achieving the postures. The remaining 77 combinations were randomly divided into two sets: set 1 consisted of 45 button combinations and set 2 consisted of 32 button combinations. The procedure for a single task is as follows: subjects started with hands in a closed fist position. An audio start cue coincided with four target buttons lit green. An audio stop cue was given after either four buttons were touched (incorrect or correct) or 4 s had passed. During the task, if a button was touched, the outer rim of the button turns red (**Figure 1B**). If all four target buttons are correctly touched, a "correct bar" lights green. Between each task, a 3.5 s break is given, during which the subject returns his/ her hand to a fist position. Subjects were instructed to complete each task as "accurate" and "simultaneous" as possible. Accuracy pertains to pressing the four correct buttons and simultaneous pertains to pressing each button simultaneously versus sequentially. In order to keep the task confined to finger movements only, the wrist was strapped to a board positioned above the touchscreen.

FIGURE 2 | The experiment consists of four phases. In Phase 1, subjects perform 3 trials of 45 tasks. Phase 2 involves different forms of training. Subjects in the task repetition group repeat the same 45 tasks twice and with extra time. If subjects are in the synergy repetition group, eight postural synergies were derived from kinematic data in Phase 1. These subjects trained on performing the eight postural synergies. Subjects in the control group were required to rest for ~10 min. In Phase 3, all subjects were retested on the tasks performed in Phase 1. In Phase 4, 32 new tasks were introduced to test transference of the new skill.

#### Phase 1

The experimental procedure (**Figure 2**) consisted of four phases: Phase 1—preliminary evaluation, Phase 2—training, Phase 3 retest evaluation, and Phase 4—transference evaluation. In Phase 1 (preliminary evaluation), subjects performed a total of three trials. Each trial consisted of performing the 45 tasks in set 1 and lasted a maximum of ~6 min. An optional 1 min break was given between trials. The first trial allowed subjects to familiarize themselves with the task and task procedure; therefore, data from trial 1 were discarded. Trials 2 and 3 were used to determine baseline performance for each subject.

#### Phase 2

In Phase 2 (training phase), subjects were trained according to their assigned group. Subjects in the task repetition training group performed 2 additional trials of the same 45 tasks from set 1, but without the time constraint (timeout of each task was set to 8 s, and intertask breaks were set to 6 s) and wrist constraint. Subjects were told to use this phase and the extra time to improve their accuracy and simultaneity. Subjects that were assigned to the synergy-training group practiced eight postural synergies derived from Phase 1, trials 1 and 2. Postural synergy derivation and synergy training is described in detail in Section "Derivation of Postural Synergies and Synergy Training"; a brief description is provided here. In synergy training, subjects were first familiarized with each of the 10 derived synergy postures, which were displayed on a computer screen (**Figure 3**). Still wearing the CyberGlove, the subject practiced performing each posture until a minimum error between the hand posture and the displayed synergy posture was reached (after approximately two to three attempts). Then, starting in a closed fist position, each posture was quickly performed for three repetitions (each posture was queued by displaying the synergy posture on the screen). The goal of this training was to have subjects become comfortable and familiar enough with each synergy posture that they are able to rapidly perform it when queued. Additionally, subjects were

explicitly told that these postures were to be learned as much as possible to "help" improve task performance. We attempted to keep training procedures for both the task repetition training and synergy-training groups as even as possible by implementing the following procedures. First, the maximum time allowed for the training phase was kept to 20 min. During this time, the task repetition training group performed ~90 postures, and the synergy-training group performed ~60 postures. Second, the following feedback was implemented in both groups. For the task repetition group, just as in Phase 1, the red button outlines indicated when buttons were touched, and a green bar ("Correct Indicator") indicated when tasks were correctly performed; for the synergy group, numerical values indicated which joints were not adequately similar to those of the displayed synergy posture. Joint angles within a 20° error range were considered acceptable. Third, both groups maintained creating postures from a closed fist position. Fourth, the time constraint was removed, so that subjects could focus on the creation of each posture. Fifth, because the wrist needed to be free while learning synergy postures, in both training groups, a wrist constraint was not used. Subjects that were assigned to the control group rested for ~10 min during the Phase 2.

#### Phase 3 and Phase 4

In Phase 3, subjects were retested with 2 trials of the same 45 tasks from set 1. This phase was used to determine improvements from baseline evaluation. In Phase 4, 32 additional tasks (set 2) were introduced. Two trials were performed. These trials were used to test the transference of motor skills gained from Phases 1–3 to similar, but untested tasks.

To compare the similarity of the 45 tasks in set 1 and the 32 tasks in set 2, we looked at the overall frequency in which each button was pressed as well as the general patterns found in each group. **Figure 4A** shows each of the 12 buttons numbered in black. **Figure 4B** shows the percent of tasks that involved each button. Buttons 5 and 12 were proportionately used more often in set 1 tasks while buttons 6 and 11 were proportionately used more often in set 2 tasks. Principal component analysis (PCA) was then used to determine if the overall required patterns are statistically similar between set 1 and set 2. Of the 12 total principal components derived, the first three are shown in **Figure 4C**. Statistically, the pattern described from **Figure 4B** and seen in PC1 of **Figure 4C** accounts for ~34% of the total variance (**Figure 4D**) for both groups. PCs 2–9 account for the remaining variance. Thus, the majority of the variance is scattered equally across the 12 buttons; however, there is a slight skew toward buttons 5 and 12 in set 1 tasks and buttons 6 and 11 in set 2 tasks. This is further discussed in Section "Discussion."

### Data Collection

A right-handed CyberGlove (CyberGlove Systems LLC, San Jose, CA, USA) equipped with 18 sensors was used to capture hand movements during the experiment at a rate of 125 Hz. In this study, only data from the metacarpophalangeal (MCP) and proximal interphalangeal (PIP) joints of the four fingers were used. The glove was calibrated for each subject using custom goniometers ranging from −10° to 90°. Once the glove was dawned, it was not removed until the study was completed. Subjects also wore a glove liner that was coated with a liquid allowing the fabric to be touchscreen compatible. CyberGlove data were recorded with the same LabVIEW program that controlled the task. Additionally, for each task, the identity of the buttons that were pressed and the task completion time (CT) were recorded.

### Derivation of Postural Synergies and Synergy Training

In this study, we hypothesized that synergy-based training provides a means of reinforcing spatial joint patterns that generalize to a large range of tasks. In order to determine these movement patterns, we used PCA, a commonly used dimensionality reduction technique for synergy derivation (Santello et al., 2002; Thakur et al., 2008; Vinjamuri et al., 2010). End postures taken from Phase 1, trials 1 and 2, provides joint configurations for 45 tasks, with 2 repetitions each. The mean posture across both repetitions was used to create an *m* × *n* joint angle matrix (*J*), where *m* is the number of tasks (*m* = 45) and *n* is the number of joints (*n* = 8). Singular value decomposition is used to approximate *J* such that:

$$J = U\sum \mathcal{R}',\tag{1}$$

where orthogonal *U* (*m* × *m*) contains left singular vectors, orthogonal *R* (*n* × *n*) contains right singular vectors, and Σ (*m* × *n*) contains the square root of singular values in its diagonal. The rows of *R*′ contain eigenvectors of *J*′*J*, or principal components. These eight principal components are considered synergies. Therefore, we have a total of *s* = 8 synergies. Importantly, although these synergies were derived from only a subset of all possible tasks (Phase 4 tasks are not included), they each emphasize specific joint patterns that can then be combined to produce new postures.

After synergies were derived, each synergy vector was multiplied by a maximum possible weight such that the joint angles

FIGURE 4 | (A) The 3 × 4 button grid required three levels of extension (labeled in blue) for each finger [index (I), middle (M), ring (R), and pinky (P)]. For reference, each button number is provided in black. (B) The percent number of tasks that involved a specific button is shown for set 1 (blue) and set 2 (red). Buttons 5, 6, 11, and 12 showed differences in frequency. (C) The first 3 (of 12) principal components derived from set 1 and set 2 tasks are shown. In PC1, weight proportions are similar to (B). In the remaining PCs, the covariance patterns are distributed. (D) The fraction of variance that each of the 12 PCs account for is shown. Distributions of variance are similar across both groups.

still fell within the range of normal movement [set from −10° to 90° for MCP joints and 0° to 90° for proximal interphalangeal (PIP) joints]. A separate LabVIEW program was used to display the resulting synergy postures. A virtual hand model (developed using the Simulink 3D Animation toolbox from MATLAB) showed the target synergy posture. As seen in **Figure 3**, top and side views were provided. Numerical indicators showed the target angle for each joint as well as the subject's current joint angle. The synergy training procedure involved the following steps:


then queues the next posture. This was repeated for a total of three rounds.

### Reconstruction

In order to determine how subjects may have recruited synergies in Phases 3 and 4, the end posture of each task was reconstructed using weighted synergies. Let *w* (1 × *s*) represents the weight of each synergy and *S* represents the synergy matrix, which is equal to *R*′. Each end posture is represented by joint angles contained in *A* (*n* × 1). The following optimization problem (Vinjamuri et al., 2010) was used in the selection of synergies and weights:

$$\text{Minimize } \|\|\boldsymbol{\omega}\|\|\_{1} + \frac{1}{\lambda} \|\|\boldsymbol{\omega}\mathbf{S} - \boldsymbol{A}\|\|\_{2}^{2},\tag{2}$$


#### Data Analysis

Task-related variables were compared across phases (Phase 1 preliminary evaluation, Phase 3—retest evaluation, and Phase 4—transference evaluation) and across the three groups—task repetition, synergy, and control. Data from the two trials in each phase are accumulated.

To measure acquired motor skill in each phase, the percent correct (PC) and their average CT were measured. Angular data recorded from eight sensors on the data glove were filtered with a 5 Hz with a low-pass Butterworth filter. Various variables were used to detect any training-related kinematic changes. This includes reaction time (RT), peak velocity (PV), time of peak velocity (tPV), and overextension (OE). For each task, RT is defined as the first time one of the eight joints reaches 1% of PV. In each task, the magnitude and time at which PV occurs in each joint were recorded and then averaged to measure PV and tPV, respectively. As a measure of movement efficiency, we cal culated OE at each joint, using the difference between a subject's maximum extension and final position, to determine if a joint was overextended.

In Phase 4, subject's performed tasks that were previously un-encountered. To determine if synergy postures were being incorporated into these movements, the final posture from each task was reconstructed using subject-specific synergies. The RE was measured as the Euclidian error between actual and recon structed postures, summed across the eight joints.

Finally, spatial trends were evaluated using the position of each button. Tasks that involved a specific button (1 of the 12) were first grouped. The percentage of tasks correct in this group was measured for each subject. This was repeated for all 12 buttons.

First, to verify parity across the three groups during Phase 1 (preliminary evaluation), a one-way analysis of variance (ANOVA) test for PC, CT, RT, PV, tPV, and OE was performed. We found that group means obscured individual subject changes. Thus, to offset subject differences, we first measured how a vari able changed between phases. Accordingly, rather than using a two-way repeated measures ANOVA, a one-way ANOVA across groups was used for each phase. Multiple comparisons were used to test significant results using a Tukey–Kramer test (*p* < 0.05). For variables specific to Phase 4 (RE, button-specific perfor mance), one-way ANOVA were used to detect differences across groups. Because of non-normal distribution in RE, even after log transformation, a non-parametric Kruskal–Wallis test was also performed. In all tests, significance was set to *p* < 0.05.

#### RES U LT S

### Task Performance and Kinematics

Results for task performance and kinematic measures are pre sented in **Table 1**. Across all phases and groups, subjects' scores for percent correct (PC) ranged from 32 to 92%. **Figure 5**, however, shows an outlier subject (subject 6), while all other performance scores were similar. CT averaged ~2 s, of which the first ~0.29 s was RT. tPV and time of peak extension (tPE) occurred at ~0.65 and ~1.14 s, respectively. Average OE across joints was ~6° but ranged from 0° to 60°.


*Means and 1 SD are given.*

Phase 1 (preliminary evaluation) measurements were tested using a one-way ANOVA to ensure parity across the groups. **Table 2** (column 2) shows that none of the variables showed significant difference across groups. However, as seen in **Table 1**, within-group SDs for all measured variables are relatively large. Thus, to reduce the effect of intersubject differences, we calculated how measurements for each variable changed across phases. **Table 2** shows group averages for these changes from Phase 1 to Phase 3 (retest evaluation) and from Phase 3 to Phase 4 (transference evaluation). ANOVA results showed a significant group difference in tPE after training (*p* = 0.034). Subjects in

both the task repetition group and synergy group were able to reach peak extension faster after training. However, the control group averaged an increase in tPE. Multiple comparisons show only a significant difference between the synergy group and control group (*p* = 0.030). A significant group difference was found for changes in PC from Phase 3 to Phase 4 (*p* = 0.019). When tested on the second set of tasks in Phase 4, subjects in the synergy training group were able to improve their performance while subjects in the task repetition training group and control group showed decreased performance. This is more clearly shown in **Figure 5**, which provides PC for each subject in different phases. Multiple comparisons showed that the average PC change for the synergy group was significantly greater than that of the task repetition group (*p* = 0.033) and control group (*p* = 0.033). A significant group difference was also found for PV. Subjects in the control group expressed an average increase of 24.3 ± 19.71 /s in PV. This was significantly different than synergy group's average decrease of 6.27 ± 16.73 /s (*p* = 0.037). No clear trends or significant differences were observed for CT, RT, and OE.

Because many of the kinematic variables are averaged across joints, we further explored how the four fingers individually executed the tasks. A temporal analysis revealed interesting trends that spanned all three groups. **Figures 6** and **7** show how the same task (task # 26 from Phase 4) was executed by a representative subject from the task repetition group, synergy group, and control group. In **Figure 6**, profiles for each of the eight recorded joints are presented and are overlaid by their reconstructed versions. In general, index and pinky MCP joints averaged the fastest times to reach peak extension, but their PIP joint extensions occurred last. The index MCP joint also had exhibited the most OE for all three groups. Time to reach peak extension in the middle and ring PIP joints averaged similar times. For each joint, we separately recomputed time to peak extension and OE. Betweengroup ANOVA results are provided in **Table 3**. At all four MCP joints, the task repetition group showed significantly greater OE than the synergy group and control group. However, M\_MCP, R\_MCP, P\_MCP, I\_PIP, and M\_PIP joints in the task repetition group reached peak extensions significantly faster than synergy and control groups (*p* < 0.05).


*Column 2 p values show that no significant differences between groups were found for any variables during Phase 1 (group averages are provided in Table 1). Changes from Phase 1 to Phase 3 and from Phase 3 to Phase 4 are presented in columns 3–5 and 7–9 for each group: task repetition training (TR), synergy repetition training (SR), and control (C). PC, percent correct; CT, completion time; RT, reaction time; PV, peak velocity; tPV, time of peak velocity; tPE, time of peak extension; OE, overextension. Significant group differences are indicated in bold (p* < *0.05).*

FIGURE 6 | The kinematic profile for task # 26 from representative subjects from each group [task repetition (TR)-blue, synergy repetition (SR)-red, and control (C)-green]. Metacarpophalangeal (MCP) joints and proximal interphalangeal (PIP) joints are shown in the top and bottom rows, respectively. Finger abbreviations are index (I), middle (M), ring (R), and pinky (P). Each joint profile has an inset that magnifies the profile at the end of the task, where differences between original (o, solid lines) and reconstructed (r, dotted lines) can be better appreciated.



*Post hoc significant results are bolded.*

*a A significant difference between task repetition and synergy repetition groups and between task repetition and control groups.*

*bSignificant difference between the task repetition and synergy repetition groups.*

*c Significant difference between task repetition and control groups and between synergy repetition and control groups.*

*TR, task repetition; SR, synergy repetition; C, control; MCP, metacarpophalangeal; PIP, proximal interphalangeal; I, index; M, middle; R, ring; P, pinky.*

### Utilization of Synergies

Subjects in the synergy training group were each trained on eight distinct postural synergies. These postures are presented in **Figure 8**. The first synergy is similar across all subjects and is characterized by MCP and PIP flexion, albeit at different magnitudes. The remaining postures emphasize alternating relationships among the fingers. Importantly, difficult and less commonly used postures have been captured by these synergies. For example, synergy 6 from subject 1 shows the ring PIP joint flexion while the pinky MPC and PIP joints are extended. Because the pinky is partly enslaved to the ring finger, this posture requires conscientious digression from natural behaviors.

Results from the task performance analysis show that the synergy group performed significantly better in the Phase 4 (transference evaluation) compared to the task repetition and control groups. Thus, the next step is to determine how well synergies were being incorporated into these movements. Correct tasks in Phase 4 were reconstructed using a subject's corresponding synergies. For comparison, synergies of the task repetition and control groups were also derived and used to reconstruct end postures. An example reconstruction of each joint was provided in **Figure 6**. RE is used as a measure of synergy usage and is presented in **Figure 9**. Note that only the error from the end posture, and not the entire time profile was used. Results show that end postures in the synergy group were reconstructed with significantly less error than both the repetition group (ANOVA, *p* = 3.8e−9; Kruskal–Wallis, *p* = 4.45e−6) and control group (ANOVA, *p* = 1.83e−7; Kruskal–Wallis, *p* = 0.001). These results show that postures used during Phase 4 more closely resembled synergies in the synergy group, than in the repetition and control groups.

### Task Analysis

Results indicate that subjects in the synergy group were able to employ postures that they were trained on. These practiced postures possibly led to the performance improvements previously described. To further explore the benefits of practicing these different hand configurations and how they may have been incorporated into the task, a spatial analysis was used to evaluate the performance of different fingers. Phase 4 tasks that involved a specific button were grouped. Then, the performance of each group across these tasks was measured. Results are shown in **Figure 10**. The distribution of performance in all 12 buttons was similar across the three groups. Specifically, all three groups showed the worst performance when moderate extension was

FIGURE 10 | Individual button-based performance was calculated for each group. The color scale indicates the percentage of tasks correct with lighter colors representing better performance. The synergy repetition group shows better performance overall but only was only significantly greater than the task repetition group (*p* = 0.032) at ring, maximum extension (indicated by the \*).



required of ring MCP and PIP joints (third column, middle row), but best performance when maximum extension was required of ring MCP and PIP joints (third column, top row). ANOVA results showed a significant group difference at this location (*p* = 0.0252). The synergy group was able to perform these tasks significantly better than the task repetition group (*p* = 0.032) but did not reach significance for the control groups (*p* = 0.058). All other button locations show that the synergy group had distributed advantages.

## DISCUSSION

Motor learning is often characterized by acquisition, retention, and transference of a new motor skill (Magill and Anderson, 2007). Quantitatively, motor learning exhibits as improvements in time and accuracy. In this study, we compared two different training groups, task repetition (gold standard) and synergy training, to assess differences in motor learning. Our results (summarized in **Table 4**) show that when the same tasks were tested, both, task repetition and synergy training group, showed greater improvements (increased percent correct and faster time to reach peak extension) than the control group (**Tables 1** and **2**). We expected subjects in the task repetition group to perform better than subjects in the synergy group during Phase 3 (retesting). Significant advantage was only demonstrated when excluding subject 6, who had outlier (third SD) performance in PC. Regardless, subjects from both these groups, at least for short term, were able to retain learned task dynamics. However, when tested on the new task, only subjects in the synergy training group were able to transfer their new motor skill, as evidenced by continued improvement in accuracy. The better performance was not concentrated by a single finger or extension level but spanned all 12 buttons as seen in **Figure 10**. It is worth noting that the button analysis showed that buttons 5 and 12 (see **Figure 4**) were used more often in set 1 tasks and buttons 6 and 11 were used more often in set 2 tasks. **Figure 10**

shows that all groups may have been affected by being tested on buttons that they were less exposed to during training (although the control group shows the same trend). In other words, button 6 and 11 locations showed lower averages compared to other buttons. However, it is unclear if this resulted from limited pre-valuation/training exposure or that the button location themselves were more difficult. Next, we discuss how both training methods may have influenced motor learning.

In our study, numerous explicit learning mechanisms were implemented. For example, if a task was performed correctly, a "correct indicator" was lit green. Subjects were verbally told to accomplish tasks simultaneously across all fingers. Additionally, subjects familiarized themselves with start and stop cues so that the task could be accomplished within the allotted time. These factors contribute to explicit learning using external cues provided by the experimenter. Subjects in the task repetition group were more exposed to explicit mechanisms because their training phase provided more time in the task environment. The extra allotted time also allowed cognitive processes to create an optimal motor plan for each task. Subjects in the synergy training group, however, were trained using different explicit mechanisms, synergies.

The development of hand synergies, as well as other motor synergies (i.e., walking), begins early on when infants use imitation and/or self-regulated learning mechanisms to achieve a grasp (Oztop et al., 2004). Between 9 and 13 months, reach and grasp motor patterns appear pre-programmed, evidenced by temporal overlap and early anticipatory movements (Lockman et al., 1984; Newell et al., 1993). Konczak et al. (1995) found that a "fine tuning" period in infants 7–18 months was characterized by stable joint torque patterns over time, and across limb segments. Further analysis of shoulder, arm, and hand kinematics showed that these infants express stereotypical kinematic patterns only after 24 months (Konczak and Dichgans, 1997). Through mainly implicit learning mechanisms, these infants stored the most effective and common motor sequences required for grasping, optimizing them throughout life. Computationally, we derive these synergies through dimensionality reduction techniques, which capture primed inter-joint coordination. In this study, a new touch task required subjects to develop new inter-joint coordination techniques in order to complete the tasks correctly and quickly. All three groups may have implicitly learned useful inter-joint coordination during the initial evaluation (Phase 1) and retest (Phase 3). However, subjects in the synergy group received more concentrated training on these inter-joint coordination patterns through postural synergy training. The reconstruction analysis (**Figure 5**) indicates that these trained postures were used during the transference tests. Additionally, kinematic analysis indicates that synergy training affected the feed-forward mechanisms (resulting from motor planning) allowing joints to reach their most extended configurations, quicker. In the task repetition group, joints were extended quicker, but this also caused significantly greater OEs in Phase 4.

Although only healthy adults were used in this study, our results show that synergy training may be able to address some of the requirements of poststroke physical therapy. For example, the question of whether therapy should be constant versus variable has been addressed in numerous studies (Lin et al., 2008; Wu et al., 2011). While constant, repetitive practice reinforces positive mechanisms, it may reduce the ability to transfer a skill (Dromerick et al., 2009) potentially because of less exposure to all the inherent task patterns (Meier and Cock, 2010). Concurrently, the synergy group concentrates only these patterns. Additionally, equal training time is given to all patterns, whether common or not. This type of training resulted in spatially broad advantages (**Figure 10**) during the transference tasks. However, intense repetition training also leads to improvements in function after stroke (Kawahira et al., 2004). Thus, a balance between repetition and synergy training may provide optimal results. Additionally, the implications of "whole versus part" training in neurorehabilitation have been explored (Schmidt and Lee, 2011; Wickens et al., 2013). For example, in relearning "reach and grasp," poststroke, it is important to maintain the overlapping temporal relationship between arm transport and hand grasp. While synergies derived in this study were static postures, training with spatio-temporal synergies (Vinjamuri et al., 2010) would allow individuals to reinforce temporal relationships. Finally, evidence suggests that in adults with neurological damage in certain brain areas, explicit instructions can lead to poorer performance than implicit instructions (Boyd and Winstein, 2003), suggesting that rehabilitation efforts need to balance how supposed explicit and implicit knowledge can be delivered. This balance may be reached with the use of synergies.

While this study attempted to model the benefits of synergybased learning for potential use in motor learning as well as hand rehabilitation, there are some limitations to consider. First, the sample size in this data set is quite small. Based on results of the main outcome measure of this study (PC in Phase 4), we would need a sample size of 20 subjects in each group for 80% power (α = 0.05). Thus, the current low sample size of *n* = 5 in each group only achieves has an extremely low power and thus, high type II (false negative) error rate. A significant group difference was indeed found (Cohen's *d* = 0.401, moderate effect), but only after accounting for intersubject differences. Future studies may have to establish a more equivalent baseline with lower SDs. Second, synergies that were used for training were not necessarily optimal because they were acquired relatively early in training stages. Moreover, they were subject specific. In a realistic setting, synergies need to be derived from healthy, skilled individuals to be used on unskilled individuals. Minor unnatural discrepancies between individuals may cause undue difficulty in training. Third, the task used in this study was created to balance novelty and finger range of movement. Other hand-related motor tasks, such as the serial RT task (Robertson, 2007), have been extensively researched in their ability to expose different motor-learning strategies. In designing the task for this study, we attempted to balance novelty with hand-related function. Further investigation is required to delineate the explicit mechanisms that may have occurred during the task and also determine their effects on long-term retention of the skill.

### ETHICS STATEMENT

The study was approved by IRB at Stevens Institute of Technology. Consent procedures were followed according to the IRB guidelines at Stevens. All subjects were individuals without any disabilities.

### AUTHOR CONTRIBUTIONS

Methods and experimental setup were designed and implemented by VP and JC, with guidance from RV. MS

### REFERENCES


helped with data collection and data analysis. MB helped create data collection program. IF helped with statistics. Manuscript was written by VP and revised and approved by all authors.

### ACKNOWLEDGMENTS

The authors would like to thank the Department of Biomedical Engineering, Chemistry, and Biological Sciences of the Stevens Institute of Technology for their continued support of ongoing research.


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Copyright © 2017 Patel, Craig, Schumacher, Burns, Florescu and Vinjamuri. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# **Low-Dimensional Synergistic Representation of Bilateral Reaching Movements**

*Martin K. Burns, Vrajeshri Patel, Ionut Florescu, Kishore V. Pochiraju and Ramana Vinjamuri\**

*Sensorimotor Control Laboratory, Department of Biomedical Engineering, Chemistry, and Biological Sciences, Stevens Institute of Technology, Hoboken, NJ, USA*

Kinematic and neuromuscular synergies have been found in numerous aspects of human motion. This study aims to determine how effectively kinematic synergies in bilateral upper arm movements can be used to replicate complex activities of daily living (ADL) tasks using a sparse optimization algorithm. Ten right-handed subjects executed 18 rapid and 11 natural-paced ADL tasks requiring bimanual coordination while sitting at a table. A position tracking system was used to track the subjects' arms in space, and angular velocities over time for shoulder abduction, shoulder flexion, shoulder internal rotation, and elbow flexion for each arm were computed. Principal component analysis (PCA) was used to generate kinematic synergies from the rapid-paced task set for each subject. The first three synergies accounted for 80.3 *±* 3.8% of variance, while the first eight accounted for 94.8 *±* 0.85%. The first and second synergies appeared to encode symmetric reaching motions which were highly correlated across subjects. The first three synergies were correlated between left and right arms within subjects, whereas synergies four through eight were not, indicating asymmetries between left and right arms in only the higher order synergies. The synergies were then used to reconstruct each natural-paced task using the *l*1-norm minimization algorithm. Temporal dilations of the synergies were introduced in order to model the temporal scaling of movement patterns achieved by the cerebellum and basal ganglia as reported previously in the literature. Reconstruction error was reduced by introducing synergy dilations, and cumulative recruitment of several synergies was significantly reduced in the first 10% of training task time by introducing temporal dilations. The outcomes of this work could open new scenarios for the applications of postural synergies to the control of robotic systems, with potential applications in rehabilitation. These synergies not only help in providing near-natural control but also provide simplified strategies for design and control of artificial limbs. Potential applications of these bilateral synergies were discussed and future directions were proposed.

**Keywords: motor control, kinematic synergies, bilateral upper limb movements, activities of daily living, principal component analysis**

## **INTRODUCTION**

The human arm is a highly complex structure with an equally sophisticated control system. Each arm possesses 11 independent degrees of freedom (DoF) defined from the pectoral girdle to the wrist, which are actuated by approximately 32 muscles (Mackenzie and Iberall, 1994). The brain, therefore, has to coordinate over 60 different controls in order to operate both arms, yet accomplishes this

#### *Edited by:*

*Alessandro Tognetti, University of Pisa, Italy*

#### *Reviewed by:*

*Matteo Bianchi, University of Pisa, Italy Claudia Casellato, Politecnico di Milano, Italy*

*\*Correspondence: Ramana Vinjamuri ramana.vinjamuri@stevens.edu*

#### *Specialty section:*

*This article was submitted to Bionics and Biomimetics, a section of the journal Frontiers in Bioengineering and Biotechnology*

> *Received: 11 September 2016 Accepted: 12 January 2017 Published: 10 February 2017*

#### *Citation:*

*Burns MK, Patel V, Florescu I, Pochiraju KV and Vinjamuri R (2017) Low-Dimensional Synergistic Representation of Bilateral Reaching Movements. Front. Bioeng. Biotechnol. 5:2. doi: 10.3389/fbioe.2017.00002* task with apparent ease. How the brain handles real-time control of two redundant, high DoF manipulators during activities of daily living (ADL) is known as the degrees-of-freedom (DoF) problem (Bernstein, 1967; Latash et al., 2007) and is the subject of much research, including the present work. Progress in this field has applications in numerous areas including motor rehabilitation, assistive and prosthetic technology, and robotic control.

Evidence suggests that the brain may control the limbs by scaling, offsetting, and temporally dilating fundamental movements encoded in the sensorimotor system (Viviani and Terzuolo, 1980; Brooks, 1986). Previous research has shown that the brain executes tasks by using certain movement patterns while preserving their relative spatiotemporal proportions. Viviani and Terzuolo (1980) have shown in handwriting tasks that increased letter size still results in similar execution times by automatically increasing writing speed. Furthermore, slowing down a writing task results in temporal dilation of a common velocity pattern, preserving the relative occurrence of velocity profile features in time while reducing velocity amplitude (Brooks, 1986).

These patterns of motion have been developed into the concept of synergies, which can be defined as "a collection of relatively independent degrees of freedom that behave as a single functional unit" (Turvey, 2007). Synergies exist in either the joint angular velocity space, in the form of kinematic synergies, or neuromuscular activity space, in the form of neuromuscular synergies. Linear discriminant analysis, singular value decomposition (SVD), principal component analysis (PCA), non-negative matrix factorization (NMF), artificial neural networks, and many other algorithms have been used in the literature to derive synergies for hand grasps, gait patterns, and single-arm motion (Merckle et al., 1998; Santello et al., 1998; Vinjamuri et al., 2010; Roh et al., 2013; Alibeji et al., 2015). NMF is typically used to derive neuromuscular synergies (Tresch and Jarc, 2009), while PCA is frequently used to derive kinematic synergies as in Mason et al. (2001) and Vinjamuri et al. (2010). PCA-derived kinematic synergies have been demonstrated to perform favorably when directly compared to those from other linear and non-linear dimensionality reduction methods when applied to hand grasp reconstruction (Patel et al., 2015a).

Recent work has been aimed at integrating synergies into the control of robotic systems with the goal of producing a simplified control scheme for high DoF devices. The authors in Chen et al. (2015) have demonstrated an anthropomorphic robotic hand that has two mechanically implemented postural synergies which could successfully grasp various objects. Several groups have also proposed autonomous, control systems for high DoF robotic and virtual hands based on two postural synergies (Wimbock et al., 2011) and four postural synergies (Rosell et al., 2011; Segil and Weir, 2013), whereas Matrone et al. (2012) have demonstrated real-time myoelectric control of a robotic hand using two postural synergies with able-bodied subjects. An EMG-based control scheme was also introduced by Artemiadis and Kyriakopoulos (2010), which controls a 7-DoF robotic arm using kinematic and muscular synergies. The review recently published by Santello et al. (2016) gives a thorough description of the state of the art concerning dexterous hand control using synergies and highlights some future directions merging synergies with compliant design. Synergies derived using NMF have also been applied to optimal movement generation for virtual arms (Fu et al., 2013) as well as myocontrol of a multi-DoF planar robotic arm using muscle synergies (Lunardini et al., 2015). So far, work has been focused on using time-invariant postural synergies in the kinematic domain and restricted to unimanual processes.

Bilateral spatiotemporal kinematic synergies such as those presented here may be used as the controlled variable in future robotic systems that can be manipulated using EMG, EEG, or some other biosignal input. Whereas postural/spatial synergies attempt to linearize joint motion relative to each other, a timevarying approach allows more flexibility to capture the non-linear behaviors inherent to complex systems. An open question for such a system is whether or not ADL tasks are within the "workspace" of a system that is only manipulated using time-varying kinematic synergies. In other words, is it possible to manipulate bilateral spatiotemporal kinematic synergies by scaling their amplitudes and temporal offsets in such a way as to replicate ADL-like tasks.

In this study, we derive spatiotemporal kinematic synergies from rapidly paced ADL tasks for 10 able-bodied subjects. Tasks that require coordination of both arms and can be classified as symmetric in-phase, symmetric out-of-phase, asymmetric, or coupled are chosen. PCA is used to derive time-varying kinematic synergies from eight joint angular velocity profiles across both arms recorded during these rapid tasks. A separate set of tasks performed at a natural pace are reconstructed using the *l*1*-*norm minimization algorithm to select optimal amplitudes and temporal offsets and dilations of these synergies. The derived synergies are characterized in terms of intersubject and interlimb correlations, accuracy of reconstruction, and trends in their recruitment levels throughout the task duration.

### **SUBJECTS AND METHODS**

The present study was conducted under IRB Approved Protocol # 2014-026/2015-022 at the Stevens Institute of Technology. Ten subjects were recruited in the study after obtaining written informed consent. Subjects performed ADL-like tasks while their movements were recorded using an electromagnetic motion tracking system (Polhemus LIBERTY). Positional data from each sensor were converted into joint angles, synergies were derived using PCA in the joint angle velocity domain, and a separate set of tasks were reconstructed from the derived synergies using the *l*1-norm minimization algorithm.

### **Data Capture**

An electromagnetic tracking system (Polhemus LIBERTY, TX4 source) was used to record positional data of the subject during each task using their proprietary software (PiMgr). The study was executed in a minimal-metal environment with a compensation map calibration executed monthly to account for disturbances due to metal in the construction of the room. The workspace was calibrated such that the origin was on the edge of the table, centered in front of the subject. Positive Z extended upwards toward the ceiling, positive X extended forward away from the subject, and positive Y extended to the subject's left. Data were captured at 240 Hz and filtered using a 3 Hz fourth-order low-pass Butterworth filter.

### **Kinematic Model**

Several groups have developed refined anatomical models with the intent of capturing kinematic data from subjects as they perform tasks. Most of this work has addressed hurdles using optical systems such as 3D interpolation of one or multiple 2D viewpoints (Sidenbladh et al., 2000; Chen et al., 2010), and soft tissue deformation (Gabiccini et al., 2013). Under guidance from Wu et al. (2005), this work utilized an electromagnetic tracking system to capture the positions of several convenient landmarks on the torso, left, and right arms with a positional accuracy of approximately 2.5 mm in *x*, *y*, and *z*. The tracking system lacks line of sight issues and readily supplies Cartesian positions of these landmarks.

Seven sensors were placed on the body as shown in **Figure 1A**. Three sensors defined the trunk of the subject, while two additional sensors per arm tracked elbow and wrist movements in space. *S*<sup>1</sup> and *S*<sup>2</sup> were placed at the lateral head of the clavicle on the subject's right and left shoulder, respectively, while *S*<sup>3</sup> was placed on the subject's right side near the middle of the rib cage on the midaxillary line. *S*<sup>4</sup> and *S*<sup>6</sup> were placed on the lateral side of the subject's elbows over the joint's center of rotation. *S*<sup>5</sup> and *S*<sup>7</sup> were placed on the dorsal side of the subject's wrists and were centered between the distal ulnar and radial heads. The filtered *X*, *Y*, and *Z* trajectories captured by the tracking system were converted to joint angles as follows.

Three shoulder angles and one elbow angle were calculated for each side of the subject: shoulder abduction/adduction, flexion/extension, and internal/external and elbow flexion/extension. These angles are calculated using six vectors: *V*should, *V*side, *V*ae*,* and *V*aw, where *a* = *L, R* to indicate the left or right arm. *V*should is a vector from the subject's left shoulder to their right shoulder sensors, *V*side is a vector from the right shoulder sensor to the sensor on the right side of the torso, *V*ae is a vector from the shoulder to the elbow on each respective side, and*V*aw is the vector from the elbow to the wrist on each respective side. These vectors were calculated as:

$$S\_i = \begin{bmatrix} x\_i \\ y\_i \\ z\_i \end{bmatrix}$$

$$V\_{\text{should}} = S\_1 - S\_2$$

$$V\_{\text{side}} = S\_1 - S\_3$$

$$V\_{\text{Rc}} = S\_1 - S\_4, \ V\_{\text{Lc}} = S\_2 - S\_6$$

$$V\_{\text{Rw}} = S\_4 - S\_5, \ V\_{\text{Lw}} = S\_6 - S\_7$$

Abduction/adduction was found by first projecting the elbow vector, *V*ae, onto the coronal plane:

$$V\_{\text{ac,proj}} = V\_{\text{ac}} - \left(V\_{\text{ac}} \cdot n\_c\right) \frac{n\_c}{||n\_c||} \tag{1}$$

where *n<sup>c</sup>* is the vector normal to the coronal plane and is defined as the cross product between *V*should and *V*side. The shoulder abduction angle, θsa, between *V*ae,proj and *V*should was found using

$$\Theta\_{\rm sa} = \cos^{-1} \frac{V\_{\rm should} \cdot V\_{\rm ac,proj}}{||V\_{\rm should}|| ||V\_{\rm ac,proj}||} \tag{2}$$

The flexion/extension angle was found in the same manner as in Eqs 1 and 2 except by projecting the vectors onto the sagittal plane using *ns*, the normal vector to the sagittal plane, which was found by normalizing*V*should. The internal/external rotation of the upper

**FIGURE 1 | (A)** Sensor placement on body. *S*<sup>1</sup> and *S*<sup>2</sup> are positioned at the lateral head of the clavicles, S3 is placed on the right side on the body's midaxillary line, *S*<sup>4</sup> and *S*<sup>6</sup> are placed on the outer side of the elbow, and *S*<sup>5</sup> and *S*<sup>7</sup> are placed on the wrists between the distal heads of the radius and ulna. *P*<sup>1</sup> indicates the coronal plane, *P*<sup>2</sup> indicates the sagittal plane, *P*<sup>3</sup> indicates the plane normal to *V*ae, and *P*<sup>4</sup> indicates the plane containing *V*ae and *V*aw. *P*<sup>3</sup> and *P*<sup>4</sup> exist for both the left and right arms. **(B)** Tasks executed during study. Each panel is labeled with a number corresponding to the task in **Table 2** which is shown. Subjects start each task with their hands in the rest positions marked by the visible red boxes.

arm was found by projecting the wrist vector, *V*aw, and *V*should onto the plane normal to*V*ae and calculating the angle between the two projections as in Eq. 2. Elbow flexion/extension was calculated using *V*aw and *V*ae in Eq. 2. Sign changes were determined by comparing the vector cross product to the corresponding normal vector on the plane. Positive angles correspond to shoulder abduction, shoulder flexion, shoulder external rotation, and elbow flexion. These joint angle calculations were performed offline using a MATLAB function to get joint angle trajectories over time, and the resulting motion profiles were differentiated to get joint angular velocities.

#### Model Validation

**Table 1** shows the accuracy of the joint angle calculations reported here, compared to goniometer measurements. An iGaging 7*′′* goniometer (Anytime Inc., Los Angeles, CA, USA) with a precision of 0.1° and resolution of 0.05° was used to enforce the measured (ground truth) readings presented in **Table 1**. Each of the eight joints (two angles per joint, indicating minimum and maximum angles) were measured independently in each trial. Sensor positions were recorded and filtered using PiMgr (see Data Capture) in 2 s recordings. Three repetitions were captured for each trial. Joint angles were calculated from the time-series position data using our model to yield 2-s long joint angle postures, which were then averaged across the 2 s to get a joint angle for each repetition of each trial. Mean and SD for the calculated angles shown in **Table 1** are computed across repetitions. The difference between the model-estimated joint angles and the ground truth measured by goniometers for minimum and maximum angles is 8.8° and 8.6°, respectively. Error normalized to ground truth measurements reveals a mean of 36% error for the minimum measurement and 10% error for the maximum measurement, in large part because the minimum angle is smaller in magnitude than the maximum. Since reported results are in the velocity domain a large part of this error is negated in differentiation.

### **Subjects**

Subjects were recruited, with written and informed consent, based on the criteria specified in the approved IRB. Healthy subjects with no history of right or left upper limb injury or weakness and no cognitive or motor impairments were allowed in the study after signing a consent form and filling out a basic medical questionnaire. Ten subjects of age 18–25 years were recruited (mean 20); of which 4 were female and 10 were self-reported right-hand dominant.

### **Experiment Procedure**

Upon arrival subjects were fitted with hook-and-loop harnesses for the position sensors. Straps were adjusted so the sensors were held firmly in place without impeding the subject's motion. Basic range of motion exercises were performed to ensure that all straps and wires were settled and that the sensors remained in the proper locations. Subjects first executed 18 rapid training tasks, during which they were instructed to execute each task as quickly as they could successfully be completed. Three repetitions of each task were completed before performing the next task in the same fixed order for every subject. Eleven testing tasks were then performed in a fixed order at the subject's natural pace with three repetitions each. Each session lasted approximately 90 min with a short break offered between task phases.

**Figure 1B** and **Table 2** show each of the tasks performed in the study. These tasks were selected based on Barreca et al. (2005) and Foti and Koketsu (2013) as a cross-section of what is performed during ADL while requiring coordinated motion of both arms and being executable in the testing environment. These tasks were grouped into four categories to describe their type of motion: symmetric in-phase, symmetric out-of-phase, asymmetric, and coupled. Symmetric in-phase motions involve mirror symmetry between the two arms. This symmetry is typically about the midline but can be present in any direction. Symmetric out-of-phase motions have the same mirror symmetry as the in-phase category, except the motions of the left and right arms are offset in time. Asymmetric motions involve no symmetry between the two arms: each arm executes a different motion trajectory from the other such as washing a dish or using a fork and knife. Coupled motions involve both arms manipulating one object, such as when moving a box or tray, which results in a fixed relationship between the endpoints of each arm. **Table 3** shows the number of each type of task present in the training and testing phase. Subjects were given instruction on what to do in each task with special care taken not to coach how to execute the motions. Subjects began each task with their hands in a flat resting position marked with red rectangles. Subjects were instructed to stop at the end of the task without returning to the rest position.

**TABLE 1 | Computation of normalized error by the difference between measured (by goniometer) and estimated (by the model) joint angles (in degrees, mean***±***SD)**.


#### **TABLE 2 | Task list as executed in experiment**.


**TABLE 3 | Summary of task categories**.


Tasks 1 and 22, the knife and fork task, involved the subject picking up a knife and fork, positioning them on a plate as if to cut food, and executing a cutting motion with the knife. Task 1 involved one cutting motion and task 22 involved three. Tasks 2 and 23, picking object off plate, involved the subject picking up a plate holding a small wooden object with their non-dominant hand and using their dominant hand to pick the object off the plate and place it on a target marked on the table. Tasks 3 and 4, scrub dish, involved the subject picking up a plate in their nondominant hand and using a sponge with their dominant hand to wipe the plate in a circular clockwise (task 3) or counterclockwise (task 4) motion for one complete cycle. Tasks 5 and 6, scrub table, consist of the subject reaching for a sponge and wiping it in a shallow upright (task 5) or upside down (task 6) V pattern along the surface of the table.

The open box task, number 7, 8, and 24, involved the subject simultaneously opening the left and right flaps (task 7), the front and back flaps (task 8), or both sets of flaps sequentially (task 27) of a medium shipping box. Tasks 9 and 25, the fold clothes task, consist of identical motions between the training and testing set. The subject grasped a large piece of cloth along the outer edges and folded the left and right thirds over the center.

Task 10, drink from cup, consisted of the subject grasping a weighted cup placed in front of them with both hands and raising it up to their mouth as if to drink. Tasks 11–14, place cup on shelf, involved the subject picking up a weighted cup with both hands and placing it on one of four locations on a small set of shelves (top left, top right, bottom left, and bottom right sections of shelf for tasks 11–14, respectively). Tasks 15–18, pick cup off shelf, has

the same sequence as 11–14 except the subject moves the cup from the shelf to the table.

Tasks 19–21, manipulate tray, involved the subject picking up a tray while tilting it to the left (task 19), right (task 21), or not tilting (task 20). Tasks 26 and 27, imagined steer wheel, involved the subject pantomiming a steering motion. Subjects were instructed to pretend to turn a steering wheel in a handover-hand fashion in either the clockwise direction (task 26) or the counterclockwise direction (task 27) for three complete cycles. Tasks 28 and 29, imagined ladder climb, involved the subject pantomiming climbing up (task 28) and down (task 29) a ladder while seated.

The measured velocity trajectories were windowed using a threshold of 5% maximum repetition velocity to identify task start/end. Windowed training tasks were then averaged across repetitions for each subject to produce 18 windowed, filtered, averaged joint angular velocity profiles for synergy derivation. Testing task data were converted to joint angular velocity as above and were windowed to task onset without averaging across repetitions. Testing tasks were only windowed to task start to ensure that task duration always exceeded synergy duration.

### **Synergy Derivation**

**Table 4** documents all symbols used in this section for reference. The measured velocity profiles for the training tasks were formatted into a matrix for each subject during data processing with *J* rows, *T*max columns, and *N* pages. For the training tasks, the study used *J* = 8 joints, *T*max = number of samples in the longest windowed training task, and *n* training tasks with *N* = 18 total.

$$V\_n = \begin{bmatrix} \nu\_1^n(1) & \dots & \nu\_1^n(T\_{\text{max}}) \\ \vdots & \ddots & \vdots \\ \nu\_f^n(1) & \dots & \nu\_f^n(T\_{\text{max}}) \end{bmatrix} = \begin{bmatrix} \nu\_1^n(1) & \dots & \nu\_1^n(T\_{\text{max}}) \\ \vdots & \ddots & \vdots \\ \nu\_8^n(1) & \dots & \nu\_8^n(T\_{\text{max}}) \end{bmatrix}$$

Tasks were padded with zeros on the end to reach *T*max length. *T*max varies for each subject and ranged from 400 to 608 samples. In order to extract spatiotemporal synergies using PCA, the 8 *× T*max dimensional velocity matrix *V<sup>n</sup>* for a given task is manipulated into a row vector, *Vn*, with *J·T*max columns where each instant of time is represented as an additional set of *J* joints. Each task's row vector is concatenated into an *N × J·T*max task matrix, *V*:

$$V = \begin{bmatrix} \overline{V}\_1 \\ \overline{V}\_2 \\ \vdots \\ \overline{V}\_n \end{bmatrix}$$

$$= \begin{bmatrix} \begin{bmatrix} \nu\_1^1(1) & \cdots & \nu\_8^1(1) & \cdots & \nu\_1^1(T\_{\text{max}}) & \cdots & \nu\_8^1(T\_{\text{max}}) \end{bmatrix} & \begin{array}{cccc} \cdots & \nu\_8^1(T\_{\text{max}}) \end{array} \end{bmatrix}$$

$$= \begin{bmatrix} \begin{bmatrix} \nu\_1^2(1) & \cdots & \nu\_8^2(1) & \cdots & \nu\_1^2(T\_{\text{max}}) & \cdots & \nu\_8^2(T\_{\text{max}}) \end{bmatrix} \\\\ \begin{bmatrix} \nu\_1^N(1) & \cdots & \nu\_8^N(1) & \cdots & \nu\_1^N(T\_{\text{max}}) & \cdots & \nu\_8^N(T\_{\text{max}}) \end{bmatrix} \end{bmatrix}$$

Principal component analysis operates on this matrix as though it were a *J·T*max-joint system observed at *N* instants of time, effectively treating the time-series velocity profiles as a single

**TABLE 4 | Definitions of variables used in Section "Materials and Methods."**


instant of a *J·T*max-joint system. The task matrix is resolved into three-component matrices using SVD:

$$V = U\sum s\_i$$

where *U* is an *N × N* square matrix with orthogonal columns, Σ is an *N × J·T*max diagonal matrix, and *S* is a *J·T*max *× J·T*max square matrix with orthogonal rows. The diagonal elements of Σ correspond to the singular values, λ, of *V.*

$$\sum = \text{diag}\{\lambda\_1, \lambda\_2, \dots, \lambda\_N\}$$

In this form, the first *m* rows of *S* correspond to the first *m* principal components, or synergies, where *m < N*:

$$S = \begin{bmatrix} s\_1^1(1) & \cdots & s\_l^1(1) & \cdots & s\_1^1(T\_{\max}) & \cdots & s\_l^1(T\_{\max}) \\ s\_1^2(1) & \cdots & s\_l^2(1) & \cdots & s\_1^2(T\_{\max}) & \cdots & s\_l^2(T\_{\max}) \\ & & & \vdots \\ s\_1^M(1) & \cdots & s\_l^M(1) & \cdots & s\_1^M(T\_{\max}) & \cdots & s\_l^M(T\_{\max}) \\ & & & \vdots \\ s\_1^N(1) & \cdots & s\_l^N(1) & \cdots & s\_1^N(T\_{\max}) & \cdots & s\_l^N(T\_{\max}) \end{bmatrix}.$$

*U,* Σ, and *S* are computed from *V* using the SVD function in MATLAB. The original *V* matrix can be approximated as *V*˜ by isolating the first *M* columns of *U*, *M × M* elements of Σ, and *M* rows of *S*. The matrix *UM*diag{λ*1,* λ*2,. . .* λ*M*} is now denoted as the weight matrix for the *n*th task and first *M* synergies for *SM*:

$$V \approx \tilde{V} = \ U\_{\mathcal{M}} \text{diag}\left\{ \lambda\_1, \lambda\_2, \dots, \lambda\_M \right\} \\ \mathbb{S}\_{\mathcal{M}} = \mathcal{W}\_{\mathcal{M}}^{\pi} \mathbb{S}\_{\mathcal{M}} \qquad \text{(3)}$$

It follows from Eq. 3 that the time-series components of *V*˜ over time can be expressed as a weighted sum of *M* principal components:

$$\vec{\nu}\_{j}^{n}(t) = \sum\_{m=1}^{M} \, \boldsymbol{\nu}\_{m}^{n} \boldsymbol{s}\_{j}^{m}(t) \tag{4}$$

Since the singular values found in Σ are related to the spread of data along each principle axis, i.e., variance in that direction, an index known as fraction of sum-squared variance can be calculated from the diagonal elements of Σ*.* This index describes the fraction of total variance accounted for by the first *m* synergies and is useful as an indicator of how many principal components are needed to adequately represent the data.

$$\frac{\lambda\_1^2 + \lambda\_2^2 + \dots + \lambda\_M^2}{\lambda\_1^2 + \lambda\_2^2 + \dots + \lambda\_N^2} \tag{5}$$

An index threshold of 95% variance is used to determine how many synergies, *M*, are required to represent the training task data.

Principal component analysis assumes stationary input variables, but the input data contain time-varying joint angular velocities. Here, "time-varying" refers to a motion consisting of a sequence of postures that change with time. However, the statistical properties of the synergies are quite stationary. PCA also assumes independent and identically distributed variables, but the input data that contain bilateral arm postures are not strictly independent (as there are biomechanical constraints that lead to joint correlations) as is the case with many real-world variables. PCA is a non-parametric method, i.e., it does not require any prior knowledge. Although this makes the application of this method simple, the method itself assumes linearity, which could be a weakness in many applications. We have compared the performance of PCA with other non-linear methods in Patel et al. (2015a) and found that PCA outperformed other methods. Using this exploratory analysis has previously led us to anatomically informing and meaningful synergies as principal components. These synergies could represent 100 postural movements with as low as six synergies with accuracy greater than 90% (Vinjamuri et al., 2010). These synergies also showed the effect on visual and tactile feedback in reaching and grasping movements (Patel et al., 2015b).

### **Reconstruction**

Before performing reconstruction, the synergies and testing data were downsampled from 240 to 60 Hz due to the relatively long duration of synergies and testing tasks leading to excessive computation time. The testing task matrix, *R*, was reformatted from a *J × T<sup>d</sup> × L* matrix for *J* joints, *T<sup>d</sup>* samples in the downsampled task, and *L* = 11 tasks into a *J·T<sup>d</sup>* row, *L* column 2D matrix with each column defined as a separate task:

$$\mathcal{R} = \begin{bmatrix} \begin{bmatrix} r\_1^1(1) \\ \vdots \\ r\_{\hat{f}}^1(1) \\ \vdots \\ r\_1^1(T\_d) \\ \vdots \\ r\_1^1(T\_d) \end{bmatrix}, & \begin{bmatrix} r\_1^2(1) \\ \vdots \\ r\_{\hat{f}}^2(1) \\ \vdots \\ r\_1^2(T\_d) \\ \vdots \\ r\_{\hat{f}}^2(T\_d) \end{bmatrix}, & \cdots & \begin{bmatrix} r\_1^L(1) \\ \vdots \\ r\_{\hat{f}}^L(1) \\ \vdots \\ r\_1^L(T\_d) \\ \vdots \\ r\_{\hat{f}}^L(T\_d) \end{bmatrix} \end{bmatrix}$$

Each task shorter than the longest task was padded with zeros to equal the same length. The objective of reconstruction is closely analogous to finding a representation of *R* in a new basis *B*. This is accomplished as an optimization problem to find the elements of column vector *C* which most closely satisfies:

$$R = BC \tag{6}$$

In this case, *B* will be made up of temporal offsets and dilations of the synergies defined by *Sm*. *B* is formed by first transposing the downsampled version of *Sm*. For notation, let *S(m)* be a J*·T<sup>s</sup>* element column vector containing synergy *m* and let [0] be a null Burns et al. Synergistic Representation of Bilateral Movements

column vector of *J* elements long. The first *T<sup>d</sup> − T*max columns of *B* are:

$$B\_D^m = \begin{bmatrix} S(m) & [0] & & [\mathbf{0}] \\ [\mathbf{0}] & S(m) & & [\mathbf{0}] \\ [\mathbf{0}] & [\mathbf{0}] & \dots & & [\mathbf{0}] \\ \vdots & \vdots & & \vdots \\ [\mathbf{0}] & [\mathbf{0}] & & S(m) \end{bmatrix}$$

where *m* = *1* is the synergy number, *D* = 0 identifies an undilated synergy (1, 2, or 3 indicating the first, second, or third synergy dilation), and *T*max is the length of the undilated synergy.

Next, the synergy is dilated by interpolating *S(m)* to be some proportion of the difference between testing task length and synergy length. In this study, we dilate each synergy to be longer than the original synergy by 25, 50, and 75% of the difference in task and synergy sample length. Each dilation, *SD(m)*, is used to form the next *T<sup>d</sup> − Tm,D* columns of *B*, where *Tm,D* is the number of samples in the *D*th dilations of the *m*th synergy. This is done by forming each column as a sequential temporal offset of *SD(m)* as done above. This process is repeated for each dilation of each synergy, resulting in a *B* matrix, which contains every temporal offset of every dilation examined of the first *M* synergies:

$$B = \begin{bmatrix} B\_0^1 & \dots \dots & B\_3^1 & \dots \dots & B\_0^M & \dots \dots & B\_3^M \end{bmatrix}^T$$

*l*1-norm minimization was used to sparsely select values for *C* which satisfy the following optimization problem:

$$\text{Minimize } \|\mathbf{C}\|\_1 + \frac{1}{\mathsf{N}} \|\mathbf{B}\mathbf{C} - \mathsf{R}\|\_2^2$$

where ||*·*||<sup>1</sup> and ||.||<sup>2</sup> represent the *l*1- and *l*2-norms, respectively, and λ is a regulation parameter. Since the columns of *B* represent each synergy in different temporal offsets and dilations, the elements of *C* serve as recruitment weights for a synergy at a particular instant of time. The reconstructed task profiles, *R*˜, can be generated by multiplying *B·C*:

$$\mathcal{R} \approx \tilde{\mathcal{R}} = BC$$

The error between the measured and reconstructed profiles across all joints *J* = *8* is computed for each task, *l* using

$$\mathbf{e}\_{l} = \frac{\sum\_{j=1}^{l} \sum\_{t=0}^{T\_{l}} \left(r\_{j\_{j}}^{l}(t) - r\_{j}^{l}(t)\right)^{2}}{\sum\_{j=1}^{l} \sum\_{t=0}^{T\_{l}} r\_{j}^{l}(t)^{2}} \tag{7}$$

### **RESULTS**

### **Extraction of Synergies Using PCA**

Spatiotemporal kinematic synergies were computed for each subject using PCA. **Figure 2** shows the squared variance for each synergy along with the fraction of the sum of squared variance averaged across subjects. Derived synergies were 2.14 *±* 0.29 s long, ranging from 1.67 to 2.53 s. The first synergy accounts for 57.98 *±* 6.4% of total variance, while the first six synergies account for 91.05 *±* 1.7% and the first eight account for 94.82 *±* 0.85%.

**and total from 1 to** *n* **synergies (line)**. The first synergy accounts for 57.98 *±* 6.35% of variance, the first six synergies account for 91.05 *±* 1.69%, and the first eight synergies account for 94.82 *±* 0.85%. Dotted line shows 0.95 threshold.

The first three of each subject's synergies were compared to each other using Pearson's correlation coefficient. Synergies were put into the column form discussed in the Section "Reconstruction" and compared. **Figure 3A** shows the Pearson's *r* 2 averaged across subject comparisons for each combination of the first three synergies, leading to 90 unique comparisons for each synergy pair. Statistically significant differences were found in each of the three groups by one-way ANOVA tables, α = 0.05. All comparisons between synergy 1, 2, and 3 were statistically significant (*p≪*0.005 for all). Tukey *post hoc* tests were computed to determine specific differences. As expected, the correlation between synergy 1 and synergy 1 was greater than the correlation between synergies 1 and 2 and between synergies 1 and 3. Correlations between synergies 1 and 2 and between synergies 1 and 3 were not found to be different from each other. All *r* 2 values are statistically different for the synergy 2 and 3 comparisons. A closer examination of the synergy correlations was conducted using Pearson's correlation coefficient,*r*. **Figure 3B** is a color-scale grid displaying the comparison between synergy one, two, and three between each pair of subjects. Synergy 1 appears positively correlated across all subjects except subject 6, who appears to have a strong negative correlation. Synergy 2 appears more mixed: there exist some positive/negative correlations that contribute to the overall mean *r* 2 of 0.3593 *±* 0.2652, although subjects 1 and 10 have statistically insignificant mean *r* values at α = 0.05 of 0.0504 *±* 0.265 (*p* = 0.5841) and 0.0594 *±* 0.4582 (*p* = 0.7074), respectively.

**Figure 4** shows the synergy velocity profiles for the first eight synergies of subject 6. Each row corresponds to a joint labeled with a letter indicating the side ("R" for right or "L" for left), the joint ("S" for shoulder or "E" for elbow), and the rotation ("A" for abduction, "F" for flexion, "I" for internal rotation). Since the synergy is unscaled, the *y* axis is a unit-less velocity amplitude, while the *x* axis is time in seconds. For subject 6, the first synergy

**FIGURE 3 | Correlation analysis of the first three synergies**. **(A)** Pearson coefficient of determination, *r* 2 , averaged across the 45 unique combinations between the specified synergies of each of 10 subjects. Statistical differences found using one-way ANOVA tables with α = 0.05 with Tukey *post hoc* tests. Comparison of synergy 1 to synergy 1 and synergy 1 to synergies 2 and 3, among all pairings of synergy 2, and among all pairings of synergy 3 yielded significant differences. **(B)** Correlation coefficient, *r*, between each subject for synergies 1, 2, and 3. Only unique pairings of subjects are shown using the upper triangle matrices. Synergy 1 appears highly positively correlated except for subject 6, who is highly negatively correlated.

**FIGURE 4 | Angular velocity profile for first eight synergies of subject 6**. Vertical axis is unitless velocity since synergy is unscaled. Rows are labeled with letters indicating the side ("R" for right or "L" for left), the joint ("S" for shoulder or "E" for elbow), and the rotation ("A" for abduction in positive direction, "F" for flexion in positive direction, "I" for internal rotation in positive direction) for each DoF. Synergy 1 and 2 involve flexion and internal rotation of shoulder along with extension of elbow, implying a reaching motion, whereas synergy 3 involves extensions at the shoulder and flexion at the elbow.

consists of bilateral shoulder flexion, abduction, and internal rotation, which suggests a forward reaching motion. The elbows both show a small initial flexion, perhaps as the subject raises their arms from the rest posture, followed by an extension motion as they complete the reach. Synergy 2 behaves similarly, except shoulder abduction is delayed relative to synergy 1 and shoulder flexion and internal rotation are executed for a longer period of time. As discussed above, synergy 3 is relatively uncorrelated among subjects. For subject 6, synergy 3 involves slight shoulder flexion followed by extension and adduction with an external rotation. The elbows appear to flex slightly, extend, then flex again.

In order to better visualize the other synergies for subject 6, each profile was integrated, multiplied by a gain, and added to the average starting joint angles of a particular subject. The gain was chosen such that the resulting angular profile remains within natural range of motion throughout the whole path. **Figure 5** shows a

subject's position averaged across the first 50 samples of all tasks. Synergies were integrated up to each time point and multiplied by a gain such that normal joint range of motion is not violated. Mirror symmetry between left/right arms can be seen in the first three synergies whereas synergies 4–8 have asymmetric motions. Subject's non-dominant hand tends to go to a single position and hold steady while the dominant hand appears to move continuously.

virtual mannequin posed at the resulting postures for six normalized time points. Subject 6, with a negatively correlated synergy 1, has a distinct reach-and-grasp motion. The downward dip of both hands observed at *t* = 0.4, or nearly halfway through the motion, could roughly correspond to the end of the reach phase and the beginning of object grasp and manipulation involving picking up an object. Note that subject 6's first synergy is strongly negatively correlated with the rest of the subjects, so synergy 1 for most of the subjects involves outward extension of the arms, similar to synergy 3 in **Figure 4**. Synergy 2 was similar, except it involved a reach-and-grasp motion instead of reach and manipulate/pick up. The shoulder motions observed in the synergy 3 profiles clearly result in an overall bilateral extension movement, with the arms spanning outward behind the back. Synergy 4 appears similar to picking up and holding a box at chest level.

The higher order synergies shown in **Figure 5** appear to show a level of handedness, with the left hand tending to go to a certain position and holding while the right hand moved in various profiles through the duration of each synergy. **Figure 6** shows the Pearson coefficient of determination, *r* 2 , averaged across subjects comparing joints 1–4 (right arm) to joints 5–8 (left arm) of all eight synergies. A one-way ANOVA (α = 0.05) was performed with Tukey *post hoc* tests to establish significant differences (ANOVA *p≪*0.005). Synergies 1 and 2 were each

**FIGURE 6 | Pearson's coefficient of determination comparing right arm joints to left arm joints of the each synergy**. One-way ANOVA and Tukey *post hoc* show that synergy 1 and synergy 2 each had significantly higher coefficients of determination than synergies 4–8, while the coefficient for synergy 3 was significantly higher than synergy 4 and 8. Statistical difference found in first three synergies implies asymmetric motion between left and right arms in higher order synergies.

significantly more correlated between left/right than Synergies 4–8, and Synergy 3 was more correlated between left/right than Synergies 4 and 8.

### **Task Reconstruction**

Reconstruction was carried out using up to eight synergies given the PCA results above and the 8-DoF nature of the bilateral arm model presented. Tasks were reconstructed with and without dilations. **Figure 7** shows the mean reconstruction error calculated using Eq. 7 and averaged across degrees of freedom, tasks, and subjects using no dilations (blue) and dilations at 25, 50, and 75% of task/synergy length difference. Synergies 1–4 show a nearly linear decrease in normalized error in both cases; each subsequent synergy begins showing less of an improvement. Recruiting the first six synergies yielded a normalized reconstruction error of 0.1757 *±* 0.0347 and 0.104 *±* 0.0161 for no dilations and three dilations, respectively. Ultimately the error reduces down to 0.062 *±* 0.0098 by synergy 8.

**FIGURE 7 | Normalized reconstruction error when recruiting from synergies 1–8 with and without dilations**. The reconstruction error for each synergy was averaged over degrees of freedom, subjects, and tasks. Dilated synergies were longer than undilated synergies by 25, 50, and 75% of the difference between minimum reconstruction task length and the synergy length. SDs are across subjects and tasks.

**Figure 8** shows examples of the reconstruction progression for left shoulder abduction using dilations for several tasks, demonstrating a clear progression from two available synergies (blue dotted line) to 8 synergies (red line). **Figure 8A** shows that the best-performing task that was repetition 2 of task 27, pretend steering wheel counterclockwise for subject 8. Reconstruction error went from 74.2% using only the first synergy to 2.22% using the first eight. **Figure 8B** shows repetition 2 of task 24, open box, for subject 5. Reconstruction error for this task was 27.9% using the first synergy and 2.41% using the first eight. **Figure 8C** shows repetition 1 of task 22, knife and fork, for subject 10. Reconstruction error was 43.8% using the first synergy and 7.32% using the first eight.

The optimization algorithm appeared to handle cyclic profiles relatively well, whereas the initial and ending phases of the tasks would often be less accurate. **Figures 8A,C** show this quality within the first 1 s of the tasks. Reconstruction was able to capture the overall shape of the task shown in **Figure 8B**, but some of the finer, irregular motions were not captured.

**Tables 5** and **6** show the integrated recruitment gain for subject 4 averaged across tasks during normalized time bins for each synergy without and with dilations, respectively. Synergies and their dilations were not allowed to be recruited beyond the time at which the last synergy and task sample would coincide. Time bins which would include "missing" recruitment gains were therefore omitted. Significant differences found using one-way ANOVA with α = 0.00089 (0.05 over 56 comparisons) between synergy recruitments with and without dilations are indicated in **Table 5** as bolded and underlined with an asterisk. For subject 4, synergies 3 and 5 in time bin 2 were significantly more recruited in the no dilations reconstruction than in the dilations reconstruction (*p ≈* 0 for both). No other synergies had significantly different recruitments.

**Table 6** shows significant differences in recruitment between dilations of each synergy in each time bin, found using one-way ANOVA tables with α = 0.00125 (0.05 over 40 comparisons) and Tukey *post hoc* tests. Differences are bolded and marked with an asterisk. The undilated synergy 1 was significantly more recruited than all dilations in time bin 1 (*p ≈* 0). The undilated synergy 8 was significantly more recruited than dilation 2 and 3 in time bin 1 (*p≪*0.00125). The first dilation of synergy 7 was significantly more recruited than the un-dilated synergy in time bin 3 (*p ≈* 0). The undilated synergy 8 was significantly more recruited than the

**FIGURE 8 | Three examples of reconstruction progression for left shoulder abduction**. **(A)** Repetition 2 of task 27 for subject 8, the best-performing reconstruction with error across all joints of 2.22%. **(B)** Repetition 2 of task 24 for subject 5, error of 2.4%. **(C)** Repetition 1 of task 22 for subject 10, error of 7.3%. The early reaching phases of tasks were typically not as accurately reconstructed as the later manipulation phases.

#### **TABLE 5 | Integrated recruitment weight without dilations for normalized time bins averaged across tasks for subject 4 (mean***±***SD)**.


*Statistical differences are marked in bold with an asterisk.*

#### **TABLE 6 | Integrated recruitment weight including dilations (***D***0,** *D***1,** *D***2,** *D***3) for normalized time bins averaged across tasks for subject 4 (mean***±***SD)**.


*Statistical differences are marked in bold with an asterisk.*

second dilation in time bin 3 (*p≪*0.00125), although the absolute value of their recruitments likely would not be different.

A similar analysis was performed after pooling all subjects. Significant differences in recruitment of an undilated synergy including and excluding dilations were found across tasks and subjects using ANOVA with α = 0.0013 (α = 0.05 for 40 comparisons). Synergy 1 was significantly less recruited in time bins 1 and 2 when dilations were included in reconstruction (*p ≈* 0). Synergy 8 was significantly more recruited in time bin 1 with dilations compared to without dilations (*p≪*0.0013). Differences in recruitment among dilations of each synergy in each time bin were found across tasks and subjects using ANOVA tables with α = 0.00156 (α = 0.05 for 32 comparisons). The undilated synergy 1 in time bin 1 is significantly more recruited than the dilated versions (*p ≈* 0). The undilated and first dilation of synergy 5 are more recruited than dilation 3, but this difference was not statistically significant (*p* = 0.0034). Dilation 4 of synergy 7 was recruited significantly more than the undilated, first, and second dilation in time bin 1 (*p≪*0.001). The undilated synergy 8 was statistically different from the first and second dilation in time bin 1 (*p* = 0.0015).

### **DISCUSSION**

Previous work in implementing postural synergies in robotic control systems (see Introduction) aims to create a simplified control scheme, which can control high-dimensional systems with a reduced number of control inputs or actuators. The present study expands on this work by evaluating time-varying spatiotemporal synergies defined for both shoulder and elbow joints. Furthermore, we believe a key question for robotic systems (such as prosthetics and other assistive devices) meant to perform ADL, is how capable such systems are of reproducing ADL-like motion. Most experiments evaluate performance via endpoint variables such as task success rate, task completion time, or endpoint accuracy. This work attempts to evaluate such a system based on its ability to replicate ADL-like kinematics measured from reach-and-grasp experiments. This representation gives a more complete representation of the state of the system through the entire motion being performed. We present the optimal kinematic performance of time-varying synergies by sparsely optimizing the recruitment of synergies in time and amplitude and also introduce temporal dilations as discussed later.

Our results indicate that although the first three synergies can account for up to 80% of variance in training data, replicating the actual kinematics of ADL tasks could require the use of higher order synergies. Using two, three, or even four synergies as in previous literature could still be placing an absolute optimal kinematic accuracy limit of over 30% error in angular velocity, as shown in **Figure 7**. In the context of robotic control and assistive robotic interfaces, our results are reminiscent of the tradeoff between the optimal performance and complexity of control. An autonomous system may be able to incorporate higher order synergies in order to finely coordinating two arms, whereas an assistive system meant to be used by a person would quickly become too complex to use. These results may be improved on by experimenting with training task composition, dividing joints into different subgroups, or several other approaches which were not examined here.

Beyond simplifying control schemes, synergies have also been proposed as an avenue to make robotic systems exhibit more humanoid behaviors with biomimetic control. Robots designed to interact and cooperate with humans are deemed more esthetically pleasing when exhibiting anthropomorphic motion and are believed to be safer since human-like motions are more readily predictable than robotic ones (Duffy, 2003; Hegel et al., 2008). Control schemes aimed at anthropomorphizing robotic motions through the use of synergies derived from reach-and-grasp tasks have been carried out by Liarokapis et al. (2015) and reviewed by Santello et al. (2016). Whereas this work leveraged postural synergies, our results indicate that time-varying synergies could be used to produce more accurate replications of human motion. It is possible that the reconstruction method applied here could be performed on non-anthropomorphic trajectories to generate a near-natural approximation of a synthetictrajectory.

The recent trend of mechanically embedding synergies into robotic systems may also be extended to time-varying principal components. The SoftHand, for example, uses a compliant structure actuated by a single principal component to achieve various grasps (Catalano et al., 2014). Implementing the complex motions described in **Figure 4** may be achieved by software-generated temporal postural synergies coupled with a mechanical design which together introduce non-linearities. These time-varying synergies in bilateral arm movements can be easily integrated in upper limb prosthetics and rehabilitation devices to render natural movements.

Our results could also have implications outside of strict robotic control. The symmetric nature of the first two synergies between the left and right arms led to an examination of the similarities between the left and right joint angular velocities of synergies 1–8 for each subject. These results indicate that lower order synergies tended to have similar profiles between the left and right arms, whereas higher order synergies were dissimilar. The lower order synergies seem to provide broad reaching motions, while higher order synergies appear to fine-tune the motion to suit particular manipulations required by a task.

The asymmetries in higher order synergies could contain information related to the handedness of the subject. Observationally, the postural visualization of the higher order synergies in **Figure 3** appears to show the left arm maintaining stable positions and motions, while the right arm follows a more complex path. This result is in line with previous work done on bimanual upper limb control, and a more detailed kinematic analysis of bilateral kinematic synergies may also be conducted to explore this link. Sainburg and Kalakanis (2000) and Bagesteiro and Sainburg (2002) found significant differences in hand path curvature and torque efficiency between dominant and non-dominant hands; the same group found advantages in the dominant arm for speed and directional control, whereas the non-dominant arm was specialized for accurate position control through increased limb impedance (Wang and Sainburg, 2007). Recently, Yokoi et al. (2014) conducted bimanual lever tasks in which the non-dominant arm operated in an artificial force field. They were able to demonstrate that the non-dominant limb was more adaptable to the force field when trained with a specific motion direction of the dominant arm, but that altering the dominant-side target location led to a decrease in performance of the non-dominant arm. In this paper, a model using movement primitives to approximate motor learning of the non-dominant arm relative to dominant arm dynamics was able to replicate this effect, supporting the idea that bilateral synergies may be used by the central nervous system to produce motion.

The synergies are hypothesized to be abstract representations of motion encoded in the sensorimotor system of the brain, which can be combined to form the complex motions required to execute ADL tasks. The actual generation and execution of complex upper limb movements is a neural process that still has much to be discovered. The original drive or goal which the sensorimotor system acts on has long been believed to be the limbic system (Brooks, 1986). The abstract goals produced by the limbic system are processed by the association cortex that then produces a plan for motion. This information is passed to the projection system, composed of the sensorimotor cortex, cerebellum, and basal ganglia, which converts the "trajectory" to a series of commands which can be passed to the spinal cord. This process is dictated by sensory feedback, developmental history, and a sense of the dynamics of the body (Brooks, 1983). The internal dynamic model has been hypothesized to be composed of both feedforward and feedback components, aimed at minimizing error while still allowing for quick responses. The existence of either of these models or the combination thereof is still much debated, with evidence pointed either way (Wolpert et al., 1998). As mentioned earlier, the studies conducted by Viviani and Terzuolo (1980) imply the projection system's ability to dilate motion building blocks in time. Single impulses at specific interneuronal sites result in coactivation of multiple muscles, as shown by Tresch et al. (1999) and Saltiel et al. (2001), and each muscle generates torque around multiple degrees of freedom in the hand and arm (Santello et al., 2013). This torque is influenced by numerous anatomical parameters which are unique to each individual, as is the resulting motion of the limb (Buchanan et al., 2004). Visual, proprioceptive, haptic, and numerous other sensory signals are then returned to the brain and integrated into motion planning.

Task reconstruction as performed in this study attempts to model the neural process described above computationally. The measured training task can be viewed as the motion plan output by the associative cortex. The synergies derived from the rapid tasks serve as the "history" programmed into the projective system which dictates how the brain converts planned motion to spinal cord commands. This would be analogous to a feedforward component of cerebellar processing discussed by Wolpert et al. (1998): given a certain output from the brain, there is expected to be some dynamic response observed at the arms that can take on an arbitrary pattern across joints. Dilations of the synergies are also provided which attempts to mimic the time-scaling ability of the basal ganglia and cerebellum. *l*1-norm minimization serves the role of the projection system by using synergies and their dilations to convert from a desired motion to a time-series of gains, which can be viewed as the time-series of signals being sent to the spinal cord. The neuromuscular and physiological construction of the limbs is approximated by multiplying the synergies by this time-series of gains, producing the final motion.

The time-series of gains generated by *l*1-norm minimization could give some insights into how useful temporal dilations of synergies could be in motion production. **Table 5** shows that recruitment of synergy 1 is reduced when including dilations in reconstruction for the first two time bins. Synergy 8 had a larger recruitment in the first time bin with dilations compared to without. **Table 6** shows that the only statistical difference between recruitment of dilations is in time bin 1 for synergies 1, 7, and 8. The undilated synergy 1 was significantly more recruited than the dilations, dilation 3 was the most recruited for synergy 7, and the undilated synergy was the most recruited for synergy 8. The most significant effect of including dilations in the recruitment process appears to be the reduction in recruitment of synergy 1 during task onset. Recruitment of the dilations of synergy 1 also appears to be relatively large compared to recruitment of the other synergy dilations.

The present experiment could be refined to further study dominant/non-dominant performance in bimanual ADL. The subject pool can be expanded to include left- and right-handed individuals, and tasks can be modified to selectively engage the left, right, or bilateral arms. The relative influence of dominant/non-dominant hemispheres on ipsilateral limb motion could be examined with a more focused data analysis by examining whether left or right side joints are better reconstructed for left- and right-handed subjects. Devising a way to "coax" reconstruction to replicate the structural asymmetry and increased influence of the dominant over the non-dominant hemisphere as opposed to *vice versa* (Snyder et al., 1995; Amunts et al., 1996; Ziemann and Hallett, 2001; Hayashi et al., 2008) could allow us to study the strength of this influence. The upper limb kinematic data recorded in this study may also be augmented with EMG and/or EEG signals.

### **CONCLUSION**

This paper presents a study of spatiotemporal kinematic synergies in bilateral arm movements during ADL tasks. These tasks were selected as a representation of the fundamental categories of bilateral motion (symmetric, asymmetric, coupled), with further work aimed at tailoring the task list to target specific aspects of sensorimotor processing. Derived synergies accounted for up to 94.82 *±* 0.85% of variance and were demonstrated to reconstruct ADL tasks to within 6.2 *±* 0.98% using the *l*1-norm minimization algorithm. The concept of temporal synergy dilations were incorporated to replicate movement processing found in the basal ganglia and cerebellum. Recruitment patterns were examined throughout task duration and found that the dilated version of a synergy was used equally as much as the undilated version of the same synergy in most time bins, with the beginning of the motion having the most difference. Potential uses of time-varying synergies to anthropomorphize robotic motion and inform mechanical construction were discussed. Interesting features of the synergies in the context of handedness corroborating others' results were discussed. We believe that synergies will be instrumental in building next-generation biomimetic prosthetics and orthotics in the near future.

### **ETHICS STATEMENT**

The present study was conducted under IRB Approved Protocol # 2014-026/2015-022 at the Stevens Institute of Technology. All the subjects gave written informed consent in accordance with the Declaration of Helsinki.

### **AUTHOR CONTRIBUTIONS**

MB designed the experiment, ran instrument calibrations, designed the kinematic model and data processing scripts (with inputs from IF and KP), executed the experiment, and wrote the manuscript. VP contributed to the data processing scripts, assisted with executing the experiment, and provided valuable feedback in writing the manuscript. RV set up the hypothesis and direction/scope of the study, provided invaluable feedback in experiment design, task selection, and data processing, coordinated subject recruitment, and provided key revisions and editing on the manuscript.

### **ACKNOWLEDGMENTS**

The authors would like to thank the Department of Biomedical Engineering, Chemistry, and Biological Sciences of the Stevens Institute of Technology for their continued support of ongoing

### **REFERENCES**


research. The authors would like to thank Christian Jensen for his assistance in performing the experiment. The authors would also like to thank the reviewers and associate editors for their valuable comments and suggestions that majorly improved the quality of their manuscript.


during bimanual actions. *J. Neurosci.* 34, 9141–9151. doi:10.1523/JNEUROSCI. 2694-13.2014

Ziemann, U., and Hallett, M. (2001). Hemispheric asymmetry of ipsilateral motor cortex activation during unimanual motor tasks: further evidence for motor dominance. *Neurophysiol. Clin.* 112, 107–113. doi:10.1016/S1388-2457(00) 00502-2

**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

The reviewer MB and handling editor declared their shared affiliation, and the handling editor states that the process nevertheless met the standards of a fair and objective review.

*Copyright © 2017 Burns, Patel, Florescu, Pochiraju and Vinjamuri. This is an openaccess article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Muscle synergies Facilitate Computational Prediction of subject-specific Walking Motions

*Andrew J. Meyer1 , Ilan Eskinazi1 , Jennifer N. Jackson2 , Anil V. Rao1 , Carolynn Patten3,4 and Benjamin J. Fregly1,2\**

*1Department of Mechanical and Aerospace Engineering, University of Florida, Gainesville, FL, USA, 2Department of Biomedical Engineering, University of Florida, Gainesville, FL, USA, 3Department of Physical Therapy, University of Florida, Gainesville, FL, USA, 4Neural Control of Movement Lab, Malcom-Randall VA Medical Center, Gainesville, FL, USA*

Researchers have explored a variety of neurorehabilitation approaches to restore normal walking function following a stroke. However, there is currently no objective means for prescribing and implementing treatments that are likely to maximize recovery of walking function for any particular patient. As a first step toward optimizing neurorehabilitation effectiveness, this study develops and evaluates a patient-specific synergy-controlled neuro musculoskeletal simulation framework that can predict walking motions for an individual post-stroke. The main question we addressed was whether driving a subject-specific neuromusculoskeletal model with muscle synergy controls (5 per leg) facilitates generation of accurate walking predictions compared to a model driven by muscle activation controls (35 per leg) or joint torque controls (5 per leg). To explore this question, we developed a subject-specific neuromusculoskeletal model of a single high-functioning hemiparetic subject using instrumented treadmill walking data collected at the subject's self-selected speed of 0.5 m/s. The model included subject-specific representations of lower-body kinematic structure, foot–ground contact behavior, electromyography-driven muscle force generation, and neural control limitations and remaining capabilities. Using direct collocation optimal control and the subject-specific model, we evaluated the ability of the three control approaches to predict the subject's walking kinematics and kinetics at two speeds (0.5 and 0.8 m/s) for which experimental data were available from the subject. We also evaluated whether synergy controls could predict a physically realistic gait period at one speed (1.1 m/s) for which no experimental data were available. All three control approaches predicted the subject's walking kinematics and kinetics (including ground reaction forces) well for the model calibration speed of 0.5 m/s. However, only activation and synergy controls could predict the subject's walking kinematics and kinetics well for the faster non-calibration speed of 0.8 m/s, with synergy controls predicting the new gait period the most accurately. When used to predict how the subject would walk at 1.1 m/s, synergy controls predicted a gait period close to that estimated from the linear relationship between gait speed and stride length. These findings suggest that our neuromusculoskeletal simulation framework may be able to bridge the gap between patient-specific muscle synergy information and resulting functional capabilities and limitations.

Keywords: biomechanics, computational neurorehabilitation, direct collocation optimal control, muscle synergy analysis, neuromusculoskeletal modeling, predictive gait optimization

#### *Edited by:*

*Ramana Vinjamuri, Stevens Institute of Technology, USA*

#### *Reviewed by:*

*Laurent Simon, New Jersey Institute of Technology, USA Jonathan B. Shute, University of Florida, USA*

> *\*Correspondence: Benjamin J. Fregly fregly@ufl.edu*

#### *Specialty section:*

*This article was submitted to Bionics and Biomimetics, a section of the journal Frontiers in Bioengineering and Biotechnology*

*Received: 01 June 2016 Accepted: 21 September 2016 Published: 13 October 2016*

#### *Citation:*

*Meyer AJ, Eskinazi I, Jackson JN, Rao AV, Patten C and Fregly BJ (2016) Muscle Synergies Facilitate Computational Prediction of Subject-Specific Walking Motions. Front. Bioeng. Biotechnol. 4:77. doi: 10.3389/fbioe.2016.00077*

## INTRODUCTION

Roughly one in six people worldwide will suffer a stroke at some point in their lifetime, with ~15 million people experiencing a stroke each year (World Stroke Organization, 2016). Due to improvements in acute stroke management, the majority of these individuals will survive their initial stroke, which has helped make stroke a leading cause of serious, long-term disability in adults worldwide (Go et al., 2013; World Stroke Organization, 2016). More than a third of stroke survivors experience significant physical disability (Lloyd-Jones et al., 2010), with walking dysfunction being among the greatest stroke-related limitations contributing to disability. While the majority of persons who suffer a stroke regain some level of ambulatory function, their gait is typically slow, asymmetrical, and metabolically inefficient (Olney et al., 1986; Roth et al., 1997). Diminished walking ability is tied to decreased quality of life, increased risk of depression, and increased risk of serious secondary health conditions (Blair et al., 1989; Mutikainen et al., 2011; Ostir et al., 2013). Restoration of walking function following a stroke is therefore both a high priority for rehabilitation and an important public health problem.

Despite recognition of the problem, current clinic-based neurorehabilitation methods produce only modest improvements in walking function for persons post-stroke (States et al., 2009; Bogey and Hornby, 2014; Winstein et al., 2016). For this reason, researchers and clinicians have explored a variety of neurorehabilitation approaches in search of an effective means to restore post-stroke walking function. These approaches include functional electrical stimulation (FES) (Popovic et al., 1999; Kesar et al., 2009, 2010; Sabut et al., 2013; Chung et al., 2014; O'Dell et al., 2014; Pilkar et al., 2014; Auchstaetter et al., 2015; Chantraine et al., 2016), ankle–foot orthoses (AFOs) (Ferreira et al., 2013; Tyson et al., 2013; Kobayashi et al., 2016), exoskeletons (Nilsson et al., 2014; Bortole et al., 2015; Buesing et al., 2015), partial body weight support (Ng et al., 2008; Lee et al., 2013) and split-belt treadmill training systems (Reisman et al., 2007; Malone and Bastian, 2014), and robotic gait trainers (Pennycott et al., 2012; Mehrholz et al., 2013; Bae et al., 2014; Hussain, 2014; Dundar et al., 2015). Each of these approaches has shown varying levels of promise for improving post-stroke walking function. However, a critical challenge is determining the treatment prescription – *which* approach to apply, *how much* of the approach to apply, and *how* the approach should be applied – that will maximize recovery of walking function for any particular individual. Furthermore, there is currently no way to identify whether a small amount of treatment provided by a combination of approaches might be dramatically more effective than a large amount of treatment provided by a single approach (Belda-Lois et al., 2011). Current treatment design methods based on trial-and-error and subjective clinical judgment cannot address these challenges, since they do not provide an objective means for predicting a patient's walking function following a specified treatment or treatment combination.

One possible approach for overcoming this challenge is to use patient-specific neuromusculoskeletal models to predict post-treatment walking function for different neurorehabilitation technologies (alone or combined) under consideration. Such models should account for how the patient interacts with the treatment approach (Mooney and Herr, 2016) so that the optimal prescription can be recommended based on objective predictions of walking improvement. A number of studies have pursued such modeling efforts by simulating the effects of FES (Riener, 1999; Heilman and Kirsch, 2003; Zhang and Zhu, 2007; Shao and Buchanan, 2008; Nekoukar and Erfanian, 2013; Sharma et al., 2014; Alibeji et al., 2015), exoskeletons (Fleischer and Hommel, 2008; Afschrift et al., 2014; Farris et al., 2014; Sawicki and Khan, 2015), orthoses (Zmitrewicz et al., 2007; Crabtree and Higginson, 2009; Silverman and Neptune, 2012), and strength training (Goldberg and Neptune, 2007; Knarr et al., 2014) on lower extremity function and walking ability in the context of stroke, spinal cord injury, amputee, and general rehabilitation. Few of these studies focused on stroke (Goldberg and Neptune, 2007; Shao and Buchanan, 2008; Knarr et al., 2014), few used threedimensional models (Fleischer and Hommel, 2008; Afschrift et al., 2014; Farris et al., 2014; Knarr et al., 2014; Sawicki and Khan, 2015), few used subject-specific models created by calibrating critical neuromusculoskeletal model parameters to movement data collected from an individual (Fleischer and Hommel, 2008; Shao and Buchanan, 2008; Knarr et al., 2014), and only one included modeling elements that accounted for subject-specific neural control capabilities and limitations (Alibeji et al., 2015). No study to date has predicted a stroke patient's complete posttreatment walking motion and speed resulting from application of a specific neurorehabilitation intervention.

As a first step toward optimizing patient interaction with stroke neurorehabilitation technologies, this study describes the development and evaluation of a subject-specific synergycontrolled neuromusculoskeletal simulation framework that can predict three-dimensional walking motions for an individual post-stroke. The main question we address is whether actuating a subject-specific neuromusculoskeletal model with muscle synergy controls (5 per leg) facilitates generation of accurate walking predictions compared to actuating the model with muscle activation controls (35 per leg) or joint torque controls (5 per leg). We hypothesize that synergy controls will work the best since they combine a low number of control signals with a subject-specific representation of the coupling between muscle activations within each leg. We collect gait data from a stroke subject walking at 0.4, 0.5, 0.6, 0.7, and 0.8 m/s on an instrumented treadmill and use data from his self-selected speeds of 0.4–0.6 m/s to develop a subject-specific neuromusculoskeletal model. We incorporate the subject-specific full-body model into a direct collocation optimal control framework to predict new walking motions for the subject. To evaluate the framework and the potential benefits of using synergy controls, we predict how the individual will walk (including cadence and stride length) at 0.5 and 0.8 m/s (conditions for which experimental data are available for comparison) using joint torque, muscle activation, or muscle synergy controls and at 1.1 m/s (a condition for which no experimental data are available) using only synergy controls. With future simulation of different neurorehabilitation approaches, our subject-specific synergy-controlled neuromusculoskeletal simulation framework may help identify optimal neurorehabilitation prescriptions that maximize recovery of walking function on an individual patient basis.

## METHODS

### Experimental Data Collection

To assist with development and evaluation of our subjectspecific synergy-controlled neuromusculoskeletal simulation framework, we collected experimental walking data from one high-functioning hemiparetic male individual with chronic stroke-related walking dysfunction (age 79 years, LE Fugl-Meyer Motor Assessment 32/34 pts, right-sided hemiparesis, height 1.7 m, mass 80.5 kg). All study procedures were approved by the University of Florida Health Science Center Institutional Review Board (IRB-01) and the Malcom Randall VA Medical Center Research and Development Committee and included approval to study individuals with stroke-related disability. Study personnel obtained written informed consent prior to participant enrollment and involvement in study activities. Study procedures were conducted in accordance with the Declaration of Helsinki. Motion capture (Vicon Corp., Oxford, UK), ground reaction (Bertec Corp., Columbus, OH, USA), and electromyography (EMG) data (Motion Lab Systems, Baton Rouge, LA, USA) were collected simultaneously while the participant walked on a split-belt instrumented treadmill (Bertec Corp., Columbus, OH, USA) at speeds ranging from 0.4 to 0.8 m/s in increments of 0.1 m/s. 0.8 m/s was the fastest speed at which the subject could walk safely without assistance. This range of speeds included the participant's self-selected walking speed of 0.5 m/s. More than 50 gait cycles were recorded at each walking speed. A static standing trial was collected for model scaling purposes. To facilitate subsequent creation of subject-specific foot–ground contact models, the participant wore Adidas Samba Classic sneakers, which have a flat sole and neutral midsole with no cushioning, and we collected additional static trials where we used a marker wand to trace the outline of each sneaker sole on the ground. Motion capture data were obtained using a modified Cleveland Clinic full-body marker set with additional markers added to the feet (Reinbolt et al., 2005). Marker motion and ground reaction data were filtered at a variable cut-off frequency of 7/tf Hz, where tf is the period of the gait cycle being processed, using a fourthorder zero phase lag Butterworth filter (Hug, 2011). This variable cut-off frequency would cause data collected at a normal walking speed to be filtered at ~6 Hz.

Electromyography data were collected from 16 muscles in each leg and processed using standard methods (Lloyd and Besier, 2003). A combination of surface and fine-wire EMG electrodes was used. Surface EMG data were collected for gluteus maximus and medius, semimembranosus, biceps femoris long head, rectus femoris, vastus medialis and lateralis, medial gastrocnemius, tibialis anterior, peroneus longus, and soleus. Fine-wire EMG data were collected for adductor longus, iliopsoas, tibialis posterior, extensor digitorum longus, and flexor digitorum longus. All EMG data were high-pass filtered at 40 Hz (Lloyd and Besier, 2003), demeaned, rectified, and then low-pass filtered at a variable cut-off frequency 3.5/tf Hz. Filtering was performed using a fourth-order zero phase lag Butterworth filter. EMG data from each muscle were normalized to the maximum value over all trials and resampled to 101 time points per gait cycle (heel strike to heel strike for the less involved left side) while keeping an additional 20 time points before the start of the cycle to permit modeling of electromechanical delay. In addition, each processed EMG signal was offset on a cycle-by-cycle basis so that the minimum value was zero.

### Neuromusculoskeletal Model Development

The subject-specific neuromusculoskeletal model that served as the foundation for our simulation framework incorporated four modeling components to account for the unique neurophysiological and musculoskeletal characteristics of the subject: (1) a subject-specific lower-body kinematic model to simulate the subject's skeletal motion, (2) subject-specific foot–ground contact models to simulate how the subject's feet interact with the ground, (3) subject-specific EMG-driven muscle moment models to simulate the subject's lower extremity joint moments, and (4) a subject-specific muscle synergy control model to simulate the subject's neural control system. Below we describe each of these four components in further detail. Unless otherwise noted, we calibrated model parameters in each component using a single representative walking trial collected at the subject's self-selected speed of 0.5 m/s.

### Subject-Specific Lower-Body Kinematic Model

Our neuromusculoskeletal model creation process started with a generic full-body musculoskeletal model (Arnold et al., 2010; Hamner et al., 2010) developed in OpenSim (Delp et al., 2007). The original model included 37 degrees of freedom (DOFs) and 44 Hill-type muscle-tendon actuators in each leg. We locked the wrist and forearm pronation–supination angles to anatomically reasonable values for walking, leaving the following 31 DOFs: 6 DOF ground-to-pelvis joint, 3 DOF hip joints, 1 DOF knee joints, 1 DOF ankle joints, 1 DOF subtalar joints, 1 DOF toe joints connecting rear foot and toe segments, 3 DOF back joint, 3 DOF shoulder joints, and 1 DOF elbow joints. We also eliminated nine muscle-tendon actuators without related EMG data (extensor hallucis longus, flexor hallucis longus, gemelli, gracilis, pectineus, piriformis, quadratus femoris, sartorius, tensor fascia latae), leaving 35 muscles per leg that actuated hip flexion-extension, hip adduction-abduction, knee flexionextension, ankle flexion-extension, and ankle inversion–eversion on each leg. We then scaled the modified model using the standing static trial marker data and the OpenSim "Scale Model" tool, where distances between markers placed over identifiable landmarks were averaged between the two sides to maintain bilateral symmetry following scaling.

Once the model was scaled, we calibrated joint and marker positions in the torso, pelvis, and lower-body portion of the OpenSim model using marker data from a representative walking trial. The calibration approach was similar to one described previously (Reinbolt et al., 2005, 2008) except that it was performed on the scaled OpenSim model using the OpenSim-MATLAB Application Programing Interface and included modifications to maintain correct bone geometry positions within the body segments (Charlton et al., 2004). To facilitate the calibration process, we created marker plates on the torso, pelvis, thighs, and shanks to which all markers on the respective OpenSim body segments were attached. To perform the actual calibration, we used non-linear least squares optimization (lsqnonlin) in MATLAB to adjust joint (knee, ankle, and subtalar in both legs) and marker plate (torso, pelvis, thighs, and shanks) positions and orientations in their respective body segments such that marker errors from repeated OpenSim inverse kinematic analyses were minimized. The optimization cost function included penalty terms that prevented large changes in joint and marker plate positions and orientations that would produce only small improvements in marker tracking. Modification of the two hip joint center locations was achieved by modifying the position and orientation of the rigid marker plate on the pelvis. For joint centers and orientations, symmetry between left and right sides of the body was maintained during the kinematic calibration process. Markers on the feet were not adjusted since their locations were well defined. The position and orientation of the toe axis in each foot and of the back, shoulder, and elbow joints was maintained from the scaled OpenSim model.

#### Subject-Specific Foot–Ground Contact Models

Following kinematic calibration, we created a subject-specific foot–ground contact model for each foot of the OpenSim model using recently developed methods (Jackson et al., 2016). The elastic foundation contact models were developed in MATLAB and used a grid of contact elements that spanned the rear foot and toes segments of each foot. To create the element grid, we started with the shoe outlines obtained from the static trial marker data and used principal component analysis to identify the principal axes of each foot (rear foot and toes segments together). Using these axes, we constructed an 11 (anteriorposterior) × 8 (medial-lateral) grid of rectangular contact elements for the left foot, where the edges of the grid extended 2.5 mm beyond the edge of the shoe outline in both directions. Forty-seven elements whose centers fell within the shoe outline were retained in the contact model, while 41 elements whose centers fell outside the shoe outline were removed. Given the locations of the MATLAB contact element centers relative to the foot markers from the static left shoe outline trial, we calculated the locations of the element centers on the OpenSim rear foot and toes segments. We then projected the left toes axis of the OpenSim model onto the contact element grid. Elements whose centers were posterior to the axis were assigned to the rear foot segment, while elements whose centers were anterior to the axis were assigned to the toes segment. The complete MATLAB/ OpenSim contact element grid for the left foot was mirrored to the right foot by aligning the principal axes of the mirrored grid with those of the right foot.

Each contact element in the foot–ground contact models generated normal force using a linear spring with non-linear damping and shear force using a continuous stick-slip friction model. For any contact element *i*, the required time-varying inputs for contact force calculations performed in MATLAB were the penetration into the floor *yi*, the normal penetration rate *y<sup>i</sup>* , and the shear slip rate *vsi* of the element center in the laboratory coordinate system as calculated by OpenSim. The normal contact force *Fi* for element *i* was calculated as (Hunt and Crossley, 1975)

$$F\_i = k\_i(\mathcal{Y}\_i - \mathcal{Y}\_0)(1 + c\dot{\mathcal{Y}}\_i) \tag{1}$$

where *ki* is the spring stiffness unique to each spring, *y*0 is the spring resting length common to all springs on the same foot and essentially adjusts the height of the floor, and *c* (= 1e–2) is a non-linear damping coefficient common to all springs. The linear spring also generates a small amount of force when the foot is off the floor, and this negligible force transitions in a smooth and continuous way to the large force produced when the spring is in contact with the ground (Anderson and Pandy, 2001; Ackermann and van den Bogert, 2010). The non-linear damping ensures that the normal contact force does not exhibit damping-related discontinuities when a spring enters or leaves contact. The shear contact force *fi* for element *i* was calculated using a simple continuous and differentiable friction model (Ackermann and van den Bogert, 2010)

$$f\_i = F\_i \left[ \mu \tanh\left(\frac{\nu\_{s\_i}}{\nu\_I}\right) \right] \tag{2}$$

where μ [= 1 (Ackermann and van den Bogert, 2010)] is a dynamic friction coefficient common to all springs and *vl* (= 5 cm/s) is a latching speed common to all springs that defines the edge of a linear transition region between zero slip rate and the start of dynamic friction. Shear contact force *fi* was applied to the element center in the direction opposite to the slip velocity vector. The direction calculation included a small constant value of 1e−4 in the denominator to avoid division by a small number when the slip speed was near zero. Once *Fi* and *fi* were calculated for each contact element, the net contact force and torque due to all contact elements in the rear foot segment were calculated with respect to the rear foot origin, and similarly for the toes segment using the toes origin (Kane and Levinson, 1985). These net contact forces and torques were then applied to their corresponding segments in the OpenSim model. This approach allowed rear foot and toes contributions to total ground reaction force to be predicted by the model.

We calibrated the spring stiffness *ki* of each contact element in both feet and the spring resting length *y*0 for all contact elements in each foot using marker and ground reaction data from a representative walking trial. We made two assumptions about the spring stiffness distribution across the bottom of the shoe to simplify the calibration process. First, we assumed that the mirrored stiffness distribution was the same for both feet. Second, we assumed that the stiffness distribution across the entire shoe bottom could be approximated by a three-dimensional parabolic surface, which possesses only six unknown coefficients rather than 47 unknown independent spring stiffness values and prevents neighboring springs from having dramatically different stiffnesses. To calibrate these six coefficients and the two resting lengths, we formulated a direct collocation optimal control problem that tracked experimental marker, ground reaction, and inverse dynamic joint torque data for the entire body with higher weight placed on matching marker position and ground reaction data for the two feet. Tracked ground reaction quantities for each foot included the three ground reaction force components and three ground reaction torque components calculated about the midfoot marker projected onto the floor (see Optimal Control Walking Predictions below for further details).

### Subject-Specific EMG-Driven Muscle Moment Models

To generate subject-specific joint moments from predicted muscle activations, we calibrated lower extremity EMG-driven muscle moment models for both legs to a large number of walking trials collected from the subject. Complete details of our EMG-driven model calibration process, and a full assessment of its ability to predict joint moments accurately for the same subject walking at multiple speeds, can be found in Meyer et al. (2016). In brief, the model calibration process used experimental walking data collected from the subject at 0.4, 0.5, and 0.6 m/s, bracketing his self-selected speed of 0.5 m/s. Ten trials from each speed were used for calibration. We adjusted three types of model parameter values in our calibration process: (1) EMG-to-activation parameter values, (2) Hill-type muscle-tendon model parameter values, and (3) surrogate musculoskeletal geometry parameter values. Below we describe each category of adjusted model parameter values in greater detail.

For our EMG-to-activation model, parameter values adjusted during calibration included electromechanical delays, EMG scale factors, activation time constants, and muscle non-linearity constants. A single electromechanical time delay between 0 and 100 ms was used for all muscles in the same leg, allowing the two legs to have different electromechanical delays. An EMG scale factor was found for each muscle in each leg, resulting in 70 different scale factors. An activation time constant for a first-order activation dynamics model (He et al., 1991) was found for each muscle, with time constants assumed to be identical for the same muscle in both legs, resulting in 35 different time constants. Deactivation time constants were assumed to be four times larger than corresponding activation time constants (Thelen, 2003; De Groote et al., 2012; Millard et al., 2013). Finally, a nonlinear constant defining the conversion of neural activation to muscle activation was found for each muscle (Buchanan et al., 2004), with non-linear constants assumed to be identical for the same muscle in the two legs, resulting in 35 different non-linear constants.

For our Hill-type muscle-tendon model, parameter values adjusted during calibration included optimal muscle fiber lengths and tendon slack lengths. We used a custom Hill-type muscle-tendon model with a rigid tendon, as a recent study has shown that use of a compliant tendon model for simulations of walking has little effect on predicted muscle activations and forces (De Groote et al., 2016). This model was implemented in MATLAB to facilitate customization of model properties. Initial Hill-type model parameter values (optimal muscle fiber length, tendon slack length, pennation angle) for each muscle were taken from the literature (Arnold et al., 2010) and assumed to be the same for both legs. Optimal fiber length and tendon slack length values were pre-calibrated to reproduce passive hip, knee, and ankle flexion moment data reported in the literature (Silder et al., 2007). Peak isometric force for each muscle was defined using regression equations for muscle volume reported in the literature (Handsfield et al., 2014) along with each muscle's optimal fiber length and a maximum muscle stress of 61 N/cm2 (Arnold et al., 2010). Maximum shortening velocity for each muscle was defined to be 10 optimal fiber lengths per second.

For our surrogate musculoskeletal geometry, parameter values adjusted during calibration included coefficients of polynomial functions defining muscle-tendon lengths as a function of spanned joint angles. For each muscle-tendon actuator in our kinematically-calibrated OpenSim model, we first created a surrogate model of muscle-tendon length using a cubic polynomial function of all spanned joint angles. Some muscles required a cubic function of only one joint angle (e.g., vastus medialis), while other muscles required a cubic function of multiple joint angles (e.g., gluteus maximus and gastrocnemius medialis). We then created corresponding surrogate models of muscle-tendon velocity and moment arms by defining muscle-tendon velocity as the first derivative of the muscle-tendon length polynomial with respect to time and each muscle moment arm as the negative of the first derivative of the muscle-tendon length polynomial with respect to the corresponding spanned joint angle (An et al., 1984). In this way, the polynomial functions defining muscle-tendon lengths, velocities, and moment arms shared common coefficients (Menegaldo et al., 2004; Sartori et al., 2012a).

To generate an initial polynomial fit for each muscle, we sampled muscle-tendon lengths and moment arms from our kinematically calibrated OpenSim model using a wide range of lower extremity joint angle combinations. The maximum and minimum value of each joint angle were allowed to go well beyond the values achieved by the subject during walking. Sampling was performed using 1000 different model poses specified using a Latin hypercube design, with muscle-tendon lengths and moment arms being calculated by an OpenSim "Muscle Analysis." After outliers were removed corresponding to situations with muscle wrapping problems, we used linear least squares regression to fit muscle-tendon length and moment arms for each muscle as a cubic polynomial function of spanned joint angles.

We used sequential quadratic programing (SQP) optimization (fmincon) in MATLAB to adjust the parameter values described above such that EMG-driven models for both legs matched lower extremity inverse dynamic and passive joint moment curves as closely as possible. Inputs to the optimization were the subject's processed experimental EMG data, joint kinematics from OpenSim "Inverse Kinematics" analyses, and joint moments from OpenSim "Inverse Dynamics" analyses from 30 selected walking trials, along with the published passive joint moment data described earlier (Silder et al., 2007). For modeled muscles without experimental EMG data (e.g., vastus intermedius) or with multiple compartments (e.g., gluteus maximus), EMG data from anatomically related muscles were used but with a separate scale factor for each muscle/compartment (Sartori et al., 2012b). Outputs were the model parameter values and predicted inverse dynamic and passive joint moments. The subject's hip internal–external rotation moment from walking was not included in the calibration process, since EMG data were not collected from primary hip external rotator muscles. Thus, the EMG-driven model for each leg matched five inverse dynamic moments (hip flexion–extension, hip adduction–abduction, knee flexion–extension, ankle flexion–extension, and ankle inversion– eversion). During calibration, optimal muscle fiber length and tendon slack length values were allowed to vary within 25% of the values produced by pre-calibration, while changes in surrogate musculoskeletal geometry were strongly penalized so that such changes would be made only if they resulted in significant improvements in joint moment matching. The calibrated EMGdriven model for each leg was verified by predicting lower extremity joint moments for walking trials withheld from calibration, including trials from faster walking speeds. These models were used in all subsequent activation- and synergy-driven optimal control simulations.

#### Subject-Specific Muscle Synergy Control Model

The standard method of performing muscle synergy analysis uses only processed experimental EMG data. Following processing, EMG data for one complete walking cycle are organized into a matrix, and non-negative matrix factorization is applied via an optimization approach to decompose the high dimensional set of processed EMG signals into a lower dimensional set of time-varying signals (which we will call "synergy controls") with associated sets of muscle weights (which we will call "synergy vectors") (Lee and Seung, 1999; Tresch et al., 1999). The synergy vector weights specify how each synergy control contributes to each processed EMG signal. Each time a synergy analysis is performed, the number of synergies to fit must be specified *a priori*. Typically only three to six synergy controls are needed to reconstruct a much larger number of processed EMG signals with a high "variability account for" (VAF), typically above 90% (Ivanenko et al., 2005; Clark et al., 2010).

There are at least two drawbacks to the standard synergy analysis approach. First, the absolute amplitude of each processed EMG signal remains unknown. Though EMG signals are commonly normalized using data from a maximum voluntary contraction trial or the movement trial with maximum signal amplitude, maximal M-wave measurements reveal that these methods do not yield the true maximum EMG value (Clark et al., 2006; Simonsen et al., 2012; Racinais et al., 2013; Cronin et al., 2015). Since EMG amplitudes affect the results of a muscle synergy analysis, this issue makes it difficult to use experimentally derived synergy information for musculoskeletal modeling purposes. Second, since standard synergy analysis only uses experimental EMG data, it does not provide any information on how the inter-muscle coupling quantified by the synergy vectors affects an individual's ability to perform specific movement tasks. Thus, a gap exists between subject-specific neural control information provided by standard muscle synergy analysis and the functional consequences of that information.

To bridge this gap while also addressing the EMG normalization issue, we perform muscle synergy analysis for our subject within the larger context of producing a dynamically consistent full-body walking motion using a subject-specific neuromusculoskeletal model (Sharif Razavian et al., 2015). With standard synergy analysis, the goal is to find a specified number of synergy controls and vectors that best match a larger set of normalized muscle EMG signals. With our approach, the goal is to find a specified number of synergy controls and vectors that best match experimental joint motions, ground reactions, lower-body inverse dynamic joint moments, and scaled EMG signals produced by our EMG-driven models. Thus, the breadth of data matched by our approach is much larger than that of a standard muscle synergy analysis. We describe our expanded approach as "dynamically consistent synergy analysis." Use of EMG-driven models within our neuromusculoskeletal model provides a unique way to address the EMG normalization issue, while finding dynamically consistent full-body walking motions with our neuromusculoskeletal model bridges the gap between neural control information and its functional consequences.

To find synergy controls and vectors that could reproduce our subject's walking data at his self-selected speed of 0.5 m/s, we followed a two-step process. First, we performed standard synergy analysis on the 35 muscle activations for each leg produced by the EMG-driven model calibration process for a representative walking trial. We performed this step on muscle activations rather than scaled EMG signals since we omitted activation dynamics from our final neuromusculoskeletal model to reduce model complexity. We incremented the number of synergies found by standard synergy analysis until the total VAF was greater than 95% and the VAF for each muscle was greater than 85%. We chose these high values since our goal was not simply reconstruction of activations but also reproduction of a dynamically consistent walking motion. Five synergies were required to achieve the target VAF values. Second, we performed a tracking optimization using direct collocation optimal control where all muscles in each leg were driven by five synergy controls. The synergy controls and associated synergy vectors were unknowns to be found by the optimization. The optimization tracked ground reactions, muscle activations, lower-body joint torques, and upper body joint motions while producing a dynamically consistent walking motion (see Optimal Control Walking Predictions below for further details).

## Optimal Control Walking Predictions

We used the subject-specific neuromusculoskeletal model described above and direct collocation optimal control to predict the subject's walking motion at 0.5 and 0.8 m/s given walking data from the most periodic trial collected at his self-selected speed of 0.5 m/s. A key advantage of collocation methods over shooting methods is that they use implicit rather than explicit simulation. Repeatedly during the non-linear programing (NLP) solution process, shooting methods perform explicit simulation to solve the system dynamics sequentially for one time frame at a time via numerical integration. This process is often unstable because either errors accumulate with each integration step or the system being simulated is inherently unstable, as with the human body during walking. By contrast, as part of the NLP solution process, collocation methods perform implicit simulation to solve the system dynamics for all time frames simultaneously with no notion of time stepping. Consequently, instabilities arising from accumulated integration errors or inherent system instabilities are eliminated, facilitating the use of gradient-based optimization for predicting motion. Another advantage of collocation is that feedback control, which is artificially introduced in explicit simulation to maintain system stability, is unnecessary since time stepping is not performed by implicit simulation. An overview of various numerical methods for solving optimal control problems, along with a brief discussion of the advantages and disadvantages of each, can be found in Limebeer and Rao (2015).

We investigated how well the predictions worked using three different control situations: joint torque controls (5 per leg – termed "torque-driven"), muscle activation controls (35 per leg – termed "activation-driven"), and muscle synergy controls (5 per leg – termed "synergy-driven"). To control the motion of each leg, the torque-driven problems used 5 joint torques rather than 35 muscles, the activation-driven problems used 35 muscles controlled by 35 independent muscle activations, and the synergy-driven problems used 35 muscles controlled by 5 independent muscle synergy controls that were linearly combined to produce 35 muscle activations. For all three control situations, hip internal–external rotation and toes flexion–extension in both legs along with the three pelvis rotations were found by tracking the corresponding joint angles from the 0.5 m/s periodic trial.

To generate our predictions, we performed a sequence of three categories of optimizations: (1) calibration optimizations, (2) tracking optimizations, and (3) prediction optimizations (see **Table 1** for overview). This sequence was needed since large-scale direct collocation optimal control problems are often sensitive to the initial guess, making it helpful to increase the complexity of the problems being solved in a gradual and systematic fashion. Furthermore, for each category of optimization, initial problems were solved where the skeletal dynamic constraints (as quantified by pelvis residual loads) were not enforced, and these constraints were gradually tightened in subsequent problems until the desired tolerance was met. Below we describe concepts common to all three optimization categories, integration of OpenSim functionality into the optimal control framework, and details for the three optimization categories.

TABLE 1 | Sequence of direct collocation optimal control problems solved using GPOPS-II to predict patient-specific walking motions at 0.5, 0.8, and 1.1 m/s.


*All optimizations were performed using a full-body patient-specific neuromusculoskeletal model with calibrated joint, musculoskeletal geometry, and muscle-tendon model parameter values (see text). Tracking optimizations were performed at 0.5 m/s to develop dynamically consistent baseline data for each of the three lower extremity control situations (joint torques, muscle activations, or muscle synergies) used in subsequent prediction optimizations. Prediction optimizations were performed for 0.5 m/s to verify optimal control problem formulation, 0.8 m/s to evaluate problem formulation, and 1.1 m/s to challenge problem formulation. Activations employ 35 controls per leg while synergies employ 5 controls per leg. Note that Problem 1.1 serves as the basis for Problem 1.2, Problem 1.2 serves as the basis for Problems 2.1, 2.2, and 2.3, and Problems 2.1, 2.2, and 2.3 serve as the basis for Problems 3.1, 3.2, and 3.3, respectively.*

#### Common Concepts

The walking predictions reported in this study were generated using GPOPS-II, a direct collocation optimal control toolbox for MATLAB (Patterson and Rao, 2014). GPOPS-II solves for the state *x*(*t*), control *u*(*t*), and static parameters *p* that minimize the cost functional:

$$J = \phi\left(\mathbf{x}(t\_0), t\_0, \mathbf{x}(t\_f), t\_f, p\right) + \int\_{t\_0}^{t\_f} \mathbf{g}\left(\mathbf{x}(t), \boldsymbol{\mu}(t), p\right) dt\tag{3}$$

subject to the constraints

$$
\dot{\mathbf{x}}(t) = f(\mathbf{x}(t), \boldsymbol{\mu}(t), t, p), \text{ (dynamic constraints)}, \qquad (4)
$$

*c f x t u t t p c* min max ≤ ( ) ( ), ( ), , , ≤ a( ) lgebraic constraints , (5)

*b b x t t x t t p b* min m *f f* ax ≤ ≤ ( ( 0 0 ), , ( ), , ) , . b( ) oundary conditions (6)

The state and control are parameterized using variable-order Gaussian quadrature orthogonal collocation methods and formulated into a NLP problem.

Within the toolbox, two NLP solvers can be utilized: SNOPT (Gill et al., 2005) or IPOPT (Biegler and Zavala, 2009). SNOPT employs a quasi-Newton SQP active set method where the inverse of the Hessian is approximated using a Broyden–Fletcher–Goldfarb–Shanno (BFGS) update. At each step in the optimization, all linear constraints are satisfied and the active set is estimated. SNOPT achieves convergence via a merit function that is the objective function plus the sum of constraint infeasibilities. IPOPT employs an interior-point method where the goal is to satisfy the constraints via a barrier function. IPOPT does not distinguish between linear and nonlinear constraints; all constraints are treated the same. IPOPT operates in two different modes. In first derivative mode, IPOPT employs a quasi-Newton method where the inverse of the Hessian is estimated using a BFGS update. In second derivative mode, IPOPT employs a full-Newton method where GPOPS-II provides a sparse representation of the lower triangular part of the Hessian of the Lagrangian of the NLP problem. Based on convergence and computation time considerations, we chose IPOPT in first derivative mode as the initial NLP solver for all optimal control problems. Though GPOPS-II contains an adaptive mesh refinement algorithm, we used a fixed mesh of 50 collocation points, divided into 10 intervals, over the entire gait cycle to reduce computation time (Ackermann and van den Bogert, 2010).

In this study, we developed a jerk-controlled inverse dynamic problem formulation using the 31 DOF skeletal dynamic equations generated by OpenSim (Delp et al., 2007). This approach was used since it exhibited improved convergence properties and solution smoothness over a directly controlled forward dynamic problem formulation (van den Bogert et al., 2011), but at the cost of added joint jerk controls. It also allowed us to violate the skeletal dynamics when starting from a poor initial guess, facilitating finding a feasible solution. For this problem formulation, we defined the control *u*(*t*) as the third time derivative of the generalized coordinates *q*(*t*) (i.e., joint jerk). The problem state consisted of the coordinate positions *q*(*t*), velocities *v*(*t*), and accelerations *a*(*t*), simplifying the dynamics of the optimal control problem to:

$$
\begin{bmatrix}
\dot{q} \\
\dot{\nu} \\
\dot{a}
\end{bmatrix} = \begin{bmatrix}
\nu \\
a \\
u
\end{bmatrix} \tag{7}
$$

To make the joint jerk controls unique, we added minimization of joint jerk to every optimal control problem. Jerk minimization terms in the cost function were scaled by *tf* <sup>6</sup> , where *tf* is the specified final time, since jerk magnitude is proportional to *tf* 3 based on analysis of analytic functions, and joint jerk squared is minimized in the cost function. Since joint jerk magnitudes change with final time, we did not want jerk minimization to affect the final times predicted by our optimizations.

For our inverse dynamic problem formulation, enforcement of the skeletal dynamics was achieved using algebraic path constraints. Two types of path constraints were used for this purpose. The first type ensured that residual forces and torques acting on the pelvis were eliminated. Each time an inverse dynamic analysis was performed with the OpenSim model, six pelvis residual loads *R*pelvis(*x*(*t*)) were calculated. To enforce dynamic consistency, we added the algebraic path constraint

$$R\_{\rm min} \le R\_{\rm pelvis} \le R\_{\rm max} \tag{8}$$

to constrain the six pelvis residual loads to be within a specified tolerance. The second type of constraint enforced consistency between net muscle moments calculated from additional controls and lower extremity joint moments calculated from inverse dynamics. The additional controls were 5 joint torques per leg for torque-driven problems, 35 activations per leg for activation-driven problems, 5 synergies per leg for synergy-driven problems. For all three control situations, we added algebraic path constraints to ensure that the additional controls balanced five lower extremity inverse dynamic joint moments in each leg (i.e., hip flexion–extension, hip adduction–abduction, knee flexion–extension, ankle flexion–extension, and ankle inversion– eversion). Convergence tolerances were set to 1 N and 0.1 Nm for residual forces and moments, respectively. When this tolerance was met, we considered the resulting motion to be dynamically consistent, requiring negligible fictitious external loads to balance the dynamic equations. All optimal control solutions presented in this study utilized this inverse dynamic problem formulation. However, the cost function, controls, and constraints varied depending on the category of optimization problem being solved.

#### OpenSim Integration

Development of dynamically consistent full-body walking predictions required integrating OpenSim functionality into the MATLAB environment in a computationally efficient manner (**Figure 1**). To perform the integration, we took advantage of two aspects of our optimal control solution process. First, use of joint jerk controls made complete OpenSim model state and state derivative information available for any time point being analyzed, making it possible to evaluate the OpenSim skeletal dynamic equations in an inverse sense and avoiding the need to calculate OpenSim model state derivatives for forward solutions.

FIGURE 1 | Flowchart demonstrating the interaction of MATLAB and OpenSim functions for synergy-driven prediction optimizations performed within the GPOPS-II MATLAB environment. For each iteration of the NLP solver, GPOPS-II provides a new guess for the model state and control, which are used by a series of MATLAB and OpenSim functions to calculate path constraints, terminal constraints (not shown – used to enforce near-periodicity in motion and ground reactions), and the cost function. The cost function requires knowledge of state and control quantities that are being tracked, which are taken from the results of the corresponding tracking optimization. Path constraints are used to satisfy skeletal dynamics and to make lower-body joint moments calculated by an OpenSim inverse dynamic analysis match corresponding joint moments calculated by a MATLAB EMG-driven model. Synergy-driven prediction optimizations use five synergy controls per leg as controls. Activation-driven prediction optimizations use 35 muscle activations per leg as controls and do not require the MATLAB muscle synergy model. Torque-driven prediction optimizations use five joint torques per leg as controls and do not require the MATLAB muscle synergy model or EMG-driven model. Calibration optimizations include additional path constraints to match additional experimental quantities (e.g., ground reaction forces and moments).

Second, use of a direct collocation solution approach allowed each time point to be analyzed independently from all others, making it possible to employ parallel processing methods on a time-point-by-time-point basis.

To take advantage of these two aspects, we created MATLAB MEX functions in C++ that parallelized two OpenSim tasks: (1) calculation of positions and velocities of all contact elements on the bottom of each foot, and (2) calculation of inverse dynamic joint torques. We parallelized OpenSim point kinematic calculations using OpenMP and OpenSim inverse dynamic calculations using MPI. For parallelized point kinematic calculations, inputs were the OpenSim model state; while for parallelized inverse dynamic calculations, inputs were the OpenSim model state and calculated external ground reactions.

Using these parallelized functions, we employed an efficient series of calculation steps each time GPOPS-II needed to evaluate the OpenSim skeletal dynamic equations across all time points. First, we called our parallel C++ point kinematics function to calculate the position and velocity of every contact element in the model. Second, we used this point kinematic information to calculate the contact force acting on each element and the net contact force and torque to be applied to the rear foot and toes segments of each foot. Third, we evaluated our surrogate musculoskeletal geometry using the current joint positions and velocities to find muscle-tendon lengths, velocities, and moment arms for all muscles in the model. Fourth, we used our Hill-type muscle model with the current muscle activations and muscle-tendon lengths and velocities to calculate all tendon forces. Fifth, we calculated muscle contributions to all joint moments by multiplying tendon forces by their associated moment arms. Sixth, we called our parallel C++ inverse dynamics function to apply the calculated ground reactions to the rear foot and toes segments of both feet as OpenSim external forces and calculate inverse dynamic joint torques. Seventh, we calculated joint torque errors for the five muscle-controlled lower-body joints in each leg by subtracting the joint torques calculated in OpenSim using inverse dynamics from the joint torques calculated in MATLAB using GPOPS-II additional controls and moment arms where needed. All of these calculations were performed in MATLAB apart from the two parallel C++ functions that called OpenSim functionality.

#### Calibration Optimizations

We performed two calibration optimizations to determine parameter values in the foot–ground contact models and to provide a starting point for subsequent tracking optimizations (see **Table 1**, top section). For both optimizations, the cost function tracked experimental marker, ground reaction, and inverse dynamic joint torque data from the periodic trial while also minimizing joint jerk. The constraints enforcing dynamic consistency and joint jerk were the only controls. IPOPT was used in first derivative mode to generate an initial guess, and then SNOPT was started at that initial guess to refine the solution. For the first optimization, static parameters were included for the six coefficients that defined the parabolic distribution of spring stiffness across the bottom of each foot and for the common spring resting length for each foot. For the second optimization, all static parameters were removed from the problem, the parameter values were fixed to those found by the first optimization, and the optimization repeated to verify that identified foot–ground contact model parameter values could closely reproduce the experimentally measured ground reactions for both feet. Root-mean-square (RMS) errors in ground reactions produced by the second optimization were within 10 N for forces and 5 Nm for moments calculated about the midfoot marker projected onto the floor.

#### Tracking Optimizations

Starting from the results of the second calibration optimization, we performed three tracking optimizations for the 0.5 m/s walking speed to provide a starting point for subsequent prediction optimizations (see **Table 1**, middle section). Each tracking optimization utilized a different method to control 5 lower-body DOFs (hip flexion–extension, hip adduction–abduction, knee flexion–extension, ankle flexion–extension, and ankle inversion–eversion) in each leg. The first tracking optimization was torque-driven (5 independent joint torque controls per leg), the second was activation-driven (35 independent muscle activation controls per leg), and the third was synergy-driven (5 independent synergy controls per leg used to construct the 35 muscle activations per leg). For all three optimizations, the cost function tracked lower-body joint torques (apart from hip internalexternal rotation and toes flexion–extension) and upper body, hip internal-external rotation, and toes joint angles from the second calibration optimization while minimizing joint jerk controls. We did not track joint angle and joint torque curves for the same joint to avoid having related terms for the same joint in the cost function. We selected joint angle tracking over joint torque tracking for the toes and upper body joints since we found that large changes in toes or arm motion required only small changes in the corresponding joint torques. We also tracked pelvis angles so that the model would maintain the proper orientation in the laboratory. Path constraints satisfied skeletal dynamics and bounded joint angle errors relative to the second calibration optimization and ground reaction errors relative to experimental data. For the torque-driven optimization, the controls were joint jerk and 5 joint torques for each leg, and algebraic path constraints were used to match 5 inverse dynamic joint torques per leg with the 5 joint torque controls per leg. The activation-driven optimization replaced the 5 torque controls with 35 lower extremity muscle activations per leg, added tracking of 35 activation controls per leg from the EMG-driven models to the cost function, and used algebraic path constraints to match 5 inverse dynamic joint torques per leg with activation controls per leg. The synergy-driven optimization was identical except that the 35 activation controls were replaced with 5 synergy controls per leg, inverse dynamic joint torque matching used synergy-constructed muscle activations, and static parameters were added to allow identification of the corresponding 5 sets of synergy vector weights. In addition, a constraint was added to force each synergy vector to have unit magnitude, making the synergy solutions unique. IPOPT in first derivative mode was used to solve all tracking optimization problems, with gradients calculated using central differencing.

#### Prediction Optimizations

Starting from the results of the tracking optimizations, we performed three prediction optimizations for the 0.5 m/s walking speed as a "sanity check," three for the 0.8 m/s walking speed as a predictive evaluation, and one using only synergy controls for a 1.1 m/s walking speed as a challenge to our simulation framework. For the 0.5 m/s and 0.8 m/s predictions, the first tracking optimization was torque-driven, the second was activation-driven, and the third was synergy-driven. For the 1.1 m/s prediction, only a synergy-driven model was used to make predictions. The goal of these optimizations was to see whether each type of control could predict not only a realistic walking motion with realistic ground reactions but also the correct period for one gait cycle, which decreases with increasing walking speed. Since the 0.5 m/s speed was the same as the tracking optimization speed, matching the experimental period closely provides a "sanity check" on the solution. Due to problems we encountered with solving these optimal control problems using free final time, we performed each prediction optimization using five values of final time: the average experimental value rounded to the nearest tenth of a second plus or minus 0.1 and 0.2 s. We then fit a parabola to the final cost function values plotted as a function of final time and took the minimum value as the predicted final time. All results reported were taken from the simulations whose final times were closest to the parabola's minimum value.

For each control situation, the optimal control problem formulation for both speeds was similar to the corresponding tracking optimization except that several cost function and constraint terms were removed and no static parameters were utilized, allowing new walking motions and ground reactions to be predicted. For all three optimizations, the cost function tracked upper body, hip internal–external rotation, and toes flexion–extension joint angles from the corresponding tracking optimization solution while minimizing joint jerk controls, and the path constraints satisfied the skeletal dynamics. The torque-driven problem added tracking of 5 lower extremity joint torques per leg found by the corresponding tracking optimization to the cost function. The activation-driven problem added 35 lower extremity muscle activations per leg to the controls, tracking of 35 activations per leg found by the corresponding tracking optimization to the cost

function, and matching of 5 inverse dynamic joint torques per leg with corresponding joint torques produced by muscle activations to the algebraic path constraints. The synergy-driven problem added 5 synergy controls per leg to the controls, tracking of 5 synergy controls per leg found by the corresponding tracking optimization to the cost function, and matching of 5 inverse dynamic joint torques per leg with corresponding joint torques produced by the synergy-constructed muscle activations to the algebraic path constraints. The synergy-driven problem used the synergy vectors found by the corresponding tracking optimization. The implicit assumption in this prediction approach is that when the subject walks at any speed, he will choose controls (joint torques, muscle activations, or synergy controls) that are "close" to those he uses at his self-selected speed of 0.5 m/s. IPOPT in first derivative mode was again used to solve all prediction optimization problems.

### RESULTS

Tracking optimizations using all three types of controls closely reproduced the subject's experimentally measured walking motion at 0.5 m/s. Simulated lower and upper body joint angles

were within experimental ranges from multiple walking cycles (**Figure 2** – first column), as were simulated lower-body joint torques (**Figure 3** – first column). However, joint torques from the torque-driven model were generally less smooth than those from the activation-driven and synergy-driven models, especially for the two hips. Simulated ground reaction forces were also within experimental ranges (**Figure 4** – first column). For the activationdriven and synergy-driven models, simulated activations were within the experimental ranges determined by the EMG-driven models (**Figures 5** and **6**), with only small changes in activations needed to produce dynamically consistent walking motions.

Prediction optimizations using all three types of controls also closely reproduced the subject's experimentally measured walking motion, gait period, and stride length at 0.5 m/s (**Table 2** – top rows). Simulated lower and upper body joint angles (**Figure 2** – second column), lower-body joint torques (**Figure 3** – second column), and ground reaction forces (**Figure 4** – second column) were extremely similar to those produced by the corresponding tracking optimizations. The most noticeable minor differences were for the torque-driven model, where the predicted hip torques were again less smooth than for the other two control types, as were the predicted

medial–lateral ground reaction forces. For the activationdriven and synergy-driven models, simulated activations were also very similar to those found by the corresponding tracking optimizations (**Figures 7** and **8**). All three models predicted a realistic gait period and stride length (**Figure 11**, first column; **Table 2**, top rows).

Only prediction optimizations that used activation controls or synergy controls closely reproduced the subject's experimentally measured walking motion, gait period, and stride length at 0.8 m/s (**Table 2** – bottom rows). For these two control types, simulated lower joint angles were within or just outside experimental ranges from multiple walking cycles, while for torque controls, every simulated lower-body joint angle went beyond the experimental ranges (**Figure 2** – third column), and the optimization predicted hiking of the paretic hip to compensate for reduced knee flexion on that side, which is a biologically plausible strategy. Interestingly, for all three control types, simulated upper body joint angles were extremely different from

those observed experimentally (see **Figure 12** for a visual comparison between experimentally measured and computationally predicted full-body motions). Similar to the joint angle results, simulated lower-body joint torques (**Figure 3** – third column) and ground reaction forces (**Figure 4** – third column) produced by activation and synergy controls generally remained within or at the edge of experimental ranges, while those produced by torque controls generally went beyond them, especially for the ground reaction forces. All three control types predicted an increase in anterior–posterior ground reaction force that was comparable to the increased range measured experimentally to achieve an increase in walking speed. For the activation-driven



*Predicted walking motions were generated using torque-driven, activation-driven, and synergy-driven models. A large difference in gait period and stride length existed between the two experimental gait speeds. All three models predicted gait period and stride length within experimental ranges for the slower speed, while only the synergydriven model predicted them within experimental ranges for the faster speed.*

and synergy-driven models, simulated activations remained within the experimental ranges determined by the EMG-driven models (**Figures 9** and **10**). The synergy-driven model predicted the most realistic gait period and stride length, which were 1.16 s compared to an experimental median of 1.14 s for gait period and 0.93 m compared to an experimental median of 0.91 m for stride length (**Figure 11**, second column; **Table 2**, bottom rows). The gait period and stride length predictions for the activation-driven model were right at the boundary of the measured experimental ranges, while the predictions for the torque-driven model were outside those ranges.

The final prediction optimization used synergy controls to simulate walking at 1.1 m/s, a condition for which no experimental data are available, and predicted a gait period of 1.12 s. Assuming a linear relationship between walking speed and stride length (Jordan et al., 2007), as defined by the subject's median speed and stride length for 0.5 and 0.8 m/s, the estimated experimental gait period for this faster walking speed would be 1.06 s. Thus, the synergy-driven model predicted not only the best gait period for the 0.8 m/s walking speed but also a physically realistic gait period with physically realistic walking motion (not shown) for an even faster walking speed without available experimental data.

### DISCUSSION

This study developed and evaluated a subject-specific synergycontrolled neuromusculoskeletal simulation framework that predicted three-dimensional walking motions for an individual post-stroke. We investigated whether actuating a neuromusculoskeletal model of the subject with muscle synergy controls (5 per leg) facilitated generation of accurate walking predictions compared to actuating the model with muscle activation controls (35 per leg) or joint torque controls (5 per leg). We found that walking predictions generated for both 0.5 and 0.8 m/s were more accurate (in terms of joint motions, joint torques, ground reactions, and final time) and converged more easily for the activation-driven and synergy-driven models than for the torque-driven model. Furthermore, the accuracy of the walking predictions at both speeds was comparable for the activation-driven and synergy-driven models, even though the synergy-driven model used 30 fewer controls per leg. When the synergy-driven model was used to predict the subject's walking motion at 1.1 m/s, a condition for which no experimental data were available, the predicted stride length was close to that calculated by the linear speed–stride length relationship fitted to experimental data from 0.5 and 0.8 m/s. Overall, these findings are encouraging and suggest that the current simulation framework could provide a useful foundation for predicting how a patient will interact with different neurorehabilitation approaches (e.g., FES, AFO, exoskeleton, robotic gait trainer, strength training) so that an optimal neurorehabilitation prescription can be identified.

We had to overcome a number of practical challenges to generate the walking predictions presented in this study. One significant challenge was obtaining convergence of our optimal control problems, which we addressed using several strategies. First, we used sequences of optimizations that gradually increased the complexity of the problem being solved and provided a good initial guess for the next level problem. Second, we compared IPOPT in first derivative mode, IPOPT in second derivative mode, and SNOPT to determine which option worked best for our problems. Third, we tracked joint angles rather than joint torques for joints not controlled by muscles, since tracking joint torques permitted large changes in toes and arm motions using only small changes in joint torques. Fourth, we replaced free final time problems with a sequence of five fixed final time problems and then fitted a parabola to the cost function values to determine the final time of the corresponding free final time problem. Fifth, we replaced an explicit dynamics formulation requiring no additional controls with an implicit dynamics formulation requiring additional jerk controls to obtain smooth motion and control predictions. The other significant challenge was obtaining reasonable computation times, which we resolved by parallelizing the computationally costly skeletal dynamics and point kinematics calculations performed by OpenSim. Once a reasonable initial guess was available, torque-driven and synergy-driven problems typically required about 30 min of CPU time to converge, though the variation in convergence time was wide, while activation-driven typically required about an hour of CPU time. On a practical basis, however, we normally perturbed the most recent solution and re-ran each optimization to help avoid entrapment in a local minimum, making estimation of total CPU time difficult. More research is needed to determine how to improve the convergence properties of these problems.

Given that numerous previous studies have generated muscle-actuated full-body forward dynamic simulations of walking (Gerritsen et al., 1998; Anderson and Pandy, 2001; Thelen et al., 2003; Ackermann and van den Bogert, 2010, 2012; Geyer and Herr, 2010; McGowan et al., 2010; Allen et al., 2013; Knarr et al., 2013, 2014; Kia et al., 2014; Dorn et al., 2015), it is worth considering the unique aspects of our approach. Only a small number have predicted new walking motions for which experimental data are not available (Anderson and Pandy, 2001; Ackermann and van den Bogert, 2010, 2012; Dorn et al., 2015). Few studies have used direct collocation optimal control methods to solve for all time points simultaneously (Ackermann

and van den Bogert, 2010, 2012), few have used subject-specific (Knarr et al., 2014) rather than scaled generic or simplified musculoskeletal models, and few have modeled individuals with neurological impairment (Allen et al., 2013; Knarr et al., 2013). No previous muscle-actuated full-body walking simulation studies have combined subject-specific EMG-driven modeling with subject-specific synergy controls to define the neural control

structure of the model. Furthermore, no previous studies have calibrated joint and ground reaction force parameters in the model to match walking data collected from a specific subject. Thus, the most unique feature of our study was combining all of the various modeling and optimization elements listed above into a single comprehensive simulation framework capable of predicting new walking motions.

Only the activation-driven and synergy-driven models were able to predict the correct walking motion and gait period closely at both 0.5 and 0.8 m/s. Since the only difference between the torque-driven model and the other two models was the control method, it is likely that numerical issues related to the use of pure torque controls were the source of the problem. This hypothesis is supported by results from two previous simulation studies. Risher et al. (1997) showed that even small inconsistencies in inverse dynamic solutions, such as those introduced by spline fitting, can produce large motion errors when the calculated joint torques are used to control a forward dynamic simulation intended to reproduce the original motion. Our approach requires spline fitting of joint torque and joint angle data so that the optimal control solver can obtain values at any desired collocation point.

Gerritsen et al. (1998) showed that muscle force–length–velocity properties provide proportional-derivative-like feedback control properties that help stabilize forward dynamic simulations of walking and prevent drift away from a desired motion. Based on the finding of these two studies, it is less surprising that the activation-driven and synergy-driven models performed better than the torque-driven model.

Although both our synergy-driven and activation-driven models generated accurate walking predictions, our synergydriven model still possesses several distinct advantages. The primary advantage is the significantly reduced number of controls compared to the activation-driven model. By using muscle synergy controls, we were able to predict highly realistic subject-specific walking motions using the same number of

controls as in the torque-driven model but without any of the problems encountered by that model. Significantly fewer controls (30 fewer per leg) reduce computational cost and complexity and make the optimal control solution process less sensitive to poor initial guesses. Though not demonstrated in our study, we believe that the synergy-driven model also has the best potential for simulating individuals with neurological impairment, since subject-specific synergy information limits how a subject can coordinate his muscles. For example, it would be interesting to eliminate one synergy at a time from the paretic leg in our model and predict the functional impact on our subject's walking pattern. Would the model's ability to reproduce the subject's walking

motion break down if a lower number of synergies were used? Synergy controls within a neuromusculoskeletal model provide an excellent avenue for simulating the functional consequences of reduced neural complexity (Allen et al., 2013).

To use our neuromusculoskeletal simulation framework for actual clinical treatment design, we would need to determine how to incorporate different treatment approaches into the framework. Simulating the effects of strength training could be achieved by increasing the peak isometric strength of individual muscles or groups of muscles. Simulating the effects of FES could be achieved by adding controls to the optimal control problem that augment the activation of one muscle or several muscles. Simulating the effects of an AFO, exoskeleton, robotic gait trainer, or exercise device could be achieved by adding a model of the device to the patient's OpenSim model [e.g., Fregly et al. (2015)] and adding static parameters and controls to the optimal control problem that account for modifiable design features of the device. The biggest challenge with simulating any of these

treatment approaches is defining an appropriate optimization cost function. When predicting how the patient's neural control system will respond to a treatment, should the cost function minimize an absolute quantity, such as metabolic cost, should it minimize neural control changes away from some experimentally measured baseline situation (as we have done here), or should it hold the patient's neural control strategy constant and change only static parameters and controls related to the treatment? The issue of predicting how a patient will interact with a treatment is a critical one for researchers to explore in the future.

Despite the high level of subject specificity in our neuromusculoskeletal model, our study possesses a number of limitations that can help inform future research efforts. First, we modeled only a single hemiparetic subject. Our goal for the present study was to develop and evaluate the initial implementation of our neuromusculoskeletal simulation framework, which required use of only a single subject. In the future, we plan to test the framework further using walking data collected from additional hemiparetic subjects. Second, we performed all model calibration steps (lower-body kinematic model, foot–ground contact model, EMG-driven model, muscle synergy model) using only a static trial and walking data, primarily from the subject's self-selected speed of 0.5 m/s. Though this limitation was planned to simplify the model calibration process, use of a wider variety of calibration movements could improve the predictive capabilities of the model. Third, we did not model any neural feedback mechanisms (e.g., from muscle spindles and Golgi tendon organs). Though the extent to which feedback mechanisms contribute to the control of walking remains controversial, it is possible that inclusion of neural feedback models could have a significant impact on our predicted walking motions (Geyer and Herr, 2010). Fourth, we evaluated our predicted walking motions using only the calibration speed and a single faster non-calibration speed, where the faster speed was only 0.3 m/s faster. A more thorough evaluation would involve a wider range of speeds (our subject was unable to walk comfortably for an extended period of time above 0.8 m/s) and movement tasks. Fifth, while we controlled most lower-body joints with muscles, we tracked experimentally measured joint motions to determine the motion of the toes, hip internal–external rotation, and all upper body joints. Better prediction of toes and upper body motion in particular would likely be achieved if these joints could be controlled by muscles as well. Finally, even though our subject was hemiparetic, we assumed that most model parameter values were the same on both sides of the body, the exceptions being electromechanical delays, EMG scale factors, and muscle synergy control structure. When we allowed model parameter values to be different on the two sides, the accuracy with which our EMG-driven modeling process could predict joint moments improved little, suggesting that our current assumptions about bilateral symmetry were reasonable.

In conclusion, we have presented a novel combination of neuromusculoskeletal and computational modeling approaches that permits prediction of subject-specific walking motions for individuals with neurological impairment. The primary strengths of the approach are the high level of subject-specificity in the neuromusculoskeletal model calibration process, including a subject-specific representation of neural control limitations and capabilities via muscle synergy controls, the flexibility of the framework for changing the optimal control problem formulation and the characteristics of the OpenSim model, the ability to tie muscle synergies to their functional consequences, and the ability to predict new walking motions. With future developments, it may be possible to use our framework to simulate different neurorehabilitation interventions and ultimately to optimize treatment prescription so as to maximize recovery of walking function on an individual patient basis.

### AUTHOR CONTRIBUTIONS

AM assisted with experimental data collection, processed all experimental data, performed all neuromusculoskeletal modeling work, formulated and ran all optimal control problems,

### REFERENCES


and participated in writing the manuscript draft. IE integrated OpenSim dynamics functionality into the GPOPS-II optimal control environment and helped with revising the manuscript draft. JJ assisted with development and calibration of the foot– ground contact models and helped with revising the manuscript draft. AR assisted with development of optimal control problem formulations, helped troubleshoot GPOPS-II optimal control issues, and helped with revising the manuscript draft. CP recruited the experimental subject, ran the experimental data collection session, provided clinical assessment of predicted walking motions, and helped with revising the manuscript draft. BF planned and supervised the entire project, organized the experimental data collection session, assisted with experimental data collection, directed collaborative efforts between AM, IE, and JJ, assisted with formulation of optimal control problems, evaluated optimal control results, and participated in writing the manuscript draft.

### FUNDING

This study was funded by the National Science Foundation under grants CBET 1404767, CBET 1159735, and CBET 1052754 and by the National Institutes of Health under grants R01 EB009351 and U54 GM072970.

### SUPPLEMENTARY MATERIAL

MATLAB code and sample experimental inputs for performing direct collocation optimal control predictions of walking will be made available at https://simtk.org/projects/synwalkingpred.


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

The reviewer JS declared a shared affiliation, though no other collaboration, with the authors AM, IE, JJ, AR, CP, and BF to the handling Editor, who ensured that the process nevertheless met the standards of a fair and objective review.

*Copyright © 2016 Meyer, Eskinazi, Jackson, Rao, Patten and Fregly. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# **A Muscle Synergy-Inspired Adaptive Control Scheme for a Hybrid Walking Neuroprosthesis**

*Naji A. Alibeji, Nicholas Andrew Kirsch and Nitin Sharma\**

*Department of Mechanical Engineering and Materials Science, University of Pittsburgh, Pittsburgh, PA, USA*

A hybrid neuroprosthesis that uses an electric motor-based wearable exoskeleton and functional electrical stimulation (FES) has a promising potential to restore walking in persons with paraplegia. A hybrid actuation structure introduces effector redundancy, making its automatic control a challenging task because multiple muscles and additional electric motor need to be coordinated. Inspired by the muscle synergy principle, we designed a low dimensional controller to control multiple effectors: FES of multiple muscles and electric motors. The resulting control system may be less complex and easier to control. To obtain the muscle synergy-inspired low dimensional control, a subject-specific gait model was optimized to compute optimal control signals for the multiple effectors. The optimal control signals were then dimensionally reduced by using principal component analysis to extract synergies. Then, an adaptive feedforward controller with an update law for the synergy activation was designed. In addition, feedback control was used to provide stability and robustness to the control design. The adaptive-feedforward and feedback control structure makes the low dimensional controller more robust to disturbances and variations in the model parameters and may help to compensate for other time-varying phenomena (e.g., muscle fatigue). This is proven by using a Lyapunov stability analysis, which yielded semi-global uniformly ultimately bounded tracking. Computer simulations were performed to test the new controller on a 4-degree of freedom gait model.

**Keywords: non-linear control, adaptive control, time-invariant synergies, functional electrical stimulation, hybrid neuroprosthesis**

### **1. INTRODUCTION**

Each year, approximately 5100 people in the USA alone are diagnosed with paraplegia due to a spinal cord injury (The National SCI Statistical Center, 2014), impairing their ability to walk again. Functional electrical stimulation (FES) and powered orthoses are two viable technologies that have the potential to restore the walking function in persons with SCI (Kralj and Bajd, 1989; Kobetic et al., 1997; Farris et al., 2011; Neuhaus et al., 2011; Esquenazi et al., 2012; del-Ama et al., 2014a; Ha et al., 2015). FES is a clinical technique in which the muscle is artificially stimulated with low level electrical currents to produce muscle contractions (Peckham, 1987). The use of FES for gait restoration is limited by the rapid onset of muscle fatigue (Binder-Macleod and Snyder-Mackler, 1993), and powered exoskeletons require batteries and larger actuators to generate the torques necessary to produce the gait motion. However, combining the two technologies may provide the benefits of both powered exoskeletons and FES-based devices and overcome their limitation when used alone. A hybrid device composed of FES and electric motors may have smaller motors because

#### *Edited by:*

*Ramana Vinjamuri, Stevens Institute of Technology, USA*

#### *Reviewed by:*

*Arturo Forner-Cordero, University of São Paulo, Brazil Naveen Kuppuswamy, Istituto Italiano di Tecnologia, Italy*

> *\*Correspondence: Nitin Sharma nis62@pitt.edu*

#### *Specialty section:*

*This article was submitted to Bionics and Biomimetics, a section of the journal Frontiers in Bioengineering and Biotechnology*

> *Received: 20 May 2015 Accepted: 04 December 2015 Published: 21 December 2015*

#### *Citation:*

*Alibeji NA, Kirsch NA and Sharma N (2015) A Muscle Synergy-Inspired Adaptive Control Scheme for a Hybrid Walking Neuroprosthesis. Front. Bioeng. Biotechnol. 3:203. doi: 10.3389/fbioe.2015.00203* a FES-induced muscle torque would be able to generate a portion of the required torque. The use of a powered exoskeleton in the hybrid device would restrict unwanted degrees of freedom (DOF), reduce stimulation duty cycle of FES, and compensate for FES-induced muscle fatigue. Thus, the hybrid device may be capable of achieving longer walking durations and have additional therapeutic benefits of FES such as muscle growth and increased bone density.

However, multiple motors and FES of lower-limb muscles introduces limb coordination and actuator redundancy (e.g., limb joint torque can be produced by an electric motor of the hybrid exoskeleton and FES of joint flexors and extensors). To date, little research has been done on the design of controllers that consider the actuator redundancy in hybrid neuroprostheses. Quintero et al. (2012) used an adaptive control gain to distribute the control effort between an electric motor and FES for producing knee extensions. A cooperative controller was designed by Ha et al. (2015) for a hybrid walking neuroprosthesis. In this controller, feedback control was used to control the motors to track a desired limb trajectory. Then, an adaptation scheme was used to modify FES profiles to match the joint torque profiles of the electric motors in future gait cycles. However, actuator redundancy was not specifically addressed, this may be because only one or two muscles were stimulated. In Kobetic et al. (2009), a hybrid neuroprosthesis that used implanted electrodes to stimulate 16 muscles was used to achieve walking in a subject with paraplegia. del-Ama et al. (2014b) developed a cooperative control strategy that balanced FES and robotic control of the hybrid neuroprosthesis. The controller used PD control for the electric motors, PID control to maintain support during stance, and an iterative learning controller to develop the stimulation profiles for the swing cycle. The algorithm also detected FES-induced fatigue by measuring decreases in the torque-time integral of the force generated by FES. Currently, these systems only use *ad hoc* finite-state machine controllers and controller stability is not guaranteed.

Given the control challenge, a synergy based closed-loop controller may be ideal to handle the actuators redundancy and high dimensionality in the system. The central nervous system (CNS) is hypothesized to control the largely overactuated musculoskeletal system by activating the individual muscle fibers in groups called synergies, or motor primitives (Sherrington, 1910; Lee, 1984; Grillner, 1985; Tresch et al., 2002; Ting, 2007). Although it is a controversial hypothesis in the field of neural control of movement, these synergies can be thought of as weighted muscle activation patterns for multiple muscles that can be combined to generate coordinated limb movements (e.g., walking or reaching). It is hypothesized that these muscle synergies act as lower dimensional motion primitives that are stored in the spinal cord. Therefore, instead of individually controlling each muscle fiber, the human brain recruits weighted synergies to simplify a task involving multiple muscles or limbs. Currently, synergies are being used for a wide variety of applications, such as musculoskeletal movement analyses or gait therapy (Vinjamuri, 2008; Berniker et al., 2009; Vinjamuri et al., 2010; An et al., 2013; Routson et al., 2013; Steele et al., 2013; Simkins et al., 2014), robot design (Catalano et al., 2012; Wu and Asada, 2014), and control engineering systems (Popovic and Popovic, 2001; Wimbock et al., 2011; Kuppuswamy et al., 2012; Kuppuswamy and Harris, 2014; Wu and Asada, 2014).

From a controls perspective, synergies may be desired for controlling large and complex systems because they can provide simpler lower dimensional controllers that may be more computationally efficient. Some studies have used synergies to achieve lower dimensional control of systems with large DOF. Kuppuswamy et al. (2012) designed a synergy-based feedforward controller to drive robotic systems with redundant actuators to an equilibrium position. In Santello et al. (1998), synergy analyses were used to gain a further understanding of hand postures. This study showed that grasping movements can be explained by the first 2–3 postural synergies. Later, this work was used to design underactuated and simplified humanoid robot hands that mimicked the postural synergies observed in the grasping tasks (Catalano et al., 2012). Synergy inspired controllers were then designed for the humanoid robot hands (Ajoudani et al., 2013). In Berniker et al. (2009), a low-dimensional linear model of non-linear musculoskeletal frog hind-limb is found empirically by using the model order reduction technique balanced truncation. This low-dimensional model is then used to identify a set of muscle synergies using an optimization algorithm which were then used with optimal control techniques to produce a range of movements. The key advantage of synergy-inspired controllers is that the control of complex high DOF systems can be accomplished more efficiently using fewer control signals.

However, synergy-inspired control, to the best of our knowledge, has not been developed for a hybrid walking neuroprosthesis. This work uses the concept of time-invariant synergies and applies them for control design of a hybrid neuroprosthesis. The synergies can be extracted using statistical tools, such as nonnegative matrix factorization (NNMF), singular value decomposition (SVD), partial least squares regression (PLSR), or principal component analysis (PCA). Typically, synergy analyses of human motion studies use NNMF. For example, An et al. (2013) used NNMF to analyze muscle synergies of standing up motions with varying seat heights and standing speeds. Steele et al. (2013) studied the impact the number and choice of muscles have on synergy analyses in a musculoskeletal model for an upper extremity task. The benefit of an NNMF algorithm is that it maintains a positive value constraint on the decomposed synergies. This constraint is essential because muscle activations processed from EMG data always have positive values. However, in the hybrid neuroprosthesis system, electric motors are also present, which can generate both positive and negative torque values. Therefore, we employ PCA, instead of NNMF, to avoid the non-negative constraint.

The open hypothesis of this paper is that the hybrid walking system is a better rehabilitative intervention for subjects with spinal cord injury, and a control theoretic result is presented to enable such a system. The key contribution of this paper is the development of an adaptive synergy-based controller for a hybrid neuroprosthesis. Dynamic optimizations were used to produce optimal inputs and gait trajectories, using a subjectspecific gait model. A PCA-based decomposition technique was used to extract time-invariant synergies and their activation profiles that were present in the optimal input space. The activation profiles were further adapted online using a gradient-based update law to be used as feedforward control. Then feedback control to the motors was used to improve the performance and robustness of the overall controller. A Lyapunov-based stability analysis was performed to yield semi-global uniformly ultimately bounded tracking. Simulations on a 4-DOF gait model with 10 actuators (FES of three antagonistic muscle pairs, three electric motors, and one walker moment) are presented to depict performance and as a proof-of-concept of the muscle synergy-inspired controller.

### **2. DYNAMIC MODEL**

A person taking one step (half of a gait cycle), using a hybrid neuroprosthesis and a walker, is modeled as a four-link musculoskeletal system as seen in **Figure 1**. The hybrid neuroprosthesis uses a hip knee ankle foot orthosis (HKAFO) that provides kinematic constraints on the user, allowing only motion in the sagittal plane. In addition, the HKAFOs typically use a wrap spring clutch that locks the knee joint of the stance leg to prevent flexion when standing. This reduces the amount of stimulation needed which decreases muscle fatigue and prolongs walking durations (Sharma et al., 2014). The stance leg is modeled as one rigid segment simulating the locking of the knee joint and the ankle is fixed to the ground because only half of the gait cycle is considered in this simulation study. The swing leg has a thigh, shank, and foot segment with three actuators at each joint: motor and FES for flexion and extension of antagonistic muscle pairs. The trunk dynamics were neglected in the model because the use of a walker allows the user to stabilize their truck. The walker is modeled as a moment acting on the stance leg to help propel the body forward and also to keep it upright. The *n*-DOF lower limb model is given as

$$M\left(q\right)\ddot{q} + C(q,\dot{q})\dot{q} + G(q) + f(q,\dot{q}) + \tau\_d(t) + \tau\_{\text{ext}}(t) = \tau,\quad \text{(1)}$$

where *q, q*˙*,* ¨*q ∈* R *n* are the angular positions, velocities, and accelerations of the leg segments, respectively. In equation (1), *M*(*q*) *∈* R *n × n* is the combined inertia of the semi-powered orthosis and human limbs in the swing phase, *C*(*q, q*˙) *∈* R *n×n* is the centripetal/Coriolis matrix, *G*(*q*) *∈* R *n* is the gravity vector, *f*(*q, q*˙) *∈* R *n* is the viscoelastic vector term that model the passive muscle model, *τ ext ∈* R *n* is the torque generated at each joint due to contact with the ground, and *τ <sup>d</sup> ∈* R *n* is any unmodeled effects or disturbances in the system. The torques at the joints are generated by including the musculoskeletal dynamics due to FES (Popović et al., 1999), an electric motor attached at each joint, and the moment generated by the walker force. The torque term is defined as

$$
\tau = b(q, \dot{q})u,\tag{2}
$$

where *b ∈* R *<sup>n</sup> <sup>×</sup> <sup>m</sup>* is the control gain matrix containing the scaling functions for the *m* inputs.

**REMARK 1:** *b*(*q*,*q*˙) and *u*(*t*) are presented for a gait model with DOF, *n* = 4, and control inputs, *m* = 10. However, without loss of generality, the control development and analysis can be extended to *n*-DOF system with *m* inputs.

The model used in this work considers a hybrid neuroprosthesis that uses electric motors and FES via surface electrodes, which non-selectively applies an external voltage potential to a muscle group to generate a contraction. In equation (2), *b*(*q*,*q*˙) and

Frontiers in Bioengineering and Biotechnology | www.frontiersin.org

muscles (antagonistic hip, knee, and ankle muscle pairs in the swing leg), three electric motors acting on each joint of swing leg (*Th*, *T<sup>k</sup>*

acting on the stance leg (*Mw*). The step length is defined as the distance from stance toe to swing toe.

, *Ta*), and a walker moment

*u*(*t*) *∈* R *<sup>m</sup>* are defined as

$$b = \begin{bmatrix} 1 & 0 & 0 & 0 \\ 0 & \psi\_{h\circ} & 0 & 0 \\ 0 & -\psi\_{h\_{cx}} & 0 & 0 \\ 0 & 0 & \psi\_{k\circ} & 0 \\ 0 & 0 & -\psi\_{k\_{cx}} & 0 \\ 0 & 0 & 0 & \psi\_{a\_{\bar{k}}} \\ 0 & 0 & 0 & -\psi\_{a\_{\bar{\alpha}}} \\ 0 & \kappa\_{\bar{h}} & 0 & 0 \\ 0 & 0 & \kappa\_{\bar{k}} & 0 \\ 0 & 0 & 0 & \kappa\_{\bar{a}} \end{bmatrix}, \quad u = \begin{bmatrix} M\_{\text{w}} \\ u\_{h\circ} \\ u\_{h\circ} \\ u\_{k\circ} \\ u\_{k\circ} \\ u\_{a\_{\bar{\alpha}}} \\ u\_{a\_{\bar{\alpha}}} \\ T\_{k} \\ T\_{k} \\ T\_{\bar{k}} \end{bmatrix}, \tag{3}$$

where subscripts *i* = *h, k, a* stand for hip, knee, and ankle joints of the swing leg. In equation (3), *u<sup>i</sup>ex , u<sup>i</sup>fx* are the stimulation inputs and *ψifx , ψiex* are the torque-length and torque-velocity relationships of the flexor and extensor muscles, and *T<sup>i</sup>* is the current input to the motor and the conversion constants (current to torque) of the electric-motor drives is *κ<sup>i</sup>* . The moment due to the walker is denoted as *Mw*. For this simulation study, hip joint actuation via FES is achieved by stimulating the inner hip muscles (Iliopsoas) for flexion and the Gluteals for extension. Knee joint actuation uses the Quadriceps muscle group for extension and Hamstrings for flexion, and the ankle joint uses the Gastrocnemius for dorsiflexion and Tibialis anterior for plantarflexion.

**ASSUMPTION 1:** The trunk dynamics were neglected in the model because the use of a walker allows the user to stabilize their trunk. However, mass of the head, arm, and torso was incorporated in the model as a point mass.

**ASSUMPTION 2:** The motion is considered only in the sagittal plane because the HKAFO puts kinematic constraints on motion in planes other than sagittal. The HKAFO system uses a wrap spring clutch that locks the knee joint of the stance leg during walking. The stance leg is modeled as one link because the knee is locked and the stance leg ankle acts as an anchor because only half of the gait cycle is considered in this study. These assumptions allow us to model the kinematics of the lower extremities as a four-link chain.

**ASSUMPTION 3:** The walker is used to help produce the required propulsion force or *Mw*. As the user pushes against the walker to pull themselves forward, the resultant force acts as a moment on the hip of the user or the stance leg, *Mw*. Therefore, the walker moment, *Mw*, is treated as an input to the system that can be computed by the developed controller.

**ASSUMPTION 4:** First order muscle activation dynamics are ignored to simplify the control design. This avoids the use of control techniques, such as integrator backstepping (Khalil, 2002), which would add the requirement of additional signals, such as the acceleration, which is typically unavailable or very noisy (Sharma et al., 2009).

**ASSUMPTION 5:** The unmodeled effects or disturbances, *τ <sup>d</sup>*, are bounded as |*τ <sup>d</sup>* | *≤ ϵ*<sup>1</sup> where *ϵ*<sup>1</sup> *∈* R <sup>+</sup> is a constant.

**ASSUMPTION 6:** The control input, *u*, can be decomposed as *u* = *Wc<sup>d</sup>* + *uloss*, where the synergies in the matrix, *W*, are bounded constants and the time-varying activation coefficients, *cd*, are bounded signals. The reconstruction error, *uloss*, is bounded by a constant.

### **3. METHODS**

### **3.1. Dynamic Optimization**

Dynamic optimization was used to compute optimal subjectspecific gait trajectories and inputs (Kirsch et al., 2013; Sharma et al., 2014). In these optimizations, the model was only restricted to achieve a certain step size and step duration (0.4 m in 75 s). The optimization computes the inputs that minimize a user-defined cost function. One of the benefits of dynamic optimization is that it can account for constraints, such as a limited range of movement and strength of a user. These constraints are accounted for by constraining the optimization to a subject-specific dynamic model. Rather than tracking able-bodied gait data, which may be suboptimal when applied in the case of subjects with paraplegia (Popović et al., 1999, 2003; Dosen and Popovic, 2008, 2009; Pandy and Andriacchi, 2010) and may result in over stimulation of the muscles and quicken the onset of FES-induced muscle fatigue, the dynamic optimizations are used to compute subject-specific optimal trajectories. The following cost function and constraints were used to compute the optimal control inputs and joint angle trajectories:

$$\begin{aligned} \min\_{\mathcal{U}} \quad & \Pi = \int\_{t\_o}^{t\_f} u^T Q u \, dt \\ \text{subject to: } & M(q)\ddot{q} + \mathcal{C}(q, \dot{q})\dot{q} + \mathcal{G}(q) + \mathcal{f}(q, \dot{q}) \\ & + \tau\_{\text{ext}}(t) = b(q, \dot{q})u \\ & q(t\_o) = q\_o \\ & q(t\_f) = q\_f \\ & u \in [u\_l, u\_u] \end{aligned}$$

where *Q ∈* R *<sup>m</sup> <sup>×</sup> <sup>m</sup>* is a symmetric positive definite weight matrix, *q<sup>o</sup>* and *q<sup>f</sup>* are the initial and final joint angle vectors corresponding to the user-defined step length, and the lower and upper bound on the inputs are defined as *u<sup>ℓ</sup>* and *uu*. These bounds allow for the computation of an optimal solution while considering the physical constraints of the system, such as the maximum torque a motor can produce or the maximum amount of force a user can produce when using a walker. The inputs to the system are bounded by realistic values. The walker moment was constrained to 100 Nm and the motors torques are constrained to 40 Nm. The optimizations were run with 75 grid points for each control input in *u*. The inputs were interpolated using a linear interpolation. A second-order Heun's method with a step size of 1 ms was used for numerical integration. This smaller step size was used to prevent numerical divergence that may occur due to the harsh non-linearities in the dynamics, e.g., ground reaction model and passive muscle models, *f*(*q*,*q*˙), which diverge around hyperflexion and hyperextension.

### **3.2. Synergy Extraction**

Let *ud*(*t*) *∈* R *<sup>m</sup>* be the desired optimal control vector containing desired stimulation and motor voltage levels to achieve the desired optimal trajectory, *qd*(*t*) *∈* R *n* . The dynamics are written in terms of the optimal control inputs and kinematic trajectories as

$$\begin{aligned} M(q\_d)\ddot{q}\_d + C(q\_d, \dot{q}\_d)\dot{q}\_d + G(q\_d) + f(q\_d, \dot{q}\_d) + \tau\_{\text{ext}}^\*(t) \quad & \{4\} \\ \equiv b(q\_d, \dot{q}\_d)u\_d(t), \end{aligned}$$

where *τ ∗ ext* is the torque created at each joint due to the ground reaction force when using the optimal inputs, and *b<sup>d</sup>* = *b*(*q<sup>d</sup> , q*˙ *d* )is the desired control gain matrix, which is bounded. By using PCA, the possibly correlated inputs, *ud*, can be transformed into linearly correlated inputs, *cd*, such as

$$
\mu\_d = \mathcal{W}c\_d\left(t\right) + \mu\_{\text{loss}},\tag{5}
$$

where *W ∈* R *m × p* are the precomputed orthogonal synergies, and *cd*(*t*) *∈* R *p* are the corresponding time-varying activation coefficients of the synergies. The PCA analysis computes *m* synergies that account for all the variability of the data. The synergies are ordered such that the first synergy accounts for most of the variance, the second accounts for the second most, and so on. Typically, the rule of thumb is to use the number of synergies, *p < m*, that would account for over 90% of the variability of the data. But since the controller also has feedback control and adapts online, less synergies can be used. Therefore, instead of using three synergies as indicated in **Figure 4**, only two were used. After dropping the *m* – *p* synergies that account for the least amount of variability in the data, the reconstructed inputs,*Wcd*, do not match the optimal inputs, *ud*. Therefore, a reconstruction error, denoted as *uloss*, is introduced in equation (5).

### **3.3. Control Development**

In this section, we develop a controller that uses the synergies extracted in the previous section to reduce the dimensionality of the feedforward component. Also to improve control performance and robustness, an update law was used to adapt the time-varying activation coefficients online and feedback control for the motors was included. The control objective is to track a continuously differentiable desired trajectory *q<sup>d</sup> ∈* R *n* . The tracking error, *e ∈* R *n* , is defined as

$$e = q\_d - q.\tag{6}$$

To facilitate the control design and stability analysis, the auxiliary error signal *r ∈* R *n* is defined as

$$r = \dot{e} + \alpha e,\tag{7}$$

where *α ∈* R <sup>+</sup> is a control gain. The closed-loop error is derived by multiplying the time derivative of equation (7) with *M*(*q*) and substituting the dynamics in equation (1) to obtain

$$M\dot{r} = M\ddot{q}\_d + \dot{\mathcal{C}}\dot{q} + \mathcal{G} + f + \tau\_d + \tau\_{\text{ext}} - bu + M\alpha \dot{e}.\tag{8}$$

This expression can be written in the form

$$M\dot{r} = -Cr + \bar{N} + N\_d + \tau\_d + \tau\_{\text{ext}} - bu - e,\tag{9}$$

where *N*˜ = *N − N<sup>d</sup>* and the auxiliary signals *N*(*e,r*) and *N<sup>d</sup>* (*t*) are defined as

$$\begin{aligned} N &= M\ddot{q}\_d + \mathcal{C}\dot{q}\_d + \mathcal{C}\alpha e + G + f + M\alpha \dot{e} + e, \\ N\_d &= M(q\_d)\ddot{q}\_d + \mathcal{C}(q\_d, \dot{q}\_d)\dot{q}\_d + \mathcal{G}(q\_d) + f(q\_d, \dot{q}\_d). \end{aligned}$$

The term *N*˜ in equation (9) can be upper bounded by using the mean value theorem as

$$\left\|\left|\bar{N}\right|\right\| \leq \rho\_1(\left\|z\right\|)\left\|z\right\|\ , \tag{10}$$

where *ρ*1(||*z*||) *∈* R is a positive monotonic bounded function and *z ∈* R 2*n* is defined as

$$z = \begin{bmatrix} r^T & e^T \end{bmatrix}^T.$$

Note that the auxiliary signal *N<sup>d</sup>* is equal to the left-hand side of the desired muscle synergy dynamics in equation (4), this allows us to substitute *bdu<sup>d</sup> − τ ∗ ext* in for *N<sup>d</sup>* resulting in

$$M\dot{r} = -Cr + \ddot{N} + \tau\_d + \ddot{\tau}\_{\text{ext}} + b\_d u\_d - bu - e,\tag{11}$$

where *τ*˜*ext* = *τext − τ ∗ ext* is the torque due to the ground reaction force mismatch and can be bounded.

**REMARK 2**: Further analysis can be done to show that the bound on *τ*˜*ext* gets smaller as the position and velocity errors get smaller, i.e., as the tracking errors approach to 0, *τ*˜*ext* will approach to 0.

By choosing the control law *u* as

$$
\mu = \mathcal{W}\vec{\mathfrak{c}} + kr,\tag{12}
$$

where ˆ*c ∈* R *p* is the estimate of *c<sup>d</sup>* and *k ∈* R *m × n* is the feedback gain that is chosen to only influence the electric motors. The estimate of the synergy activation coefficient updates according to the following gradient-based update law with the projection algorithm

$$\dot{\hat{c}} = \text{proj}\left(\dot{c}\_d + \Gamma \mathcal{W}^T b\_d^T r\right),\tag{13}$$

where Γ *∈* R *p × p* is a symmetric positive definite learning rate gain matrix. The projection algorithm imposes an upper and lower bound on ˆ*c*, which is used in the stability analysis. More details of this algorithm can be seen in Khalil (2002). The purpose of the adaptation in the activation coefficient is to improve the feedforward component after reconstruction loss and to overcome any system uncertainties. After using equations (5) and (12), equation (11) becomes

$$M\dot{r} = -Cr + \ddot{N} + \tau\_d + \ddot{\tau}\_{\text{ext}} + b\_d u\_{\text{loss}} + b\_d W \overline{c} + \ddot{b} W \overline{c} - b k r - e,\tag{14}$$

where ˜*c ∈* R *p* and ˜*b <sup>∈</sup>* R *<sup>n</sup>×<sup>m</sup>* are defined as

$$
\bar{c} = c\_d - \hat{c}, \quad \bar{b} = b\_d - b.
$$

Using the mean value theorem, Assumption 5, and the property of the projection algorithm, the following terms can be bounded as

$$\left| \left| \bar{b} \right| \right| \le \rho\_2(\left| \left| z \right| \right) \left| \left| z \right| \right|, \ \left| \left| \mathcal{W} \hat{c} \right| \right| \le \epsilon\_2, \ \left| \left| \bar{\tau}\_{\text{ext}} + b\_d \mu\_{\text{loss}} \right| \right| \le \epsilon\_3, \ \left| \left| \bar{c} \right| \right| \le \delta,\tag{15}$$

where *ρ*2(||*z*||)*∈* R is a positive monotonically increasing bounded function and *ϵ*2, *ϵ*3, *δ ∈* R <sup>+</sup> are constants.

**THEOREM**: The controller designed in equations (12) and (13) ensures semi-global uniformly ultimately bounded tracking provided that the following gain conditions are met:

$$K\_{\min} > \frac{\left(\rho\_1(||z||) + \epsilon\_2 \rho\_2(||z||)\right)^2}{2}, \ \gamma\_{\min} \left\{bk - \gamma I\right\} > 0,$$

where *γmin*{*·*} denotes the minimum eigenvalue of a square matrix and *Kmin ∈* R <sup>+</sup> is a subsequently defined constant.

*Proof:* See Supplementary Material.

### **4. SIMULATION RESULTS**

### **4.1. Optimization and Synergy Extraction Results**

The optimization results are shown in **Figures 2** and **3**. **Figure 2** shows the optimal joint angle trajectories. **Figure 3** shows the optimal control inputs. The optimal contributions from the motor and FES can be adjusted by tuning the weights in the cost function.

The two synergies and their activation profiles extracted through PCA can be seen in **Figure 5**. Note that the scaling factors in the synergies on the left and the time-varying activation coefficients on the right can have negative values. This makes it harder to interpret what influence each synergy has on the system, but it is unavoidable when PCA is used. Also in the optimizations, the inputs to the stimulation channels are constrained to be positive, but after extracting the synergies, this property was lost. This results in negative stimulation values that are not applicable with FES because muscles are unidirectional actuators. Therefore, when implementing the controller, any negative stimulation inputs were set to 0.

### **4.2. Tracking Results**

The newly developed controller in equation (12) was simulated on a four-link rigid body gait model, developed in SimMechanics [MathWorks, CA, USA]. The head, arms, and torso were modeled as a point mass at the hips. The stance leg was modeled as a single link with a fixed knee joint and a pinned ankle joint. The swing leg was modeled with four-links: thigh, shank, and foot. Each link in the swing leg had redundant actuation, i.e., an electric motor and FES for the muscle flexors and extensors. The influence of the walker was modeled as a moment acting on the stance leg. This moment was used to help propel the body forward and help keep the body stable and upright. The unmodeled effects or disturbances, *τ <sup>d</sup>*, was incorporated by injecting uniformly distributed noise into the four joints. The masses and lengths for each limb were taken from anthropometric data (Winter, 2009), and the muscle parameters of a subject with SCI were taken from Popović et al. (1999) and Dosen and Popovic (2009). The ground reaction force was realized on two contact points: the toe and heel. The model uses a spring-damper system in the vertical direction and a static or kinetic friction model in the horizontal direction when the foot is in contact with the ground. More information on the specifics of this ground reaction model can be found in Geyer and Herr (2010).

To explore the efficacy of the controller, the simulations were done with four cases. Case 1 considered the synergies as the feedforward component but with no adaptation, i.e., *Wc<sup>d</sup>* in equation (5). Case 2 considered the synergies with adaptation, i.e., *Wĉ* with the adaptive law in equation (13). Case 3 considered both the synergies with adaptation and feedback control, i.e., equations (12) and (13). Case 4 considered the full optimal inputs computed in the optimizations with feedback control. Only the motors and walker moment were used as effectors to provide feedback. The control gains used in the cases that included feedback control were *k* = 10 and *α* = 100. In the two cases where adaptation was present, the learning rate used for the two synergies were 0.0175 and 0.001. The results are shown in **Figures 6**–**10**. The root mean squared error (RMSE) for the four joints for each case can be seen in **Table 1**. Of all the cases, the third and fourth cases were found to provide the best performance. In the first case, the feedforward component provides just enough control input to produce the

**FIGURE 3 | Optimal inputs to the walker moment, electric motors, and stimulation channels to reproduce the optimal gait trajectories**.

**be accounted for based on the number of synergies considered**. Rule of thumb would indicate using three synergies, but since the controller is not solely dependent on the feedforward component less synergies can be used.

movements but fails to clear the ground to complete the step. This is because the toe makes contact with the ground model early and begins to drag. In the second case, the swing leg joint angles match the desired profiles better and almost complete the walking step but the swing foot does not reach the floor in the allotted time of 0.75 s. In the third and fourth cases, the trajectories match the desired profiles almost perfectly.

### **5. DISCUSSION**

A muscle synergy approach can be useful for engineered systems with redundancy in effectors. For example, the research by de Rugy et al. (2013) mentions the usefulness of muscle synergies in FES-based systems. The muscle synergy principle has also been suggested as a hierarchical control framework for redundant manipulators (Todorov et al., 2005; Artemiadis and Kyriakopoulos, 2010), brain–machine interface-based control (Vinjamuri et al., 2011), and for the design and control of a humanoid robotic hand (Cho et al., 2007; Rosmarin and Asada, 2008; Catalano et al., 2012; Grioli et al., 2012). In our proposed adaptive control scheme, we showed that the synergy-based approach can be modified to provide a lower dimensional feedforward controller and combined with a feedback controller to control a hybrid walking neuroprosthesis.

As shown in the simulations, the new controller (Case 3) performs as expected only when both the adaptive feedforward and feedback components were active. However, in Case 1, when two synergies were used alone, the key characteristics of the

optimized gait was reconstructed, but the inputs from the two synergies were not enough to clear the ground and complete a full step as can be seen in **Figure 10**. This was likely caused by the reconstruction error, *uloss* in equation (5), due to the PCA

decomposition. Evidence of this can be seen by comparing the optimal inputs in **Figure 3** and the feedforward inputs in Case 1, as shown in **Figure 7**. To overcome the reconstruction error due to the synergy decomposition, we proposed adding an adaptive

**FIGURE 7 | Control inputs for Cases 1 and 2 of the simulations**. Note that the control input profile shapes, after PCA decomposition, in Case 1 may not be similar to the optimal inputs in **Figure 3**.

component and a feedback component to the synergy controller. In Case 2, the adaptive synergies provided sufficient control inputs to complete the walking step as well as enable the foot to clear the ground during the swing phase but the swing knee joint angle does not end at 0° as seen in **Figure 10**. This is evident in **Figure 6**, near the 40% gait cycle region, where the swing hip, knee, and ankle profiles showed improved tracking of the desired profile. In Case 3, the feedback control to the motors further improved the performance and the actual gait trajectories tracked the desired profile almost perfectly. In this case, the adaptive feedforward control may have given an approximate desired control input, and the feedback control fine tuned the input to further minimize the error.

In **Figure 8**, it can be seen that the amount of feedback motor torque and feedforward motor torque are comparable in magnitude. This indicates that the feedback is not doing all the work in this case. The need for the feedback torque is necessary because after dimensionality reduction, the feedforward component may not be enough to reproduce the movement due to reconstruction loss. However, in Case 4's results (**Figure 9**), where optimal inputs instead of reconstructed inputs were used, it can be seen that feedback control still played the same role as it did in Case 3. This is because the optimizations that computed the feedforward components did not consider system disturbances.

It can then be concluded that even if one were to use more synergies (greater than two), the feedforward component would

**FIGURE 9 | Control inputs for Case 4 of the simulations**. The feedback's contribution was used only in the walker moment and motor torques.

**FIGURE 10 | The gait sequence for the four cases, a step length of 0.4 m with a step duration of 0.75 s was used**. Since the errors for the third and fourth cases are so close their gait sequences look identical.

**TABLE 1 | The root mean squared (RMS) error for the four simulated cases**.


*Case 3 that had two synergies and feedback and Case 4 that had full optimal feedforward and feedback had the smallest RMS errors, followed by Case 2 that had just adaptive feedforward, and then Case 3 with the non-adaptive feedforward.*

still not be enough. But the benefit of decomposing the optimal inputs and truncating the amount of synergies used is the reduced amount of data needed in the real-time implementation of the controller. That is to say, instead of having the 10 signals with 750 data points each (0.75 s at 1 kHz) from the optimal inputs, the feedforward controller uses two signals and a matrix *W ∈* R 2*×*10 in this case. Therefore, the feedforward component was reduced from using 7500 data points to using only 1520 data points.

The limitation of PCA is that the decomposed synergies may not be easily interpreted. For example, in each synergy there is a scaling factor for each of the *m* control inputs and some synergies may have negative scaling factors. A negative scaling factor may not have any physical meaning (e.g., the stimulation inputs are always positive). Also, adaptation in one activation coefficient changes the scaling factors of all the control inputs in the corresponding synergy, which may result in a non-gait like motion. Interpretation of the synergies becomes even more inscrutable when PCA results in activation coefficients that can be negative.

The new control development is based on time-invariant synergies, which means that all the inputs within a synergy set were activated synchronously and temporal delays were not considered. Perhaps, the use of time-varying synergies, which have a spatial and temporal component, would result in less synergies and a more effective feedforward component. Also, synergies specific to the optimized gait data were extracted which means that they may not span the full input space of the system. However, the developed controller is general enough to be implemented on larger systems with more degrees of freedom and may be used with any set of synergies. A general set of synergies that are applicable to multiple tasks/movements, such as different step lengths and gait speeds, sitting/standing, or ascending/descending stairs, would provide a comprehensive data set to accomplish a control design for the hybrid neuroprosthesis. An optimization algorithm, such as the one used in Berniker et al. (2009), may be used to extract a more general set of synergies from a reduced model (lower dimensional) and used with this controller for a general task. While the focus of this paper was on designing automatic control methods that can handle actuator redundancy, gait optimizations in our result can be improved by using high fidelity gait models or optimization methods, such as in Ackermann and van den Bogert (2010)**.** Our future work will explore extracting muscle synergies based on optimization of these high DOF models and implementing these controllers on human subjects.

One method to find a generalizable set of synergies would be to have the optimizations compute a common set of synergies and the time-varying activation profiles to achieve multiple walking speeds and other tasks directly as opposed to computing the optimal inputs then extracting the synergies. The resulting set of synergies may be more generalizable than the current set extracted through PCA. Another benefit of using optimizations to extract the synergies is the possibility of more restrictions, e.g., nonnegative scaling factors in synergies (a limitation of PCA) or non-negative stimulation after synergy transformation.

In order to test these types of controllers in experimental trials, the controller must be scaled-up to achieve motions other than gait, such as sitting and standing. This can be achieved by designing a library of synergies that encompass walking, sitting, and standing. Also, because these optimizations are model based, extensive system identification experiments are required to find the subject-specific parameters that are used in the models. Undoubtedly, validating the synergy-inspired controller is not completely feasible with surface FES because the hip flexors and extensors are harder to access. However, during experimental implementation of this controller, only a motor could be used at the hip joint while muscle synergy-inspired controller can still be verified for redundant actuation at the knee and ankle joints. Alternatively, an invasive FES system, such as in Triolo et al. (1996), may provide access to over 40 different lower-limb muscles. Therefore, a synergy-inspired controller may be a very good candidate for the hybrid neuroprosthesis system proposed in Kobetic et al. (2009).

### **6. CONCLUSION**

In this paper, an adaptive synergy-based controller was presented for a hybrid walking neuroprosthesis. The controller used optimal inputs and trajectories, computed from dynamic optimizations, that were performed on a subject-specific gait model. A PCA algorithm was used to extract synergies from the optimal inputs to be used as a feedforward component to the controller. An update law was derived, using Lyapunov stability analysis, to adapt the time-varying activation coefficient of the synergies online. In addition, a feedback PD controller was used to make the controller more robust to disturbances. The efficacy of the controller was demonstrated in simulations on a four-link gait model with 10 actuators, including a walker moment, electric motors, and FES of the muscle flexors and extensors. Future work will focus on using time-varying synergies and different adaptation schemes, such as the adaptation of the scaling factors in the synergies.

### **ACKNOWLEDGMENTS**

The authors would like to thank Henry Phalen for his assistance with developing the gait models in SimMechanics. This work was funded in part by the National Science Foundation award number 1462876 and the University of Pittsburgh's Competitive Medical Research Fund.

### **SUPPLEMENTARY MATERIAL**

The Supplementary Material for this article can be found online at http://journal.frontiersin.org/article/10.3389/fbioe.2015.00203

### **REFERENCES**


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Copyright © 2015 Alibeji, Kirsch and Sharma. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# On the Origin of Muscle synergies: invariant Balance in the co-activation of agonist and antagonist Muscle Pairs

*Hiroaki Hirai1 \*, Fumio Miyazaki1 , Hiroaki Naritomi2 , Keitaro Koba1 , Takanori Oku1 , Kanna Uno1 , Mitsunori Uemura1 , Tomoki Nishi3 , Masayuki Kageyama3 and Hermano Igo Krebs1,4,5,6,7*

*1Department of Mechanical Science and Bioengineering, Graduate School of Engineering Science, Osaka University, Toyonaka, Japan, 2Department of Neurology, Senri Chuo Hospital, Toyonaka, Japan, 3Department of Rehabilitation, Senri Chuo Hospital, Toyonaka, Japan, 4Department of Mechanical Engineering, Massachusetts Institute of Technology, Cambridge, MA, USA, 5Department of Neurology, University of Maryland School of Medicine, Baltimore, MD, USA, 6Department of Rehabilitation Medicine I, School of Medicine, Fujita Health University, Toyoake, Japan, 7 Institute of Neuroscience, University of Newcastle, Newcastle upon Tyne, UK*

#### *Edited by:*

*Ramana Vinjamuri, Stevens Institute of Technology, USA*

#### *Reviewed by:*

*Arturo Forner-Cordero, University of São Paulo, Brazil Olivier Ly, University of Bordeaux, France*

> *\*Correspondence: Hiroaki Hirai hirai@me.es.osaka-u.ac.jp*

#### *Specialty section:*

*This article was submitted to Bionics and Biomimetics, a section of the journal Frontiers in Bioengineering and Biotechnology*

> *Received: 20 May 2015 Accepted: 06 November 2015 Published: 24 November 2015*

#### *Citation:*

*Hirai H, Miyazaki F, Naritomi H, Koba K, Oku T, Uno K, Uemura M, Nishi T, Kageyama M and Krebs HI (2015) On the Origin of Muscle Synergies: Invariant Balance in the Co-activation of Agonist and Antagonist Muscle Pairs. Front. Bioeng. Biotechnol. 3:192. doi: 10.3389/fbioe.2015.00192*

Investigation of neural representation of movement planning has attracted the attention of neuroscientists, as it may reveal the sensorimotor transformation essential to motor control. The analysis of muscle synergies based on the activity of agonist–antagonist (AA) muscle pairs may provide insight into such transformations, especially for a reference frame in the muscle space. In this study, we examined the AA concept using the following explanatory variables: the AA ratio, which is related to the equilibrium-joint angle, and the AA sum, which is associated with joint stiffness. We formulated muscle synergies as a function of AA sums, positing that muscle synergies are composite units of mechanical impedance. The AA concept can be regarded as another form of the equilibrium-point (EP) hypothesis, and it can be extended to the concept of EP-based synergies. We introduce, here, a novel tool for analyzing the neurological and motor functions underlying human movements and review some initial insights from our results about the relationships between muscle synergies, endpoint stiffness, and virtual trajectories (time series of EP). Our results suggest that (1) muscle synergies reflect an invariant balance in the co-activation of AA muscle pairs; (2) each synergy represents the basis for the radial, tangential, and null movements of the virtual trajectory in the polar coordinates centered on the specific joint at the base of the body; and (3) the alteration of muscle synergies (for example, due to spasticity or rigidity following neurological injury) results in significant distortion of endpoint stiffness and concomitant virtual trajectories. These results indicate that muscle synergies (i.e., the balance of muscle mechanical impedance) are essential for motor control.

Keywords: muscle synergy, motor primitives, mechanical impedance, reference frame, virtual trajectory, endpoint stiffness, electromyography

### INTRODUCTION

Voluntary movement requires sensorimotor transformation between extrinsic and intrinsic frames of reference (Kandel et al., 2012). To execute movement with specific endpoint characteristics, including aspects of kinematics, force, and impedance, the sensorimotor transformation may directly map the muscle space into the task space; the central nervous system (CNS) needs to regulate muscle activities to meet the endpoint's kinematic and kinetic specification. If the muscle space directly relates to the task space, the endpoint movement could be planned or predicted based on the reference frame in the muscle space in which the motor commands from the CNS to the muscles are encoded. The reference frame in the muscle space provides a framework to explain how humans plan, adjust, and achieve a desired endpoint movement when governing multiple muscles in the executing limb.

However, the neuromusculoskeletal system is neurologically and mechanically redundant. The inverse problem (i.e., movement planning) involves an infinite number of possible solutions to a given task. One hypothesis for solving this ill-posed problem is to exploit the stereotypical patterns of coordination, or muscle synergies. Synergies are classes of movement patterns that are functional groups of structural elements in the regulation and control of movement (Bernstein, 1967). The synergy hypothesis emphasizes that the CNS utilizes the functional structure at different motor levels (neurons, muscles, and joints) to simplify motor control. There is much evidence to show that the natural solution to the distribution problem results in highly robust endpoint's kinematics (Morasso, 1981; Lacquaniti et al., 1983; Flash and Hogan, 1985; Shadmehr and Mussa-Ivaldi, 1994) and kinetics (Hogan, 1985; Mussa-Ivaldi et al., 1985; Flash and Mussa-Ivaldi, 1990; Tsuji et al., 1995) features, called invariant characteristics (Zatsiorsky and Prilutsky, 2012). Motor invariance could provide a clue for understanding the mechanism underlying voluntary movements, because the CNS may impose or exploit these constraints to solve the degrees-of-freedom problem (Bernstein, 1967) essential for motor control.

However, it is an open question: are muscle synergies fundamental primitives or consequences of other primitives? Some researchers have considered motor synergies to be building blocks of movement (d'Avella et al., 2006; Latash, 2008; Cheung et al., 2009; Dominici et al., 2011; Bizzi and Cheung, 2013). However, other researchers have considered that at least some types of motor synergies are not primitives but composites of mechanical impedance (Hogan and Sternad, 2012).

While many aspects of motor control and coordination remain controversial, such as movement reference frame, motor redundancy, and motor primitives (neurological or mechanical origin of motor synergies), our objective in this paper is to provide some evidence supporting the concept that muscle mechanical impedance might provide key insights into unravel motor control intertwined relationships.

In our previous work, we reconsidered muscle synergies from a mechanical engineering aspect and associated them with the reference frame in the muscle space (Uno et al., 2014). The mathematical formulation was theoretically attractive because it suggested that muscle synergies were a function of co-activations by agonist–antagonist (AA) muscle pairs (i.e., composites of mechanical impedance). Moreover, the muscle synergies were viewed as invariant functional modules representing the reference frame in the polar coordinates centered on the specific joint (e.g., shoulder) at the base of the body. Thus, we hypothesized that muscle synergies are consequences of the balance of mechanical impedance, which represents the reference frame in the muscle space.

In this work, we examine our hypothesis from the viewpoint of motor control, learning, and recovery. If muscle synergies are primitives for motor control, learning, and recovery, it would be expected that common synergies are extracted across a variety of different tasks, different subjects, and different motor skills of the subjects. Also, the investigation of muscle synergies for a subject with neuromotor deficits would provide insight into the extent of muscle-synergy invariance, since the fundamental motor functions may be damaged by abnormal muscle tone, which is a common feature after neurological injury. In this study, we tested two experimental paradigms: (1) muscle synergies on motor adaptation and (2) muscle synergies on motor recovery. In our view, muscle synergies strongly relate to mechanical impedance. We also discuss endpoint stiffness and concomitant virtual trajectories in the context of muscle synergies.

### METHODS

### Apparatus

Multiple muscles in the human neuromuscular system are responsible for coordinating and regulating movement while negotiating within the dynamic environment. The establishment of a systematic framework to explain motor synergies, mechanical impedances, and virtual trajectories is a challenge to the comprehensive understanding of motor control and learning. Assuming that the investigation of multiple muscle activities would lead to a deeper understanding of the neural mechanism underlying voluntary movements, we developed a kinesiological analysis device that enables us to estimate those intrinsic motor characteristics from electromyography (EMG) signals during movement. **Figure 1** shows an overview of the system we call the "synergy analyzer*"*. The system consists of a display, a screen table, a chair with harnesses, an arm-support cart with low-friction ball wheels, a motion capture system, and an EMG measurement system.

The subjects sat in the chair with both shoulders fixed in the harnesses and performed voluntary arm movements while looking at the 65-inch display [or a 1.20 m × 0.86 m (width × height) screen table] in front of them. The upper limb was placed on the arm-support cart at shoulder height to eliminate the influence of gravity and restrict arm movement on the horizontal plane. In order to conceptualize the upper limb as a two-link structure, the wrist joint was fixed to the arm-support cart. During the subject's voluntary arm movements, kinematics and EMG signals were recorded synchronously. Each joint position (left shoulder, right shoulder, left or right elbow, and left or right hand) of the upper limbs was measured using an optical motion capture system with eight cameras (OptiTrack; NaturalPoint, Inc., Corvallis, OR, USA) at 100 Hz.

trajectories.

FIGURE 1 | Experimental setup. (A) Display; (B) screen table; (C) EMG signals; (D) muscle synergies [*u*R(*s*) (top left), *u*φ(*s*) (top right), and *u*φ×R(*s*) (bottom left)]; (E) endpoint-stiffness ellipse; (F) equilibrium point. The subject performed spiral or circle tracing with the non-dominant/dominant hand in a horizontal plane while monitoring a display showing the ideal trajectory. The EMG activities during movements were recorded to analyze muscle synergies, endpoint stiffness, and virtual suggested that muscle synergies were a function of co-activations by agonist–antagonist (AA) muscle pairs (i.e., composites of mechanical impedance). Moreover, the muscle synergies were viewed as invariant functional modules representing the reference frame in the polar coordinates centered on the specific joint (e.g., shoulder) at the base of the body. Thus, we hypothesized that muscle synergies are consequences of the balance of mechanical impedance, which

In this work, we examine our hypothesis from the viewpoint of motor control, learning, and recovery. If muscle synergies are primitives for motor control, learning, and recovery, it would be expected that common synergies are extracted across a variety of different tasks, different subjects, and different motor skills of the subjects. Also, the investigation of muscle synergies for a subject with neuromotor deficits would provide insight into the extent of muscle-synergy invariance, since the fundamental motor functions may be damaged by abnormal muscle tone, which is a common feature after neurological injury. In this study, we tested two experimental paradigms: (1) muscle synergies on motor adaptation and (2) muscle synergies on motor recovery. In our view, muscle synergies strongly relate to mechanical impedance. We also discuss endpoint stiffness and concomitant virtual trajec-

Multiple muscles in the human neuromuscular system are responsible for coordinating and regulating movement while negotiating within the dynamic environment. The establishment of a systematic framework to explain motor synergies, mechanical impedances, and virtual trajectories is a challenge to the comprehensive understanding of motor control and learning. Assuming that the investigation of multiple muscle activities would lead to a deeper understanding of the neural mechanism underlying voluntary movements, we developed a kinesiological analysis device that enables us to estimate those intrinsic motor characteristics from electromyography (EMG) signals during movement. **Figure 1** shows an overview of the system we call the "synergy analyzer*"*. The system consists of a display, a screen table, a chair with harnesses, an arm-support cart with low-friction ball wheels, a motion capture system, and an EMG

The subjects sat in the chair with both shoulders fixed in the harnesses and performed voluntary arm movements while looking at the 65-inch display [or a 1.20 m × 0.86 m (width × height) screen table] in front of them. The upper limb was placed on the arm-support cart at shoulder height to eliminate the influence of gravity and restrict arm movement on the horizontal plane. In order to conceptualize the upper limb as a two-link structure, the wrist joint was fixed to the arm-support cart. During the subject's voluntary arm movements, kinematics and EMG signals were recorded synchronously. Each joint position (left shoulder, right shoulder, left or right elbow, and left or right hand) of the upper limbs was measured using an optical motion capture system with eight cameras (OptiTrack; NaturalPoint, Inc., Corvallis, OR,

represents the reference frame in the muscle space.

tories in the context of muscle synergies.

METHODS

Apparatus

measurement system.

USA) at 100 Hz.

Electromyography signals of six upper limb muscles [deltoid posterior, deltoid anterior, triceps brachii (long head), biceps brachii, triceps brachii (lateral head), and brachioradialis] were measured with a multi-telemeter system (WEB-5000; Nihon Kohden Corp., Japan) at 1000 Hz. Surface electrodes were attached to the appropriate places on measured muscles as previously described (Criswell, 2010; Perotto, 2011), after cleansing the skin with alcohol (<10 kΩ). The obtained EMG signals were analyzed after the following procedures: bandpass filtering (10–450 Hz), fullwave rectification, smoothing, and normalization to maximum voluntary contraction (MVC), which was reported as percentage of MVC. We followed standard procedure to determine the MVC for each muscle (Hislop and Montgomery, 2007). The synergy analyzer then estimated the muscle synergies, endpoint stiffness, and virtual trajectories from the measured movement data, while superimposing these motor indices in real time (refresh rate, 10 Hz) onto the actual images captured from the top-view camera. The estimation results were provided to the subject on the display (or screen table) for use during biofeedback training.

### Experimental Paradigm

This study focused on the roles of muscle synergies, endpoint stiffness, and virtual trajectories during voluntary training and rehabilitation. To clarify the evolution of these motor characteristics, we performed two experiments.

#### Experiment 1: Motor Adaptation After Training

Eight young subjects (all males, 23 ± 1 years old, right-handed) volunteered for the first experiment. No subject reported any history of neuromuscular disease. The experiment was approved by the Institutional Review Board of Osaka University, and all subjects provided written informed consent before participation.

Each subject performed spiral tracing as fast as possible without touching the lines with his non-dominant (left) hand in a horizontal plane (**Figure 1**). The maximum radius of the spiral was 21 cm. The visual presentation of the ideal trajectory and current hand position were provided on a display in front of the subject; the ideal spiral trajectory had a spacing of 1.0 cm between lines on the display, which was equivalent to 3.5 cm in the task space. The center of the spiral in the task space was adjusted to correspond to each subject's hand position in a natural posture. The movement included 5.75 clockwise rotations from outside to inside. To become familiar with the procedure, the subjects performed 20 trials as practice before the first baseline measurement. The subject was then asked to perform the task 50 times per day for 8 days. On the first and last days, the kinematics and EMG signals during the task were measured to analyze the muscle synergies, endpoint stiffness, and virtual trajectories.

#### Experiment 2: Motor Recovery After Rehabilitation

Two elderly subjects, a healthy subject (male, 61 years old, righthanded) and a post-stroke subject (male, 74 years old, righthanded), volunteered for the second experiment. The experiment was approved by the Institutional Review Boards of Osaka University and Senri Chuo Hospital, and both subjects provided informed consent. The healthy subject was a control subject who was of the same generation as the other subject. The post-stroke subject was an acute-stage inpatient with mild-to-moderate rightside hemiplegia but was able to carry out verbal communication. The post-stroke subject performed the experiment twice, before and 2.5 months after rehabilitation.

Each subject performed circle tracing with his dominant (right) hand in a horizontal plane. Because the spiral tracing task was difficult for the post-stroke subject before rehabilitation, we selected a similar trajectory in a smaller circle (radius: 10 cm) so that the task would be easier for him. The post-stroke subject performed the task as fast as possible with his affected hand without any kind of assistance, while the healthy subject performed the task at slow speed (movement time: about 4 sec) to match his movements to those of the post-stroke subject. The kinematics and EMG signals during the task were measured to analyze the muscle synergies, endpoint stiffness, and virtual trajectories.

### Data Analysis AA Ratio and AA Sum

The human upper arm was modeled as a two-link structure with six muscles (**Figure 2**). We selected the four mono-articular muscles and two bi-articular muscles relevant to the shoulder and elbow movements in a horizontal plane. The chosen muscles were indexed as follows: deltoid posterior (*M*s,ext), deltoid anterior (*M*s,flex), triceps brachii (long head) (*M*se,ext), biceps brachii (*M*se,flex), triceps brachii (lateral head) (*M*e,ext), and brachioradialis (*M*e,flex). These six muscles comprise three pairs of AA muscles. The mono-articular muscle pair around the shoulder joint (*M*s,ext and *M*s,flex), bi-articular muscle pair around the shoulder and elbow joints (*M*se,ext and *M*se,flex), and mono-articular muscle pair around the elbow joint (*M*e,ext and *M*e,flex) are the fundamental functional units for coordinating and regulating the shoulder and elbow joint movements to control hand movement; each muscle pair is composed of two muscles that have opposite (i.e., agonist and antagonist) functions.

To characterize the motor functions of an AA muscle pair around the joint(s), *j*, we defined the following meta-parameters (the AA ratio, *r*j, and AA sum, *s*j) as the control variables:

$$r\_{\circ} = \frac{m\_{\text{j.ext}}}{m\_{\text{j.ext}} + m\_{\text{j.flex}}}, \left(j = \text{ s}, \text{ se, e}\right) \tag{1a}$$

$$s\_j = m\_{j, \text{cat}} + m\_{j, \text{flex}}, \left( j = \text{ s}, \text{ se, e} \right) \tag{1b}$$

where the subscript *j* indicates the joint(s) and corresponds to any one of the shoulder (s), shoulder and elbow (se), and elbow (e) joints; *m*j,ext and *m*j,flex are the EMG activities of the extensor and flexor muscles around the joint(s) *j*. **Table 1** lists the motor functions of these AA muscle pairs. The AA ratio contributes to the equilibrium position of the joint angle(s), and the AA sum contributes to the mechanical impedance of the joint(s). Details of the mathematical theory on the AA concept have been previously published (Ariga et al., 2012; Pham et al., 2014; Hirai et al., 2015).

#### Muscle Synergies

One hypothesis for the spatial and temporal control of limb movements with multiple muscles is the use of hierarchical coordination. In the previous section, we considered the coordination of agonist and antagonist muscles as the lowest level of coordination. This

TABLE 1 | Definitions and functions of agonist–antagonist (AA) ratio (r) and AA sum (s).

functions of the shoulder and elbow joint angles (θs, θe).


*EP, equilibrium point.*

section explains the next level of coordination, intra-limb coordination, and develops the AA concept under the model shown in **Figure 2**. To derive the relationship among the equilibrium points, AA ratios, and AA sums, we used the following assumptions: (1) each muscle can be described as a spring system whose elastic coefficient and natural length are adjusted according to an EMG signal; (2) the moment arm of each joint is equal and constant; and (3) the lengths of the upper arm (from shoulder joint to elbow joint) and forearm (from elbow to the center of wrist) are equal. In the mathematical sense, the assumption (1) means that the contractile force of a muscle *F*(*m*) can be expressed by

$$F(m) = K(m)(l - l\_0(m))\tag{2}$$

where *K*(*m*) is the muscle stiffness at EMG activity level *m*, and *l* and *l*0(*m*) are the muscle length and natural length of the muscle at EMG activity level *m*. *K*(*m*) and *l*0(*m*) are

and

$$d\_\circ(m) = \frac{C\_3}{K(m)} + C\_4 \tag{4}$$

*K m*( ) = − *C m*1 2 ( ) *C* (3)

where *C*1, *C*2, *C*3, and *C*4 are constant coefficients that represent the properties of the muscle. The details of our assumption with mathematical formulation have been published previously (Ariga et al., 2012; Hirai et al., 2015). Based on these assumptions, the displacement of the equilibrium-joint angles at the shoulder and elbow, **θEP** = (θs,EP, θe,EP)T , can be described with the following equation, using the AA ratios and the AA sums (Pham et al., 2014; Uno et al., 2014; Hirai et al., 2015):

$$\begin{aligned} & \left(\Theta\_{s,\text{EP}}\right) \\ &= C \left( \frac{\frac{s\_s s\_{sc}}{s\_s s\_{sc}} + s\_s s\_{sc}}{\frac{s\_s s\_{sc}}{s\_s s\_{sc}} + s\_s s\_{sc}} + \frac{s\_s s\_{sc}}{s\_{sc} s\_{sc} + s\_s s\_{sc}}} \cdot \frac{\frac{s\_s s\_{sc}}{s\_{sc}}}{\frac{s\_s s\_{sc}}{s\_s s\_{sc} + s\_{sc} s\_{sc}} + s\_s s\_{sc}}} \cdot \frac{\frac{-s\_s s\_{sc}}{s\_{sc} s\_{sc} + s\_{sc} s\_{sc}}}{\frac{s\_s s\_{sc}}{s\_{sc} s\_{sc} + s\_{sc} s\_{sc}} + s\_s s\_{sc}} \right) \\ & \times \left( \frac{r\_s - \frac{1}{2}}{r\_{sc} - \frac{1}{2}}\right) \\ & \times \left( \frac{1}{r\_c - \frac{1}{2}}\right) \end{aligned} \quad \begin{aligned} & \frac{-s\_s s\_{sc}}{s\_{sc} + s\_{sc} s\_{sc} + s\_{sc} s\_{sc}} \\ & \quad \frac{s\_s s\_{sc}}{s\_{sc} + s\_{sc} s\_{sc} + s\_{sc} s\_{sc}} \\ & \quad \frac{s\_s s\_{sc} + s\_{sc} s\_{sc}}{s\_{sc} + s\_{sc} s\_{sc} + s\_{sc} s\_{sc}} \end{aligned} \quad \begin{aligned} & \frac{-s\_s s\_{sc}}{s\_{sc} + s\_{sc} s\_{sc} + s\_{sc} s\_{sc}} \\ & \quad \frac{s\_s s\_{sc} + s\_{sc} s\_{sc}}{s\_{sc} + s\_{sc} s\_{sc} + s\_{sc} s\_{sc}} \\ & \quad \frac{s\_s s\_{sc}}{s\_{sc} + s\_{sc} s\_{sc}} + s\_{sc} s\_{sc}} \end{aligned}$$

(5)

where *C* is the coefficient determined by the muscle characteristics and the moment arm, *r* is the AA ratio vector (*r*s, *r*se, *r*e)T , *s* is the AA sum vector (*s*s, *s*se, *s*e)T , and *q*s(*s*) and *q*e(*s*) are defined as follows:

$$q\_s\left(\mathbf{s}\right) = \frac{1}{s\_s s\_{sc} + s\_c s\_{sc} + s\_s s\_c} \left(s\_s s\_{sc} + s\_s s\_c, s\_c s\_{sc}, -s\_c s\_{sc}\right)^\mathrm{T} \tag{6a}$$

$$q\_{\boldsymbol{\epsilon}}\left(\mathbf{s}\right) = \frac{1}{s\_{\boldsymbol{s}}s\_{\boldsymbol{s}\boldsymbol{\epsilon}} + s\_{\boldsymbol{\epsilon}}s\_{\boldsymbol{s}\boldsymbol{\epsilon}} + s\_{\boldsymbol{s}}s\_{\boldsymbol{\epsilon}}}\left(-s\_{\boldsymbol{s}}s\_{\boldsymbol{s}\boldsymbol{\epsilon}}, s\_{\boldsymbol{s}}s\_{\boldsymbol{s}\boldsymbol{\epsilon}}, s\_{\boldsymbol{s}\boldsymbol{\epsilon}}s\_{\boldsymbol{s}\boldsymbol{\epsilon}} + s\_{\boldsymbol{s}}s\_{\boldsymbol{\epsilon}}\right)^{\mathrm{T}} \tag{6b}$$

Note that *q*s(*s*) and *q*e(*s*) are composed of the AA sum only. As shown in equation (5), the AA ratio controls the equilibriumjoint angle linearly if *q*s(*s*) and *q*e(*s*) satisfy the condition of being a constant. However, one problem is motor redundancy: the dimension of the AA ratio space always exceeds the dimension of the joint space. The synergy hypothesis emphasizes the use of coordination in solving this ill-posed problem (Bernstein, 1967). We used this hypothesis to introduce a method for extracting the muscle synergies from the human musculoskeletal model. The essence of this technique is that the EP at the endpoint is described based on the polar coordinates system centered on the shoulder (**Figure 2**). The kinematics of the two degrees-of-freedom arm with the shoulder angle, θs, and elbow angle, θe, determine the unique endpoint position, *p* = (*R*, φ)T , in the polar coordinates:

$$\mathbf{p} = \begin{pmatrix} R \\ \Phi \end{pmatrix} = \begin{pmatrix} 2L\cos\frac{\left|\Theta\_{\epsilon}\right|}{2} \\ \pi - \left(\Theta\_{\epsilon} + \frac{\Theta\_{\epsilon}}{2}\right) \end{pmatrix} \tag{7}$$

where *L* is the length of the upper arm and forearm. By considering a small deviation of *p* and substituting Eq. (5) into Eq. (7), we can obtain the relationships among the endpoint EP, AA ratios, and AA sums:

$$\begin{aligned} \Delta \mathbf{p}\_{\rm FP} &= \mathbf{J}\_{\rm R\phi}(\boldsymbol{\Theta}) \cdot \boldsymbol{\Delta} \boldsymbol{\Theta}\_{\rm FP} \\ &= \begin{pmatrix} \mathbf{C}\_{\rm R} \left( \boldsymbol{\Theta}\_{\rm \epsilon} \right) & \mathbf{0} \\ \mathbf{0} & C\_{\rm \epsilon} \end{pmatrix} \begin{pmatrix} \boldsymbol{\mathsf{q}}\_{\rm \epsilon}^{\top} \left( \boldsymbol{s} \right) \\ \boldsymbol{\mathsf{q}}\_{\rm \epsilon} \left( \boldsymbol{s} \right) + \frac{\mathbf{q}\_{\rm \epsilon} \left( \boldsymbol{s} \right)}{2} \end{pmatrix} \begin{pmatrix} \boldsymbol{\Delta \mathsf{r}\_{\rm \epsilon}} \\ \boldsymbol{\Delta \mathsf{r}\_{\rm \epsilon}} \\ \boldsymbol{\Delta \mathsf{r}\_{\rm \epsilon}} \end{pmatrix} \\ &\approx \begin{pmatrix} \mathbf{C}\_{\rm R} & \mathbf{0} \\ \mathbf{0} & \mathbf{C}\_{\rm \epsilon} \end{pmatrix} \begin{pmatrix} \boldsymbol{\mathsf{q}}\_{\rm \epsilon}^{\top} \left( \boldsymbol{s} \right) \\ \boldsymbol{\mathsf{q}}\_{\rm \epsilon} \left( \boldsymbol{s} \right) \end{pmatrix} \begin{pmatrix} \boldsymbol{\Delta \mathsf{r}\_{\rm \epsilon}} \\ \boldsymbol{\Delta \mathsf{r}\_{\rm \epsilon}} \\ \boldsymbol{\Delta \mathsf{r}\_{\rm \epsilon}} \end{pmatrix} \end{aligned} \tag{8}$$

where *J*<sup>R</sup> T s e φ φ θ θ ( ) ( , ) ( , ) **<sup>q</sup>** <sup>=</sup> <sup>∂</sup> ∂ *<sup>R</sup>* is a Jacobian matrix that relates the joint

space to the task space described in the polar coordinates; *C*R(θe) and *C*φ are coefficients determined by the muscle characteristics, moment arm of each joint, and upper arm/forearm length *L*. Moreover, *C C L* R e <sup>e</sup> θ <sup>θ</sup> ( ) <sup>φ</sup> <sup>=</sup> sin <sup>2</sup> can be approximated as a constant *C C* R R <sup>e</sup> ( ) = ( ) θ when the elbow is flexed enough during the movement, where θe is the mean angle of the elbow joint. A remarkable feature of our method is that the formulation is based on the polar coordinates. Owing to the good linear approximation between the task space described in the polar coordinates and the joint space (Mitsuda et al., 1997), the above equation is satisfied in a relatively broad range of work space. Equation (8) indicates that the displacement of the endpoint EP in the polar coordinates can be estimated by projecting the three-dimensional AA ratio vector Δ*r*[=(Δ*r*s, Δ*r*se, Δ*r*e)T] onto the two-dimensional

subspace composed of *C*R*q*e(*s*) and *C*<sup>φ</sup> *q s q s* s <sup>e</sup> ( ) + ( ) 2 . Based on this informative relationship, we defined the muscle-synergy

vectors as

$$\mu\_{\mathbb{R}}\left(\mathbf{s}\right) = \frac{q\_{\boldsymbol{\varepsilon}}(\mathbf{s})}{\left|q\_{\boldsymbol{\varepsilon}}(\mathbf{s})\right|}\tag{9a}$$

$$\mu\_{\phi}(s) = \frac{q\_{\ast}(s) + \frac{q\_{\ast}(s)}{2}}{\left| q\_{\ast}(s) + \frac{q\_{\ast}(s)}{2} \right|} \tag{9b}$$

$$\mu\_{\mathbb{R}\times\phi} \left( s \right) = \frac{\mu\_{\mathbb{R}} \left( s \right) \times \mu\_{\phi} \left( s \right)}{\left| \mu\_{\mathbb{R}} \left( s \right) \times \mu\_{\phi} \left( s \right) \right|} \tag{9c}$$

where *u*R(*s*) and *u*φ(*s*) indicate the unit vectors for the distributions of the AA ratio vector in the radial and tangential directions, and *u*<sup>R</sup>×φ(*s*) is defined as the unit vector in the null direction (i.e., the zero space). Muscle synergy in the null direction is not considered to directly contribute to the movement of the endpoint EP but is felt to regulate the endpoint stiffness (Uno et al., 2014). These synergy vectors are the bases for the endpoint EP's movement in the radial, tangential, and null directions. Note that muscle synergy is a function of the AA sum only. In our definition, the muscle synergy represents the balance of mechanical impedance by co-activations of AA muscles and plays a role as the reference frame in the muscle space for the endpoint EP movement. It is worth noting that muscle synergy becomes constant when *q*s(*s*) and *q*e(*s*) satisfy the condition of being constants. This assumption is not trivial, but the validity of this assumption (i.e., musclesynergy invariance) is confirmed in the later sections.

#### Endpoint Stiffness

Endpoint stiffness is another index of mechanical impedance, while muscle synergy indicates the balance of mechanical impedance by co-activations of the AA muscles. Assuming a linear relationship between single muscle activation and the corresponding muscle stiffness, the joint stiffness *K*j(*s*) in the static condition can be expressed as the following function of AA sums:

$$\mathbf{K}\_{\circ}(\mathbf{s}) = k\_{\circ} \begin{pmatrix} s\_{\circ} + s\_{\circ \circ} & s\_{\circ \circ} \\ s\_{\circ \circ} & s\_{\circ} + s\_{\circ \circ} \end{pmatrix} \tag{10}$$

where *k*j N·m/rad is a gain constant to convert the AA sums to joint stiffness. Under dynamic conditions, such as the presence of a force load, an additional term depending on the hand position and hand force is required (McIntyre et al., 1996). However, we ignored this effect for simplicity, assuming that hand force was minimal in our task. Then, endpoint stiffness *K*e(*s*, **θ**) can be obtained as follows:

$$\mathbf{K}\_{\rm c}(\mathbf{s}, \boldsymbol{\Theta}) = \left(\boldsymbol{J}\_{\rm xy}^{\rm T}(\boldsymbol{\Theta})\right)^{-1} \cdot \mathbf{K}\_{\rm j}(\mathbf{s}) \cdot \boldsymbol{J}\_{\rm xy}^{-1}(\boldsymbol{\Theta}) \tag{11}$$

where *J*xy T s e ( ) , , **<sup>q</sup>** <sup>=</sup> ( ) ( ) ∂ ∂ θ θ *x y* is the Jacobian matrix that associates

the joint space with the task space in Cartesian coordinates. The endpoint-stiffness matrix can be graphically represented as a stiffness ellipse calculated based on the eigenvalues and eigenvectors of the matrix (Hogan, 1985; Mussa-Ivaldi et al., 1985; Flash and Mussa-Ivaldi, 1990).

#### Virtual Trajectories

By projecting the deviation vector of the AA ratio onto the muscle-synergy vectors, we can obtain the change in the EP at the endpoint. We defined the deviation of synergy activation coefficients (Δ*w*R, Δ*w*φ, and Δ*w*<sup>R</sup>×φ) as the inner products of the muscle-synergy vectors [*u*R(*s*), *u*φ(*s*), and *u*R×φ(*s*)] and the deviation vector of the AA ratio ∆ = *r r* ( ) − *r* , where *r* is the AA ratio at the basis position.

$$
\Delta \boldsymbol{\omega}\_{\rm R} = \boldsymbol{\mu}\_{\rm R}^{\rm T}(\mathbf{s}) \cdot \Delta \mathbf{r} = \boldsymbol{\mu}\_{\rm R}^{\rm T}(\mathbf{s}) \cdot (\boldsymbol{r} - \overline{\boldsymbol{r}}) \tag{12a}
$$

$$
\Delta \boldsymbol{\omega}\_{\boldsymbol{\phi}} = \boldsymbol{\mu}\_{\boldsymbol{\phi}}^{\mathrm{T}}(\mathbf{s}) \cdot \Delta \boldsymbol{\mathbf{r}} = \boldsymbol{\mu}\_{\boldsymbol{\phi}}^{\mathrm{T}}(\mathbf{s}) \cdot \left(\boldsymbol{\mathbf{r}} - \overline{\boldsymbol{\mathbf{r}}}\right) \tag{12b}
$$

$$
\Delta \boldsymbol{\omega}\_{\mathbb{R} \times \boldsymbol{\phi}} = \boldsymbol{\mu}\_{\mathbb{R} \times \boldsymbol{\phi}}^{\mathrm{T}}(\mathbf{s}) \cdot \Delta \boldsymbol{\tau} = \boldsymbol{\mu}\_{\mathbb{R} \times \boldsymbol{\phi}}^{\mathrm{T}}(\mathbf{s}) \cdot (\boldsymbol{\tau} - \boldsymbol{\overline{\tau}}) \tag{12c}
$$

The deviation of the endpoint EP is then expressed as

$$
\begin{pmatrix}
\Delta \mathcal{R}\_{\text{EP}} \\
\Delta \phi\_{\text{EP}}
\end{pmatrix} = \begin{pmatrix}
\mathcal{\alpha}\_{\text{R}} \Delta \mathcal{W}\_{\text{R}} \\
\mathcal{\alpha}\_{\phi} \Delta \mathcal{W}\_{\phi}
\end{pmatrix} \tag{13}
$$

where αR and αφ are the gain constants to adjust the scale of the muscle-synergy activation coefficients to the scale of the virtual trajectory, and αR and αφ correspond to *C*R and *C*φ in Eq. (8). The displacement of the endpoint EP in the polar coordinates, *p*EP = (*R*EP, φEP)T , can be calculated from a linear combination of muscle-synergy activation coefficients as

$$\mathbf{p}\_{\rm EP} = \begin{pmatrix} \mathbf{R}\_{\rm EP} \\ \boldsymbol{\Phi}\_{\rm EP} \end{pmatrix} = \begin{pmatrix} \overline{\boldsymbol{R}}\_{\rm EP} + \boldsymbol{\Delta}\boldsymbol{R}\_{\rm EP} \\ \overline{\boldsymbol{\Phi}}\_{\rm EP} + \boldsymbol{\Delta}\boldsymbol{\Phi}\_{\rm EP} \end{pmatrix} = \begin{pmatrix} \overline{\boldsymbol{R}}\_{\rm EP} + \boldsymbol{\alpha}\_{\rm R}\boldsymbol{\Delta}\boldsymbol{\omega}\_{\rm R} \\ \overline{\boldsymbol{\Phi}}\_{\rm EP} + \boldsymbol{\alpha}\_{\boldsymbol{\phi}}\boldsymbol{\Delta}\boldsymbol{\omega}\_{\boldsymbol{\phi}} \end{pmatrix} \tag{14}$$

where *R*EP and φEP are the polar coordinates of the endpoint EP at the basis position. In the rest condition at the basis position, we assumed that the actual position and EP position at the endpoint became equal. Finally, the endpoint EP in the Cartesian coordinates can be obtained by the following transformation:

$$
\begin{pmatrix} \boldsymbol{\omega}\_{\rm EP} \\ \boldsymbol{\nu}\_{\rm EP} \end{pmatrix} = \begin{pmatrix} \boldsymbol{R}\_{\rm EP} \cos \phi\_{\rm EP} \\ \boldsymbol{R}\_{\rm EP} \sin \phi\_{\rm EP} \end{pmatrix} \tag{15}
$$

The virtual trajectory is a time series and is a succession of EPs at the endpoint. The EP can be represented as a point in the configuration space of muscle synergies, and the virtual trajectory can be identified by tracking the point over time in the muscle-synergy space. Reference control based on EPs or virtual trajectories, that is to say, the EP hypothesis (Feldman, 1966, 1986; Feldman et al., 1990; Feldman and Latash, 2005), has been an influential hypothesis for motor control. Our formulation may give an insight to unify the different ideas of muscle synergies, endpoint stiffness, and virtual trajectories.

### RESULTS

### Experiment 1: Synergy Analysis of Motor Adaptation

The spiral test is a reliable measure of accuracy and speed in upper limb movements; it is usually used in rehabilitation as a qualitative assessment to provide feedback to patients with coordination disorders, such as cerebellar ataxia or Parkinson's disease (Verkerk et al., 1990). We adopted this measure as an index to reflect the evolution of movement in the non-dominant hand through voluntary training, although the subjects were neurologically and physically intact. The subjects were scored on the time spent to complete the task, with a penalty time added for touching or crossing the lines; the score was defined as the sum of the time spent (from start to goal), the number of times the spiral line was touched multiplied by 3, and the number of times the spiral line was crossed multiplied by 5. The kinematics-assessment score greatly improved for the eight subjects through 8 days of training. The average score for all subjects was 62.1 ± 23.3 (mean ± SD) on the first day and 23.3 ± 11.8 on the last day, respectively, indicating the enhancement of motor performance.

**Figure 3** shows a typical AA ratio and AA sum before and after training for one subject (Subject #1). The AA ratio is an explanatory variable ranging from 0 to 1, and the AA sum is an explanatory variable ranging from 0 to 2. The AA ratio and AA sum indicate the degree of extension of the equilibrium-joint angles and of the increase of joint stiffness, respectively. Note that both variables vary with time because they are calculated from EMG signals during movement.

**Figure 4** shows the change in muscle synergies for the eight subjects before and after training, using the method described in the previous section. In each graph, the left, central, and right groups of the three-bar set (red, green, and blue) illustrate the muscle synergy in the radial direction [*u*R(*s*)], tangential direction [*u*φ(*s*)], and null direction [*u*<sup>R</sup>×φ(*s*)], respectively. The three colored bars in each muscle synergy represent the element values of the muscle-synergy vector, and each value quantifies the contribution of AA muscle activities to the shoulder, shoulder and elbow, and elbow joint movement, respectively. For a summary of the mean changes and SDs of muscle synergies, see **Table 2**. **Table 3** illustrates the inner-product (IP) values between muscle-synergy vectors computed from EMG signals in Experiment 1, indicating the similarity of muscle synergies in both inter-individual and intra-individual variations.

**Figure 5** shows typical endpoint stiffness before and after voluntary training (the first and last days of training) for Subject #1. The endpoint-stiffness ellipses during movement were compared between corresponding hand positions. **Figure 6** shows

typical actual and virtual trajectories before and after voluntary training for Subject #1. The red and green circles are the start and goal points, and the arrow indicates the direction in which each trajectory progresses. **Figure 7** shows the actual and virtual trajectories in the radial and tangential directions, which correspond to those trajectories in **Figure 6**. **Figure 7** also shows the change in movement time, indicating significant improvement in motor performance. The observed evolution characteristics in these figures are representative of those for all eight subjects.

### Experiment 2: Synergy Analysis of Motor Recovery

The functional independence measure (FIM) score is a widely used scale of disability severity that quantifies the impact of impairment on the performance of daily activities (Granger et al., 1986; Carr and Shepherd, 2010). The FIM score (maximum score: 126) of the post-stroke subject in this study was 44 points before rehabilitation and 67 points after 2.5 months of rehabilitation. These scores indicate that the subject's motor function improved through therapist-based exercise in rehabilitation. In agreement with the FIM score's change, the average movement time for the post-stroke subject in our task improved from about 6 s to about 4 s before and after rehabilitation; the average movement time for the healthy subject was about 4 s.

Since obvious recovery was observed in the post-stroke subject, we then compared the motor indices of muscle synergies, endpoint stiffness, and virtual trajectories, which characterize the coordination and regulation of multiple muscle activities before and after rehabilitation. **Figure 8** shows the changes in the AA ratios and AA sums for the post-stroke subject before and after 2.5 months of rehabilitation, as well as the changes in the AA ratios and AA sums for the same-generation healthy subject. **Figure 9** shows the changes in muscle synergies, endpoint stiffness, and virtual trajectories, which can be estimated by the proposed algorithm with the AA ratio and AA sum. The actual trajectory was also plotted on the graph of the virtual trajectory in **Figure 9** as one of the indices of motor recovery, although significant change was not observed. Details of the mean changes and SDs of muscle synergies are summarized in **Table 4**. **Table 5** compares the IP values between muscle-synergy vectors for different variations in Experiment 2: inter-individual variations, intra-individual variations, and intra-task variations.

# DISCUSSION

### Muscle Synergies as Reference Frames in Muscle Space

Focusing on the coordination among activities of AA muscles, here we discuss the relationships among the muscle synergies, endpoint stiffness, and virtual trajectories. To our knowledge, muscle synergy is a coordination index defined as a function of co-activations of AA muscles. It is a composite unit associated with mechanical impedance and is also a functional module representing the reference axis in the polar coordinate system for the displacement of an EP in the task space. In short, muscle synergies represent the reference frame in the muscle space

FIGURE 4 | Muscle synergies from the first and last days of voluntary training for eight subjects. The three-bar sets represent the muscle synergies in the radial direction (*u*R), tangential direction (*u*φ), and null direction (*u*<sup>R</sup>×φ). Each colored bar in the muscle synergy indicates the contribution of agonist–antagonist muscle activities to the shoulder (red), shoulder and elbow (green), and elbow joint (blue) movement. Muscle synergies exhibited similar patterns regarding both intra- and inter-individual variations during training, demonstrating the existence of common and invariant reference frames for motor representation that are independent of the level of motor learning.

that may be used in motor planning for endpoint control. Endpoint stiffness is another index of mechanical impedance; the balance of the co-activations of AA muscles determines not only the muscle synergies but also the shape and orientation of the endpoint-stiffness ellipse. The virtual trajectory is a time sequence of EPs at the endpoint. The EP can be represented as a point in the configuration space of muscle synergies, and the virtual trajectory can be identified by tracking the point over time in the muscle-synergy space. The mathematical relationships among these inter-winding motors were derived from the physical modeling of the musculoskeletal structure with multiple AA muscles.

### Physics-Based Approach to Muscle-Synergy Extraction

Our approach may provide a new perspective in understanding motor control and learning. Motor synergies are usually extracted by applying statistical techniques to explanatory variables, such as joint angles and EMG signals, which may be the set of motor states resulting from CNS commands based on fewer motor modules. The results of factor decomposition are, however, not necessarily interpretable with such explanatory variables even though the factors successfully reduce the dimensionality of movement. Therefore, the physical meaning of motor synergies is not clear in most cases, in particular, in the case of muscle synergy because EMG includes information on both kinematic and kinetic aspects; muscles work for both joint displacement and joint impedance. EMG is phenomenologically interpreted as an electrical signal originating from the depolarization of the muscle fibers. However, the relationship between muscle activation and movement is not fully understood. We assumed that the CNS controls the equilibrium state and mechanical impedance for multi-joint movements by changing specific neurophysiological parameters (Feldman et al., 1990) and that EMG consequently reflects at least these two pieces of information. Then, the statistical analysis of original EMG signals may result in yielding the makeshift factors, which are usually task-dependent and/or subject-dependent.

To gain insight into the physical meaning of muscle synergies, this study examined the AA concept using the following explanatory variables: the AA ratio, which is related to the equilibriumjoint angle, and the AA sum, which is associated with the joint stiffness. Since the AA concept originates from the control of a robotic system with antagonistic pneumatic artificial muscles, muscle synergy extracted under the AA concept has a clear physical meaning. Similar ideas for the control of AA muscles can be found in the field of neuroscience [e.g., the ratio of the tensions of AA muscles (Lestienne et al., 1981; Bizzi et al., 1984), mechanical impedance and co-activation of AA muscles (Hogan, 1984), and the control of the EP and level of co-contraction for joint movement (Feldman et al., 1990)]. However, our AA concept is strictly different from these. The AA concept can be regarded as another form of the EP hypothesis (Feldman, 1966, 1986; Feldman et al., 1990; Feldman and Latash, 2005) and can be extended to the novel concept of EP-based synergies (Pham et al., 2014; Uno et al., 2014; Hirai et al., 2015).


#### TABLE 2 | Element values of muscle synergies before and after voluntary training on spiral tracing (A) first training day and (B) last training day.

TABLE 3 | Inner-product values between muscle synergies (experiment 1).


It is also worth noting that the muscle synergy derived from our approach is composed of AA sums only (see Eqs 9a–c). This formulation means that the muscle synergies themselves are not motor primitives but consequences of modulation of mechanical impedance, which may be one of the motor primitives. We believe that our findings are in line with the idea of dynamic primitives, which Hogan and Sternad recently argued (Hogan and Sternad, 2012). Nevertheless, muscle synergies may play roles as functional modules – that is, a reference frame in muscle space. Muscle synergies represented as the balance of mechanical impedance may be called "kinetic synergies." Thus, our study is categorized as a physics-based approach and is clearly different from most studies, which are categorized as statistical-based approaches (d'Avella et al., 2006; Cheung et al., 2009, 2012; Dominici et al., 2011; Bizzi and Cheung, 2013; Roh et al., 2013) for extracting muscle synergies. Although we do not discuss null synergy much in this paper, the idea of null synergy, which the statistical approach cannot extract from the data, is informative. The physics-based approach is a powerful way to reverse engineer the control mechanism underlying the neuromusculoskeletal system in the dynamic environment. For more details on muscle synergies based on the AA concept, refer also to our recent publications (Koba et al., 2014; Oku et al., 2014, 2015; Uno et al., 2014).

### Muscle Synergies, Endpoint Stiffness, and Virtual Trajectories in Motor Adaptation

How do muscle synergies or the balance of mechanical impedance affect motor enhancement if they represent a reference frame in muscle space? We measured the similarity of muscle-synergy vectors among subjects before and after voluntary training, based on the IP value of the corresponding two muscle-synergy vectors (**Table 3**). The results revealed that the muscle synergies before

tends to be far smaller than that in the radial direction.

and after training exhibited similar patterns regarding both intra-individual variations and inter-individual variations. It is also notable that the muscle synergies were held almost constant despite being calculated from time-varying AA sums. The SDs of muscle synergies were sufficiently small for all subjects (**Figure 4**,

**Table 2**). These results demonstrate the invariant kinetic characteristics that the CNS may exploit for movement planning.

It is mathematically evident that the muscle synergies represent the bases of the polar coordinates. The results suggest that the invariant reference frame for motor representation is encoded into time-varying biological signals (**Figure 3**) and that the reference frame is not only common among normal subjects but also independent of the level of learning. Thus, we hypothesize that the muscle synergies may be functional modules to link the muscle space to the task space and that they may be a coordinate system for motor control. Moreover, the invariance of muscle synergies may be related to the stable characteristics of endpoint stiffness since the muscle synergies represent the balance of mechanical impedance by co-activations of AA muscles. In our task, the shape and orientation of the endpoint-stiffness ellipse did not change much during training; however, its size changed (**Figure 5**). The orientation of the major axis of the ellipse tended to keep tilting toward the direction connecting the shoulder and the endpoint (i.e., the radial direction). This indicates that endpoint stiffness in the tangential direction always tends to be far smaller than that in the radial direction.

In contrast to these hard-wired characteristics in the CNS, virtual trajectories showed drastic changes with motor enhancement. The virtual trajectories were organized from disordered patterns into smooth spiral patterns that rotated in the opposite direction of the actual trajectories (**Figure 6**). As shown in **Figure 7**, in both cases before and after training, the virtual trajectories showed oscillating movements that preceded the actual trajectories with similar sequences showing gradually decreasing amplitudes. However, the different phase relationship between the actual and virtual trajectories emerged in each direction after training. In the radial direction, high endpoint stiffness caused endpoint EP movement with about a 0° phase shift. In the tangential direction, the far smaller endpoint stiffness caused endpoint EP movement with about a 180° phase shift. The coupling of these directional mechanical impedances yielded a counterintuitive observation, i.e., the opposite rotation of virtual trajectories. This phenomenon can be observed in fast movements. The finding indicates that

joints, respectively. Each data set was normalized with respect to a period of movement time. The changes in the AA ratio and AA sum indicate that rehabilitation resulted in change in the control of EP and stiffness around each joint. In particular, the AA sum was improved although it was still far from the level observed from the healthy subject. (Note the different range of the graphs for AA sums before and after rehabilitation.)

the CNS requires an internal model (Gomi and Kawato, 1997) to achieve dynamic compensation in the process of motor control and learning.

### Muscle Synergies, Endpoint Stiffness, and Virtual Trajectories in Motor Recovery

In contrast to our results from Experiment 1, significant changes (IP < 0.9) were observed in the muscle synergies for the poststroke subject before and after rehabilitation (**Table 5**). This observation relates to the disrupted inter-joint coordination commonly observed in arm movements after stroke. In our view, the alteration of muscle synergies indicates a breach in the basis for motor control, and it may influence the reference frame essential for sensorimotor transformation. In the case of the post-stroke subject, the abnormal co-activation of bi-articular AA muscles (green bars in **Figure 9**) yielded different muscle synergies, especially in the tangential direction. The SDs of abnormal muscle synergies were then within a tolerance (small) level and, thus, the abnormal muscle synergies could also be regarded as the invariant bases for the polar coordinates. In other words, the reference frame in the muscle space was held by other coordinated muscles.

Before rehabilitation, the post-stroke subject may have exploited motor redundancy to regulate multiple muscles in order to manage his impairments following neurological injury, and he may therefore have achieved an invariant coordination different from that of normal subjects. Since the muscle synergies represented a balance of mechanical impedance by co-activation of AA muscles, the changes in muscle synergies significantly affected the endpoint-stiffness characteristics. The endpointstiffness ellipse of the post-stroke subject before rehabilitation was elongated, and the orientation of its major axis indicated a more clockwise rotation than that observed in the healthy subject. The primary cause may have been the hypertonicity of the bi-articular

Frontiers in Bioengineering and Biotechnology | www.frontiersin.org



#### TABLE 5 | Inner-product values between muscle synergies (experiment 2).


muscles; an increase in the co-activation of bi-articular muscles (*s*se) tends to modify the eigenvalues and eigenvectors of the joint stiffness matrix *K*j(*s*), enlarging the stiffness ellipse and rotating its major axis in a clockwise direction. These characteristics of the endpoint-stiffness ellipse may explain the typical dysfunctions: flexor synergy (characterized by simultaneous shoulder abduction and elbow flexion) and extensor synergy (characterized by simultaneous shoulder adduction and elbow extension) (Brunnstrom, 1970). Moreover, alteration of muscle synergies results in significant distortion of not only endpoint stiffness but also concomitant virtual trajectories. The virtual trajectory moved within a limited range and tended to move in the fixed direction of the minor axis of the endpoint-stiffness ellipse.

However, rehabilitation caused a fundamental change in motor control in the post-stroke subject. The muscle synergies after rehabilitation exhibited similar patterns (IP > 0.9) to those of the healthy subject (**Table 5**). This result indicates that the poststroke subject regained the normal patterns of muscle synergies in the process of recovery. Moreover, these patterns were similar to the patterns extracted from the subjects during the fast-spiraltracing task (IP > 0.9). This case study provides only preliminary evidence for common muscle synergies across a variety of different tasks, different subjects, and different motor skills of the subjects. The endpoint stiffness also recovered, along with the recovery of the corresponding muscle synergies. The shape of the endpoint-stiffness ellipse was shortened, and the orientation of the major axis rotated slightly counter-clockwise. These results provide evidence that the post-stroke subject was on the course of recovery. This analysis was based on data after 2.5 months of rehabilitation, and it is supposed that the post-stroke subject has the potential to recover motor function with further rehabilitation. Interestingly, in the process of recovery, the muscle synergies regained normal patterns earlier than did the virtual trajectories. The virtual trajectory did not fully recover after 2.5 months of rehabilitation. This indicates that the muscle synergies playing a role as the reference frame in the muscle space are fundamental for motor control. This is a reasonable conclusion because the virtual trajectories are defined by the configuration space of muscle synergies.

### FUTURE DIRECTIONS FOR MOTOR REHABILITATION

Imbalance of intra-limb coordination is one of the causes of motor impairment. If our hypothesis is true, muscle synergy (i.e., the balance of mechanical impedance) should be carefully taken into account when selecting methods of rehabilitation. The importance of impedance control has been discussed frequently, but care is required in its application because impedance control can result in either improved or worsened outcomes based on the way it is used. Muscle synergy may be an index for exploring the appropriate assistance application of impedance control. It would also provide an additional measure to clinical assessment such as the Fugl–Meyer assessment and others. Again, our hypothesis is that muscle synergies, the balance of mechanical impedance, represent a reference frame in the muscle space. This study tested our hypothesis to confirm the feasibility of the practical use of muscle synergy, such as in the assessment, diagnosis, and intervention planning for stroke rehabilitation.

The results of this study can be summarized as follows: (1) muscle synergy is an invariant balance of muscle mechanical impedance; (2) muscle synergies represent a reference frame in the muscle space; and (3) the common muscle synergies were found among different tasks (fast-spiral-tracing with the nondominant hand and slow-circle-tracing with the dominant hand), different subjects from different generations (i.e., from subjects aged in their 20s to subjects aged in their 70s), and different levels of motor skill (beginner, experienced, and in a patient after rehabilitation). Further data collection and analysis from different situations will strengthen our hypothesis; it would be useful to discuss the relationship with other internal representations such as eye-centered, head-centered, and world-centered reference frames. Our future work includes the development of novel approaches for robotic therapy, particularly for the lower extremity. Robotic therapy, especially for lower extremity function, is currently in the early stage of development. The next generation of robot-aided neuro-rehabilitation requires correctly assessing the effect of interventions and collecting clinical evidence to develop an efficacious intervention. Stroke rehabilitation requires sensorimotor coordination. By combining the methods of *synergy assessment* and *robotic therapy*, we will develop a novel robotic intervention, test, and validate it in the framework of muscle synergies.

In conclusion, this study showed evidence that the muscle synergies play a central role in motor representation and internal model formation. Muscle synergies are not primitives but

### REFERENCES


functional modules of mechanical impedance. The balance of muscle mechanical impedance is essential for motor control, learning, and recovery.

### ACKNOWLEDGMENTS

We would like to thank Drs. Neville Hogan and Dagmar Sternad for insightful comments on our study. This work was supported by JSPS Grant-in-aid for Scientific Research (KAKENHI), Grant Numbers 24360096 and 15H03949. HK was supported in part by the NIH Grant R01HD069776-03.


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Copyright © 2015 Hirai, Miyazaki, Naritomi, Koba, Oku, Uno, Uemura, Nishi, Kageyama and Krebs. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

*Shota Hagio1,2 and Motoki Kouzaki2 \**

*<sup>1</sup> Japan Society for the Promotion of Science, Tokyo, Japan, 2 Laboratory of Neurophysiology, Graduate School of Human and Environmental Studies, Kyoto University, Kyoto, Japan*

Redundancy in the musculoskeletal system was supposed to be simplified by muscle synergies, which modularly organize muscles. To clarify the underlying mechanisms of motor control using muscle synergies, it is important to examine the spatiotemporal contribution of muscle synergies in the task space. In this study, we quantified the mechanical contribution of muscle synergies as considering spatiotemporal correlation between the activation of muscle synergies and endpoint force fluctuations. Subjects performed isometric force generation in the three-dimensional force space. The muscle-weighting vectors of muscle synergies and their activation traces across different trials were extracted from electromyogram data using decomposing technique. We then estimated mechanical contribution of muscle synergies across each trial based on cross-correlation analysis. The contributing vectors were averaged for all trials, and the averaging was defined as action direction (AD) of muscle synergies. As a result, we extracted approximately five muscle synergies. The ADs of muscle synergies mainly depended on the anatomical functions of their weighting muscles. Furthermore, the AD of each muscle indicated the synchronous activation of muscles, which composed of the same muscle synergy. These results provide the spatiotemporal characteristics of muscle synergies as neural basis.

#### *Edited by:*

*Zhi-Hong Mao, University of Pittsburgh, USA*

#### *Reviewed by:*

*Arturo Forner-Cordero, University of São Paulo, Brazil Nitin Sharma, University of Pittsburgh, USA*

*\*Correspondence: Motoki Kouzaki kouzaki.motoki.4x@kyoto-u.ac.jp*

#### *Specialty section:*

*This article was submitted to Bionics and Biomimetics, a section of the journal Frontiers in Bioengineering and Biotechnology*

> *Received: 20 April 2015 Accepted: 30 October 2015 Published: 13 November 2015*

#### *Citation:*

*Hagio S and Kouzaki M (2015) Action Direction of Muscle Synergies in Three-Dimensional Force Space. Front. Bioeng. Biotechnol. 3:187. doi: 10.3389/fbioe.2015.00187*

Keywords: muscle activity, electromyogram, non-negative matrix factorization, force fluctuations, mechanical pulling direction, cross-correlation analysis

### INTRODUCTION

The fundamental problem in motor control is how the central nervous system (CNS) controls the immense number of variables in the musculoskeletal system (Bernstein, 1967). To simplify the redundancy, the CNS may modularly organize the muscles through the hard-wired neural circuit referred to as muscle synergy (Tresch et al., 1999; d'Avella et al., 2003; Ting and Macpherson, 2005; Hagio and Kouzaki, 2014). To clarify the underlying mechanisms in motor control based on muscle synergies, it is important to examine how muscle synergies are represented and modulated in the neural circuitry (motor level) and to investigate whether muscle synergies function as the actuator to produce movement (task level) (Alessandro et al., 2013). Many researchers statistically calculated task-dependent muscle synergies from electromyogram (EMG) dataset in motor level (d'Avella et al., 2006, 2008; Torres-Oviedo and Ting, 2007, 2010; Hug et al., 2010; Roh et al., 2012, 2013; Hagio et al., 2015), whereas model-based approaches showed the low dimensionality in the task level (Berniker et al., 2009; Neptune et al., 2009; Allen and Neptune, 2012). To uniformly identify the relationship of the low dimensionality between motor and task levels, it is necessary to quantify the net contribution of individual muscle synergies in the task space.

Several approaches were conducted to demonstrate correlations between muscle synergy recruitment levels and biomechanical outputs. During perturbed standing, functional muscle synergies were calculated, which reflect the mapping of the endpoint force vector (Torres-Oviedo et al., 2006; Chvatal et al., 2011). Previous research estimated the mechanical contribution of each muscle synergy (called as synergy-to-force mapping) by assuming the linear relationship between EMG (further linearly decomposed into muscle vectors of muscle synergies) and endpoint force in isometric condition (Berger and d'Avella, 2014). These techniques were advantageous to quantify the force vector produced by each muscle synergy in the force space. However, the force vectors did not contain the temporal contribution of muscle synergies, which is important property to regard muscle synergies as neural basis. Although our previous study directly compared the spatiotemporal correlation between the activation coefficients of muscle synergies and endpoint force fluctuations during voluntary isometric conditions, demonstrating the significant correlation between them (Hagio and Kouzaki, 2015), the mechanical contribution of muscle synergies in the task space was not estimated. An appropriate approach has been taken using *EMG-weighted averaging* (EWA) method (Kutch et al., 2010; Imagawa et al., 2013). This was formulated as a non-invasive technique instead of the spike-triggered averaging (STA), i.e., a well-established method to extract the force associated with single motor unit (SMU) contractions, based on the hypothesis that surface EMG is indeed analogous to a superposition of SMU action potentials and its cross-correlation with endpoint force should produce the equivalent of an average spike-triggered force averaged across multiple motor units (Kutch et al., 2010). In this study, we developed this technique to evaluate the action direction (AD) of muscle synergies, which represented the net contribution of individual muscle synergies in the three-dimensional endpoint force. It should be noted that we assumed the neural basis of muscle synergies: the estimated activation of muscle synergies represents the summation of the individual basis constructing muscle synergies, which might have been regarded as spinal interneuron in the previous studies (Hart and Giszter, 2010; Overduin et al., 2014).

In the muscle synergy hypothesis, the primary problem is still whether the CNS actually modulates muscle synergies in the neural circuit. Many empirical findings showed the neural basis of muscle synergies by examining the relationship between statistically calculated muscle synergies and activation of spinal interneuron in frogs (Hart and Giszter, 2010) or activation of motor cortical neurons in rhesus macaques (Overduin et al., 2014). However, it is reported that low dimensionality as statistically calculated muscle synergies might be due to task or biomechanical constraints (Kutch and Valero-Cuevas, 2012). Accordingly, the problem remains controversial (Bizzi and Cheung, 2013). In the concept of the synchronous muscle synergy, which is discriminated from other muscle synergy models, such as timevarying muscle synergy (d'Avella et al., 2003), muscles organized in the same muscle synergies may be synchronously activated. Therefore, cross-correlation analysis will lead to the correlation between the activation of the target muscle and endpoint force, which are generated by the muscles grouped in the same muscle synergy; the AD of a muscle will reflect the mechanical contribution not only of the muscle but also of the other muscles synchronously activated due to a muscle synergy as hard-wired modular controller. Hence, the examination of the relationship among the ADs of muscles weighted by the same extracted muscle synergy will make it possible to approach identifying the neuronal basis of muscle synergies. In this study, we examined the presence of muscle synergies by calculating AD of each muscle.

Consequently, the main purpose of the present study was to quantify the contribution of muscle synergies in the task space. To this end, we estimated the AD of muscle synergies during multi-directional force generation in three-dimensional force space. Furthermore, we verified the hypothesis of the neural basis of muscle synergies by examining the relationship between the ADs of individual muscles and the estimated structure of muscle synergies which the relevant muscles belong to. This study demonstrates the relationship of low dimensionality due to muscle synergies between in the motor and task levels.

### MATERIALS AND METHODS

### Subjects

Five male subjects voluntarily participated in this study. Their mean (±SD) age, height, and body mass were 23.8 ± 1.1 years, 173.9 ± 3.8 cm, and 67.4 ± 6.5 kg, respectively. All subjects were healthy, had no history of any neurological disorder, and had corrected-to-normal vision. Subjects provided written informed consent to participate in the study after receiving a detailed explanation of the purposes, potential benefits, and risks associated with participation. All procedures used in this study were in accordance with the Declaration of Helsinki and approved by the Committee for Human Experimentation at the Graduate School of Human and Environmental Studies, Kyoto University.

### Experimental Setup

Each subject laid on their left side on a bed with the right leg supported horizontally by a sling (**Figure 1A**; Hagio and Kouzaki, 2014, 2015). The knee and hip joints were applied with the angles of 90° from full extension. Isometric endpoint forces surrounding the right ankle were produced for a total of 10 s at 2 different intensities (20 and 40 N) in each of 32 different directions in the three-dimensional force space (**Figure 1C**); in total, 64 trials were randomly conducted with a rest period of 30 s between each trial and of 10 min between 2 blocks which is composed of 32 trials, respectively. The directions were equally distributed in 30° increments along horizontal plane to cover the anterior side on this plane. On sagittal plane, force was applied from six directions (0°, 30°, 60°, 90°, 120°, and 135°) considering the knee extension torque and/or hip joint torque (Hof, 2001). We then measured isometric endpoint forces, which were composed of three force vectors, *F*x, *F*y, and *F*z referring to hip abduction–adduction, knee extension–flexion, and hip flexion–extension movements, respectively (**Figures 1B,D**), using a tri-axial force transducer (LSM-B-500NSA1, Kyowa, Tokyo, Japan) attached to the subject's

bed with the right leg supported horizontally by a sling. Visual feedback of produced and target forces was displayed to the subject on a computer screen (Hagio and Kouzaki, 2014). (B) Using a tri-axial force transducer attached to the subject's right ankle, three-dimensional forces, *F*x, *F*y, and *F*z, were measured (Hagio and Kouzaki, 2014). The positive values of three axes are corresponding to hip abduction (+*F*x), knee extension (+*F*y), and hip flexion (+ *F*z) movement directions, respectively. (C) Thirty-two desired target force directions (blue dots: 20 N, red dots: 40 N) in the three-dimensional force space. (D) The force trajectories across each target direction in the force intensity of 40 N for a representative subject.

right ankle (Kouzaki et al., 2002; Hagio et al., 2012). The resultant force vector was calculated based on the three force vectors, i.e., *F* = *F*<sup>x</sup> + *F*<sup>y</sup> + *F*z; the resultant vector length represented the intensity of the force, i.e., | | *F* = *FFF* . x y <sup>z</sup> <sup>222</sup> + + In each trial, the subjects viewed the produced force vector and the desired force vector as a target on a visual display.

### Electromyography

Surface EMGs were recorded from eight muscles mainly activated in the task space in this study: the rectus femoris (RF), vastus lateralis (VL), vastus medialis obliquus (VMO), vastus medialis longus (VML), vastus intermedius (VI), sartorius (SR), adductor longus (AL), and gluteus medius (GM) (Hagio and Kouzaki, 2014, 2015). EMGs were recorded using bipolar Ag–AgCl electrodes. Each electrode had a diameter of 5 mm, and the inter-electrode distance was 10 mm. We used a small inter-electrode distance to prevent cross-talk between neighboring muscles (Imagawa et al., 2013). A reference electrode was placed on the lateral epicondyle of femur. The EMG signals were amplified (MEG-6116M, Nihon-Kohden, Tokyo, Japan) and band-pass filtered between 5 and 1000 Hz. All electrical signals were stored with a sampling frequency of 2000 Hz on the hard disk of a personal computer using a 16-bit analog-to-digital converter (PowerLab/16SP; AD Instruments, Sydney, NSW, Australia). The raw EMG traces were high-pass filtered at 35 Hz using a zero-phase-lag fourth-order Butterworth filter, after which they were demeaned, rectified, and low-pass filtered at 40 Hz (Chvatal et al., 2011). The filtered traces were then divided into 100 time bins per second and averaged across each bin (i.e., resampled at 100 Hz). The same procedures were conducted across each corresponding rest period, and the difference between the two traces served as the net EMG (Hagio and Kouzaki, 2015).

For the extraction of muscle synergies, the muscle activity data for each muscle were assembled to form an EMG data matrix. We first constructed the EMG data matrix (*M*), which consisted of temporal sequence for 10 s of each muscle activity in each trial, i.e., 8 muscles × 64,000 variables (32 directions × 2 force levels × 10 s × 100 samples). The EMG values of each muscle were normalized to the maximum value for all of the muscles across all desired directions such that each value was between 0 and 1. Then, each muscle data vector was normalized to have unit variance to ensure the activity in all muscles was equally weighted.

### Extraction of Muscle Synergies

We extracted muscle synergies from the data matrix of the EMG recordings (*M*) using non-negative matrix factorization (NMF) (Lee and Seung, 1999; Tresch et al., 1999; Hagio and Kouzaki, 2014, 2015; Hagio et al., 2015) as following equation:

$$M = \sum\_{i=1}^{N} W\_i C\_i + \mathfrak{e} \quad \left(W\_i \ge 0, C\_i \ge 0\right),$$

where *W*i represents the contribution of each muscle to synergy i, and an individual muscle may contribute to multiple synergies. The composition of the muscle synergies does not change among the conditions, but each synergy is multiplied by a scalar activation coefficient (*C*i) that changes among conditions: the column of *C*i consisted of 64,000 variables (32 directions × 2 force levels × 10 s × 100 samples). ɛ is the reconstructed error. The synergy weighting and activation coefficient matrices were normalized such that the individual muscle-weighting vector was the unit vector.

To select the smallest number of muscle synergies (*N*syn) that resulted in an adequate reconstruction of the muscle responses, we extracted between 1 and 8 muscle-weighting matrices of muscle synergies and activation coefficient matrices from the EMG data matrices that were obtained from each subject. We subsequently verified the goodness-of-fit between the original (*M*) and recon-

structed *M W* r i*C*<sup>i</sup> i N <sup>=</sup> <sup>=</sup> ∑1 data matrices as the amount of total

variability explained (*R*<sup>2</sup> ) depending on the number of muscle synergies (*N*). We used a linear regression procedure (d'Avella et al., 2006) to determine *N* value after which the *R*<sup>2</sup> curve is approximately straight as assuming that the increase of *R*<sup>2</sup> with adding *N* value is due to noise-based variation. We performed linear regression on the entire *R*<sup>2</sup> curve and progressively removed the smallest *N* value from the regression interval. We then compared the mean square residual errors of the different regression lines and selected the least *N* value (*N*syn), a mean squared error in the regression line from which to the maximum *N* value was <10<sup>−</sup><sup>4</sup> . For *N*syn muscle synergies, both muscle-weighting and activation-coefficient matrices were defined.

For the verification that the extracted muscle synergies depend on not the methodological but physiological factors, it is needed to judge whether the resultant dimensionality in the muscle activation space using the NMF analysis was lower than the chance level or not. To this end, EMG data matrix was constructed using the shuffled EMG data across each muscle. It should be noted that these shuffled EMG data contained the same value, range, and variance for each muscle although the relationships between muscle activations were removed. We then calculated *R*<sup>2</sup> value between the original and reconstructed EMG data matrices across each of one to eight muscle synergies.

## Grouping of Similar Muscle Synergies Across Subjects

Functional sorting of the muscle synergies across each subject was initially performed by grouping muscle synergies based on the values of cosine similarity (*r* > 0.78; *p* < 0.01) to that of an arbitrary reference subject using an iterative process (Hagio and Kouzaki, 2014, 2015). If two synergies in one subject were assigned to the same synergy group, we defined a pair of synergies with the highest correlation as the same group of synergies. Subsequently, an averaged set of similar muscle synergies for all subjects were computed, and the similarity between the averaged muscle synergies and each synergy grouped across the subjects was quantified.

### Evaluating Action Direction of Muscle Synergies and Muscles

We estimated the three-dimensional AD of muscle synergies and muscles by developing EWA method (Kutch et al., 2010; Imagawa et al., 2013). **Figure 2** provides a diagram of how the method operates. The EWA is based on a cross-correlation of EMGs and force signals. Such analysis was performed over an approximately steady period of force fluctuations lasting 10 s out of the time course used in prior analysis. We used a series of estimated activation coefficients of each muscle synergy (*C*i; i = 1, 2, … , *N*) along with three force components (*F*x, *F*y, and *F*z) for cross-correlation analysis (**Figure 3A**). For the estimation of ADs of muscles, cross-correlation analysis was performed between the processed surface EMGs from individual muscles and each of the corresponding three force components. Each correlation coefficient was first quantified temporally and spatially based on a time lag from 0 to 200 ms, during which the traces reached its peak magnitude (**Figure 3B**). We used the time lag, on which the most peak magnitude of the three was estimated, to define the time-to-peak and used the corresponding time lag to define the net correlation coefficient of remaining components. According to the correlation coefficients of each component, the force vector in the three-dimensional space was determined across each trial (**Figure 3C**). Then, we defined the AD of the muscle synergy or muscle, which was the averaging of force vectors for all selected trials (see detail below) after the correlation coefficients underlying the force vector were transformed with Fisher's *Z* transformation (Fisher, 1934).

To verify the physiological validity of the ADs, we considered the electromechanical delay (EMD) of each muscle. The EMD was referred to as the time lag between EMG and mechanical force response (Cavanagh and Komi, 1979; Norman and Komi, 1979), corresponding to the time-to-peak of cross-correlation in this study. Furthermore, to increase a validity of this technique, we adopted the trials, which were comprised in three time bins around the peak time bin of the histograms across each muscle, for estimating force vectors (see Results). If the same peak time bins were observed in histograms, we selected the time bin, which was close to 100 ms. In the case of muscle synergies, we determined the correlation coefficients of the time-to-peak value between 50 and 150 ms,

FIGURE 2 | Block diagram to estimate action direction of a muscle synergy. Muscle weightings and activation coefficients of muscle synergies were first extracted from processed electromyogram (EMG) data for all trials using non-negative matrix factorization (NMF). The cross-correlation analysis was then performed between the traces of three force components and activation coefficient of a muscle synergy recorded and estimated in each trial to determine each correlation coefficient and the time-to-peak. If the correlation was physiologically valid, the correlation vector was plotted in three-dimensional space as a force vector. These procedures (surrounded by a dashed line) were repeated for all trials. Finally, we estimated the action direction of the muscle synergy, which was the averaging of force vectors after the correlation coefficients underlying the force vector were transformed with Fisher's *Z* transformation. The details were described in the Section "Materials and Methods."

which indeed reflected the EMD of muscles constructing the muscle synergies, based on the EMD of lower limb muscles in the previous study (Vos et al., 1990) and in this study (see Result).

### Methodological Identification of Action Direction

To validate the analysis for the estimation of ADs, we performed methodological identifications. We verified that the distribution of force vectors as a result of correlation coefficients was not due to a secondary product of the methodology but due to a physiological factor, i.e., the relationship between the muscle activation and endpoint force. To this end, we calculated force vectors with the same procedure as estimating ADs of muscle synergies (or muscles), using the three force components and shuffled activation traces of muscle synergies (or EMG data) in which temporal sequences were shuffled across each muscle synergy (or muscle) (**Figure 4**). The force vectors, which time-to-peak value was physiologically meaningful, i.e., between 50 and 150 ms based on the previously calculated EMD (Vos et al., 1990), were adopted. We then quantified the distribution of the force vectors as a resultant vector length [ *R r* =|| ||; norm of the force vector averaged for each force vector (*r*), i.e., length of AD vector (Fisher, 1995)]. This procedure was repeated 100 times using bootstrapping to resample each shuffled activation trace of muscle synergy (EMG data) (Efron, 1993). We then estimated 95% bootstrap confidence intervals for the overall resultant vector length. If a resultant vector length calculated by actual dataset was out of the 95% confidence interval, the distribution of the force vectors was not determined by chance but included physiological information.

### RESULTS

### Directional Tuning of EMG Activity

**Figure 5** shows the muscle activations across each target direction in a representative subject. The activation of each muscle was broadly and specifically tuned with three-dimensional force direction. RF and VML were predominantly activated for between forward (+*F*y) and upward (+*F*z) force directions, which required knee extension and hip flexion torques, whereas VL, VMO, and VI were mainly activated forward (+*F*y) and close to downward (−*F*z) directions. It should be noted that the net knee extension torque, which does not involve the hip flexion or extension torques, was biased toward this direction on the force space. Hence, these mono-articular knee extensors produce the net knee extension torque. In the case of SR, AL, and GM, they generated hip flexion, adduction, and abduction torques, respectively.

### Muscle Synergy

In this study, we extracted five or six muscle synergies which accounted for 92.9 ± 2.75% of the total data variability (*R*<sup>2</sup> ) in the five subjects, and the *R*<sup>2</sup> value for same number of synergies were definitely higher than the case of shuffled dataset across each muscle (**Figure 6**: top). Additionally, the data were sufficiently reconstructed across each muscle and each target direction, as determined by *R*<sup>2</sup> averaged for all muscles and all directions: 92.3 ± 2.86 and 91.2 ± 4.24%, respectively (**Figure 6**: third and bottom). **Figures 7A,B** show five extracted muscle synergies and their activation coefficients across each target direction in a representative subject, respectively. The synergy *W*1, which was mainly constructed by mono-articular knee extensors (VL, VMO, and VI), was activated in forward (+*F*y) and downward (−*F*z) directions, i.e., the range around net knee extension direction, and around medial direction (−*F*x). The synergy W2, which contained RF, VML, and SR, was dominant for forward (+*F*y) and upward (+*F*z) directions generated by both knee extension and hip flexion torques and was also broadly activated in medial (−*F*x) and lateral (+*F*x) directions. The synergy *W*3 was mainly composed of SR, and activated around upward direction (+*F*z) produced by hip flexion torque. The synergy *W*4 having GM dominantly contributed to

component yielded the traces of correlation coefficients peaking at time lag (between EMGs constructing activation coefficients of the muscle synergy and force) about 100 ms. The time-to-peak is shown as a vertical line (see detail in Section "Materials and Methods"). ×, the point in which the most peak magnitude of the correlation coefficients was observed. (C) Force vector was determined as a unit vector based on the correlation coefficients of three components in threedimensional force space. The correlation coefficient was described as the radius of a plot on the surface of unit sphere. These procedures were corresponding to a block diagram surrounded by a dashed line in Figure 2.

lateral force (+*F*x), which was generated by hip abduction torque. The synergy *W*5, which was constructed by RF, VL, VI, and AL, was activated around medial direction (−*F*x) produced by hip adduction torque.

### Action Direction of Muscle Synergy

**Figure 7C** represents the AD of each muscle synergy (red), which was defined as the averaging for individual force vectors (blue) resulting from cross-correlation analysis. We verified the significance in the distribution of the force vectors across each muscle synergy (*p* < 0.05; see detail in Section "Materials and Methods"). The ADs were approximately corresponding to the activation range of the muscle synergies: the synergy *W*1 contributed around net knee extension torque [(0.215, 0.680, −0.701)]; (*F*x, *F*y, *F*z)]; the synergy *W*2 was dominant for knee extension and hip flexion torques (−0.093, 0.850, 0.519); the synergy *W*3 was mainly activated for hip flexion torque (−0.180, 0.507, 0.843); the synergy *W*4 dominated hip abduction torque (0.999, 0.013, −0.048); and the synergy *W*5 generated knee extension, hip flexion, and hip adduction torques (−0.567, 0.818, 0.103). **Figure 7D** shows the time-to-peak histograms of each synergy at a time lag of 0 to 200 ms, which represents the time lag between the activation onset of muscle synergy and the onset of mechanical force response. Each muscle synergy had the peak time bin around a time lag of 100 ms [117.2, 103.3, 98.5, 115.2, and 101.4 (ms); mean value in *W*1–5, respectively).

The muscle weighting and ADs of muscle synergies for all subjects are shown in **Figures 8** and **9**, respectively. The synergy *W*1, which was mainly constructed by mono-articular knee extensors, i.e., VL, VMO, VML, and VI, was extracted from all subjects with high similarity (*r* > 0.936). The ADs of the synergy *W*1 were distributed in the similar direction generated by knee extension torque in four of five subjects, whereas the AD of one subject (ID4) denoted more medial direction (−*F*x) than the others. The synergy *W*2 weighting RF and VML was observed in all subjects (*r* > 0.940). The AD of the synergy *W*<sup>2</sup> was similar across each subject, which directional force was produced by the combination of hip flexion and knee extension torques. The synergy *W*3 dominantly composed of SR was included in all subjects (*r* > 0.998). The ADs of the synergy *W*<sup>3</sup> were consistently directed approximately hip flexion direction (+*F*z) for all subjects. The synergy *W*4, which was constructed by the combination of GM and other muscles, was similar across all subjects (*r* > 0.953). The ADs of the synergy *W*4 mainly denoted the lateral direction (+*F*x), but in one subject (ID3) the AD was biased to the hip flexion direction (+*F*z) because of the influence of SR. The synergies *W*5 and *W*6, which commonly contained AL, were extracted 2 of 5 subjects, respectively (*r* > 0.965 and *r* > 0.976, respectively). The ADs of these synergies were distributed in the medial direction (−*F*x). The subject-specific synergies were observed in two subjects, the ADs of which depended on the composition of these muscle synergies. These

results demonstrated the robustness and specificity of muscle synergies and their ADs across each subject.

### Action Direction of Muscles on the 3-D Force Space

As illustrated in **Figure 10**, we estimated the AD of each muscle (red) to validate the hypothesis that neural-based muscle synergy would synchronously control organized muscles. We verified the significance in the distribution of the force vectors across each muscle (*p* < 0.05; see detail in Section "Materials and Methods"). These ADs were approximately corresponding to the range of the muscle activation direction. However, they represented the characteristics of each muscle more distinctly. Although VL, VMO, VML, and VI, mono-articular knee extensors, are generally assumed as functionally similar muscles, the ADs were different among them. The AD of VL denoted the force direction generated by both knee extension and small hip flexion torques [(−0.113, 0.990, −0.086); (*F*x, *F*y, *F*z)]. This contribution to the off-axis hip joint torque would be due to the synchronous activation with RF, which generates knee extension and hip flexion torques, in the synergy *W*5. The similar result was observed in VI (0.274, 0.958, −0.088). On the other hand, the AD of VMO was around the direction produced only by the knee extension torque (0.166, 0.680, −0.714) because VMO was contained only in the synergy *W*1, which AD (0.215, 0.680, −0.701) was similar to the AD of VMO, and not associated with the bi-articular muscle, RF. The AD of VML was in the force direction produced by both knee extension and hip flexion torques (−0.245, 0.936, 0.252) and was strongly similar to the AD of RF (−0.288, 0.871, 0.398), both of which was included in the synergy *W*2. Interestingly, the ADs of the knee extensors also directed either medial or lateral side, indicating the synchronous activations of the hip adductor, AL, or hip abductor, GM, through the synergy *W*5 and *W*4, respectively. The ADs of the SR, AL, and GM were also affected by the synchronization with the different muscles in the same muscle synergies [SR (0.002, 0.298, 0.955), AL (−0.470, 0.790, 0.393), and GM (0.977, 0.164, 0.135)]. These results suggest that the AD of a muscle reflect the anatomical function of the muscle and different muscles, which are synchronously activated through the muscle synergies.

To verify the physiological validity of the ADs, we calculated the EMD of each muscle. **Figure 11** shows the EMD histograms of each muscle for all analyzed trials at a time lag of 0 to 200 ms. Each muscle had the peak time bin around a time lag of 100 ms [102.5, 103.7, 104.8, 107.4, 103.9, 104.7, 101.9, and 108.5 (ms); RF, VL, VMO, VML, VI, SR, AL, and GM, respectively). The average time-to-peak values were similar to the values of the previous study (Vos et al., 1990), indicating that the estimated ADs in this study would be physiologically valid.

### DISCUSSION

The primary goal of this study was to quantify the mechanical contribution of muscle synergies in the task space. To this end, we estimated the AD of lower limb muscle synergies during isometric force-maintaining tasks on three-dimensional force space. The five or six muscle synergies were identified across each subject. The ADs of these muscle synergies approximately denoted the direction based on the anatomical function of the weighted muscles in the task space. Furthermore, the AD of each muscle reflected each anatomical function and a synchronous contribution with different muscles, which belonged to the same muscle synergies.

### Action Direction of Muscle Synergies

Many researchers have long studied low-dimensional organization of the spinal motor system and resulting movements. Low dimensionality in the task space, which was induced by the stimulation of spinal interneuron, was first observed in frog and rat as force field (Giszter et al., 1993; Saltiel et al., 1998). Modular organization of muscle activations (so-called muscle synergy) was then statistically estimated (Tresch et al., 1999), which would produce the low dimensionality in the task space. A few studies showed the relationship of the low dimensionality between task level and motor level. Novel method was conducted to estimate synergy-to-force-mapping, which represented the linear relationship between the activation of muscle synergies and endpoint force in the isometric condition (Berger et al., 2013; Berger and d'Avella, 2014), whereas synergy-to-force-mapping vector did not contain the contribution of unmesurable muscles and did not completely explain the net contribution of muscle synergies in the task space. Different approach calculated functional

muscle synergies from the data matrix, which contained muscle activation matrix and corresponding endpoint force matrix, by reducing their dimension together using NMF algorithm (Torres-Oviedo et al., 2006; Chvatal et al., 2011). The counter evidence of the muscle synergy hypothesis, however, showed that the estimated low-dimensionality in muscle activation was due to the biomechanics of the limb, which constrains musculotendon length changes (Kutch and Valero-Cuevas, 2012). Thus, the constraints of limb geometry relevant to task space can lead to the low dimensionality in the measured EMG activity. Therefore, the identification of neural-based muscle synergies will require examining not only the spatial contributions of muscle synergies in the task space but also temporal contributions, which include both movement-based and neural-based fluctuations (Hagio and Kouzaki, 2015). For this reason, the previous studies only evaluating the spatial contribution of muscle synergies could not provide the direct evidence whether the estimated contribution in the task space was arose from neural-based muscle synergies (Torres-Oviedo et al., 2006; Chvatal et al., 2011; Berger and d'Avella, 2014). In this study, we could directly estimated the net contribution of muscle synergies in the task space while considering temporal correlation between the activation of muscle synergies and endpoint force in the basis of the physiological criteria, i.e., time delay from the onset of the muscle synergy activation to the resulting force (indeed, this delay was between the onset of the muscle activation constructing muscle synergies and the force) (**Figure 7**). Our method in the basis of the previous technique (Kutch et al., 2010) made it possible to regard muscle synergies as neural basis and to directly quantify the spatiotemporal contribution of muscle synergies to the endpoint force.

Variability was observed across each force vector constructing AD of muscle synergy (**Figure 7C**; blue dots). The possible reason of this variability was due to synchronous activation with the other muscle synergies. In the methodology, the previous study showed that synchronization of motor units having different pulling directions distorts the estimate of the pulling direction by STA (Kutch et al., 2007). On the other hand, in the physiological aspect, the merging of muscle synergies was observed depending on the force-generating capability of muscles, which might result from the simultaneous recruitment of a few different muscle synergies (Hagio and Kouzaki, 2014). In the basis of the fact, the AD would be estimated by the correlation between a target muscle synergy and endpoint force, which was generated by the combination of the target muscle synergy and the synchronously activated muscle synergies. The different possible reason of this variability attributed the mechanical property of motor units. If the neural basis of muscle synergies exists as spinal interneuron (Hart and Giszter, 2010; Overduin et al., 2014), these interneurons control individual motor units having a broad range of the pulling direction (Thomas et al., 1986, 1990). This fact suggests that the mechanical contribution of muscle synergies was varied depending on the recruited motor units, which were activated according to a neural property, such as the Henneman's size principle: if the neural input from spinal interneuron to motor units was increased, motor units are recruited in turn from the smallest to the largest. This variability could make it possible for flexible force generations in a broad range of the task space by the combination of a small number of muscle synergies (Roh et al., 2012; Hagio and Kouzaki, 2014). These results also suggest that the ADs of muscle synergy defined in this study represent

line on *R*<sup>2</sup> curve from that number of muscle synergy to the maximum was computed. We selected the least number of muscle synergies (red circle), which MSE was <10−<sup>4</sup> (red dash line). Third: *R*<sup>2</sup> value across each muscle. Each line represents the *R*<sup>2</sup> value of the certain number of muscle synergies. Bottom: *R*<sup>2</sup> value across each of 32 target directions. Each line represents the *R*<sup>2</sup> value of the certain number of muscle synergies. Syn, synergy.

the average of variability of pulling direction of muscle synergies arose from a range of the pulling direction of motor units. Therefore, not simple combinations of ADs of different muscle synergies but flexible modulation within a muscle synergy might determine the produced endpoint forces.

### Action Direction of Muscles

The second effort in this study was to provide evidence that muscle synergies were of neural origin. To this end, we hypothesized that the AD of each muscle reflects the mechanical contribution of different muscles, which belong to the same muscle synergy, based on the consideration that if muscles are synchronously activated by the muscle synergies, cross-correlation analysis leads to the correlation between the activation of the target muscle and the endpoint force generated by the combination of these muscles. Indeed, the ADs of knee extensors (VL, VMO, VML, and VI) were different from each other depending on the muscle synergies, which these muscles belong to, despite their similar anatomical function (**Figure 10**). The results indicated that the muscles spanning different joints, such as bi-articular, RF, AL, and GM, affected the ADs of these muscles. The previous studies conducted the novel method focusing on the synchronous recruitment of each muscle through muscle synergies and showed the low-dimensional structure in the EMG activity (Krouchev et al., 2006; Drew et al., 2008; Krouchev and Drew, 2013). The extracted clusters, however, were relatively more than the estimated muscle synergies using decomposing technique, such as NMF (Krouchev et al., 2006). The method and idea in the present study could demonstrate the synchronous recruitment of muscles due to muscle synergies extracted by NMF. Moreover, because the method focused on both high and low force fluctuations during constant force

analysis, representing the time lag between the onset of activation of muscle synergy and force responses (exactly, the delay between the onset of activation in muscle level and the force responses).

generation, the correlation might reflect not the co-contraction of muscles but the synchronization of motor units constructing the measured EMG signals, which have the innervation from the same muscle synergy. This temporal property of muscle synergies provides the provided evidence that muscle synergies are of neural origin.

The estimation of muscle ADs also provided the EMD of each muscle, i.e., the time lag between the EMG and mechanical force response, which peak of the distribution was different across each muscle. Muscle synergies were composed of any muscles, which EMDs were variable. This difference among EMDs apparently confounds a motor control because the mechanical responses induced by descending neural input to muscle synergy may be out of alignment among muscles. Each muscle synergy, however, had roughly constant peak time lags between the activation of muscle synergy and force responses (**Figure 7D**). This result implies that the motor units having the similar EMDs in a muscle or in more different muscles compose a muscle synergy. Therefore, muscle

represents cosine similarities between the averaged muscle synergies estimated from the initial sorting and each original synergy grouped across each subject (see Materials and Methods). The synergies across each subject were grouped into six groups (*W*1–6) and two subject-specific muscle synergies (last row; gray and orange).

synergies are constructed considering the complexity in the misalignment of EMDs and achieve the accurate force generations. In the different scheme, such temporal lag between muscles within the same synergy was previously observed as "time-varying synergies," which has a fixed temporal profile (d'Avella et al., 2003), suggesting that this delay consider the difference of EMDs among muscles.

Furthermore, the difference of the muscle ADs among functionally similar muscles, especially between VML and the other vasti muscles (VL, VMO, and VI) (**Figure 10**), might reflect not only the modularity due to muscle synergies but also the inherent relationship between muscles and force responses based on the intrinsic characteristics in the musculoskeletal system. It is known that the EMG activities between VML and VL were different because of the discordancy of physiological parameters, such as physiological cross-sectional area (PCSA) and pennation angle of muscle fiber (Akima et al., 2000, 2001; Ward et al., 2009; Watanabe and Akima, 2011) or contribution to torque (Zhang et al., 2003). Moreover, this result could reflect the divergence of relationship between these muscles and the bi-articular muscle, RF, suggesting the stronger association between RF and VML than the other knee extensors. On the other hand, the ADs of VL and VI were mainly distributed in the same area. As illustrated in **Figure 7A**, the synergy *W*1 mainly consisted of the mono-articular knee extensors, in which the weightings of VL and VI were similar to each other. It is reported that they are fused at posterolateral side (Willan et al., 1990) or have relatively equivalent physiological parameters, such as PCSA (Akima et al., 2000, 2001). Hence, this result reflects the morphological and physiological similarity between VL and VI. Additionally, it is generally accepted that the

principal function of VMO is to control tracking of the patella by overcoming the lateral forces imposed by the other vasti muscles. This fact led us to the speculation that VMO does not have the specific AD. This fact was also ensured by the incoherent timeto-peak value of VMO (**Figure 11**). The calculated AD, which directed the net knee extension directions, might reflect the association between VMO and VL, which applies lateral-directed forces to the patella.

Furthermore, the AD of VI and VMO also directed to the lateral, whereas RF, VL, and VML contributed to medial force. The similar result was previously reported and suggested that balanced off-axis torques and forces are necessary for appropriate three-dimensional patellar tracking and tibiofemoral movement, and different quadriceps components need to be coordinated to generate appropriate off-axis and extension torque around knee joint (Zhang et al., 2003). Therefore, the ADs in the three-dimensional force space reflected such complicated relationships of quadriceps muscles.

### Existence of Hard-Wired Muscle Synergies

The primary problem in the muscle synergy hypothesis is whether a muscle synergy is a hard-wired neural system. Many researchers addressed the problem in some empirical studies. Hart and Giszter (2010) showed that activation of spinal interneurons in frogs was related to statistically calculated muscle synergies rather than individual muscles, indicating the neural-based structure of a muscle synergy as a spinal interneuron (Hart and Giszter, 2010). In addition, the connectivity between motor cortical neurons and muscle synergies was demonstrated by testing the similarity between statistically extracted muscle synergies evoked by intracortical microstimulation and hand movements in rhesus macaques (Overduin et al., 2012, 2014).

FIGURE 10 | Action directions of muscles. The ADs (red) and force vectors (blue) across each muscle. The length of each vector was uniformed with their unit vectors, and the correlation coefficient was described as the radius of each plot distributed on the surface of unit sphere. Data shown are for all subjects and selected trials (see detail in Section "Materials and Methods"). The positive values of three axes are corresponding to hip abduction (*F*x), knee extension (*F*y), and hip flexion (*F*z) movement directions, respectively.

FIGURE 11 | Electromechanical delay (EMD). Histograms of time-to-peak value across each muscle referred to as the time lag between the onset of EMG and mechanical force responses. Data shown are for all trials and all subjects.

In humans, a virtual surgery technique that rearranged muscle architecture demonstrated a hard-wired modularity in a neural circuit by testing the prediction that modularity due to muscle synergies interfered the adaptation to perturbations that are incompatible with the muscle synergies (Berger and d'Avella, 2014). However, a low-dimensional structure as statistically calculated muscle synergies might be a secondary product due to some decomposing techniques, such as NMF or independent component analysis (Bell and Sejnowski, 1995; Hart and Giszter, 2004), in task or biomechanical constraints (Kutch and Valero-Cuevas, 2012). Accordingly, to reveal the problem whether a muscle synergy is of neural origin, it is important to focus on not only the spatial but also the temporal structure of muscle synergies, which comprehended information of both a mechanical and a neural property. In this study, we demonstrated the mechanical contribution of the muscles, which activation was synchronized with the different muscles, due to a modular control of muscle synergies (**Figure 10**). Furthermore, the procedure, which considered temporal correlation between activation traces of muscle synergies and endpoint force fluctuations containing various information of neural property (Hagio and Kouzaki, 2015), enabled to estimate the ADs of muscle synergies (**Figure 7**). These results suggest that low dimensionality in muscle space was due to not simply a biomechanical constraint but a neural constraint as a hard-wired muscle synergy.

In summary, we could quantify the mechanical contribution of lower limb muscle synergies during isometric force-generating tasks in three-dimensional force space around a right ankle as

### REFERENCES


considering spatiotemporal correlation between activation of muscle synergies and endpoint force. Furthermore, the ADs of knee extensors were different despite functionally similar muscles, which depended on the other muscles weighted by the same muscle synergies, suggesting that muscles were synchronously activated through a hard-wired constraint as a muscle synergy. These results provide strong evidence that neural-based muscle synergies spatiotemporally contribute to the low-dimensional force generation in a task space.

### AUTHOR CONTRIBUTIONS

Conception and design of the experiments: SH and MK. Collection, analysis, and interpretation of data: SH. Drafting the article or revising it critically for important intellectual content: SH and MK. Final approval of the version to be published: SH and MK.

### ACKNOWLEDGMENTS

This work was supported, in part, by a grant from the Descente and Ishimoto Memorial Foundation for the Promotion of Sports Science (MK).


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Copyright © 2015 Hagio and Kouzaki. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Deep reduced PEDOT films support electrochemical applications: biomimetic color front

### **Jose G. Martinez, Beatriz Berrueco and Toribio F. Otero\***

Center for Electrochemistry and Intelligent Materials, Escuela Técnica Superior de Ingeniería Industrial, Universidad Politécnica de Cartagena, Cartagena, Spain

#### **Edited by:**

Federico Carpi, Queen Mary University of London, UK

#### **Reviewed by:**

Virgilio Mattoli, Istituto Italiano di Tecnologia, Italy Zhi-Hong Mao, University of Pittsburgh, USA

#### **\*Correspondence:**

Toribio F. Otero, Center for Electrochemistry and Intelligent Materials, ETSII, Universidad Politécnica de Cartagena, Aulario II, C/Carlos III, s/n, Cartagena, 30203, Spain e-mail: toribio.fotero@upct.es

Most of the literature accepts, despite many controversial results, that during oxidation/reduction films of conducting polymers (CPs) move from electronic conductors to insulators. Thus, engineers and device's designers are forced to use metallic supports to reoxidize the material for reversible device work. Electrochromic front experiments appear as main visual support of the claimed insulating nature of reduced CPs. Here, we present a different design of the biomimetic electrochromic front that corroborates the electronic and ionic conducting nature of deep reduced films.The direct contact PEDOT metal/electrolyte and film/electrolyte was prevented from electrolyte contact until 1 cm far from the metal contact with protecting Parafilm®.The deep reduced PEDOT film supports the flow of high currents promoting reaction induced electrochromic color changes beginning 1 cm far from the metal-polymer electrical contact and advancing, through the reduced film, toward the metal contact. Reverse color changes during oxidation/reduction always are initiated at the film/electrolyte contact advancing, under the protecting film, toward the film/metal contact. Both reduced and oxidized states of the film demonstrate electronic and ionic conductivities high enough to be used for electronic applications or, as self-supported electrodes, for electrochemical devices.The electrochemically stimulated conformational relaxation model explains those results.

**Keywords: conducting polymers, electrochromic front, redox reactions, ionic conductivity, reduced films**

#### **INTRODUCTION**

Conducting polymers (CPs) submitted to electrochemical reactions have been proposed as a very simple material model (reactive macromolecules, ions and solvent) of the intracellular matrix from living cells (Otero and Martinez, 2013a). Driven by electrochemical reactions they originate biomimetic devices such as artificial muscles and actuators, electrochromic windows (UV-vis or IR), fast charge/discharge batteries, or supercapacitors mimicking electric organs or new artificial chemical synapses (Otero et al., 2012). Designing such biomimetic devices requires conductive (electronic and ionic) materials. Designers and development engineers approaching to those materials envisaging new applications realize that most of the literature asserts the insulating nature of films of conducting polymers in its reduced state claimed by the conducting/insulator transition model (Ofer et al., 1990;Aoki and Kawase, 1994; Zykwinska et al., 2003; Heinze et al., 2010). Different designs of the electrochromic front border show that the oxidation of deep reduced electrochromic films supported by a glass always starts at the polymer–metal interface used for the film connection with the electrical generator (Tezuka and Aoki, 1989; Tezuka et al., 1995, 1996). The final conclusion is that the reduced film is an insulator forcing the reaction beginning only through those polymer chains in direct contact with the metal. In those designs, the ensemble glass, polymer film, and metal are immersed inside the electrolyte. As final consequence reduced self-supported films of those materials are discarded as basic component of the above-mentioned devices and as electronic conductors for electrochemical purposes or devices.

Different experimental results contradict the insulating nature of deep reduced films of CPs. High spin (Petr and Dunsch, 1996; Zykwinska et al., 2003; Osterholm et al., 2008b) and charged states (Osterholm et al., 2008a) content in reduced films were detected by EPR or Raman spectroscopic studies. Full polymeric electrochromic devices, not including any metal contact inside the device, have been developed (Invernale et al., 2010). The conductivity of freestanding polypyrrole films reduced at high cathodic potentials for long times keeps high counterion content and electronic conductivities over 10−<sup>3</sup> S cm−<sup>1</sup> measured in inert atmosphere (Otero and Ariza, 2003; Otero and Martinez, 2014b). Deep reduced films support metal electrodeposition from aqueous solutions with flow of high current densities (Otero and Ariza, 2003). Freestanding films of CPs can be reduced by slow potential sweeps up to high cathodic potentials (more cathodic that −2V) in different electrolytes (solvents and salts) and then reoxidized during the subsequent anodic sweep getting stationary voltammetric responses (Otero et al., 2014). Artificial muscles constituted by freestanding films on isolating flexible tapes (Otero et al., 1993), or by interpenetrated polymer networks (Plesse et al., 2005) or by freestanding bilayer of two CPs (Kaneto et al., 1995) also support stationary voltammetric cycles from the reduced (supposed isolating) state to the oxidized states giving reverse bending movements.

The Electrochemically Stimulated Conformational Relaxation (ESCR) model states (Otero and Angulo, 1993; Otero et al., 1995, 1996, 1997; Grande and Otero, 1998; Otero and Padilla, 2004), and the Structural Chemical Kinetics (SCK) (Otero and Martinez, 2013b) corroborates that the film oxidation/reduction

induces molecular (conformational) and macroscopic (swelling, shrinking, compactions, and relaxation) processes. After reaction induced structural closing the packed material traps over 20% of the counterions involved in the film redox processes, as proved by EDX analysis (Otero and Martinez, 2014b), which only can be expulsed very slowly through the packed film at high reduction overpotentials. Thus, after a deep reduction at high cathodic potentials any CP film keeps counterion and balancing polaron concentrations high enough to give electronic conductivities higher than 10−<sup>3</sup> S cm−<sup>1</sup> .

The electrochromic front methodology was designed to support the isolating and porous nature of deep reduced CPs (Tezuka and Aoki, 1989; Tezuka et al., 1995, 1996). Films from oligomeric solutions were casted on an insulating support, as glass. A metal film (sputtered or by simple contact) allows the electrical contact at the film top. The full system (a fraction of the metal, the metal/polymer contact, and the polymer film) was immersed in the electrolyte. After an anodic potential step the oxidation induced color change starts at the metal/polymer interface and the electrochromic front advances toward the film bottom. Ulterior designs from Smela's group, protecting now the film surface from the electrolyte contact letting the lateral borders free to contact the electrolyte, indicate that always the film reaction and the color change start at the electrode borders advancing toward the electrode center (Wang et al., 2004; Wang and Smela, 2009a,b).

In order to clarify this controversy, we will re-design here the electrochromic front experiment.

#### **MATERIALS AND METHODS**

PEDOT/PSS films were obtained by evaporation from Aldrich aqueous solution on a glass plate of 2 cm<sup>2</sup> . In order to ensure the electrical contact, a platinum foil of 1 cm<sup>2</sup> of area was put in contact with the polymeric film contacting 3 mm of the film top. The electrolyte–polymer contact was prevented in most of the film surface area by surrounding the ensemble under pressure and strain with two Parafilm® layers. About 1 mm of the PEDOT/PSS film remains uncoated at the film bottom to allow the PEDOT contact with the electrolyte. At the top, 2 mm of the Pt foil keep uncoated to allow the electrical clamp contact (**Figure 1A**). The electrochemical cell was a transparent tank containing 0.2 M LiClO<sup>4</sup> (Aldrich) aqueous solution. An ITO electrode (4 cm<sup>2</sup> ) was used as counter electrode. The reference electrode was a Crison Ag/AgCl (3 M KCl). First the PEDOT/PSS was deep reduced at −1V for 5 min to guarantee a deep reduced initial state. Then it was submitted to consecutive square potential waves from −1.00 V for 10 s to 0.20 V for 10 s. **Figure 2** shows two pictures of the cell with the oxidized electrode (transparent blue light color) and reduced electrode (blue dark color).

To ensure stationary responses, eight consecutive square potential waves were applied to the electrode. Color changes were recorded in parallel using a compact Sony video camera. Images were treated by Virtual Dub software and by ImageJ software to evaluate de the color gradient evolution between consecutive frames.

#### **RESULTS AND DISCUSSION**

The reduced PEDOT film is casted on an isolating and transparent glass (**Figure 1A**). At the top a Pt foil allows, by direct contact, the electronic contact and the current flow. The ensemble is surrounded, under strain and pressure, with a transparent Parafilm® in order to protect most of the deep reduced film, the film–metal interface and most of the metal from the direct electrolyte contact. At the electrode top, 1 mm of uncoated Pt foil allows the electrical connection, through a metal clamp, with the generator. At the electrode bottom, 1 mm of unprotected PEDOT film allows the CP/electrolyte contact (**Figure 1A**). Thus the Parafilm® protects from the electrolyte direct contact 10 mm, from the metal/PEDOT contact to the PEDOT/electrolyte contact, of the PEDOT film surface and lateral sides. This new design reproduces, at a larger scale, a theoretical transversal strip from the metal until the solution (**Figure 1B**) in the middle of a CP film coating a metal electrode. Here, the metal–polymer contact is far from the solution, furthermost than in any film coating a metal. Only a small fraction (1 mm) of the film remains uncovered at the film bottom (imitating the

two Parafilm® layers under strain to prevent the direct electrolyte contact. Around 1 mm of the CP remains uncoated at the bottom allowing there the direct CP/electrolyte contact. **(B)** The new design mimics a transversal cut of a conducting polymer (CP) film: metal/CP/electrolyte.

**PEDOT–PSS film casted on glass showing a transversal Pt foil on the top allowing the clamp electrical contact**. A half of the Pt foil and most of the conducting polymer was coated with two layers of transparent Parafilm®. **(A)** Reduced PEDOT–PSS film. **(B)** Oxidized PEDOT–PSS film.

polymer/solution interface of the coated metal electrode). If the deep reduced film is an insulator requiring the metal/polymer contact inside the electrolyte to start the polymer re-oxidation both, film oxidation and color change will be inhibited.

**Figure 2A** shows the blue dark uniform color of the polymer film after reduction at −1.00 V for a long time. **Figure 2B** shows the blue light color of the polymer film inside the electrolyte after oxidation at 0.2 V for a long period of time. Starting from the deep reduced film, after a potential step from −1.00 to 0.20 V, the polymer oxidation, visualized by the color change from blue dark to blue light, is initiated (**Figure 3A**) at the electrode bottom narrow strip in direct contact with the electrolyte. The electrochromic front advances through the PEDOT film from the film bottom, underneath the transparent Parafilm®, toward the polymer/metal contact, **Figure 3A** from the left picture to the right picture.

Thus, the oxidation induced color change is initiated at the narrow PEDOT strip in direct contact with the electrolyte, at the electrode bottom,1 cm far from the film/metal interface at the electrode top. The electrons extracted from the PEDOT strip chains during its oxidation must flow to the potentiostat through the 1 cm deep reduced film present between this strip and the metal contact. This result unambiguously indicates that the electronic conductivity of 1 cm of the dark blue and deep reduced film at −1V for long time supports the current flow required to initiate the PEDOT oxidation at the other film end: the reaction starts at the place where counterions, required for the reaction charge balance, are available from the solution. This oxidation drives the film color change from blue dark (reduced) to blue light (oxidized) and the oxidation progress drives the advance of the blue dark oxidized front by consumption of the reduced blue light film toward the polymer/metal contact. The color advance toward the metal contact underneath the Parafilm® indicates that the ionic conductivity through the oxidized film from the solution toward the oxidized/reduced front is also high enough to allow the reaction progress.

By stepping the potential back to −1.00 V the color changes back to dark blue (reduced) and the new generated reaction front also advances from the film bottom to the metal contact (**Figure 3B**, from the left picture to the right picture). The ionic conductivity through the reduced film underneath the Parafilm® toward the solution is also high enough to allow the advance of the reduced front.

The evolution of the front border is improved by color subtraction. **Figures 3C,D** were obtained by difference between the first image (showing the film at the beginning of the potential step), and each the subsequent ones. **Figures 3E,F** were obtained by binarization (transparent/black related to a threshold color) of **Figures 3C,D**.

The same results above described for the movement of the oxidized or reduced front is reproduced every time when the electrode is submitted to consecutive square potential steps: both, oxidation and reduction processes start, every time, at the polymer/electrolyte interface advancing underneath the Parafilm® toward the polymer/metal interface. This stationary reproducibility also sustains that the conductivity of the deep reduced film is always high enough to allow the initiation of the polymer oxidation 10 mm far from the metal/polymer contact. If, as proposed by the conducting/insulator model the reduced polymer was an insulator, its re-oxidation at 10 mm from the polymer/metal contact should become prohibited.

Those results unambiguously corroborate that both states (oxidized and reduced) of the PEDOT film present high enough electronic and ionic conductivities to support film electrochemical reactions taking place far away from the metal contact. That means that engineers and designers can use self-supported films of CPs as electrodes for any electronic or electrochemical application (actuators and artificial muscles; batteries and supercapacitors; smart windows, glasses, or mirrors; smart membranes with tuned transversal ionic conductivity; and chemical storage for drug delivery or artificial chemical synapse, ionic sensors, biosensors and proprioceptive sensors, and so on) (Otero et al., 2012;Otero and Martinez, 2013a, 2014b).

The origin of the conducting/insulator transition model during the electrochemical reduction of CPs apparently comes from the theoretical calculations of the insulating nature of an isolated, ideal, and neutral (without any charge) chain of any CP. The relatively high electronic and ionic conductivity of deep reduced films here deduced or the contradictory results presented at the introduction: high spin states and charged states (EPR and Raman results), high concentration of counterions in deep reduced films (XPS), film reduction reaction going on up to very high cathodic potentials or different devices, as artificial muscles, only constituted by polymers giving stationary voltammetric responses (reduction and re-oxidation) at very low potential sweeps (to get a deep reduced state) up to−3.5 V,does not contradict the calculated insulating nature of neutral individual chains. According with the ESCR model (Otero et al., 1995, 1996, 1997), CPs relax, swell, shrink, and compact under oxidation/reduction control. Those

**144**

structural changes are corroborated by determination of dimensional changes during oxidation/reduction (Otero and Martinez, 2014a), by its application to develop artificial muscles (Otero et al., 1992) or smart membranes, which transversal ionic flow can be tuned by the oxidation state (swollen or shrunk) of the film and by the SCK model (Otero and Martinez, 2013b). In films of CPs the oxidized material (polymer, balancing counterions, and solvent) presents a swollen structure. During reduction the materials, exchanging anions or cations, trap up to 30% of the counterions (and the balancing positive charges on the chains) inside the film (Otero et al., 2014). The reduction reaction rate becomes slower (Otero and Martinez, 2013b) going on up to very cathodic potential limits. Getting a full-reduced film (without any counterion inside) becomes a very difficult (for usual experimental

times) task for films thicker than 0.5µm. So, any reduced material keeps counterions and balancing polarons presenting a high electronic conductivity, as underlined in the literature. Getting lower electronic conductivities than 10−<sup>4</sup> S cm−<sup>1</sup> requires very long reduction times (days or weeks), at high cathodic overpotentials (more cathodic that −1V) and quite thin (<0.1µm) films. According with the Ohm's law, higher conductivities than 10−<sup>4</sup> S cm−<sup>1</sup> can support several milliampere per square centimeter of current flow and fast electrochemical reactions, as those observed in **Figures 2** and **3**.

The relatively high electronic conductivity of deep reduced films, and results from **Figure 3**, also give some light on other controversial point. Where the oxidation of a deep reduced film of a CP coating a metal starts? The chronoamperometric responses

during re-oxidation present a large maximum (Otero and Boyano, 2003; Otero and Martinez, 2013b) indicating that the re-oxidation begins by nucleation of the oxidized material. Supporters of the conducting/insulator transition model advocate a nucleation starting at the polymer/metal interface of the porous polymer film. Thus extracted electrons flow to the metal from those insulating polymer chains touching it: by oxidation those chains become conducting, originating a reaction front that advances from the polymer/metal interface toward the polymer/electrolyte interface. Results presented in this paper unambiguously demonstrate that the oxidation of deep reduced films in contact with a metal begins at the polymer/electrolyte interface. We can conclude that the oxidation of reduced and compacted films is initiated by nucleation– relaxation (**Figure 4**) at the polymer/solution interface, as proposed by the ESCR model (Grande and Otero, 1998; Otero and Boyano, 2003; Otero and Padilla, 2004), advances toward the polymer/metal interface originating expanding cylindrical columns of oxidized polymer. From those expanding columns a good theoretical modelization of the electrochemical responses is attained: the ESCR model.

#### **CONCLUSION**

Deep reduced films of PEDOT–PSS protected from the direct electrolyte contact present an electronic conductivity high enough to support the initiation of electrochromic changes, driven by oxidation/reduction reactions, at the polymer/electrolyte contact located 1 cm apart from the metal/film electrical contact. The electrochromic front advances, consuming the deep reduced film, from the film bottom to the polymer/metal contact at the electrode top. Those results support one of the hypothesis from the ESCR model: the oxidation of deep reduced films of CPs start at the polymer/electrolyte interface; in opposition to the conducting/insulator transition model stating that this oxidation begins at the polymer/metal interface of the porous (ionic conductor) and electronic insulator film, advancing from there toward the polymer/electrolyte interface.

An important technological consequence merges from the relatively high electronic and ionic conductivity of deep reduced films: they can be used as self-supported electrodes or as electronic conductors by engineers and designers to develop electronic or biomimetic electrochemical devices as artificial muscles and actuators; smart membranes tuning the transversal ionic flow by the membrane oxidation-swollen or reduced-packed state; smart drug (pharmaceutical, fertilizer, and neurotransmitter) deliverer; artificial chemical synapse; batteries and supercapacitors; smart windows, mirrors and glasses; sensors biosensors and proprioceptive sensors and devices; and so on.

#### **AUTHOR CONTRIBUTIONS**

The three authors participated in the conception of the experiments. Experimental work was performed by BB with help of JM and TO. All authors have participated in writing and have approved the final version of the draft. All authors agree to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

### **ACKNOWLEDGMENTS**

We acknowledge financial support from Spanish Government (MCINN) Projects MAT2011-24973, Fundación Séneca Project 08684/PI/08. JM acknowledges Spanish Education Ministry for a FPU grant (AP2010-3460). BB thanks Fundación Séneca for an R&D, Innovation and Technology Transfer grant.

#### **REFERENCES**


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 02 October 2014; accepted: 28 January 2015; published online: 11 February 2015.*

*Citation: Martinez JG, Berrueco B and Otero TF (2015) Deep reduced PEDOT films support electrochemical applications: biomimetic color front. Front. Bioeng. Biotechnol. 3:15. doi: 10.3389/fbioe.2015.00015*

*This article was submitted to Bionics and Biomimetics, a section of the journal Frontiers in Bioengineering and Biotechnology.*

*Copyright © 2015 Martinez, Berrueco and Otero. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*